The Opportunity Cost of Debating Facts

I read this New York Times editorial this morning, How the Truth Got Hacked, and it reminded me of a situation at work, back when I first joined Host Analytics some four years ago.  This line, in particular, caught my attention:

Imagine the conversation we’d be having if we weren’t debating facts.

Back when I joined Host Analytics, we had an unfortunate but not terribly unusual dysfunction between product management (PM) and Engineering (ENG).  By the time the conflict got to my office, it went something like this:

PM:  “ENG said they’d deliver X, Y, and Z in the next release and now they’re only delivering X and half of Y.  I can’t believe this and what am I going to the customers and analysts who I told that we were delivering …”

ENG:  “PM is always asking us to deliver too much and we never actually committed to deliver all of Y and we certainly didn’t commit to deliver Z.”

(For extra fun, compound this somewhat normal level of dysfunction with American vs. Indian communication style differences –including a quite subtle way of saying “no” – and you’ll see the real picture.)

I quickly found myself in a series of “he said, she said” meetings that were completely unproductive.  “We don’t write down commitments because we’re agile,” was one refrain.  In fact, while I agree that the words “commitment” and “agile” generally don’t belong in the same sentence, we were anything but agile at the time, so I viewed the statement more as a convenient excuse than an expression of true ideological conflict.

But the thing that bugged me the most was that we had endless meetings where we couldn’t even agree on basic facts.  After all, we either had a planning problem, a delivery problem, or both and unless we could establish what we’d actually agreed to deliver, we couldn’t determine where to focus our efforts.  The meetings were a waste of time.  I had no way knowing who said what to whom, we didn’t have great tracking systems, and I had no interest in email forensics to try and figure it out.  Worse yet, it seemed that two people could leave the same meeting not even agreeing on what was decided.

Imagine the conversation we’d be having if we weren’t debating facts.

In the end, it was clear that we needed to overhaul the whole process, but that would take time.  The question was, in the short term, could we do something that would end the unproductive meetings so we take basic facts in evidence and then have a productive debate at the next level?  You know, to try and make some progress on solving our problems?

I created a document called the Release Scorecard and Commitments document that contained two tables, each structured like this.

At the start of each release, we’d list the major stories that we were trying to include and we’d have Engineering score their confidence in delivering each one of them.  Then, at the end of every release, PM would score how the delivery went, and the team could provide a comment.  Thus, at every post-release roadmap review, we could review how we did on the prior release and agree on priorities for the next one.  Most importantly, when it came to reviewing the prior release, we had a baseline off which we could have productive discussions about what did or did not happen during the cycle.

Suddenly, by taking the basic facts out of question, the meetings changed overnight.  First, they became productive.  Then, after we fully transitioned to agile, they became unnecessary.  In fact, I’ve since repeatedly said that I don’t need the document anymore because it was a band-aid artifact of our pre-agile world.  Nevertheless, the team still likes producing it for the simple clarity it provides in assessing how we do at laying out priorities and then delivering against them.

So, if you find yourself in a series of unproductive, “he said, she said” meetings, learn this lesson:  do something to get basic facts into evidence so you can have a meaningful conversation at the next level.

Because there is a massive opportunity cost when all you do is debate what should be facts.