Kapost Engineering

Recent Posts


Recent Comments


Archives


Categories


Meta


You CAN (And Should) Measure Software Engineering Performance

Nader AkhnoukhNader Akhnoukh

You know a great engineer when you see one. (And we’ve got a team full of them at Kapost) Subjectively, it’s easy. They’re productive, make good architectural decisions, care about tests, think about infrastructure, monitoring, edge cases, and performance. They pay back technical debt as they go, and on top of it all are a pleasure to work with.

So why measure at all?

In the early days of a startup resolute perseverance and individual heroics are often enough to make you successful.

Then, if you’re fortunate, you find that you have product market fit (hurray!) and your challenge shifts to scaling your company. Almost always scaling your people will be harder than scaling your product. To learn more about this challenge check out Micheal Lopp’s The Old Guard.

Individual heroics are no longer enough. Now you need incredible coordination of each group in the company. Inertia is a powerful force and can crush a young company. That coordination you need to be successful is driven by great visibility and great accountability across the entire company. You’re looking to build a well oiled learning machine, a neural network that takes in input and as quickly as possible adjusts course.

To build this well oiled machine, everyone in the company needs to know how their team is doing and how every other team is doing.  This has long been the case with Sales and Marketing teams. They’ve got a clear “number”. Engineering, however, has long been a black box, subjectively easy to measure, but objectively a no man’s land.

To grow into the great company you want your startup to be, this has to change.

Ok, but is it even possible?

Now the question boils down to how we should measure ourselves.

The most obvious measure is productivity. But what does that mean? Lines of code? Every decent developer knows that reducing line count is often better than increasing it. Even in the early eighties there were misgivings to this approach.

How about hitting GA dates? There are two big problems here. Setting a date months out implies a waterfall approach which for a dynamic startup is generally not the best approach. Secondly, and more importantly, anyone can hit a date at the expense of quality.

Hmm…so what are we to do?

The conventional wisdom is that Engineering teams can not be measured effectively so we shouldn’t even try.

While I hold these two legends of our industry in high esteem, I respectfully disagree. Software engineering teams CAN be measured… it’s just a little complicated.

The Building Blocks of a Great Engineering Team

Let’s examine the components of a high functioning team:

Putting it All Together

I’d argue that if your team has high achievement of each of those components then you’ve got a high functioning team. At Kapost we set goals for each of those components each quarter. For some teams and some components were already doing well (hurray!), and we put in a “threshold goal” which means we’re just trying to maintain that level. In other areas we want to improve so we set a goal of improving a number by the end of the quarter.

Then we measure ourselves every iteration. Improvement goals are split up over the course of the quarter, so for example if we’re trying to get our code climate score for a particular repo up from 3.3 to 3.6, the goal for the first iteration is 3.35, then 3.4, etc.

Finally we weigh each of these metrics to give us a blended score. These weights vary from quarter to quarter depending on what we want to focus on. This is the current blend.

The average of the score for each iteration is your final score for the quarter and this is what we base bonus payouts on!

We’ve been doing this for about a year, and it is working great so far. While the blended metrics take a little bit of effort to measure, with a clear dashboard every team knows how they’re doing, what needs improvement, and the whole company knows how we’re doing as an engineering organization.

I hope this helps other engineering teams as they struggle through this tricky topic and I’d love to hear how other teams have tackled this!

—–

1 I personally detest the term “sprint”.  Sprinting through an iteration is a great way to burn out a team.  A healthy team is capable of high output for an indefinite amount of time.  It’s much more akin to a marathon than a sprint.  Thus we use the term “iteration”.

leading a great team, building great software.

Comments 0
There are currently no comments.