I recently had the opportunity to attend and speak at the CTO Craft Con in London, a fantastic gathering of technology leaders focused on engineering leadership, culture, and strategy.
It was an insightful experience, and I wanted to take a moment to share some of my key takeaways from the conference.
Metrics: The "So What?" Test
One of the recurring themes at the conference was how metrics are used in engineering. Are the metrics we measure useful? Do they pass the "So what?" test?
In smaller organizations, especially, no single metric will perfectly align with a founder’s vision or goals. It’s crucial to think about the impact of what we measure and ensure it’s meaningful.
Engineering metrics alone are rarely enough, especially as teams become more deeply integrated with Product. Teams must understand not only their engineering output but also their impact on customers.
Metrics Shape Behavior, For Better or Worse
Measuring something can change behavior, sometimes in unintended ways. When teams know they’re being evaluated on a particular metric, they may optimize for it—even if that doesn’t necessarily lead to better outcomes.
This is especially true when metrics are reported outside the team; too much emphasis on numbers can lead to frustration or even resistance from engineers.
That said, internal metrics can be incredibly valuable when used correctly. The key is to focus on using them to improve capabilities rather than just chasing numbers.
Using Metrics to Identify Bottlenecks
Resolving one metric often reveals bottlenecks elsewhere in the organization. For example, fixing a deployment speed issue might expose gaps in testing or product discovery.
The key is not just to look at metrics in isolation but to understand the bigger picture.
I also liked the perspective that metrics should help rationalize the delivery plan to stakeholders.
If a team spends a high percentage of its time on maintenance and BAU (business-as-usual) activities, having data to support those concerns can be a powerful way to drive discussions about resourcing and priorities.
Metrics Aren’t Everything
Too much autonomy isn’t always a good thing. Some of the best engineering work happens when there’s strong collaboration, which requires the right balance between autonomy and shared decision-making.
Many engineering teams function well, but dysfunction often lies elsewhere—particularly in product discovery. Even the best engineering teams can struggle when the broader organization isn’t aligned—for example, misalignment in business requirements early on can lead to rework and scope creep.
Finally, an important caution: you can become imprisoned by your metrics if you’re trying to make a significant change. When evolving your team or processes, sometimes you have to be willing to step back from strict measurement and focus on the bigger picture.
Recommended Reading
For those interested in diving deeper into these topics, two books came up repeatedly at the conference that I’d highly recommend:
- Sooner Safer Happier: A great book on improving ways of working and delivering value.
- Accelerate: The must-read book on engineering performance and the DORA metrics.
Until next time,
Kendrick Curtis, VP of Technology
.svg)
