Introducing Pulse to help companies achieve elite engineering performance

At Codacy, we envision a future where everyone can impact the world by crafting complex software with confidence and focus at the speed of thought. In the spirit of this vision, we want to find new ways to help teams build software.

Up until now, we were focused on helping developers and their managers standardize code quality during specific steps of the software development workflow. After working with thousands of companies in virtually every industry that software touches, we now understand that to be truly impactful we must do two things:

• Help the whole engineering team, which must include engineering leadership given that digital transformation can only happen when everyone is on board.
• Connect engineering data to business outcomes, bringing relevance to the great work done by engineering teams.

Today we’re announcing Pulse and setting ourselves to guide engineering teams to elite performance. We want to help optimize engineering teams for successful business outcomes by ensuring they’re culturally healthy, technically able, and process-aware.

Pulse is the blueprint and ruler that lets you assess the health of your engineering culture and guides you in the most effective way to elite performance, so you can be focused on your business goals:

• Measure the key metrics from Accelerate in a standard way
• Understand and keep track of your engineering team performance in one place
• Obtain guidance on which DevOps practices can have the most impact if adopted, depending on your current stage
• Have an easy and flexible way to integrate with your current stack and software delivery workflow

Water is wet: measuring and improving engineering is hard

The DevOps and Developer Tools landscape continues to grow at an accelerated pace. While larger players push for the consolidation of tools by creating more product value, innovative technologies and tools rely on bottom-up developer adoption to create their own market niches.

However, despite the existence of this heterogeneous and dynamic market, most companies still struggle to consistently improve their software development and delivery capabilities as a means to take their business performance to new heights. Furthermore, high-performing companies continue to pull away from the pack of low-performing companies that haven’t developed an ingrained culture of continuous improvement yet.

Keeping track of and improving the productivity of teams as an organization scales is hard. From the moment engineering grows beyond one team, the engineering leader quickly struggles to keep track of day-to-day groundwork and must rely on indicators and feedback from representatives of each team instead.

We quickly learned from our user interviews that reporting the work of engineering is far from trivial:

• There’s a lack of meaningful engineering metrics. You may have tried different approaches yourself: number of commits, lines of code, pull request count, velocity points, “impact”. These may give you a sense of how much your engineers produce, but they hardly cover the scope of their activities in the software development lifecycle. Other products in the market are still focused on measuring these output metrics. Further, these create bad incentives that promote busyness, and not impact or outcomes.
• There’s no easy way to collect engineering metrics. They require someone that can collect all relevant metrics in one place and existing solutions are hard to implement and costly to maintain. The immense variability of software development workflows and tooling rules out standard, out-of-the-box solutions.

Engineering leaders struggle to implement DevOps best practices. Tracking key performance metrics is just the starting point. Teams can only achieve higher levels of performance with ROI-based prioritization on what needs to be addressed next and practical guidance on how to actually implement these technical, cultural, and process best practices.

Published in 2018, the book Accelerate condenses half a decade of research into evidence-based and actionable insights that revolutionize how we think about engineering practices, such as the correlation between engineering excellence and business success; valuing outcomes over outputs; and key metrics to consistently measure and keep track of engineering performance.

Engineering leaders of any modern organization need help to measure and boost the impact of their teams towards higher-level business goals, and the Accelerate key metrics came to prove that they are the vehicle to achieve that success. However:

• The metrics are somewhat abstract and difficult to interpret unequivocally since the goal of the research described in the book was only to classify organizations into broad groups.
• Building an internal custom solution to measure these metrics is time-consuming and costly to maintain, as reported in many of our user interviews.
• Integrations are hard to implement and if any team wants to adopt a new tool, they must do more work. This undermines one of the most important DevOps principles, the need for teams to be fully autonomous.

Want to try measuring your Pulse?

The first version we’re testing is heavily inspired by the concepts explored in Accelerate, as well as in DORA’s research program and the latest State of DevOps Report. It defines and measures the key metrics established in Accelerate: lead time for changes, deployment frequency, mean time to restore (MTTR), and change failure rate.

With Pulse, we are now able to provide a “full-stack” company-wide performance overview. We do this using our best in class code quality standardization, while also giving the tools necessary for engineering leaders to justify the investment in quality to their executive leads by quantifying the ROI in the broader goal of improving business performance.