Home Developer Code review process: improved developer productivity

Code review process: improved developer productivity

Author

Date

Category

According to our survey of 680 developers, the code review process had an overall positive impact on developer productivity which we discuss below.  We find that code reviews increase code quality and developer productivity on several dimensions (for other information on our first set of developer survey insights see here).


Step 1. Identifying the most important practice to improve code quality

It is widely assumed that code reviews have a number of benefits. A casual browse of the literature shows that the code review process has benefits such as:

  • improving code quality;
  • facilitating team collaboration;
  • homogenizing code standards and code styles;
  • identifying bugs early in the development process;
  • onboarding new developers and spreading best practices and knowledge within the organization.

However we wanted to contrast the intuition with hard and cold numbers.

The first step was to ask an open question: what change to their development process had the biggest impact to code quality? Developers were free to provide as many answers as they wanted. The answers were then labelled and categorized, as shown in the chart below.

Question 1: what change to your development process had the biggest impact on code quality?

Graph showing what change in development process has the biggest impact to code quality

Result: 1 in 5 respondents answered that code reviews was the single most important activity to to improve code quality.

Of all the answer provided by the developers, code reviews was by far the most prevalent answer, with nearly 1 in 5 (specifically 19%) suggesting that code reviews was the single most important activity to to improve code quality.

Step 2. Measuring  impact of code review process on developer productivity

There’s an obvious trade-off between time spent reviewing code and time spent coding new features. So we asked developers what proportion of their time they spend fixing bugs.

Question 2: what proportion of your time do you spend fixing bugs?

Results:

The above table is pretty self-explanatory. If you assume that a week consists of 40 hours, developers who work in companies where the code is reviewed systematically before deployment spend 4 hours less per week fixing bugs.

But it gets better.

Because the other side of the coin is whether the time saved fixing bugs actually translates in the more fun, productive stuff — that is, building new features. So we asked the following question:

Question 3: what proportion of your time do you spend building new features?

Results:

It turns out that people who who review code spend 4.4 hours per week more building new features (again, assuming a 40-hour week).

Step 3. Measuring the impact of code reviews on code quality

Having established that developers who have their code reviewed end up spending 10% more of their time on building new features and 10% less on fixing bugs, the next set of question relates to code quality: how much of an improvement do code reviews contribute?

To find out, we ask two questions, starting with…

Question 4: How do you rate code quality in your project?

Here we asked point blank developers how they rated the quality of their project, on a scale from 1 (very bad) to 5 (very good). The average scores are displayed in the table below.

Those did not do code reviews had an average score below the median (i.e. 3), whereas those who did do code reviews had an average quality score above the median 3. This code quality score is obviously highly subjective, so we asked another question:

Question 5: do you need more time to maintain code?

Developers here asked whether they needed more time to maintain code, and had only possible options: yes, or no. The result is as follow:

Whether or not the code is reviewed, in general there was a feeling that more time was needed to review code (in fact, time pressures were one of the main complaints that developers reported). But again, the difference between those who do code reviews and those who don’t is stark: over three-quarters of developers who didn’t have their code reviewed felt that they needed more time to maintain the code, compared to 59% for those who did.

Final thoughts

Our research confirms what is known intuitively by most developers. It also complements insights from a variety from case studies first reported by McConnell’s “Code Complete” book. We’ll conclude this blog post by adding a few quotes from the book:

“… software testing alone has limited effectiveness — the average defect detection rate is only 25 percent for unit testing, 35 percent for function testing, and 45 percent for integration testing. In contrast, the average effectiveness of design and code inspections are 55 and 60 percent. Case studies of review results have been impressive:

* In a software-maintenance organization, 55 percent of one-line maintenance changes were in error before code reviews were introduced. After reviews were introduced, only 2 percent of the changes were in error. When all changes were considered, 95 percent were correct the first time after reviews were introduced. Before reviews were introduced, under 20 percent were correct the first time.

* In a group of 11 programs developed by the same group of people, the first 5 were developed without reviews. The remaining 6 were developed with reviews. After all the programs were released to production, the first 5 had an average of 4.5 errors per 100 lines of code. The 6 that had been inspected had an average of only 0.82 errors per 100. Reviews cut the errors by over 80 percent.

* The Aetna Insurance Company found 82 percent of the errors in a program by using inspections and was able to decrease its development resources by 20 percent.

* IBM’s 500,000 line Orbit project used 11 levels of inspections. It was delivered early and had only about 1 percent of the errors that would normally be expected.

* A study of an organization at AT&T with more than 200 people reported a 14 percent increase in productivity and a 90 percent decrease in defects after the organization introduced reviews.

* Jet Propulsion Laboratories estimates that it saves about $25,000 per inspection by finding and fixing defects at an early stage.”

Further reading

  • Seven Truths About Peer Reviews (Karl E. Wiegers). Link.
  • Code complete. (Steve McConnell). Link.
  • Code Reviews: Just Do It. Link. Coding Horror. Link.

Thanks for reading! If you enjoyed it, hit that heart button below 🙂

You can also read more about this and other great learnings in our ebook about code reviews:

Click to get the ebook

Happy code reviews!


About Codacy

Codacy is used by thousands of developers to analyze billions of lines of code every day!

Getting started is easy – and free! Just use your  GitHub, Bitbucket or Google account to sign up.

GET STARTED

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Subscribe to our newsletter

To be updated with all the latest news, offers and special announcements.

Recent posts

New Public Product Roadmap – See What's In Store For Q1

We’re excited to release a public product roadmap for 2020.  Now, those outside Codacy can see what our team is working on...

4 Trends To Boost Software Quality in 2020

Today’s abundance of technology has created an increased demand for higher software quality. With companies aware of this, spending on IT worldwide...

7 Reasons Dev Teams Choose Codacy Over Other Automated Code Review Tools

Today more than 100,000 developers at leading global organizations like PayPal and Adobe across industries use Codacy (read customer case studies). ...

Introducing GitHub Sync For Seamless User Management

Announcing GitHub Sync: Synced Organizations For GitHub We are thrilled to announce "GitHub Sync" -- automatic user and repository...

Ringing In 2020 With Codacy

It’s been an exciting 2019 at Codacy and we expect more excitement in 2020. Over the last 12 months we’ve built out...