AppSec in the Age of Continuous Integration and Deployment

There is a core benefit to moving to continuous integration in your development pipeline.

Speed.

Continuous integration/continuous deployment (CI/CD) aims to automate and streamline software development. It allows your team to rapidly push code to integrate, test, and deploy to production environments. This faster deployment leads to shorter feedback cycles, meaning everything speeds up with CI/CD.

But this focus on speed in CI/CD pipelines can sometimes lead to "YOLO’ing" security concerns. Teams want to push code, and security checks can be seen as a bottleneck or an impediment to their velocity. However, neglecting application security (AppSec) practices in pursuing speed can have severe consequences.

The Core Challenges With CI and Security

A development team uses a CI/CD pipeline to work on a web application. They have a well-established workflow where developers commit code changes, triggering automated builds, unit tests, and deployments to staging and production environments.

Now, the organization decides to integrate application security practices into the pipeline to enhance the application's security posture. They introduce the following security measures:

1. Static Application Security Testing (SAST): The team integrates a SAST tool into the pipeline to analyze the source code for potential security vulnerabilities. After each commit, the SAST tool scans the codebase, identifying issues such as cross-site scripting (XSS), SQL injection, and insecure configurations. Our 2024 State of Software Quality report found that 42% of developers use SAST tools. 
   
   
2. Dynamic Application Security Testing (DAST): A DAST tool is added to the pipeline to perform runtime analysis of the application. It simulates attacks and identifies vulnerabilities that may not be detectable through static analysis alone.
   
   
3. Dependency Scanning: The team incorporates a tool to scan the application's dependencies for known vulnerabilities. This helps identify any vulnerable third-party libraries or packages used in the application.

What happens? Well, first, you get an increase in build time. The addition of SAST and dependency scanning tools increases the overall build time. These tools need to perform their analysis on the codebase, which can take significant time depending on the size and complexity of the application. As a result, the CI/CD pipeline may take longer to complete, delaying the deployment process.

You might also get false positives. SAST and DAST tools may flag issues that are not actual vulnerabilities. Developers must triage these findings, investigating each reported issue to determine its validity. This additional effort can slow the development process as developers must allocate time to review and address the security findings.

When genuine security vulnerabilities are identified, developers must prioritize and address them. This may involve code changes, configuration updates, or dependency upgrades.

The time spent on remediation efforts can vary depending on the complexity and severity of the vulnerabilities, further impacting the development timeline. Then, if critical security vulnerabilities are discovered during the security testing phase, deployment may be delayed until those issues are resolved. This ensures that the application is not released with known security flaws but can prolong the market time.

Initially, this doesn’t look good. The cost-benefit analysis seems to land squarely on the cost. But this negates two important considerations:

1. The long-term benefits of enhanced security often outweigh the short-term impact on speed.
2. The development of DevSecOps methodologies and better tooling to increase velocity while incorporating security.

Let’s look at those.

The Long-Term Benefits of Enhanced Security

By detecting and addressing security vulnerabilities early in development, organizations can significantly reduce the risk of costly security breaches, data leaks, and reputational damage.

This risk mitigation is critical because the cost of a security breach can be astronomical financially and in terms of customer trust. According to IBM's Cost of a Data Breach Report 2023, the average cost of a data breach reached $4.45 million.

Beyond the immediate financial impact, a security incident can lead to long-term reputational damage and straightforward loss of customer confidence. Integrating security practices into the CI/CD pipeline can help organizations meet compliance requirements and industry standards, avoiding potential legal and financial repercussions.

Fixing security vulnerabilities becomes more complex and expensive as the software progresses through the development lifecycle. A vulnerability that could have been quickly addressed during the coding stage might require extensive rework and patching if discovered in production. The National Institute of Standards and Technology (NIST) estimates that:

On average a bug found in coding/unit testing takes 2.4 hours to fix, whereas an average bug found in post-product release takes 13.1 hours to fix.  

So, that is a 5X return on finding and fixing issues in development rather than production, leading to the next problem: technical debt. Addressing security issues early prevents the accumulation of technical debt related to security vulnerabilities. This saves time and resources in the long run, as fixing security flaws becomes more complex and expensive in later stages of development.

How DevSecOps and Better Tooling Increase Velocity

DevSecOps practices emphasize the automation and integration of security tools into the CI/CD pipeline. This allows for continuous and efficient security testing without manual intervention, reducing the time and effort required for security checks.

The core idea here is the shift-left approach. By shifting security testing to the left (i.e., earlier in the development process), DevSecOps enables the identification and fixing of security issues before they propagate to later stages. This reduces technical debt and the costly hours (and money) of fixing bugs in production.

This starts with each developer making security thinking a part of their development process. This might be through secure coding practices, security-focused IDE plugins, or local security scans. Developers can fix potential issues quickly without disrupting the pipeline's flow by catching potential issues early.

But, the code review is the principal component of better security early in development. Every Code Review Is a Security Review. But one of the reasons this doesn’t happen is that manual security code reviews are laborious. Thus, automation is vital. You need the security tooling to analyze every new or modified line of code for vulnerabilities. That will include:

• Static Application Security Testing analyzes source code to identify common security vulnerabilities, such as cross-site scripting (XSS) and SQL injection.
  
  
• Supply Chain Security monitors code dependencies for known vulnerabilities, CVEs, and risks in open-source libraries.
  
  
• Hard-Coded Secrets Detection scans code for exposed sensitive information such as API keys, passwords, certificates, and encryption keys.
  
  
• Infrastructure-as-Code Configs scanning assesses Terraform, CloudFormation, and Kubernetes configuration files for misconfigurations and security risks.
  
  
• Dynamic Application Security Testing performs simulated attacks on web applications to identify vulnerabilities and weaknesses in the front end.
  
  
• Cloud Security Posture Management identifies misconfigurations and security risks in cloud infrastructure across major cloud providers.
  
  
• Penetration Testing proactively identifies exploitable vulnerabilities in a system before malicious actors can leverage them.

By implementing DevSecOps practices and leveraging advanced tooling, organizations can increase their development velocity while ensuring that security remains an integral part of the CI/CD process. This allows for faster and more secure software delivery, striking the right balance between speed and security.

If your organization can take advantage of this opportunity, each of the components of automated code security reviews mentioned above is available as part of Codacy Security. It gives your team “DevSecOps in a box” and easily finds and fixes vulnerabilities within your code review process. If you want to learn more about Codacy Security, AppSec, or DevSecOps, you can reach out to us or sign up to start with better security today.