Mitigating technical debt with developer-driven security

Let’s talk about debt 

Most everyone knows now that cybercrime has become a major issue facing our global economy. As of 2022, the average cost of a data breach in the United States amounted to $9.44 million, up from $9.05 million in the previous year. It’s important not to ignore the cost of insecure code and its accumulated technical debt. According to the 2022 Consortium for Information and Software Quality: The Cost of Poor Software Quality report, it is estimated that the cost of poor software quality in the US has grown to at $2.41 trillion and the accumulated software technical debt has grown to $1.52 trillion. 

The burgeoning costs of addressing insecure code and its technical debt have become the biggest obstacle to making any changes to existing code bases - thus leaving them vulnerable to exploitation and external threats. The state of software security is facing an existential crisis - we know we have to improve our security posture as well as address accumulated technical debt, but the barriers are huge: 

• There are an estimated 300,000 unfilled software developer and IT related jobs in the US with a projected growth rate of 15% 
• It’s predicted that by 2025, 40% of IT budgets will be spent simply maintaining tech debt
• 1/3 of developers’ weekly hours on average are spent addressing tech debt

Quick fixes are risky - and cost more long term 

What is technical debt and why is it so important? Tech debt accumulates when decision makers go for a short-term solution to a software development problem—instead of a more exhaustive, long-term solution. This comes with a substantial hidden cost that organizations must pay later. Much like a maxed-out credit card, technical debt has two main components:

• Principal - refers to the total cost of refactoring or fixing software so that it reaches a desired level of maintainability and security.
• Interest - the extra effort that developers spend making those changes to address the technical debt alone, and not new functionalities. Every minute spent on not-quite-right code adds interest to the debt.

One can eventually reach a state of “technical bankruptcy” when the cost of new features, bug fixes, and maintenance exceeds the project budget - sinking the value of the software application significantly. 

However, some debt accumulation, just like in life, is normal and in most cases, somewhat expected. 

Ideally, all software developers should reduce bugs as much as possible before shipping code. However, they are faced with a tough tradeoff: To be competitive, an organization might want to deliver features or products to customers quickly at a minimum cost. As a result, the quality of the application suffers because developers' KPIs are based on the speed of the delivery, and the initial cost to build it. What’s missing from the picture is the accumulated deficiencies and potential vulnerabilities baked into the code. This leaves it ripe for bugs or security vulnerabilities down the line or worse, exploitation by bad actors. 

But there lies the conundrum: Is there a different way to ship products quickly without accumulating a massive amount of technical debt? 

The cost of finding and fixing deficiencies and vulnerabilities is the largest single expense in the software development lifecycle. The earlier in the development lifecycle issues are found, the more cost-effective the overall delivery will be. 

Technical debt can evolve into security debt

Many developers try to circumvent this tradeoff by using open source code to help them move quickly and ideally, use an already vetted solution. However, relying heavily on open source software often presents its own risks: 

• 82% of the open source components were found to be out of date (i.e. unpatched or not well supported) 
• 75% of codebases contained vulnerabilities (up from 60% in 2018), and 49% contained high-risk vulnerabilities 
• An average of 82 vulnerabilities were identified per codebase 

This proliferates a subset of technical debt - security debt. Security debt is the accumulation of vulnerabilities in a software application that makes it harder or even impossible to protect data and systems from an attack.

One of the most notorious examples is Equifax, the credit reporting giant breached in 2017 because it had failed to patch a known vulnerability in Apache Struts, a popular open-source web application framework. The patch had been available for months, but the breach compromised the crucial personal data of more than 147 million people.

Therefore, greater attention must be given to secure coding practices as many applications have reached a critical mass in not only their technical debt but the density of security weaknesses and vulnerabilities in the application itself.

This can result in huge losses, that can either be tangible or intangible: 

Reputational damage: The loss of customer trust can have an extremely negative impact down the road. This may include damage to the brand, lost sales, and costly legal problems as a result of a breach. 

Regulatory and compliance impact: If a security breach can cause a company to miss a deadline and/ or contractual obligations. A failure to meet an SLA can land a company in trouble with regulators, resulting in significant fines. 

Remediation costs: Extra work is often needed following a failure or outage to make up for the loss in productivity.

Preventing technical and security debt in the SDLC

Many organizations are already shifting their budget to create a stronger security posture. Last year, Google committed $10 billion over 5 years to fund a program to strengthen cybersecurity.  The Biden administration also requested $2.1 billion in the 2022 discretionary budget for the Cybersecurity and Infrastructure Security Agency (CISA). 

Providing more resources and training to help bolster the professional growth and knowledge of your developers can be the first step in establishing quality standards for all code shipped into production. 

The costs to find and fix vulnerability or defect exponentially grows the later in the software development cycle it’s found and addressed. And as we’ve seen, with so much time spent on addressing technical and security debt, organizations are creating their own losses by forgoing innovation and time spent on new features or products. 

In 2022, a majority of developer teams said DevOps or DevSecOps was their methodology of choice, and it’s no surprise why. DevSecOps integrates security at every stage of the software development lifecycle to deliver better and more secure applications. Security and Development teams continue to work in silos and have tension, but it’s clear that this needs to change to help businesses succeed. DevOps is part of how organizations are trying to break down barriers and reshape culture. The fundamental goal of DevSecOps is to increase collaboration between AppSec/ Security with developers from the very beginning of the software development lifecycle.

Implementing a new way of thinking about addressing technical debt and security doesn’t have to be a monumental feat. Establishing a proactive mindset through training is critical when trying to improve the security awareness and skills of an organization’s developer community. A robust secure-coding education for developers ensures that learning is ongoing, interactive, relevant, and contextual is a necessity. A truly holistic approach must consider what is needed to foster a genuine developer-led security culture. It may require changing the focus from the typical ways of managing and building developer teams.

Creating a culture change isn’t easy, but Secure Code Warrior helps you to identify your security champions and help equip developers and organizations with the right skills to tackle today’s ever-changing security challenges. 

Launching an engaging and scalable secure code program is a worthy investment because of the long-term preventative approach to security, instead of the reactive way of the past. This ultimately helps to mitigate the costly risks of a breach, educate developers on how to find and fix vulnerabilities quickly, and facilitate a more agile way of focusing on product development and accelerated time to market.