This article originally appeared in Security Boulevard. It has been updated and syndicated here.

‍

It’s almost ingrained in our collective psyche that more of a good thing is an even better thing. If you have one of something that you really like, then there is nothing wrong with having two or three, or even more. After all, you can’t have too much of a good thing, right?

Unfortunately, while that logic works for a few things, there are times when it quickly breaks down at higher volumes. In some cases, you begin to experience diminishing returns. How much extra value or joy is that 13th piece of chocolate cake really going to bring you? And then there are other times when adding too much of a good thing can become something that is a significant risk profile for you or your organization.

Too many cybersecurity tools

A critical area where too much of a good thing has quickly become a bad one is with cybersecurity tools, which have blossomed out of control at most organizations in recent years. A study in Dark Reading found that most Chief Information Security Officers rely on an average of 55 to 75 distinct security products or applications to protect their networks. And yet, attacks are still getting through. According to the Verizon 2023 Data Breach Investigations Report, successful attacks are on the rise, and the most complex among them are taking longer than ever to detect.

How can attackers circumvent what seems to be on the surface an impossible gauntlet of 75 or more cybersecurity tools? The fact is that they often do trip alarms, but human defenders are too busy maintaining their defensive tools or responding to thousands of daily alerts to notice. Having lots of security tools can actually give attackers the cover they need to remain undetected.

A recent report in TechRadar highlighted the negative consequences of having too many cybersecurity tools. Among companies surveyed, 71% felt they had more tools than their cybersecurity teams could ever successfully manage. This was actually causing their security posture to get worse as more tools were added. In fact, contrary to the belief that more tools equates to more security, the overwhelming majority of those surveyed reported feeling much less secure because of all the cybersecurity tools installed throughout their environment.

The situation has only gotten worse with the massive move to the cloud at most organizations. One of the reasons that layering lots of cybersecurity tools into a network to cover every possible avenue of attack got popular was because in an era of almost totally on-premise assets, the strategy kind of worked. Or at least organizations didn’t run into diminishing returns and negative consequences quite so quickly. In cloud environments, however, the more tools you add, the more complexities and vulnerabilities you produce.

Besides the work that goes into maintaining overlapping tools, the other big problem with a tools-centric approach is the ocean of false positives that will certainly occur as more tools come online. Chasing down false positives can drain all the time away from human security personnel for no real benefit to the organization. And meanwhile, a real attack can easily hide within all the false alarms. Cybersecurity professionals may never find real threats until it’s far too late.

A better way

It would be disastrous to remove all of the security tools from your environment. But you also don’t want to have so many that the key tools that could really help out aren’t given enough attention. It’s important to find the right selection of tools, and to make sure that you don’t have too many draining your time and resources.

The key to making tool consolidation work is to simultaneously invest in a human-led approach to security. And this should include using an asset that has traditionally not been deployed in that role: teams of developers tasked with coding the very apps and software being targeted by attackers.

Although developers traditionally have not been tasked with security, this is changing. In fact, encouraging developers to concentrate on security is a key facet of DevSecOps movements where everyone takes some responsibility for deploying secure applications. Nobody expects developers to suddenly become security experts or to bear the primary responsibility for security at their organizations, but teaching them how to write secure code, and rewarding them for a job well done, can go a long way to setting the stage for the elimination of all those overlapping security tools. 

If you start with good, secure code, then you can easily begin to eliminate some of the hundreds of cybersecurity tools designed to scan for common exploits and vulnerabilities. Eventually you will foster an environment where developers create secure code, and a few select cybersecurity tools can act as an additional check that is easy for security teams to monitor and maintain without being overloaded by too much of a supposedly good thing.

Cyberattacks are, without a doubt, on the rise. According to the Verizon 2021 Data Breach Investigations Report, the threat landscape is more dangerous today than ever before. Organizations of all sizes are experiencing a higher volume of attacks and a greater sophistication level from threat actors who are targeting them. And the success rates for attackers are also skyrocketing.

Analyzing the most recent attacks helps to reveal some of the most common vulnerabilities and techniques being used by hackers during this unprecedented blitz against cyber defenses. Some of the most popular attacks, such as those that made the Open Web Application Security Project’s (OWASP) Top 10 Security Risks and Vulnerabilities for 2021, involved stealing or otherwise compromising credentials. And according to security research conducted by Akamai, the overwhelming majority, almost 75%, directly targeted the credentials held by APIs. 

The rise and possible ruin of APIs

It’s no wonder that application programming interfaces, mostly just called APIs, are on the rise within almost every organization’s networks. They are a critical component of most cloud-based services, which are rapidly taking over the functions of on-prem assets at most companies, organizations, and government agencies. You almost can’t run any sort of business or task these days without the cloud, especially those that are public-facing. And that means that APIs are going to certainly be the glue that holds quite a few services together in every network.

The amazing thing about APIs is that they are mostly small and unobtrusive in terms of network resource allocation. And they are completely flexible so that they can be tasked with performing almost any job. At their core, APIs are individual pieces of software tailored to control or manage a particular program. They can be utilized to perform very specific functions, like accessing data from a host operating system, application, or service.

Unfortunately, it is this very same flexibility, and the fact that they are often small and overlooked by security teams, that makes APIs attractive targets. Most APIs are designed by developers for total flexibility so that they can, for example, continue to function even if the core program they are managing is modified or changed. And there are few standards. Almost like snowflakes, many APIs are unique in that they are created to serve a particular function with a single program on a specific network. If they are coded by developers who aren’t very security-aware, or who are not concentrating specifically on security, then they can and likely will have any number of vulnerabilities that attackers can find and exploit.

Sadly, the problem is quickly getting out of hand. According to Gartner, by 2022, vulnerabilities involving APIs will become the most frequent attack vector across all cybersecurity categories.

The key reason that attackers want to compromise APIs is not so that they can take over whatever specific function the API performs, but instead to steal the credentials associated with it. One of the biggest problems with APIs, in addition to being ripe with vulnerabilities, is that they are often way over-permissioned in regards to their core functionality. For simplicity’s sake, most APIs have near administrator-level access on a network. If an attacker gains control of one, they can often use its permissions to launch deeper and more substantial incursions into a network. And because the API has permission to perform whatever tasks the attacker is redirecting them toward, their actions can often bypass traditional cybersecurity monitoring because the API is not breaking any rules thanks to its access-all-areas VIP backstage pass.

If organizations are not careful, the rise of APIs within their network and their clouds can also spell big trouble if they are targeted by attackers.

Defending the APIs

As dangerous as the situation with APIs is becoming, it’s far from hopeless. There is a big effort through movements like DevSecOps to help make developers more security-aware, and to bring security and best practices into all aspects of software creation from development to testing and deployment. Including API security as part of that training will be critical for any organization that wants to buck the trend of API exploitation through 2022 and beyond.

That said, there are a few really good best practices that can be implemented right now in terms of API security.

The first thing is to include tight identity controls for all APIs. You should almost consider them to be like human users when assigning permissions. Just because an API is only designed to do a specific function, you have to think about what could happen if an attacker is able to compromise it. Consider using role-based access control. Ideally, you should ultimately be applying the principles of zero trust to your APIs and users, but that is often a long road. Good identity management is a good place to start. Just be sure to include APIs as part of that program.

You should also tightly control the various calls that are being made by your APIs as much as possible. If you limit those calls to very context-centered requests, then it will be much more difficult for an attacker to modify them for nefarious purposes. You can even layer your APIs, with an initial API making a highly contextual call to another API that knows exactly what to look for, and what to ignore. That can be an effective way to limit the functionality available to a threat actor even if they are able to exploit and compromise an API within that chain.

The threats leveled against APIs can certainly seem overwhelming. But by implementing best practices along with assisting and rewarding developers who become security champions, the situation can seem a lot less hopeless. With good training and practice, you can erect a robust security program that gives attackers little room to maneuver even if they should somehow compromise one of your tiny but essential API workhorses.

When was the last time you took a road trip? Depending on where you are in the world, it may only be a recent return to the agenda, but really, nothing is better than the open road and getting a change of scenery. 

Unless you’re a software vulnerability, of course.

We’ve spoken at length about the dangers posed by lax cybersecurity measures in the automotive industry, with the likes of Tesla and Jeep already working with security researchers, finding exploitable bugs that could have lead to serious safety issues if not discovered early and fixed promptly. We’ve also spoken about how, in general, software security is still in the Wild West. Software is everywhere we turn, and for many connected devices, vehicles, and their peripherals, the security measures needed go well beyond end-user education and vigilance. 

API vulnerabilities are becoming especially insidious, with malicious API traffic growing more than 300% in the past six months alone. This is rather concerning, given that modern vehicles are essentially APIs on wheels. They’re connected, very chatty with other applications, and could be caught up in targeted attacks as one of many vulnerable endpoints. 

When your EV charger says too much

Connected vehicles have come under scrutiny for their software safety, but what about their accessories? The genius crew at Pen Test Partners uncovered several code-level vulnerabilities in six home electric vehicle charging brands, as well as a widespread public EV charging network. 

Who cares about a charger? What could an attacker gain? Unfortunately, one of the downsides of powerful, deep tech working overtime for us is that generally, these devices have a bad case of TMI. EV chargers communicate with companion mobile apps via API, in a cloud-based environment, and all of that can be vulnerable to exploitation if not coded and configured securely. APIs, by design, open the floodgates for communication between apps, and if these endpoints are not carefully configured, too much could be shared - or worse - accessed via a vulnerable app backdoor.

Pen Test Partners discovered extremely dangerous vulnerabilities that could have lead to millions of EV chargers being hijacked, several instances of API authorization issues that allowed account takeover and remote control/access to an account, and even the possibility of disrupting the power grid through synchronized control of multiple EV devices. These issues were all patched, but the fact that a few lines of code were all that stood between attackers and complete disruption of core functionality and service infrastructure is deeply concerning. 

It’s not like it’s mastermind stuff, either. Wallbox, for example, had two insecure direct object references (IDOR) in their API, which would have allowed account takeover if exploited. IDOR falls under broken authentication, which sits at number two in the OWASP Top 10 API vulnerabilities. It’s as common as dirt, which points to a failing in the learning and implementation of quality code. We can’t persist with connecting sensitive devices and apps through a myriad of buggy communication pathways, and poorly configured APIs are just that. 

Working securely with automotive APIs takes education and patience

The frustrating thing with API security is that it is touted as a new wave of cybersecurity disaster to try and mitigate, when really, it’s just a new setting for the same old problems we have seen for decades in web development. Cross-site scripting, injection, misconfiguration: sound familiar?

Recent indicators from organizations like NIST are promising, and show that software security is becoming more regulated and standardized. However, we still fall well short of the required experts to even put a dent in the amount of safeguarding that needs to happen on the deluge of code that is written every day. Developers need to be elevated in their security knowledge and responsibilities, and it’s not on them to take the initiative. If you’ve got a team working on embedded systems in appliances, or APIs that could turn a car into someone’s remote control toy, then you must make sure they are equipped with what they need to stop introducing common vulnerabilities. 

The differences between a secure API and one vulnerable due to XSS are minimal, for example, but developers need to be shown the nuances that differentiate a poor coding pattern from a good one. In addition to that, lazy development processes are often business as usual in API configurations, with many given vast permissions over-and-above the minimum requirements for it to perform its set tasks, thus opening so much extra threat surface and potential data theft. These factors need to be considered during the build, but if they are not ingrained into acceptable development practices, the process will continue to be a risk factor.

Avoiding the new threat actor playground

The dramatic increase in APIs as a target of threat actors shows that attention is moving to a perceived low-hanging fruit… and in this instance, it is one that could be a pipeline to significant paydirt, in addition to potential threats to life in the form of potential vehicle takeover. 

Leaving API security up to chance is a sure-fire way to introduce problems later on, with potentially devastating consequences at worst, and frustrating rework and low performance at best. It should be a critical consideration as part of the communication ecosystem of software, and high on the list of a best-in-class security program. The key to this is treating each API as though it’s a human, and assessing what access it should have. Should Jim from Accounting have access to all sensitive legal documents for the entire company? Probably not, and generally, access control is correctly determined in the case of real-world personnel. The same cannot be said for APIs, and it’s important to remember that they are powerful chatterboxes that will let everyone know your secrets if they’re not configured with the same zero-trust methods of everything else.

The organization must be on high alert, and developers are the eyes needed on the ground to create quality code that is free from these vulnerable portals to despair. It’s time to give them the opportunity to grow and thrive as security-aware engineers, with the right mindset for this goal and the hands-on skills to make the right decisions in critical stages of creation.

A version of this article appeared as a feature for the Forbes Technology Council. It has been updated and syndicated here.

‍

There is nothing quite like being on the wrong side of a data breach. At first, there might be denial, then panic. Once all the expletives have been aired and the CISO has had to do a 2am conference call with public relations, it’s time to roll up your sleeves and get to work on securing endpoints, systems, and quickly eliminating any potential attack vectors. It’s no picnic, to say the least. 

And yet, this is a reality that will dawn on many organizations in the future, and one that absolutely must be prepared for with a comprehensive cybersecurity incident response plan. The problem, however, is that this reactive strategy is where much of the time, resources, and effort is concentrated, instead of working to prevent or reduce the potential severity of cyberattacks up-front. It’s a little bit like calling an ambulance for a suspected heart attack; the outcomes are often a lot less positive - not to mention more damaging - than if preventative health measures were in force before it was too late. 

To that end, what does a preventative plan look like? Let’s explore how security pros can employ all the tools at their disposal to mitigate ever-increasing cyber risk, every day:

Understand the scope of work that lies ahead 

It seems obvious, but the “right” plan to mitigate cyber risk does have nuances between industries, and it’s important to understand what is needed up-front to reach the desired outcome.

What security problems currently exist? What time and resources are they taking up? How many of them are recurring issues? These are important factors, and will give you a foundational starting point. Consider any roles that need filling, gaps in tooling, and what is needed from an expertise and tool perspective to secure endpoints and reduce the attack surface, while preempting other areas of potential risk. 

A recent report revealed that eleven industries saw a serious vulnerability, across at least half of their applications, every day for the past year. In particular, the utilities, public administration, and professional services industries took 288 days on average to patch known vulnerabilities. This is incredibly slow, giving an attacker more than enough time to do serious damage if those vulnerabilities are discovered before a patch can be applied. This, coupled with the probability of organizations experiencing a data breach approaching 30%, is a sobering reminder that incident reaction is not enough, and the stakes are simply too high to brace for the impact of a large-scale cyberattack and hope for the best.

Prepare to get buy-in for cultural change

Shaking up the status quo does tend to raise a few eyebrows, but the truth is, security programs should be in a constant state of continuous improvement. Every component should stay relevant, and new developments should be assessed and factored in. 

Placing emphasis on a preventative - as opposed to reactive - approach may not be widely understood outside of the security team, especially if a big, bad, security incident has not taken place. It might be seen as something that isn’t broken and doesn’t need fixing. In this instance, getting executive buy-in is essential. Some of the more pertinent points for them to consider are:

• The time and cost savings in preventative measures, such as role-based training and related tools, as opposed to the potential cost of a critical incident 
• How finding and fixing vulnerabilities now keeps releases on time, with fewer showstoppers from the security team
• Why preparing for and preempting potential security risks, from the development team right through to release, saves more time (no to mention significant cash) overall. To put it into perspective, late-stage vulnerabilities uncovered in the testing phase - or worse, post-production - can raise costs as much as 3000% on average.
  

It’s vital that proposed cultural changes are aligned with business goals, even if they seem uncomfortable at first. 

Security awareness is something, security skills are everything

As an industry, we talk frequently about the importance of security awareness, and this is an increasingly critical component of every member of staff in an organization. However, it is not enough to stop at lip service and passive training, especially for those in technical positions.

Put simply, anyone who is touching code is a potential security risk if they’re not equipped with the skills to code securely. General awareness of basic security parameters is a good start, but without contextual knowledge of good, secure coding patterns, poor habits prevail, and it’s this lack of quality development skill that attackers rely upon to do their dirty work. 

Don’t write off your developers.

Though the attitude is shifting, many organizations are structured in such a way that developers are rarely a true consideration in security mitigation plans. Some industries - like banking and finance - have stringent compliance and regulatory requirements that result in heightened security practices and training across the board, for all staff. And while they’re certainly ahead of other verticals, just about every organization on the planet could benefit from an in-house army of security-aware developers, all with a baseline ability to sniff out common security bugs before they’re committed. Most are nowhere near achieving this critical piece of the security program puzzle - and it’s necessary if we ever have a hope of securing the deluge of code that increases in volume year-on-year.

Preventative security should begin the moment fingers touch the keyboard to create software, but developers cannot be expected to bridge the security skills gap alone. They need the right toolset and contextual guidance to reach a higher standard of code quality, and the best results are always achieved when it’s part of their everyday work, not an afterthought that is sporadically rolled out whenever annual compliance requirements roll in.

A version of this article appeared in TechCrunch. It has been updated and syndicated here.

‍

As if we hadn’t had enough disruptions in the Year That Cannot Be Named, 2021 started off with a deafening bang for the U.S. government, in the form of one of the worst data breaches on record. The SolarWinds incident was a devastating, sophisticated nation-state attack that sent several government departments and large organizations into a panic, scrambling to secure their endpoints. Virtually every end-user of the SolarWinds software was compromised, and the incident made it abundantly clear that cybersecurity and defense in software supply chains are critical.

It’s obvious that cybersecurity is important, but what does it actually mean in the context of supply chains?

The state of cybersecurity in the average development team

An enormous amount of software is being produced every single day, representing billions of lines of code every year, and that is only increasing with the demand created by our progressively digital lifestyles. The global developer population is on track to swell to almost 29 million by 2024, and currently, no formal certification exists to assess and certify their ability to code securely. That’s not to say that every developer produces or re-uses insecure code, but there is undoubtedly a sustained risk that basic security weaknesses can be introduced into the software we trust with our data. 

Developers are assessed on their ability to create features and ship code as fast as possible. Security hasn’t been a benchmark for their success in most organizations, but that sentiment is starting to change as more companies realize the potential they hold in preventing common security bugs at the earliest stage of software creation. However, realization and implementation are two different things, and since most tertiary development courses omit any context around secure coding practices, it’s often up to a developer’s workplace to make up the shortfall. If skills building and knowledge sharing are infrequent or irrelevant, then it will likely be ineffective. And so, the cycle of recurring vulnerabilities through lack of skill development remains unbroken. 

Naturally, it is not the responsibility of your average software developer to solve the world’s cybersecurity woes; after all, organizations hire those very expensive security professionals for a reason. Security gurus are in short supply, however, and developers can certainly play a role in reducing the strain. 

But where does this leave us - and the vendors creating software for critical infrastructure and sensitive organizations - in terms of preventing a devastating cyberattack? It’s going to require a shift in the status quo of software procurement, at the very least.

The pitfalls that stand in the way of a secure software supply chain

The tired adage of a chain only being as strong as its weakest link is, unfortunately, just as true when it comes to software supply. It really doesn’t matter if your company has come to the party with beefed up security best practices, investment in developer upskilling, and a move towards a functional DevSecOps environment (i.e. everyone sharing the responsibility for software being made as securely as possible); if you use software from a vendor that has security problems, you will inherit them into your ecosystem and bear the consequences.

Sure, the security team should be helping to assess the safety of third-party additions to the tech stack, but decisions can be made based on a business need with little choice among solutions. At this point, it can be a trust exercise. Does the vendor care about security as much as your company does? And can the vendor actually assess the risks as only you could understand them, as well as the assets you need to protect?

Transparency is an all-important factor in assessing the security viability of vendor software additions. Are they up-front with their own security practices? They should take pride in their approach to keeping data safe, and it should be a top priority. If the security practices are not published anywhere, or no information is available, there is a strong chance that security is not top-of-mind. The vendor should be able to answer technical questions, and independent certifications like ISO27001 and SOC2 wouldn’t hurt either. Also, if you can’t “look under the hood” and scan for vulnerabilities as part of your own internal due diligence and security practices, forget it. 

With demand driving such fast-paced implementation of software needs, especially if vendor code is being folded into existing systems to perform actions in a new context, both the vendor and buyer need to be at the top of their game, and both should have their developers as the boots on the ground to pick up common security bugs and flaws before they ship. There could be hundreds - or thousands - of dependencies compromised if a new addition to the existing spider web of tech solutions is added to, and one small failure could lead to a catastrophic undoing. 

So, what’s the solution? Code everything in-house, from scratch? If it was 1998, that might make sense. However, just as we no longer “Ask Jeeves” where the closest car wash is, we need to implement realistic safeguards that work in today’s context.

There is still no silver bullet, but there are solutions

For buyers, security assessment of vendor software and development practices should be a priority of the overall security program and risk mitigation plan. Ask questions around their certifications, practices, and security reputation of their developers.

Vendors (and indeed any companies creating software), must be prepared to demonstrate that security is top-of-mind, and should focus on upskilling. Security-skilled developers are in high demand, and with the right tools and support, they can be built up from your existing team and empowered to defend against attacks resulting from common vulnerabilities. But don’t throw any old training at them. Give them time to thrive with security tooling that is complementary to their existing workflows. Make it as easy as possible, and watch those pesky bugs start to thin out among the code powering the business.

The bottom line is that any software risks are immediately exacerbated if they are part of the supply chain, affecting all users and any systems where vulnerable components have been utilized. If vendors aren’t as serious about security as the companies implementing their software - or both the vendor and organization are lacking in their security program - then more  devastating, far-reaching supply chain attacks like SolarWinds will inevitably become the norm - and this is a critical problem for everyone.

Although it may seem counterintuitive to anyone working outside of software development, many of the professionals employed in application security over the years have worked in those critical roles with little or no programming experience. These AppSec professionals are part of the team that is responsible for making sure that no vulnerabilities creep into the applications that have become the lifeblood of many industries and organizations, and yet few of them can actually directly evaluate or fix the code themselves. 

Instead of coming from a coding background, many security professionals approach their roles from the perspective of key knowledge around attack vectors, threats, exploits, and business risk; they have a limited view of code. While not every AppSec guru has the same skillset, a typical day for many involves working with code reviewers and scanning tools to ensure that programs and systems are secured according to organizational standards, or relevant industry and government frameworks. They then write up reports about their findings, and send back information on the attack vector that may break the code. It is then up to developers to make necessary fixes, no matter how disruptive it may be to current work.

The reason the situation developed this way is because the prevailing logic over the years was that the job of protecting networks and applications was so vast, that it didn’t make sense to expect everyone working in cybersecurity to perform every role. Deep coding skills were left to the developers, and little value was placed on the ability to write or edit code farther down the development pipeline.

That mindset is changing fast, and that presents a unique opportunity for developers to make the lucrative jump and career shift into AppSec. Not every developer will want to embrace the so-called dark side, and many developers aren’t particularly positive in their opinions regarding AppSec teams. But for those who do, there has never been a better time to grab that increasingly tempting brass ring.

DevSecOps drives nearly every industry

One of the biggest factors in elevating the value of security-aware programmers and developers in any organization, is the almost universal move to embrace more agile development practices like DevSecOps. When development, security, and operations are combined, cybersecurity becomes a shared responsibility integrated into the development of new software from end to end. In that environment, the ability to code is increasingly being seen as a valuable asset across the board, and this is especially true for engineers who also inherently understand security. 

An AppSec professional who not only understands cybersecurity at a high level, but also the code that makes everything work, is inherently more valuable to any organization than someone whose knowledge is concentrated on the theoretical. Being able to quickly discover and evaluate vulnerabilities found within code, and then mitigate those problems, is at the core of why DevSecOps is seeing such popularity.

Developers working in AppSec also bring another big advantage to any organization that employs them. Coming from the development side of the house makes it easy for them to talk with developers about security and vulnerabilities. It also makes it much easier to become coaches for the development teams, helping them to become better coders. Over time, they might even be able to remove the “dark side” stigma from AppSec and help to unify teams within software development across an organization.

The cybersecurity skills shortage is getting worse

Shakespeare mused that it’s an ill wind that blows nobody any good. What he meant was that even the darkest situation probably benefits someone. The cybersecurity skills shortage is a great example of this.

The shortage of personnel is being felt acutely almost everywhere. In a recent survey conducted by the Center for Strategic and International Studies, 82% of IT decision-makers said their organizations suffered from a shortage of cybersecurity skills, and 71% said that the shortage had resulted in direct and measurable damage to their organizations. To put this crisis in an even better perspective, the report pointed out that just in the United States alone, there were more than 520,000 unfilled cybersecurity jobs in 2020 for a field where only about 940,000 are employed.

The cybersecurity personnel shortage is bad news for organizations trying to protect their infrastructure, business and data from an increasingly dangerous threat landscape. But it makes a good opportunity for developers looking to get into AppSec and security. Chances are, that cybersecurity and AppSec positions are available almost everywhere. And with cybersecurity positions taking an average of 21% more time to fill these days, salaries are rising across the board.

Making the jump to AppSec

There may never be a better time for developers to make the lucrative jump to the sunny security side of life. Security-aware developers are no longer seen as just part of a stopgap security method, but are instead filling out a full and respected role as cybersecurity defenders. This is especially true for organizations that have embraced DevSecOps and other more agile development methodologies. And the cybersecurity talent shortage means that positions are available at nearly every company, government agency or organization. Those with the right skills can pick and choose where they want to work.

Moving to AppSec may not be for everyone, and of course, most developers will remain focused on building amazing features. But for those who are considering making the jump, investing in security training to augment their existing coding skills can open up a lot of doors. The best AppSec people come out of engineering, because they deeply understand the tech and have empathy for the plight of their fellow developers. DevSecOps means that everyone is now responsible for security anyway, so why not take advantage of the current critical skills shortage to advance your career into application security? There has never been a better time to make a positive move for yourself, your family, and your career.

There are plenty of pop culture references to rogue AI and robots, and appliances turning on their human masters. It is heavily steeped in science fiction fun and fantasy, but with IoT and connected devices becoming more prevalent in our homes, so too should the conversation around cybersecurity and safety. Software is all around us, and it’s very easy to forget just how much we’re relying on lines of code to do all those clever things that provide us so much innovation and convenience. Much like web-based software, APIs, and mobile devices, vulnerable code in embedded systems can be exploited if it is discovered in the wild by an attacker. 

While it’s unlikely that an army of microwaves is coming to enslave the human race (although, the Tesla bot is a bit concerning) as the result of a cyberattack, malicious cyber events are still possible. Some of our cars, planes, and medical devices also rely on intricate embedded systems code to perform key tasks, and the prospect of these objects being compromised is not only alarming, but potentially life-threatening.

Much like every other type of software out there, developers are among the first to touch the code, right at the beginning of the creation phase. And much like every other type of software, this can be the breeding ground for insidious, common vulnerabilities that could go undetected before the product goes live. 

Developers are not security experts, nor should any company expect them to play that role, but they can be equipped with a far stronger arsenal to tackle the kind of threats that are relevant to them. Embedded systems - typically written in C and C++ - will be in more frequent use as our tech needs continue to evolve, and specialized security training for the developers on the tools in this environment is essential. 

Exploding air fryers, rogue vehicles… Are we sitting ducks?

While there are some standards and regulations around all secure development to keep us safe, we need to make far more precise, meaningful strides towards all types of software security. It might seem far-fetched to think of a problem that can be caused by someone hacking into an air fryer, but it has happened in the form of a remote code execution attack (allowing the threat actor to raise the temperature to dangerous levels), as has vulnerabilities leading to vehicle takeovers. 

Vehicles, in particular, are especially complex, with multiple embedded systems onboard, each taking care of micro functions; everything from automatic wipers, to engine and braking capabilities. Intertwined with an ever-increasing stack of communication technologies like WI-Fi, Bluetooth, and GPS, the connected vehicle represents a complex digital infrastructure that is exposed to multiple attack vectors. And with 76.3 million connected vehicles expected to hit roads globally by 2023, that represents a monolith of defensive foundations to lay for true safety.

MISRA is a key organization that is in the good fight against embedded systems threats, having developed guidelines to facilitate code safety, security, portability and reliability in the context of embedded systems. These guidelines are a north star in the standards that every company must strive for in their embedded systems projects.

However, to create and execute code that adheres to this gold standard takes embedded systems engineers who are confident - not to mention security-aware - on the tools. 

Why is embedded systems security upskilling so specific?

The C and C++ programming languages are geriatric by today’s standards, yet remain widely used. They form the functioning core of the embedded systems codebase, and Embedded C/C++ enjoys a shiny, modern life as part of the connected device world.

Despite these languages having rather ancient roots - and displaying similar vulnerability behaviors in terms of common problems like injection flaws and buffer overflow - for developers to truly have success at mitigating security bugs in embedded systems, they must get hands-on with code that mimics the environments they work in. Generic C training in general security practices simply won’t be as potent and memorable as if extra time and care is spent working in an Embedded C context. 

With anywhere from a dozen to over one hundred embedded systems in a modern vehicle, it’s imperative that developers are given precision training on what to look for, and how to fix it, right in the IDE. 
‍

Protecting embedded systems from the ground floor is everyone’s responsibility

The status quo in many organizations is that speed of development trumps security, at least when it comes to developer responsibility. They’re rarely assessed on their ability to produce secure code, but rapid development of awesome features is the gold standard. The demand for software is only going to increase, but this is a culture that has set us up for a losing battle against vulnerabilities, and the subsequent cyberattacks they allow. 

If developers are not trained, that’s not their fault, and it’s a hole that someone in the AppSec team needs to help fill by recommending the right, accessible (not to mention assessable)  programs of upskilling for the entire development community. Right at the beginning of a software development project, security needs to be a top consideration, with everyone - especially developers - given what they need to play their part. 

Getting hands-on with embedded systems security problems

Buffer overflow, injection flaws, and business logic bugs are all common pitfalls in embedded systems development. When buried deep in a labyrinth of microcontrollers in a single vehicle or device, it can spell disaster from a security perspective.

Buffer overflow is especially prevalent, and if you want to take a deep dive into how it helped compromise that air fryer we talked about before (allowing remote code execution), check out this report on CVE-2020-28592.

Now, it’s time to get hands-on with a buffer overflow vulnerability, in real embedded C/C++ code. Play this challenge to see if you can locate, identify, and fix the poor coding patterns that lead to this insidious bug:
‍

‍

How did you do? Visit www.securecodewarrior.com for precision, effective training on embedded systems security.

As an application security professional, it’s your job to ensure the cyber safety of your organization’s applications. You’re not, however, responsible for writing the code the application runs on. Engineers within the development team are. So how do you make sure that they’re developing those systems with security in mind? 

Most likely you’re doing some or even all of the following: 

• Reviewing all code for security flaws and reporting them back to the development team to fix. 
• Enforcing a strict peer review process throughout your secure development lifecycle. 
• Having regular application assessment/penetration tests done by internal or external security teams.
• Implementing scanning tools to pick up on vulnerabilities.
  

These are great best practices, but they’re also expensive and similar to taking a round of antibiotics every time you get sick. Not only does that come with a high cost, it loses efficacy and can weaken your immune system over time. 

How do you go about actually ensuring the code that developers ship is written securely in the first place? 

Secure coding aside, first think about how people learn. Most of us are visual learners and we learn through doing. And yet secure code training is often provided as a ‘tick-the-box’ activity and is not relevant to a developer’s daily work. It’s designed to prove that developers have undergone security training often to comply with industry standards, and not for developers to actually retain that knowledge, let alone enjoy the learning process. 

Another way humans tend to learn is through our mistakes in the same way that our immune system does. T-Cells remember what sort of pathogens they have encountered and successfully eradicated in the past, so that they can protect against them in the future. This is exactly the role that developers should play in your secure SDLC. 

It’s unrealistic to expect them not to make mistakes, but you can prepare them in a way to be able to recognize coding patterns that will translate to security vulnerabilities in the future. 

This is also how a robust peer review process becomes so powerful. Just because one developer doesn’t notice a security flaw, does not mean that another won’t. And the better trained the development team is as a whole, the more likely vulnerabilities will be caught in their tracks and never make it to production. 

Software vulnerabilities are like pathogens

Software vulnerabilities are like pathogens in the sense that you have to remember them in order to fight them. With pathogens, our immune system often needs to be exposed multiple times before it remembers how to fight it in order to avoid severe sickness or worse. 

A successful cyber attack from vulnerable software can severely cripple or kill an organization. But if developers are introduced to software vulnerabilities first in a controlled environment, they can work to build immunity to the threats by increasing and regularly practicing their secure coding knowledge and skills. 

Expose developers to security flaws in a controlled environment

We can never fully protect ourselves from getting sick, but there are things we can do to boost our immune systems and stay as healthy as possible. Things like regular exercise, healthy eating, and plenty of sleep are amongst the lifestyle choices commonly associated with a strong immune system. But all of those things require a bit of effort and they must be continuous. Going on a jog every day for a week or giving up drinking for a month will barely make a dent in your overall health. It’s also not advisable to go out and run a 10k the first day we take up running. We first need to expose our hearts and muscles to the exercise. We also know that it takes a bit of experimenting until we find a good balance for our body and healthy foods and exercises that we love.

It’s not so different when it comes to secure software development. Learning happens over time and with practice, and developers need the same on-going training to regularly boost their secure coding skills. Not to mention, software development is always evolving and adapting, meaning the vulnerabilities are too. That’s why a simple training course is not enough. Developers need regular upskilling in order to be familiar enough with potential threats to be properly equipped to defend against them. 

Aim to achieve herd immunity within the development team

A single person can’t prevent any and all security issues. It’s great to have security champions on the team, but to get the best protection, the more people that have learned about security vulnerabilities and how to prevent them, the better chance your organization has in preventing them. Again it’s not much different than how the immune system has different types of T-Cells for different purposes. Every single developer is part of a team that ensures security. If they’re empowered to take responsibility, do it well, and even enjoy doing it, then you can create herd immunity against cyber threats within the development team as a result.

Keep security top of mind with repeat exposure

Our brains learn in a similar way that our immune systems work. German psychologist Hermann Ebbinghaus was a pioneer in the field of memory and learning. He deduced that learning has to occur over time and with multiple learning sessions. When we’re in school, we’re never expected to maintain new knowledge after the first introduction. First the information is presented to us, then we practice it with guidance, and then we practice it on our own. And even after we have learned it well enough to pass an exam, the information tends to be forgotten shortly after if we don’t regularly use the knowledge we have spent the time and effort to learn. How many of us can claim we remember our high school French?

So how can we possibly believe that a single day of looking at slides and listening to someone talk about security would actually lead to those developers in attendance coding more securely?

Patterns of recurring vulnerabilities show us that this simply doesn’t work. 

How do you achieve secure development immunity?

The answer lies in our nature. Our bodies and minds work the same way and they provide beautiful solutions to problems, as long as we work with them and not against them. 

To ensure that your applications are secure, you need to start with upskilling developers to write secure code. Otherwise AppSec will continue to spend all of their time reviewing all code for security flaws and reporting the same recurring vulnerabilities back to development only to be quickly fixed with nothing learned. And then do it all over again for the next release.

So let’s reiterate.

If you work together with your development managers to do that, you’ll not only be implementing a secure SDLC and ticking the security training requirement for compliance box, you’ll be making a real-world impact on the development process. To put a cherry on top of it all, AppSec will no longer be encountering and reporting repeat vulnerabilities back to development teams and developers will spend less time fixing them. That means they can spend more time creating and improving the amazing software that makes our world better. 

Ready to upskill your development team? Go ahead and book a demo with us.

At Secure Code Warrior, we are always looking to expand our training coverage. To empower embedded developers and security managers to build secure embedded systems, we are tapping into the world of embedded systems security. In this post, we will share the key things developers and architects should know about building secure embedded systems. 

An embedded device is an independent computing unit, with its own microprocessor-based hardware and software. They are typically used to perform dedicated functions, either independently, or as parts of larger systems.

What is an embedded device and system?

Embedded devices are purpose-built computing systems with minimalistic hardware and software, programmed to perform specific functions. Depending on their purpose, the complexity of embedded devices can vary. 

If an embedded system has to perform complicated, resource-intensive operations, its hardware may comprise multiple processors, with various interconnected peripherals. If, however, it’s designed to do a simple task, it may just contain a single microcontroller. 

Some embedded systems are black-boxes with no user interface, while others can have detailed, graphical user interfaces.

Examples of embedded devices

Embedded devices find applications across multiple industries; from aerospace to computing, and from home appliances to healthcare. Here are a few examples:

• A thermostat within an air conditioner. 
• A fitness tracker, with small electrical components running a tiny operating system, that can do nothing more than record and sync your health stats.
• Digital climate control systems.
• Navigation systems within an airplane.
• Digital watches.
• GPS devices.
• Wi-Fi components within microwave ovens and other consumer electronics.
• Blood pressure and heartbeat monitors.
• Networking component within an MRI machine that sends data to the cloud.

Is a laptop an embedded device?

Short answer: No. Long answer:

By definition, an embedded device encapsulates all the software and hardware it needs to perform its functions. A laptop indeed wraps up various hardware components in a metal body, and encompasses all the software it needs to deliver its feature set.

However, unlike typical embedded systems, which have a pre-defined, minimal feature set, a laptop is more general purpose and can be used to do a wide array of activities.

Embedded devices in Internet of Things

Embedded systems are at the heart of IoT. Your smart fridge can be controlled via your mobile device, because of an embedded device. It’s an embedded device in your security system that lets you view CCTV footage from a thousand miles away. And when you push a button to put your car on auto-pilot, behind-the-scenes, it’s an embedded device that does all the magic.

Embedded software development life cycle

The following steps are involved in the typical software development life cycle of an embedded device:

Step 1: Purpose and requirements

An embedded device has a distinct and precise feature set. This is why it’s very important to explicitly state its purpose and requirements before starting the development. Creating a detailed design document for the system is recommended. Answer questions like:

• What functions will the device perform?
• Will it be a part of another machine, or will it function on its own?
• Should the device have a user interface? If yes, does a command line suffice, or is a graphical UI needed?
• What size, cost, or power consumption limits must the device adhere to?
• Are there any performance benchmarks? E.g. is the device expected to respond in real-time, or within certain thresholds?

Step 2: System architecture

Once we have identified the unique system requirements, we are ready to design the system architecture. Answer questions like:

• What hardware components will be required to develop the device? This includes identifying the circuits, processor chips and micro-controllers, along with any required internal and external peripherals. 
• How much power will the components require? 
• Will the device be connected to the internet?
• Which interfaces must be added/developed to allow the device to connect to other devices, or the larger machine?
• How will you implement encryption? (think algorithms, key storage etc.)
• How will you identify and avoid any potential vulnerabilities, exploits, and malware?

Step 3: Select the operating system

The choice of operating system often dictates the efficacy of your embedded applications, so choose wisely. Windows for IoT may score high in graphics, but lacks hardware support. Embedded Linux and Android are free, whereas VxWorks and Windows for IoT have licensing costs. 

With open-source systems like Linux and Android, your developers have more control over the kernel; however, with proprietary systems, the default feature set of the kernel is all that you can work with. 

It’s also important to consider the security implications while choosing an operating system. Do security patches get released quickly, in case any vulnerabilities are discovered? Is there any built-in protection against the most common cybersecurity attacks?

Step 4: The development tools

Different programming languages and frameworks have various pros and cons. Depending on your feature, speed, and reliability requirements, you may choose one language/framework over the other. 

If your application is web-based, and you want high performance, go with Java.
However, if you want the fastest throughput, go with C/C++. If you want the best third-party library support, go with Python.

Step 5: Code, refactor, test, code some more

Once you have decided on the development platform, you are ready to start coding. Remember that embedded systems are sensitive devices with limited hardware and software resources. It’s thus of pivotal importance to keep in mind the best practices regarding application security and performance.

Code reviews help in optimizing code and identifying any potential bugs. You also need to test as rigorously as possible. Ensure that you compile an exhaustive list of all the test-cases that the device may encounter when used in production.

Step 6: Maintenance and support

Just like any other application/device, the life cycle doesn’t end when the embedded device has been shipped and installed. As the device gets used in production, new use-cases are identified, which require new features to be added. Any reported bugs may also require new firmware updates to be released.

Top embedded programming languages

There are many reputable organizations, like IEEE and TIOBE, that rank programming languages by popularity and use. However, regardless of whether you check the TIOBE index, or the IEEE Spectrum, you will notice a common theme.

The top 3 languages are always C, Python, and Java. The TIOBE index for July 2021 ranks C at #1, Java at #2, and Python at #3. According to IEEE Spectrum’s ranking for embedded programming languages, the order is: Python, Java, and then C.

How does firmware written in Java, C, or Python differ?

Different languages manage memory, interactions with the OS, and runtime, in different ways. E.g. Java applications execute in a specialized runtime environment known as Java Virtual Machine, or JVM. 

Memory management within C/C++ is manual, however, in Java or Python, you don’t have to worry about managing memory yourself; the language does it for you. What’s more is that in C, you have to collect your garbage yourself; i.e. if you dynamically allocate a piece of memory, you have to explicitly free it yourself. If you don’t, it will lead to a memory leak. In Java or Python, garbage collection is done automatically. However, C is intrinsically much faster than Java or Python.

One thing that C and Java (but especially C) lack in comparison to Python is the availability of third-party libraries. Python has a richer database of libraries, which makes it much easier for developers to add certain features.

Overall, choose C if:

• You want a low-level interface with the operating system and hardware.
• You want the best possible performance.
• Your developers are skilled at C/C++.
• Don’t need too much external/third-party support.

Go with Python if:

• You want to implement data science or machine learning algorithms.
• You want rich, third-party library support.
• You are okay with trading-off speed for ease-of-use.
• You have limited experience in embedded development.
• You don’t need a low-level interface with the OS or hardware.

Go with Java if:

• You are building web applications.
• You want more ease-of-development than C, but still want more performance than Python.
• Don’t need too much external/third-party support.

Will C slowly stop being a top choice for embedded development?

High-level languages make it easier to code by abstracting away the complexities that simply can’t be avoided in C. However, for decades, despite the release of fancier languages and frameworks, C has remained a top choice in embedded development. 

This is because the speed, performance, and reliability it offers, are still unparalleled. The fact that Python and large parts of the Java ecosystem (including the runtime, language, and compiler) are written in C, speaks volumes about the language’s efficiency and durability.

With that said, C is undeniably harder to learn for new developers, who prefer the simplicity of languages like Python. This is creating a noticeable decline in the size of the C developer community. The rise in development of AI and ML applications is also driving people to choose Python over C, simply because it offers greater library support.

It’s important to remember that Python has only recently become part of the top embedded programming language debate. Yes, its adoption may continue to rise in the years to come, but we don’t expect C to ever stop being a top choice for embedded development.

Why secure coding is important in C/C++

A lot can go wrong while writing code in C/C++. Accessing a memory address that no longer points to valid data, or incorrectly sharing data between threads, can cause your entire application to crash. 

For embedded devices, these implications are far more significant, as a crash inside them can cause a much larger machine to stop functioning. E.g. if the auto-pilot module in a car shuts down, the car will no longer be able to avoid obstacles.

Here are more reasons why secure coding of C/C++ applications is so important:

• Insecure memory handling (null pointers, memory corruption, stack overflows, buffer and heap overflows) cause the entire application to crash.
• Pointers are part and parcel of C and C++. Handling them requires utmost care, and in-depth knowledge of underlying concepts.
• Garbage collection is manual. This makes it compulsory for programmers to explicitly delete any dynamically allocated memory. Unfree or incorrectly freed memory can lead to memory leaks or corruption.
• There is a lack of standardized APIs for advanced data structures (e.g. hash tables, sets etc.), which forces developers to reinvent the wheel.

It’s absolutely critical that developers are trained in secure coding for C/C++ applications. Secure Code Warrior provides a personalized learning platform with interactive challenges, courses, and assessments that can enable developers to write secure C/C++ code. Here, we are talking about language:framework-specific content, not just a tweak on general conceptual training.

Check out how we help automotive and transportation industries. Or try an embed-focus coding challenge today to experience how we empower developers to write secure embedded applications!‍