Is Your Product Secure Enough? (Part 2 of 2)

For the past 30 years it has become a widely expected, common practice for the Enterprise Software vendors to commit to always shipping zero-defect software. Where any issues were to be found, vendors committed upfront to correct them within predetermined timeline depending on the issue severity. For example, customers would expect all Critical issues to be resolved within 30 days, and all High severity issues  to be resolved within 60 or 90 days.

These days however, it's becoming increasingly evident that this fixed timeline approach, while well intentioned, is impractical and in some ways delusional.

Software vendors can no longer commit in good faith to a fixed number of days it will take to address discovered/reported issues because

• The number of product security/quality issues we will face in the future is unknown. At any point in time, a new issue may be reported to or discovered by vendors internally (e.g. using code scanning tools).
• The amount of time to resolve or mitigate any security/quality issue is also unknown in advance.

Software businesses should classify all known security and quality issues into two groups, based on how they impact their operations. 

• Group 1: This category mandates we drop all our planned activities and throw at it as many of our resources as we have to address the issues as fast as possible (and yes, however improbable it may sound, we may run out of resources, much like some hospitals ran out of beds at the peak of COVID). You may designate these as all Sev 1 (and potentially Sev 2) quality and CVSS 9.0 and higher security issues. I want to stress this category of work trumps everything else. This applies to issues reported to us as well as to issues we discover ourselves.

• Group 2: This category of issues is managed in a planned manner using managed resource allocation (reviewed/adjusted periodically, e.g. quarterly) and disciplined prioritization. This may include Sev 3/4 quality and CVSS 8.9 and below security issues. For this to work, we must diligently rank all the issues on the list from 1 to N. At any given point, a developer or team assigned to work on this should grab from the list what’s at the top, not what has been sitting the longest on the list. Prioritizing High, Medium, or Low will not work as it leaves discretion to developers to pick from the pool of many issues using their discretion. If we don’t rank, we will not necessarily be addressing the most critical issues at the time. For security issues, we need to have the CVSS score calculated before making the change to the product to reliably justify the complete cost of making the change (not just editing the code). That cost for vendor and customers includes testing by us, packaging and distribution, communication internally and to customers, training our support, customers downloading and deploying a modified version of our product, and then in some cases testing their implementations before ultimately pushing them to their users.

The approach described here will scale better in the face of growing velocity of incoming security challenges and increased uncertainty in a complex world of enterprise software.

What is your experience with this?

Is Your Product Secure Enough? (Part 1 of 2)

There are different types of security vulnerabilities in every product, and I stress, every product.

• Those that no one discovered yet

• Those that have been discovered quietly, the world does not know about and
1. no one has exploited yet, or 
2. someone is quietly exploiting them as we speak 

• Those that the vendor knows about and has not yet made the patch available (for any number of reasons)

• Those for which the patch exists, but the users have not applied it (for any number of reasons)

My point is there is no such thing as defect/bug/vulnerability free software.

It’s a vendor's responsibility to find the best possible balance between the cost of the solution and the value it provides to the customers, minus the risk of harm it can cause.

A lot has been written about Zero-Defect Software, and while the intent is certainly nobel, organizations that solely focus on this, are missing the point. 

• This blog about Zero-Defect Software concept written through the eyes of QA lead (Nyall Lynch), applies equally well to dealing with security defects, commonly referred to as vulnerabilities.

The key question Product Managers need to answer: "What level of risk, given the nature of our product and its application, is acceptable" for us as a vendor. In other words: "Is our build secure enough to ship?" 

As well, responsible vendor will ensure the product releases already in customer hands remain secure enough for continued use until these releases reach the end of communicated maintenance window.

In Part 2 I will share some practices that can help large scale enterprise software vendors to continue delivering value in the face of growing uncertainty about product security risks.