As part my responsibilities with governance of IT and Technology, I need to be able to validate certain things that team members, and contractors say. Recently, I had a solo developer inform me that they develop their software securely, but they do not follow and have not implemented any Secure Software Development Lifecycles. And that got me thinking, how would I validate that this developer actually is writing source code securely? I mean, they may be, but they may also be writing their code in such a way that there are still security issues such as not locking down a controller that allows non-approved users to create accounts.

And it is possible that the solo developer may be capable of writing secure software. But I had my doubts and needed a way of evaluating the statement that the software is developed securely.

There are 5 things that I did to evaluate if it is possible that the software is developed securely without direct involvement to the overall development process:

• Technical vulnerability analysis tools.
• Review the software/programs requirements.
• Review the tests performed.
• Reaction time: a vulnerability scenario.
• Questions for the developer to answer.

Technical Vulnerability Analysis

The first way is easy, I can use a variety of tools to probe the system externally. This requires little to no further interactions with the developer and usually reveals things such as insecure front-end libraries (jQuery, or other JavaScript libraries), and it may also reveal that the web server or proxy server has been setup incorrectly, though that may be the responsibility of a different person/team. But it also has the potential to reveal that a version of a framework (such as dotnet Framework) is vulnerable to certain attacks. I will use a variety of tools that include ZED Attack Proxy, Greenbone, and Nessus.

Typically, it is not that there is a presence of vulnerabilities that causes me to question if the software is not developed securely. Instead, I am looking at the list of detected CVEs and evaluating the age, the severity, complexity of exploit, and probing to see if there has been any form of mitigation applied.

For example, if the highest CVE has a CVSS rating of 6 or higher, is more than 2 years old, and lacks any mitigation measures (such as a Web Application Firewall, a proof-of-concept exploit executes, monkey-patched code, or a developer-introduced hotfix), I will start to side with the software is not developed securely.

But, if the highest CVE that I see has a CVSS rating of 3, its ID is indicated that it is only 3 months old, or there is an indication that there is a mitigation in place. I am going to start siding with the software may be developed securely.

Review the requirements

Often this will require that the developer provides me with access to the requirements documentation and gathering system. If none exists, or is extremely ad-hoc, that is a further indicator that the software is not developed securely (because how can any tests be performed if the requirements are not even recorded?).

But, if a system such as Azure DevOps, or Atlassian Jira is in use, I can review the requirements. Sometimes the security requirements are not explicitly called out, but it may indicate that thought and design has been put into things such as user access, permissions, and controls.

One of the biggest tells, is that there is no documentation anywhere of how user authorisation will work. Even if it is stated that “we have roles and specific permissions”, if it is not documented how these will apply to accounts, then the system is unlikely to have been developed securely.

Review the tests performed

This one can typically be skipped over if the requirements are not documented. If the system is not being documented, then it is not going to be tested in a way that will be constructive for checking for security, much less the overall system behaviour or intent.

However, if there is a formal testing system in place, I evaluate the level of maturity. Here are some key points I consider:

Continuous Integration/Continuous Deployment (CI/CD): Does one exist? How well does it run? I evaluate the CI/CD pipeline to ensure that tests are run automatically with each code change. A robust CI/CD pipeline can help catch security issues early in the development process. However, if a CI/CD is not in place, this may also play into the reaction time scenarios.

Test Coverage: Ensure that the tests cover all critical functionalities, including security features. This includes unit tests, integration tests, and end-to-end tests. While I do not have any requirement that a certain percentage of the system be formally tested, it is more concerning when there are none. It may also be a tell about the maturity and age of the project more than the individual developer if the entire scope of tests are not present.

Automated Testing: Check if automated testing tools are used. Automated tests can quickly identify security issues and ensure that new code changes do not introduce vulnerabilities. This may include tools such as Dependabot in GitHub to let the developer know when a vulnerable dependency has been detected.

Security-Specific Tests: Look for tests specifically designed to identify security vulnerabilities, such as SQL injection, cross-site scripting (XSS), and cross-site request forgery (CSRF). This also can include evaluating any tests that evaluate the configuration files for any embedded web servers (Kestrel) or proxy servers that may be in use.

Test Results: Review the results of the tests. Look for any failed tests and how they were addressed. Consistent failures or ignored test results can indicate potential security issues.

But, if none of this in place, I am very likely to be in the camp of, the software is not developed securely. If it is in place, the developer may be developing securely.

Reaction time: a vulnerability scenario

This is creating a scenario where the developer is informed of a fictious security vulnerability that has been found in the system and is under active exploit in the wild. The mitigation is to increment the version number of the package as it has been patched and does not require any additional development work.

A developer who has a robust CI/CD should be able to deploy a new version in a very short time frame, usually in less than 2 hours based on that scenario. Depending on the maturity of the CI/CD, it may be able to be done completely automatically. However, if they are needing to remote into a system and manually copy changes between their development environment, and the production environment, it is likely that they are not developing securely.

If the developer is unable to deploy a rapid fix, then it is unlikely that they are developing securely. The other scenarios that I also use are:

• Close a logical security issue such as locking down how users can create an account.
• Update a configuration to prevent system leaks user information.
• Update a configuration of the server to prevent leakage of system details.
• “Rolling back” an update that introduced a security issue.

While a rapid deployment can be faked, or completed rapidly once off, if a CI/CD is not in place it is an indicator that the system is not being built in repeatable way, may be susceptible to a supply chain attack, and cannot roll out security fixes in a timely manner. This last point is even more critical with the number of exploits gaining proof of concepts shortly after being patched by the vendor.

Audit questions for the developer

To further evaluate the security of the software, I ask the developer a series of audit questions. These questions help assess their understanding of secure coding practices and their implementation. Here are some sample questions:

1. Secure Coding Practices:
   • What is your understanding of the OWASP top 10?
   • How do you ensure that your code is free from common vulnerabilities such as SQL injection and XSS?
2. Code Reviews:
   • Are code reviews conducted? If so, what is the process?
   • How do you ensure that security issues are identified and addressed during code reviews?
3. Dependency Management:
   • How do you manage third-party dependencies?
   • How do you ensure that the dependencies you use are free from known vulnerabilities?
   • How do you ensure that the dependencies on your local development environment are the same versions as the ones in the production environment?
4. Authentication and Authorization:
   • How do you handle user authentication and authorisation?
   • What measures do you take to ensure that only authorised users can access sensitive data and functionalities?
5. Data Protection:
   • How do you protect sensitive data, both in transit and at rest?
   • What encryption methods do you use to secure data?
   • How are the application credentials protected?
6. Security Testing:
   • What security testing tools do you use?
   • How often do you perform security testing on your codebase?
7. Testing, Staging, Production:
   • What sort of separation is there between the environments?
   • What data is used in the development, testing, staging environments?

By asking these questions, I can gain a better understanding of the developer’s approach to security and identify any potential gaps in their practices.

But is it developed securely?

The purpose of this process is to ensure that I can believe the answer that the developer gave me of: “The software is developed securely”. And I can believe the consideration for specific security requirements has been taken on during the development. While not every test needs to be met, the majority should be, especially when industry standards and best practices have established defined SSDLCs. Without this adherence, it is difficult to trust the developer is developing securely.

Additionally, the risk profile of the client, project, or system plays a crucial role in this assessment. This also has the effect of getting to understand the specific implementations that the developer has done so as not to be checklist risk management.