The penetration testing report is a document that provides information about the vulnerabilities that the web application, network segment, or mobile app contains. More importantly, you can read about recommended actions to mitigate security issues.

However, there are certain things that you will not find in the penetration testing report.

Security guarantee

First and foremost, no penetration testing team will ever guarantee that they detect all vulnerabilities in your application. That is simply impossible. Not only penetration testing but any type of security verification cannot guarantee that. Software is very complex these days and the complexity increases every day. A modern web application has often different layers distinguished such as the front-end that is responsible for presenting data to users, back-end controllers responsible for processing requests, API as a means to access the application programmatically, and models that implement connection with the databases or some external data sources. Developers don’t code everything from scratch. They use public libraries and frameworks to deliver desired functionality faster and make it easier to enhance in the future. To make all this possible there has to be a webserver involved that processes HTTP requests, maybe some caching mechanism, or load balancer to handle high traffic. The webserver often uses additional extensions to allow certain capabilities. All of this could be wrapped in some containerization technology to make the deployment more manageable and deployed in the cloud to make it easier or cheaper.

I think you know where I’m going right now. Even before the web application is deployed, it utilizes a massive amount of code lines from different components. As you can imagine, many lines of such code can introduce vulnerability to the whole system.

I’m not even going down to low-level architecture where processors of different physical machines can execute undesired actions.

This covers only the complexity part of the equation. There is also the fact that the technologies that your application is using are updated every day.

The image below illustrates some technologies involved in the typical application process.

Needless to say that no security engineer, penetration tester, developer, or any security solution can fully protect all of the software involved.

This is similar to financial investments, where no one can guarantee that you will earn money if you invest in stock X.

Risk vs severity

Another thing that is not present in most penetration testing reports is the risks associated with detected vulnerabilities and business impacts. Whenever you see a vulnerability in your report and a “critical, high, medium, low, info, none” next to it, it is not a risk but a vulnerability severity (unless the proper risk assessment was conducted). Vulnerability severity can be calculated or simply assigned by the penetration tester who discovers the vulnerability. In Zigrin Security we use a well-known Common Vulnerability Scoring System (CVSS) to calculate it. Among other things, it reflects the technical impact of the vulnerability on confidentiality, integrity, and availability. So an example vulnerability that allows exfiltrating the database of the application could be rated as high severity. What it does not reflect, is the business impact, the likelihood of the attack, and the skills of the threat actors that could exploit it. This is because all the above not covered factors are not known to the penetration tester who analyzes your application. For example, the penetration tester does not know if the database that can be exfiltrated contains a large or small portion of your company’s data. If it’s small, even though the technical impact may be high, the business impact may be low. Therefore, it will be more important to fix other vulnerabilities that impact a wider audience or more business-critical assets.

The difference is especially important when there are numerous vulnerabilities and a limited budget for fixes. In that case, your company has to spend resources firstly on the vulnerabilities that pose the most risk to your company, users, and business. Not necessarily the vulnerabilities that have the highest technical severity.

Secure elements

Penetration testing’s core objective is to find vulnerabilities that can later be fixed or mitigated. The processes behind discovering vulnerabilities may be different depending on the methodology, tester, and so on. However, one is certain. Testers execute hundreds of tests manually and thousands of semi-automated tests. As a result of this intense testing, only a fraction of a percentage leads to an exploitable vulnerability.

Because of that, it’s impossible, or rather not productive to report all exploitation tests that failed. Even having a report with all the test cases executed and failed would be counter-productive as it would require the penetration tester to spend time documenting them instead of looking for vulnerabilities in other places of the application. Moreover, a recipient of such a report would get a false sense of security seeing 99,9% of items green (secure).

Additionally, there are multiple ways of testing for one single vulnerability. This means that even if the penetration tester tries to exploit a particular vulnerability in a specific component of a web application, there can be several potential ways to do the exploitation differently. It could also be possible to find a different vulnerability in the exact same component.

I hope this article shows you what not to expect out of the penetration testing report in contrast to previous articles.