Web technology detection and fingerprinting in Invicti

The Achilles’ heel of modern web applications

Modularity is crucial for rapid software development, so it is no surprise that today’s web applications can contain as little as 10% of custom code. The rest is scaffolding: frameworks, libraries, and external components that do the heavy lifting so application developers can focus on business logic. This is especially important in agile development, where work is divided into smaller parts that need to be completed in a matter of weeks. The shift from monolithic to modular applications has also affected application security. The growing awareness that security needs to be an integral part of the software development lifecycle (SDLC) means that custom application code usually goes through some sort of security testing to catch at least the most obvious security flaws. But since custom code is only one part of the total attack surface, attackers are increasingly turning their attention to weaknesses in the underlying technologies. This is where the reliance on convenient ready-made components becomes a liability. Today’s web applications, including commercial products, combine custom code and open-source components, with the latter commonly making up from 70% to 90% of the codebase. The problem is that as many as 79% of applications use libraries that are never updated after being included in the code, not even for critical security updates. Many of these components are used in thousands of applications worldwide, making it easy for attackers to find installations that use known vulnerable libraries. This is the web application aspect of supply chain vulnerabilities that were called out in Joe Biden’s recent presidential order as a priority area for cybersecurity.

Finding the many ways you can be vulnerable

Dynamic application security testing (DAST) solutions probe a running application for weaknesses. To accurately mimic the actions of real-life attackers, advanced products such as Invicti don’t stop at scanning your application for vulnerabilities but also detect its underlying technologies. Apart from using a cutting-edge heuristic scanning engine to detect previously unknown vulnerabilities, Invicti also has its own vulnerability database for reporting out-of-date libraries, frameworks, and other application components. The vulnerability database (VDB) brings together all of Invicti’s security checks and serves as a repository of known web technologies along with their versions and security status. The VDB is carefully curated by Invicti’s security researchers and periodically updated based on multiple sources of vulnerability intelligence to ensure that the latest web application CVEs and other known issues are covered. VDB updates are automatically reflected in the Invicti user interface and available for use in your scan policies.

3 steps to technology identification

Invicti starts by identifying JavaScript libraries, frameworks, runtimes, web servers, databases, and other components. Successfully identified technologies are reported for your information, and the scanner then attempts to detect the specific versions in use. Any versions that are discovered are reported as version disclosure issues since this information can be useful to attackers. Here is a typical request and response that allows the scanner to identify a specific version:

Having discovered the version, Invicti then queries its vulnerability database to see if a newer version is available and whether the current version has known vulnerabilities. Depending on the result, you will then get a warning about an out-of-date or vulnerable technology. All the reports and warnings about identified technologies are presented in the Technology section in the main menu for easy management. Here you can see all the technologies used in all your applications. For each one, you see the number of sites that use the technology, the number of identified versions, the version branches, the number of out-of-date versions, and the number of issues found. The overall security status for a technology corresponds to the highest severity vulnerability found across all installed versions.

Component fingerprinting

Identifying web application technologies and components is not a trivial task, so Invicti uses a variety of techniques to extract as much accurate intelligence as it can. Some products explicitly advertise their presence in response headers, so header analysis is the first and easiest step. For popular web applications such as WordPress, the product can also be reliably identified based on known headers, cookies, and directory structures. Invicti gathers all this information during the crawling phase. The scanner also runs more advanced pattern recognition on responses received during crawling to identify web servers, application servers, frameworks, programming languages, and more. Invicti then queries its vulnerability database for information about the latest available versions and known security issues with specific products and versions. For other components such as JavaScript libraries, Invicti  searches the script code for known patterns and also attempts to call functions specific to a known library or version. This lets the scanner accurately identify libraries at runtime even if they have been modified manually and don’t exactly match the original source. The scanning engine can fingerprint many popular libraries this way, such as jQuery, React, Angular, and Bootstrap. Again, if multiple versions are found (which is very common), Invicti will report the branches, the number of versions, and the overall security status for that library.

Get the full picture

A quality automated solution for dynamic security testing is a must-have in any modern web development workflow, especially for agile software projects. Invicti’s advanced heuristic vulnerability scanner with Proof-Based Scanning can reliably detect a wide variety of security issues in your entire web application as executed, covering vulnerabilities in both your custom code and external dependencies. Technology fingerprinting takes this a step further to help you manage your entire web application stack and avoid vulnerabilities introduced by outdated third-party components, even if you are not actively using a specific vulnerable function or module in your applications. You can also use the Technologies page to find abandoned servers and installations that needlessly increase your overall attack surface. Combined, these results give you a complete picture of your web application security posture – all from one reliable solution. For a detailed description of all the available information, see our support page for the Technologies feature.