Detecting Buffer-Overflow vulnerabilities using REVEN

Feb 11, 2021
by Quentin
Categories: REVEN -
Tags: Reverse Engineering - Vulnerability Detection - Analysis API - Taint - REVEN -

The REVEN Buffer-Overflow (BoF) detection capability is built on the top of the Use-after-Free (UaF) script. Therefore, it is best to read the UaF article before this one. We will explain how the UaF detection Jupyter notebook led us to develop a Buffer-Overflow detection Jupyter notebook (available on Github) and how we applied it to CVE-2020-17087 and CVE-2021-3156.

Detecting BoF using REVEN based on the UaF Jupyter notebook

The detection of the UaF presented in the previous article is composed of two steps:

  1. Detecting the allocations in the trace of the target scenario
  2. Analyzing the pointer dereferencings to detect UaF usage by following the pointer using the taint

Both UaF and BoF are invalid pointer usages, UaF occurs when the pointer is dereferenced after it has been freed, and BoF when the pointer is used to dereference memory outside the boundaries of the allocated buffer.

Accordingly, we changed the last step of the UaF detection algorithm to find if any buffer was dereferenced out-of-bounds, using the start address and the size of all buffers allocated during the recorded execution timespan.

The current algorithm only works on heap allocations and will not detect BoFs occurring on stack-allocated buffers. Extending this algorithm is a work in progress. We already extended it to detect allocations using malloc/free on Linux because CVE-2021-3156 is a Linux CVE.

The BoF Jupyter notebook has mostly the same input parameters as the UaF Jupyter notebook.

Results and Discussion

The root cause of CVE-2020-17087 is an integer-overflow which leads to a badly sized allocation, itself leading to a heap buffer-overflow. Our scenario for CVE-2020-17087 only contains 45 millions instructions with the integer-overflow and the heap buffer-overflow. This is relatively tiny for a REVEN scenario. It was achieved by using the automatic recording feature and a lightened Windows 10 VM.

CVE-2021-3156 is a heap buffer-overflow in the sudo library that could lead to privilege escalation. Our scenario for CVE-2021-3156 is containing 145 millions instructions.

The remainder of this section discusses the results obtained along several axes using these scenarios.


The two main steps of the algorithm were measured separately: finding Allocation/Deallocation and detecting BoF. The analysis was also done using 3 different focus for each scenario: first a specific buffer, second a specific binary in a specific process, and last the whole trace. This impacts a lot the number of addresses to be analyzed by the algorithm and its performance.

Scenario Perimeter analyzed Finding Allocation/Deallocation pairs Finding BoF
CVE-2020-17087 Buffer at 0x7c5a1450 23s 4min 45s
CVE-2020-17087 cng.sys binary 23s 10min 9s
CVE-2020-17087 Whole trace 34s 43min 52s
CVE-2021-3156 Buffer at 0x132498cd0 4s 17s
CVE-2021-3156 binary 4s 11m58s
CVE-2021-3156 Whole trace 7s 2h 23min 42s

As explained in the UaF article, we have already identified several areas of improvement that are also valid for BoF detection.

An important difference between UaF and BoF detections is that it wasn’t possible to use the same memory optimization as in the UaF Jupyter notebook, because the memory history is based on physical addresses whereas a BoF is done in the logical space and not the physical space. This slows down the execution of the BoF algorithm compared to UaF.


Like performance, accuracy also varies according to the scope analyzed.

Scenario Perimeter analyzed # Searched Addresses Total reported BoFs Confirmed BoFs Probably false positive
CVE-2020-17087 Buffer at 0x7c5a1450 1 1 1 0
CVE-2020-17087 cng.sys binary 4 1 1 0
CVE-2020-17087 Whole trace 670 21 1 20
CVE-2021-3156 Buffer at 0x132498cd0 1 1 1 0
CVE-2021-3156 binary 129 11 1 10
CVE-2021-3156 Whole trace 395 104 1 103

As the UaF article explains, the majority of false positives come from the mismatch between the general taint propagation semantic and the “pointer aliasing semantic” that is required for tracking pointers very accurately.

“False-positive” in the glibc

With the notebook we also detect a lot of BoF in the glibc on Linux. That’s because it is purposely reading after a buffer in a controlled manner (ensuring that the access won’t trigger a page fault leading to a crash) in some functions (e.g strncmp and strlen) to have the best optimization possible.

As those BoF are controlled and only on read they aren’t dangerous as long as the controls are correct.

Note that the false positive BoFs related to this aren’t filtered out in the current version and are included in the numbers reported above.

Usability & Perimeter

The use and the perimeter of this Jupyter notebook are the same as those of the Jupyter UaF notebook because they are based on the same allocation detection and pointer tracking algorithms.


This notebook is already available to detect BoF vulnerabilities. The REVEN traces used for this article are also available to our users: contact support.

For future work, the final thought is that refining the tracking of pointers will greatly improve the accuracy and the performance of both the UaF Jupyter notebook and the BoF Jupyter notebook. Extending the allocation detection will allow users to detect both UaF and BoF on the stack.

Next post: REVEN OpenLab - Feb 18th, 2021
Previous post: Finding uses of cryptographic functions and the data encrypted by an application