SOURCE ARCHIVE
EXTRACTED CONTENT
137,303 charsarXiv:2603.25354v1 [cs.CR] 26 Mar 2026
Master’s Thesis (Academic Year 2025)
Multi-target Coverage-based Greybox Fuzzing
Institute of Information Security Graduate School of Information Security
5543501 Masami Ichikawa
Supervisor Kuniyasu Suzaki
January 2025
Abstract of Master’s Thesis (2025)
Multi-target Coverage-based Greybox Fuzzing
Summary
In recent years, fuzzing has been widely applied not only to application software but
also to system software, including the Linux kernel and firmware, and has become a power- ful technique for vulnerability discovery. Among these approaches, Coverage-based grey- box fuzzing, which utilizes runtime code coverage information, has become the dominant methodology. Conventional fuzzing techniques primarily target a single software compo- nent and have paid little attention to cooperative execution with other software. However, modern system software architectures commonly consist of firmware and an operating sys- tem that operate cooperatively through well-defined interfaces, such as OpenSBI in the RISC-V architecture and OP-TEE in the ARM architecture. In this study, we investigate fuzzing techniques for architectures in which an operating system and firmware operate cooperatively. In particular, we propose a fuzzing method that enables deeper exploration of the system by leveraging the code coverage of each coop- erating software component as feedback, compared to conventional Single-target fuzzing. To observe the execution of the operating system and firmware in a unified manner, our method adopts QEMU as a virtualization environment and executes fuzzing by booting the system within a virtual machine. This enables the measurement of code coverage across software boundaries. Furthermore, we implemented the proposed method as a Multi-target Coverage-based Greybox Fuzzer called MTCFuzz and evaluated its effectiveness.
Keywords
Fuzzing, Coverage-based Greybox Fuzzing
Graduate School of Information Security
Institute of Information Security
5543501 Masami Ichikawa
i
Contents
1 Introduction 1 1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Objectives and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Organization of This Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Fuzzing 6 2.1 Fuzzing Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Greybox Fuzzing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Code Coverage Measurement Techniques . . . . . . . . . . . . . . . . . . . . 9 2.3.1 The code coverage measurement in the AFL . . . . . . . . . . . . . . 10 2.3.2 The code coverage measurement in the Linux kernel . . . . . . . . . 14 2.4 Background and Limitations of Coverage-Guided Fuzzing . . . . . . . . . . 14
3 Proposed Approach 15 3.1 Multi-target Coverage-based Greybox Fuzzing Method . . . . . . . . . . . . 15 3.2 Design of MTCFuzz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.3 Implementation of MTCFuzz . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.4 Summary of the Proposed Approach . . . . . . . . . . . . . . . . . . . . . . 30
4 Evaluation 32 4.1 Experimental Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2 Experimental Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2.1 Coverage Evaluation of OpenSBI Base Extension . . . . . . . . . . . 33 4.2.2 Coverage evaluation of OP-TEE by long running . . . . . . . . . . . 34 4.2.3 QEMU performance overhead . . . . . . . . . . . . . . . . . . . . . . 34 4.2.4 Address filter performance . . . . . . . . . . . . . . . . . . . . . . . . 34 4.2.5 Snapshot load performance . . . . . . . . . . . . . . . . . . . . . . . 35 4.2.6 Fuzzing OP-TEE cryptographic API . . . . . . . . . . . . . . . . . . 35 4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.3.1 Coverage Evaluation of OpenSBI Base Extension results . . . . . . . 35 4.3.2 Coverage evaluation of OP-TEE by long running results . . . . . . . 36 4.3.3 QEMU performance overhead results . . . . . . . . . . . . . . . . . . 42 4.3.4 Address filter performance results . . . . . . . . . . . . . . . . . . . . 44 4.3.5 Snapshot load performance results . . . . . . . . . . . . . . . . . . . 45 4.3.6 Fuzzing OP-TEE cryptographic API . . . . . . . . . . . . . . . . . . 45 4.4 Summary of Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
ii
5 Discussion 48 5.1 Effectiveness of the Proposed Approach . . . . . . . . . . . . . . . . . . . . 48 5.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.3 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.4 Summary of Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6 Related Work 51
7 Future Work 54
8 Conclusion 55
Acknowledgment 55
References 47
A Terminology 61
B Additional Details 62
iii
List of Figures
1.1 Memory layout on RISC-V . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 TrustZone for Cortex-A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 Example of basic blocks as displayed by Ghidra . . . . . . . . . . . . . . . 12 2.2 Example of basic block transitions in AFL . . . . . . . . . . . . . . . . . . 13
3.1 Execution flow across software boundaries . . . . . . . . . . . . . . . . . . 16 3.2 Architecture of MTCFuzz based on QEMU. . . . . . . . . . . . . . . . . . . 17 3.3 Overview of the fuzzing workflow in MTCFuzz. . . . . . . . . . . . . . . . . 28 3.4 Coverage visualization in HTML . . . . . . . . . . . . . . . . . . . . . . . . 29 3.5 Coverage differences between multi and single . . . . . . . . . . . . . . . 30
4.1 Coverage difference between Multi and Single (2 hours) . . . . . . . . . . . 38 4.2 Total coverage of the Multi and Single (2 hours) . . . . . . . . . . . . . . . 38 4.3 Total kernel coverage Multi vs. Single (2 hours) . . . . . . . . . . . . . . . 39 4.4 Total firmware coverage Multi vs. Single (2 hours) . . . . . . . . . . . . . . 39 4.5 Total coverage Multi vs. Single (8 hours) . . . . . . . . . . . . . . . . . . . 40 4.6 Total coverage Multi vs. Single (24 hours) . . . . . . . . . . . . . . . . . . 40 4.7 Total coverage Multi vs. Single 2h/8h/24h running results merged . . . . . 41 4.8 Visualized qemu overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
iv
List of Tables
1.1 Software Classification Based on Address Ranges Observed in trace logs . . 4
2.1 Common options for instrumentation . . . . . . . . . . . . . . . . . . . . . . 11
4.1 Experimental environments . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2 Software versions used in the experiments . . . . . . . . . . . . . . . . . . . 33 4.3 Configuration files used in fuzzing experiments . . . . . . . . . . . . . . . . 33 4.4 Number of campaigns that achieved full coverage . . . . . . . . . . . . . . . 36 4.5 Average number of coverage campaigns . . . . . . . . . . . . . . . . . . . . . 36 4.6 hackbench: Process and Thread Performance (Env1) . . . . . . . . . . . . . 43 4.7 hackbench: Process and Thread Performance (Env2) . . . . . . . . . . . . . 43 4.8 Performance and Overhead Comparison by CPU (hackbench process mode) 43 4.9 Fuzzing Execution Counts under Different Filter Configurations . . . . . . . 44 4.10 Statistical Summary of Test Execution Counts . . . . . . . . . . . . . . . . 45 4.11 Comparison of test case execution counts with and without snapshot loading 45
6.1 Comparison of system fuzzing frameworks . . . . . . . . . . . . . . . . . . . 52
v
List of Algorithms
2.1 Basic Fuzzing Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Greybox Fuzzing Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1 Coverage Classification for Kernel and Firmware Address Spaces . . . . . . 23 3.2 VM Snapshot Save via QMP . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.3 VM Snapshot Load via QMP . . . . . . . . . . . . . . . . . . . . . . . . . . 26
vi
List of Listings
2.1 Simple if condition example . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 AFL’s code coverage instrumentation logic. . . . . . . . . . . . . . . . . . . 10 2.3 AFL’s QEMU mode code coverage instrumentation logic. . . . . . . . . . . 10 2.4 Compiler flags for KCOV instrumentation in the Linux kernel . . . . . . . . 11 2.5 Example output of KCOV in trace-pc mode. . . . . . . . . . . . . . . . . . 11 3.1 Address filters setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 Definition of APIs for code coverage instrumentation . . . . . . . . . . . . . 21 3.3 Implementation of instrumentation code in TCG translation process . . . . 22 3.4 Implementation of writting trace log to file . . . . . . . . . . . . . . . . . . 22 3.5 QEMU command-line option for snapshot feature . . . . . . . . . . . . . . . 24 3.6 Execute test case via ssh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.7 Resolving a kernel address using addr2line . . . . . . . . . . . . . . . . . . . 29 4.1 Fedora 43 total symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.2 Total symbols in tested RISC-V kernel . . . . . . . . . . . . . . . . . . . . . 35 4.3 bpftrace on-CPU profile script . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4 Kernel OOPS log by CVE-2025-40031. . . . . . . . . . . . . . . . . . . . . . 46 B.1 Linux kernel target list used in CGF with Multi-Target Coverage Feedback 62 B.2 optee_os target list used in CGF with Multi-Target Coverage Feedback . . 62
vii
Chapter 1
Introduction
1.1 Background and Motivation
In recent years, fuzzing has been widely used not only in application software but also in lower-level areas such as the Linux kernel and firmware. In particular, Coverage-based Greybox Fuzzing(CGF), which uses code coverage at runtime as feedback, has become mainstream due to its efficient search capabilities. In open source software development, the OSS-FUZZ project [6] is continuously fuzzing over 1,000 projects as of March 2025. In the Linux kernel, fuzzing is performed continuously using syzkaller [2]. As with soft- ware testing, fuzzing can be classified into three types: whitebox-, blackbox-, and greybox fuzzing. In recent years, the greybox fuzzing method, which utilizes code coverage infor- mation during program execution, has become widely used. The main greybox fuzzers are AFL++ [19], which primarily targets application software, and libFuzzer [5], which can fuzz a variety of applications and libraries. The syzkaller is a widely used fuzzer targeting operating systems (OS), and supports fuzzing not only for the Linux kernel but also for BSD-based OSes such as FreeBSD and NetBSD, as well as Android and Windows. Fuzzers include not only tools developed for testing specific software, but also those that are capable of targeting a wide range of programs. However, in typical fuzzing tools, only one piece of software is selected as the fuzzing target. As a result, code coverage information from other software components that operate in cooperation with the target, such as database systems or firmware, is generally not taken into account. However, modern system software is generally configured so that multiple pieces of software operate cooperatively through specific interfaces. Examples include collaboration between Linux and OpenSBI in the RISC-V architecture [32], the Normal World where Linux runs, and the Secure World where Trusted OS runs in TrustZone [31] in the ARM architecture. In this paper, TrustZone refers to TrustZone for Cortex-A. These architectures use the same physical memory as the OS, but memory is isolated by address ranges. For example, Figure 1.1 shows the RISC-V memory layout, and the blue box shows the shared memory area between OpenSBI and the Linux kernel. Figure 1.2 shows the TrustZone architecture. Each software component is isolated by its address range that is the same as RISC-V. Table 1.1 shows a classification of software components in the TrustZone environment based on address ranges extracted from fuzzing execution traces. Thus, it can be confirmed
1
that multiple software components each occupy their own memory address ranges within
the system memory.
Software Physical memory
components linear address OxFFFFFFFFFFFFFFFF
X
Linux Kernel Userland address
space
shared memory(Linux
kernel and OpenSBI)
| OpenSBI i OpenSBIspaceJ address
0x0
Figure 1.1: Memory layout on RISC-V, where the User application, Linux kernel, and OpenSBI operate in isolated address ranges while interacting through a shared memory region.
Because the OS and firmware are developed independently and share the same physi-
cal memory through well-defined interfaces, incorrect input validation or inconsistent state handling at these interfaces can propagate across components. Since they are often de- veloped by different teams or communities, gaps in the interpretation of specifications or assumptions about interface behavior can also become sources of subtle bugs. For example, data that should have been rejected by the firmware may instead be processed normally due to an implementation flaw, leading to subtle inconsistencies between OS and firmware states. Such issues have been observed in practice; for example, OpenSBI has fixed bugs related to feature value checking [25]. If such a bug directly causes a firmware crash, it can be detected by fuzzing the firmware. However, if the crash occurs later due to state
2
ARM (Cortex-A)
TrustZone architecture
Normal World Secure World
ELO: User Application Application
EL2: Hypervisor | 1
I |
EL3: Secure | Trusted Firmware-A I
Figure 1.2: TrustZone architecture for Cortex-A, illustrating the separation between the
Normal World and the Secure World.
inconsistency on the OS side, it may be difficult to detect using fuzzing that targets only
the firmware.
In summary, conventional CGF mainly targets a single software component. Hence,
it does not measure code coverage across software boundaries even in software where
multiple components operate cooperatively. As a result, newly reached code coverage on
the cooperating software side cannot be detected or utilized as feedback, and in modern
system software with complex cooperative execution environments, this limitation leads
to reduced fuzzing efficiency.
3
Table 1.1: Software Classification Based on Address Ranges Observed in ARMv8 Trust- Zone fuzzing trace logs.
Address Ranges Software
0xe090000 – 0xe0a00c4 Trusted Firmware-A
0xe100000 – 0xe19f4c8 Trusted OS
0x40000400 – 0x402ab510 Trusted Application
0xaaaaab0e55d4 – 0xaaaae95ff1ac Linux: User Application 0xffff814d3ac4 – 0xffff80008110f928 Linux: Kernel Land
1.2 Objectives and Contributions
In this study, our aim is to propose a fuzzing approach that simultaneously targets
multiple cooperating software components and leverages their respective code coverage
information as feedback, thereby enabling deeper system exploration than conventional
methods.
• We enabled code coverage measurement without modifying the source code or bina-
ries of the fuzzing target software.
• We realized coverage tracing of executed basic blocks by QEMU.
• We demonstrated the generality of our approach through empirical evaluation across
multiple architectures (ARM and RISC-V).
• We showed that using the code coverage of cooperating software components as
feedback can improve the exploration capability of fuzzing.
• We developed a prototype fuzzer called the Multi-target Coverage-based Greybox Fuzzer (MTCFuzz) that implements the proposed approach [3].
• We discovered a previously unknown vulnerability, assigned to CVE-2025-40031 [10].
• We received the OWS Research Award (2025) at the Computer Security Symposium
2025 [11].
1.3 Organization of This Thesis
The organization of this thesis is as follows. Chapter 2 explains the fundamental tech- nologies of fuzzing, including fuzzing techniques, greybox fuzzing, and code coverage mea- surement, and clarifies the limitations of conventional coverage-guided fuzzing. Chapter 3 describes the proposed Multi-target Coverage-based Greybox Fuzzing approach, presents its design, and outlines the prototype implementation, MTCFuzz. Chapter 4 presents the experiments conducted to evaluate the effectiveness of the proposed method and discusses the results. Chapter 5 discusses the effectiveness and limitations of the proposed approach based on the evaluation results. Chapter 6 reviews related research on QEMU-based fuzzing
4
frameworks and fuzzers targeting kernels and Trusted OS. Chapter 7 describes future work to further enhance and apply the proposed method. Finally, Chapter 8 concludes this thesis.
5
Chapter 2
Fuzzing
Fuzzing is a major software testing method these days. The technique of testing programs by providing random inputs was empirically known to be effective as early as the 1950s, when programs were entered using punch cards [39]. At that time, discarded punch cards were sometimes used as test inputs. In 1990, Miller et al. conducted an empirical study to evaluate the robustness of UNIX utilities against random, invalid inputs, introducing the term “fuzz“. In this paper, the term “fuzz“ refers to random characters used for this testing method [28]. Motivated by accidental crashes caused by line noise during remote logins, they systematically fed random inputs to 88 UNIX utility programs across multiple systems. Their results showed that 25-33% of the tested programs crashed or hung, revealing numerous latent bugs such as buffer overflows, unchecked return values, and pointer errors.
2.1 Fuzzing Techniques
Fuzzing is a testing technique in which programs are exercised using randomly gen- erated inputs. As such, it encompasses multiple approaches. Fuzzing can be categorized into several types. One common way to classify fuzzing is based on whether the internal structure of the software under test (SUT) is taken into account, similar to whitebox and blackbox testing in software testing. According to this classification, fuzzing can be divided into three types: whitebox-, blackbox-, and greybox fuzzing. In addition, in terms of test case generation methods, there are Mutation-based fuzzing, which mutates existing test case templates, and Generation-based fuzzing, which creates input from scratch. Several fuzzing approaches are named using Color-based terminology, such as black and white. These colors are used in the same sense as blackbox and whitebox testing in software testing. In this chapter, we describe the main techniques among them.
Whitebox fuzzing:This approach analyzes the source code of the SUT and generates test cases using knowledge of internal structure and data structures in the SUT. Therefore, these internal knowledge can be used to guide test case generation, enabling higher code coverage. In whitebox fuzzing, fuzzers are implemented using symbolic execution [23, 35, 40].
6
Blackbox fuzzing:Unlike whitebox fuzzing,blackbox fuzzing does not analyze the SUT. Test cases are therefore generated on the basis of visible specifications, such as file formats, API interfaces, etc. The test result is judged by detecting crashes or abnormal signals. However, because blackbox fuzzing has no knowledge of the program’s internal structure, it faces challenges to achieve high code coverage [22]. For example, in the function shown in Listing 2.1, the probability that x equals 0xdeadbeef , which is 1/232 . This is a very small probability, making it extremely unlikely that random mutations in blackbox fuzzing will exercise the path that evaluates the condition to true.
Listing 2.1: Simple if condition example
1 bool f( u32 x)
2 {
3 if (x == 0 xdeadbeef)
4 return true ;
5
6 return false ;
7 }
Algorithm 2.1 Basic Fuzzing Algorithm
Require: Seed input S
1: S† ← ∅
2: repeat
3: s ← ChooseNext(S)
4: s′ ← MutateInput(s)
5: t ← Execute(s′)
6: if t is crashed then
7: S† ← S† + s′
8: end if
9: until Timeout reaches or all the seeds are checked
Ensure: Crash inputs S†
Mutation-based fuzzing:In this approach, seeds that serve as templates for inputs are prepared, and during the mutation phase, these seeds are mutated to generate new test cases. Once the sample data are prepared, fuzzing can begin immediately. Although this approach allows data to be modified without requiring detailed knowledge of file formats or protocol specifications, the quality of generated test cases largely depends on the templates that were initially provided [29].
Generation-based fuzzing:Unlike Mutation-based fuzzing, Generation-based fuzzing does not rely on templates as the basis for test cases; instead, it constructs test cases from scratch. Therefore, it lacks the convenience of Mutation-based fuzzing, which can be easily executed once the sample data are prepared. However, because test cases are Generated- based on the specifications, Generation-based fuzzing can produce higher quality test cases
7
than Mutation-based fuzzing. Miller et al. reported that, in their study on the mutation of the PNG file format, Mutation-based fuzzing achieved only 24% of the code coverage obtained by Generation-based fuzzing on average [29].
Snapshot-based fuzzing:This approach saves the runtime state and restores that saved state after each test execution, thereby returning the execution environment to its pre- test condition. This enables efficient testing by avoiding the need to re-execute the setup steps required for each test. Moreover, some approaches do not use snapshots solely to save and restore the state, but instead mutate the saved state itself [34]. Chaofan et al. implemented ItyFuzz [34], it freezes the program state and performs fuzzing using two complementary techniques: traditional mutations applied to transaction inputs and mutations applied directly to the saved state itself. Because the smart contracts targeted by ItyFuzz treats state as a critical component, mutating the state enables the discovery of state dependent bugs that cannot be revealed by input only mutations.
2.2 Greybox Fuzzing
The Greybox fuzzing is major fuzzing technique nowadays. As its name suggests, this approach lies between whitebox and blackbox fuzzing. This approach does not analyze the SUT to the same extent as whitebox fuzzing; instead, it performs lightweight analysis or instrumentation and leverages that information to guide test case generation. Thus, while greybox fuzzing performs only lightweight analysis, the predominant technique is to measure code coverage and use it as feedback for test case generation. This methods that leverage code coverage in this manner are referred to as Coverage-based Greybox Fuzzing (CGF). The basic algorithm for blackbox fuzzing is given by the steps shown in Algorithm 2.1. The algorithm executes the tests as follows: 1) select a seed (line 3), 2) mutate the selected seed (line 4), 3) run the test (line 5), and 4) observe the execution and, if the program crashes, save the test case used (lines 6–7). The corresponding algorithm for CGF is illustrated in Algorithm 2.2. This algorithm is used by AFL [41]. There are two differences from black box fuzzing: the first is the AssignEnergy function at line 4 and the second is the IsInteresting check at line 10.
AssignEnergy: In blackbox fuzzing, no priorities are assigned to seeds, and all seeds are mutated uniformly. The AssignEnergy function assigns a priority, also known as energy, to each seed and determines how many mutations to perform for that seed according to its assigned energy (the assignment of priority at line 4 of Algorithm 2.2 and the loop at line 5 correspond to this behavior). The AssignEnergy function assigns energy to each seed primarily based on the following criteria. It takes a seed as input and assigns energy according to that seed. In the AFL, the energy assigned by the AssignEnergy function is calculated based on execution time, basic block code coverage, and the time at which the seed was generated [12]. In AFL, the functions calculate_score and fuzz_one correspond to AssignEnergy.
8
Algorithm 2.2 Greybox Fuzzing Algorithm
Require: Seed input S
1: S† ← ∅
2: repeat
3: s ← ChooseNext(S)
4: p ← AssignEnergy(s)
5: for i from 1 to p do
6: s′ ← MutateInput(s)
7: t ← Execute(s′)
8: if t is crashed then
9: S† ← S† + s′
10: else if IsInteresting(t) then
11: S ← S + s′
12: end if
13: end for
14: until Timeout reaches or all the seeds are checked
Ensure: Crash inputs S†
IsInteresting: The IsInteresting function determines whether a test case, although not causing a program crash, has produced an interesting test result. Here, “interesting” refers to cases such as reaching new code coverage or receiving a signal during execution. Test cases determined to be interesting by the IsInteresting function are added as new seeds. This enables, for example, to perform mutations on seeds that have reached new code coverage, which can in turn be expected to yield further coverage gains. Because blackbox fuzzing observes only crashes and does not monitor other behaviors, it cannot detect behavioral changes caused by different test cases. In contrast, CGF inspects the program’s runtime behavior through the IsInteresting function, enabling the generation of more effective test cases. In AFL, the functions save_if_interesting and has_new_bits in afl-fuzz.c correspond to IsInteresting.
2.3 Code Coverage Measurement Techniques
There are several methods to measure the coverage of the code. These include using compiler provided code coverage instrumentation [21, 38], extending compiler functionality to embed custom coverage instrumentation code [41, 19], embedding coverage instrumen- tation code as part of the software itself [13], measuring coverage through virtualized environments such as QEMU [1], and using hardware tracking features to collect coverage information [33]. Each method offers its own advantages; however, when fuzzing multiple independently developed software components simultaneously—such as a Linux kernel and firmware the method for measuring code coverage must be unified between components. If the granularity of coverage differs among targets, it becomes difficult to integrate feedback between them. In this section, we describe three code coverage measurement techniques: SUT compile-
9
Listing 2.2: AFL’s code coverage instrumentation logic.
1 cur_location = < COMPILE_TIME_RANDOM >;
2 shared_mem[ cur_location ^ prev_location ]++;
3 prev_location = cur_location >> 1;
Listing 2.3: AFL’s QEMU mode code coverage instrumentation logic.
1 if ( block_address > elf_text_start && block_address <
elf_text_end) {
2
3 cur_location = ( block_address >> 4) ^ ( block_address <<
8) ;
4 shared_mem[ cur_location ^ prev_location ]++;
5 prev_location = cur_location >> 1;
6
7 }
time instrumentation and QEMU mode coverage instrumentation by AFL [41], and the KCOV feature in the Linux kernel [13].
2.3.1 The code coverage measurement in the AFL
First, we explain how AFL embeds coverage instrumentation into the SUT. In AFL, a dedicated wrapper is used to compile the SUT, and during compilation, the code shown in Listing 2.2 is inserted in each basic block(BB). A basic block is a maximal sequence of instructions that is executed linearly, without any jumps or branches inside the block. An example of a basic block is shown in Figure 2.1. The value of cur_location on line 1 is a unique identifier determined at the time of compilation for each basic block of the SUT. The variable prev_location represents the basic block that was executed immediately prior to reaching the current one. In line 2, the XOR of cur_location and prev_location is used as an index in the shared memory array, and the number of transitions to the current BB is incremented at that index. In line 3, prev_location is updated to cur_location shifted right by one bit. This technique distinguishes the direction of transitions: moving from BBi to BBj and from BBj to BBi results in different indices, allowing the fuzzer to differentiate the transition directions. For example, consider a transition from BBi to BBj . Suppose that in BBi, cur_location is 0x4000 and prev_location is 0x1000. In this case, the index in the shared memory becomes 0x5000, and prev_location is updated to 0x2000. Then, when execution reaches BBj , where cur_location is 0x3000, the shared-memory index becomes 0x1000. If the execution later changes from BBj to BBi, prev_location is updated to 0x1800 in BBj . When the execution reaches BBi, the shared-memory index becomes 0x4000^0x1800 = 0x4400. Thus, the indices used for the transitions BBi → BBj and BBj → BBi differ, allowing the fuzzer to distinguish the direction of the transition. Figure 2.2 illustrates the transition states in this example.
10
Table 2.1: Common options for instrumentation
Option Inserted functions
-fsanitize-coverage=trace-pc __sanitizer_cov_trace_pc
-fsanitize-coverage=trace-cmp __sanitizer_cov_trace_cmp{1,2,4,8}
__sanitizer_cov_trace_const_cmp{1,2,4,8}
__sanitizer_cov_trace_switch
Listing 2.4: Compiler flags for KCOV instrumentation in the Linux kernel
1 kcov - flags -y
+= - fsanitize - coverage = trace - pc
2 kcov - flags -$( CONFIG_KCOV_ENABLE_COMPARISONS ) += - fsanitize
- coverage = trace - cmp
Next, we describe how code coverage is measured in QEMU mode. Because QEMU mode does not require compiling the SUT as part of the fuzzing setup, no instrumentation code is embedded into the SUT itself. Instead, coverage is measured within QEMU by treating a basic block as a unit of instrumentation [41]. When QEMU performs emulation, it does not execute the target architecture’s machine code directly. Instead, it translates the code into an intermediate representation called Tiny Code Generator (TCG) [15], using basic blocks as a translation unit. The method used in QEMU mode to measure code coverage is shown in Listing 2.3. In this approach, coverage is recorded only when the executed address lies within the text section of the ELF binary. This is because, in the ELF format, machine instructions are placed in the text section [24]. The sub- sequent operations on cur_location, shared_mem, and prev_location follow the same procedure as described above. However, unlike compile time instrumentation, the value of cur_location is derived by combining shifts and XOR operations on the basic block address to ensure its uniqueness.
Listing 2.5: Example output of KCOV in trace-pc mode.
1 0 xffffffff81d88e26
2 0 xffffffff81d88c20
3 0 xffffffff81e1940e
4 0 xffffffff81e19497
5 0 xffffffff81e195fe
6 0 xffffffff81d88d6f
11
‘undefinedundefined <UNASSIGNED> <RETURN>
queue_directory
...6290 PUSH RBP
...6291 MOV
...6294 PUSH R14
PUSH R13
...6298 MOV R13D,EDX
...629b PUSH R12
...629d MOV R12,RSL
...62a@ PUSH RBX
...62a1 MOV RBX,RDI
...62a4 MOV
CALL <EXTERNAL>::malloc
...62ae TEST RAX,RAX
..62b1 JZ LAB_001062fc
3 LAB_@@1062fc
«..62fc CALL xalloc_die
00106203
...62b3 MOV - 5
...62b6 TEST R12,R12
..62b9 JZ LAB_001062c6
...62bb MOV RDI,R12
|...62be CALL xstrdup
...62c3 MOV R12,RAX
v ¥
Figure 2.1: Example of basic blocks as displayed by Ghidra. A basic block consists of a sequence of instructions ending at a branch instruction, and control flow diverges to different successor blocks according to the branch.
12
Transition from BB; to BB;
curlocation: 0x4000 prev_location: 0x2000 | cur_location: 0x3000
0x1000 shared_mem: 0x5000 shared_mem: 0x1000
prev_location: 0x2000 prev_location: 0x1800
Transition from BB; to BB;
[=
0x4000| prev_location: 0x1800 0x3000 prev_location:
09000
shared_mem: 0x4400 shared_mem: 0xa000
prev_location: 0x2000 prev_location: 0x1800
Figure 2.2: Example of basic block transitions in AFL. Transitions from BBi to BBj and from BBj to BBi are distinguished.
13
2.3.2 The code coverage measurement in the Linux kernel
KCOV is a feature of the Linux kernel designed to support CGF. Unlike AFL, KCOV does not embed its own instrumentation code. Instead, it relies on the sanitizer cover- age features provided by GCC [4] and LLVM [7] to insert coverage hooks. With these Compiler-based mechanisms, the compiler determines the instrumentation points and, upon reaching each point, invokes functions defined by the sanitizer specification. The function names and interfaces invoked at these points are standardized, and developers must implement their handlers according to this specification. Moreover, many of the sanitizer defined functions are shared between GCC and LLVM. The common interfaces available in both compilers are summarized in Table 2.1. When the CONFIG_KCOV option is enabled in the Linux kernel configuration, the compiler is invoked with the sanitizer coverage options shown in Listing 2.4. The compiler option -fsanitize-coverage ac- cepts either trace-pc or trace-cmp as its argument. The trace-pc option traces only program counter addresses, whereas the trace-cmp option additionally traces values in- volved in conditional branches, including operands used in comparisons. In the Linux kernel, trace-pc is used as the default coverage instrumentation method, and enabling CONFIG_KCOV_ENABLE_COMPARISONS allows the kernel to additionally trace comparison data observed during conditional branches. KCOV writes code coverage data to the file /sys/kernel/debug/kcov, and the fuzzer reads this file to obtain and analyze coverage information during fuzzing. The output format of KCOV is simple; in trace-pc mode, it consists of a sequence of addresses, as shown in Listing 2.5.
2.4 Background and Limitations of Coverage-Guided Fuzzing
In this chapter, we discuss the major fuzzing techniques and provide an overview of CGF, which has become the mainstream approach in recent years. In particular, we explain the characteristic fuzzing algorithm of CGF and describe core code coverage mea- surement techniques, focusing on the AFL instrumentation and the KCOV feature in the Linux kernel. Although each of these techniques has its own advantages, they are funda- mentally designed for Single-target fuzzing. As a result, it is difficult to apply a uniform coverage measurement method across software components that are developed and built independently, such as an operating system and firmware. Furthermore, measuring cov- erage across software boundaries, for example, during transitions from OS to firmware is inherently challenging. Therefore, in system configurations where independently devel- oped components, such as the operating system and firmware, operate cooperatively, it is essential to employ a unified coverage measurement approach that does not depend on the build process of each component. Building on this understanding, the next chapter introduces our proposed method, which leverages unified coverage feedback across mul- tiple cooperating software components. Based on this understanding, the next chapter introduces our proposed Multi-target Coverage-based Greybox Fuzzing method.
14
Chapter 3
Proposed Approach
In the previous chapter, we described the general techniques used in CGF. As discussed earlier, CGF relies heavily on code coverage measurement; however, existing methods mea- sure coverage for only a single software component, either by embedding instrumentation at compile time or by collecting coverage information during fuzzing execution. Con- sequently, these approaches do not consider how to measure code coverage in a unified manner across multiple software components that operate cooperatively within a system. In this chapter, we describe the design and implementation of our proposed method, Multi-target Coverage-based Greybox Fuzzing, along with its prototype fuzzer, MTCFuzz, which we developed to realize this approach.
3.1 Multi-target Coverage-based Greybox Fuzzing Method
In our proposed method, Multi-target Coverage-based Greybox Fuzzing, tracing the execution of multiple software components is essential. Figure 3.1 illustrates a scenario in which an application invokes a function in a library, the C library delegates process- ing to the Linux kernel via a system call, the Linux kernel further delegates processing to OpenSBI, and the result is returned to the application in reverse order. In the ex- ample, software boundaries exist not only at the application level, but also across the C library, the Linux kernel, and the OpenSBI. Although these software components are developed independently and use different build processes, they all run within a single system environment. Therefore, if the behavior of the software running on this system can be observed, it is possible to trace execution across software boundaries. Based on the observation, our proposed method adopts QEMU [17] as the execution environment and traces the execution of machine-level instructions within QEMU, thereby allowing execution to be traced across multiple software boundaries. Unlike regular CGF, which measures coverage for only a single software component, our method unifies the coverage of multiple cooperating components and uses it as a single feedback signal for fuzzing.
Unified code coverages: In the proposed method, a single input triggers execution across multiple software components. Consequently, the feedback used for test case evaluation must also be unified
15
Software Boungary Software Boundary Software Boundary
Application
C Library Linux Kernel OpenSBI
H | Hi| H|
Library call
System call
1 i
| ul ul ||
Figure 3.1: Execution flow across software boundaries, where execution propagates from
applications to the OS and firmware.
across all targets. Let C(Tkerneli ) and C(Tfwj ) denote the coverage observed in each execution unit. As shown in Equation 3.1, we define the unified multi-target coverage as N N i o C ⋃ᵏ ⋃ multi = C(Tkernel) ∪ C(Tfwj ) . (3.1) i=1 j=1
Here, the subscripts kernel and fw denote the Linux kernel and firmware (e.g., OpenSBI, Trusted OS, Trusted Application), respectively. The superscripts i and j represent the indices of the execution units observed during a single fuzzing execution, where Nk and No denote the total number of execution units observed in the kernel and firmware, re- spectively. In summary, instead of evaluating each component independently, our proposed method treats the combined coverage space of the OS and firmware as a single exploration domain.
Measurement of code coverage: In each test execution, the code coverage observed
during the execution of a test case is evaluated after the test case has finished. In MTC-
Fuzz, QEMU-based instrumentation is started at the beginning of each test execution and
terminated when the test completes. The instrumentation results are recorded to a file.
The sequence of recorded code coverage from the beginning to the end represents the
execution flow of the code for that test case. During fuzzing, the Linux kernel executes
not only the target functionality under test but also unrelated components such as the
scheduler and memory management. As a result, if the coverage of such an irrelevant code
is included in the measurement, IsInteresting may return undesirable judgments. To
address this issue, MTCFuzz provides an address based filtering mechanism. The address
16
Host Linux machine
QEMU
\
Linux System Firmware/TrustZone
. ecallsme
Linux Kernel Firmware/Trusted OS
system call |= 4 GDB Ey i:
Test
i amp
i
Blue: Fuzzing target ssh GDB script Green: Test harness Orenge: Userland apps Purple: QEMU component
QMP protocol
Figure 3.2: Architecture of MTCFuzz based on QEMU.
filters are specified in a JSON file, as shown in Listing 3.1. In the current implementation at the time of writing this paper, the filter configuration supports two types of target: kernel and firmware. The kernel filter is used to specify the address ranges of the Linux kernel, while the firmware filter is used to specify the address ranges of firmware components, such as OpenSBI, Trusted OS, and Trusted Applications. The code coverage data removed by filtering is concatenated into a single string and hashed, and the result- ing hash value is recorded as the result of the test execution for the corresponding seed. By managing the execution flow as a whole rather than individual traced addresses, this approach enables differences in loop iteration counts, such as those in for and while loops, to be taken into account. The computed hash value is stored as part of the seed’s metadata, and the number of times the same code coverage has been observed in different seeds is aggregated. This count is then used by the power scheduler when calculating the energy assigned to each seed.
3.2 Design of MTCFuzz
In this section, we describe the implementation of MTCFuzz. The fuzzing execution
environment of our proposed method is shown in Figure 3.2. When performing fuzzing with MTCFuzz, the main software components include the fuzzer itself (MTCFuzz ), the target operating system (e.g., Linux), and the target firmware (e.g., OpenSBI or OP- TEE environments such as Trusted OS). MTCFuzz is responsible for managing the entire fuzzing process. The management of the fuzzing process includes launching QEMU, saving
17
Listing 3.1: Address filters setting
1 " address_filters ": { 2 " kernel ": [ 3 { 4 " name ": " riscv_ext_d_validate " , 5 " lower ": "0 xffffffff80010764 " , 6 " upper ": "0 xffffffff80010772 " 7 } , 8 ... 9 ] , 10 " firmware ": [ 11 ... 12 ]
and loading snapshots, seed mutation, code coverage checking, execution monitoring, and crash detection. MTCFuzz is designed to fuzz low-level software components such as kernels and firmware. If the fuzzing process were managed on Linux inside the virtual machine, a crash of the Linux kernel would cause the process responsible for managing the fuzzing campaign itself to stop, making it impossible to continue fuzzing. Therefore, in MTCFuzz, the manage- ment of the fuzzing process is not performed inside the virtual machine, but is instead implemented as a process running on the host operating system. While the fuzzing process itself is managed by MTCFuzz, the execution of the test harness must be performed inside the virtual machine. For this reason, MTCFuzz transfers the test harness into the guest operating system during the setup phase using mechanisms such as scp or the 9p file sys- tem, and then launches the test harness via ssh. Consequently, the ability to communicate with the guest operating system via ssh is a mandatory requirement for MTCFuzz.
3.3 Implementation of MTCFuzz
Measuring code coverage in QEMU: QEMU provides a debugging feature that traces the machine instructions executed [16]; however, this feature continuously records the execution from the start to the end of QEMU. As a result, it is not suitable for selective instrumentation, only for the execution of individual test cases. When measuring code coverage in QEMU, it is therefore essential to be able to arbitrarily control the start and end of the measurement. In addition, QEMU must send the measurement results to a destination that can be read by the fuzzer. To meet these requirements, we extend the QEMU QMP API [14] to MTCFuzz by adding new commands to starting and stopping coverage tracking. The definitions of these APIs are shown in Listing 3.2. The API for starting code coverage measurement is mtcfuzz-trace-start, which takes a file name as the filename parameter and writes the trace log to the specified file. To stop the measurement, mtcfuzz-trace-stop is executed.
18
MTCFuzz performs instrumentation at the timing when QEMU translates guest ma- chine instructions into TCG. This translation is performed on a basic block basis [15], and therefore the tracing granularity in MTCFuzz corresponds to the level of QEMU basic blocks. Moreover, since TCG translation is used for all architectures supported by QEMU, the instrumentation mechanism can be implemented without introducing architecture- specific code, enabling a simple and portable design. The implementation of the QEMU instrumentation code is shown in patch format in Listing 3.3. The changes made to the existing source code consist only of adding calls to mtcfuzz_record_tb_exec in two func- tions and therefore have minimal impact on the original codebase. As a result, even when upgrading the QEMU version, it is relatively easy to keep the implementation up to date. mtcfuzz_record_tb_exec is the function shown in Listing 3.4.
The code coverage measurement and filtering: The code coverage traced by QEMU is filtered in MTCFuzz. Filters are described as a list in JSON format in a configuration file, as shown in Listing 3.1. The removal of unnecessary data from the traced coverage is performed by the analyze_coverage pro- cedure. The algorithm of analyze_coverage is shown in Algorithm 3.1. In this process, among the measured code coverage, if an address belongs to the target of measurement, the number of executions of the address is recorded in Kcov (Linux kernel coverage) or Fcov (firmware coverage). During the filtering process, the target address is first checked to determine whether it is already registered in Kcov or Fcov. If the address has already been registered, its count is incremented. If the address is not registered in either list, AddrInFilters is invoked to determine whether the address is included in the filtering list. If this check is performed naively, the computational complexity required to examine a single address is O(N ), and if there are M addresses to be examined, the total complexity becomes O(M N ). Therefore, as the number of filtering targets increases, the performance of the filtering process is expected to degrade. To address this issue, MTCFuzz sorts the filter list in ascending order and applies binary search to the filtering procedure, thereby reducing the complexity of examining a single address to O(log N ). Consequently, the total complexity required to examine all traced addresses becomes O(M log N ). In addition, the measured addresses are stored in Kcov or Fcov, and before executing the binary search, the algorithm checks whether the target address is already registered in one of these data structures (implemented as python dictionary type). This membership test requires O(1) expected time. Therefore, in the filtering process, the best-case complexity is O(1), while the worst-case complexity is O(M log N ). In the early stage of fuzzing campaign, new code coverage is likely to be discovered frequently. Thus, a large number of new code coverage points are detected shortly after fuzzing begins. therefore, for a certain period after the start of fuzzing, the computational complexity remains close to O(M log N ). However, as the number of test executions increases, the number of newly discovered coverage points decreases, and the slope of the curve becomes gradually smaller. Consequently, while the computational complexity of the filtering process is O(M log N ) in the initial phase of fuzzing, it approaches O(1) as the fuzzing. Let N be the number of filters, K be the total number of distinct addresses that have appeared at least once so far, and T be
19
the total number of address checks performed throughout the fuzzing campaign (i.e., the sum over all campaigns, all tests, and all PCs). Under these definitions, the total cost (T otalCost) can be expressed as shown in Equation 3.2, and the amortized complexity can be expressed as shown in Equation 3.3. As fuzzing progresses, the discovery rate of new coverage decreases, and the increase in T becomes dominant over that of K . As a result, the ratio K/T converges to zero and the computational complexity approaches O(1). This functionality is not merely a supporting mechanism for collecting code coverage, but a core component that directly influences fuzzing efficiency, and therefore plays an essential role in CGF.
TotalCost = O(T + K log N ), (3.2)
O( T + KT log N ) = O(1 + K log N ) . (3.3)
T
20
Listing 3.2: Definition of APIs for code coverage instrumentation
1 ## 2 # @mtcfuzz - trace - start : 3 # 4 # Start a fuzzing trace . 5 # 6 # @filename: Name of the file to store the trace . 7 # 8 # Since : 10.0 9 # 10 # .. qmp - example :: 11 # 12 # -> { " execute ": " mtcfuzz - trace - start " , 13 # " arguments": { " filename ": " trace . log " } } 14 # <- { " return ": {} } 15 ## 16 { ’ command ’: ’ mtcfuzz - trace - start ’ , 17 ’data ’: { ’ filename ’: ’str ’ }} 18 19 ## 20 # @mtcfuzz - trace - stop : 21 # 22 # Stop a fuzzing trace . 23 # 24 # Since : 10.0 25 # 26 # .. qmp - example :: 27 # 28 # -> { " execute ": " mtcfuzz - trace - stop " } 29 # <- { " return ": {} } 30 ## 31 { ’ command ’: ’ mtcfuzz - trace - stop ’}
21
Listing 3.3: Implementation of instrumentation code in TCG translation process
1 diff -- git a / accel / tcg / cpu - exec . c b / accel / tcg / cpu - exec . c 2 index 8163295 f34 .. b20bd8f11d 100644 3 --- a / accel / tcg / cpu - exec . c 4 +++ b / accel / tcg / cpu - exec . c 5 @@ -41 ,6 +41 ,8 @@ 6 # include " tb - context . h " 7 # include " internal - common . h " 8 # include " internal - target . h " 9 +# include " trace / mtcfuzz_trace . h " 10 + 11 12 /* - icount align implementatio n . / 13 14 @@ -603 ,6 +605 ,7 @@ void cpu_exec_ st e p _a t om i c ( CPUState * cpu ) 15 16 cpu_exec_ent er ( cpu ) ; 17 / execute the generated code / 18 + m t c f u z z _ r e c o r d _ t b _ e x e c (( uint64_t ) pc ) ; 19 trace_exec_tb ( tb , pc ) ; 20 cpu_tb_exec ( cpu , tb , & tb_exit ) ; 21 cpu_exec_exit ( cpu ) ; 22 @@ -904 ,6 +907 ,7 @@ static inline void cpu_loop_ex e c_ t b ( CPUState * cpu , TranslationB lo c k * tb , 23 vaddr pc , Translation Bl o ck ** last_tb , 24 int * tb_exit ) 25 { 26 + m t c f u z z _ r e c o r d _ t b _ e x e c(( uint64_t ) pc ) ; 27 trace_exec_tb ( tb , pc ) ; 28 tb = cpu_tb_exec ( cpu , tb , tb_exit ) ; 29 if ( tb_exit != TB_EXIT_REQ U ES T ED ) {
Listing 3.4: Implementation of writting trace log to file
1 void m t c f u z z _ r e c o r d _ t b _ e x e c( uint64_t pc ) 2 { 3 qemu_mutex_l oc k (& mtcfuzz_t b_ t ra c e l o ck ) ; 4 5 if (! mtcfuzz_tb _ tr a ce _ fp ) { 6 qemu_mutex u nl o ck (& mtcfuzz_tb t r a ce _ lo c k ) ; 7 return ; 8 } 9 10 fprintf ( mtcfuzz_tb_trace_fp , "0 x %" PRIx64 "\ n " , pc ) ; 11 qemu_mutex un l oc k (& mtcfuzz_t b_ t r ac e _l o c k ) ; 12 }
22
Algorithm 3.1 Coverage Classification for Kernel and Firmware Address Spaces Require: Covered PC list (hex strings) C Require: Kernel coverage map Kcov, firmware coverage map Fcov Require: Kernel filters Kfilter, firmware filters Filter Require: Kernel start list Kstarts, firmware start list Fstarts Ensure: Updated coverage maps Kcov, Fcov 1: A ← ∅ ⊲ List of all observed PCs 2: Kfound ← false 3: Found ← false 4: for all pchex ∈ C do 5: pc ← HexToInt(pchex) 6: if pc ∈ Kcov then 7: Kcov[pc] ← Kcov[pc] + 1 8: A ← A + pc 9: continue 10: else if pc ∈ Fcov then 11: Fcov[pc] ← Fcov[pc] + 1 12: A ← A + pc 13: continue 14: end if 15: if AddrInFilters(pc, Kfilter, Kstarts) then 16: Kcov[pc] ← 1 17: Kfound ← true 18: A ← A + pc 19: continue 20: end if 21: if AddrInFilters(pc, Filter, Fstarts) then 22: Fcov[pc] ← 1 23: Found ← true 24: A ← A + pc 25: continue 26: end if 27: end for
23
Snapshot: MTCFuzz uses QEMU’s snapshot functionality to save and load status of virtual ma- chine. These operations are performed through the QMP API. Snapshot saving through QMP is performed using the snapshot-save command, which stores the snapshot data on a storage device attached to the virtual machine. Consequently, as part of the setup process, MTCFuzz creates a dedicated qcow2 file to store snapshots and launches QEMU with snapshot storage attached by the command-line option shown in Listing 3.5. In this configuration, the storage device is assigned the ID snapshot0. This ID is required when saving snapshots using the snapshot-save API. The general procedure for saving a snapshot is shown in Algorithm 3.2. First, MTCFuzz establishes a connection to QMP using ConnectQMP. Then it checks whether the node_name of the snapshot storage is already set; if not, the node is searched using FindBlockDevice. Once the node is found, the execution of the virtual machine is suspended using StopVM. The snapshot target devices are then configured, the snapshot-save API parameters are prepared, and the snapshot-save command is executed. After the snapshot has been successfully saved, the virtual machine execution is resumed and the connection to QMP is closed, completing the procedure. The procedure for loading a snapshot requires the node_name of the storage device where the snapshot is stored. If the node name is not already set, it is obtained using FindBlockDevice, in the same manner as in the save operation. The virtual machine execution is then suspended using StopVM, the parameters for the snapshot-load API are configured, and the snapshot-load command is executed. Once the snapshot loading is complete, the execution of the virtual machine is resumed, and the connection to QMP is closed, completing the operation.
Listing 3.5: QEMU command-line option for snapshot feature
drive file =/ home / build / projects / mtcfuzz / work / fuzz - snapshot . qcow2 , if = none , format = qcow2 , id = snapshot0
24
Algorithm 3.2 VM Snapshot Save via QMP Require: QMP socket path, snapshot device name Ensure: Snapshot is saved successfully 1: if ConnectQMP() == false then 2: return false 3: end if 4: if node_name is not set then 5: node_name ← FindBlockDevice() 6: end if 7: if node_name is null then 8: DisconnectQMP() 9: return false 10: end if 11: StopVM() 12: devices ← [node_name] 13: if rootf s_device_name exists then 14: devices ← devices ∪ {rootf s_device_name} 15: end if 16: args ← { job-id: GenerateSnapshotJobID("save"), 17: tag: “mtcfuzz-snapshot”, 18: vmstate: node_name, 19: devices: devices } 20: QMPExecute(“snapshot-save”, args) 21: ResumeVM() 22: DisconnectQMP() 23: return true
25
Algorithm 3.3 VM Snapshot Load via QMP Require: QMP socket path, snapshot tag Ensure: VM state is restored from snapshot 1: if ConnectQMP() == false then 2: return false 3: end if 4: if node_name is not set then 5: node_name ← FindBlockDevice() 6: end if 7: StopVM() 8: args ← { job-id: GenerateSnapshotJobID("load"), 9: tag: “mtcfuzz-snapshot”, 10: vmstate: node_name, 11: devices: [node_name] } 12: QMPExecute(“snapshot-load”, args) 13: ResumeVM() 14: DisconnectQMP() 15: return true
26
Fuzzing workflow in MTCFuzz: Figure 3.3 illustrates the basic fuzzing execution workflow of MTCFuzz. The workflow can be roughly divided into five phases:
initialization,
test case preparation,
test case execution,
evaluation of test execution results, and
post-execution processing.
The blue rectangles in Figure 3.3 represent the initialization phase. This phase includes
one-time setup operations performed before starting fuzzing, such as creating a working directory on the host running MTCFuzz, loading the initial seed corpus, launching the virtual machine, creating a working directory inside the virtual machine, and transferring the test harness to the virtual machine. The orange rectangles correspond to the test case preparation phase, which includes selecting a seed, assigning energy to the selected seed, creating a snapshot if it has not been created yet, and generating a test case by mutating the seed. The yellow rectangles represent the test case execution phase, which consists of starting code coverage tracing, executing the test harness, and stopping code coverage tracing. The green rectangles indicate the evaluation phase, in which code coverage is evaluated based on the execution results. The red rectangles represent the post-execution processing phase. If the system crashes, the virtual machine is restarted; otherwise, the system state is restored to the pre-execution state by loading the snapshot. When the energy assigned to the current seed is exhausted, a new seed is selected; otherwise, mutation of the current seed is continued. MTCFuzz advances fuzzing by repeatedly executing this workflow. Test cases generated by MTCFuzz on the host side are either transferred to the virtual machine as files via the scp command or directly supplied as arguments to the test harness. Since the test harness is executed inside the virtual machine, it is invoked via remote command execution over ssh, as shown in Listing 3.6.
Listing 3.6: Execute test case via ssh
ssh -p 10022 localhost / path / to / testharness arg1 arg2 ...
27
ee reas | EE |
or 14 Ett cose |
|3. ne 15. Add tho testcase
Startvirtual | 5. Save a snapshot if 12. Stop code 10 sed 1 now code |
machine its not created coverage tracing
9.Croate caso
rectory by mutatingᵃ testhe seed >
system
crashed
{ =
Yes
test hamess 17. Load
5. Sendremote work resat tho
vi sco snapsh sito⁰ he machinei system
crashed
Ble: ial sep
Prepare
Yelow: Execute est
tes case
Orange: test case
Green: Evalue resul
[ves Figure 3.3: Overview of the fuzzing workflow in MTCFuzz.
Visualizing code coverage: MTCFuzz provides a feature that visualizes the code coverage executed after fuzzing in an HTML format. It enables users to correlate executed coverage with the corresponding source code. This feature allows users to inspect source code regions that were executed during fuzzing. Although this functionality is primarily developed to visually compare the results of the proposed method with those of a baseline approach, it can also be used for post-fuzzing inspection in standard fuzzing workflows that do not involve such comparisons. Since MTCFuzz measures coverage at the basic block granularity, the corre- spondence between source code lines and coverage information is not strictly one-to-one. Nevertheless, the ability to identify executed regions is helpful for debugging purposes. In the HTML view, executed lines are highlighted with background colors: lines executed by both the proposed method and the baseline approach are shown in green, lines executed only by the proposed method are shown in blue, and lines executed only by the baseline approach are shown in orange. As illustrated in Figure 3.4, selecting a source file displays a legend explaining these background colors at the top of the page. An example of visualiz- ing executed regions within a selected source file is shown in Figure 3.5, where the number of executions is displayed for each executed line. In the Figure 3.5, the blue-highlighted
28
+ home home /build/projects/sres/opensbi/1ib/shi/shi_ecall_base.c
build
+ projects Covered by both single and malts
wares Covered only by mules
Covered only by single
+ linux coverage
arch
Milt Single Line Source
» include 1
v opensbi 2 * SPDX-License-Identifier: BSD-2-Clause
+ firmware
4 * Copyright (c) 2020 Western Digital Corporation or its affiliates
El
sbi_uraph 6 * Authors:
7 + Anup Patel <anup.patelewdc.con>
sbi_ecallc & + Atish Patra
sbi_ecall base.
11 #include <sbi/sbi_ecall.h>
12 #include <sbi/sbi_ecall_interface.h>
13 #include <sbi/sbi error.h>
Figure 3.4: Coverage visualization in HTML, illustrating the meaning of background col-
ors.
line at line 56 indicates that it was executed twice only by the proposed method.
This functionality correlates executed basic blocks with source code and the target
software to be built with debug information enabled. As a result, this feature cannot be
used when fuzzing closed-source software for which source code is unavailable. To resolve
executed basic blocks to source code locations, we uses the addr2line command. The
addr2line tool can be applied to binaries in the ELF file format, as shown in Listing 3.7.
In addition to specifying target addresses directly, addr2line supports batch processing
by reading multiple addresses from a file. In this example, addr2line outputs the function
name, source file path, and line number corresponding to each address. In MTCFuzz, the
addresses to be resolved are written to a file and processed in a single invocation of the
addr2line command to obtain this information efficiently. Although addr2line operates
on binaries in the ELF file format, firmware such as OpenSBI is typically deployed as a raw
binary rather than an ELF file. However, for debugging purposes, such firmware projects
also generate binaries in the ELF format. Therefore, when invoking addr2line, MTCFuzz
specifies the corresponding ELF file instead of the raw binary that is actually loaded at
runtime.
Listing 3.7: Resolving a kernel address using addr2line
$ riscv64 - linux - gnu - addr2line -e ./ vmlinux -f -p -a 0 xffffffff8001b340 0 xffffffff8001b340 : __sbi_ecall at / home / build / projects / srcs / linux / arch / riscv / kernel / sbi_ecall.c :41
29
syscallh out->value = MAJOR <<
+ kernel “ SBI_SPEC_VERSION_MAJOR_OFFSET) &
entrys MASK
4s (SBI_SPEC_VERSION_MAJOR <<
fous a6 SBI_SPEC_VERSTON_MAJOR_OFFSET);
rae 47 out->value = out->value | SBI_ECALL
process.c 0 break;
shi_ecallc
signal.c 49 case SBI_EXT_BASE_GET_IWP_ID:
18 24 se out->value = sbi_ecall_get_inpid();
traps s1 break;
+ include 52 case SBI_EXT_BASE_GET_IMP_VERSION:
+ asm.generic 6 10s out->value = OPENSBI_VERSION;
+ bitops 54 break;
+ linux 55 case SBI_EXT_BASE_GET_MVENDORID:
entry-commonh 1 56 out->value = ;
thread info 57 break;
+ opensbi 55 case SBI_EXT_BASE_GET_MARCHID:
firmware 4 1s out->value = ;
+ include 60 break;
sbi 61 case SBI_EXT_BASE_GET_MIWPID.
"libˢᵇⁱᵗʳᵃᵖʰ 16 1 2 out->value =
break;
64 case
sbi_ecall.c 65 ret = sbi_ecall_base_probe(regs->ad, &out->value);
sbi_ecall_base.c break;
Figure 3.5: Illustrating coverage differences between the proposed method (multi) and the baseline (single). The blue-highlighted line at line 56 was executed only in the multi configuration.
3.4 Summary of the Proposed Approach
In this chapter, we proposed the Multi-target Coverage-based Greybox Fuzzing approach
that integrates and utilizes the code coverage of multiple software components cooperating within a system. Conventional CGF primarily targets a single piece of software; therefore, in systems where software cooperates with other components, such as the Linux kernel and OpenSBI, it is difficult to observe the behavior of the cooperating software components. In addition, Compiler-based coverage instrumentation provided by GCC or Clang is only applicable to software that is built with such instrumentation enabled, and in systems where multiple software components operate cooperatively, it is difficult to measure code coverage continuously in temporal order. In our proposed method, QEMU is adopted as the execution environment, and the code coverage of software running on QEMU is measured within QEMU. It enables unified cov- erage measurement regardless of how each software component is built. Furthermore, by extending QMP to provide measurement control functionality and by utilizing the snap- shot mechanism to reset the execution environment, the influence of previous fuzzing runs on the system state can be eliminated, thereby enabling a practical fuzzing environment. In the proposed method, the regions for which code coverage should be measured can be specified, enabling the address filtering mechanism that excludes executions unrelated to the target functionality from the coverage measurement. The computational complexity of the filtering process in the proposed method is O(1) in the best case and O(M logN ) in the worst case, while computational complexity towards O(1) on average. Therefore, even if the number of registered filters increases, its impact on filtering performance remains
30
small. Based on these designs, we implemented a prototype system called MTCFuzz and constructed a platform capable of evaluating the effectiveness of the proposed method on multiple CPU architectures. In the next chapter, we evaluate the effectiveness of the proposed method through experiments and present the results. In the next chapter, we evaluate how effectively MTCFuzz explores cooperative system software in practice.
31
Chapter 4
Evaluation
In this chapter, we describe evaluation environments, evaluation methods, and evalu- ation results.
4.1 Experimental Environment
The experiments were conducted using the two environments shown in Table 4.1. The
experiments were conducted on two architectures: RISC-V and AArch64. The software used for these architectures is listed in Table 4.2. The fuzzing experiments were performed using the Docker container provided by MTCFuzz.
4.2 Experimental Methodology
Our proposed method improves the capability of CGF by unifying and using the code
coverage obtained from multiple software components. For the experimental evaluation, MTCFuzz provides two execution modes: a mode that utilizes the coverage of multiple software components (which realizes the proposed method) and a mode that utilizes the code coverage of only a single software component (as in conventional fuzzers). We refer to execution with the proposed method as multi, and execution using only single software code coverage as single. In this study, the single mode uses only the Linux kernel code coverage. There is virtually no difference in the execution code between these two modes. In both cases, code coverage is collected; however, in the single mode, the firmware side coverage is always treated as if no new coverage has been discovered. As a result, the
Table 4.1: Experimental environments
Environment OS Hardware
Env1 Ubuntu 24.04 CPU: Intel Core i7-9700K
Memory: 64 GB
Env2 Fedora 43 CPU: Intel Core i7-14700
Memory: 64 GB
32
Table 4.2: Software versions used in the experiments
Environment Component Version
RISC-V Linux Kernel 6.16-rc1
OpenSBI v1.6
Manifest f92aacd1b103af137438f2e434303ffa5dff3d09
OP-TEE Linux Kernel 6.14.0-gb8a233c2155a
OP-TEE 4.7.0-rc1
Table 4.3: Configuration files used in fuzzing experiments
Experiment Configuration file
4.2.1
: Coverage Evaluation of OpenSBI Base Extension configs/opensbi/coverage_test
4.2.2
: Coverage evaluation of OP-TEE by long running configs/optee/xtest_fuzz_1002
4.2.4
: Address filter performance configs/opensbi/coverage_test
4.2.5
: Snapshot load performance configs/opensbi/coverage_test
4.2.6
: Fuzzing OP-TEE cryptographic API configs/optee/xtest_fuzz_1001
number of steps required for code coverage measurement is identical in both modes, and no performance difference arises from the coverage measurement process itself. In our experimental evaluation of the proposed method, code coverage effectiveness is assessed by comparing the results of multi and single using this functionality. In all experiments, the number of QEMU instances launched by MTCFuzz was set to one. The configuration files used for each fuzzing experiment are summarized in Table 4.3.
4.2.1 Coverage Evaluation of OpenSBI Base Extension
In this evaluation, we used the RISC-V architecture with Linux kernel 6.16-rc1 and OpenSBI v1.6 in Env2. The evaluation focuses on measuring and analyzing the cover- age of case statements within the sbi_ecall_base_handler function, which handles the Base Extension in lib/sbi/sbi_ecall_base.c of OpenSBI. The Base Extension pro- vides fundamental APIs, such as those that return the implemented specification version. According to the RISC-V Supervisor Binary Interface Specification, the Base Extension defines seven APIs [32]. In the OpenSBI implementation, the extension ID is stored in the eid variable, and the function ID is stored in the fid variable. The Base Extension has a value of 0x10, and sbi_ecall_base_handler executes the corresponding process according to the fid value. To perform branching based on the value of fid, this imple- mentation uses a switch statement. In this evaluation, the mutation target was set to fid and the initial seed used a value of 0 for it. Based on this initial seed, the fuzzing was conducted in two independent sets, each consisting of 50 campaigns of 5 minutes, resulting
33
Listing 4.1: Fedora 43 total symbols
1 $ uname -r
2 6.17.8 -300. fc43 . x86_64
3 $ sudo wc -l / proc / kallsyms
4 403481 / proc / kallsyms
in a total of 100 fuzzing campaigns.
4.2.2 Coverage evaluation of OP-TEE by long running
In this experiment, we compare the number of covered code paths between multi and single during fuzzing of the shared memory functionality used jointly by the Linux kernel and the Trusted OS.
4.2.3 QEMU performance overhead
For QEMU performance evaluation, we used a RISC-V Linux environment running kernel version 6.16-rc1 as the guest OS. The number of virtual CPUs in the guest environ- ment was set to one. To measure performance, we used hackbench [8], which is included in the Ubuntu 24.04 rt-tests package. The hackbench is a benchmark and stress testing tool for the Linux kernel scheduler. To run a stress test by hackbench, it does not use storage. Therefore, test result is unaffected by I/O operations. This allows us to measure kernel processing performance in a purely CPU- and scheduler-bound manner. We evaluate two configurations of hackbench: the process mode(hackbench -l 600 -g 10 –process) and the thread mode(hackbench -l 600 -g 10 –thread). Both configurations were executed with 600 loops and 10 groups. As a baseline for comparison, we used the same version of QEMU(v9.2.3) built and used by MTCFuzz.
4.2.4 Address filter performance
For the evaluation of the address filter, the same fuzzing test described in 4.2.1 is used. The number of configured filters increases from 21 to 215, and a fuzzing campaign was run for 10 minutes for each test. The upper limit of the number of filters was set to 215. As shown in Listing 4.2, there are 51,779 symbols in the tested kernel. Also, as shown in Listing 4.1, the number of symbols in Fedora 43, a widely used Linux distribution (Linux kernel version 6.17.8-300.fc43.x86_64), is 403,481. It is corresponds to log2(403,481) ≈ 18.62, and therefore we consider that setting the maximum number of filters to 215 is sufficient for the purpose of this experiment.
34
Listing 4.2: Total symbols in tested RISC-V kernel
1 # uname -a 2 Linux buildroot 6.16.0 - rc1 #2 SMP Sun Aug 24 03:10:39 UTC 2025 riscv64 GNU / Linux 3 # wc -l / proc / kallsyms 4 51779 / proc / kallsyms
4.2.5 Snapshot load performance
MTCFuzz loads a snapshot before the execution of each test case and restores the environment to its pre-execution state. This mechanism enables each test to be executed without being affected by the results of previous tests and is used to minimize the occur- rence of bugs that depend on runtime state. However, restoring the snapshot for every test case may introduce performance overhead during fuzzing. Therefore, in the experiment, we compared the number of test executions between two configurations: (i) restoring the snapshot before each test case (the default behavior of MTCFuzz ), and (ii) continuing fuzzing without restoring the snapshot. The experiment uses the same configuration as in 4.2.1 (Coverage Evaluation of OpenSBI Base Extension), and both configurations—normal execution of MTCFuzz and the execution with snapshot loading disabled—were evaluated. For each configuration, five times a set of 10-minute fuzzing runs was conducted and the results were compared.
4.2.6 Fuzzing OP-TEE cryptographic API
In this experiment, we constructed a test harness that mutates parameters associ- ated with shared memory used in cryptographic operations in OP-TEE and conducted an evaluation based on this setup. The target cryptographic operation accepts, as input pa- rameters, a buffer containing plain text data to be encrypted and its length, and produces, as output parameters, a buffer for the resulting cipher text and its length. For APIs that take both a buffer and its corresponding size as arguments, inconsistencies between the actual buffer size and the specified size may lead to memory corruption. Consequently, this evaluation aimed to detect vulnerabilities caused by such buffer size mismatches.
4.3 Experimental Results
4.3.1 Coverage Evaluation of OpenSBI Base Extension results
In this experiment, a fuzzing campaign was set to 5 minutes and two sets of 50 execu- tions were performed, resulting in a total of 100 fuzzing campaigns. Table 4.4 shows the number of campaigns in which all branching points of the switch statement were fully covered in each test set and the total number of fully covered. The results indicate that multi achieved cover all branch points 27 times in Set 1, 28 campaigns in Set 2, then total 55 campaigns (55%). In contrast, single achieves cover all branch points 19 times in Set
35
Table 4.4: Number of campaigns that achieved full coverage
Mode Set 1 Set 2 Total
multi 27 28 55
single 19 15 34
Table 4.5: Average number of coverage campaigns
Mode Set 1 Avg. Set 2 Avg. Overall Avg.
multi 6.34 6.14 6.24
single 5.92 5.86 5.89
1, 15 campaigns in Set 2, then a total of 34 campaigns (34%). Table 4.5 shows the average
code coverage rate for each set, as well as the overall average code coverage rate. From
the average results, multi achieved an improvement of approximately 5.9% over single
in terms of coverage. These experimental results demonstrate that the proposed method
improves the code coverage rate by approximately 20% compared to the conventional CGF
that relies on the code coverage of a single software component. This indicates that the
proposed method can achieve higher code coverage in a shorter period of time than the
conventional approach.
4.3.2 Coverage evaluation of OP-TEE by long running results
On Env1, we conducted fuzzing experiments for both multi and single, consisting of five campaigns with a duration of two hours each. Since the number of test executions varies across campaigns, we summarized the increase in code coverage for the first 4,688 test executions, which is a minimal test execution count at the time. For comparison, we computed the average code coverage for each test execution for both multi and single. Figure 4.1 visualizes the coverage difference between multi and single (multi − single) for each test execution. In this graph, the zero value on the y-axis represents the origin where there is no difference in code coverage between multi and single. Positive values indicate that multi achieves higher code coverage, whereas negative values indicate that single achieves higher code coverage. As shown in Figure 4.1, in the comparison aligned to 4,688 test executions which are the same as above , the Multi configuration achieves higher coverage than the Single configuration at nearly all execution points, and the gap between them increases further in the latter half of the execution. These results indicate that, within a limited execution time budget, the multi provides higher exploration efficiency than the single. Figure 4.2 illustrates the growth of code coverage. As the number of test executions in- creases, the gap between multi and single becomes larger. Furthermore, at nearly all ex- ecution points, multi consistently achieves higher code coverage than single. Figure 4.3 and Figure 4.4 show the growth of the code coverage separately for the kernel and the firmware. In the firmware code coverage, the coverage achieved by single catched up with that of multi around the 2,700th test execution; however, multi subsequently con- tinued to discover additional coverage and ultimately succeeds in detecting more coverage
36
than single. From the experiment, we can conclude that the proposed method, multi, achieves code exploration efficiency on average higher than the conventional CGF targeting a single software component single. Here, the evaluation had been conducted using 2-hour fuzzing campaigns. In order to evaluate the behavior over a longer execution period, both multi and single fuzzing campaigns were configured to run for 8-hour fuzzing campaign, and each configuration was executed 4 times. The average of these runs was shown in Figure 4.5. In the figure, a relatively large difference between multi and single can be observed up to around 6000 test executions. Although this difference gradually decreases as the number of executed tests increases, multi continues to maintain higher coverage than single. From Figure 4.2 and Figure 4.5, the following trend can be observed. In short fuzzing runs, multi exhibits higher exploration efficiency and is able to discover more code cov- erage than single. However, when fuzzing is executed for a longer period of time, the difference in the number of detected coverage elements between multi and single gradu- ally decreases. This behavior can be interpreted as reflecting the difference in exploration efficiency between the proposed method (multi) and the conventional method (single), as discussed by Liyanage et al. [26]. In addition, as a reference, we conducted a single 24-hour fuzzing campaign run, the results of which are shown in Figure 4.6. To ensure a fair comparison between multi and single, the analysis was limited to the first 49,612 test executions in both cases. Figure 4.7 combines the results of the 2h, 8h, and 24h experiments, showing that multi discovers new coverage more efficiently than single in the early stages of fuzzing campaign. The functions targeted for the measurement of code coverage in this experiment are listed in Listing B.1, Listing B.2, and Figure 4.6 in Ap- pendix B (Additional Details). The functions contained in the files under these directories are treated as coverage measurement targets.
37
Coverage Difference (Multi - Single) 12 Multi - Single —— un
2 9 5
8
2 8 7 8 6 2 sh S g H 43
2
1
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Number of tests
Figure 4.1: Coverage difference (Multi - Single) between the Multi and Single configura- tions, aligned to 2 hours test executions.
Total Coverage (Multi vs Single)
160 Multi Avg —— Single Avg ——
140
120
8g 100 8 ke] 80
60
40 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Number of tests
Figure 4.2: Total coverage of the Multi and Single configurations, aligned to 2 hours of test executions.
38
Total Kernel Coverage (Muli vs Single)
65
| Multi Avg ——
60 Single Avg ——
55
50
& 45 5
3 3
30
25
20
15
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Number of tests
Figure 4.3: Total kernel coverage of the Multi and Single configurations, aligned to 2 hours of test executions.
Total Firmware Coverage (Multi vs Single)
100
Multi Avg ——
single Avg ——
90
80
& 70 3 3 o 60
50
40
30
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Number of tests
Figure 4.4: Total firmware coverage of the Multi and Single configurations, aligned to 2 hours of test executions.
39
Total Coverage (Multi vs Single) 8h running
240
Single Avg ——
200 | Multi Avg ——
200
180
& 160 §⁵ wo 2 120
100
80
60
40
0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000
Number of tests
Figure 4.5: Total coverage of the Multi and Single configurations, aligned to 8 hours of test executions.
Total Coverage (Multi vs Single) 24h running
300 Multi Avg —— Single Avg ——
250
200
8g 150 8 ke] 100
50
0
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
Number of tests
Figure 4.6: Total coverage of the Multi and Single configurations, aligned to 24 hours of test executions.
40
Total Coverage Comparison (Multi vs. Single, 2n/8h/24h) 300
5
200
g g 7 § El 150 wr ke] 100
Multi Avg(2h) ——
50 single Avg(2h) ——
Multi ——
Single ——
Multi
Single Avg(24h) ——
0
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
Number of tests
Figure 4.7: Total coverage of the Multi and Single configurations, 2h/8h/24h running results merged. Vertical dashed lines indicate the execution boundaries corresponding to 2h, 8h, and 24h runs.
41
4.3.3 QEMU performance overhead results
In the experiment, the number of CPUs allocated to the Docker container was set to
one. In addition, a P-core was assigned on the i7-14000 platform(Env2).
To summarize the results, this experiment aimed to quantify the performance over-
head introduced by MTCFuzz on the baseline QEMU. The results for Env1 are shown in
Table 4.6, and those for Env2 are presented in Table 4.7. In the process mode, both Env1
and Env2 exhibited a median overhead increase of approximately 45–65%. In the thread
mode, Env1 showed an overhead increase of approximately 50%, while Env2 exhibited a
substantially larger increase of approximately 83%. Table 4.8 summarizes the results for
each execution environment. This figure shows that the overhead varies depending on the
CPU. Although the overhead is smaller on the i7-9700K than on the i7-14000, the run-
time performance of the i7-14000 is overall better. This can be attributed to the higher
computational capability of the i7-14000 compared with the i7-9700K, which allows it
to perform more computations; however, this also leads to a larger overhead associated
with recording coverage information. A larger overhead was observed in Env2 compared
to Env1. This can be attributed to the higher computational performance of the i7-14000
used in Env2 relative to the i7-9700K. Consequently, the impact of bottleneck components
becomes more pronounced on i7-14000.
During hackbench execution, we measured the execution behavior of the QEMU using
the bpftrace script shown in Listing 4.3 on the Env2 host machine and aggregated the
sampled stack traces into a stack flame graph. The result is shown in Figure 4.8. From
this stack flame graph, we observed that the call sequence from mtcfuzz_record_tb_exec
to __fprintf_chk appears four times among the top-ranked execution paths. This indi-
cates that the execution of fprintf for trace logging introduces a significant performance
overhead.
Listing 4.3: bpftrace on-CPU profile script
1 profile : hz :99 / pid == $1 / { 2 @ [ ustack ] = count () ; 3 } 4 5 END { 6 print (@) ; 7 }
42
Table 4.6: hackbench: Process and Thread Performance (Env1)
Process Thread
QEMU Type Min Max Mean Median Min Max Mean Median
Baseline QEMU 28.343 29.324 28.710 28.638 23.062 27.886 23.633 23.292
MTCFuzz QEMU 41.236 43.062 41.743 41.644 34.677 39.488 35.546 35.084 Overhead (%) 45.5 46.8 45.4 45.5 50.3 41.6 50.4 50.6
Table 4.7: hackbench: Process and Thread Performance (Env2)
Process Thread
QEMU Type Min Max Mean Median Min Max Mean Median
Baseline QEMU 14.229 14.586 14.381 14.365 11.138 13.293 11.471 11.337
MTCFuzz QEMU 21.242 26.978 23.668 23.761 18.104 24.647 21.308 20.807 Overhead (%) 49.3 85.0 64.5 65.4 62.5 85.4 85.8 83.5
Table 4.8: Performance and Overhead Comparison by CPU (hackbench process mode)
Process (s) Thread (s) Process O
CPU Min Max Mean Median Min Max Mean Median Min Max Me
i7-9700K 28.343 29.324 28.710 28.638 23.062 27.886 23.633 23.292 45.5 46.8 45 i7-14000 14.229 14.586 14.381 14.365 11.138 13.293 11.471 11.337 49.3 85.0 64
qemu overhead
gb₁| i|
Figure 4.8: Visualization of stack frames for QEMU execution overhead. This figure shows a stack frame visualization aggregated from QEMU’s stack traces collected during the execution of hackbench. Each frame represents an executed function, and the width of a frame indicates the relative proportion of execution time spent in that function.
43
4.3.4 Address filter performance results
The performance of the filtering process was evaluated by increasing the number of
filters from 21 to 215 and repeating this test three times. The results are shown in Table 4.9. The number of test executions ranged from 441 to 449 in Test 01, from 441 to 449 in Test 02, and from 444 to 451 in Test 03, regardless of the number of filters. As shown in Table 4.10, the standard deviation in all tests remained around 2, indicating that this variation can be attributed to the noise of the measurement at runtime. These results confirm that the performance of the filtering process is not significantly affected even as the number of filters increases. Since the filtering algorithm has a time complexity of O(M · log2 N ), a filtering scale that would cause a notable performance impact for example, doubling the execution time would require a filter count of the order of 230 = 1,073,741,824. However, this corresponds to an extremely large number of symbols for software typically targeted in fuzzing. In such a case, rather than configuring a large number of fine-grained filters, it would be simpler and more practical to apply coarse-grained filtering, such as using the entire address range of the Linux kernel and the entire address range of OpenSBI. This approach would, in turn, result in a smaller number of filters. Therefore, we conclude that the performance of the filtering mechanism in MTCFuzz does not pose a practical limitation in realistic system environments.
Table 4.9: Fuzzing Execution Counts under Different Filter Configurations
Number of Filters Test 01 Test 02 Test 03
2(21) 441 441 451
4(22) 443 445 448
8(23) 445 445 448
16(24) 448 448 449
32(25) 449 443 448
64(26) 443 447 446
128(27) 442 444 449
256(28) 443 447 449
512(29) 441 446 447
1024(210 ) 445 446 444
2048(211 ) 446 441 449
4096(212 ) 446 444 449
8192(213 ) 448 445 445
16384(214 ) 444 446 446
32768(215 ) 441 449 445
44
Table 4.10: Statistical Summary of Test Execution Counts
Test Mean Std. Dev. Min Max
Test01 444.33 2.57 441 449
Test02 445.13 2.22 441 449
Test03 447.53 1.89 444 451
4.3.5 Snapshot load performance results
The results of the test executions with and without snapshot loading are shown in
Table 4.11. Across all five runs, disabling snapshot loading results in an approximately
12% improvement in execution throughput. In the current implementation of MTCFuzz,
a snapshot is loaded before each test execution in order to reset the system state and
eliminate the effects of previously executed test cases. Considering this design choice, an
execution throughput overhead of approximately 12% introduced by snapshot loading is
within an acceptable range.
Table 4.11: Comparison of test case execution counts with and without snapshot loading
Snapshot load enabled Snapshot load disabled Performance improvement rate (%)
Test01 449 507 12.92
Test02 448 506 12.95
Test03 448 505 12.72
Test04 447 503 12.53
Test05 448 503 12.28
Average 448.0 504.8 12.68
4.3.6 Fuzzing OP-TEE cryptographic API
As a result of this evaluation, we discovered a vulnerability in the optee driver in the
Linux kernel that is caused by insufficient validation of shared memory sizes. Specifically, in the error-handling path of the routine that sets up shared memory based on the buffer and buffer size passed to the API. Due to an issue in the cleanup logic for shared memory during error handling, a NULL pointer dereference leads to kernel crash. The kernel panic log observed during this reproduction is shown in Listing 4.4. In our experiments, we used Linux kernel version 6.14.0-gb8a233c2155a. We also ver- ified that the vulnerability is reproducible in the Linux kernel version 6.17-rc5, which is the latest available mainline kernel version at the time of discovery. We confirmed that the bug exists in the mainline kernel as well. Then, we reported the conditions under which this bug is triggered, along with repro- duction steps and a reproduction script, to the OP-TEE development mailing list. We also tested the bug fix patch developed by the maintainers and cooperated in the validation of the bug fix [9]. The vulnerability was quickly fixed, and after the fix was merged into the mainline Linux kernel, it was assigned to CVE-2025-40031 [10].
45
Listing 4.4: Kernel OOPS log by CVE-2025-40031.
1 [ 16.143987] Unable to handle kernel NULL pointer dereferen ce at virtual address 0000000000000008 2 [ 16.144141] Mem abort info : 3 [ 16.144215] ESR = 0 x0000000096000004 4 [ 16.144246] EC = 0 x25 5 ** replaying previous printk message ** 6 [ 16.144246] EC = 0 x25 : DABT ( current EL ) , IL = 32 bits 7 [ 16.144271] SET = 0, FnV = 0 8 [ 16.144289] EA = 0, S1PTW = 0 9 [ 16.144308] FSC = 0 x04 : level 0 translation fault 10 [ 16.144325] Data abort info : 11 [ 16.144335] ISV = 0, ISS = 0 x00000004 , ISS2 = 0 x00000000 12 [ 16.144346] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 13 [ 16.144358] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 14 [ 16.144412] user pgtable : 4k pages , 52 - bit VAs , pgdp =0000000048 b34b00 15 [ 16.144432] [0000000000000008] pgd =0800000040 bb3403 , p4d =0000000000000000 16 [ 16.144876] Internal error : Oops : 0000000096000004 [#1] SMP 17 [ 16.146429] Modules linked in : 18 [ 16.146775] CPU : 0 UID : 0 PID : 148 Comm : xtest Not tainted 6.17.0 - rc5 #58 PREEMPT 19 [ 16.146995] Hardware name : linux , dummy - virt ( DT ) 20 [ 16.147181] pstate : 21402005 ( nzCv daif + PAN - UAO - TCO + DIT - SSBS BTYPE = - -) 21 [ 16.147330] pc : unpin_user_pages +0 x78 /0 xd0 22 [ 16.147763] lr : unpin_user_pages +0 xa0 /0 xd0 23 [ 16.147842] sp : ffff800084403d20 24 [ 16.147912] x29 : ffff800084403d20 x28 : fff00000054aa300 x27 : 0000000000000000 25 [ 16.148089] x26 : 0000000000000000 x25 : 0000000000000000 x24 : 0000000000000000 26 [ 16.148235] x23 : fff00000004fb5a8 x22 : 0000000000000001 x21 : 000000000000000 d 27 [ 16.148401] x20 : fff0000000b2f9c0 x19 : 0000000000000011 x18 : 0000000000000001 28 [ 16.148544] x17 : 0000000000000000 x16 : 0000000000000000 x15 : 0000000000000000 29 [ 16.148659] x14 : 0000000000000002 x13 : 0000000000000002 x12 : 0000000000037 d0f 30 [ 16.148786] x11 : fff0000001dad708 x10 : 000000000000003 f x9 : 0000000000000 d1b 31 [ 16.148925] x8 : 00000000000007 e0 x7 : 0000000000000001 x6 : 000000000000000 d 32 [ 16.149039] x5 : ffffffffffffffff x4 : ffffffffffffffff x3 : 000000000000000 e 33 [ 16.149167] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffffc1ffc0fd68c0 34 [ 16.149351] Call trace : 35 [ 16.149520] unpin_user_pages +0 x78 /0 xd0 ( P) 36 [ 16.149684] tee_shm_put +0 x134 /0 x184 37 [ 16.149783] tee_shm_fop_release +0 x14 /0 x24 38 [ 16.149866] __fput +0 xcc /0 x2dc 39 [ 16.149925] fput_close_sync +0 x40 /0 x108 40 [ 16.149991] __arm64_sys_close +0 x38 /0 x7c 41 [ 16.150058] invoke_syscall +0 x48 /0 x110 42 [ 16.150127] el0_svc_common . constprop .0+0 x40 /0 xe8 43 [ 16.150227] do_el0_svc +0 x20 /0 x2c 44 [ 16.150303] el0_svc +0 x34 /0 xf0 45 [ 16.150369] el0t_64_sync_handler +0 xa0 /0 xe4 46 [ 16.150439] el0t_64_sync +0 x198 /0 x19c 47 [ 16.150629] Code : aa0203e3 eb02027f 54000109 f8627a82 ( f9400444 ) 48 [ 16.150940] ---[ end trace 0000000000000000 ]--- 49 [ 16.151230] Kernel panic - not syncing : Oops : Fatal exception 50 [ 16.151466] SMP : stopping secondary CPUs 51 [ 16.151838] Kernel Offset : disabled 52 [ 16.151911] CPU features : 0 x000000 ,0000 d180 ,2 bbe33e1 ,957 e7f3f 53 [ 16.152019] Memory Limit : none 54 [ 16.152284] ---[ end Kernel panic - not syncing : Oops : Fatal exception ]---
46
4.4 Summary of Evaluation
In this chapter, we evaluated the effectiveness of the proposed method in terms of code exploration capability, the QEMU overhead introduced by code coverage measurement, the processing performance of the address filtering mechanism, and the overhead caused by restoring snapshots. In addition, we successfully discovered a new CVE by performing fuzzing against the OP-TEE API. The evaluation results demonstrated that the proposed method achieves higher code coverage than conventional CGF that targets only a single software component, and is able to discover a larger number of execution paths within a limited time. Furthermore, the performance overhead introduced by extending QEMU to support code coverage mea- surement, as well as the cost associated with the address filtering mechanism, remained within a practical range, confirming that the proposed method is feasible as a practical fuzzing approach. From these results, we conclude that the proposed method, which integrates and uti- lizes the code coverage of multiple software components, is effective in improving explo- ration efficiency in cooperative execution environments where conventional approaches face difficulties.
47
Chapter 5
Discussion
Based on the evaluation results obtained in chapter 4, this chapter discusses the effec- tiveness and limitations of the proposed approach.
5.1 Effectiveness of the Proposed Approach
In sections 4.3.1 and 4.3.2, we evaluated the code coverage exploration capability, and in both experiments, the proposed method achieved higher code coverage than the existing approach. This result can be interpreted as follows. In conventional CGF, when execution crosses software boundaries, for example, when execution transitions from the Linux kernel to OpenSBI due to the ecall, even if no new coverage is discovered on the Linux kernel side, new coverage may be reached on the OpenSBI side. However, because CGF targeting only the Linux kernel does not utilize the coverage information obtained in OpenSBI, IsInteresting is likely to judge that the executed test case did not yield useful results. In contrast, the proposed method continues the coverage measurement even when execution paths transition from the Linux kernel to OpenSBI. Therefore, coverage that is previously invisible to conventional CGF can now be utilized, and even in cases where no new coverage is discovered on the Linux kernel side, IsInteresting can judge the test case as beneficial if new coverage is detected in OpenSBI. For this reason, we consider that the ability of the proposed method to measure coverage across software boundaries contributed to its higher coverage exploration capability compared with conventional CGF. The proposed method enables fuzzing in architectures where multiple software com- ponents operate cooperatively. Therefore, it is particularly effective in software systems whose functionality is not confined to a single software component, such as environments in which the kernel and the firmware operate in cooperation.
5.2 Limitations
MTCFuzz leverages QEMU as the fuzzing execution environment, which enables com- prehensive monitoring of the behavior of all software running inside QEMU. However, the current design assumes that only a single fuzzing target environment is executed within a
48
QEMU instance. Although it is technically possible to emulate multiple systems by run- ning nested QEMU instances inside the Linux environment launched by MTCFuzz, the current implementation does not support such configurations.
5.3 Challenges
There are two main challenges in MTCFuzz. The first challenge, as demonstrated in the experiments in the Chapter 4, is that the overhead of code coverage measurement using QEMU is relatively high. The second challenge is that the creation of test harnesses must be carried out by the fuzzing practitioner.
Improving the performance of QEMU-based coverage measurement: Execution efficiency is essential for fuzzing and reducing overhead is an important issue. In the current implementation, fprintf is invoked every time mtcfuzz_record_tb_exec is called, and the bpftrace profiling results showed that this routine constitutes a major bottleneck. To mitigate this bottleneck, the following improvements can be considered:
Store coverage data in memory until writing to a file in order to reduce the number of fprintf executions.
Replace file I/O with shared memory communication between the fuzzer and QEMU to eliminate the overhead of fprintf entirely.
In the case of approach (1), the question is how frequently the buffered data should be flushed. As shown in Section 4.2.1, one execution of the test harness records approximately 35,000 addresses on average. Since the output is written as text rather than binary, each 64-bit value must be converted to a hexadecimal string (up to 16 characters), and with the 0x prefix and newline, one entry requires up to 19 bytes. If a 4096-byte buffer is used, writing the data whenever approximately 200 entries accumulate would result in only 175 write operations per run, reducing the number of fprintf calls by approximately 99.5%. Therefore, this approach has the potential to significantly reduce the measurement overhead. In the case of approach (2), file I/O can be eliminated completely, removing the fprintf overhead entirely. However, when using shared memory, QEMU must track how much data has been written, while the fuzzer must manage how much has been consumed. If QEMU overwrites the buffer before the fuzzer finishes reading it, the recorded coverage information becomes unreliable. Therefore, although this approach removes the fprintf overhead, it requires careful buffer management between QEMU and the fuzzer.
Improving test harness creation: Although MTCFuzz provides a fuzzing framework, the creation of test harnesses must be performed by the user. Since the primary fuzzing targets of MTCFuzz include kernels, firmware, Trusted OSs, and Trusted Applications, test harnesses must be implemented using the APIs provided by these components. Unlike file-format fuzzing (e.g., PNG or ELF), where valid initial seeds can be used and mutated, API fuzzing requires knowledge of execution dependencies, parameter preparation, and ordering of API invocations. For
49
example, the write system call requires a file descriptor (fd) as its first argument, which must be obtained beforehand using open. Furthermore, the fd may refer to a regular file, a pseudo file, or even a network socket. In addition, the execution state can vary depending on operations such as modifying the file offset via lseek or holding a lock using flock in another thread before write is executed. Consequently, constructing appropriate test har- nesses for API-based fuzzing is not a trivial task. Therefore, although MTCFuzz provides a framework for Multi-target fuzzing, the burden of test harness development remains a significant challenge. Addressing this issue would further enhance the practicality and efficiency of fuzzing using the proposed approach.
5.4 Summary of Discussion
In this chapter, we discussed the effectiveness and limitations of the proposed approach. Based on the evaluation results in Chapter 4, the proposed method was shown to enable code coverage measurement across software boundaries in systems where multiple software components operate cooperatively. As a result, it expands the range of code exploration compared to conventional CGF and enables more effective exploration. At the same time, we identified several practical challenges, including the overhead of QEMU-based coverage measurement and the burden of creating test harnesses. Overall, the proposed approach is both meaningful and practical, although there re- mains room for improvement. These findings form the basis for the future enhancements and extensions that will be addressed in Chapter 7.
50
Chapter 6
Related Work
In this chapter, we discuss related work based on two key aspects of the proposed approach: its implementation as a QEMU-based fuzzing framework and its focus on system software such as kernels and Trusted OS.
QEMU-based Fuzzing Frameworks: As studies that employ QEMU as the execution environment for fuzzing, Schumilo et al. proposed kAFL [33], and Malmain et al. proposed LibAFL QEMU [27]. The kAFL measures code coverage using hardware support, namely Intel Processor Trace (Intel PT). Since coverage measurement is performed using hardware tracing, the overhead is extremely low, reported to be less than 5% [33]. It supports fuzzing of operating systems such as Linux and Windows. Because coverage is obtained via Intel PT, software modification (e.g., rebuilding with instrumentation) is not required. Basic blocks that should be excluded from coverage measurement can be registered in a blacklist so that they are ignored. As kAFL relies on Intel PT, the host machine must be equipped with an Intel Skylake or later processor. In addition, the host operating system must be Ubuntu 20.04 or later, or Debian Bullseye or later, with a Linux kernel customized for kAFL. Although QEMU is used as the fuzzing execution environment, the supported target architectures are limited to x86 and x86-64, and architectures such as ARM and RISC-V are not supported. Furthermore, kAFL does not adopt snapshot restoration because it is considered costly; instead, it continuously uses the same VM instance, and non-deterministic behavior such as interrupts is filtered using the blacklist mechanism. The LibAFL QEMU is a fuzzing framework implemented as part of LibAFL [20]. The LibAFL QEMU customizes QEMU so that QEMU, which normally operates as a standalone application, can be embedded and controlled as a library from the fuzzer. The code coverage measurement is performed inside QEMU, and coverage is collected at the basic block granularity in TCG. While kAFL allows users to configure coverage exclusion using a blacklist, LibAFL QEMU enables specifying coverage regions using an allow-list mechanism. It also incorporates several techniques to support efficient snapshot restoration: 1) device state management relies on existing mechanisms in QEMU, 2) memory snapshots are restored using differential state management, 3) block devices are launched in read-only mode while write operations are redirected to a shadow region in RAM. As a result, reverting a snapshot can be completed simply by discarding the shadow region. The operating systems and CPU architectures that can be fuzzed with
51
LibAFL QEMU follow those supported by QEMU, and no software rebuilding is required for coverage measurement. In both kAFL and LibAFL QEMU, test suites must be implemented by the developer who performs fuzzing.
Fuzzing for Kernel/Trusted OS: syzkaller is a fuzzer that targets multiple operating systems, including Linux, FreeBSD, and Windows. It supports QEMU as an execution environment; however, syzkaller uses upstream or distribution provided QEMU as-is and does not require any customization of QEMU. In addition to QEMU-based execution, fuzzing can also be performed on physical hardware. When fuzzing the Linux kernel, code coverage is measured using KCOV [13], and therefore the target Linux kernel must be built with KCOV enabled. One of the notable features of syzkaller is its domain-specific language, syzlang. Using syzlang, system call specifications are defined, and based on these specifications, syzkaller automatically generates valid system call invocation sequences and corresponding test programs. Although developers are required to add new system call specifications manually, a large number of definitions are already available in the official repository, allowing practical fuzzing to be performed immediately after cloning and building the source code. In addition to system calls provided by the kernel, pseudo- syscalls can also be defined [30]. Using these specifications as input, syzkaller constructs test programs and mutates them during execution, efficiently expanding code coverage. Although syzkaller is already a highly capable fuzzer, many approaches have been proposed based on it. For example, SyzDirect [37] performs Directed Greybox Fuzzing for the Linux kernel, and SyzParam [36] focuses on fuzzing device drivers via sysfs modifiable parameters. Furthermore, SyzTrust, proposed by Wang et al., is a fuzzer targeting Trusted Execution Environments (TEE) and is also based on syzkaller. SyzTrust focuses on fuzzing Trusted OSs and executes on real hardware rather than QEMU. Since mechanisms such as KCOV are not available as in Linux, coverage measurement is carried out using a debug probe. To fuzz Trusted OS components, SyzTrust utilizes the GP TEE Internal Core API, the specifications of which are described using syzlang. API execution against the Trusted OS is triggered from a Trusted Application.
Table 6.1: Comparison of system fuzzing frameworks
kAFL LibAFL QEMU MT
Target CPU architecture x86-64 / x86 QEMU-supported CPUs QEMU-su Host OS Ubuntu / Debian Any Language C Rust P Performance X (X) Instrumentation Method Hardware-assisted (Intel PT) Emulator-based (QEMU) Emulator- Target Model Single-target Single-target Mu Snapshot Save / Restore No Yes State Reset Mechanism Reset Snapshot Sn Representative Use Case Kernel fuzzing Binary / kernel fuzzing OS–Firmware Summary and Positioning: Table 6.1 summarizes the key differences between kAFL, LibAFL, and our proposed system, MTCFuzz. In summary, existing QEMU-based fuzzing frameworks focus on maximizing the capabilities of QEMU and introducing necessary customizations in order to perform efficient fuzzing. Meanwhile, kernel-oriented fuzzers
52
support efficient test generation by enabling developers to construct test harnesses us- ing appropriate combinations of system calls and APIs. However, these frameworks and fuzzers are fundamentally designed for Single-target fuzzing: while they are highly effective within their respective software domains, they do not provide unified Multi-target cover- age feedback across cooperatively executing software components. Our proposed method addresses this limitation by enabling integrated coverage measurement across software boundaries.
53
Chapter 7
Future Work
As future work, we plan to integrate the proposed method into LibAFL QEMU, one of the state-of-the-art fuzzing frameworks. While the proof-of-concept implementation MTCFuzz provides only the minimum necessary fuzzing functionality, LibAFL QEMU offers advanced capabilities such as high-performance snapshot mechanisms, flexible ex- ecution control, and support for modern fuzzing techniques including cmplog [18]. By realizing our approach—measuring and leveraging code coverage across multiple software boundaries—on top of LibAFL QEMU, we expect to extend conventional Single-target fuzzing workflows to cooperative execution environments such as kernel–firmware and ker- nel–Trusted OS systems. Furthermore, this integration is expected to facilitate broader adoption of our approach and enable large-scale, realistic evaluations using the latest fuzzing ecosystem.
54
Chapter 8
Conclusion
This thesis addressed the challenge of fuzzing environments in which multiple software components operate cooperatively, such as operating systems with firmware or Trusted OS. Conventional fuzzing techniques primarily focus on a single software component and have not sufficiently explored the unified use of code coverage information in such cooperative execution environments. To overcome this limitation, we proposed a method Multi-target Coverage-based Grey- box Fuzzing that leverages QEMU to enable unified execution measurement of multiple software components running within a virtualized environment. To demonstrate the prac- ticality of the proposed approach, we developed a prototype implementation, MTCFuzz, and evaluated it in environments where multiple software components operate in coordi- nation. The evaluation results showed that the proposed method achieved higher code cover- age than conventional CGF. Furthermore, the runtime overhead introduced by coverage measurement remained within a practical range. These results indicate that the proposed method can effectively enhance code exploration in environments where multiple software components execute cooperatively. This study demonstrates the feasibility of fuzzing multiple software components in an integrated manner and provides a foundation for future research aimed at improving performance, enhancing automation, and expanding applicability to a broader range of systems.
55
Acknowledgment
I would like to express my deepest gratitude to Professors Ariyasu Suzaki, Eiji Kuwana,
and Atsuhiro Goto of the Institute of Information Security for their invaluable guidance and support throughout this research. I am also deeply grateful to the members of the Suzaki Laboratory and the Institute of Information Security for their helpful advice, in- formation sharing, and continuous encouragement. I would further like to thank my col- leagues at my company for their understanding, support that contributed to this research.
56
References
[1] AFLplusplus/qemu_mode/README.md at stable · AFLplusplus/AFLplusplus — github.com. https://github.com/AFLplusplus/AFLplusplus/blob/stable/qemu_ mode/README.md. [Accessed 28-07-2025].
[2] GitHub - google/syzkaller: syzkaller is an unsupervised coverage-guided kernel fuzzer — github.com. https://github.com/google/syzkaller/. [Accessed 04-10-2025].
[3] GitHub - masami256/MTCFuzz — github.com. https://github.com/masami256/ MTCFuzz. [Accessed 03-01-2026].
[4] Instrumentation Options (Using the GNU Compiler Collection (GCC)) — gcc.gnu.org. https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options. html. [Accessed 22-11-2025].
[5] libFuzzer: a library for coverage-guided fuzz testing. LLVM 22.0.0git documentation — llvm.org. https://llvm.org/docs/LibFuzzer.html. [Accessed 25-07-2025].
[6] oss-fuzz/projects at master · google/oss-fuzz — github.com. https://github.com/ google/oss-fuzz/tree/master/projects. [Accessed 24-11-2025].
[7] SanitizerCoverage — Clang 22.0.0git documentation — clang.llvm.org. https:// clang.llvm.org/docs/SanitizerCoverage.html. [Accessed 22-11-2025].
[8] Ubuntu manpage: hackbench - scheduler benchmark/stress test. https://manpages. ubuntu.com/manpages/noble/en/man8/hackbench.8.html. [Accessed 23-11-2025].
[9] [BUG] tee_shm: NULL pointer dereference in unpin_user_pages() on invalid shm pages - OP-TEE - lists.trustedfirmware.org — lists.trustedfirmware.org. https:// lists.trustedfirmware.org/archives/list/op-tee@lists.trustedfirmware. org/thread/IE4UTIKLYYMVEQB5I3IQ5CGYG5M5JI33/, 2025. [Accessed 13-01-2026].
[10] cve.org. https://www.cve.org/CVERecord?id=CVE-2025-40031, 2025. [Accessed 24-11-2025].
[11] OWS2025 — iwsec.org. https://www.iwsec.org/ows/2025/, 2025. [Accessed 06- 01-2026].
[12] Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. Coverage-based greybox fuzzing as markov chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16), pages 1032–1043, 2016.
57
[13] ©The kernel development community. KCOV: code coverage for fuzzing – The Linux Kernel documentation — docs.kernel.org. https://docs.kernel.org/dev-tools/ kcov.html. [Accessed 28-07-2025].
[14] The QEMU Project Developers. Documentation/QMP - QEMU — wiki.qemu.org. https://wiki.qemu.org/Documentation/QMP. [Accessed 06-12-2025].
[15] The QEMU Project Developers. Documentation/TCG - QEMU — wiki.qemu.org. https://wiki.qemu.org/Documentation/TCG. [Accessed 20-11-2025].
[16] The QEMU Project Developers. Emulation — QEMU documentation — qemu.org. https://www.qemu.org/docs/master/about/emulation.html# execution-log. [Accessed 01-12-2025].
[17] The QEMU Project Developers. QEMU — qemu.org. https://www.qemu.org/. [Accessed 26-11-2025].
[18] Andrea Fioraldi, Daniele Cono D’Elia, and Emilio Coppa. Weizz: Automatic grey- box fuzzing for structured binary formats. In Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis (ISSTA 2020), pages 1–13, 2020.
[19] Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. AFL++: Com- bining incremental steps of fuzzing research. In Proceedings of the 14th USENIX Workshop on Offensive Technologies (WOOT 20), 2020.
[20] Andrea Fioraldi, Dominik Christian Maier, Dongjia Zhang, and Davide Balzarotti. Libafl: A framework to build modular and reusable fuzzers. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CSS ’22), pages 1051–1065, 2022.
[21] Free Software Foundation, Inc. Gcov (Using the GNU Compiler Collection (GCC)) — gcc.gnu.org. https://gcc.gnu.org/onlinedocs/gcc/Gcov.html. [Accessed 28- 07-2025].
[22] Patrice Godefroid. Random testing for security: blackbox vs. whitebox fuzzing. In Proceedings of the 2nd international workshop on Random testing: co-located with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007), pages 1–1, 2007.
[23] Patrice Godefroid, Michael Y Levin, David A Molnar, et al. Automated whitebox fuzz testing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), volume 8, pages 151–166, 2008.
[24] Xinuos Inc. 3. Sections — ELF Object File Format 4.3 DRAFT documentation — gabi.xinuos.com. https://gabi.xinuos.com/elf/03-sheader.html. [Accessed 22-11-2025].
58
[25] Clément Léger. lib: sbi: fwft: check feature value to be exactly 1 or 0 · riscv-software- src/opensbi@e8717d1 — github.com. https://github.com/riscv-software-src/ opensbi/commit/e8717d126401435896b0e96c18e187f0b2431d5e, 2024. [Accessed 26-07-2025].
[26] Danushka Liyanage, Marcel Böhme, Chakkrit Tantithamthavorn, and Stephan Lipp. Reachable coverage: Estimating saturation in fuzzing. In 2023 IEEE/ACM 45th In- ternational Conference on Software Engineering (ICSE), pages 371–383. IEEE, 2023.
[27] Romain Malmain, Andrea Fioraldi, and Aurélien Francillon. Libafl qemu: A library for fuzzing-oriented emulation. In Proceedings of the Workshop on Binary Analy- sis Research (BAR 24), colocated with the Network and Distributed System Security Symposium (NDSS 24), 2024.
[28] Barton P Miller, Lars Fredriksen, and Bryan So. An empirical study of the reliability of unix utilities. Communications of the ACM, 33(12):32–44, 1990.
[29] Charlie Miller, Zachary NJ Peterson, et al. Analysis of mutation and generation-based fuzzing. Independent Security Evaluators, Tech. Rep, 4, 2007.
[30] notselwyn. Tickling ksmbd: fuzzing SMB in the Linux kernel — pwning.tech. https://pwning.tech/ksmbd-syzkaller/, 2023. [Accessed 26-12-2025].
[31] Arm Limited (or its affiliates). Documentation Arm Developer — de- veloper.arm.com. https://developer.arm.com/documentation/100690/0201/ Arm-TrustZone-technology?lang=en. [Accessed 25-07-2025].
[32] RISC-V Platform Runtime Services Task Group. Risc-v supervisor binary interface specification: Version 2.0. Technical report, RISC-V International, 1 2024. Ratified specification. No changes are allowed.
[33] Sergej Schumilo, Cornelius Aschermann, Robert Gawlik, Sebastian Schinzel, and Thorsten Holz. kafl: Hardware-assisted feedback fuzzing for os kernels. In 26th USENIX security symposium (USENIX Security ’17), pages 167–182, 2017.
[34] Chaofan Shou, Shangyin Tan, and Koushik Sen. Ityfuzz: Snapshot-based fuzzer for smart contract. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2023), pages 322–333, 2023.
[35] Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. Driller: Augmenting fuzzing through selective symbolic execution. In Proceedings of the Network and Distributed System Security Symposium (NDSS ’16), number 2016, pages 1–16, 2016.
[36] Yue Sun, Yan Kang, Chenggang Wu, Kangjie Lu, Jiming Wang, Xingwei Li, Yuhao Hu, Jikai Ren, Yuanming Lai, Mengyao Xie, et al. Syzparam: Incorporating runtime parameters into kernel driver fuzzing. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CSS ’25), pages 1484–1498, 2025.
59
[37] Xin Tan, Yuan Zhang, Jiadong Lu, Xin Xiong, Zhuang Liu, and Min Yang. Syzdirect: Directed greybox fuzzing for linux kernel. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CSS ’23), pages 1630–1644, 2023.
[38] The Clang Team. Source-based Code Coverage – Clang 22.0.0git documentation — clang.llvm.org. https://clang.llvm.org/docs/SourceBasedCodeCoverage.html, 2007. [Accessed 28-07-2025].
[39] Gerald M. Weinberg. Fuzz Testing and Fuzz History — secretsofcon- sulting.blogspot.com. https://secretsofconsulting.blogspot.com/2017/02/ fuzz-testing-and-fuzz-history.html. [Accessed 15-08-2024].
[40] Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. {QSYM}: A practical concolic execution engine tailored for hybrid fuzzing. In Proceedings of the 27th USENIX Security Symposium (USENIX Security 18)), pages 745–761, 2018.
[41] Michal Zalewski. Tool. American fuzzy lop (afl) fuzzer. https://lcamtuf.coredump. cx/afl/technical_details.txt. [Accessed 09-11-2025].
60
Appendix A
Terminology
• Fuzzing: A software testing technique that executes a program with numerous automatically generated inputs to uncover crashes or vulnerabilities.
• Seed: An input used as a template for generating new test cases through mutation during fuzzing.
• Initial seed: A seed input explicitly provided by the user at the start of a fuzzing campaign, which serves as the initial template from which subsequent seeds are generated.
• Fuzzing campaign: A continuous fuzzing execution session performed under fixed testing conditions.
• OpenSBI: RISC-V firmware that provides Supervisor Binary Interface (SBI) ser- vices to the operating system.
• OP-TEE: Trusted execution environment software running in the Secure World of ARM TrustZone.
• TrustZone: ARM security architecture that separates execution environments into Secure World and Normal World.
• Basic block: A straight-line sequence of instructions without internal branching.
• Code coverage: A measure of which parts of a program have been executed during testing.
• Test case: An input executed on the software under test (SUT) during fuzzing. In coverage-guided fuzzing, promising test cases are retained and used as new seeds.
61
Appendix B
Additional Details
Listing B.1: Linux kernel target list used in CGF with Multi-Target Coverage Feedback
1 linux / drivers / tee
2 linux / drivers / tee / optee
Listing B.2: optee_os target list used in CGF with Multi-Target Coverage Feedback
1 optee_os / out / arm / core / tee 2 optee_os / out / arm / core / lib / zlib 3 optee_os / out / arm / core / lib / libtomcrypt 4 optee_os / out / arm / core / lib / libtomcrypt / src / mac / hmac 5 optee_os / out / arm / core / lib / libtomcrypt / src / mac / omac 6 optee_os / out / arm / core / lib / libtomcrypt / src / modes / xts 7 optee_os / out / arm / core / lib / libtomcrypt / src / modes / ctr 8 optee_os / out / arm / core / lib / libtomcrypt / src / modes / cbc 9 optee_os / out / arm / core / lib / libtomcrypt / src / modes / ecb 10 optee_os / out / arm / core / lib / libtomcrypt / src / math / fp 11 optee_os / out / arm / core / lib / libtomcrypt / src / math 12 optee_os / out / arm / core / lib / libtomcrypt / src / encauth / ccm 13 optee_os / out / arm / core / lib / libtomcrypt / src / ciphers / aes 14 optee_os / out / arm / core / lib / libtomcrypt / src / ciphers 15 optee_os / out / arm / core / lib / libtomcrypt / src / pk / ed25519 16 optee_os / out / arm / core / lib / libtomcrypt / src / pk / ecc 17 optee_os / out / arm / core / lib / libtomcrypt / src / pk / pkcs1 18 optee_os / out / arm / core / lib / libtomcrypt / src / pk / dsa 19 optee_os / out / arm / core / lib / libtomcrypt / src / pk / x25519 20 optee_os / out / arm / core / lib / libtomcrypt / src / pk / dh 21 optee_os / out / arm / core / lib / libtomcrypt / src / pk / rsa 22 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / oid 23 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / octet 24 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / sequence 25 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / set 26 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / generalized ti m e 27 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / object_ide n ti fi e r 28 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / utf8 29 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / integer 30 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / ia5 31 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / bit
62
32 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / short_intege r 33 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / printable_ st ri n g 34 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / teletex_stri n g 35 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / boolean 36 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / custom_type 37 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / utctime 38 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / choice 39 optee_os / out / arm / core / lib / libtomcrypt / src / pk / asn1 / der / general 40 optee_os / out / arm / core / lib / libtomcrypt / src / pk / ec25519 41 optee_os / out / arm / core / lib / libtomcrypt / src / hashes 42 optee_os / out / arm / core / lib / libtomcrypt / src / hashes / sha2 43 optee_os / out / arm / core / lib / libtomcrypt / src / hashes / helper 44 optee_os / out / arm / core / lib / libtomcrypt / src / misc 45 optee_os / out / arm / core / lib / libtomcrypt / src / misc / ssh 46 optee_os / out / arm / core / lib / libtomcrypt / src / misc / pkcs5 47 optee_os / out / arm / core / lib / libtomcrypt / src / misc / crypt 48 optee_os / out / arm / core / lib / libtomcrypt / src / misc / pkcs12 49 optee_os / out / arm / core / lib / libtomcrypt / src / misc / base64 50 optee_os / out / arm / core / lib / libfdt 51 optee_os / out / arm / core / drivers / clk 52 optee_os / out / arm / core / drivers / rstctrl 53 optee_os / out / arm / core / drivers 54 optee_os / out / arm / core / drivers / gpio 55 optee_os / out / arm / core - lib / libmbedtls / mbedtls / library 56 optee_os / out / arm / core - lib / libutils / isoc 57 optee_os / out / arm / core - lib / libutils / isoc / arch / arm 58 optee_os / out / arm / core - lib / libutils / isoc / newlib 59 optee_os / out / arm / core - lib / libutils / compiler - rt / lib / builtins 60 optee_os / out / arm / core - lib / libutils / ext 61 optee_os / out / arm / core - lib / libutils / ext / arch / arm 62 optee_os / out / arm / lib / libunw 63 optee_os / out / arm / ta / avb 64 optee_os / out / arm / ta / trusted_keys 65 optee_os / out / arm / ta / remoteproc 66 optee_os / out / arm / ta / remoteproc / src 67 optee_os / out / arm / ta / pkcs11 68 optee_os / out / arm / ta / pkcs11 / src 69 optee_os / out / arm / ta_arm64 - lib / libmbedtls / mbedtls / library 70 optee_os / out / arm / ta_arm64 - lib / libutils / isoc 71 optee_os / out / arm / ta_arm64 - lib / libutils / isoc / arch / arm 72 optee_os / out / arm / ta_arm64 - lib / libutils / isoc / newlib 73 optee_os / out / arm / ta_arm64 - lib / libutils / compiler - rt / lib / builtins 74 optee_os / out / arm / ta_arm64 - lib / libutils / ext 75 optee_os / out / arm / ta_arm64 - lib / libutils / ext / arch / arm 76 optee_os / out / arm / ta_arm64 - lib / libutee 77 optee_os / out / arm / ta_arm64 - lib / libutee / arch / arm 78 optee_os / out / arm / ta_arm64 - lib / libdl
63