SOURCE ARCHIVE
EXTRACTED CONTENT
97,378 charsRISC Whisk: Unleashing the Power of Software Fuzzing on Hardware
Nandita Singh
Thesis submitted to the Faculty of the
Virginia Polytechnic Institute and State University
in partial fulfillment of the requirements for the degree of
Master of Science
in
Computer Engineering
Matthew Hicks, Chair
Angelos Stavrou
Wenjie Xiong
April 21, 2023
Blacksburg, Virginia
Keywords: Software Fuzzing, Hardware Fuzzing, RISC-V Processors. Copyright 2023, Nandita Singh
RISC Whisk: Unleashing the Power of Software Fuzzing on Hard-
ware
Nandita Singh
(ABSTRACT)
In the hardware industry, the fabrication of a chip with hardware bugs represents a crit- ical concern due to the permanent and irreversible nature of the process. The detection of bugs in intricate designs, such as those found in central processing units (CPUs), is a highly challenging and labor-intensive task, which leaves little margin for error. Modern CPU verification techniques often employ a blend of simulation, formal and emulation veri- fication to guarantee the accuracy of the design. Although these methods are successful in identifying various types of design flaws, they still have some limitations. The biggest limita- tions is achieving comprehensive coverage of all conceivable scenarios and exceptional cases which may interrupt a core and put it in a halt state. We are presenting a design agnostic methodology involving a three-stage process for verification of a multi-core 32-bits RISC-V processor. This methodology leverages software fuzzing and utilizing state-of-the-art tools to analyze CPU’s design after converting it into an equivalent software model. Our approach for hardware fuzzing incorporates the use of a sparse memory matrix as external memory to hold the inputs and state of the core, which are encountered during the fuzzing process. This approach has significantly increased the efficiency of our fuzzing process, enabling us to achieve a 609x improvement in the fuzzing rate compared to prevalent hardware fuzzing techniques. To further optimize our process, we precisely constrained the inputs of the fuzzer to provide only valid test scenarios, which has eliminated the crash overhead of the fuzzer. By doing so, we have improved the accuracy of our testing results and reduced the time
and resources required to analyze potential vulnerabilities. Our verification techniques are implemented using open-source tools, making our fast and cost-effective process accessible to a wide range of hardware engineers and security professionals. By leveraging the benefits of sparse memory and precise input constraints, our approach to hardware fuzzing offers a powerful and efficient tool for identifying potential hardware vulnerabilities and defects.
RISC Whisk: Unleashing the Power of Software Fuzzing on Hard-
ware
Nandita Singh
(GENERAL AUDIENCE ABSTRACT)
In the world of technology, computer chips play a crucial role in almost everything we do. These chips are designed to perform specific tasks and are used in a variety of devices, such as smartphones, computers, and gaming consoles. It’s crucial that these chips are free of bugs or errors because even a small flaw can lead to disastrous consequences, such as system crashes, data loss, or even security breaches. However, testing computer chips for bugs is a challenging and labor-intensive task, especially when it comes to complex designs like central processing units (CPUs). This is because CPUs are responsible for carrying out a wide range of operations and are made up of many intricate components, making it difficult to identify and fix any issues that arise during the testing process. To overcome these challenges, engineers and researchers have developed various testing methods, including simulation, formal verification, and emulation verification. These methods are effective in identifying most types of design flaws, but they still have some limitations. For instance, they may not be able to cover all conceivable scenarios or exceptional cases that could cause a CPU to malfunction. To address these limitations, we have developed a new testing method that leverages software fuzzing. Fuzzing is a technique in which million of random inputs are given to the program to find unexpected behaviour of the design. We are using state-of-the-art tools to analyze the CPU’s design after converting it into an equivalent software model. This approach is called hardware fuzzing. To implement hardware fuzzing, we have used special memory system called a sparse memory matrix. This system is used
to hold the inputs and state of the CPU during the testing process. By doing this, we are able to increase the efficiency of the fuzzing process by 609x compared to other hardware fuzzing techniques. This means we can test the CPU much faster and more accurately than before. To further optimize the process, we have constrained the inputs of the fuzzer to only include valid test scenarios. This eliminated the crash overhead of the fuzzer, which improved the accuracy of the testing results and reduced the time and resources required to analyze potential vulnerabilities.This new testing method is implemented using open- source tools, which makes it accessible to a wide range of hardware engineers and security professionals. In other words, anyone can use this method to test their CPU designs quickly and cost-effectively.
Dedication
I am humbled and honored to dedicate my thesis to four exceptional individuals who have had a profound impact on my academic and personal growth. First and foremost, I express my sincere gratitude to my advisor, Dr. Matthew Hicks, for his guidance, expertise, and encouragement. His mentorship has been instrumental in shaping my research work and has motivated me to strive for academic excellence. I also dedicate this thesis to my beloved parents, Col. Sanjay Singh and Dr. Poonam Chauhan and my sister Gitanjali Singh, whose unconditional love, support, and belief in me have been the driving force behind my success. Their encouragement have been my constant source of strength, and I am forever indebted to them for their immeasurable contributions towards making me the person I am today.
vi
Acknowledgments
I owe the successful completion of my thesis to the immense support I am receiving from my friends and lab mates. I want to express my heartfelt gratitude to my friends, particu- larly Pratijay Guha, Abhinav Sethi, and Suhani Khurana, for providing encouragement and support throughout this challenging journey. Their constant presence and positive attitude are inspiring me to keep pushing forward. Furthermore, I would like to express my deep appreciation to my lab mates, Harrison Williams, Prakhar Sah and Rishi Ranjan. There invaluable assistance and expertise have played a crucial role in broadening my knowledge spectrum and shaping the final outcome of my thesis. Finally, I express my deep gratitude to my boyfriend, Vishal Malik, who serves as a pillar of strength in my journey. His love and his constant presence by my side give me the strength and motivation to keep going, even during the toughest times. I recognize that I could not have accomplished this feat without him.
vii
Contents
List of Figures xi
List of Tables xiii
1 Introduction 1
2 Background 5
2.1 Design Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Design Debugging Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Memory Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Software Fuzzing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 RISC-V Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.6 Open RISC 1000 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Design 16
3.1 First Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.1 Memory Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.2 Direct Programming interface . . . . . . . . . . . . . . . . . . . . . . 20
3.1.3 Random Instruction generator . . . . . . . . . . . . . . . . . . . . . . 21
viii
3.1.4 Constraint block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Second Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1 Executable binary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2.2 Input Seed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2.3 Custom Mutator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 Third Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4 Implementation 30
4.1 First Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.2 Testbench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.1.3 Functional hardware to software converter tool . . . . . . . . . . . . 31
4.2 Second Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2.1 Executable binary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2.2 Initial Seed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2.3 Custom Mutator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.3 Third Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5 Review of Literature 36
6 Results and Evaluation 40
6.1 Killing crash overheads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
ix
6.1.1 Picorv custom mutator . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.1.2 Sodor 1, Sodor 3 and Sodor 5 custom mutator . . . . . . . . . . . . . 42
6.1.3 Mor1kx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.2 Performance Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.2.1 Line Coverage of Processors over time . . . . . . . . . . . . . . . . . 45
7 Conclusions 49
8 Summary 50
9 Future Work 51
Bibliography 52
x
List of Figures
1.1 Mutation tree of initial set of ISA instructions by 3 percent . . . . . . . . . 4
2.1 Software Fuzzing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1 Stage1: Converting Hardware into executable binary . . . . . . . . . . . . . 17
3.2 First stageurcation of memory . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 Stage2: Register indirect addressing mode . . . . . . . . . . . . . . . . . . . 23
3.4 Stage2: Relative addressing mode . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5 Stage2: STORE instruction for write operation . . . . . . . . . . . . . . . . 24
3.6 Stage2: Block diagram representing the AFL fuzzing of the Hardware exe- cutable binary using custom mutator . . . . . . . . . . . . . . . . . . . . . . 25
3.7 Custom Mutator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.8 Stage3 Infrastructure implemented to collect line coverage of the processor . 28
4.1 Executable binary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.1 Time over coverage graph of picorv . . . . . . . . . . . . . . . . . . . . . . . 46
6.2 Time over coverage graph of Sodor1 . . . . . . . . . . . . . . . . . . . . . . . 46
6.3 Time over coverage graph of Sodor3 . . . . . . . . . . . . . . . . . . . . . . . 47
6.4 Time over coverage graph of Sodor5 . . . . . . . . . . . . . . . . . . . . . . . 47
xi
6.5 Time over coverage graph of Mor1kx . . . . . . . . . . . . . . . . . . . . . . 48
xii
List of Tables
3.1 Mentioning memory interface of RISC processors, type of memory interface determines the memory modeling of the processor . . . . . . . . . . . . . . . 20
6.1 Fuzzing speed of the RISC-V cores. . . . . . . . . . . . . . . . . . . . . . . . 45
xiii
Chapter 1
Introduction
In today’s society, electronic hardware plays an omnipresent role, and various electronic de- vices, such as smartphones, computers, and other equipment, rely on Integrated Circuits (ICs) as a crucial component. These ICs have revolutionized the way we communicate and work, making them an essential aspect of modern life. The global economy heavily relies on the semiconductor industry, which had a total market size of $527.88 billion in 2021 and is projected to have a market of 1380.79 billion in 2029 [1]. However, the design and manufac- ture of ICs come with significant challenges. The chip verification process is one of the most critical aspects of this process, which entails testing and verifying the chip’s functionality before releasing it to the market. In 1994, the historic Intel Pentium FDIV bug caused a significant chip failure due to design verification issues. Erroneous calculations resulted from insufficient verification of the chip’s floating-point division unit during the design phase [17]. The bug impacted certain financial and scientific applications and caused significant public outrage. Since then, the semiconductor industry has heavily focused on chip design verifi- cation, making it a crucial aspect of the ASIC flow. According to Siemens Wilson Research Group 40% - 50% of the total development time for an average IC design project is invested in design verification of the chip [7]. Therefore, the verification process is a critical part of the IC design process and is essential for ensuring the reliability and quality of the final product. The hardware industry faces the primary challenge of reducing the time and cost of the verification process while maintaining high levels of quality and reliability in electronic
1
2 CHAPTER 1. INTRODUCTION
devices. To address this challenge, researchers and engineers are exploring new approaches to chip design and verification that include using equivalent software models of the hardware. These techniques can significantly improve the efficiency and accuracy of the verification process. ICs and the semiconductor industry play an essential role in the global economy and our daily lives. As the demand for electronic devices continues to grow, developing more efficient and cost-effective methods for chip verification is crucial to ensure consumers can continue relying on these devices. Verification methods for electronic chips were relatively unsophisticated and inefficient in the late 1990s [3]. However, over time, these techniques have evolved to become more advanced and capable of conducting thorough and exhaustive testing of the chip’s functionality before it is released into the market. As the complexity of chips has increased, the potential for intentionally creating small hardware components that can be used to carry out attacks has also increased, making identifying such hardware an essential aspect of maintaining system security. Therefore, it is crucial to develop robust and effective methods for identifying potentially malicious hardware, as failure to do so could compromise the security of the entire system. Hardware design verification is an essential process in the development of electronic devices, but it can be time-consuming and expen- sive. Traditional methods of design verification focus on finding incorrect RTL code, which is inefficient for exhaustively testing the design’s functionality. To address this challenge, researchers and engineers are exploring new approaches, including the use of software fuzzing techniques. The proposed method involves creating a software equivalent of the hardware model that can be fuzzed using software fuzzers. This approach leverages the advantages of fuzzy logic to explore the states of processors. Unlike traditional design verification, which focuses on finding incorrect RTL code, our method uses fuzzing to generate random scenarios that exhaustively test the design’s functionality. Specifically, a fuzzer takes an executable binary of the design code and runs randomly generated instructions into it. After each cycle, the fuzzer evaluates the coverage, and if it finds new coverage from the test case, it mutates
3
the input with a given percentage to explore states above the captured state of the design. This process allows the fuzzer to explore potential corner cases that may have been missed by traditional design verification methods. The use of software fuzzing techniques can sig- nificantly improve the efficiency and accuracy of the verification process, reducing the time and cost of hardware design. Furthermore, it allows for the identification of difficult-to-find bugs that may have otherwise gone undetected. This approach can be particularly useful in identifying potentially malicious hardware components that could compromise system se- curity. The proposed method of using software fuzzing to identify difficult-to-find bugs in hardware is a promising approach that could significantly improve the verification process’s efficiency and effectiveness. Our proposed method supplements the dynamic functional de- sign verification process by using software fuzzing to make the detection of hardware bugs more comprehensive and efficient. We leverage the power of software fuzzing to uncover difficult-to-detect bugs, which could significantly improve the efficiency and accuracy of the verification process, ultimately leading to more reliable and higher quality hardware designs. Identifying processor bugs presents a challenging task due to the inherent complexity of contemporary processor designs. Fuzzing techniques have emerged as a viable solution to mitigate this challenge. Fuzzing enables the execution of a multitude of test cases on the pro- cessor core and facilitates the recording of processor states that diverge from the normative behavior. In this way, using fuzzing techniques can enable effective bug identification while also enhancing the reliability and quality of processor designs. In our research, we apply the software fuzzing technique to five 32-bit processors, namely Picorv, Mor1kx, Sodor 1, Sodor 3, and Sodor 5. These processors are a combination of simple and pipeline processors that exhibit different behaviors when executing a series of instructions. Our implementation of the fuzzing technique involves creating a software equivalent of the hardware model that can be subjected to software fuzzing using fuzzers. In our approach, we take into account that each processor core may have a different memory model depending on the memory interface
4 CHAPTER 1. INTRODUCTION
it is connected to. To perform fuzzing on these cores, we generate random instructions for the core to execute when attempting to read a value from a new memory location within the instruction memory range. These instructions are generated in a set ISA format compatible with the architecture of the processor being tested. We then use the fuzzer to mutate the initial set of instructions and generate branches of test cases with a small percentage of change from the original instructions. As the fuzzer identifies interesting scenarios, the tree of test cases grows to explore additional states of the core. It’s important to note that the generation and mutation of instructions are controlled to avoid invalid instructions that may cause the processor to crash or produce incorrect results. Using this approach, we can effi- ciently test the different processors in our research, which include both simple and pipeline processors that behave differently when executing a series of instructions.
mutated mutated mutated mutated seed 1 seed 2 TTT seedn
mutated mutated mutated mutated mutated mutated mutated mutated seed1 10f seednof1 | (seed1of)2 2 of seed3 10f seed3 n of seedn 1 of || seedn n of
Figure 1.1: Mutation tree of initial set of ISA instructions by 3 percent
We apply the fuzzing technique on the processors and collect the executable files of the core with the coverage-enable feature to execute a series of intriguing test cases on it. We utilize hardware-level coverage monitoring to evaluate the RTL (register transfer level) of the core, which helps us identify the fuzzed portions of the core. In case of any system crashes, we can pinpoint which specific piece of code is responsible for it.
Chapter 2
Background
In this chapter, we discuss the basic concepts used to build our research project. We will be discussing the design verification and debug methods used for verification, the memory interfaces used in our cores such as wishbone interface and general memory interface, and software fuzzing.
2.1 Design Verification
The process of design verification is crucial in engineering to ensure that the final product or system adheres to the specified design requirements and functions as intended [14]. The process involves a series of rigorous tests and evaluations to verify the accuracy and completeness of the design and to identify and rectify any design flaws or errors that may arise during the testing phase. The goal of design verification is to ensure the reliability, performance, and safety of the final product. The process typically employs a combination of simulation, modeling, and physical testing techniques to validate the design. In the semiconductor industry, design verification involves thorough testing of the design at various levels, including the functional, register transfer level (RTL), and physical level. This includes simulation, emulation, and testing to verify the correctness and functionality of the design. Verification encompasses testing for performance, power consumption, thermal issues, and other critical parameters [4]. Design verification in the semiconductor industry is critical
5
6 CHAPTER 2. BACKGROUND
in reducing the risk of expensive and time-consuming design failures, ensuring that the final product is of high quality, and maintaining the industry’s reputation for delivering reliable and high-performance products. Therefore, design verification is a vital aspect of the semiconductor design process, and its successful implementation is crucial to ensure the success of the final product and the company. There are six types of design verification conducted in order to verify the sanity of the chip before tapping out:
Functional verification: Functional verification: In functional verification, random con- strained inputs are given to the design simulation and the functionality of the design is compared to the expected output. It is done at IP, system, and gate level [16].
Formal verification: Formal verification: In formal verification, a mathematical model of the design is used to compare the functionality of the design with the correspond- ing output. The verifi- cation process is carried out using automated tools that use mathematical algorithms to exhaustively analyze the system’s behavior and verify its correctness [8].
Coverage-based verification: This technique involves measuring the coverage of the RTL design’s functional space, including code coverage, branch coverage, and func- tional coverage. Coverage-based verification helps to ensure that the RTL design has been thoroughly tested and that all possible scenarios have been covered [18].
Assertion-based verification: This technique involves the use of assertions to specify the expected behavior of the RTL design. Assertion-based verification is useful for verifying complex functionality and identifying errors and bugs in the RTL design [19].
FPGA-based verification: This technique involves mapping the RTL design onto an FPGA (Field Programmable Gate Array) and testing its functionality. FPGA-based
2.2. DESIGN DEBUGGING METHODS 7
verification is useful for testing the design’s performance in a real-world environment
[15].
- Emulation-based verification: This technique involves mapping the RTL design onto an emulation platform to verify its functionality in a real-world environment. Emulation- based verification is used to test the design’s performance, power, and timing behavior [12].
Through these techniques, design verification helps ensure the quality, reliability, and func- tionality of semiconductor designs.
2.2 Design Debugging Methods
Debugging plays a vital role in the design verification process, and various methods are utilized to detect and rectify errors or defects in a design. The following are the common debugging techniques used in design verification:
Waveform Debugging: This method involves analyzing the simulation waveform to identify the cause of a particular error or behavior in the design. It helps to identify timing or signal integrity issues, as well as logic errors. The verification team analyzes the simulation waveform by zooming in on specific areas of the waveform to identify any issues [23].
Coverage Analysis: This technique involves measuring the extent to which the design has been tested, including code coverage, branch coverage, and functional coverage. It helps to identify untested or under-tested parts of the design. The verification team performs coverage analysis to ensure that the design is thoroughly tested and that all possible scenarios have been covered [11].
8 CHAPTER 2. BACKGROUND
Assertion-based Debugging: This technique involves using assertions to check the cor- rectness of the design. If an assertion fails, it indicates an error in the design. Assertion- based debugging is useful for identifying design flaws and errors in complex designs. The verification team creates assertions to specify the expected behavior of the design, and checks whether the design meets those specifications [6].
Emulation-based Debugging: This method involves running the design on an emulation platform to debug it in a real-world environment. It helps to identify performance or power-related issues that may not be detected in simulation. The verification team uses an emulation platform to test the design’s behavior under real-world conditions, and identify any issues that may arise [6].
Formal Debugging: This technique involves using formal verification techniques to identify design errors or bugs. Formal debugging can help to identify hidden errors or design flaws that may be difficult to detect using other debugging methods. The verification team uses formal verification tools to analyze the design and identify any hidden errors or bugs [2].
Debugging with testbenches: This technique involves using testbenches to simulate the design and identify errors or defects. Testbenches can be designed to check specific functionalities of the design and help to identify design errors or bugs. The verification team designs testbenches to simulate the design and checks whether it meets the desired functionality [5].
Debugging is a continuous process throughout the design verification cycle. The verification team uses multiple debugging techniques to ensure the correctness, reliability, and perfor- mance of the final product. By using these techniques, the verification team can identify and rectify any design flaws or errors, ensuring that the final product meets the specified
2.3. MEMORY INTERFACE 9
design requirements and functions as intended.
2.3 Memory Interface
A memory interface acts as an interface between a core processor or a digital system and a memory module, facilitating the transfer of data between the processor and the memory module. Memory interfaces are essential components in systems that require high-speed ran- dom access memory (RAM). The memory interface comprises various components, including address and data buses, control signals, and timing circuits. The address bus selects a spe- cific memory location from which to read or write data, whereas the data bus facilitates the transfer of data between the microprocessor and the memory module. The control signals regulate the timing of the data transfer, and the timing circuits ensure that data is trans- ferred with the correct timing parameters. In our research, we use two memory interfaces [24].
- General memory interface with memory signals: A memory interface serves as a bridge between a microprocessor or digital system and a memory module, facilitating the transfer of data. It comprises several crucial components, including the address and data buses, control signals, and timing circuits. The address bus is responsible for selecting the specific memory location being accessed, and the number of address lines determines the maximum number of memory locations that can be accessed. Similarly, the data bus is responsible for transferring data between the microprocessor and the memory module, and the number of data lines determines the width of data that can be transferred in a single access. Control signals, such as chip select, read enable, write enable, and output enable, are used to manage the data transfer and timing between the microprocessor and memory module. The chip select signal is used to select the
10 CHAPTER 2. BACKGROUND
memory module being accessed, while the read enable signal is used to initiate a read
operation, and the write enable signal is used to initiate a write operation. The output
enable signal is used to enable the output buffer of the memory module, allowing
the data to be transferred to the microprocessor.A well-designed memory interface is
essential to ensure efficient and accurate data transfer between the microprocessor and
memory module. It must be optimized for the specific requirements of the system, such
as the data transfer rate, latency, and power consumption. In addition, the interface
must be designed to handle the different types of memory modules, such as dynamic
random access memory (DRAM), static random access memory (SRAM), and flash
memory. The use of appropriate design techniques and simulation tools can help in
designing and verifying a reliable and high-performance memory interface.
- Memory interface using wishbone interface: In a Wishbone-based memory interface, the microprocessor communicates with the memory module through the Wishbone bus. The memory interface comprises two crucial components: the Wishbone master and the Wishbone slave. The Wishbone bus functions as the primary communication interface between the master and the slave, enabling the master to access and control the memory module. The Wishbone master, which is typically the microprocessor, initiates the memory read or write requests and generates the control signals, including the address and data signals, to communicate with the slave. The slave, on the other hand, responds to the master’s requests by sending the requested data back to the master. To facilitate the communication between the master and the slave, the memory interface that utilizes Wishbone comprises several components, such as an address decoder, data buffer, and control circuitry. The address decoder is responsible for decoding the address signals generated by the Wishbone master and selecting the appropriate memory location in the memory module. The data buffer is used to
2.4. SOFTWARE FUZZING 11
temporarily store the data being transferred between the master and the slave. The
control circuitry generates the necessary timing and control signals to ensure that the
data transfer occurs accurately and efficiently. The timing signals control the timing
of data transfer to ensure that data is transferred at the right time and in the right
sequence. The control signals, on the other hand, control the direction and speed of
data transfer, ensuring that the data is transferred in the desired direction and at the
desired speed [20].
2.4 Software Fuzzing
Software fuzzing is based on fuzzy logic which is a mathematical logic that facilitates rea- soning with uncertain or imprecise information, utilizing the concept of partial truth, where truth values can vary between completely true (1) and completely false (0), as well as in- termediate values. This logic is beneficial in cases where precise boundaries or definitions are challenging to establish, or when the available information contains ambiguity or un- certainty. It has been implemented in multiple areas, including decision-making, control systems, artificial intelligence, and engineering [25]. Fuzzing is a software testing technique that involves running a Program Under Test (PUT) with inputs that are not expected by the developer, i.e., fuzz inputs. The goal of fuzz testing is to find security-related bugs, including program crashes, by executing the PUT using inputs sampled from a fuzz input space that protrudes the expected input space of the PUT. A fuzzer is a program that performs fuzz testing on a PUT, and a fuzz campaign is a specific execution of a fuzzer on a PUT with a specific security policy. Fuzz configurations control the parameters of the fuzz algorithm, which can vary depending on the type of fuzzer used. The three major steps of fuzzing are typically input generation, input modification, and program execution. A fuzzer creates a
12 CHAPTER 2. BACKGROUND
collection of input values at random or based on a predetermined set of test cases during the input generation phase. The fuzzer alters the generated inputs by randomly changing some of their values, adding errors, or changing their structure during the input mutation step. The test program is fed the updated inputs during the program execution step, and its behavior is scrutinized. The fuzzer detects a bug if the application fails or behaves strangely. Numerous software systems, including network protocols, operating systems, and web appli- cations, have flaws that can be found through fuzzing. Fuzzing is very useful for discovering vulnerabilities that are challenging to spot using other testing methods, as those brought on by buffer overflows, format string flaws, or race situations [21]. Fuzzing techniques come in
Input
data that
Software Fuzzer gives
coverage
increase.
Run the
Initial Test-case input/mutated input
with executable
binary
Executable binan Mutates the ae Coverage check Crashes
Figure 2.1: Software Fuzzing
a variety of forms, such as black-box and white-box fuzzing. In black-box fuzzing, the fuzzer regards the software under test as a ”black box” and is unaware of its internal workings. White-box fuzzing makes use of the program’s source code or binary code, which the fuzzer
2.5. RISC-V PROCESSORS 13
may access, to direct the input generation procedure. Input generation methods include feedback-based fuzzing, random and mutation-based fuzzing, and many others. Without any prior knowledge of the program’s input domain, random fuzzing involves the fuzzer pro- ducing entirely random inputs. When using mutation-based fuzzing, the fuzzer starts with a collection of initial inputs and changes them by adding, removing, or changing the values of each one. In feedback-based fuzzing, heuristics like coverage or distance-based metrics are used by the fuzzer to direct the input generation process in response to the program’s output.
2.5 RISC-V Processors
An emerging form of open-source processor architecture are RISC-V processors. RISC-V, which stands for ”Reduced Instruction Set Computing Five,” is the name of a class of mi- croprocessor architecture that is intended to be straightforward and uncomplicated to use. As an open standard, the RISC-V architecture enables anyone to create their own RISC-V processor without having to pay royalties or licensing costs. They are open-source, RISC-V processors offer for more flexibility and customization than proprietary processor designs, which is why they are becoming more and more popular in the industry. The RISC-V architecture is also intended to be modular, making it simple to swap out and upgrade various processor parts as necessary. The RISC-V architecture is made to be extremely scalable, making it appropriate for a variety of applications, from high-performance com- puter systems to low-power microcontrollers. The energy-efficiency of RISC-V processors is another feature that makes them perfect for usage in battery-operated devices. A nonprofit organization called the RISC-V Foundation works to further the creation and use of the RISC-V architecture. The organization helps firms and developers interested in creating
14 CHAPTER 2. BACKGROUND
RISC-V processors by offering them resources and support. A large number of businesses, including Western Digital, NVIDIA, and Qualcomm, are RISC-V Foundation members and are working actively to build RISC-V-based solutions. With numerous standard versions offering alternative instruction sets and addressing modes, the RISC-V ISA (Instruction Set Architecture) is made to be very flexible. Hardware designers and software developers can choose from a variety of RISC-V standard versions, including as RV32I, RV32E, RV32M, RV64I, RV64M, and RV128I. The base integer instruction sets for 32-bit and 64-bit proces- sors, respectively, are designated as RV32I and RV64I, and they offer a fundamental set of instructions for arithmetic and logic operations, data movement, and control flow. For em- bedded systems with constrained memory and computing power, the RV32E instruction set is a more compact and effective variant of the RV32I instruction set. The base instruction set is expanded to enable 128-bit integers in the RV128I version, while the RV32M and RV64M variants add support for integer multiplication and division operations. The RISC-V ISA is made to be extendable in addition to the conventional instruction sets, enabling the in- clusion of customized instructions to support particular applications. By using user-defined instruction set extensions (ISA extensions) that can be added to the base instruction sets, this extensibility is made possible. For specialised applications like encryption or signal pro- cessing, ISA extensions can be utilized to provide specialized instructions. These extensions can be made to integrate easily with the existing RISC-V instruction sets [9].
2.6 Open RISC 1000 Architecture
The OpenRISC 1000 architecture uses a 32-bit RISC processor with a load-store architecture, which implies that all instructions only work with registers and memory. It contains a fixed- length instruction set and a straightforward instruction encoding technique that facilitates
2.6. OPEN RISC 1000 ARCHITECTURE 15
speedy decoding and execution of instructions. It supports both big-endian and little-endian byte ordering. The OpenRISC 1000 architecture’s modular structure, which enables cus- tomization and optimization for various applications, is one of its primary characteristics. It features a modular pipeline that can be set up to handle various instruction and data caches as well as various pipeline stages. Because of its adaptability, the OpenRISC 1000 architecture can be tailored for certain use cases like high-performance computing, low-power embedded systems, or real-time applications.
Chapter 3
Design
There are 3 main stages involved in building software fuzzing of 32 bit processors. In the first stage, we convert the design under test into executable binaries using a tool which has the capability to create one to one hardware to software translation. While creating software binaries, we make sure that the hardware is compiled with the compiler compatible with our software fuzzer. In this stage, we get the C equivalent of hardware code which is given to the second phase for software fuzzing. In the second stage, we mutate the executable binaries with the custom mutator which has constraints to take care of the hardware requirements such as keeping the jumps in the memory bond, having load and store instruction byte/word compatible depending upon the instruction used, etc. In the last stage, we evaluate how software-based fuzzing on hardware gives the coverage increase with respect to time. To achieve this, we calculate the speed of the fuzzing core and the growth of line coverage at the register transfer level.
3.1 First Stage
The first stage of hardware design involves three main components: core design, creating an infrastructure that supports compressed memory for holding instructions, and developing a test bench that generates random instructions based on the instruction set architecture used by the core. We integrate core design and memory model in the top in this stage. The
16
3.1. FIRST STAGE 17
test bench provides the system with clock and reset signals, and the core begins reading from external memory once it is out of reset. To enable fuzzing and eventual mutation, valid instructions are loaded into the external instruction memory at zero runtime. Our test bench consists of two blocks: one that generates random instructions and another that provides constraints based on the core’s memory implementation. We establish handshaking between the memory model and test bench by utilizing a direct programming interface between the two languages.
C- Test
Top Memory Model
Clock
Reset
Address bys
RISC or” |
Processor
Figure 3.1: Stage1: Converting Hardware into executable binary
3.1.1 Memory Modeling
When implementing conventional hardware memories, the size of the memory is very huge as compared to the actual utilisation of the memory while running one test case. To solve this problem we propose to use a sparse memory matrix instead of a conventional memory, by doing so we will reduce the size of the memory. We will be creating memory space that will
18 CHAPTER 3. DESIGN
actually be used by the processor to boot after it comes out of reset. Using this method will compact our external memory which can give us high fuzzing performance as the memory required to store the initial input states will decrease drastically. In hardware fuzzing, we capture the internal states of the processor. Since the program for fuzzing is a binary version of the RTL design of the core, we capture the states of the internal registers to achieve greater coverage. We use two compact memories to capture these states. In one memory, we filter out the unwanted data written by the core and reserve the instructions and internal register states in another. When the core performs a load instruction from a memory location, the memory location can either have a preexisting data/instruction, which will be loaded into an internal register, or it can point to an empty memory location. In the latter case, we create a new memory location and then store a randomly generated data or a random instruction into it, depending upon the accessed address. This way, we can preserve the initial instructions as well as the new internal register state. On the other hand, when the core performs a store operation, it stores the random data given in its instruction into an external memory, which is not a state of the core that we are interested in preserving. Therefore, we write this data in our secondary memory. Our primary memory holds all the instructions and the internal register state of the core, whereas the secondary memory only holds the unwanted output store data from the core. When fuzzing RISC-V processors, our expectation is to only receive crashes when the processor halts with a valid RISC-V instruction indicating a design bug. Therefore, one of the biggest challenges in designing this system is to avoid having any false positives. To overcome this hurdle, we have designed two algorithms:
- Memory modeling is done differently for different cores depending upon the memory interface of the respective cores, with the availability of instruction and data buses playing a crucial role. For cores having a common memory bus for both data and instruction memory, crashes occur after a few clock cycles because the program counter
3.1. FIRST STAGE 19
keeps incrementing and expects an instruction. Due to the lack of First stageurcation
of these memories, data values also get stored in this memory resulting in a trap
exception. For example, if we have a word store instruction at address 0x4 to store
a randomly generated immediate value to a memory location 0x16, as soon as the
program counter reaches address 0x16, the processor halts with a trap exception since
it has encountered an invalid instruction. To eliminate this problem, we have divided
the memory into two parts: the first part ranging from 21̂-21̂6, which only comprises
instructions, and the second section of the memory ranging from 21̂7-23̂2, which is
utilized for loading and storing random data.
Top
Memory Model
clk
Sparse memory Foy 0x10000 matrix
|Oxfffc |ᴼˣᶠᶠᶠᶠᶠᶠᶠᶜ
Intelligence to segregate
Figure 3.2: First stageurcation of memory
- We perform constraint-based instruction generation to avoid false crashes during fuzzing of RISC-V processors. This is particularly important for cores with common memory buses for data and instruction memory. If the program counter enters the data mem- ory, the core will halt with a trap exception, leading to a false crash. To prevent this, we constrain the instructions to jump only within the address range of instruction
20 CHAPTER 3. DESIGN
memory. For separate instruction and data buses, this gives an out of memory range
error, while for common memory buses, it halts due to an invalid instruction. To opti-
mize this algorithm, we consider the worst-case scenario where the first instruction is
a jump instruction to the last instruction-memory location. In this case, fuzzing will
halt after executing only two instructions. To avoid this issue, we fuzz for 200 clock
cycles for each fuzzing iteration and move the range of jump to :
instruction_memory_end_address - (4*(time_taken_to_execute_one_instruction)).
Memory interface for respective cores
Cores Type of memory interface Separate/Common bus
Picorv General memory interface Common bus
Mor1kx Wishbone interface Separate bus
Sodor1 General memory interface Separate bus
Sodor3 General memory interface Separate bus
Sodor5 General memory interface Separate bus
Table 3.1: Mentioning memory interface of RISC processors, type of memory interface de- termines the memory modeling of the processor
3.1.2 Direct Programming interface
The direct programming interface (DPI) acts as a bridge between the testbench and memory model to facilitate communication at runtime. Whenever the program counter reaches a new address space, the memory model invokes a constraint block to produce random parameters that satisfy all hardware requirements and avoid false positives. The memory model then creates a new entry in the primary memory with the instruction generated by the instruction generator.
3.1. FIRST STAGE 21
3.1.3 Random Instruction generator
For generating random instructions, we use an inbuilt function that is initialized based on the current time to generate pseudo-random integers. The generated integers are used to select parameters from a pool, which are used to create an instruction. Our research focuses on creating designs for the RISC-V 32i and OpenRISC 1000 architectures. For cores based on the RISC-V 32i instruction set, the instructions consist of R/I/S/U formats, which include two sets of source registers, a set of destination registers, two functions (funct3 and funct7), an immediate offset, and an offset that signifies the group of instructions. Based on the random selection of these parameters, instructions are created with their respective formats. Cores based on OpenRISC ORBIS32 have two sets of source registers, a set of destination registers, immediate data, and an opcode with specific bits hardcoded for specific instructions. Once the parameters are selected, the instruction is created based on its specific format.
3.1.4 Constraint block
In the constrained block we write logic to solve the problem which breaks the system and give us false crashes. Following are the challenges and the solutions:
- Currently, we utilize a spare memory matrix in our design to prevent potential crashes caused by attempts to access non-existent memory locations. When conducting soft- ware fuzzing, if a test-case tries to access an uncreated memory area, it will be flagged as a crash due to a segment fault. To address this issue, we analyze the address of the jump and take appropriate action based on its location. If the address falls within the data memory range, we create and load the memory with random data. Conversely, if it is an instruction memory address, we constrain the instruction accordingly before loading it into the memory.
22 CHAPTER 3. DESIGN
This approach allows us to avoid crashes and maintain the stability of our system.
We ensure that the offset used during a load or store operation of data or instruction to any memory location is byte, half-word, or word bound, depending on the specific instruction. To achieve this, we always perform an AND operation with zero on bit zero of the offset to ensure that it remains half-word bound. Similarly, to maintain word bound offset, we perform an AND operation with zero on the last two bits of the offset. By following this approach, we guarantee that the memory access is always correctly aligned and prevent any potential issues that may arise due to misaligned memory access.
During a register indirect addressing mode, the base address of the jump is held in the internal register of the core. However, it is possible that the internal register may hold a value that is not word bound due to prior instruction execution. This could lead to an invalid memory access when attempting to jump to a location using this register. To ensure that the jump always targets a valid address location, we create two new memory locations. In the first memory location, we insert an ADDI instruction that randomly loads the internal register with a valid word-bound base address. In the consecutive memory location, we load the register indirect mode instruction. Additionally, we constrain the instruction to have a word-bound offset, which is then added to the internal register’s base address.
When dealing with relative address instructions, we calculate the offset range based on the program counter value. While selecting a random value for the offset, we ensure two things. Firstly, the offset must be word-bound to guarantee that the instruction is correctly aligned with the memory. Secondly, we verify that the instruction falls within the range of the instruction memory, ensuring that the instruction is valid and executable.
3.1. FIRST STAGE 23
Memory model @
DPI 0
Sparse
Instruction with indirect
= memory access. (Instr1)
as Where, Rd = internal
ep
RIM = random immediate value whichisword bounded
Figure 3.3: Stage2: Register indirect addressing mode
ET + Rand_val inside{
DPI a 0O:end_of_imem - PC}
Send PC value Bes] Offset= Rand_val
mode &
Sparse memory matrix
DPI
Figure 3.4: Stage2: Relative addressing mode
24 CHAPTER 3. DESIGN
- When dealing with a STORE instruction in fuzzing, the primary challenge we face is that it breaks the fundamental concept of segregating the instruction and data memory. Writing a random value to an address within the instruction memory can cause the system to fail, as it encounters an invalid instruction, which halts the fuzzing process. To overcome this challenge, we write the values into a secondary memory to maintain the integrity of the memory segregation concept. This approach also enables us to utilize the full range of memory for write operations without disrupting the execution of the program.
Dpi interface
Sparse Dpi interface memory matrix
memory
Read
Figure 3.5: Stage2: STORE instruction for write operation
3.2 Second Stage
In the second stage we have the software equivalent of the core and its binary. For fuzzing
the core we need three crucial components: executable binary of the core, input seed and
3.2. SECOND STAGE 25
custom mutator to mutate the initial test case based on the mutation percentage.
Executable Sg ++ binary Check the discard coverage
Custom
Mutator
Interesting
test-cases
Figure 3.6: Stage2: Block diagram representing the AFL fuzzing of the Hardware executable
binary using custom mutator
3.2.1 Executable binary
When converting hardware to software using conversion tools for processor compilation, the compilers can be switched to fuzzer-compatible compilers to create a binary that is capable of fuzzing using coverage collection intelligence. The conversion tool generates equivalent software classes for the hardware, which are then compiled to create the desired binary that can be further mutated. It is crucial to compile the core with a fuzzer-compatible compiler in order to mutate efficiently and obtain more edge coverage in each fuzzing iteration. Be- fore compiling the core, the testbench should be equipped to capture input files from the
26 CHAPTER 3. DESIGN
command line in order to be in sync with the fuzzer command format.
3.2.2 Input Seed
When starting fuzzing, the program loads an initial seed into the external memory of the core. It runs a processor for a constant number of clock cycles in each fuzzing cycle, which can be calculated by dividing the constant clock cycles by the number of clock cycles required to execute one instruction. The initial seed consists of instructions from all the opcodes to spread the mutation across the executable binary. The initial seed is the primary input to the fuzzer, and it keeps mutating in every iteration based on the selected mutation parameters and percentage. When the fuzzer discovers a mutated test case that provides more coverage, it identifies it as an interesting input and saves it in a queue. All the mutated inputs saved in the queue are further mutated to find more coverage. The fuzzer can find test scenarios that deeply explore the code and reach states that cannot be reached by manual testing. It can also retain and further explore uncovered areas of the code.
3.2.3 Custom Mutator
Generally, fuzzers have an inbuilt mutator that includes features such as flipping bits, bytes, performing random arithmetic operations, trimming the input, etc. However, when fuzzing hardware binaries, using fuzzer-based mutators can result in mutated test cases modifying instructions into invalid instructions leading to false crashes. Therefore, we use our custom mutator to change the parameters of the instructions. To use a custom mutator, we not only have to enable this mode but also have to explicitly disable the trim property of the fuzzer, else it will randomly chop off the instructions resulting in interruption while fuzzing. We have 3 custom mutators for each type of RISC-V architecture and memory type. Before
3.2. SECOND STAGE 27
beginning to fuzz the core, we set the mutation percentage that we want to achieve. This then calculates the number of lines to be mutated and randomly selects the instruction number that corresponds to a particular key-value pair. In our design, the key is the address of the memory where the instruction is stored, and the value is the valid instruction stored in the corresponding key address. After getting the instructions to be mutated, it randomly selects the parameter that will be altered in the instruction. Once the parameter is selected, it is mutated to other possible values, which will keep the instruction format intact but cover a different scenario.
Custom Mutator
Testcase Calculate Randomly Segregate With the selects the address and 100 number instructions to instruction entries instr. mutate.
31 2019 1514 122 1 76
rst func3 rd opcode I type
instruction,
Disintegrate the instruction into its field value.
to Select any value other Reconstruct Return a
be mutated than the original from the mutated the pool of possibilities. instruction file.
Figure 3.7: Custom Mutator
28 CHAPTER 3. DESIGN
3.3 Third Stage
In this stage of the design, we are collecting coverage data for the hardware description language (HDL). The fuzzer is helping us to generate a collection of test cases that have interesting inputs, resulting in an incremental increase of coverage at the binary level. How- ever, we are primarily interested in capturing coverage at the register transfer level (RTL), so we need to build an infrastructure to capture line coverage of the core at the hardware level. This will enable us to more accurately analyze the performance of the core and identify any areas that need improvement.
Database
With —
coverage
report
| Annotated: Incremental|
RTL Coverage
Executable report binary with Interesting coverage test-cases enabled Python script to plot coverage over time graph
Figure 3.8: Stage3 Infrastructure implemented to collect line coverage of the processor
To evaluate the effectiveness of the software fuzzing at the hardware level, we analyze the
verilog lines covered by the testcases generated by software fuzzer. We follow the steps below:
1. We create a parallel directory that contains the source code of the processors and
an executable binary with line-coverage enabled. Enabling line-coverage allows us to
3.3. THIRD STAGE 29
monitor the range of lines covered by an individual testcase. In this setup, we use a
set of software fuzzer generated testcases as our sample points.
We use a script to run the line-coverage enabled binaries sequentially with the testcases in relation to time. Each execution produces an output report that determines the number of lines covered by that individual testcase. These reports are dumped in a separate directory.
We implement a script which merge the coverage reports incrementally with respect to time to obtain the line coverage of the processor over time.
Chapter 4
Implementation
4.1 First Stage
In the first stage of our research, we aim to convert the hardware design into software. The primary objective is to create a software that is the closest equivalent to machine language. To achieve this, we target to convert the hardware language into C language, which has been the fundamental language for low-level programming. Therefore, we explore tools that can provide us with an optimal hardware-to-C conversion. In addition, we also need to consider that the tool we select must function as a simulator, allowing us to analyze and debug the hardware crashes.
4.1.1 Hardware
We are conducting research on five open source RISC-V cores, namely Picorv, Sodor1, Sodor3, Sodor5, and Mor1kx. To analyze and evaluate these processors, we are using their Verilog code which is integrated with a memory model in a top file. This top file is then called in a C-based testbench file, where the processor is provided with a clock and taken out of reset for further executions.
30
4.1. FIRST STAGE 31
4.1.2 Testbench
We build our testbench in C language due to its compatibility with Verilator and high- speed execution as it is closer to the machine level. To ensure faster execution, we write our constraint and instruction generator blocks in C language. For the memory model, we use System Verilog language as it gives us the benefits of hardware structure and software concepts. Connecting the input/output signals of the core to a memory model requires a hardware module structure that is sensitive to time and provides us with the leverage to exploit high-level programming concepts such as creating new memory allocations at runtime. To achieve interaction between C and System Verilog, we use direct programming interface (DPI), which is supported by Verilator. This allows us to call System Verilog functions directly in C and C functions directly in System Verilog. As we create our memory in System Verilog, we use associative arrays as our memory. In order to achieve high space and timing performance, our optimum selection of memory is a memory that is like a sparse matrix, from which we can easily retrieve any value. Associative memories are collections of key-value pairs, in which the key and value hold a relation between them. In our case, keys are the address, and the values are instruction/data depending on the address range keys lie in.
4.1.3 Functional hardware to software converter tool
We are choosing Verilator as our hardware description language (HDL) to C language con- verter and functional simulator. Verilator has been chosen over other tools due to its event and time awareness which is required for transition from static event scheduling to dynamic event scheduling. It is a widely accepted high performance tool in the verification commu- nity which supports both Verilog and SystemVerilog(SV) based code. Verilator also provides
32 CHAPTER 4. IMPLEMENTATION
support for object oriented programming in SV. The functional simulations in Verilator en- able us to create a C executable which can be used by software fuzzer for fuzzing. This tool provides us with the advantage of having event driven simulations which makes it easier to detect bugs and errors in our hardware designs. Overall, Verilator is an optimum choice for our project as it allows us to efficiently convert our HDL code into a software which is closer to the machine language, and perform functional simulations which are essential for detecting any design flaws.
4.2 Second Stage
For hardware executable binary fuzzing, we are utilizing AFL++ fuzzer. We selected AFL++ because it allows us to create our own mutator, a key requirement for instruction-based fuzzing. Traditional software fuzzing mutates the initial seed using various logical or arith- metic operations, bit and byte flipping, and other techniques, but since instructions executed on a processor must be in a specific format, we need to implement our own custom mutator. AFL++ offers cutting-edge fuzzing and a built-in custom mutator API, providing us with comparable, reproducible, and reusable research on our processors. AFL++ requires three primary inputs: the processor’s executable binary, the initial seed, and a custom mutator that continually mutates the initial and queued seeds.
4.2.1 Executable binary
During the generation of the executable binary in first stage, it is essential to ensure that the binary is compiled using AFL++ compilers to enable detection of the coverage increase by mutated test cases. AFL++ has an inbuilt decision-making component that identifies
4.2. SECOND STAGE 33
interesting scenarios and stores the corresponding test cases in an output queue, based on coverage increase. The tool discards those mutated test cases where coverage does not in- crease. To enable this decision-making component in AFL++ to function correctly, we need to change the compilers to AFL++. Once the executable binary is provided to the AFL++ tool, we also need to give the initial seed at the command line as per the AFL++ command format. To fulfill this requirement, we write a code in our C testbench to read a file from the command line and load its values to our memory model. With every fuzzing iteration, AFL++ provides a new mutated test case to the executable binary, and the processor exe- cutes slightly different sets of instructions. Gradually, we achieve coverage over a period of 24 hours. AFL++ provides a custom mutator API that facilitates state-of-the-art fuzzing and enables the creation of custom mutators for mutating initial and queued seeds.
4.2.2 Initial Seed
We choose to run one fuzzing iteration for 200 clock cycles, so we prepare the initial seed file accordingly. We determine the number of clock cycles required to execute one instruction and divide 200 by that number to obtain the number of instructions needed in the initial seed file. The initial seed file is in the form of a .hex file and contains a collection of addresses and instructions.
4.2.3 Custom Mutator
We implemented our custom mutator using Python3 for its efficient mutation capabilities. The mutation percentage is set to 3%, and the mutator takes an initial seed, mutates it by 3%, and checks for interesting test cases. If a test case is interesting, the mutator will further mutate it. By the end of the 24-hour fuzzing period, we observe a gradual increase
34 CHAPTER 4. IMPLEMENTATION
0
1 abab3793
2 c9ed4c93
3 a2111913
4 290c4b13
Figure 4.1: Executable binary
in coverage over time.
4.3 Third Stage
In the final stage of our evaluation, we aim to assess the performance of hardware fuzzing just like software. To achieve this, we calculate the instructions executed per second and line coverage of the Verilog code of the processor. To calculate the instructions/second, we use the information provided by AFL++. As we run AFL++, a dashboard appears which shows the execution/second information. Since each test case has a constant number of instructions, we can calculate the number of instructions executed by the processor in a second. For line coverage evaluation, we need to trace back the coverage at the Verilog level. To do this, we create an execution binary of the processor from Verilator with the coverage enabled feature. This generates the execution binary with breakpoints that capture the line coverage when the test cases are run with it. Since AFL++ has created various interesting test cases that provide progressive coverage, we build our infrastructure using python3 and bash scripts to achieve this. The python3 script enables us to run the executable binary with all test cases, giving us coverage by each test case. Once the coverage report from each test case is generated, we run a bash script to merge the coverage with respect to time. We
4.3. THIRD STAGE 35
use another Python script to plot the coverage over time graphs.
Chapter 5
Review of Literature
In this field, there are several related works being done, such as hardware fuzzing, which involves simulating the behavior of a Device Under Test (DUT) and testing various inputs for code sequences using a program called Hardware Simulation-based Fuzzing. If an incor- rect code sequence is given, the fuzzer forks a new process, which is equivalent to restarting the DUT, and tries again. An SVA (SystemVerilog Assertions) is broken if the correct code sequence is provided, and the fuzzer records this as a software crash. Researchers then investigate these crashes to find any errors in hardware design. During fuzzing, they use both invariant testing and (gold) model checking to spot bad hardware behavior [22]. To confirm the accuracy of a hardware design, two methods are used: invariant checking and (gold) model checking. Invariant testing verifies that the design’s properties have not been broken using a collection of assertions, such as SystemVerilog Assertions or software-side C/C++ assertions. They determine whether a set of conditions is always true when the hardware design is being implemented. If an assertion is broken, a bug has been discov- ered. On the other hand, (gold) model checking compares the simulated behavior of the DUT to a distinct model of the DUT’s right behavior that is emulated in software. The researchers aim to develop a ”golden” reference model that illustrates what the hardware ought to accomplish given various inputs. After that, they make a comparison between this golden model and the DUT’s real fuzzing behavior. If there is a discrepancy between them, a bug has been discovered. While the researchers have made significant progress in the
36
37
field of hardware fuzzing, there are a few limitations to their approach. For instance, their tool, Hardware Fuzzing HSB, covers only a portion of the SystemVerilog language, which may limit its compatibility with certain hardware designs. Additionally, their method relies on the presence of a software fuzzer to provide code sequence inputs for the DUT. These limitations suggest that further research may be necessary to develop more comprehensive and versatile hardware fuzzing tools and methods. The field of hardware fuzzing has seen significant progress. However, there are a few limitations to their approach that need to be addressed. For instance, the Hardware Fuzzing HSB tool covers only a portion of the SystemVerilog language, which may limit its compatibility with certain hardware designs. Additionally, their method relies on the presence of a software fuzzer to provide code se- quence inputs for the DUT. In our research, we have tackled the limitations of existing hardware fuzzing methods by developing our own custom mutator. Our mutator not only reduces the dependency on software fuzzer to generate input sequence but also minimizes the crash overhead. One of the major advantages of our approach is the elimination of the manual labour required to identify if a crash is a valid crash or a false positive. By using our custom mutator, we have been able to significantly improve the efficiency and effectiveness of hardware fuzzing, allowing for the identification and mitigation of hardware vulnerabilities with greater speed and accuracy. In this field, researchers have also developed RFuzz, which introduces MetaReset transformations to address the issue of deterministic test execution in FPGA-accelerated simulation. They define the idea of Mux Toggle Coverage, which can be obtained during FPGA-accelerated simulation and utilized as input to the fuzz testing process. To find bugs, they employ manual waveform inspection, comparing the execution to a reference model, or a variety of customized invariant (assertion) checkers. They use dif- ferent manual and automatic coverage criteria to ensure that the device under test (DUT) has been covered to an appropriate extent. The researchers in RFuzz introduce the idea of Mux Toggle Coverage, which may be obtained during FPGA-accelerated simulation and
38 CHAPTER 5. REVIEW OF LITERATURE
applied as feedback to the fuzz testing procedure. However, their strategy is only applicable to simulations that can be accelerated using an FPGA and the method might not work for all designs and necessitates substantial computational resources. In contrast, our research have overcome these limitations by introducing hardware fuzzing techniques at the simula- tion level with sparse memory which reducing the fuzzing time and require lesser storage for saving the interesting testcases. This approach allows for more comprehensive testing of the DUT, regardless of whether it can be accelerated using an FPGA or not, and reduces the computational resources required for testing. In the field of hardware security, researchers have made significant strides in developing new methods for hardware fuzzing. One such method is RFuzz, which uses MetaReset transformations to address the issue of deterministic test execution in FPGA-accelerated simulation. They introduce the concept of Mux Toggle Coverage, which can be obtained during FPGA-accelerated simulation and used as input to the fuzz testing process. To detect bugs, they use a combination of manual waveform inspection, comparison of execution to a reference model, and customized invariant (asser- tion) checkers. They also employ different manual and automatic coverage criteria to ensure that the DUT is adequately covered. However, the RFuzz method has some limitations. For example, it is only applicable to simulations that can be accelerated using an FPGA and may not work for all designs. Furthermore, it requires significant computational resources, which can limit its scalability and effectiveness. In contrast, our research introduces a new approach to hardware fuzzing that overcomes these limitations. We have developed hard- ware fuzzing techniques that can be applied at the simulation level with sparse memory. This reduces the fuzzing time and requires less storage for saving the interesting testcases. Our approach allows for more comprehensive testing of the DUT, regardless of whether it can be accelerated using an FPGA or not, and reduces the computational resources re- quired for testing. With this approach, we can significantly reduce the time and resources required for hardware fuzzing, making it more accessible and practical for a wider range
39
of applications. The research team has also developed DIFUZZ RTL, an RTL fuzzer that uses register-coverage guided fuzzing techniques to identify state transitions in the finite state machine of RTL designs. They have created several techniques, such as cycle-sensitive register coverage guiding, asynchronous interrupt event handling, and a unified CPU input format with Tilelink protocols to support various CPU RTLs. By taking into account spe- cific RTL design characteristics, DIFUZZ RTL can generate test cases that trigger unknown bugs in CPU RTL designs and identify the vulnerable states that cause these issues [10]. However, the research team also encountered some limitations in their work. They found that cycle-insensitivity prevented them from accurately capturing states, while complicated multiplexer wiring resulted in scalability issues. Additionally, writing new code for different designs depending on their functionality posed a challenge in terms of scalability. Injecting code into the present RTL also had implications in terms of effort and resource levels. In contrast, our research employs a processor-agnostic approach and can be used to fuzz any processor developed with the RISC-V 32i and 1000 architectures. The functionality of the processor is independent of our fuzzing process, and we do not encounter the same limitations as the DIFUZZ RTL. By focusing on the RISC-V architecture, we can ensure compatibility with a wide range of processors and avoid the scalability and wiring issues encountered by the DIFUZZ RTL. Our approach provides an efficient and effective method for identifying unknown bugs and vulnerable states in RTL designs.
Chapter 6
Results and Evaluation
In this section, we will assess our design and address two key questions. Firstly, we will examine how the constrained custom motor has assisted in eliminating the crash overheads of the software fuzzer. Secondly, we will discuss how we achieved high core performance through the use of sparse memory matrix as our external memory.
6.1 Killing crash overheads
Software fuzzers typically manipulate input data during every fuzzing cycle. They utilize various schemes to modify the input data and achieve greater coverage. The inbuilt mutator carries out a range of arithmetic and logical operations on the input data to make it different from the previous entry. The software fuzzer has the ability to identify inputs that can increase the program’s coverage, and once it has compiled all of the interesting inputs, it begins to mutate them to explore deeper program states. Additionally, software fuzzers often use a trim functionality that removes certain parts of the input and evaluates how the program responds to it. When fuzzing with processors, using invalid instructions can cause the processor to hang and become irretrievable, requiring a reset to recover. Such scenarios result in crashes during software fuzzing. Once the hardware of the processor is converted into equivalent software, it should only be fed with valid inputs; otherwise, the output will be filled with numerous crashes. After completing the fuzzing, we evaluate the crashes by
40
6.1. KILLING CRASH OVERHEADS 41
loading the crashed input to the processor and analyzing the core’s waveforms. Relying on the inbuilt mutator can result in thousands of crashes that must be manually sorted by the verification engineer. To reduce this effort, we develop our own custom mutator to only supply the processor with valid input. The inbuilt intelligence then directs the software fuzzer to further mutate the interesting, valid inputs to reach deeper states of the processor. By leveraging the custom mutator feature of AFL++, we are able to eliminate unnecessary crashes or false positives.
6.1.1 Picorv custom mutator
We create a custom mutator based on the RISC-V 32i instruction set to facilitate fuzzing for the picorv processor. We meticulously define the fields of the instructions for all R/I/S/U formats to maintain the format of the instruction. During each cycle, the custom mutator disintegrates the instruction by its field and replaces the randomly generated parameter with the original parameter in the instruction, thus exploring different parts of the program. Since the picorv has the same bus for instruction and data memory, we segregate the memory based on the memory range, which further requires us to constrain the selection of the new offset value to meet the requirement of redirecting the program counter in the instruction memory. We also identify whether the input is a data entry or an instruction to be mutated based on the address of the input entry. If the address is less than 21̂7-4, then the entry corresponds to an instruction; otherwise, it’s a data entry. By doing so, we achieve 24 hours of fuzzing of the processor with zero false positive crashes.
42 CHAPTER 6. RESULTS AND EVALUATION
6.1.2 Sodor 1, Sodor 3 and Sodor 5 custom mutator
We create a custom mutator for Sodor cores based on the RISC-V 32i instruction set, which allows us to mutate and repack instructions in the same way as picorv instructions. The cores have separate instruction and data memory, which relaxes the constraint on jumps to data memory. Since we only mutate the instruction memory, we do not need to identify if the entry is data or instruction, and all the values present as well as changed must be of valid R/I/S/U format. The custom mutator codes all the intelligence for analyzing the instructions and recreating a new instruction to provide us with crash-free fuzzing for our processor. The mutator also takes into account parameters such as internal registers 1 and 2, a destination register, and function type, which determine the operation in the specific category of the instruction. For example, opcode 0x13 is the category of all the immediate instructions, and the function determines whether it’s an immediate ADD or a XOR. We can expect crashes only in case of an actual fault in the hardware where the core gets stuck due to an unwanted series of instructions.
6.1.3 Mor1kx
The Mor1kx processor is built on the openRisc 1000 architecture, which requires a different custom mutator design compared to the ones used for picorv and Sodor cores, as the fields and instruction creation are distinct for this architecture. Since Mor1kx has a different bus interface for instruction and data memory, we only include the intelligence of selecting a parameter and altering it with the various valid possibilities of that particular field in that instruction, thus exploring a new set of coverage for the processor. Mor1kx instructions have several constant values for determining the group of instructions. In case of variations of the group type, we repack the instruction based on the respective format. Custom mutators have
6.2. PERFORMANCE ANALYSES 43
been instrumental in reducing the manual sorting of crashes and enhancing the performance of the fuzzer by utilizing fewer storage resources.
6.2 Performance Analyses
During system startup, the processor relies on memory as the source of its initial instructions, with the program counter starting to boot from the memory’s address space. In typical hardware systems, the memory size is determined by the number of address lines. For instance, a 32-bit address bus results in a maximum memory size of 4.29 gigabytes. However, during software fuzzing on the processor, converting memory to an equivalent software model requires significant storage space, which decreases the fuzzing efficiency. To speed up the fuzzing process, a sparse memory matrix is used to load initial instructions to the processor, which are then mutated in each fuzzing iteration over a 200-clock cycle period. As each instruction takes two clock cycles to execute, only 100 instructions can be stored in memory at the beginning of the fuzzing process. Thus, instead of using a memory size of 4.29 gigabytes, a sparse memory of just 100 words can significantly reduce memory usage while increasing the speed of analyses and mutations by the fuzzer. In our research we are mutating the input by 3 percent since mutating with lower percentage gives better results with time compared to having high percentage of mutation [13].
- Picorv: Picorv is a RISC V processor that is based on the RISC V 2019 instruc- tion architecture and can handle the RISC-V 32i instruction set, which comprises the R/I/S/U format. To calculate the instruction per second speed, we determine the num- ber of executions per second of the fuzzing logic. For Picorv, the execution per second is currently 161/second. Each test case executed consists of 100 instructions, with two clocks required to execute each instruction. Therefore, the number of instructions per
44 CHAPTER 6. RESULTS AND EVALUATION
second is calculated as 16100/sec.
Sodor1: Sodor1 is a processor core with a single-stage pipeline, meaning that once an instruction is fetched in the first stage, another instruction is immediately added to the queue for execution. The core’s execution rate is 139 instructions per second, but the first two clock cycles are used to initialize the core. As a result, in test cases, a total of 99 instructions or test cases will be executed, resulting in an instructions-per-second rate of 13,761.
Sodor 3: Sodor 3 is a processor with a 3-stage pipeline, allowing for an execution speed of 199.4 instructions per second and resulting in an instruction-per-second rate of 19,740. This execution speed is notably higher than that of Picorv and other single- stage pipeline cores, due to the additional stages in the Sodor 3 pipeline.
Sodor 5: Sodor 5 is a processor with a 5-stage pipeline, allowing for parallel execution of each task by dividing it into five stages. This results in the highest execution speed among the Sodor cores, with a rate of 272.4 testcase executions per second. The instruction-per-second rate for Sodor 5 is 26,967. The speed metrics are also provided in Table 6.1.
Mor1kx: The Mor1kx processor is based on the open RISC 1000 Architecture and has distinct mutation parameters compared to other processors. It boasts an execution speed of 249.7 instructions per second, resulting in an instruction-per-second rate of 24,720.
6.2. PERFORMANCE ANALYSES 45
Fuzzing speed of RISC cores
Cores Fuzzing iterations /second
Picorv 161
Mor1kx 249.7
Sodor1 139
Sodor3 199.4
Sodor5 272.4
Table 6.1: Fuzzing speed of the RISC-V cores.
6.2.1 Line Coverage of Processors over time
To evaluate the line coverage of the processors, we utilize interesting inputs generated by the fuzzer and run them with the processor binaries, which capture the line coverage of the processor. The resulting graphs depict the gradual increase of line coverage over time as the fuzzer explores new states of the processor. The y-axis of these graphs represents the logarithmic measure of the fuzzing time, while the x-axis determines the percentage increase of the line coverage. However, in the Verilog code of the processor, some code sections may never get exercised due to specific parameter selection or the occurrence of an exceptional interrupt. Hence, although the graphs show a gradual increase in coverage, it is not possible to achieve 100% coverage in a real-time system.
46 CHAPTER 6. RESULTS AND EVALUATION
Time vs Coverage
70
65
60
g
&
g 3 3S
50
a5
40 10° 107% 1073 1072 107! Time (hrs)
Figure 6.1: Time over coverage graph of picorv
Time vs Coverage
78
76
74
g 8 72 8
3 © 70
68
66
64
10° 107% 107? 102 10-2 10° 10! Time (hrs)
Figure 6.2: Time over coverage graph of Sodor1
6.2. PERFORMANCE ANALYSES 47
Time vs Coverage
60
58
g
3 4 2 56 3
54
52
107° 107% 1073 1072 10° 10° 10! Time (hrs)
Figure 6.3: Time over coverage graph of Sodor3
Time vs Coverage
70
65
60
g
855 3 3 3S
50
a5
40
10° 1074 107 1072 107! 10° Time (hrs)
Figure 6.4: Time over coverage graph of Sodor5
48 CHAPTER 6. RESULTS AND EVALUATION
Time vs Coverage
62.5
60.0
= 575
) g
3 8
52.5
50.0
415
1075 10° 107 1072 107! 10° 10* Time (hrs)
Figure 6.5: Time over coverage graph of Mor1kx
Chapter 7
Conclusions
We have developed a processor hardware fuzzer by converting the RISC processor’s hardware into a software model. Our fuzzer injects the equivalent program with millions of valid inputs, which explore different states of the processor. We eliminate the crash overhead of the fuzzer by using a custom mutator with an algorithm that generates constrained instructions. Typically, in hardware fuzzing, when a core creates a non-maskable interrupt, the processor goes into a state where its execution freezes. The core tends to reach this state when it executes an invalid instruction. However, we have constrained our interrupt generation in a way that ensures the hardware never hits this scenario. By using this technique, we drastically reduce the effort required to filter out false positives and actual crashes to the hardware bugs. We also utilize a sparse memory matrix that helps the software fuzzer to fuzz efficiently. Instead of having a large memory in every fuzzing cycle, we only use valid inputs, which reduces the processing time of the fuzzer. This approach makes it easier for the fuzzer to analyze and mutate the inputs. Based on our results, we can conclude that our coverage has increased over a period of 24 hours, and we have explored all the possible scenarios that can be covered using hardware fuzzing at a speed 609x faster than the existing hardware fuzzer.
49
Chapter 8
Summary
In our research, we introduce a new design-agnostic methodology that leverages software fuzzing to verify a multi-core 32-bit RISC-V processor. We convert the CPU’s design into an equivalent software model, allowing us to analyze it using state-of-the-art tools for software analysis. This approach helps us to identify potential vulnerabilities and defects in the hardware design efficiently and accurately. Our hardware fuzzing approach involves using a sparse memory matrix as external memory to store the inputs and state of the core that we encounter during the fuzzing process. This technique has significantly increased the efficiency of our fuzzing process, enabling us to achieve a 609x improvement in the fuzzing rate compared to prevalent hardware fuzzing techniques. To further optimize our process, we constrain the inputs of the fuzzer to provide only valid test scenarios in the form of instructions understood by the respective processor. This approach eliminates the crash overhead of the fuzzer and improves the accuracy of our results. By doing so, we can reduce the time and resources required to analyze potential vulnerabilities in the hardware design.
50
Chapter 9
Future Work
Our hardware fuzzing system for RV32I and 1000 architecture cores is agnostic to the vendor, making it independent of a specific hardware device. This characteristic enables us to extend our research to other architectures, providing a more comprehensive hardware fuzzing system for various processors. We aim to further extend our hardware fuzzing system to support 64-bit processors and incorporate it as a supplement verification technique. The team’s ultimate goal is to develop a system-on-chip (SoC) that includes multiple peripherals and intellectual properties of network protocols connected to the processor to test the behavior of the entire system and identify potential issues. By incorporating the hardware fuzzing system into an SoC, we can ensure that the system operates correctly, thus increasing the system’s applicability and allowing it to be used with a broader range of hardware devices. This would enable us to identify potential issues with hardware devices and mitigate them early in the design and development process. Our hardware fuzzing system aims to be a supplement verification technique, ensuring that hardware devices are thoroughly tested and verified to operate correctly. With the addition of the SoC, our system will provide an even more comprehensive verification technique, testing the entire system’s behavior, and identifying potential issues that may arise.
51
Bibliography
[1] Semiconductor market size & share. https://www.fortunebusinessinsights.com/ semiconductor-market-102365. Accessed: 2023-4-7.
[2] Bijan Alizadeh and Payman Behnam. Formal equivalence verification and debugging techniques with auto-correction mechanism for rtl designs. Microprocessors and Mi- crosystems, 37(8):1108–1121, 2013.
[3] Clair Brown and Greg Linden. Semiconductor capabilities in the us and industrializing asia. In 2008 Industry Studies Conference Paper, 2008.
[4] Wen Chen, Sandip Ray, Jayanta Bhadra, Magdy Abadir, and Li-C Wang. Challenges and trends in modern soc design verification. IEEE Design & Test, 34(5):7–22, 2017.
[5] Mehdi Dehbashi, André Sülflow, and Goerschwin Fey. Automated design debugging in a testbench-based verification environment. Microprocessors and Microsystems, 37(2): 206–217, 2013.
[6] Rolf Drechsler et al. Advanced formal verification, volume 122. Springer, 2004.
[7] Harry Foster. Part 3: The 2022 wilson research group functional verification study. https://blogs.sw.siemens.com/verificationhorizons/2022/10/30/ part-3-the-2022-wilson-research-group-functional-verification-study/, October 2022. Accessed: 2023-4-15.
[8] Osman Hasan and Sofiene Tahar. Formal verification methods. In Encyclopedia of Information Science and Technology, Third Edition, pages 7162–7170. IGI Global, 2015.
52
BIBLIOGRAPHY 53
[9] Roland Höller, Dominic Haselberger, Dominik Ballek, Peter Rössler, Markus Krapfen- bauer, and Martin Linauer. Open-source risc-v processor ip cores for fpgas—overview and evaluation. In 2019 8th Mediterranean Conference on Embedded Computing (MECO), pages 1–6. IEEE, 2019.
[10] Jaewon Hur, Suhwan Song, Dongup Kwon, Eunjin Baek, Jangwoo Kim, and Byoungy- oung Lee. Difuzzrtl: Differential fuzz testing to find cpu bugs. In 2021 IEEE Symposium on Security and Privacy (SP), pages 1286–1303. IEEE, 2021.
[11] Jing-Yang Jou and Chien-Nan Jimmy Liu. Coverage analysis techniques for hdl design validation. Proc. Asia Pacific CHip Design Languages, pages 48–55, 1999.
[12] Arkadiusz Koczor, Łukasz Matoga, Piotr Penkala, and Adam Pawlak. Verification approach based on emulation technology. In 2016 IEEE 19th International Symposium on Design and Diagnostics of Electronic Circuits & Systems (DDECS), pages 1–6. IEEE, 2016.
[13] Caroline Lemieux and Koushik Sen. Fairfuzz: A targeted mutation strategy for increas- ing greybox fuzz testing coverage. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pages 475–485, 2018.
[14] Paul G Maropoulos and Dariusz Ceglarek. Design verification and validation in product lifecycle. CIRP annals, 59(2):740–759, 2010.
[15] Richard Munden. ASIC and FPGA verification: a guide to component modeling. Else- vier, 2004.
[16] Andrew Piziali. Functional verification coverage measurement and analysis. Springer Science & Business Media, 2007.
54 BIBLIOGRAPHY
[17] D. Price. Pentium fdiv flaw-lessons learned. IEEE Micro, 15(2):86–88, 1995. doi: 10.1109/40.372360.
[18] Amer Samarah, Ali Habibi, Sofiene Tahar, and Nawwaf Kharma. Automated coverage directed test generation using a cell-based genetic algorithm. In 2006 IEEE International High Level Design Validation and Test Workshop, pages 19–26. IEEE, 2006.
[19] Hassan Sohofi and Zainalabedin Navabi. Assertion-based verification for system-level designs. In Fifteenth International Symposium on Quality Electronic Design, pages 582–588. IEEE, 2014.
[20] Ayas Kanta Swain and KamalaKanta Mahapatra. Design and verification of wishbone bus interface for system-on-chip integration. In 2010 Annual IEEE India Conference (INDICON), pages 1–4. IEEE, 2010.
[21] Ari Takanen, Jared D Demott, Charles Miller, and Atte Kettunen. Fuzzing for software security testing and quality assurance. Artech House, 2018.
[22] Timothy Trippel, Kang G Shin, Alex Chernyakhovsky, Garret Kelly, Dominic Rizzo, and Matthew Hicks. Fuzzing hardware like software. In 31st USENIX Security Symposium (USENIX Security 22), pages 3237–3254, 2022.
[23] Ray Turner. System-level verification-a comparison of approaches. In Proceedings Tenth IEEE International Workshop on Rapid System Prototyping. Shortening the Path from Specification to Prototype (Cat. No. PR00246), pages 154–159. IEEE, 1999.
[24] Hualong Zhao, Hongshi Sang, Tianxu Zhang, and Yebin Fan. Gemi: A high performance and high flexibility memory interface architecture for complex embedded soc. In 2008 International Conference on Computer Science and Software Engineering, volume 4, pages 62–65. IEEE, 2008.
BIBLIOGRAPHY 55
[25] Xiaogang Zhu, Sheng Wen, Seyit Camtepe, and Yang Xiang. Fuzzing: a survey for roadmap. ACM Computing Surveys (CSUR), 54(11s):1–36, 2022.