RISC-V Processor Verification with Coverage-guided Aging*

Niklas Bruns1, Vladimir Herdt1,2, Eyck Jentzsch3, Rolf Drechsler1,2
1Institute of Computer Science, University of Bremen, Germany
2Cyber-Physical Systems, DFKI GmbH, Bremen, Germany
3MINRES Technologies GmbH, 85579 Neubiberg, Germany
nbruns@uni-bremen.de, vherdt@uni-bremen.de, eyck@minres.com, drechsler@uni-bremen.de

Abstract

In this extended abstract we present an efficient approach for processor verification at the Register-Transfer Level (RTL), using a cross-level setting with an Instruction Set Simulator (ISS) as a reference model. We leverage a custom instruction stream generator tailored for RISC-V, which produces one endless instruction stream at runtime. Moreover, we employ a coverage-guided aging concept which ensures a more uniform distribution of the generated instructions by tracking and updating coverage information in the ISS and dynamically providing feedback to the instruction stream generator. Our case study with an industrial pipelined 32 bit RISC-V processor demonstrates the effectiveness of our approach.

1 Extended Abstract RISC-V instruction sets are supported by selecting and combining the respective constraint-based specifications. RISC-V [10, 11] is a free and open-source ISA that en- Execution results between the ISS and RTL processor core ables a royalty-free processor design and implementation. are compared through execution log files. While this fea- It is designed in a very modular way with optional standard ture set makes RISC-V DV very powerful in general, it also instruction set extensions around a mandatory base integer has some major weaknesses. In order to keep the frame- instruction set and the ability to integrate additional custom work generic, the generated tests use a restricted instruc- instruction sets to build highly application-specific proces- tion set to avoid problems with infinite loops and platform- sors. These properties made RISC-V very popular in in- dependent memory access operations. Moreover, by gener- dustry and academia. From the verification perspective, ating tests one by one, only comparatively short instruction however, the extensive modularity adds additional com- sequences are considered, and the state of the processor un- plexity. Besides the modern features provided by RISC-V der test is regularly reset for each new test execution. Fur- and any micro-architectural specific optimizations of the thermore, the co-simulation has an inherent performance processor, such as pipelining and branch prediction, the overhead due to the extensive filesystem communication, verification tools also need to be able to deal with the large since each RISC-V assembly test needs to be compiled, configuration space offered by RISC-V. loaded onto the respective simulator, and produce a log Recently, approaches specifically tailored for RISC-V ver- file for comparison. Finally, the test generator is not de- ification have emerged. The baseline is provided by the signed to be dynamically guided by coverage information official RISC-V unit and compliance tests [2, 1], which are obtained from the test execution progress. Many of these directed test suites. More comprehensive simulation-based issues have been addressed by a recent academic work [7]. testing is enabled by approaches that rely on continuous as- It generates endless instruction streams and integrates the sembly test generation. Different approaches have been de- ISS with the RTL core in a very efficient co-simulation vised that generate tests by integrating randomized instruc- compiled into a single binary with in-memory communi- tion templates [3], leverage constraint-based specification cation. The setup allows to generate instructions without mechanism to define coverage requirements [6], utilize any restrictions, i.e., arbitrary combinations of load/store coverage-guided fuzzing techniques to generate tests [8, 5], and Control and Status Registers (CSRs)1 instructions, as and use symbolic execution techniques to find specific in- well as infinite loops, are supported, which enables a very puts [9]. A particularly promising approach in this con- comprehensive test approach. However, the approach is text is pursued by Google’s open-source RISC-V Design still limited as it does not collect or employ runtime cover- Verification (DV) framework. It leverages a method based age information to assess and guide the test generation pro- on co-simulation, i.e. it employs an Instruction Set Sim- cess. Instead, the instruction stream generators are based ulator (ISS) as a functional reference model for the RTL on a simple randomized test strategy which makes it very processor under test. As a backbone it applies constraint- difficult to continuously achieve a broad and deep test cov- based specification techniques in SystemVerilog to gener- erage in endless instruction streams. ate RISC-V assembly tests one after another. Different In this extended abstract, we summarize our work from [4], *This work was supported in part by the German Federal Ministry where we proposed a novel approach for cross-level ver- of Education and Research (BMBF) within the project Scale4Edge under 1 contract no. 16ME0127 and no. 16ME0135, and within the project VerSys In the CSRs, the processor stores additional instruction results to en- under contract no. 01IW19001. able sophisticated hardware/software interactions.

ification that conceptually builds upon the previous aca- [4] N. Bruns, V. Herdt, E. Jentzsch, and R. Drech- demic work [7] and addresses the aforementioned limita- sler. Cross-level processor verification via end- tions. The foundation is a randomized coverage-guided less randomized instruction stream generation with instruction stream generator that produces an endless and coverage-guided aging. In DATE, 2022. accepted. unrestricted instruction stream that evolves dynamically [5] V. Herdt, D. Große, and R. Drechsler. Closing the at runtime based on observed coverage information. We RISC-V compliance gap: Looking from the negative also leverage an ISS as a reference model in a tight co- testing side. In DAC, 2020. simulation setting. Coverage information is continuously [6] V. Herdt, D. Große, and R. Drechsler. Towards spec- updated based on the execution state of the ISS and we em- ification and testing of RISC-V ISA compliance. In ploy the novel concept of coverage-guided aging to smooth DATE, pages 995–998, 2020. out the coverage distribution of the randomized instruc- tion stream over time. Our experiments with the 32-bit [7] V. Herdt, D. Große, E. Jentzsch, and R. Drechsler. pipelined RISC-V core of the MINRES The Good Core Efficient cross-level testing for processor verification: (TGC) series demonstrate the effectiveness of our approach A RISC-V case-study. In FDL, 2020. in achieving a much more regular coverage distribution of [8] V. Herdt, D. Große, H. M. Le, and R. Drechsler. the randomized instruction stream via coverage-guided ag- Verifying instruction set simulators using coverage- ing. For more details please refer to [4]. guided fuzzing. In DATE, pages 360–365, 2019. [9] V. Herdt, S. Tempel, D. Große, and R. Drechsler. 2 References Mutation-based compliance testing for RISC-V. In ASP-DAC, 2021. [1] RISC-V compliance task group. https:// [10] A. Waterman and K. Asanovi´c, editors. The RISC-V github.com/riscv/riscv-compliance. Instruction Set Manual; Volume I: Unprivileged ISA. [2] RISC-V ISA tests. https://github.com/riscv/ 2019. riscv-tests. [11] A. Waterman and K. Asanovi´c, editors. The RISC-V [3] RISC-V torture test generator. https: Instruction Set Manual; Volume II: Privileged Archi- //github.com/ucb-bar/riscv-torture. tecture. 2019.