RISC-V instruction stream generator

A RISC-V instruction stream generator is a randomized, coverage-guided test-generation component tailored to the RISC-V Instruction Set Architecture (ISA). It is designed to produce an endless, unrestricted instruction stream at runtime for use in cross-level processor verification, where an RTL processor core is checked against an Instruction Set Simulator (ISS) used as a functional reference model.

Background and motivation

RISC-V is a free and open-source ISA that enables royalty-free processor designs and is highly modular, with optional standard extensions around a mandatory base integer instruction set and the ability to integrate custom instruction sets. From a verification standpoint, this modularity, combined with micro-architectural features such as pipelining and branch prediction, creates a large configuration space that verification tools must cover.

Several RISC-V-specific verification approaches have emerged:

• Directed test suites provided by the official RISC-V unit and compliance tests.
• Continuous assembly test generation, using randomized instruction templates, constraint-based specification, coverage-guided fuzzing, and symbolic execution techniques.
• Google's open-source RISC-V Design Verification (DV) framework, which uses co-simulation with an ISS as a functional reference model for the RTL processor under test, and constraint-based specification techniques in SystemVerilog to generate RISC-V assembly tests one after another. The DV framework allows generating instructions without arbitrary restrictions and supports arbitrary combinations of load/store and Control and Status Registers (CSR) instructions as well as infinite loops, enabling a very comprehensive test approach.

Despite its breadth, the RISC-V DV framework has notable limitations:

1. It uses a restricted instruction set to avoid problems with infinite loops and platform-dependent memory accesses.
2. Tests are generated one by one, so only comparatively short instruction sequences are considered and the processor state is regularly reset.
3. The co-simulation has performance overhead from filesystem communication (each test must be compiled, loaded, and produce a log file for comparison).
4. The test generator is not dynamically guided by coverage information obtained from the test execution progress.

The randomized coverage-guided instruction stream generator

The instruction stream generator described in the underlying work addresses these limitations by:

• Producing an endless and unrestricted instruction stream that evolves dynamically at runtime based on observed coverage information.
• Generating instructions without restrictions, supporting arbitrary combinations of load/store and CSR instructions as well as infinite loops.
• Operating in a tight co-simulation setting in which an ISS is used as a reference model, with coverage information continuously updated based on the execution state of the ISS.
• Providing feedback from the ISS to the generator so that the stream adapts dynamically as coverage evolves.

Unlike the simple randomized test strategy of earlier generators, this approach leverages runtime coverage information to guide the stream, making it possible to continuously achieve a broad and deep test coverage in endless instruction streams.

Coverage-guided aging

The generator employs the concept of coverage-guided aging to smooth out the coverage distribution of the randomized instruction stream over time. By tracking and updating coverage information in the ISS and dynamically providing feedback to the instruction stream generator, the generator ensures a more uniform distribution of generated instructions across the explored coverage space.

Case study

The effectiveness of the approach was demonstrated on the 32-bit pipelined RISC-V core of the MINRES The Good Core (TGC) series, an industrial RISC-V processor. The experiments showed that the use of coverage-guided aging achieves a much more regular coverage distribution of the randomized instruction stream compared with a plain randomized strategy.

Relation to prior and follow-up work

The generator builds upon prior academic work on efficient cross-level testing for processor verification (a RISC-V case-study) and is described in detail in the DATE 2022 paper Cross-level processor verification via endless randomized instruction stream generation with coverage-guided aging by N. Bruns, V. Herdt, E. Jentzsch, and R. Drechsler. It is summarized in the MBMV 2022 extended abstract RISC-V Processor Verification with Coverage-guided Aging by the same authors together with Eyck Jentzsch of MINRES Technologies.

Limitations

While the generator removes the restrictions of earlier instruction stream generators (such as avoiding infinite loops or platform-dependent memory accesses), it is still bounded by the limits of randomized generation: even with coverage guidance, ensuring systematic coverage of all corner-case behaviors in a complex pipelined core remains an open challenge.