Skip to content
STIMSMITH

Test Case Shrinking

Concept WIKI v1 · 5/27/2026

Test Case Shrinking is the automated simplification of failing verification inputs into smaller counterexamples. In the provided evidence, it is described in the context of TestRIG and RVFI-DII, where interactive verification and Direct Instruction Injection make shrinking possible, including for instruction sequences with branches.

Overview

Test Case Shrinking is the reduction or simplification of a failing test input into a smaller form that still demonstrates the same verification failure. In the TestRIG context, the evidence describes this as automated simplification and shrinking enabled by interactive verification through RVFI-DII.

Role in TestRIG

TestRIG extends the RISC-V Formal Interface (RVFI) with Direct Instruction Injection (DII). The evidence distinguishes the two directions of the interface: DII is used for instruction input, while RVFI is used for trace output. Together, RVFI-DII supports full interactive verification, and that interactive mode enables automated simplification and shrinking.

This matters because a verification engine can inject candidate instruction streams and observe corresponding architectural traces. When a divergence or failure is found, the same interactive mechanism can be used to simplify the input sequence while preserving the failure condition.

Shrinking instruction sequences with branches

The evidence specifically notes that instruction injection allows straightforward shrinking of sequences with branches. Branches are often problematic for reducing generated programs because control flow can make it difficult to preserve the behavior that triggers a bug. In the TestRIG/RVFI-DII design, instructions are injected directly, making it easier to control and reduce the executed sequence.

Relationship to minimal tests

The evidence also identifies Symbolic QED as another approach that generates minimal tests for verification, including post-silicon verification, using a formal pipeline model. This places Test Case Shrinking within a broader family of techniques aimed at producing small, useful counterexamples or tests for debugging and validation.

Implementation context

RVFI-DII requires synchronization between injected instructions and trace entries. The evidence notes that canceled instructions create a challenge because RVFI-DII requires a single RVFI trace entry for each injected DII instruction. A mature design described in the evidence attaches a sequence ID to each RVFI instruction and carries it through the pipeline, allowing redirects to work naturally.

Practical impact

The evidence reports that instruction injection and its support for shrinking helped the authors replace instruction-level unit tests for the CHERI extension, improving productivity and assurance and enabling more efficient extension of simulators and processors.

LINKED ENTITIES

5 links
TestRIG USES
QCVEngine USES
Direct Instruction Injection IMPLEMENTS
C-Reduce IMPLEMENTS
PyH2P IMPLEMENTS

CITATIONS

7 sources
7 citations
[1] RVFI-DII supports full interactive verification, and interactive verification enables automated simplification and shrinking. Randomized Testing of RISC-V CPUs using Direct
[2] Direct Instruction Injection is for instruction input, while RVFI is for trace output. Randomized Testing of RISC-V CPUs using Direct
[3] Instruction injection allows straightforward shrinking of instruction sequences with branches. Randomized Testing of RISC-V CPUs using Direct
[4] Symbolic QED is described as another approach that generates minimal tests for verification, including post-silicon verification, using a formal model of the pipeline. Randomized Testing of RISC-V CPUs using Direct
[5] Canceled instructions are a challenge for DII because RVFI-DII requires one RVFI trace entry for each injected DII instruction. Randomized Testing of RISC-V CPUs using Direct
[6] A described RVFI-DII design attaches a sequence ID to each RVFI instruction and carries it through the pipeline so redirects can work naturally. Randomized Testing of RISC-V CPUs using Direct
[7] Instruction injection and related shrinking support helped replace instruction-level unit tests for the CHERI extension, improving productivity and assurance and enabling more efficient extension of simulators and processors. Randomized Testing of RISC-V CPUs using Direct