Skip to content
STIMSMITH

Instruction-Level Unit Testing

Concept WIKI v1 · 6/2/2026

Based on the cited TestRIG paper, instruction-level unit testing refers here to testing CPUs with short injected instruction sequences and then minimizing failing cases with shrinking. The evidence specifically supports smart shrinking, non-shrinkable initialization sequences, and embedded assertions as key techniques for isolating interesting instruction-level failures.

Overview

In the cited TestRIG work on randomized testing of RISC-V CPUs using direct instruction injection, instruction-level unit testing is represented by executing instruction sequences directly against an implementation and then analyzing counterexamples at instruction granularity. The paper emphasizes reducing failing sequences to small, interpretable tests rather than keeping long randomized traces.[1]

Core techniques supported by the evidence

Smart shrinking

The paper states that direct instruction injection primarily relies on QuickCheck's built-in shrinking, but augments it with smart shrinking functions that both remove instructions and transform sequences to simplify them.[1] The example given starts from a longer failing sequence for an artificial add bug and shows how shrinking can:

  1. remove irrelevant instructions, and
  2. propagate an instruction's output register to later operands,

so that the counterexample is reduced further to a shorter two-instruction form.[1]

This makes instruction-level tests easier to interpret because the remaining sequence more directly exposes the errant behavior.[1]

Non-shrinkable sequences

The paper also describes non-shrinkable sequence annotations.[2] These are used to preserve required initialization state when shrinking would otherwise remove setup code and produce trivial or unhelpful failures. The example in the paper is an implementation that did not initialize floating-point registers; a non-shrinkable initialization sequence let testing progress past trivial counterexamples to more interesting divergences involving exception conditions and rounding modes.[2]

Assertions inside sequences

Instruction sequences can include assertions, such as checking that the value written by the previous instruction was non-zero.[3] According to the paper, these assertions allow a test to fail without requiring a divergence, and sequences with assertions do not require tandem verification to discover a failure.[3] The authors note that they used such assertions to test limits of implementation-defined behavior.[3]

Why this matters

From the cited evidence, the value of instruction-level unit testing is not only generating instruction sequences, but also turning failures into compact, informative counterexamples. Smart shrinking, preserved setup through non-shrinkable regions, and embedded assertions together support faster diagnosis and bring-up by highlighting the smallest sequence that still demonstrates the problem.[1][2][3]

Relation to TestRIG

The evidence places these techniques in the context of TestRIG's QCVEngine, which the paper compares with the RISC-V test suite and RISCV-DV in a coverage study.[4] Within the provided evidence, TestRIG is therefore a directly relevant comparison point and implementation context for instruction-level unit testing techniques based on direct instruction injection.[4]

[1]: See citation "Smart shrinking removes and transforms instructions to minimize failing sequences." [2]: See citation "Non-shrinkable sequences preserve required initialization during shrinking." [3]: See citation "Assertions allow failures without divergence and without tandem verification." [4]: See citation "TestRIG QCVEngine is the implementation context discussed in the evidence."

LINKED ENTITIES

1 links
TestRIG COMPARES_WITH

CITATIONS

7 sources
7 citations
[1] Smart shrinking removes and transforms instructions to minimize failing sequences. Randomized Testing of RISC-V CPUs using Direct
[2] Smart shrinking can propagate an instruction's output register to future operands, enabling additional reduction of a counterexample. Randomized Testing of RISC-V CPUs using Direct
[3] Non-shrinkable sequences preserve required initialization during shrinking. Randomized Testing of RISC-V CPUs using Direct
[4] A non-shrinkable initialization sequence helped move past trivial counterexamples caused by uninitialized floating-point registers to more interesting divergences in exception conditions and rounding modes. Randomized Testing of RISC-V CPUs using Direct
[5] Assertions allow failures without divergence and without tandem verification. Randomized Testing of RISC-V CPUs using Direct
[6] The paper uses assertions in instruction sequences to test the limits of implementation-defined behavior. Randomized Testing of RISC-V CPUs using Direct
[7] TestRIG QCVEngine is the implementation context discussed in the evidence. Randomized Testing of RISC-V CPUs using Direct