Overview
Smart Shrinking is a technique for simplifying failing instruction sequences after a counterexample has been found during randomized RISC-V CPU testing. In the TestRIG context, a counterexample is found by QCVEngine, after which QuickCheck's built-in list-shrinking can remove instructions and retest the shorter sequence. Smart Shrinking extends this basic approach by not only deleting instructions, but also transforming instructions to simplify the trace while preserving the failing behavior.
How it works
The baseline shrinking step uses QuickCheck's list-shrinking strategy: it removes subsequences from the instruction list and reruns the test, hoping to discard instructions that are irrelevant to the observed failure.
Smart Shrinking adds domain-specific transformations on top of that baseline. One described transformation propagates an instruction's output register into later input operands. This can expose additional opportunities for ordinary list shrinking to remove now-redundant move-like instructions.
For example, for an artificial hardware bug where the least-significant bit of add results is stuck at zero, an initial failing sequence includes instructions that generate values, move a value through another register, include an irrelevant xori, and finally execute the buggy add:
addi x7, x4, 123 # Generate odd immediate
addi x5, x3, 42 # Generate even immediate
addi x6, x7, 0 # Move x7 to x6
xori x1, x5, 745 # Irrelevant
add x1, x5, x6 # Perform buggy add
Built-in list shrinking can remove the irrelevant xori and one other instruction, leaving:
addi x7, x4, 123 # Generate odd immediate
addi x6, x7, 0 # Move x7 to x6
add x1, x5, x6 # Perform buggy add
Smart Shrinking can then propagate x7 directly into the final add, making the move unnecessary and allowing another list-shrinking pass to reduce the counterexample further:
addi x7, x4, 123 # Generate odd immediate
add x1, x5, x7 # Perform buggy add
Simplification library
The technique also includes a library of simplifications used during shrinking. These simplifications are intended to remove esoteric instructions that perform mundane functions and distract from the root cause of the failure. The stated goal is to produce smaller and clearer counterexamples that highlight the relevant errant behavior.
Relationship to test-case shrinking
Smart Shrinking extends general test-case shrinking. Instead of relying only on structural deletion of list elements, it adds instruction-aware rewrites that can make additional deletions possible and produce a more diagnostic failing sequence.