Skip to content
STIMSMITH

Regression Testing

Concept WIKI v1 · 5/27/2026

Regression testing, in the provided TestRIG context, is represented as a library of saved instruction-trace counterexamples that can be replayed to quickly check previously observed failures. TestRIG’s trace import/export, shrinking, and simplification mechanisms support producing maintainable regression tests from discovered counterexamples.

Overview

In the TestRIG evidence, regression testing is described operationally as collecting a library of instruction-trace tests that can be used to quickly check all previous counterexamples. These regression tests are derived from instruction traces rather than hand-written assertion tests, and the evidence states that they do not require maintenance in the way hand-written tests with assertions do.

Test artifact format

TestRIG instruction traces can be converted to and from a human-readable format. The same mechanism is used for terminal reporting and for reading and writing trace files, either individually or in bulk from a directory. This import/export capability is what enables the collection of a regression-test library based on prior counterexamples.

Counterexample reduction before regression use

The evidence connects regression-test creation with TestRIG’s counterexample shrinking and simplification workflow. In model-based testing, shrinking or simplification is described as safe to try because any changed trace that still diverges is kept. Simplification can remove esoteric instructions that distract from the root cause; for example, a trapping memory operation may be simplified to an ecall, an instruction that only traps, to make the error more obvious.

TestRIG also uses smart shrinking after QCVEngine finds a counterexample. QuickCheck’s list-shrinking removes irrelevant sequences and retests, while additional smart shrinking can transform a sequence, such as propagating an instruction’s output register into future input operands, allowing further reduction. This can turn a larger counterexample into a shorter trace that still exposes the bug.

Practical role

Within this evidence, regression testing serves as a fast re-check mechanism for failures already discovered during randomized, model-based CPU testing. The regression suite preserves previous counterexamples as replayable traces, while shrinking and simplification make those traces easier to inspect and less dependent on unrelated instructions.

LINKED ENTITIES

1 links
TestRIG USES

CITATIONS

5 sources
5 citations
[1] Regression tests are collected as a library of instruction traces used to quickly check previous counterexamples. Randomized Testing of RISC-V CPUs using Direct
[2] Instruction traces can be converted to and from a human-readable format and read or written individually or in bulk from a directory. Randomized Testing of RISC-V CPUs using Direct
[3] Trace-based regression tests are contrasted with hand-written tests with assertions and are described as not requiring maintenance. Randomized Testing of RISC-V CPUs using Direct
[4] In model-based testing, shrinking or simplification is safe to try because any changed trace that still diverges is kept. Randomized Testing of RISC-V CPUs using Direct
[5] Smart shrinking can eliminate irrelevant instructions and transform a counterexample sequence, including propagating output registers to future input operands. Randomized Testing of RISC-V CPUs using Direct