Skip to content
STIMSMITH

Directed-Random Test Sequence Generation

Concept WIKI v1 · 5/27/2026

Directed-random test sequence generation is a processor-verification approach that combines randomized instruction sequences with directed mechanisms such as templates, constraints, assertions, and shrinking to reach architecturally interesting states and reduce failing cases.

Overview

Directed-random test sequence generation combines random stimulus with mechanisms that steer generated instruction sequences toward useful architectural states. In the RISC-V CPU testing context described by TestRIG, generators can construct addresses within a defined memory range and emit random loads and stores; a shortened counterexample for a processor bug was reduced to three overlapping memory operations: two loads with a store between them. [Directed-random memory stimulus]

The approach can also be template- or model-directed. IBM's Genesys-Pro is described as being built on templates that intelligently solve for desired deep states. Related CHERI work generated tests from a formal CHERI-MIPS ISA model, compiled from L3 to HOL4, and used constraint solving to automatically generate instruction sequences that reach a desired state without triggering undefined behavior. The same general approach was also applied to the CHERI ARM Morello instruction set starting from a Sail model. [Template and constraint-directed generation]

Counterexamples and shrinking

A key practical feature is counterexample reduction. TestRIG's QCVEngine uses QuickCheck list shrinking after a counterexample is found, removing instructions and retesting to eliminate irrelevant instructions. It also adds smart shrinking functions that transform sequences, such as propagating an instruction's output register into later input operands, enabling further reduction. [Smart shrinking]

Sequences may also be annotated as non-shrinkable when setup must be preserved. One cited use was initialization to avoid trivial divergences from uninitialized floating-point registers, allowing testing to progress to more interesting exception-condition and rounding-mode divergences. [Non-shrinkable setup]

Assertions

Generated sequences can include assertions, such as checking that the value written by a previous instruction was non-zero. These assertions allow a test to fail without a tandem-verification divergence and have been used to test limits of implementation-defined behavior. [Assertion-based failures]

Role in development

The TestRIG paper characterizes model-based randomized testing as enabling counterexample-driven development. In contrast to conventional processor test-driven development, which requires a basic working design before architectural unit tests can be useful, counterexample-driven development can provide reduced stimulus from basic features through advanced interactions. [Counterexample-driven development]

Related tools

  • Genesys-Pro: described as a template-based system for intelligently solving for desired deep states. [Template and constraint-directed generation]
  • RISCV-DV: identified as a RISC-V generator used as a comparison point in a coverage study against TestRIG QCVEngine and the RISC-V test suite. [Coverage comparison]

LINKED ENTITIES

2 links
riscv-dv IMPLEMENTS
Genesys-Pro IMPLEMENTS

CITATIONS

7 sources
7 citations
[1] Directed-random memory stimulus Randomized Testing of RISC-V CPUs using Direct
[2] Template and constraint-directed generation Randomized Testing of RISC-V CPUs using Direct
[3] Smart shrinking Randomized Testing of RISC-V CPUs using Direct
[4] Non-shrinkable setup Randomized Testing of RISC-V CPUs using Direct
[5] Assertion-based failures Randomized Testing of RISC-V CPUs using Direct
[6] Counterexample-driven development Randomized Testing of RISC-V CPUs using Direct
[7] Coverage comparison Randomized Testing of RISC-V CPUs using Direct