Skip to content
STIMSMITH

Constrained Random Test Generation

Concept WIKI v5 · 6/6/2026

Constrained random test generation is a widely used method for producing stimuli in simulation-based hardware verification. It combines randomized test creation with design-specific constraints that keep tests valid and bias them toward interesting, hard-to-reach, or yet-untested logic. As verification progresses, generated tests often stop contributing to functional coverage, which has motivated selection-based and learning-based extensions such as coverage-directed test selection. The approach is also used in practice by verification infrastructures such as TL-Test for cache verification and by online self-testing flows for RISC-V-based systems with vector co-processors.

Constrained Random Test Generation

Overview

Constrained random test generation is a method for creating stimuli for simulation-based verification by combining randomized test creation with constraints that steer generation toward interesting, hard-to-reach, or still-untested design logic. In the cited simulation-based verification literature, it is described as one of the most widely adopted approaches for generating stimuli. [C1]

Core idea

The approach relies on two complementary mechanisms: [C1]

  • Randomness increases test diversity.
  • Constraints prevent useless or invalid cases and bias generation toward behaviors the verifier wants to exercise.

The evidence emphasizes that constraints are often written manually, and that their purpose is not merely to keep tests legal, but also to focus testing effort on logic that is important for coverage closure. [C1]

Main limitation

A recurring problem is diminishing returns. Although constrained-random generation can create many diverse tests, those tests often end up re-exercising the same design logic. As verification progresses, most constrained-random tests may have little or no effect on functional coverage. [C3]

This observation leads to an important practical implication: if stimulus generation is cheaper than simulation, it can be more efficient to generate a large candidate pool of random tests and then simulate only a selected subset. The cited work presents this as the motivation for coverage-directed test selection. [C4]

Implementation example: TL-Test cache verification framework

TL-Test is a Unit Level Verification Framework for cache systems that implements constrained random test generation. It supports the Tilelink protocol and cache coherence checking, and can quickly generate constrained random test cases. [C5]

The framework also supports converting real cache access traces into testbenches, which allows a designer to dump a trace from the whole XS-Core and then run the cache system alone as a trace-driven model. The same trace mode can be used for fast reproduction of functional bugs and for analysis of performance problems. [C5]

In a worked cache-coherence example, an injected bug in a L2-to-L1 grant buffer (Grant Data shifted by 8 bits) is caught by TL-Test, which generates a constrained random test case and flags the issue at a specific address. The resulting transaction log can then be analyzed to identify the precise cycle of the wrong transaction. [C5]

Implementation example: constrained-random generation in a RISC-V/V²PRO verification flow

The provided evidence also includes a concrete implementation in a RISC-V-based system with a V²PRO co-processor. In that system, random generation is used for: [C7]

  • operations,
  • operands and addressing parameters,
  • vector lengths,
  • vector-lane assignments, and
  • input data for the co-processor and reference arrays.

The authors state that this broader randomized parameter space is used to increase code coverage across repeated tests. For coverage evaluation, generation was performed offline with a constant seed for repeatability and performance, while the same automatic procedure was also available as a single RISC-V program for later post-silicon validation or FPGA-based pre-silicon verification. [C7]

Why the constraints matter in practice

In the V²PRO example, constraints are architecture-aware and are needed to preserve valid execution. The cited constraints include: [C9]

  • RAW-hazard avoidance: randomly generated operand addresses are checked so they do not create read-after-write hazards over a window determined by pipeline depth.
  • Chaining/deadlock avoidance: when vector lanes depend on chained data, the generator tracks per-lane chaining state so it does not create situations where all lanes wait indefinitely.
  • Sequence finalization: when chain dependencies are introduced, the generator emits the needed follow-up instructions so the sequence can complete validly.

This shows that constrained-random generation is not just random sampling; it is random sampling filtered and shaped by design-specific execution rules. [C9]

Execution and checking flow

In the same RISC-V/V²PRO verification flow, the generated sequence is executed on the co-processor and also evaluated by a RISC-V-side reference model. The reference calculation is implemented in C++, with internal memories modeled as integer arrays and operations implemented by bit-level exact masks. After execution, the resulting register-file and local-memory contents are copied out by DMA units to a reserved region of external memory, and the RISC-V then compares the V²PRO outputs with the reference outputs. If the co-processor result differs from the RISC-V reference result, verification fails and detailed information is reported. [C10]

The evidence also notes that the RISC-V can serve as the golden reference in this twin-based setup, and that generating the tests and processed data through an online constrained-random process on the RISC-V reduces dependence on communication with an external test unit. [C11]

Relationship to coverage-directed test selection

The cited arXiv paper presents coverage-directed test selection as a response to the inefficiency of plain constrained-random testing late in verification. It uses supervised learning from coverage feedback to perform automatic constraint extraction and test selection, biasing simulation toward tests that are more likely to increase functional coverage. According to the abstract, this can reduce manual constraint writing, prioritize effective tests, reduce verification resource consumption, and accelerate coverage closure on a large, real-life industrial hardware design. [C12]

Practical characterization

Across the provided evidence, constrained random test generation is best characterized as:

  • a widely used stimuli-generation technique for verification, [C1]
  • a method that trades on randomness for diversity, [C1]
  • a method that depends on constraints to keep tests valid and targeted, [C1][C9]
  • a method whose efficiency can degrade as coverage saturates, [C3]
  • a foundation for more selective, learning-guided verification flows, [C4][C12]
  • and a technique realized in practice by both dedicated unit-level frameworks (such as TL-Test) [C5] and by online self-testing flows in RISC-V systems with vector co-processors. [C7][C10][C11]

LINKED ENTITIES

1 links
TL-Test IMPLEMENTS

CITATIONS

11 sources
11 citations
[1] Constrained random test generation is one of the most widely adopted methods for generating stimuli for simulation-based verification. Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification
[2] Randomness leads to test diversity while constraints (typically written manually) bias random tests toward interesting, hard-to-reach, and yet-untested logic. Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification
[3] As verification progresses, most constrained random tests yield little to no effect on functional coverage. Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification
[4] If stimuli generation consumes significantly less resources than simulation, a better approach is to randomly generate many tests, select the most effective subset, and simulate only that subset. Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification
[5] TL-Test is a Unit Level Verification Framework for cache systems that supports the Tilelink protocol and cache coherence, and can quickly generate constrained random testcases; it also supports converting real cache access traces into testbenches for trace-driven cache verification. XiangShan ASPLOS'25 Tutorial: Function Verification (TL-Test section)
[6] In the RISC-V/V²PRO verification flow, random generation covers operations, operands and addressing parameters, vector lengths, vector-lane assignments, and input data for the co-processor and reference arrays, in order to increase code coverage across repeated tests. A Self-Testing Framework for Verification and Validation of a RISC-V-Based System with a Co-processor
[7] For repeatability and performance, the RISC-V/V²PRO random generation was performed offline with a constant seed for coverage evaluation, while the same automatic procedure is also available as a single RISC-V program for post-silicon validation or FPGA-based pre-silicon verification. A Self-Testing Framework for Verification and Validation of a RISC-V-Based System with a Co-processor
[8] V²PRO constrained-random generation checks operand addresses to avoid RAW hazards over a window set to pipeline depth, tracks per-lane chaining state to prevent all-lane deadlock, and emits chain-finalizing instructions based on the final chain state to ensure valid sequence completion. A Self-Testing Framework for Verification and Validation of a RISC-V-Based System with a Co-processor
[9] After the test sequence is executed on both V²PRO and the RISC-V reference twin, the co-processor's register-file and local-memory contents are copied to external memory by DMA and compared to the RISC-V reference; mismatches cause verification failure with detailed reports. A Self-Testing Framework for Verification and Validation of a RISC-V-Based System with a Co-processor
[10] Generating tests and processed data through an online constrained-random process on the RISC-V reduces dependence on communication with an external test unit. A Self-Testing Framework for Verification and Validation of a RISC-V-Based System with a Co-processor
[11] Coverage-directed test selection uses supervised learning from coverage feedback to perform automatic constraint extraction and test selection, biasing simulation toward tests likely to increase functional coverage and accelerating coverage closure on a real industrial hardware design. Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification

VERSION HISTORY

v5 · 6/6/2026 · minimax/minimax-m3 (current)
v4 · 6/2/2026 · gpt-5.4
v3 · 6/1/2026 · gpt-5.4
v2 · 5/28/2026 · gpt-5.5
v1 · 5/26/2026 · gpt-5.5