Skip to content
STIMSMITH

Directed Testing

Concept WIKI v2 · 6/10/2026

Directed testing is a stimulus technique in which pre-selected, specific data values or sequences are applied to a design under test, rather than randomized values. It is used in hardware/RISC-V verification, custom processor benchmarks, and module-level testing of autonomous driving systems to target known corner cases, validate compliance, and close coverage gaps—most often in combination with constrained-random stimulus.

Directed Testing

Directed testing is a stimulus technique in which a testbench implements tests using specific, pre-selected data values and sequences for the design under test (DUT). It is contrasted with random and constrained-random testing, which generate stimuli with varying degrees of freedom. Directed testing is most often used to target known corner cases, validate compliance with specifications, and close specific coverage gaps, and is typically deployed alongside (not in place of) constrained-random stimulus. [Directed testing definition]

The paradigm appears across multiple verification domains, including hardware/SoC verification, RISC-V processor validation, custom processor benchmarks, and module-level testing of autonomous driving systems.

Stimulus Paradigms

Simulation-based verification distinguishes three stimulus paradigms, all of which SystemVerilog supports: [Paradigm definitions]

  • Directed testing. The testbench implements tests using specific data values. For example, for a memory system, an engineer might select appropriate data values, write them to selected memory locations, and read them back. The acknowledged limitation is that this approach can miss errors at unselected addresses or with unselected data values.
  • Random testing. The testbench generates unconstrained random values, which can find more errors but, unless simulations run for very long periods, may still miss certain problems.
  • Directed-random testing. Constraints control how random the data values are. For example, the engineer can ensure that some memory locations are tested exhaustively and that corner cases (e.g., minimum and maximum addresses) are definitely covered. Functional coverage models can then be used to measure and guide verification so tests adequately exercise the design. [Coverage guidance]

The Doulos tutorial frames directed testing as the traditional simulation approach, with random and directed-random paradigms later added to broaden exploration and to retain precision over corner cases. [Traditional directed testing]

Role in RISC-V Processor Verification

RISC-V verification is complicated by the ISA's modular structure and wide range of optional extensions. Achieving comprehensive coverage typically requires more than one verification or comparison methodology and always requires more than one stimulus technique. Random stimuli are excellent for uncovering unanticipated behaviors, but some areas—privilege-mode transitions, page table walks, and memory protection—may not be fully exercised by random generation alone. Directed suites are used to systematically validate such features and to help close gaps exposed by coverage analysis. [RISC-V need for directed tests]

Complementarity with Constrained-Random Stimulus

The most effective strategy combines constrained-random and directed tests: random stimulus discovers the unexpected, while directed suites guarantee compliance with the specification. Compliance is necessary but not sufficient for verification. [Random-directed complementarity]

STING and Directed Stimulus

STING is a bare-metal, software-driven generator for RISC-V that produces C++-based random streams and ASM-style directed tests, built on a lightweight kernel, libraries, and device drivers. STING includes a programming framework for developing directed tests, and uses stimulus graphs to control the scheduling of both random and directed tests. The generated programs are portable across simulation, emulation, FPGA prototypes, and silicon and are architecturally self-checking, enabling a shift-left methodology in which tests developed during RTL bring-up remain useful in later validation stages and even in silicon. [STING directed stimulus]

In practice, STING has exposed issues such as deadlocks in page-table walks, mishandling of the fence.i instruction, floating-point NaN quirks, and cache coherence conflicts. [STING bug examples]

ImperasTS Directed Suites

While STING uncovers unexpected behaviors, the ImperasTS family of directed suites systematically addresses gaps revealed in coverage analysis. Together, STING provides discovery and ImperasTS provides targeted closure. The family includes: [ImperasTS scope]

  • TS-ISA: Architectural validation tests, similar to compliance suites, included with ImperasDV licences.
  • TS-VECT: Targeted suites for vector extensions.
  • TS-MMU / PMP / ePMP: Directed suites for virtual memory and memory protection features.

These directed suites are configured to match the user's RISC-V processor and target areas where random stimulus often leaves gaps. For example, after coverage analysis revealed weak points in Sv39 and Sv48 page table walks, adding TS-MMU tests exposed a subtle ordering issue in TLB flush logic. [ImperasTS bug example]

Use in Custom Processor Benchmarks

Directed (handwritten) tests are commonly used as the basis for custom processor benchmarks. In a large-scale RISC-V verification study, a custom benchmark of 40 programs was executed across multiple cores. The benchmark included manually written tests for each RV32I instruction, plus corner cases such as stores followed by loads and instruction combinations that trigger forwarding structures. The simulation infrastructure was deliberately format-agnostic and "can handle from handwritten to randomly generated tests." Such custom benchmarks leverage the precision of directed stimulus to target instruction-level behavior and architectural corner cases that random generation might not reach. [Custom benchmark with directed tests]

Module-Directed Testing in Autonomous Driving Systems

The directed-testing paradigm has also been applied at the module level in autonomous driving system (ADS) testing. MoDitector (2025) introduces the first root-cause-aware testing method for ADS, with a module-directed testing strategy that includes module-specific feedback, adaptive seed selection, and mutation. Unlike black-box approaches that only identify system-level failures such as collisions, MoDitector designs module-specific oracles for the perception, prediction, planning, and control modules, then guides the generation of test scenarios that provoke failures in a given target module. The goal is to pinpoint which specific module is responsible for a failure in order to support effective debugging and repair. [Module-directed testing in ADS]

Related Technique: Coverage-Directed Test Selection

A related but distinct approach is coverage-directed test selection (CDG), which uses supervised learning from coverage feedback to bias selection toward tests with a high probability of increasing functional coverage. In simulation-based hardware verification, where constrained-random tests are abundant but most have little effect on functional coverage, CDG can reduce manual constraint writing, prioritize effective tests, lower verification resource consumption, and accelerate coverage closure on large industrial designs. CDG differs from directed testing in that it does not hand-author test content; instead, it learns to rank randomly generated tests by their likely coverage impact. [Coverage-directed test selection]

Practical Limitations

Directed testing is widely used, but the evidence identifies several limitations:

  • Coverage of unanticipated behaviors. Because directed tests are written from the engineer's prior understanding of the design, they may miss unanticipated failure modes, addresses, or data values. [Doulos directed limitation]
  • Less effective than reference-model comparison. A large-scale RISC-V verification study states that some methods "rely on directed testing, which is generally less effective than reference model comparison verification methods." [SSCAD critique of directed testing]
  • Not a replacement for random stimulus. The cited methodology treats directed testing as one part of a combined strategy for coverage closure and bug prevention; directed tests provide precision but cannot anticipate all subtle corner cases. [Hybrid methodology]

Summary

Directed testing is a foundational stimulus technique used to validate specific design behaviors, close coverage gaps, and demonstrate compliance. In practice, it is most effective when combined with constrained-random stimulus for breadth and reference-model comparison for root-cause detection, and it has been extended to module-level testing in domains such as autonomous driving.

LINKED ENTITIES

1 links
Custom Benchmark USES

CITATIONS

13 sources
13 citations
[1] Directed testing implements tests using specific, pre-selected data values and sequences, contrasted with random and constrained-random paradigms, and may miss errors at unselected addresses or with unselected data values. Testbench Automation and Constraints Tutorial - Doulos
[2] SystemVerilog supports directed, random, and directed-random testing, and functional coverage models can be used to measure and guide verification so tests adequately exercise the design. Testbench Automation and Constraints Tutorial - Doulos
[3] RISC-V verification requires more than one stimulus technique; random stimuli are useful for breadth, but features such as privilege-mode transitions, page table walks, and memory protection may not be fully exercised by random generation alone, motivating directed suites. RISC-V Test Generation: Random, Directed, and Coverage
[4] The most effective strategy combines constrained-random and directed tests: random stimulus discovers the unexpected while directed suites guarantee compliance, and compliance is necessary but not sufficient for verification. RISC-V Test Generation: Random, Directed, and Coverage
[5] STING is a bare-metal, software-driven generator for RISC-V that produces C++-based random streams and ASM-style directed tests, includes a programming framework for developing directed tests, uses stimulus graphs to schedule random and directed tests, and generates programs portable across simulation, emulation, FPGA prototypes, and silicon that are architecturally self-checking. RISC-V Test Generation: Random, Directed, and Coverage
[6] STING has exposed real bugs including deadlocks in page-table walks, mishandling of fence.i, floating-point NaN quirks, and cache coherence conflicts. RISC-V Test Generation: Random, Directed, and Coverage
[7] The ImperasTS directed suite family includes TS-ISA (architectural validation, similar to compliance), TS-VECT (vector extensions), and TS-MMU/PMP/ePMP (virtual memory and memory protection), configured to match the user's RISC-V processor and targeting areas where random stimulus leaves gaps. RISC-V Test Generation: Random, Directed, and Coverage
[8] After coverage analysis revealed weak points in Sv39 and Sv48 page table walks, adding TS-MMU tests exposed a subtle ordering issue in TLB flush logic. RISC-V Test Generation: Random, Directed, and Coverage
[9] A custom benchmark of 40 programs used in large-scale RISC-V verification included manually written (directed) tests for each RV32I instruction and corner cases such as stores followed by loads and instruction combinations triggering forwarding structures; the simulation infrastructure was format-agnostic and accepted both handwritten and randomly generated tests. Large-Scale RISC-V Processor Verification Using Automated ...
[10] Some verification methods rely on directed testing, which is generally less effective than reference model comparison verification methods. Large-Scale RISC-V Processor Verification Using Automated ...
[11] MoDitector introduces a module-directed testing strategy for autonomous driving systems that combines module-specific oracles with module-specific feedback, adaptive seed selection, and mutation to provoke failures in a target module (perception, prediction, planning, or control) and identify the responsible module. MoDitector: Module-Directed Testing for Autonomous Driving Systems
[12] Coverage-directed test selection uses supervised learning from coverage feedback to bias test selection toward stimuli with a high probability of increasing functional coverage, reducing manual constraint writing and accelerating coverage closure on large industrial designs. Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification
[13] Directed testing provides structure and targeted validation but cannot anticipate all subtle corner cases, so the cited methodology treats it as one part of a combined strategy rather than a replacement for random or constrained-random stimulus. RISC-V Test Generation: Random, Directed, and Coverage

VERSION HISTORY

v2 · 6/10/2026 · minimax/minimax-m3 (current)
v1 · 5/25/2026 · gpt-5.5