Directed Test Generation
ConceptDirected test generation is the practice of producing stimulus that intentionally targets specific scenarios, architectural features, or known regression cases, in contrast to random or constrained-random stimulus that explores a broad state space. Evidence from the Coverage-driven Functional Test Generation paper (Koo and Mishra) frames directed test generation as one of three test-generation categories (random, directed, directed-random) and motivates automated directed test generation using formal methods such as model checking and SAT-based bounded model checking, where functional faults expressed as negated temporal properties drive test synthesis for pipelined processors including MIPS. An industrial case study (Koo, Mishra, Bhadra, Abadir, MTV'06) extends the approach to an industrial Power Architecture processor, introducing decompositional model checking, time-step-based property decomposition, and an algorithm for merging partial counterexamples into a global one. Complementary evidence from the CARRV 2022 RISC-V architectural-testing slides extends the concept to ISA-conformance / architectural-compliance testing, where RISCV-CTG generates controlled, directed tests from coverpoints with data-propagation analysis to ensure signatures are influenced by the tests. Additional stimulus-engine evidence (PPIGen/SGen) shows a practical hybrid flow in which hand-written directed tests dominate the regression suite and are supplemented by randomized exercisers.
First seen 5/26/2026
Last seen 7/3/2026
Evidence 15 chunks
Wiki v5
WIKI
Directed Test Generation
Overview
Directed test generation is the practice of authoring or producing stimulus that intentionally targets specific scenarios, architectural features, or known regression cases, in contrast to random or constrained-random stimulus that explores a broad state space. In microprocessor and hardware verification it is used alongside random exercisers as part of a hybrid verification strategy, because directed tests provide precise, scenario-specific coverage while random tests provide broad, unexpected exploration. [C1] [C2] [C3]
NEIGHBORHOOD
No graph connections found for this entity yet. It may appear in future ingestion runs.
explore full graph →RELATIONSHIPS
6 connectionsThe hybrid methodology combines directed suites with constrained-random stimulus to achieve coverage closure.
ImperasTS provides directed test suites targeting specific RISC-V features for structured verification.
Directed test generation targets micro-architectural faults
SGen piggybacks onto PPIGen's directed mode to generate tests.
PPIGen supports a directed mode for generating test programs.
Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V ← uses 90% 1e
The paper presents directed test generation as the approach for creating efficient tests.
LINKED ENTITIES
1 linksCITATIONS
26 sources26 citations — click to expand
[1] Functional verification is widely recognized as a major bottleneck in processor design, with pre-silicon logic bugs growing 300-400% per generation in successive Intel IA-32 designs, and functional verification accounting for up to 70% of design development time and resources. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022
[2] Simulation using test programs is the most widely used form of processor verification; formal methods provide completeness only at the component level due to state explosion. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022
[3] Directed tests can reduce overall validation effort because shorter tests can obtain the same coverage goal as random tests, motivating automated directed test generation; hand-written directed tests are laborious and error-prone. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022
[4] The Coverage-driven Functional Test Generation methodology converts negated desired behaviors (functional faults) into temporal-logic properties and uses model checking to produce counterexample test programs for pipeline-interaction coverage. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods
[5] The Coverage-driven flow iteratively generates a list of functional faults, selects one, uses a model checker or SAT solver to produce a test program for it, runs fault simulation to find all detected faults, removes them, and repeats until all faults are detected. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods
[6] Bounded model checking (BMC) combined with SAT solving addresses state explosion in model checking; the methodology applies bound determination per property and decomposition of the formal processor model and properties. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods
[7] The methodology was successfully applied to MIPS processors. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods
[8] PPIGen supports a directed mode that takes a C/assembly source plus a config file and produces initialization code and linker scripts, with config registers at reset values unless overridden; it also supports a random/exer mode driven by a single config file. PPIGen / SGen C++11 stimulus-engine evidence
[9] SGen piggybacks onto PPIGen's directed mode, producing an assembly program and config file that PPIGen's directed flow processes; SGen builds directed-style tests from inst/seq/tests libraries with C++11 lambda-based sequences and weighted_set randomization. PPIGen / SGen C++11 stimulus-engine evidence
[10] RISC-V architectural testing objectives: ensuring ISA implementation meets all requirements under all conditions; RISC-V is micro-architecture agnostic and extremely configurable, making an exhaustive suite impossible but limited-area suites feasible; AT is a subset of design verification. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022
[11] Challenges for RISC-V architectural test suites: standardized test format with signature-based comparison, an ISA coverage specification format, a tool to measure coverage, and a tool to generate efficient, directed tests as the ISA spec and testing scopes grow. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022
[12] RISCV-CTG generates controlled and directed tests from coverpoints, with low barriers to entry (Python + RISC-V), data-propagation analysis to ensure signatures are influenced by the tests, and open source availability; no negative testing is performed. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022
[13] PPIGen directed vs. random mode (config registers at reset unless overridden; randomized in random mode). PPIGen stimulus-engine evidence
[14] SGen produces assembly+config and feeds it through PPIGen's directed flow; sequences composed from inst/seq/tests libraries with randutils. SGen stimulus-engine evidence
[15] SGen heavily uses C++11 features: lambda functions, improved <random> distributions/generators, std::function, regular expressions, and the auto specifier. SGen C++11 stimulus-engine evidence
[16] A typical SGen bare_sequence registers instruction objects with a weighted_set so that, when fed through PPIGen's directed flow, the resulting test mixes targeted instruction selection with controlled randomization. SGen C++11 stimulus-engine evidence
[17] In RTL regression practice, suites are composed mostly of directed tests with a small selection of PPIGen and SGen exercisers; SGen catches RTL bugs more consistently than PPIGen; directed failures alone do not necessarily block releases. RTL regression evidence
[18] Over a one-month period, 50,000 total exerciser runs (25,000 per generator) produced 1,731 total failures (171 from PPIGen, 1,560 from SGen). Exerciser-scale evidence
[19] CLOTHO is a directed-test-generation framework combining a static analyzer and a model checker to detect serializability violations in SQL database-backed Java applications and translate them into concrete test inputs. CLOTHO database directed test generation
[20] Coverage Directed Test Generation (CDG) augmented with an LLM as a 'Verilog Reader' outperforms random testing on designs within the LLM's comprehension scope. LLM-aided hardware CDG
[21] Modern processor performance improvements come with high design complexity from deeply pipelined superscalar designs, dynamic scheduling, and dynamic speculation; verification complexity is directly proportional to design complexity. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study
[22] Industrial practice typically uses random and biased-random ISA-level test generation for simulation-based validation, but ISA-level techniques have difficulty activating micro-architectural artifacts and pipeline functionalities because pipeline interactions and timing details cannot be derived from an ISA-only specification. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study
[23] The MTV'06 methodology takes an architectural specification containing both micro-architectural structure and ISA behavior, with micro-architectural features including pipelined and clock-accurate behaviors; properties are automatically generated from a pipeline (graph) coverage functional fault model and additional properties can be added for combined pipeline stage rules and corner cases. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study
[24] For automatic test generation, the MTV'06 methodology uses decompositional model checking with both the processor model and properties decomposed; the notion of time steps enables property decomposition based on clock cycles, and an algorithm merges partial counterexamples from decomposed properties into a global counterexample for the target property. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study
[25] Prior approaches have difficulties with industrial processors due to high complexity of even a single pipeline path and mainly focus on the data path; data (opcode/operands) sits in a single stage while control signals (functional unit/buffer status) may spread across multiple stages, making model partitioning and counterexample merging harder and motivating improvements in decomposition and merging. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study
[26] Experiments on a Power Architecture-based industrial processor showed promising results in test generation time and test program length. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study