Directed Test Generation

Concept

Directed test generation is the practice of producing stimulus that intentionally targets specific scenarios, architectural features, or known regression cases, in contrast to random or constrained-random stimulus that explores a broad state space. Evidence from the Coverage-driven Functional Test Generation paper (Koo and Mishra) frames directed test generation as one of three test-generation categories (random, directed, directed-random) and motivates automated directed test generation using formal methods such as model checking and SAT-based bounded model checking, where functional faults expressed as negated temporal properties drive test synthesis for pipelined processors including MIPS. An industrial case study (Koo, Mishra, Bhadra, Abadir, MTV'06) extends the approach to an industrial Power Architecture processor, introducing decompositional model checking, time-step-based property decomposition, and an algorithm for merging partial counterexamples into a global one. Complementary evidence from the CARRV 2022 RISC-V architectural-testing slides extends the concept to ISA-conformance / architectural-compliance testing, where RISCV-CTG generates controlled, directed tests from coverpoints with data-propagation analysis to ensure signatures are influenced by the tests. The Austrochip 2011 ISS-as-golden-model paper on OR1200 documents the classical motivation that traditional hand-written directed tests in assembly or C/C++ are insufficient for complex designs because their expected results must be known in advance, motivating the use of golden models such as an ISS for comparison. Additional stimulus-engine evidence (PPIGen/SGen) shows a practical hybrid flow in which hand-written directed tests dominate the regression suite and are supplemented by randomized exercisers.

First seen 5/26/2026

Last seen 8/13/2026

Evidence 16 chunks

Wiki v6

WIKI

Directed Test Generation

Overview

Directed test generation is the practice of authoring or producing stimulus that intentionally targets specific scenarios, architectural features, or known regression cases, in contrast to random or constrained-random stimulus that explores a broad state space. In microprocessor and hardware verification it is used alongside random exercisers as part of a hybrid verification strategy, because directed tests provide precise, scenario-specific coverage while random tests provide broad, unexpected exploration. [C1] [C2] [C3]

READ FULL ARTICLE →

NEIGHBORHOOD

No graph connections found for this entity yet. It may appear in future ingestion runs.

explore full graph →

RELATIONSHIPS

7 connections

Hybrid Verification Methodology ← uses 98% 2e

The hybrid methodology combines directed suites with constrained-random stimulus to achieve coverage closure.

ImperasTS ← implements 97% 2e

ImperasTS provides directed test suites targeting specific RISC-V features for structured verification.

micro-architectural fault uses → 95% 2e

Directed test generation targets micro-architectural faults

SGen ← uses 100% 2e

SGen piggybacks onto PPIGen's directed mode to generate tests.

PPIGen ← implements 100% 1e

PPIGen supports a directed mode for generating test programs.

Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V ← uses 90% 1e

The paper presents directed test generation as the approach for creating efficient tests.

Verifying Open Source CPU Cores using Instruction Set Simulators in OVM Environments ← mentions 90% 1e

The paper mentions directed test generation as a traditional but insufficient verification method.

LINKED ENTITIES

3 links

Verifying Open Source CPU Cores using Instruction Set Simulators in OVM Environments MENTIONS Extracted graph relationship

random test generation SEE_ALSO Broad-state-space counterpart to directed test generation; together with directed tests forms the directed-random hybrid category.

Model Checking SEE_ALSO Used as the test-generation engine that produces counterexamples for negated temporal-logic properties (functional faults).

CITATIONS

18 sources

18 citations — click to expand

[1] Directed test generation is one of three stimulus-generation categories (random, directed, directed-random) used for processor validation, and directed tests can reduce overall validation effort because shorter tests can achieve the same coverage goal as random tests. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study

[2] Hand-written directed tests are laborious, time-consuming, error-prone, and infeasible to author comprehensively, motivating automated directed test generation. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study

[3] Functional verification consumes 50% to 70% of design resources (time and effort) and is considered a bottleneck in the development of modern computing systems. Verifying Open Source CPU Cores using Instruction Set Simulators in OVM Environments

[4] Traditional directed tests written in assembly or high-level languages such as C/C++ are insufficient for verification of complex designs because their results must be known in advance for comparison. Verifying Open Source CPU Cores using Instruction Set Simulators in OVM Environments

[5] In ISS-based verification of an open-source CPU core such as OR1200, the ISS is implemented before the core for performance analysis on the instruction set and is reused as the golden model; this speeds up implementation and verification while saving resources. Verifying Open Source CPU Cores using Instruction Set Simulators in OVM Environments

[6] Coverage-driven directed test generation defines pipeline-interaction behaviors via a graph-based processor model, converts negated desired behaviors (functional faults) into temporal-logic properties, and iteratively generates a test for each fault using a model checker or SAT solver while using fault simulation to mark co-detected faults. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[7] Model checking is used as the test-generation engine by applying the negated temporal property to a model checker so that the produced counterexample is precisely a test program exhibiting the target fault. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[8] Traditional model checking does not scale to modern processor designs due to state explosion, motivating bounded model checking with SAT solving plus per-property bound determination and decomposition of model and properties. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[9] Prior test-generation approaches for industrial processors focus mainly on the data path rather than the control path, and control signals spreading across pipeline stages and buffers complicate model partitioning and counterexample merging. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study

[10] The MTV'06 industrial methodology takes an architectural specification containing both structure (micro-architectural details) and behavior (instruction set), automatically generates temporal-logic properties from a functional fault model such as pipeline (graph) coverage, uses decompositional model checking with time-step-based property decomposition, and merges partial counterexamples into a global counterexample; experiments on a Power Architecture-based processor showed promising test-generation time and program length. Directed Micro-architectural Test Generation for an Industrial Processor: A Case Study

[11] The CARRV 2022 slides identify four challenges to maintaining a high-quality RISC-V architectural test suite: standardized test format (RISC-V Architectural Test SIG Test Format with signature-based tests compared to a golden model signature), a standard ISA-coverage specification format, a coverage-measurement tool, and a tool to generate efficient, directed tests as the ISA specification and testing scopes grow. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022

[12] RISCV-CTG is a directed test generator that produces controlled, directed tests from ISA coverpoints with no negative testing, and uses data-propagation analysis to ensure the test signatures are influenced by the tests; the tool is simulator/implementation-independent and requires only basic Python and RISC-V knowledge. Automating Generation and Maintenance of a High-Quality Architectural Test Suite for RISC-V - CARRV at ISCA 2022

[13] PPIGen supports directed mode (C or `.S` source plus a `.cfg` file, configuration registers at reset by default) and random/exer mode (a config file constraining random variables, configuration registers randomized); both modes share init-code and linker-script generation. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[14] SGen piggybacks on PPIGen's directed mode by emitting an assembly program and a config file that flow through PPIGen, providing sequence-based test generation built on `inst`, `seq`, `tests` libraries and a `randutils` package with a `weighted_set`, using C++11 features such as lambdas, improved `<random>`, `std::function`, regular expressions, and `auto`. Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[15] An RTL designer's regression suites are composed mostly of directed tests supplemented by a small selection of PPIGen and SGen exercisers; SGen caught more RTL bugs than the standard regression suite and PPIGen, and directed failures alone did not necessarily prevent release (only one of three reported regression series had directed failures that would have blocked release). Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[16] Over a one-month period, 25,000 exercisers per generator were run (50,000 total runs) yielding 1,731 total failures (171 PPIGen, 1,560 SGen). Coverage-driven Functional Test Generation for Processor Validation using Formal Methods

[17] CLOTHO is a directed-test-generation framework that combines a static analyzer and a model checker to detect serializability violations in SQL database-backed Java applications on weakly-consistent storage and to translate them back into concrete test inputs. CLOTHO: Directed Test Generation for Weakly Consistent Database Systems

[18] LLM-aided Coverage Directed Test Generation (CDG) uses an LLM as a Verilog Reader that grasps Verilog code logic to generate stimuli reaching unexplored code branches and outperforms random testing on designs within the LLM's comprehension scope. VerilogReader: LLM-Aided Hardware Test Generation