Skip to content
STIMSMITH

Reinforcement learning-based directed test generation

Technique WIKI v1 · 6/25/2026

A functional-verification technique that formulates directed test generation as a reinforcement-learning (RL) problem, in which an agent learns a policy for constructing or extending test programs so as to drive coverage toward pre-specified verification targets. The approach was introduced by Pfeifer, Zimpel, Andrade et al. (DATE 2020), who applied it to directed test generation for shared-memory verification.

Definition

Reinforcement learning-based directed test generation is a directed test generation technique for hardware / pre-silicon verification in which the search for stimuli that hit a pre-selected verification target is cast as a sequential decision-making problem and solved with reinforcement learning. Instead of relying on unconstrained or constraint-random biasing (refs. [5], [10], [17]) or on Bayesian-network / coverage-directed heuristics (ref. [11]), an RL agent iteratively selects actions that extend or mutate a test program and is rewarded according to how much closer the resulting stimulus brings the verification campaign to its goal.

Origin work

The technique was introduced in:

PFEIFER N., ZIMPEL B. V., ANDRADE G. A. G., et al. A reinforcement learning approach to directed test generation for shared memory verification. 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), Grenoble, France, 2020: 538–543. (reference [12] of the surveyed bibliography)

The paper applies RL to the problem of generating test programs that exercise specific scenarios in the shared-memory subsystem of a multiprocessor design under verification, framing the construction of each test as a Markov decision process and training a policy to maximise a coverage-oriented reward.

Place in the directed-test-generation landscape

Within the family of directed test-generation methods, the reinforcement-learning variant is closely related to, but distinct from, the following techniques also cited in the surveyed bibliography:

  • Coverage-directed test generation (CDG), exemplified by the Bayesian-network-based scheme of Braun, Fine and Ziv (ref. [11]), which uses learned probabilistic models to bias stimulus generation toward coverage holes.
  • Constraint-satisfaction-problem (CSP)-based random test generation, as in Bin et al. (ref. [17]) and surveyed by Ayadi et al. (ref. [15]), where stimulus constraints are encoded declaratively and solved by a CSP engine.
  • Constrained-random verification for RISC-V (refs. [5], [7], [13]), which combines architectural constraints with random stimulus generation.
  • Coverage-guided / endless randomized instruction-stream generation with coverage-guided aging (refs. [6], [18]), which adapts stimulus generation at the ISA level using coverage feedback.
  • Specification-based test-program generation (ref. [4]) and data-constraint generation with SMT solvers (ref. [9]), which use formal specifications rather than learned policies to direct stimulus generation.

The RL approach generalises the learned biasing idea behind CDG: rather than learning a static probabilistic model of the input space, an RL agent learns a closed-loop policy that conditions on the current state of the design-under-verification (e.g. achieved coverage, reached architectural state) and updates its behaviour online.

Relation to instruction-set and RISC-V verification

The surveyed bibliography places the technique in the broader context of instruction-set and RISC-V verification, where directed test generation is used to reach corner-case instruction sequences and architectural states (refs. [3], [7], [8], [13], [14]). The same body of work motivates RL-based approaches: classical constraint-random and coverage-directed methods struggle to compose long, ordered instruction sequences needed to expose specific shared-memory or ISA-level bugs, while an RL agent can be rewarded for emitting exactly the sequence of instructions required to reach a target microarchitectural state.

Status of evidence

The only direct evidence available for this entity is the bibliography reference (chunk 334384eb-e429-4f79-a89f-a63add44011d, ref. [12]) in which Pfeifer et al.'s DATE 2020 paper is cited. Specific implementation details of the RL formulation (state space, action space, reward shaping, learning algorithm) are therefore not derivable from the supplied evidence and are deliberately not asserted in this article.

LINKED ENTITIES

1 links
A reinforcement learning approach to directed test generation for shared memory verification USES

CITATIONS

4 sources
4 citations
[1] Reinforcement learning-based directed test generation was introduced by Pfeifer, Zimpel, Andrade et al. at DATE 2020 in their paper 'A reinforcement learning approach to directed test generation for shared memory verification' (pp. 538–543). A RISC-V Test Sequences Generation Method Based on Instruction ... (reference list, ref. [12])
[2] The technique is a form of directed test generation, i.e. stimulus generation aimed at pre-specified verification targets, as opposed to unconstrained or purely random stimulus generation. A RISC-V Test Sequences Generation Method Based on Instruction ... (reference list, refs. [5], [10], [11], [17])
[3] RL-based directed test generation is closely related to coverage-directed test generation using Bayesian networks (Braun, Fine and Ziv, ref. [11]) and to CSP-based random test program generation (Bin et al., ref. [17]; Ayadi et al., ref. [15]). A RISC-V Test Sequences Generation Method Based on Instruction ... (reference list, refs. [11], [15], [17])
[4] The technique is situated in the broader context of instruction-set / RISC-V verification, where directed generation is used to reach corner-case instruction sequences (refs. [3], [7], [8], [13], [14]). A RISC-V Test Sequences Generation Method Based on Instruction ... (reference list, refs. [3], [7], [8], [13], [14])