Regression Test Stimulus Isolation

Regression Test Stimulus Isolation is a technique in simulation-based processor verification for identifying a small subset of generated stimuli that achieves high functional coverage and can therefore be reused in regression testing. The concept appears in work on automating processor verification using recurrent neural networks, where coverage feedback from simulation is used to guide pseudorandom stimulus generation and to isolate compact, high-value stimulus sets.^[1]

Context

In simulation-based verification of processors, a common approach is to generate test stimuli using pseudorandom generators (PRGs), apply those stimuli to processor inputs, and monitor the resulting functional coverage to assess verification completeness.^[1] Stimuli may be represented in different ways, including:

• Bit vectors applied directly to processor input ports.
• Programs loaded into the processor’s program memory.^[1]

The goal of stimulus generation is to exercise as much of the design’s functionality as possible. However, large volumes of pseudorandom stimuli can make regression suites costly to run. Regression Test Stimulus Isolation addresses this by extracting a smaller set of stimuli that still provides high coverage.

Technique

The cited approach uses a recurrent neural network (RNN) to dynamically alter constraints for a pseudorandom generator.^[1] The RNN receives coverage feedback from simulation of the design under verification and uses that feedback to influence subsequent stimulus generation.^[1]

The overall process can be summarized as:

1. Generate processor verification stimuli using a constrained PRG.
2. Simulate the processor design with those stimuli.
3. Collect coverage feedback from the simulation.
4. Feed coverage information to an RNN.
5. Dynamically adjust PRG constraints based on the RNN’s output.
6. Continue generation until coverage improves or closure is reached.
7. Isolate a small set of high-coverage stimuli suitable for regression testing.^[1]

Purpose in Regression Testing

Regression testing typically requires repeatedly running test suites after design or verification changes. A full pseudorandom stimulus corpus may be large, redundant, or expensive to execute. By isolating a compact set of stimuli with high achieved coverage, verification teams can reduce regression workload while preserving meaningful coverage of processor functionality.^[1]

Reported Results

In the reported experiments, the method was demonstrated on processors provided by Codasip. The processors were chosen because their coverage state spaces were reasonably large and varied across processor types.^[1] The authors report two main outcomes:

• Faster coverage closure compared with unguided stimulus generation.
• Isolation of a small set of high-coverage stimuli that can be reused for regression tests.^[1]

Although the experiments used Codasip processors, the authors state that the presented techniques are broadly applicable.^[1]

Significance

Regression Test Stimulus Isolation is significant because it connects coverage-driven verification with regression efficiency. Instead of treating pseudorandom generation only as a way to maximize coverage during initial verification, the method also uses coverage feedback to identify reusable regression assets. This supports faster recurring verification cycles while retaining stimuli that are known to exercise important processor behavior.^[1]

References

^[1]: Martin Fajčík et al., “Automation of Processor Verification Using Recurrent Neural Networks,” arXiv:1803.09810, presented at MTVCon 2017. DOI: 10.48550/arXiv.1803.09810.