Skip to content
STIMSMITH

Testcase Extraction Feature

Concept WIKI v1 · 5/25/2026

Testcase Extraction Feature

Overview

The Testcase Extraction Feature was a capability provided in the VCS 2009.12 release for constraint-random verification performance analysis. Its stated purpose was to automatically extract the slowest partition from each randomize call, enabling focused investigation of constraint-solver bottlenecks.[1]

The feature appears in the context of profiling constrained-random generation for x86 opcode generation, where solver runtime and memory usage were key concerns.[1]

Purpose

In constrained-random verification, a randomize call may involve many variables and constraints. Profiling can identify expensive randomization activity, but further analysis often requires isolating the specific part of the constraint problem that dominates runtime. The testcase extraction feature addressed this by extracting the slowest partition from each randomization call automatically.[1]

This made it easier to create smaller, focused testcases for performance measurement and solver comparison.[1]

Technical Context

The evidence discusses two solver modes:

  • RACE solver, described as the default solver in the runtime comparison.[1]
  • BDD solver, which elaborates the entire solution space of a randomize call before selecting a solution.[1]

The BDD solver can consume substantial memory because it elaborates the full solution space. That solution space is cached to speed subsequent randomization calls.[1] In this environment, identifying expensive randomization partitions was especially useful for understanding runtime and memory behavior.

Role in Performance Analysis

The testcase extraction feature was used in a methodology for comparing two verification architectures:

  1. Use constraint-profile data to identify costly randomization behavior.
  2. Select randomize results for two opcodes.
  3. Create a small testbench that randomized those opcodes repeatedly.
  4. Measure CPU runtime for each solver in isolation, avoiding side effects from other testbench components.[1]

The VCS 2009.12 testcase extraction feature supported this workflow by automatically extracting the slowest partition from each randomize call.[1]

Relationship to Constraint Profiling

The article evidence references several forms of profiling data, including:

  • cumulative randomize CPU runtime,
  • individual randomize CPU runtime,
  • individual partition CPU runtime,
  • memory data for constraint solving.[1]

The testcase extraction feature is specifically tied to the partition-level view: it extracts the slowest partition from each randomization call, allowing that partition to be analyzed separately.[1]

Example Use Case

A representative use case is performance tuning of constrained-random opcode generation. In the cited study, randomizing instructions by first choosing an opcode category reduced the number of active variables and constraints. This made the solver’s problem smaller and improved both runtime and memory usage.[1]

The extracted testcase could be used to isolate the expensive randomization partition and compare solver behavior across implementation styles or solver modes.[1]

Reported Impact in the Surrounding Study

Although the evidence does not provide standalone performance numbers for the testcase extraction feature itself, it describes a broader performance investigation in which extracted or isolated randomization testcases were used to compare architectures. In that study:

  • the multiple-class architecture ran faster than the single-class architecture with both solvers,
  • the default RACE solver showed a reported 4× speedup,
  • the BDD solver showed a reported 2× speedup,
  • the newer implementation had 7× fewer constraints than the original.[1]

These results were attributed primarily to reducing the number of variables and constraints presented to the solver.[1]

Limitations of Available Evidence

The provided evidence states the existence and purpose of the feature, but does not specify:

  • command-line options,
  • generated testcase format,
  • exact extraction output,
  • supported SystemVerilog constructs,
  • integration details with VCS profiling reports.

Therefore, those details are not documented here.

References

[1]: Evidence ID 9dce017b-1bb8-4e8e-8b45-0d14653a98c1.