Overview

blacklisting instructions is a staged verification technique used to keep generated tests functional when parts of a design are still under development. In the cited RISC-V vector accelerator verification environment, the verification team initially blacklisted many instructions from generated tests because some design modules were still in development for much of the verification process. This allowed functional tests to run at each iteration instead of being dominated by known unsupported or incomplete instruction behavior.

Context of use

The technique was applied in a UVM-based verification flow that used RISCV-DV to generate random RISC-V assembly tests and provide vector instructions for testing a Vector Processing Unit. The team adapted RISCV-DV to match RVV 0.7.1 and added capabilities such as generating vsetvli instructions, modifying memory-operation generation to vary element width and vector length, selecting data-page initialization patterns, and constraining memory addresses to avoid memory exceptions.

Workflow

1. Start with broad exclusion. While design modules were still being developed, many generated-test instructions were placed on a blacklist.
2. Run functional tests at each iteration. The blacklist made it possible to run tests that were expected to be meaningful for the current implementation state.
3. Fix errors and implement missing features. As errors were corrected and missing features were completed, instructions were progressively removed from the blacklist.
4. Re-enable instruction coverage. Once more of the design was ready, the team started whitelisting instructions, which increased the number of errors found because more instruction behavior was being exercised.
5. Use regressions before merge. When an error was tentatively fixed, regressions were run before changes could be merged.

Observed effects

Blacklisting reduced the apparent number of errors during phases where some instructions were not implemented, because those instructions were not being exercised. Later, as instructions were whitelisted after fixes and feature completion, the number of detected errors increased temporarily. This increase reflected expanded test scope rather than necessarily worse design quality. In the later testing phase, continued fixes led to a decrease in errors.

Practical role

In this environment, blacklisting instructions served as a control mechanism for incremental verification. It helped the team avoid blocking the test flow on known incomplete functionality, while still preserving a path toward full instruction enablement as implementation maturity improved.