Verification Plan

Definition

A verification plan is an artifact used in functional verification that specifies what must be verified, how it will be verified, and how completely it has been verified. The cited literature describes the verification plan in three complementary ways:

In assertion-based functional verification (ABV) of an RTL design, the verification plan is a document that bundles the design description and functional specification, the functional coverage goals, the code coverage goals, and the assertions used to check errors in the RTL model and to drive coverage. [arxiv:2106.11461v1]
In system-level verification planning, verification planning is formulated as a sequential decision-making problem that specifies a set of verification activities (VAs) and correction activities (CAs) to be carried out at different phases of system development. Verification activities identify errors and defects, while correction activities fix them. Traditional planning methods consider only VAs as decision choices; jointly planning VAs and CAs into a joint verification-correction strategy (JVCS) is the open formulation. [arxiv:2204.00925v1]
In microprocessor functional verification, the verification plan is the central input that drives automatic pseudo-random test-program generation: it "induces smart sets of tests that carry out the verification tasks" for the processor design. [IBM Research — Genesys, DATE 1999]

Typical Contents in ABV

Based on the cited ABV case study, a verification plan for an RTL block (e.g., a March-algorithm-based MBIST controller) typically enumerates:

The design description and functional specification of the block being verified.
Functional coverage goals — the behaviors and corner cases that must be exercised.
Code coverage goals — structural coverage targets on the RTL.
Assertions — properties that check for errors in the RTL model and that contribute to functional coverage measurement.

The plan is used to measure the level and quality of verification, typically by tracking how many of the planned functional coverage targets are reached during simulation. In the cited MBIST-controller study most functional metrics reached the planned goal of 100%, and an ABV-driven plan enabled roughly 32% fewer directed test cases (25 vs. a maximum of 88 random cases) while still achieving approximately 97% total functional coverage. [arxiv:2106.11461v1]

Role in Verification Planning as a Decision Problem

In larger-scale system verification, verification planning is cast as a sequential decision-making problem in which the plan selects a sequence of verification and correction activities across development phases. The JVCS literature observes that traditional planning methods consider only verification activities as decision choices; jointly planning verification and correction activities is challenging because the two activity spaces differ. A verification plan in this formulation thus encodes not only "what to check" but also "how to act on the defects found," and can be searched using bandit-based tree search and tree-based ensemble learning to handle large systems. [arxiv:2204.00925v1]

Tooling: Genesys and Genesys-Pro

The IBM Genesys tool (Adir et al., DATE 1999) is an automatic pseudo-random test-program generator whose methodology for microprocessor verification "relies on a verification plan which induces smart sets of tests that carry out the verification tasks." The methodology was applied to verify an x86 design, and the paper argues that following such a plan-driven methodology could have helped avoid known escape bugs such as the two infamous Pentium floating-point bugs. [IBM Research — Genesys, DATE 1999]

Genesys-Pro is the successor and the current main test-generation tool for functional verification of IBM processors, including several complex processors. Its template language "considerably reduces the effort needed to define and maintain knowledge specific to an implementation and verification plan," i.e. the tool consumes the implementation- and plan-specific knowledge captured in a verification plan in order to produce functional test programs. [IBM Research — Genesys-Pro, IEEE D&T]

In both cases the verification plan is the knowledge source that the test generator reads in order to produce a smart, directed set of tests that carries out the verification tasks.