Definition
An Architecture Verification Program (AVP) is a test program used in the design verification of hardware processors. AVPs are written to satisfy verification requirements derived from a processor's architectural specification, and are required to run correctly on any realization of that architecture. They check the conformance of processor designs to the specified architecture (see [AVPGEN]—A Test Generator for Architecture Verification, IBM Research, IEEE Transactions on VLSI Systems, 1995).
Purpose and Role in Processor Verification
The goal of processor verification is to ensure equivalence between a processor and its architectural specification. Two principal approaches exist: a formal proof, or exhaustive simulation. The complexity of modern processors renders the formal approach impractical for most industrial applications, and the size of the test space makes exhaustive simulation impossible.
In industrial practice, design verification is therefore generally carried out by simulating sets of assembly-level test programs on a hardware simulator (driven by a hardware description language model such as VHDL, IEEE standard 1076-1987) and comparing the results with the output predicted by a behavioral simulator. If prototype hardware is available, the test programs may be run directly on that. The behavioral simulator is normally written for software development purposes prior to the availability of hardware and represents the expected behavior given the architecture specification. AVPs form a critical part of this verification effort by providing stimulus that exercises the architectural contract, independent of any particular implementation choices.
Test Coverage Scope
Requirements derived from the architecture specification usually call for the testing of every instruction, under three categories of conditions:
- Normal conditions — typical operand and state configurations.
- Boundary conditions — edge cases of operand ranges, register/memory addressing, and resource limits.
- Exception conditions — invalid opcodes, privilege violations, and other error-triggering scenarios.
Because an AVP must run correctly on any realization of the architecture, its correctness criteria are anchored to the architectural specification rather than to implementation details such as clock cycle, cache size, or pipeline depth.
Relationship to Implementation Verification Programs (IVPs)
AVPs are distinct from Implementation Verification Programs (IVPs). While AVPs test conformance to the architectural specification, IVPs are derived from the design work-book (which defines major design details such as clock cycle and cache size) and the actual HDL design, and typically test the functionality of implementation-specific features such as caches, pipelines, and individual hardware units (for example, a Carry Look-Ahead adder or a bit-rotator). Together, AVP and IVP test suites provide complementary coverage: architectural conformance versus implementation correctness.
Generation Approaches
Manual Generation
Traditionally, AVPs were written manually. The process involved first compiling a list of test requirements from the architecture specification, then writing tests to cover that list. This approach is labor-intensive, allocating a substantial proportion of processor design effort to verification, and frequently produced tests that were too simple, as test engineers found it difficult to define complex situations.
Automatic Generation
Automatic test program generation has been used to improve both productivity and quality. Earlier systems include A. Aharon et al., "Verification of the IBM RISC System/6000 By a Dynamic Biased Pseudo-Random Test Program Generator" (IBM Systems Journal, April 1991) and EP-A-453394. Such generators are themselves complex software systems — an example described in the prior art spans about 150,000 lines of C code — reflecting the complexity of modern processor architectures, which can include hundreds of instructions and around ten functional units.
AVPGEN
AVPGEN is a test program generator developed at IBM for producing AVPs that check the conformance of processor designs to the specified architecture. It has been used successfully to debug many IBM S/390 processors and is an integral part of the design process for these processors.
AVPGEN introduces several novel concepts:
- Symbolic execution and constraint solving — used to generate effective tests.
- Biasing techniques — to focus tests on interesting conditions that are likely to activate bugs.
- Inverse problem solving — unlike many earlier systems that make biased random choices, AVPGEN often chooses intermediate or final values first and then solves for the initial values that can lead to the desired values.
- SIGL (Symbolic Instruction Graph Language) — a language provided in AVPGEN that allows users to specify templates with symbolic constraints. The combination of user-specified constraints and the biasing functions is used to focus the generated tests on interesting conditions.
AVPGEN was published in the IEEE Transactions on VLSI Systems in 1995.
Industrial Context
Modern processor architectures are complex. A typical architecture includes hundreds of instructions, a few dozen resources (such as main memory, general-purpose registers, special-purpose registers, and complex functional units like floating-point units, address translation mechanisms, and external interrupt mechanisms), and an architecture description spanning a few hundred pages. This complexity motivates the use of automated AVP generation in industrial design verification flows.