Architecture-level Verification
Definition
Architecture-level verification refers to the process of validating that a specific hardware implementation conforms to an instruction set architecture (ISA) specification and correctly, as well as completely, implements the instructions defined by that specification. It is typically carried out through the use of test suites—collections of test cases designed to exercise different aspects of the target ISA.
Role in RISC-V Vector (RVV) Verification
In the RISC-V ecosystem, architecture-level verification is formalized through Architecture Compliance Testing. For the RISC-V Vector Extension (RVV), this is known as Vector Architecture Compliance Testing (VACT), which ensures that a vector hardware implementation conforms to the RVV specification and correctly implements vector instructions.
Challenges
Architecture-level verification of RISC-V Vector cores presents a complex challenge because vector instructions exhibit diverse behavior under various instruction parameters and core configurations. A vector instruction test suite must therefore be capable of addressing all legitimate combinations of vector instructions across all different configurations to accomplish comprehensive verification targets. Vector instructions are used to process multiple streams of data in parallel, and the configuration space (including parameters such as VLEN) further expands the verification surface.
Test Suites Used
To address these challenges, several open-source RVV test suites are commonly used or compared:
- Imperas — provides riscvOVPsim and the Vector Test Suite (VTS) Community Edition; only the 32-bit vector test suite is released, supporting VLEN 128b and 256b only, resulting in reduced coverage of some vector instruction categories.
- RIOS Labs Vector Test Generator — an RVV automated test generator created in collaboration with the RISC-V Foundation; uses the RVV Sail Model as the standard golden model and allows users to select specific instructions and configuration parameters.
- RISCV-Torture
- RISCV-DV
- FORCE-RISCV — covers the complete RVV v1.0 specification up to a VLEN of 4096 on a stand-alone basis.
- Yang's Generator
- Tenstorrent
Comparative Findings
A comparative analysis of seven open-source vector ISA test suites (Imperas, RIOS Labs, RISCV-Torture, RISCV-DV, FORCE-RISCV, Yang's Generator, and Tenstorrent), evaluated using the RVV Specification v1.0 as a reference, found that:
- FORCE-RISCV provides the most comprehensive single-suite coverage of the complete RVV v1.0 specification, up to VLEN = 4096.
- Imperas is fairly exhaustive but limited to VLEN = 256.
- Combining multiple test generators can provide a fairly comprehensive verification strategy for vector cores.