Coverage-guided test generation
ConceptCoverage-guided test generation is a family of testing techniques that steer the production of tests or instruction streams using coverage-related feedback. The provided evidence documents its application in three domains: (1) RISC-V scalar and vector processor verification, where randomized instruction-stream generators evolve at runtime based on observed coverage (tight co-simulation with an ISS in the scalar case, and the RVVTS framework combined with grammar-based generation for RISC-V Vector Extension); (2) LLM-driven software testing, where a code-aware prompting strategy (SymPrompt) decomposes test generation into execution-path-aligned stages; and (3) deep learning system testing, where combinatorial-coverage criteria are adapted into a CT coverage-guided test generation technique.
First seen 5/26/2026
Last seen 9/5/2026
Evidence 9 chunks
Wiki v3
WIKI
Overview
Coverage-guided test generation is a class of testing techniques that use coverage-related feedback to steer the production of tests or instruction streams. The evidence covers three distinct application domains: RISC-V processor verification (both scalar and vector/RVV), LLM-driven software test generation, and deep learning system testing.
RISC-V scalar processor verification
NEIGHBORHOOD
2 nodes · 1 edgesgraph · coverage-guided test generation · depth=1
RELATIONSHIPS
4 connectionsInstruction injection is used to inject instructions that cover specific coverage points, implementing coverage-guided test generation.
Coverage-guided Aging extends coverage-guided test generation by smoothing the coverage distribution over time.
RVVTS combines coverage-guided test generation with single-instruction isolation.
Bayesian network-based test generation is a form of coverage-guided test generation.
LINKED ENTITIES
1 linksCITATIONS
15 sources15 citations — click to expand
[1] A randomized coverage-guided instruction stream generator produces an endless, unrestricted instruction stream that evolves dynamically at runtime based on observed coverage information, leveraging an ISS as a reference model in a tight co-simulation setting with the ISS and RTL core compiled into a single binary communicating in-memory. Cross-Level Processor Verification via Instruction Stream Generation
[2] Coverage information is continuously updated based on the execution state of the ISS, and the novel concept of Coverage-guided Aging is employed to smooth out the coverage distribution of the randomized instruction stream over time, enabling a broad and deep coverage to find intricate corner-case bugs in the RTL core. Cross-Level Processor Verification via Instruction Stream Generation
[3] The verification framework comprises an Instruction-Injector, a Coverage-Observer, a Core-Adapter, the RTL-Core, the RTL-Memory, the ISS, and the ISS-Memory; the Instruction-Injector feeds instructions into both the RTL core and the ISS, while the Coverage-Observer tracks execution-state information used to drive the coverage-guided evolution of the instruction stream. Cross-Level Processor Verification via Instruction Stream Generation
[4] Experiments are performed on the 32-bit pipelined RISC-V core of the MINRES The Good Core (TGC) series, achieving a much more regular coverage distribution of the randomized instruction stream. Cross-Level Processor Verification via Instruction Stream Generation
[5] A prior academic approach integrates the ISS with the RTL core in a very efficient co-simulation compiled into a single binary with in-memory communication, supporting arbitrary combinations of load/store and CSR instructions as well as infinite loops, but does not collect or employ runtime coverage information; it relies on a simple randomized test strategy, making it difficult to continuously achieve a broad and deep test coverage in endless instruction streams. Cross-Level Processor Verification via Instruction Stream Generation
[6] SymPrompt is a code-aware prompting strategy for LLMs in test generation that deconstructs the testsuite generation process into a multi-stage sequence, each driven by a prompt aligned with execution paths and exposing relevant type and dependency focal context, enabling pretrained LLMs to generate more complete test cases without additional training. Code-Aware Prompting: A study of Coverage Guided Test Generation in Regression Setting using LLM
[7] SymPrompt is implemented using the TreeSitter parsing framework; evaluated on challenging methods from open-source Python projects, it enhances correct test generations by 5x, bolsters relative coverage by 26% for CodeGen2, and improves coverage by over 2x for GPT-4 compared to baseline prompting strategies. Code-Aware Prompting: A study of Coverage Guided Test Generation in Regression Setting using LLM
[8] A combinatorial testing (CT) coverage-guided test generation technique was adapted for deep learning systems by proposing coverage criteria for DL systems, with evaluation indicating CT is a promising avenue for testing DL systems. Combinatorial Testing for Deep Learning Systems
[9] RVVTS supports both positive and negative testing strategies and automates the verification process from test generation to failure analysis, combining grammar-based, coverage-guided test generation with Single Instruction Isolation and Code Minimization. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV
[10] The automated RVVTS verification flow includes grammar-based, coverage-guided test generation, instrumentation and build, functional coverage measurement, execution on a reference simulator (Spike) and the DUT, detection of architectural state deviations, and subsequent failure minimization. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV
[11] RVVTS has demonstrated the ability to uncover previously unknown bugs in widely used simulators, including three bugs in the SystemC-based RISC-V VP++ and two in the QEMU emulator. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV
[12] For a fixed (SEW, LMUL), the RVV specification permits vl ∈ {0, ..., VLMAX} and vstart ∈ {0, ..., vl} with VLMAX = ⌊LMUL × VLEN / SEW⌋, yielding ((VLMAX + 1) × (VLMAX + 2))/2 distinct (vl, vstart) pairs — at least a quadratic growth in configurations per instruction even before masks or other mode bits. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV
[13] For Ara with VLEN = 4096 bits, vtype set to SEW = 8 bit, LMUL = 1 (no grouping), VLMAX = ⌊1 × 4096/8⌋ = 512, leading to (512 + 1) × (512 + 2)/2 = 131,841 distinct (vl, vstart) pairs for a single vtype configuration. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV
[14] RVVTS combines coverage-guided test generation with a single-instruction isolation technique to generate high-quality RVV tests and automatically isolate failing instructions and provide minimized test cases; the RVVTS-generated RVV test sets achieve a functional coverage of >90% and employ both positive and negative testing. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV
[15] The paper demonstrates integration of the PULP Ara RVV vector processor into the RVVTS verification flow using an AraRunner built on a verilated Ara RTL model and a small test harness based on the existing Ara Top-Level. From Generation to Failure Categorization: An Open-Source automated RTL Verification Framework for RVV