Overview

RISC-V Torture is a RISC-V test-generation tool. The public repository is ucb-bar/riscv-torture, described as “RISC-V Torture Test” and implemented primarily in Scala.

In the DATE 2019 paper “Verifying Instruction Set Simulators using Coverage-guided Fuzzing,” RISC-V Torture is integrated as the “RISC-V Torture testcase generator” in an evaluation comparing directed RISC-V ISA tests, Torture-generated tests, and a coverage-guided fuzzing approach for instruction set simulator verification.

Generation model described by the evidence

The paper characterizes Torture tests as randomly generated tests. It also states that Torture receives no execution feedback, so each generated test is independent of previous tests. In the same discussion, the authors describe the RISC-V Torture test generator as producing valid instruction sequences.

This makes RISC-V Torture a testcase-generation baseline for valid RISC-V instruction streams, rather than a coverage-feedback-driven mutational fuzzer in the evaluated setup.

Evaluation in instruction set simulator verification

In the DATE 2019 evaluation, RISC-V Torture was run with 1,000, 5,000, and 10,000 generated tests. The reported generation-and-execution times were 5,280 seconds, 26,108 seconds, and 52,168 seconds respectively.

Across those Torture configurations, the paper reports branch coverage of 74.30% on the instrumented ISS under test. Increasing the generated test count from 1,000 to 10,000 produced only limited coverage changes in several reported functional metrics; for example, V(RD) increased from 52.17% to 56.52%, and V(I imm) increased from 9.09% to 63.64%.

The same table reports that Torture-generated tests found errors V1 and V2 in the ISS under test and H2 in Forvis, with no Spike error reported for the Torture rows.

Limitations noted in the paper

The authors attribute the limited coverage increase from 1,000 to 10,000 Torture tests to the absence of execution feedback: Torture generates each test independently of previous executions. They further argue that because RISC-V Torture only generates valid instruction sequences, it would not detect some errors found by their coverage-guided fuzzer, regardless of the number of Torture-generated testcases.

Evidence limitations

The provided evidence supports RISC-V Torture’s repository identity, implementation language, role as a testcase generator, and its behavior in the cited DATE 2019 evaluation. It does not provide command-line usage, configuration syntax, supported ISA profiles, or detailed generation algorithms.