Skip to content
STIMSMITH

Assertion-Based Testing

Concept WIKI v1 · 5/30/2026

Assertion-Based Testing uses explicit assertions as executable checks within tests. In the supplied evidence, assertions are used both as failure conditions inside randomized RISC-V CPU test sequences and as a fine-grained unit for regression test selection in software testing.

Overview

Assertion-Based Testing is a testing approach in which explicit assertions encode expected properties or conditions that a system should satisfy during a test. In the provided evidence, assertions appear in two closely related roles:

  • as executable checks inside generated or hand-authored test sequences, where an assertion can cause a test to fail; and
  • as fine-grained units for selecting which parts of a regression test suite to rerun after code changes.

Assertions as test oracles

In the TestRIG/QCVEngine setting for randomized RISC-V CPU testing, test sequences can include assertions such as a condition that the value written by the previous instruction was non-zero. These assertions make it possible for a test to fail without observing a divergence between two implementations, and the evidence notes that such assertion-bearing sequences do not require tandem verification to discover a failure.

The same evidence shows assertions being used in a shrunken counterexample for a CHERI-related vulnerability: after an illegal cSetBoundsImmediate operation and a load through the resulting capability, the sequence reads an L1 cache-miss counter and uses .assert rd_wdata == 0x0 "" to check the expected counter value. The text explains that a preceding .noshrink region was needed to initialize counters so that the final assertion would be deterministic.

Assertions as a regression-test selection unit

A separate software-testing usage is described in the public context for Fine-Grained Assertion-Based Test Selection. The paper addresses the cost of running full regression test suites after every code change and proposes selecting affected tests at the level of individual assertions rather than coarser entities such as test classes. The approach analyzes test code at statement level and treats test assertions as the selection unit.

In the reported evaluation on 11 open-source subjects, the assertion-level selection approach increased selection precision for all subjects, reduced overall test time by an average of 63%, and made regression testing 7–38% faster than two compared state-of-the-art test-selection techniques.

Relationship to QCVEngine

QCVEngine is connected to this concept through TestRIG’s support for assertion-bearing sequences. The evidence states that once QCVEngine finds a counterexample, QuickCheck shrinking is used to reduce it, and it separately describes that sequences can include assertions that allow failures without tandem verification.

LINKED ENTITIES

1 links
QCVEngine USES

CITATIONS

4 sources
4 citations
[1] In TestRIG/QCVEngine, sequences can include assertions, and those assertions can make a test fail without tandem verification. Randomized Testing of RISC-V CPUs using Direct
[2] A TestRIG counterexample used `.assert rd_wdata == 0x0` after counter initialization, with `.noshrink` used to keep initialization needed for deterministic assertion behavior. Randomized Testing of RISC-V CPUs using Direct
[3] Fine-Grained Assertion-Based Test Selection analyzes test code at statement level and treats test assertions as the unit for regression test selection. Fine-Grained Assertion-Based Test Selection
[4] The assertion-based test-selection evaluation reported increased selection precision across 11 open-source subjects, an average 63% reduction in overall test time, and regression testing 7–38% faster than two compared techniques. Fine-Grained Assertion-Based Test Selection