Benchmark Workload
Definition
A benchmark workload is a standardized collection of programs, kernels, or computational tasks used to evaluate the performance, overhead, or functional behavior of a computing system or platform. Benchmark workloads are typically selected to exercise representative components of the system under test — such as the CPU pipeline, memory subsystem, interconnects, or protection mechanisms — so that quantitative metrics (e.g., instruction count, execution time, latency, power consumption) can be measured and compared across implementations.
Use in Virtual Platform Evaluation (CHERI-RISC-V VP++)
In the context of the CHERI-RISC-V VP++ virtual platform (presented at ASP-DAC 2026), benchmark workloads were used to evaluate the capability-based CHERI architecture on a SystemC/TLM virtual prototype. The case study performed a CheriBSD boot followed by running 10 benchmark workloads on the virtual platform, exercising the full CHERI trust and protection chain, including the instruction set architecture (ISA), memory management unit (MMU), memory, privilege levels, and capability mechanism.
Specific Benchmark Workloads
The CHERI-RISC-V VP++ evaluation used the following benchmark workloads running on CheriBSD:
- dhrystone — a synthetic integer benchmark measuring CPU integer performance
- whetstone — a synthetic floating-point benchmark
- peg — a pattern-matching / parsing-related workload
- linear (
linear_100x100-sp,linear_10k-sp) — linear algebra kernels - alg-mid (
alg-mid_125k) — mid-sized algorithm kernel - loops-all-mid — loop-intensive kernel
- nnet_test — neural network test workload
- parser — parser workload
- radix2 (
radix2_big_64k) — radix-2 computation - sha (
sha_test) — cryptographic hash benchmark - zip (
zip_test) — compression workload
Instruction-Class Breakdown
During measurement, instructions in each workload were classified into the following categories to characterize CHERI-induced overhead:
- Integer Access — integer memory accesses
- Float Access — floating-point memory accesses
- CHERI Access — capability-related memory accesses
- Integer Others — non-memory integer instructions
- Float Others — non-memory floating-point instructions
- CHERI Others — non-memory capability-related instructions
Measured Overhead
Running these benchmark workloads on the CHERI-RISC-V VP++ produced the following CHERI-enabled performance overhead figures:
- Peak software overhead: 1.83× (relative to a non-CHERI baseline)
- Average software overhead: 1.47×
- Lowest observed overhead: ~0.42× (in some per-workload configurations)
These overhead numbers reflect the per-workload instruction-count breakdown across the integer, floating-point, and CHERI instruction classes, providing a fine-grained view of where capability operations introduce cost.
Broader Use in Systems Research
Beyond capability-architecture evaluation, benchmark workloads are commonly used in other domains:
- Real-time systems security: The MiBench automotive benchmark workloads were used on a Raspberry Pi platform to evaluate the REORDER schedule-obfuscation protocol, which protects dynamic-priority real-time systems against side-channel attacks that exploit deterministic timing behavior under EDF scheduling.
- On-chip networks: The PARSEC benchmark workloads were used to evaluate multicast routing algorithms (e.g., Dynamic Partition Merging) on networks-on-chip for chip multiprocessors and many-core systems.
Role in System Validation
Benchmark workloads serve not only a performance-measurement role but also a robustness and validation role. In the CHERI-RISC-V VP++ work, the platform's robustness was verified using TestRIG (DII/RIG/RVFI) with more than 2 million directed tests, while benchmark workloads provided the higher-level end-to-end exercise of the full CHERI protection chain at the operating-system level. CHERI memory-protection enforcement was confirmed during workload execution — for example, an in-address-space security exception on a capability store caused the operating system to terminate the offending program while keeping the overall system running.