Difftest and Co-Simulation

Relevant source files

Purpose and Scope

This document describes XiangShan's difftest framework and co-simulation infrastructure, which provides comprehensive functional verification by comparing execution behavior against golden reference models. The difftest system enables cycle-accurate validation of architectural state, detection of implementation bugs, and automated regression testing across various workloads.

The infrastructure supports multiple simulation backends (Verilator and VCS), integration with functional simulators (NEMU and Spike), and advanced features like fork-based checkpointing and Profile-Guided Optimization (PGO).

Difftest Architecture Overview

The difftest framework implements a co-simulation approach where the XiangShan design (Design Under Test, or DUT) runs in lockstep with a golden reference model, comparing architectural state at commit points to detect discrepancies.

High-Level Difftest System

Sources: .github/workflows/emu.yml109-114 scripts/xiangshan.py47-56 scripts/xiangshan.py162-170

Golden Reference Models

XiangShan supports two golden reference models for differential testing, each with distinct characteristics suited for different verification scenarios.

NEMU (NJU Emulator)

NEMU is the primary golden model used in XiangShan verification. It is a functional RISC-V simulator that provides:

• Fast execution: Optimized interpreter implementation.
• Full system support: Supports all privilege modes (M/H/S/U) and virtual memory.
• Extension coverage: Implements RV64GCBV with hypervisor extension.
• Checkpoint/restore: Native support for state snapshots via .zstd or .gz files scripts/xiangshan.py41-45

The NEMU reference library is typically located at /nfs/home/share/ci-workloads/NEMU and is loaded via the --diff argument scripts/xiangshan.py47-94

Spike ISA Simulator

Spike serves as an alternative reference model:

• Official RISC-V ISA simulator: Maintained by RISC-V International.
• Specification compliance: Strict adherence to ISA specifications.

Spike can be selected using the --spike flag, which automatically replaces the reference model path from nemu-interpreter to spike scripts/xiangshan.py79-96

Golden Model Comparison Matrix

Sources: scripts/xiangshan.py47-96

Difftest Interface and Commit Verification

The difftest system verifies architectural state at instruction commit boundaries. The interface between the XiangShan RTL and the C++ testbench is critical for ensuring that the reference model stays in sync with the DUT.

Commit-Time State Comparison

Difftest Signal Categories

The difftest interface exports multiple categories of signals for comprehensive verification:

1. Architectural State: Program Counter (PC), GPRs, FPRs, and Vector registers.
2. CSRs: Machine, Supervisor, and Hypervisor mode registers.
3. Memory Operations: Load/Store addresses and data for memory consistency checks.
4. Exceptions: Exception cause and PC to verify trap handling logic.

The interface is checked during CI using difftest/scripts/st_tools/interface.py to ensure SimTop.sv matches the expected difftest-interface.sv .github/workflows/emu.yml106-114

Sources: .github/workflows/emu.yml106-114 scripts/xiangshan.py162-170

Fork-Based Fast Checkpointing (LightSSS)

XiangShan implements a fork-based checkpointing mechanism to accelerate simulation of long-running workloads like SPEC benchmarks. This allows the simulator to "warm up" once and then fork multiple child processes to simulate different regions of interest (ROI).

Fork Mechanism Architecture

Key Parameters for Checkpointing

Sources: scripts/xiangshan.py90-717 .github/workflows/emu-performance.yml122-124

Test Execution Infrastructure

The test execution system is orchestrated by scripts/xiangshan.py, which manages building the emulator, setting environment variables, and running specific test suites.

Key Test Execution Arguments

NUMA-Aware Core Allocation

For multi-threaded emulation, the script implements get_free_cores() to avoid CPU contention. It scans per-core utilization and binds the simulation process to specific cores using numactl scripts/xiangshan.py93-670

Sources: scripts/xiangshan.py74-723

Performance Profiling with PGO

To maximize simulation speed, XiangShan supports Profile-Guided Optimization (PGO). This involves a two-stage build process:

1. Profiling Run: Build an instrumented emulator and run a workload (e.g., CoreMark) to collect profile data scripts/xiangshan.py102-105
2. Optimized Build: Rebuild the emulator using the collected profile data to optimize hot paths.

In CI, PGO is used during the build stage of performance tests .github/workflows/emu-performance.yml56-61

Sources: scripts/xiangshan.py102-723 .github/workflows/emu-performance.yml56-61

Summary of Verification Methodology

The verification methodology combines functional correctness checks with performance validation:

1. Unit Tests: cputest and riscv-tests for basic ISA correctness scripts/xiangshan.py54-350
2. System Tests: Booting Linux to verify privilege modes and virtual memory .github/workflows/emu.yml143-147
3. Performance Regression: Running SPEC CPU 2006 checkpoints to track IPC and detect performance regressions .github/workflows/emu-performance.yml143-203
4. Multi-Core Validation: Verifying cache coherence and memory ordering using XSNoCDiffTopConfig and dual-core NEMU .github/workflows/emu.yml98-105

Sources: .github/workflows/emu.yml98-147 scripts/xiangshan.py336-350 .github/workflows/emu-performance.yml143-203