Source bd318f89... — STIMSMITH

SOURCE ARCHIVE

SHA256: bd318f8904ea0479923f9e2c75e71eb9ab341d6388c7a7e4ec5c0a50376f904d

URL: https://hosted-files.sched.co/riscvglobalforum2020/0c/Imperas%20verifying%20all%20the%20flexibility%20of%20RISC-V%20talk%20at%20Global%20Forum%202020.pdf

TYPE: application/pdf

SIZE: 9037.1 KB

FETCHED: 9/30/2026, 10:12:16 PM

EXTRACTOR: liteparse

CHARS: 33,400

EXTRACTED CONTENT

33,400 chars

RISC-V September 3, 2020

[|

Verifying all the flexibility of RISC-V within SoC DV test plans

Simon Davidmann, President & CEO, Imperas Software Lee Moore, Sr. Application Engineer, Imperas Software RISC-V Global Forum 2020 © Imperas Software, Ltd.

Agenda

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • SoC level verification • Summary

Agenda

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • SoC level verification • Summary

RISC-V Presents New Challenges • RISC-V is a new ISA – an open standard ISA • Managed by the non-profit RISC-V International (www.riscv.org) • This means any designer can build a processor implementation • (today (mid 2020) there are almost 100 RTL designs including open source and proprietary) • Traditionally, processor IP … • comes from, and is maintained by, the ISA owner • is single sourced • comes fully verified and compliant to that specific ISA • All users need to do is to verify using integration tests • There is no “standard” approach and there are few available tools for processor verification

• The RISC-V industry / ecosystem needs to adapt best practices for SoC verification to processor verification

To be more specific about the [|] RISC-V DV Problem

• Arm processor IP • ~ 1015 verification cycles per processor • Verification of interface between NoC and processor • 1,000s of SoC designs successfully produced • Similar approaches for ARC, MIPS, Tensilica, …

• RISC-V IP • How well verified is an individual processor (from processor IP vendor, open source, self-built)? • How to verify processor subsystems, especially for AI/ML architectures? • How well verified is interface between NoC and processor? • How to deal with custom instructions?

Agenda

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • SoC level verification • Summary

Compliance Testing

• The device works within the envelope of the agreed specifications • “Have you read and understood the specification” • Testing of the instructions should • Attempt to use all registers as source and destination (not combinations) • Attempt to operate on all bits which compose the immediate values (1 / 0) • Capture a signature in memory region indicating the test result • Based upon a particular hardware configuration • Compare the signature against a known good reference • Static (pre defined signature extraction) • Dynamic (runtime generation from YAML configured reference)

Compliance Testing (2)

• Testing of the instructions does NOT • Attempt to stress all possible aspects of functional verification, e.g.: • All possible combinations of instruction parameters (2-in, 1-out = 32,768) • All possible data values • Attempt to expose possible micro-architectural aspects • Attempt to exercise behavior which generates an exception • Illegal instructions (unsupported extensions) • For example, do not test for missing M instructions in context of RV32I • Illegal conditions (misaligned fetch, load, store) These tasks all come under the Design Verification (DV) heading

[]

Compliance Testing (3)

• Test Qualification – assess how good/effective the tests are • Functional Coverage analysis • Mutation Fault Simulation - Testing analysis (Imperas work in progress) • Provides Decode Coverage • Sees if observe changes on all bits of legal decodes

[|

Compliance Testing (4)

• Test Qualification • Functional Coverage analysis • Mutation Fault Simulation - Testing analysis (Imperas work in progress) • Provides Decode Coverage • Sees if observe changes on all bits of legal decodes • Verified against RV32I test suite • 48 hand coded directed tests (average 150 instructions each) • https://github.com/riscv/riscv-compliance/tree/master/riscv-test-suite/rv32i/src

Compliance Testing (5)

• Compliance RV32I Base Instruction Testing • (Nov 2019) 48 tests, 97% functional ‘basic’ coverage, ~35% ‘extended’ coverage • Imperas RV32/64V vector instruction Compliance Test Suite • 90 different vector engine configurations x ~500 vector instructions • (Aug 2020) 1,500-7,000 tests (depends on vector engine configuration)

• RISCV International’s compliance suites are still a work in progress

Page 11    © 2020 Imperas Software Ltd.    September 2020

Agenda

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • SoC level verification • Summary

Key Issue – Reference I Comparison Ihnperas

• One thing compliance, directed, random have in common... • Is a need for a reference implementation to compare with

• So why is there a need for a reference as part of hardware design verification ? • Comparison for the observed behavior • Covering all possible aspects of the ISA envelope

• And – it needs to represent the exact design and architecture being tested, for example: • XLEN • Vectors: VLEN, SLEN, ELEN, (version: 0.8, 0.9, 1.0, …) • Bit Manipulation (version: 0.91, 0.92, 1.0) • Custom Extensions • M+U (No S) • Hardware LSU Misalignment Support (no exceptions) • CSR: MTVEC Read Only • …

RISC-V Reference Choices • RISC-V is highly configurable pay I= • So it can get a little …. complicated 16% 00 FH Dg • 60 Questions ? YR 7A 0 5 sl 0 =p 3 00 STR

  Bos     N        ~~

riscvOVPsim as the Reference Model for Compliance Testing ras • Industrial quality, free ISS / reference model for compliance testing • https://github.com/riscv/riscv-compliance = 2 • • https://github.com/riscv/riscv-bitmanip Model is built using Open Virtual Platforms (OVP) APIs PU model • Implements full RISC-V envelope • Configurable for all features and versions • Includes full open source Apache 2.0 model elf > riscvOVPsim 4 • Kept up to date for specification changes (cpu+memory) re • Works ‘out of the box’ with full tracing, debug, and Semihosted many other options Filel/O • Video: https://www.imperas.com/riscvovpsim-complete-risc-v- Imperas riscvOVPsim Compliance Simulator iss-bare-metal-software-development-and-specification- compliance-test • Has some limitations which make it not appropriate as a reference model for RTL design verification (DV) • Imperas have other simulators/models for those tasks Page 15 © 2020 Imperas Software Ltd. September 2020

Additional Capabilities Needed for a DV Reference Model • Support for multi-hart processors and concurrency • Support for custom instructions • Support for injection of external / asynchronous events • Including control of external interrupts controllers • Support for control of Debug Mode interactions • Support for step-and-compare DV flow • With reference model encapsulated in SystemVerilog testbench

Page 16    © 2020 Imperas Software Ltd.    September 2020

OVP RISC-V Model and Imperas Simulator as Reference

                                •  Support for multi-hart processors
                                •  Support for custom instructions

RISC-V • Support for injection of external / asynchronous events Reference • Support for step-and-compare DV flow Model & • Used as golden reference in RISC-V Foundations’ Simulator Compliance Suite and Bit Manipulation group http://www.imperas.com/riscv • In use as reference with customers for RTL DV • Public announcements include • Mellanox Technologies • OpenHW Verification Task Group • Google open source ISG project • RISC-V International Compliance Working Group

Imperas Processor Models I Components

                         Add memory-based components (e.g.
    MMC models           loosely-coupled caches, devices)



Instruction Decode
Instruction Behavior
       Tightly-coupled Cache              Instruction    Debug
        Exception modelling               Disassembly  Interface Core processor VMI model
                                                         (RSP)
             TLB / MMU
        Asynchronous Events

Extension / SemiHosting / Instrumentation Libraries

Create and manage resources (eg Registers) Add instruction extensions (e.g. caches) shared between multiple processor instances Add processor independent I/O support Add instrumentation into application (no overhead)

=> complete proven technology, methodology, with development tools – 12 public ISAs, 7 proprietary ISAs, 300+ models

Page 18    © 2020 Imperas Software Ltd.        September 2020

Shared Resource

Processor State

Flow to add new custom [| instructions Develop New Characterize New Characterize C Application Custom Instructions Instructions in Application • Instruction Accurate Simulation • Design Instructions • Instruction Accurate Simulation • Trace / Debug • Add to Application • Trace / Debug • Timing Simulation • Add to Model • Timing Simulation • Function Timing / Profiling • Add Timing • Function Timing / Profiling Release & Deploy • Check RISC-V Compliance • Use as reference for RTL Design Verification Optimize & Document model • Use in Imperas/OVP Platforms, EPKs • Instruction Coverage • Heterogeneous / Homogeneous • Line Coverage • Multi-core, Many-core • Instruction Performance • Imperas Multi-Processor Debug, VAP tools • Generate PDF model doc • Port OS, RTOS (Linux, FreeRTOS…) • Use in many simulation envs (inc. SystemC) • Deliver to end users

Agenda Ihnperas

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • Step and compare methodology • Directed tests • Instruction stream generation • Test generation and execution • SoC level verification • Summary

Two choices for Directed Testing: Im eras self check vs reference compare • Test Encoded Self Checking • Tests are written with expected behaviour encoded • Tests can introspect the state and (self) diagnose faults • Reference Comparison Checking • Tests are written without predicting the result • A reference is consulted for the correct value

        // Device Under Test                          // Reference
        int a = 4; int b = 5;                         int Ra = 4; int Rb = 5;
        int c = a + b;                                int Rc    = Ra + Rb;
        // c == ?                                     // Rc     == 9

        assert(c ==  Rc) // external (@runtime or post-processed)

Two choices for compare DV Methodology: Post process Trace/Signature Compare vs Ihnperas Step and Compare

• Short answer: bottom line is DV resources used • DV is already taking 70% of the design resources, so efficiency of these resources is critical • With post process comparison (trace or signature) • Failures are not known until after the simulation has completed • this can be a long time for a complex test • Errors might get programs into e.g. loop conditions • Binary result – not shows where the instruction/state change is that diverged • Often waste significant simulation and staff resources to diagnose/track down • Step and compare enables failures to be flagged and the simulation stopped when the failure occurs

Page 22    © 2020 Imperas Software Ltd.    September 2020

|







                                        SystemVerilog module
                                    Config and      Ref:   OVP model   > —
                                    control file           (cpu)        >

                                        Semihosted
                                                           File I/O      Debug: GDB or eGui Eclipse

                                    a SystemVerilog module
*   OVP model is encapsulated into

+   A model of RISC-V Foundation'sstandard ISA and ISA extensions (RV32/64 IMAFDC + B + V)
    *     Includes variants for all standard configurations
+   Single processor instantiation with Memory etc all in SystemVerilog
«Interfaces: reset, step, address bus, data bus, interrupts, etc.

+   Like riscvOVPsim it has full trace and logging capabilities

+   Does work with a side-port for GDB or Eclipse eGui debug or Imperas Multi Processor debugger (eGui MPD)

And — users can extend it with custom instructions, state etc, ...

Page 23 © 2020 Imperas Software Ltd. September 2020

Imperas OVP Model Encapsulation

       Addr[31:0], Data[31:0], R/W, clk, reset, …                               |             State Access:
       SystemVerilog Interface                                              ||                •    GPR, CSR
       memRead() memWrite() step()        Dimes                             | eave            •    FPR, VEC, …
       DPI       DPI      DPI     …   DPI                                   | fen                Async Events:
       busReadCB() busWriteCB() step()                                                           • Interrupts

           OVP RISCV-V CPU model object
           SystemVerilog module                                                          •   Easy access from SystemVerilog to
                                                                                             state inside OVP RISC-V CPU

• OVP model is a binary shared object of a RISC-V CPU model model • Encapsulated into a SystemVerilog module, using SystemVerilog DPI • Easy to trigger events from • Instanced in SystemVerilog design or testbench like any module SystemVerilog into OVP model • Control interface so SystemVerilog can load programs, reset, clock, stop, start, … • Access to OVP model state registers and events from SystemVerilog Page 24 © 2020 Imperas Software Ltd. September 2020

Step and Compare Flow

        nu                 SystemVerilog Testbench

               DUT     DUT: RISC-V RTL      Control
              memory
              GCC/        Step
              LLVM        SystemVerilog module    &
               Ref        Compare
              memory            Imperas RISC-V        results.log

RISCV.S RISCV.elf Reference Model

• Testbench loads .elf program into both memories, resets CPUs (RTL and OVP model) • Steps CPUs (DUT and reference), extracting data, and comparing • There is no stored log file – test log data is dynamic and requires two targets to be run and compared

   nVeril ay tional ~~

a ‘ ata Functional C 0 SystemVerilog Coverage yr= Arne a

    ad
— |


                oey     | DUT: RISC-V RTL      :                           CoverGroups                           |             |
                        g                                                            &
                    SystemVerilog module
           |    memc     OVP model        z                                 CoverPoints                         results.log
           \               (cpu)                                            covergroup    andi   cg   with function     sample(ins    t ins);
                                                                                option.per     instance    =   1;
                                                                                cp   rd        coverpoint      get  gpr
                                                                                cp   rsl         coverpoint     get     gpr                          ins.ops(l].key,
                                                                                cp   _immll       coverpoint     get                                 )  {
   Testbench        has class of        and                                             bins  neg
                        CoverGroup                                                      bist  sos    =
   CoverPoints for each instruction                          Group+ 5
°  CPU functional coverage sampled on ‘retire                   rs ad                                 AT LEAST   = INSTANCE       rik BIN    MAX     red MISSING
   .instruction”.   ’   event                                CATEGORY   EXPECTED   UNCOVERED      COVERED     PERCENT
                                                             Variables  96         51             45          46.88
   Coverage directly from simulation as part of              VARIABLE  EXPECTED      UNCOVERED   COVERED   PERCENT      GOAL   WEIGHT   AT LEAST  AUTO  BIN MAX COMMENT
   testbench,         not through post process of log files            2                         15        40.08        100    1        1
                                                             cp _rs2   32        16              16        50.00        100    1        1         [:]

Page 26                 © 2020 Imperas Software Ltd.                                                                                                 September 2020

Key Issue for Directed Testing: I | Functional Coverage

= 2 msc in proses Cass Typeco. (Coverage (Gost Sais nc Merge nancesootyGets coverage reas 100004 | cg 100 99.55% a(t) cgcg 100.00% 100 100.00% EE v Name. Class Type Coverage Goal |% of Goal Included _|Merge_insiances | oH co. 80.00% 100 EEN] aul) = 026 10000% 100 100.00 EE 2 cpᶜᵍ co. 100 50.00% oR 10000% 100 10000% Tres 10000% 100 10000% EE © cg₅₂ 100 EEE 10000 100 NEN wo) soncc mi aoc 10 cg. 100 10000 aul) owas acon co. 100 Ev TWEsal cg 000% autof1) + A 12 sign 10000% 100 10000% EEN oo Tesicg co. 10000% 100 10000 EE 535 bn 100 10000% Tesi 10000% 100 10000% EE 10000 EES leer) SHa cy | = .TYPE add_cg group ClassType |% of Goal |Stus included 5 on riscv_instr_co. 100.00% 100 od + riscv_instr_co. 100.00% 100 100.00% 5 CROSS of group. Ses + fscy inst 10000% 100 100.00% IE | Jorgen od ns cg 10000% 100 10000% EEN - 100.00% IE st cg 2 CVPadd_cg:cp_rd co. 10000% 100 100.00% NEN re rel) + CVP riscv_instr_co. 100.00% 100 100.00% [EEE 8 bin 8 bin 320 1 10000% — 10000% 100 100.00% NEN <auopo 33 1 10000% EE CVP add_cg:cp_gpr_harzard fiscv_instr_co. 100.00% 100 100.00% EE 8 wi 8 v 8bin 53 1 10000% EEE ocg ++ CVP nscy_inst_co.. 10000% 100 100.00% NEN sam cy El bin BL 98 Teen CROSS Tt cg een, | wd®⁺ Resoadd_cg:cp_sign_cross riscv_instr_co. 100.00% 100 100.00% eine us 1 10000% EN riscy_instr_co. 10000% 100 100.00% EEN Dies f== > rod +2 TYPE risev_instr_co. 99.55% 100 99.55% (IEE 8 bn autofS0) 403 1 100.00% EE bin 409

ol TYPE auipc_cg fiscv_instr_co. 80.00% 100 50.00% INT) auto(1 8pris] 4 10000% EE 80.009 100 50.00% EEN) H = donne
cg 10000% 100 100.00% IE nao] 51
TYPEsi cg 100.00% 100 100.00% EEG auto(1 8 binautojA3] 39 1 10000% IEE
TYPE fiscv_instr_co... 100.00% 100 100.00% INE Fo - © 000% 100 3 1 10000 EE +₊2 tiscy_inst_co. 10000% 100 NE binauoisz) J 100.00 o 100 100.00%²I 8 9 1 100000 10000% 100 100.00% a 1 +d cp riscv_instr_co. 10000% 100 100.00% NEN 5 381 100.00% 100 100.00% (I
of TYPE xori_cg 10000% 100 100.00% EE v auto(1 - 4, EN fener co 00 00% 20 as 1100.00 EE Automated instruction coverage reporting from Questa SystemVerilog ws 1 415 functional coverage based on the Imperas RISC-V reference model 31 EE me uo EY Coverage images from Mentor Questa SystemVerilog UVM Simulator Page 27 © 2020 Imperas Software Ltd. September 2020
```
 [|
```

Instruction Stream Generation Ihnperas

• Instruction stream generation (ISG) generates random streams of instructions • Generator given guidance to target specific instruction types and values • Many constraints required to get legal instruction sequences • No predicted results, relies upon reference comparison

• This is just constrained random generation repurposed to processor DV • Constrained random generation is a well-established part of SoC DV flows

Page 28    © 2020 Imperas Software Ltd.    September 2020

Google RISC-V Instruction Stream Generation eras

• High quality SystemVerilog UVM DV infrastructure • Open source (Apache 2.0) © Google Cloud • Drives a RISC-V core through corner cases and pushes it Open Source to the limit SystemVerilog • Requires reference and DUT to generate instruction UVM trace disassembly RISC-V • Traces compared as post-process (neutral CSV format) Instruction Stream • Can compare values and program flow Generator • • dependant upon target capability https://github.com/google/riscv-dv Provides functional coverage for test quality, and to aid guidance

Instruction Stream Generator | Baseline Flow Oo Google Cloud RTL Simulation Metrics.log (a) Google Cloud Open Source RISC-V RTL Open Source SystemVerilog & memory SystemVerilog UVM RISC-V Functional RISC-V SETS Coverage Instruction Imperas ISS Imperas add Stream RISCV.S (cputmemory) Generator Riscvelf Imperas.log Vectors (~~500) Bitmanip (~~100)

• Google: open source riscv-dv instruction stream generator • EDA Partners : SystemVerilog design + UVM simulator for RTL • working with Cadence, Mentor, Synopsys, and Metrics RTL simulators • Imperas: model and simulation golden reference of RISC-V CPU • Imperas have added Vector and to the Functional Coverage • Imperas: extending this flow to support step-and-compare DV Bitmanip extension instructions methodology (not yet publicly released)

OpenHW RISC-V DV Flow 5

UVM Environment for CV32/64 cores
testcase.S                                                                 CORE-V"
                                                                        Re-use Assertions

Manually written Instruction Memory Memory Mapped from Test-Programs loaded from hexfile Virtual Peripherals Formal Verification

Toolchain invoked by UVM run-flow vp addr SVA dbg Debug Agent Agents driven by

                                                                 int  Interrupt independent of
         GCC                            data     Phe:                           UVM Sequences
                                                  EE

Random compile Instruction elf2hex I&D CORE-V" r Agent Test-Program Generator mem_model CV32/64 RTL All CSRs, GPRs & PC Most test-programs crt0.S link.ld from Google generator Test-Program Step Environment Functional Coverage Compare&

RM runs same UVM RISC-V Reference Model program as Core

    make SIMULATOR=<sim> +UVM_TEST=riscv-dv-test
157                     © OpenHW Group                           27 May 2020
    •     Imperas is the RISC-V golden reference model

Page 31                 © 2020 Imperas Software Ltd.                       September 2020

                                                                            I
Case Study:                 lowRISC Ibex                                        []

                Ibex Core
           debug_req_i

                IF Stage                    ID Stage                   EX Block

            addr_o          Prefetch          Decoder     IM   PC       OpA
                                                               IM       OpB
                            Buffer                             IM           ALU
            rdata_i   32                ID                     RF
                                        EX
                                              Controller       IM       OpA MULT
                            Comp                               RF       OpB DIV
                            Decoder
                                              Reg File  RdA                           addr_o
                                                        RdB        CSR         LSU |            32
                                                        Wr                            wdata_o
                                                                                      rdata_i   32

           • Ibex is a small 32 bit RISC-V CPU core (RV32IMC/EMC) with a two stage
           pipeline, previously known as zero-risky (PULP)
           • https://github.com/lowRISC/ibex
Page 32                                 © 2020 Imperas Software Ltd.                              September 2020










Instruction Mem

Data Mem

Case study: ibex core verification In pe ras

lllegal/hint instruct... 25.0% Debug mode 31.3%

          Others
          6.3%

          Pipeline issue
          023%                       Memory access fault
                                                   12.5%

          Interrupt
          18.8%

(a) Google Cloud Categories of found bugs Image courtesy of Google LLC

Bugs Found Using ISG ee Approach

    0 YN SSTATUS J J14d Fence — —
    1 EE mse ALU
    1 I I YI

    Cache line access racing privileged CSR access FENCE operation failure


TN — >]



page fault handling    Incorrect branch execution  ALU corner case bug


Image courtesy of Google LLC

Page 34    © 2020 Imperas Software Ltd.    September 2020

Valtrix Builds Executable Test [| Benches 1 Build STING Imperas₂ Model Run on Imperas RISC-V Reference

• Imperas working with Valtrix STNG STNG • riscvOVPsim as reference model Code ELF • Alternative/complementary = approach to Instruction Stream Generator ne Tost Source polar

STING DUT ELF* Image courtesy of Valtrix Systems

Agenda

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • SoC level verification • Summary

SoC Level Verification ras

• So now you have your CPU core verified… • What about:

• Verification of the interface between the processor or processing element and the NoC • Simple answer is that NoC verification IP is used • This takes some effort, which is taken for granted with traditional processors

• Verification of processing elements with multiple RISC-V cores, as is common in AI/ML SoCs? • This flow is still evolving / being invented

Agenda

• RISC-V verification issues • Compliance is not verification • Reference models and custom instructions • Processor IP verification • SoC level verification • Summary

[|

Summary Ihnperas

• DV is a critical issue for RISC-V processor IP and SoCs • Compliance testing is a subset of DV • Reference models are needed, and are now available • Directed testing, instruction stream generation and test generation/execution are being used for processor IP DV • Step and compare methodology provides the most efficient DV flows • More work is needed – the efficient DV of RISC-V processors is evolving

Page 39    © 2020 Imperas Software Ltd.    September 2020

Thank you