py m P : A SUMMIT An introduction to RISC-V processor verification techniques Lee Moore moore@imperas.com Aimee Sutton aimees@imperas.com

@ImperasSoftware

    I
    []
RISC-V Design Verification

Constrained Interrupts Signature Random

    ISS UVM

    RVVI ISG

Trace Functional compare coverage

Page 2    © Imperas Software Ltd.

I
[]

Agenda

• RISC-V Design Verification challenges • RISC-V design verification techniques • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 3 © Imperas Software Ltd.

RISC-V Design Verification [] S challenges • Processor verification has been a niche discipline • Proprietary techniques • No industry-standard best practices or verification IP • Until recently… (stay tuned) • Techniques from the ASIC/SoC verification are insufficient • New methods are required • Take advantage of what has worked in the ASIC world • Add to it and adapt for RISC-V

Page 4 © Imperas Software Ltd.

Agenda

• RISC-V Design Verification challenges • RISC-V design verification techniques • Post-simulation trace file compare • Self-checking tests and Signatures • Step-and-compare • Step-and-compare with asynchronous events • Verification IP using RVVI • Demo video • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 5 © Imperas Software Ltd.

Post-simulation trace file Imperas compare • Components • Test programs • Can be generated by an ISG – Instruction Stream Generator • Instruction Set Simulator (ISS) • DUT and Tracer • RTL simulator • Comparison script

Page 6 © Imperas Software Ltd.

          I
          []

Test programs

• Directed tests • Write your own • Compliance tests (RISC-V International) • Commercial test suites (e.g. Imperas PMP and Vector) • OpenHW directed test suites (synchronous & asynchronous) • Instruction stream generators (ISG) • Configurable to match processor extensions • Open source solutions • E.g. riscv-dv (CHIPS Alliance) • Commercial solutions • E.g. Valtrix STING

Page 7 © Imperas Software Ltd. RISC-V SUMMIT

                                                   []

Instruction Set Simulators

• ISS • Simulate the execution of a program on a processor • Produce a trace file output • Open source solutions • E.g. spike • Commercial/closed-source solutions Imperas ISS • E.g. riscvOVPsimPlus (cpu+memory) RISCV.elf imperas Imperas_trace.log

Page 8 © Imperas Software Ltd. RISC-V ASUMMIT

                                                             []
DUT + Tracer

• DUT
     •    RTL for RISC-V processor        Testbench
     •    Memory model and bus i/f
•    •    Ability to load test program into memory    RISC-V Core RTL
   Tracer                                                  (DUT)
     •    Extracts information needed for DV
          •  E.g. PC, register values
     •    Bespoke to particular microarchitecture          Memory
     •    Often written by processor designers
     •    Can use RVVI-TRACE standard

Page 9        © Imperas Software Ltd.        RISC-V

Tracer

                                                   [ ]

Trace compare: Process

                                                   DUT_trace.log

          RISC-V                               RISC-V RTL
       Instruction     GCC/                    & memory
          Stream       LLVM                        compare
        Generator                              Imperas ISS
                                               (cpu+memory)
       RISCV.c             RISCV.elf     Imperas     Imperas_trace.log

• Run random generator (ISG) to create tests • Simulate using ISS; write trace log file • Simulate using RTL; write trace log file • Run compare program to see differences / failures Page 10 © Imperas Software Ltd. y RISCV

      []
Trace compare: Pros and Cons

• Pros: • Availability of generic RISC-V simulators (e.g. riscvOVPsimPlus from Imperas) • Simple to set up and use • Cons: • Incompatible trace formats • Must run RTL simulation to the end • Cannot debug live • Difficult to verify asynchronous events (e.g. interrupts, debug requests) • Not a comprehensive DV strategy

Page 11 © Imperas Software Ltd. RISC-V ASUMMIT

           []

Agenda

• Background: RISC-V Design Verification challenges • RISC-V design verification techniques • Post-simulation trace file compare • Self-checking tests and Signatures • Step-and-compare • Step-and-compare with asynchronous events • Verification IP using RVVI • Demo video • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 12 © Imperas Software Ltd. < RISC-V ASUMMIT

                                              []

Self-checking tests

• Components: • RISC-V processor (DUT) and test program; optionally ISS • Process: Application RISC-V RTL “Test Passed” • Each test program checks its .elf & memory results • Prints message to log • Or writes bit to memory

Page 13 © Imperas Software Ltd. RISC-V

                                                             []

Signature comparison

• Components: • RISC-V processor (DUT) and test program; ISS • Process: • Run the test program on the DUT and save the output (signature file) Application riscvOVPsimPlus RISCV.org • Run ISS, write signature file .elf (cpu+memory) Signature file • Compare/diff file results • This is the approach taken by RISCV | Compare International for their architectural Application RISC-V RTL RISCV.org validation (“compliance tests”) .elf & memory Signature file

Page 14 © Imperas Software Ltd. RISC-V

Self-checking tests & Signatures: [] Pros and Cons Inpera S

• Pros: • Simple to set up and execute • Free ISS: https://github.com/riscv-ovpsim • Free compiler: https://github.com/Imperas/riscv-toolchains • RISC-V compliance tests freely available • Cons: • Directed tests cover a subset of processor functionality • Not a complete DV strategy

Page 15 © Imperas Software Ltd. RISC-V ASUMMIT

           []

Agenda

• RISC-V Design Verification challenges • RISC-V design verification techniques • Post-simulation trace file compare • Self-checking tests and Signatures • Step-and-compare • Step-and-compare with asynchronous events • Verification IP using RVVI • Demo video • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 16 © Imperas Software Ltd. RISC-V ASUMMIT

    []

Step and compare

• Components • Test programs (can be generated by an ISG) • Processor reference model • DUT and tracer • RTL simulator • Step-and-compare logic

Page 17 © Imperas Software Ltd. RISC-V ASUMMIT

               []
Processor reference model

• Reference model requirements:
 •         Configurable to select RISC-V ISA extensions
 •         Ability to add customizations (e.g. instructions, CSRs)
 •         Can run in lock-step with the RTL simulator (co-sim)
 •         Ability to “step” reference model at significant events (retire, trap)
 •         Functions to query state of model for comparison
                     User Extension:
             RISC-V       custom
           Base Model  instructions
                            &
                           CSRs
               imperas
Page 18    © Imperas Software Ltd.        RISC-V

DPI

                                                 [ ]

Step and compare: Process

                           Testbench
          RISC-V                    RISC-V RTL
       Instruction     GCC/          & memory  Control
          Stream       LLVM                     Step &
        Generator                   Reference  compare     Results.log
       RISCV.c             RISCV.elf  model

• Reference model is encapsulated in a SystemVerilog testbench • Control block steps both DUT and reference model • Extracts data from each; compares results • Differences reported immediately Page 19 © Imperas Software Ltd. RISC-V ASUMMIT

Step-and-compare: [] Pros and Cons Imperas • Pros: • Instruction by instruction lock-step comparison • Comparison of execution flow, program data, internal state • Errors are flagged immediately – no runaway simulations • Detects synchronous bugs • Cons: • Step-and-compare logic can be fragile and error prone • Does not easily verify asynchronous events

Page 20 © Imperas Software Ltd. < RISC-V ASUMMIT

           []

Agenda

• Background: RISC-V Design Verification challenges • RISC-V design verification techniques • Post-simulation trace file compare • Self-checking tests and Signatures • Step-and-compare • Step-and-compare with asynchronous events • Verification IP using RVVI • Demo video • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 21 © Imperas Software Ltd. RISC-V ASUMMIT

    I
    []

Step and compare + Async

• Components • Test programs (can be generated by an ISG) • Processor reference model • DUT and tracer • RTL simulator • Asynchronous event drivers (e.g. UVM agents) • Step-and-compare logic +

Page 22 © Imperas Software Ltd. RISC-V

Step and compare + Async:                    []
Process

                                         Debug     Interrupt
                                         driver      driver

           RISC-V         GCC/    |]     RISC-V RTL
 Instruction                              & memory   Control
           Stream         LLVM                        Step &
 Generator                                   compare     Results.log
               RISCV.c        RISCV.elf  Reference
                                           model   Testbench
           • Asynchronous events are driven into the DUT
           • Step and compare logic informs reference model about
           async events
Page 23                       © Imperas Software Ltd.        RISC-v

Tracer

Async step and compare: Imperas Pros and Cons • Pros: • All the benefits of step-and-compare • Responds to asynchronous events • Cons: • Step-and-compare logic can be fragile and error prone • Implementation of async event handling is not reusable • Async events not connected to the reference - can conceal bugs • Significant effort to implement and maintain

Page 24 © Imperas Software Ltd. RISC-V ASUMMIT

           []

Agenda

• Background: RISC-V Design Verification challenges • RISC-V design verification techniques • Post-simulation trace file compare • Self-checking tests and Signatures • Step-and-compare • Step-and-compare with asynchronous events • Verification IP using RVVI • Demo video • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 25 © Imperas Software Ltd. RISC-V ASUMMIT

    I
    []

RISC-V Processor VIP

• Requirements: • Configurable, extendable RISC-V processor reference model • Standard interface to receive tracer data • Standard way to receive asynchronous events • Methods to configure, control and query the reference model • Mechanism to compare DUT state with the reference model and report errors/mismatches • A method to verify DUT response to asynchronous events

Page 26 © Imperas Software Ltd.

                                                           [ ]
Standard interface: RVVI                                   Innperas

• RVVI = RISC-V Verification Interface      Testbench
 •         https://github.com/riscv-verification/RVVI
• Work has evolved over 2 years        Simulation
 •         Imperas, EM Micro, SiLabs, OpenHW        control
• Standardize communication                                   RISC-V
between testbench and RISC-V VIP                    RISC-V    Verification
                                                              IP
• Two parts:                                         Core
                                                     RTL
 •         RVVI-TRACE: signal level interface to    (DUT)
           RISC-V VIP
 •         RVVI-API: function level interface to
           RISC-V VIP

Page 27        © Imperas Software Ltd.

Tracer RVVI-TRACE RVVI-API

                                                []
RVVI-TRACE

• Defines information to be extracted by
tracer
• SystemVerilog interface                       valid
• Includes functions to handle     RISC-V   |   insn[..]
asynchronous events                 Core        . . .
•     E.g. interrupts, debug req    RTL        net_push() | RISC-V
                                   (DUT)               Verification IP
                                            i  net_pop()
• https://github.com/riscv-
verification/RVVI/tree/main/RVVI-VLG

Page 28        © Imperas Software Ltd.

Tracer

RVVI-TRACE

                                        I
RVVI-API    rvviRefEventStep()
            rvviRefGprsCompare()
            rvviRefPcCompare()     • Standard functions that RISC-V
            rvviRefCsrsCompare()   • processor VIPs need to implement
—                                   Supports a step-and-compare
                                    methodology
           rvviRefGprGet()         • C and SystemVerilog versions
                                    available
           rvviRefPcGet()          • https://github.com/riscv-
           rvviRefInsBinGet()       verification/RVVI/blob/main/include
           rvviRefCsrGet()          /host/rvvi/rvvi-api.h

Page 29         © Imperas Software Ltd.

RVVI-API

ImperasDV components
Configurable reference

Testbench
ImperasDV
  RISC-V
Reference
  Model

    Configuration

trace2api







SystemVerilog C

Page 30       © Imperas Software Ltd.

RVVI-TRACE

RVVI-API

ImperasDV components
Control and Introspection

Testbench
ImperasDV
  RISC-V
Reference
  Model

    Configuration

trace2api    Synchronization






SystemVerilog C

Page 31       © Imperas Software Ltd.    Rimi

RVVI-TRACE

RVVI-API

ImperasDV components  I
Asynchronous events

Testbench
ImperasDV
  RISC-V
Reference
  Model

    Configuration

trace2api        Synchronization

Predictive
  engine



SystemVerilog C

Page 32       © Imperas Software Ltd.    Rimi

RVVI-TRACE

RVVI-API

ImperasDV compoents
Comparison

Testbench
ImperasDV
                      RISC-V
                    Reference
                      Model

    Configuration

trace2api        Synchronization

    Predictive
    engine

                    Scoreboard

    SystemVerilog C Pass/Fail
                  determination

Page 33        © Imperas Software Ltd.    December 22

RVVI-TRACE

RVVI-API

ImperasDV components
Coverage and logging

Testbench
    ImperasDV
                          RISC-V
                        Reference
                          Model

         trace2cov
             Configuration

         trace2api    Synchronization

             Predictive
         trace2log    engine

                        Scoreboard

    SystemVerilog     C Pass/Fail
                      determination


Page 34    © Imperas Software Ltd.

RVVI-TRACE

RVVI-API

   []

Verification IP + RVVI: process

• Instantiate VIP in a testbench Testbench • Connect tracer using RVVI-TRACE i/f • DUT and reference model run the Foor same program • Retire, trap events communicated Core iⁱ over RVVI ou”) ron | • Internal state continuously compared = |rene • RVVI-TRACE monitored for async = d__ • events 2 Predictive engine verifies legal determination scenarios

Page 35 © Imperas Software Ltd. RISC-V

           []

Verification IP using RVVI

• Pros: • Errors are flagged immediately • Finds synchronous and asynchronous bugs • Checking is done for you • Reusable across different core DV projects • Interchangeable due to standard interface (RVVI) • Ease of use • Training, documentation, and support

• Cons: • Cost of VIP licenses

Page 36 © Imperas Software Ltd. RISC-V ASUMMIT

           []

Agenda

• Background: RISC-V Design Verification challenges • RISC-V design verification techniques • Post-simulation trace file compare • Self-checking tests • Step-and-compare • Step-and-compare with asynchronous events • Verification IP using RVVI • Demo video • Techniques from ASIC/SoC DV • How to choose the right technique?

Page 37 © Imperas Software Ltd. RISC-V ASUMMIT

Demonstration

• DUT: OpenHW Group CV32E40X RISC-V processor • Simulation: passing test • Simulation: failing test • Simulation: asynchronous event bug

Page 38 © Imperas Software Ltd.

[] VIDEO: Passing test [©

• 1:22 ran tn i TT gery ImperasDV for RISC-V processor verification | Ch imperas | ii eee i en == i REE ey

= — = Ts bench cantor Cor Ea 15 bench VP | mode 00cht

                                                               ETT

Page 39 © Imperas Software Ltd. RISC-V

  I

VIDEO: Failing test

• 2:59

Page 40 © Imperas Software Ltd.

           Asynchronous events                              Inoperas

RISC-V Core pins OSE WY WY WO VO WY WO i fF i i i 1 i i i i fr i i i i i 1 wi ft 1 1 i i i i 1 1 i i J J ia FS A i6 / i7 / RVVI-TRACE iff InterruptT taken, but which one? valid retire trap PC MEPC

      MoAUsE
Page 41        © Imperas Software Ltd.                                     RISC-V
                                                                           4summIT

VIDEO: Asynchronous

• 4:38

Page 42 © Imperas Software Ltd. ivi

    []

Agenda

• Background: RISC-V Design Verification challenges • RISC-V design verification techniques • Techniques from ASIC/SoC DV • Verification planning • Functional coverage • Assertions • How to choose the right technique?

Page 43 © Imperas Software Ltd.

           []

Verification planning

• Start with the end in mind. What are your verification goals? • Capture them in a plan • How you will measure that they have been met? • Directed test, coverpoint, assertion? • Capture this in the plan too • Metric-driven verification is popular • Common metrics: code coverage, functional coverage, all tests passing, no new bugs found for a period of time • Sample open source verification plans: • https://github.com/openhwgroup/core-v-verif/tree/master/cv32e40p/docs/VerifPlans

Page 44 © Imperas Software Ltd.

[]

Functional coverage

• Define goals for verification • Measure that goals are achieved • Measure the effectiveness of constrained-random stimulus • Requires EDA tools to capture, merge, display coverage results • Requires many simulations to achieve coverage closure • Industry-standard best practice for ASIC/SoC

Page 45 © Imperas Software Ltd.

    [ ]

RISC-V Functional Coverage

For a processor there are different types of functional coverage required: • Standard ISA architectural features • unpriv. ISA items: mainly instructions, their operands, their values => these are standard and the same for all RISC-V processors – it is the spec… • Customer core design & micro-architectural features • priv. ISA items, CSRs, Interrupts, Debug block, … • pipeline, multi-issue, multi-hart, … • Custom extensions, CSRs, instructions

Page 46 © Imperas Software Ltd.

RISC-V Instructions (Standard ISA =
architectural feature)

• There are many different instructions in the RV64 extensions: • Integer: 56, Maths: 13, Compressed: 30, FP-Single: 30, FP-Double: 32 • Vector: 356, Bitmanip: 47 Krypto-scalar: 85 • P-DSP: 318 • For RV64 that is 967 instructions… • Each instruction needs SystemVerilog covergroups and coverpoints • 10-40 lines of SystemVerilog for each instruction • 10,000-40,000++ lines of code to be written • Not design or core specific

Page 47    © Imperas Software Ltd.

Generating functional coverage source files from machine =
readable ISA definition        ra S
          RISC-V                                    Configuration:
privilege and un-privilege  | Imperas RVFCgen = xlen, csr, csrFields,
   ISA machine readable
        definition        | | | …                   compliance, DV
                                                extensions, options, …

        Functions to convert
           RVVI-TRACE to                                     …
        Functional Coverage
             structures

    clk Functional Coverage                                  …
              sampling

             Hand coded        Generated SystemVerilog source
    SystemVerilog
Page 48        © Imperas Software Ltd.                       RISC-V










RVVI-TRACE

RV32I RV64I RV32E RV64E RV32A RV64A RV32M RV64M RV32C RV64C RV32F RV64F RV32D RV64D RV32B RV64B RV32Ks RV64Ks RV32V RV64V RV32P RV64P RV32PMP RV64PMP RV32CSR RV64CSR Exceptions Exceptions Interrupts Interrupts Debug Debug MMU MMU Hypervisor Hypervisor User defined User defined

                                                              []

Functional coverage examples

• riscvISACOV • https://github.com/riscv-verification/riscvISACOV • OpenHW Group core-v-verif • https://github.com/openhwgroup/core-v-verif/tree/master/cv32e40s/env/uvme/cov

                                            8.08%         27373981 6.86%
4     Ix               obj add              74.79%        113/126 89.68%
                       & cprd               100%          32/32(100%)
                       & cp_rd_sign         100%          3/3(100%)
                       &                    100%          32/32(100%)
                       &                    33.33%        1/3(33.33%)
                       & prs2               100%          32/32(100%)
                       & cp_rs2_sign      ET] 66.67%      2/3(66.67%)
                       &                    50%           1/2 50%
                       &                    100%          2/2(100%)
                       & cmp_rd_rs2_eq     = 50%          1/2(50%)
                       & cmp_rd_rs2_eqval   100%          2/2(100%)
                       & cmp_rs1_rs2_eq     50%           172(50%)
                       &                    100%          2/2(100%)

Page 49 AB © Imperas Software Ltd.22.22% 2/9(22.22%) RISC-V SUMMIT

    []

Assertions

• Popular languages: SVA (SystemVerilog), PSL • Concurrent assertions • Rules to check behaviour over time • Can be used to verify micro-architectural details • Can be written by RTL designers • Can be reused in formal verification • “Cover” properties contribute to functional coverage

Page 50 © Imperas Software Ltd.

    []

Agenda

• RISC-V Design Verification challenges • RISC-V design verification techniques • Techniques from ASIC/SoC DV • How to choose the right technique? • DUT considerations • Technology questions • Hybrid methodologies

Page 51 Confidential and Proprietary © Imperas Software Ltd.

DUT considerations affecting [] verification method Imperas • Is this a new design? • Have you started from a commercial IP core? • What is the magnitude of your change? • What does your IP vendor recommend for verification? • Are you using or modifying an open-source core? • Can you find evidence of verification done to date? • Can you reuse or build upon existing DV infrastructure? • What is your goal? • Research project, sell/provide IP, tape out • What is your requirement for reuse? • Across teams, future projects, etc.

                                                                RISC-V

Page 52 © Imperas Software Ltd.

Technology considerations [] affecting verification method Imperas • Verification language • SystemVerilog, VHDL, C/C++, Python? • Some methodologies only available in a certain language • E.g. functional coverage (SV), UVM (SV, Python), OSVVM (VHDL) • UVM • Widely adopted and industry proven • Good body of knowledge / online resources available • Strengths: virtual sequences, configuration database, messaging • Weaknesses: limited choice of RTL simulation tools, heavy-weight solution • Build it yourself, use open-source, or use Verification IP? • Cost of VIP licenses vs cost of time and effort to build

Page 53 © Imperas Software Ltd.

        []

Hybrid methodologies

• Post-simulation trace file compare + VIP • Use trace file compare for ISA / unprivileged tests • Use verification IP for complex scenarios: • Sync and Async exceptions • Corner cases • Make sure to combine functional coverage results • Pros: can save on license costs • Cons: effort required to build, maintain, and co-ordinate two separate verification environments

Page 54 © Imperas Software Ltd. RISC-V

                         [ |

Thank you NE • Any questions?

• . ISG

RISC-V Functional VIP coverage RVVI e • Lee Moore (moore@imperas.com) = • Aimee Sutton (aimees@imperas.com)

Page 55 © Imperas Software Ltd. RISC-V ASUMMIT

mp

RISC-V