Rapid Fault Injection Simulation by Hash-based Differential Fault Effect Equivalence Checks

  Johannes Geier∗, Leonidas Kontopoulos∗, Daniel Mueller-Gritschneder†, Ulf Schlichtmann∗

∗Technical University of Munich, Germany, {johannes.geier, leonidas.kontopoulos, ulf.schlichtmann}@tum.de †TU Wien, Austria, daniel.mueller-gritschneder@tuwien.ac.at

Abstract—Assessing a computational system’s resilience to is costly: Firstly, the fault space spanning all possible fault hardware faults is essential for safety and security-related systems. configurations (space and time) grows larger the lower the Fault Injection (FI) simulation is a valuable tool that can increase level of abstraction. Secondly, the simulation performance at confidence in computational systems and guide hardware and software design decisions in the early stages of development. lower levels of abstraction is much lower, e.g., in the range of However, simulating hardware at low levels of abstraction, such thousands of instructions per second vs. millions of instructions as Register Transfer Level (RTL), is costly, and minimizing per second for Instruction Set Simulator (ISS) [11]. Speed-up the effort required for large-scale FI campaigns is a significant techniques that address this issue can be put in two categories: objective. This work introduces Hash-based Differential Fault Pre-FI and post-FI. Among others pre-FI methods include: Effect Equivalence Checks to automatically terminate experiments early based on predicting their outcome. We achieve this by Statistical Fault Injection (SFI) [12], [13] target to reduce the matching observed fault effects to ones already encountered in required number of simulations to sample the fault space. previous experiments. We generate these hashes from differentials Fault Equivalence Pruning (FEP) [14]–[16] collapse multiple computed by repurposing existing fast boot checkpoints from injections to a single equivalent one but was so far only a state-of-the-art acceleration method. By integrating these ap- applicable to errors in memories or register files, but not micro- proaches in an automated manner, we can accelerate a large-scale FI simulation of a CPU at RTL. We reduce the average simulation architectural states. Checkpoint Restore Boots (CRBs) [17]– time by a factor of up to 25 compared to a factor of around 2 to 5 [19] save the effort to boot the system for each FI simulation for state-of-the-art techniques. While maintaining 100 % accuracy, repeatedly. Post-FI techniques try to minimize the effort to we can recover the faulty state through the stored differentials. observe a faulted system’s behavior: Mixed-level Simulations Index Terms—Fault injection, Simulation, Checkpoints, (MLSs), such as [11] and [20], switch from a lower to a higher Register-transfer-level, Fault effect equivalence level of abstraction to speed-up the simulation when possible. I. Introduction However, such mixed-level simulations require considerable knowledge of the underlying Device Under Test (DUT) and, Faults or defects can occur in different variations in com- more importantly, a large effort to set up a simulator that putational systems. In integrated circuits, we can distinguish allows switching between abstraction levels. This is why most between hard (permanent) and soft (temporary) faults. Perma- of the works target Central Processing Units (CPUs) where nent faults refer to (mainly) irreversible defects in the hardware, an ISS enables switching between Instruction Set Architecture e.g., a shorted wire or defect transistor. Soft errors [1], on (ISA) and RTL. In [17], Dynamic Fault Collapsing (DFC) was the other hand, often occur due to transient faults that do proposed as another technique that checks whether a single bit not destroy hardware and are, therefore, reversible. They result during a previous FI simulation was corrupted at a certain point from temporary upsets, seemingly at random due to radiation- in time, assuming any FI targeting this bit at this time will lead induced Single Event Upsets (SETs) [1], [2], or deliberately by to the same error effect and, hence, can be omitted. an adversary via electromagnetic pulses [3], [4] or lasers [5], In this work, we propose a new method for further accelerat- [6]. In the context of road vehicle safety, ISO 26262 [7] ing FI simulation campaigns without compromising accuracy. norms software-level (part 6) and hardware-level (part 11) Fault We build on the idea of DFC of [17]. As a major improvement, Injections (FIs) as methods to increase confidence in a system’s the proposed approach not only omits simulation runs that resilience to hard and soft errors. In contrast to post-silicon FI, corrupt a bit, as seen in previous simulations, but instead checks where experiments are conducted by exposing the device to in narrow checkpoint intervals whether the possible multi-bit radiation, FI simulation allows evaluation at the early stages fault effect pattern was already observed in any previous FI run. of the development, i.e., pre-silicon. Works such as [8]–[10] If the same error pattern is found, the simulation can be termi- have shown that simulations at lower levels of abstraction, e.g., nated early as the outcome is known from the previous FI run. A micro-architecture or Register Transfer Level (RTL), outper- straightforward implementation of this causes large overheads form higher levels of abstraction in terms of accuracy. However, to store and compare the immense number of checkpointed simulating hardware at low levels of abstraction, such as RTL, fault effect patterns as many FI campaigns run millions of FIs, This research was funded by the German Federal Ministry of Education and diminishing the speed-up gained by the proposed method. To Research within the project MANNHEIM-FlexKI, funding no. 01IS22086L. counter this effect, we additionally propose an efficient storage

technique that only stores the difference to the golden reference         fault effect                        outcome
    run and a hashing technique for highly efficient comparisons.         Tolerance               mitigate    Failure
In summary, this work has the following contributions:
(1)  Dynamic Fault Effect     Equivalence  Checks:          A new      mask      latent|mask              unrecover-
         method that terminates FI runs early when a previous run                                         able
(2)  already observed the fault effect.                              Fault activate Error detect Detected             silent
     Differential Checkpointing (diffs)     for efficient storage                                              data corrupt.
          of fault effect patterns. These checkpoints can also be     Fig. 1: Fault and fault effect transitions and classes
     exploited for post-simulation analysis of fault propagation.
(3)            A Hash-based method to compare fault effect check-
         points: We use a database to host hashes of (2) to allow    equals the already conducted experiment. DFC allows FEP by
         active experiments comparison of their own diffs against    building up a database of fault effect records but does perform
          all previously seen ones enabling early termination via    FEEC of an active experiment against this database.
     low-cost Fault Effect Equivalence Check (FEEC).                  Accelerating remaining experiments:            Both SFI and FEP

To demonstrate the approach, we extend an existing open- are powerful techniques. However, there will be a number source, cycle-accurate RTL FI tool to support (1) and (2) of experiments left that have to be conducted. A simulation without modification or knowledge of the DUT, i.e., treat the tool can try to minimize the pre-FI simulation cost (warmup) RTL as a black box. We implement (3) in a server-client by fast-forwarding the simulation state as close as possible framework. Clients simulate a RISC-V-based System on Chip to the FI point. In [11], a MLS is introduced that allows (SoC) with the CPU being the FI target. For benchmarking, we switching between ISS and RTL models of a CPU, where also implemented the state-of-the-art techniques CRB, MLS, the ISS conducts the warmup phase. For deterministic FI and masking checks. We reduce the average simulation time campaigns, a low-cost warmup can be achieved via CRB, for FI campaigns by a factor of up to 25 against pure RTL notably applied to RTL FI in works such as [17]–[19]. The compared to a factor of around 2 to 5 for state-of-the-art speed- idea is to record the simulation state at certain checkpoints of up techniques CRB and MLS. a fault-free reference simulation, which are then used in the subsequent FI simulations. A checkpoint that is as close as II. Related Work possible to the FI point is selected for each fault configuration. Reducing the number of required simulations: In [12], For a uniformly sampled SFI, CRB can cut the simulation Leuveugle et al. adapted established statistical methods to FI cost in half [19]. While CRB effectively removes the warmup simulations of a hardware accelerator. One major contribution segments from a FI campaign, after the fault is injected, its is their finding that for randomly sampled campaigns, not the impact needs to be observed. [11] and [20] implement two fault space (population F) defines the number of required post-FI acceleration techniques in their MLS: Firstly, masking experiments (chosen sample N), but the chosen error margin checks and secondly switch-back to ISS from RTL. Whereas and confidence level with which the campaign is set. [13] [20] compares the faulty micro-architecture against a golden extends this by increasing the sample size until a target error reference state, [11] simulates a modified RTL of a CPU, margin is reached. Unfortunately, SFI is only relevant for FI which contains shadows of its internal registers. This allows campaigns whose goal is to be statistical in nature but can tracking fault propagation. If the primary and shadow registers lack the observability of rare behaviors or strong localities match by value, the fault has vanished, and the experiment of fault patterns. FEP [14]–[16] is another notable technique can terminate early. If the fault is only present in the ISA that aims to reduce the number of simulations needed to be registers or has left the DUT, e.g., corrupted memory, the executed. The underlying method is based on finding equivalent simulation can switch up to the ISS. A major drawback of fault configurations before conducting the experiments. For this approach is its complexity and limit to abstractable DUTs, example, if a variable is written (DEF) and read (USE) at e.g., CPUs. The MLS has to be extended w.r.t. knowledge of a later time, all same-value FIs between these two points abstractable states, i.e., which RTL registers map to which ISA are equivalent. Only one experiment from the equivalence set registers. Furthermore, to allow observing the fault propagation, would be simulated, and the others would be pruned from the the simulator must evaluate both primary and shadow states, simulation set, assuming their outcome would be equivalent. doubling the simulation cost during cooldown. Furthermore, Unfortunately, equivalence classes are harder to find and not the simulation must stay at RTL to avoid losing information as frequent on lower levels of abstraction, such as synchronous due to abstraction if the micro-architecture is corrupt. RTL. For example, micro-architectural registers of a pipelined III. Hash-based Differential Fault Effect Equivalence Checks CPU may be read and updated in the same cycle, minimizing potential DEF/USE intervals. In [17], Berrojo et al. describe In this work, we slightly adjust the FI terminology introduced DFC, a variant of FEP. During the simulation of an experiment by [21] to categorize faults and their effects. Figure 1 depicts a, they compute a diff against the golden reference. If the diff this in a simplified diagram. We distinguish between an injected indicates exactly one-bit deviation in bit b at time t, then the fault and its effects, which means a fault might mask or activate outcome of future experiments injecting a bit-flip in b at time t to an error, after which it may either lead to the outcomes

     TABLE I: Symbols used in this work                                    μᴿ               t c₁      t c₂                          t cC       T  μᴿ     t c₁            t c₂         t cC T
 T         set of time steps of reference simulation
 IB        set of injectable states (e.g., bits)                           μᵃ                         boot                                        μᵃ              δ a          δ a
 F         set of injectable time steps I ⊂ T                              μˡ            boot                                                     μˡ     boot     t₁ᶜ           t₂ᶜ
           fault space, for single-bit faults: F      = B × I
 (α, β)ⁿ   experiment n ∈ N, with (α, β)ⁿ ∈ F                              μᵐ                                                                     μᵐ                            δ m=0
                                                                                                                                                                                   tc
 N         sample of unique experiment configs                             μᵇ                                                                                                      2
 oⁿ        N ⊂ F ∧ (α, β)ⁱ , (α, β)ʲ ∀i, j ∈ N                                                                                                    μᵇ                            δᵇₜ₂c=δᵃₜ 2c
 μⁿ        outcome of experiment n                                         Fig. 2: Checkpoint Restore Boot (CRB) (left) and CDIF (right)
           state of an experiment n at time t ∈ T
     tc    set of time steps where a checkpoint exists
 T
 ⊕μᴿᵗᶜ     state of the reference simulation at time tᶜ ∈ T c
           bit-wise exclusive OR                                           B. Fault Effect Equivalence Checks (FEEC)
           checkpoints μᴿₜc     with tᶜ ∈ T c                                                                                         In contrast to FEP and Fault Outcome Prediction (FOP),
                                                                           FEEC is not limited by analysis of the executed workload or
failure or tolerance. If detection mechanisms exist in software            knowledge of the DUT. If the states of two experiments are
or hardware, an error might be mitigated (e.g., corrected) or              equal at any point in time, their outcomes will be equivalent:
deemed unrecoverable (e.g., reset needed). Undetected, a failure                                              μax = μby                        ⇒ oa = ob     x, y ∈ T                    (3)
might also occur if the FI results in erroneous behavior (e.g.,
a Silent Data Corruption (SDC)). Lastly, an error could not                Based on (3), we could build up a database that hosts snap-
affect the system behavior or output by being masked or latent             shots of simulation states of all conducted experiments at
(benign). We will use the symbols presented in Tab. I for                  all simulated times. Although this would allow FEEC for
mathematical equations, algorithms, and results.                           future experiments, the storage and search logic costs would
A. Checkpoint Differentials (CDIF)                                         be immense. A possible solution would be to compress the
                                                                           information while retaining the capability to perform FEEC.
           Given that the simulation states for a fault-free simulation                1) Reusing Differentials:                                             We can also perform FEEC for an

exists (μR), we can compute the differential (diff) for an experiment b by finding a matching diff in the DoD ∆.The experiment a at time t with: idea is that since (1) lets us reverse from a diff back to the full δat = μat ⊕ μRt ⇒ μat = μRt ⊕ δat (1) state, we can perform (3) by comparing an active experiment’s To compute the diff, we need a reference state μRt during diff against entries in the database. However, since we are simulation. Works such as [11] simulated a fault-free pipeline computing the diffs at CRB checkpoints to allow reusing the alongside the faulty one. However, since we already have fault- checkpoints, using (1) to reverse from a diff back to a full state free states from the golden reference run that generated the fast- needed in (3) only works for equal points in time: forward CRB checkpoints, we can reuse them to compute the δax = δby ⇔ μRx ⊕ μax = μRy ⊕ μby, with x , y ∧ μRx , μRy (4) diff at the points in time where a checkpoint exists: Checkpoint ⇎ μax = μby ⇏ oa = ob Differential (CDIF). Fig. 2 (right) depicts this for a set of As an example, the accumulate register of a Multiply- checkpoints at times tc ∈ T c. Experiments a, b, l, and m choose Accumulate (MAC) unit is activated (read) at time z by a MAC checkpoints as close as possible to the FI. Next, they advance instruction. Two diffs, δa and δb, with x < z < y, might be the simulation time until it reaches another checkpoint tc where equivalent, i.e., indicate the same faulty state in the register,x y they can compute a diff against the checkpoint state μRtc per (1).

1. Masking Checks: The first trivial check that can be although experiment a will see the fault being activated at time performed is Checkpoint Masking (CMSK), e.g., the fault in z and b not because it was injected after z. By selecting an experiment m in Fig. 2 has vanished which is indicated by the equivalent point in time t ∈ T , the left-hand and right-hand side diff being 0 at t2c. The experiment can assume the reference μRt of the equivalence check cancel out and leave the states μat simulation’s outcome oR via (2): and μbt from which FEEC can be derived: δt = 0 ⇔ μt ⊕ μRt = 0 ⇔ μt = μRt ⇒ o = oR (2) δat = δbt ⇔ μRt ⊕ μat = μbt ⊕ μRt , with μRt ⊕ μRt = 0 (5) 2) Database of Diffs (DoD) ∆: Since ⊕ is symmetric, we ⇔ μat = μbt ⇒ oa = ob

can also reverse the faulty state μat from the golden state μRt Limitation: Searching for matches in the diff database ∆ is and δat (right-hand side (1)). This gives us a powerful tool to costly due to element-wise comparisons. Compressing the fault analyze fault effect propagation after the campaign. For most effect into a single value is more efficient, e.g., a hash value. experiments, the Hamming Weight lets us store individual diffs 2) Hashing Fault Effect: Let h be the value of a collision- sparsely, e.g., Coordinate list format (COO). We can describe free [22] hash function H on. Next, we can compute a signature this DoD ∆ as a set of tuples from experiment identifiers (n), ha = H(μa) of a fault effect μa. If we can not find the timestamps (t), and diff values (δ): signaturex hxax in a database containing signatures of alreadyx ∆ = {(n, tc, δntc )1, (n, tc, δntc )2, ...} with ni ∈ N ∧ tic ∈ T c conducted experiments, we add hax to the database and keep simulating until the experiment finishes. In contrast, if we can

find the signature in the campaign database, added by a previous                                                     Algorithm 1: CDIF+FEEC simulation pseudo-code
experiment, e.g., hby, we can infer FEEC from (3) via                                                             1           RunCampaign (N, S)
                                         hax = hby   ⇒ μax = μby ⇒ oa = ob,        with x, y ∈ T.          (6)         In: Fault configurations N, Simulator S
                                                                                                                       Result: Outcomes O, diffs ∆ and hashes Γ
Limitations:                                                  Hash functions are one-way, i.e., non-reversible    2    O ← ∆ ← Γ ← {}                                       ;empty sets
[23] which means we lose all information about an experiment’s                                                    3    μR, T c ← ref(S)                           ;generate checkpoints
state by hashing it. If we want to analyze the experiment,                                                        4    N ← sort(N, less)                           ;sort by earliest FI
e.g., fault propagation, we must re-execute it. Furthermore,                                                      5    foreach     fault (α, β)n   ∈ N do
hash functions are computationally expensive, so constantly                                                       6     EOS ← false                        ;init end-of-simulation flag
computing them can result in considerable overhead.                                                               7     tc     ← max({tc | tc ∈ T c     ∧ tc < βn})
3) Hash-based Differential FEEC:                                                        From Equations (4) and          boot
                                                                                                                  8     t ← tc                                 ;set simulation state to
(5), we know that differential FEEC is only safe at equivalent                                                                          boot
                                                                                                                  9     μ ← μR                          ;closest to FI checkpoint state
points in time such that simply hashing diffs H(δ                                                                                        tc
                                                                                                t) and storing   10                 tᵇᵒᵒᵗ
them without the timing information could lead to incorrect                                                      11     while        < βn ∧ ¬EOS do
matches in the database lookup. Let                                                hˆ        be the value of a   12         EOS ← advance(S)                             ;advance to FI
collision-free hash on a diff δat                        combined with a seed s:                                        μ ← μ + αn                                        ;inject fault
                                                                                                                 13     while ¬EOS do
                                                            hˆ a  = Hˆ (s, δa)                                   14
                                                               s              t                                             tnextc   ← min({tc | tc ∈ T c     ∧ tc > t})        ;select
                                                                                                                               next closest checkpoint
By choosing                                                   s to be the diffs timestamp, we can enforce that   15         while t < tc           ∧ ¬EOS do
the hashes of equal-value diffs from different points in time do                                                                         next
                                                                                                                 16                  EOS ← advance(S)
not match, even if the diffs would match:                                                                        17         if      ¬EOS then
                                                  hˆ a  , hˆ b,    with δa = δb    ∧ x , y                       18                  δ ← μ ⊕ μR                 ;diff vs sim checkpoint
Assuming the hashes of two matching diffsˆ           x  y               x     y        δat    = δbt were built   19         else                   tnextc
with H and the same time seed t, their values would match and                                                    20         hˆ     ←δH←ˆ (t μ ⊕)  μRtEOSc       ;diff vs EOS checkpoint
allow safe FEEC:                                                                                                 21                      , δ                         ;compute diff hash
                                                                                                                 22         if      ∃(r, hˆ r)i ∈ Γ,     hˆ = hˆ ri  then
    hˆ at                                    = hˆ bt ⇒ μat = μbt  ⇒ oa = ob,       with δat   = δbt              23                  on ← or                     ;reference found match
4) Database                                          of (Diff)        Hashes  Γ:   We can     describe     the   24                  EOS ← true                        ;stop simulation
Database of (Diff) Hashes (DoH) Γ as a set of tuples from                                                        25         else     Γ ← Γ ∪ {(n, hˆ )}                ;add hash to DoD
hash values ˆh and experiment identifiers (n):                                                                   26                  ∆ ← ∆ ∪ {(n, t, δ)}               ;add diff to DoH
                                                                                                                 27
    Γ = {(n, hˆ ntc )1, (n, hˆ ntc )2, ...},             with ni ∈ N ∧ tic ∈ T c                                 28                  if  EOS then
                                                                IV. Implementation                               29                                  on ← eval(μ, δ) ;determine outcome
                                                                                                                 30                 O ← O ∪ {on}              ;done. add outcome to DoO
                                                      We implemented our method, Hash-based Differential Fault
Effect Equivalence (CDIF+FEEC), in an RTL FI tool and a
server-client simulation framework.                                                                              After computing the hash (L. 21), the DoH Γ is searched for
A. Simulation Flow                                                                                               a match in L. 22. If found (L. 23-24), we can set the outcome

Our CDIF+FEEC simulation flow is described in Alg. 1. It of our experiment to the match’s and break the cooldown expects unique fault experiments N to be run on the simulator loop. If we do not find a matching hash, we add our hash S . In the preparation phase (L. 2-4), the algorithm resets its to the DoH and the diff to the DoD (L. 26-27). Unless we outputs O, ∆, and Γ, issues a fault-free reference simulation encountered EOS (L. 28), the cooldown continues with the next to generate checkpoints, and sorts the faults by their injection closest checkpoint. Otherwise, the outcome is evaluated from times (earliest first). The elaboration phase starts in L. 5, where the final state and diff (L. 29). For all experiments, we add a run is started for each fault in the campaign. lines 6-9 their respective outcome to the Database of Outcomes (DoO) describe the CRB as depicted in Fig. 2. Next, the simulator O. This can either be a reference to another experiment (FEEC) performs the warmup by advancing to FI time β (lines 10- or an actual evaluation of the final simulation state per Fig. 1. 11) and applies the fault (L. 12, shown here as an addition of the fault α). In lines 13-29, the experiment is in cooldown: B. RTL Fault Injection Simulator As long as no FEEC succeeded or the simulator reports End- The simulation basis is the open-source software (OSS) Of-Simulation (EOS), the next closest checkpoint is selected Verilator, an Verilog RTL to C++ compiler [24]. Transient and advanced to (L. 14-16). Next, the diff is computed. In FI capability is provided by another OSS vRTLmod [25] case we encounter EOS, we select the corresponding final that transforms Verilator’s output, referred to as verilated simulator state as the 2nd operand (L. 20); else, we use the RTL (vRTL), to allow injection in sequential states, i.e., next checkpoint state once the simulation advanced to it (L.18). flip-flop-level FI. We implement CRB with Verilator’s built-

server:   rsp: FI config   client:                                   TABLE II: Considered simulator features and related works
               req: jobs               active                         simulator             description            related work
configs     req: add outcome           faulty
                                       simulator                      RTL          no features                         baseline
outcomes                                                              CRB          Checkpoint Restore Boot            [17]–[19]
 diffs       req: add diff             inactive/                      MLS          CRB, Mixed-level Simulation       [11], [20]
                                       reference                      CMSK         CRB, Checkpoint Masking                 ours
 hashes   req: lookup/add hash         simulator                      CDIF+FEEC    CRB, Checkpoint Differential, and       ours
    Fault Effect Equivalence Check

Fig. 3: Simplified FI simulation server/client architecture TABLE III: Benchmark (BM) execution times |TBM | and single- in save/restore feature. This enables serialization (checkpoint bit FI fault space FBM from injection bits B and cycles IBM store) and de-serialization (restore) of a current simulation state BM |TBM| [cc] |IBM| [cc] |FBM = B × IBM| to a file. To allow generating diffs as described in Sec. III-A, aha-mont 18,616 12,646 118 · 10⁶ we extend vRTLmod tool’s code generation to automatically huffbench 292,157 285,006 2.67 · 10⁹ generate functions that compute a diff in COO format from picojpeg 757,363 750,979 7.04 · 10⁹ two instances of the now fault injectable vRTL module. This wikisort 1,692,368 1,675,850 15.7 · 10⁹ extension works on the RTL in black-box mode since vRTLmod analyzes its input vRTL for C++ variables representing sequen- tial states. Although we instantiate multiple vRTL modules for B. Experimental Results the simulator, only one is faulted and stimulated with inputs. A All simulator configurations conduct the same set of uni- second dummy instance is used to restore checkpoints (μR) for formly distributed and unique single-bit, single-cycle fault the diffs in the cooldown phase. An FI campaign is simulated experiments per BM. The goal of our experiment is not to by a server-client system sketched in Fig. 3 inspired by [26], evaluate the safety/security of the DUT but to support our where the authors proposed an open-source ISS-based fault claims regarding speed-up without accuracy loss. For this, we attack framework. Similarily, we use HDF5 [27] to manage assume the pure RTL configurations (RTL and CRB) as the our databases.Alg. 1 is mostly implemented client-side, except ground truth. Tab. IV reports the fault outcomes per BM and the FEEC check. Clients initiate it by a lookup/add hash simulator feature of equal campaigns N with 111,000 unique request to the server, which updates the DoH Γ and responds experiments. We consider the erroneous outcome classes SDC accordingly. Tab. II lists the simulator features we implemented. and Detectable Unrecoverable Error (DUE). We classify an The referenced works are not meant to be interpreted as a experiment as DUE when the CPU enters an exception or one-to-one equivalence. We implement MLS by activating the hangs (after 2× the expected cc were executed) and as an second simulator, depicted inactive in Fig. 3, and coupling it SDC when the BM output is erroneous. The Benign are all with the faulty one acting as a live tracking core and allowing experiments that did not affect the BM’s output. However, the us to switch back to ISS once a diff indicates a pure ISA fault. state of the DUT at EOS is either clean (masked), deviates in V. Evaluation execution time, or has corrupted bits in memory or registers (latent). Only MLS deviates with DUEs and SDCs from RTL. We evaluate the performance of our CDIF+FEEC method The reasons are the ISS’s inaccurate timing and a stricter bus against other techniques in a common experimental setup. protocol (I/O exceptions). For example, hang experiments at A. Experimental Setup RTL became an SDC at ISS due to the ISS executing more instructions than the RTL in the same amount of time. The pure We host the system depicted in Fig. 3 with 63 worker threads RTL simulators, including CDIF+FEEC, show a 100% match on a server with two Intel® Xeon® Silver 4314 processors at in outcomes. However, masked experiments differ due to low 2.4 GHz. As DUT, we choose the open-source 32-bit RISC- checkpoint count (e.g., aha-mont64), no masking checks at V CPU cv32e40p [28], [29] in a minimalistic SoC. The RTL ISS level, and CDIF+FEEC performing checks at EOS, which contains 9,380 sequential logic bits (B), including architectural results in no unclassified experiments remaining. and micro-architectural registers. The DUT executes a subset 1) Simulation Performance: From Tab. IV, we can see that of EmbenchTM (Tab. III). Usually, the benchmarks (BMs) run for the non-FEEC simulators (CMSK and MLS), about 50% of multiple rounds to estimate a CPU’s performance, which would experiments have to finish simulating. This rate can be reduced result in additional masking and most BMs having a similar by deploying FEEC, which automatically filters out many latent runtime. Therefore, we reduce each BM’s round count to fault effects. Next, we evaluate the performance w.r.t. to size of one. The number of inject-cycles (|IBM |) is smaller than the campaigns. We do this by issuing the same uniformly sampled respective BM’s execute-cycles (|TBM |) because we do not inject configurations N. As Tab. V reports, the average simulation in boot and return code. For single-bit FI, the fault space per times per experiment (ts) of non-FEEC features are not affected BM can be expressed as a Cartesian product of inject-cycles and by the sample size, e.g., a single wikisort experiment at pure bits. Checkpoints are placed every 10,000 clock cycles (ccs), RTL takes 63.9 s to complete for |N| = 100, 000. Except for e.g., aha-mont only has one and wikisort has 169. aha-mont64, we observed a speed-up of roughly 2× when

TABLE IV: Classification in erroneous (SDC, DUE) and be-  TABLE V: Performance in average experiment simulation time

nign outcomes for equal experiments but different simulators (ts) in seconds and speed-up factor vs. baseline RTL (×).

|N| :111,000  erroneous        benign                               |N| : 10,000  100,000     1,000,000
simulator   SDC   DUE    mask  time  latent      unclass   simulator  ts[s]    ×  ts[s]     ×  ts[s]              ×
RTL        5461   2783      0     0       0       102756   RTL         0.74    -   0.73     -
CRB        5461   2783      0     0       0       102756   CRB         0.55  1.3   0.55   1.3
CMSK       5461   2783  27738     0       0        75018   CMSK        0.47  1.6   0.49   1.5
MLS        5461   2783  47591     0       0        55165   MLS         0.60  1.2   0.60   1.2   0.60           ∼1.2
CDIF+FEEC  5461   2783  56485   188   46083            0   CDIF+FEEC   0.47  1.6   0.39   1.9   0.40           ∼1.8
RTL        2224   4807      0     0       0       103969   RTL         11.7    -   11.6     -
CRB        2224   4807      0     0       0       103969   CRB          6.1  1.9    6.1   1.9
CMSK       2224   4807  48620     0       0        55349   CMSK         3.7  3.2    3.8   3.0
MLS        2229   4795  47445     0       0        56531   MLS          3.0  3.9    3.0   3.9
CDIF+FEEC  2224   4807  53984   310   49675            0   CDIF+FEEC    3.1  3.8    1.5   7.6    1.0            ∼11
RTL         676   2559      0     0       0       107765   RTL         29.9    -   27.4     -
CRB         676   2559      0     0       0       107765   CRB         15.0  2.0   15.0   1.8
CMSK        676   2559  56661     0       0        51104   CMSK         7.7  3.9    8.1   3.4
MLS         681   2553  48560     0       0        59206   MLS          6.6  4.5    6.8   4.0
CDIF+FEEC   676   2559  59357  1719   46689            0   CDIF+FEEC    5.9  5.1    2.4    11    1.3            ∼21
RTL         330   2864      0     0       0       107806   RTL         65.6    -   63.9     -
CRB         330   2864      0     0       0       107806   CRB         33.2  2.0   33.4   1.9
CMSK        330   2864  58141     0       0        49665   CMSK        16.8  3.9   17.3   3.7
MLS         328   2996  49159     0       0        58517   MLS         14.2  4.6   14.2   4.5
CDIF+FEEC   330   2864  59865  2072   45869            0   CDIF+FEEC   12.8  5.1    5.0  12.8   2.53            ∼25

enabling CRB, which is as expected and consistent with related of 2 to 5. While maintaining 100% accuracy, we can recover works, such as [18], [19]. This can be explained by the uniform faulty states through the stored checkpoint differentials, which distribution of the sample N where, on average, experiments can be used for detailed analysis, such as fault propagation. start at half BM runtime 0.5|TBM |, which means, on average, only the remaining half has to run. Masking checks (CMSK) can speed-up by another factor of about 2× to 4× vs. RTL. RTL This can be explained by the fact that the speed-up seems 60 CRB CMSK proportional to the masking rate (50% → 2×, see Tab. IV). 50 MLS MLS allows a total speed-up of around 4.6×. Enabling FEEC CDIF+FEEC ours |N|=10000 has the most significant effect on the simulation performance, 40 CDIF+FEEC ours |N|=100000 CDIF+FEEC ours |N|=1000000 where CDIF+FEEC outperforms all other considered simu- 30 lators. For example, the largest sample on the longest BM 20 (wikisort, |N| = 1, 000, 000) has a reduction of ts to 2.53 s, which means an estimated speed-up of 25× vs. RTL. Fig. 4 10 plots the measurements for ts from Tab. V over the BM’s 0 executed cc per experiment. CDIF+FEEC’s performance is split 0 200 400 600 800 1000 1200 1400 1600 for different sample sizes |N|. In contrast, Fig. 5 plots ts per benchmark execute-cycles |TBM|[1 · 10³ cc] BM over a normalized sample size. As expected, for all non- Fig. 4: Average simulation time per experiment ts over bench- FEEC features, ts is constant, but there is a downward trend mark execute-cycles |TBM | in ccs for all considered simulators for increasing sample sizes for CDIF+FEEC: The probability of later experiments matching increases with the number of CMSK fault effects in the DoH, which is higher for larger samples. MLS 15 CDIF+FEEC ours VI. Conclusion wikisort We present a fast and accurate speed-up technique to ac- 10 celerate RTL FI simulations. Our method of Hash-based Dif- picojpeg ferential Fault Effect Equivalence (CDIF+FEEC) can be used without intricate knowledge of the DUT (black box) or addi- 5 tional analysis. To demonstrate the approach, we implemented huffbench CDIF+FEEC in vRTLmod [25], [30], an open-source RTL 0 aha-mont FI tool and a server-client simulation framework simulating 10−5 10−4 10−3 10−2 a RISC-V-based SoC. We measured a speed-up of up to 25 normalized sample size |N|/|FBM|[%] times against pure RTL simulation, significantly outperforming Fig. 5: Average simulation time per experiment ts over chosen various existing methods, which we benchmarked at a factor sample size |N| normalized for benchmark fault space size |FBM |

                                                                                  wikisort picojpeg huffbench        aha-mont BM

experiment avg. sim. time t s [s] experiment avg. sim. time t s [s]

                                                                                  wikisort picojpeg huffbench        aha-mont BM

      References                                                                [22]  A. Russell, “Necessary and sufficient conditions for collision-free hash-

[1] R. Baumann, “Soft errors in advanced computer systems,” IEEE Design [23] ing,” Journal of Cryptology, vol. 8, p. 87–99, 1995. & Test of Computers, vol. 22, no. 3, pp. 258–266, 2005. Y. Zheng, T. Matsumoto, and H. Imai, “Structural properties of one-way [2] E. H. Ibe et al., “Radiation-induced soft errors,” in VLSI Design and Test hash functions,” in Advances in Cryptology-CRYPTO’ 90, A. J. Menezes for Systems Dependability, S. Asai, Ed. Tokyo: Springer Japan, 2019, [24] and S. A. Vanstone, Eds. Springer Berlin Heidelberg, 1991, pp. 285–302. pp. 57–127. W. Snyder, P. Wasson, D. Galbi, and et al. Verilator. [Online]. Available: [3] A. Dehbaoui, J.-M. Dutertre, B. Robisson, P. Orsatelli, P. Maurine, [25] J.https://github.com/verilator/verilator and A. Tria, “Injection of transient faults using electromagnetic pulses Geier and D. Mueller-Gritschneder, “vrtlmod: An llvm based Practical results on a cryptographic system,” 2012, journal of Cryptology open-source tool to enable fault injection in verilator rtl simulations,” ePrint Archive: Report 2012/123. in Proceedings of the 20th ACM International Conference on Computing Frontiers, ser. CF ’23. New York, NY, USA: Association [4] N. Moro et al., “Electromagnetic fault injection: Towards a fault model for Computing Machinery, 2023, p. 387–388. [Online]. Available: on a 32-bit microcontroller,” in Proceedings - 10th Workshop on Fault https://doi.org/10.1145/3587135.3591435 Diagnosis and Tolerance in Cryptography, FDTC 2013, 2013, pp. 77–88. [26] F. Hauschild, K. Garb, L. Auer, B. Selmke, and J. Obermaier, “ARCHIE: [5] J. G. van Woudenberg, M. F. Witteman, and F. Menarini, “Practical optical A QEMU-based framework for architecture-independent evaluation of fault injection on secure microcontrollers,” in 2011 Workshop on Fault faults,” in 2021 Workshop on Fault Detection and Tolerance in Cryptog- Diagnosis and Tolerance in Cryptography, 2011, pp. 91–99. raphy (FDTC), 2019, pp. 20–30. [6] B. Selmke, J. Heyszl, and G. Sigl, “Attack on a dfa protected aes by [27] S. Koranne, “Hierarchical data format 5: HDF5,” in Handbook of Open simultaneous laser fault injections,” in 2016 Workshop on Fault Diagnosis Source Tools. Boston, MA: Springer US, 2011, pp. 191–200. and Tolerance in Cryptography (FDTC), 2016, pp. 36–46. [28] M. Gautschi et al., “Near-threshold RISC-V core with DSP extensions for [7] International Organization for Standardization, “ISO 26262:2018 (all scalable IoT endpoint devices,” IEEE Transactions on Very Large Scale parts): Road vehicles - functional safety,” 2018. Integration (VLSI) Systems, vol. 25, no. 10, pp. 2700–2713, 2017. [8] H. Cho, S. Mirkhani, C. Cher, J. A. Abraham, and S. Mitra, “Quantitative [29] OpenHW Group. (2024) CV32E40P - GitHub repository. [Online]. evaluation of soft error injection techniques for robust system design,” in Available: https://github.com/openhwgroup/cv32e40p Proceedings of the 50th Annual Design Automation Conference, ser. DAC [30] vRTLmod. (2024). [Online]. Available: https://github.com/tum-ei- ’13. New York, NY, USA: Association for Computing Machinery, 2013. eda/vrtlmod [9] M. Maniatakos, N. Karimi, C. Tirumurti, A. Jas, and Y. Makris, “Instruction-level impact analysis of low-level faults in a modern micro- processor controller,” IEEE Transactions on Computers, vol. 60, no. 9, pp. 1260–1273, 2011. [10] M. Maniatakos, C. Tirumurti, A. Jas, and Y. Makris, “AVF analysis acceleration via hierarchical fault pruning,” in 2011 Sixteenth IEEE European Test Symposium, 2011, pp. 87–92. [11] D. Mueller-Gritschneder, U. Sharif, and U. Schlichtmann, “Performance and accuracy in soft-error resilience evaluation using the multi-level processor simulator ETISS-ML,” in 2018 IEEE/ACM International Con- ference on Computer-Aided Design (ICCAD). IEEE Press, 2018, p. 1–8. [12] R. Leveugle, A. Calvez, P. Maistri, and P. Vanhauwaert, “Statistical fault injection: Quantified error and confidence,” in 2009 Design, Automation & Test in Europe Conference & Exhibition, 2009, pp. 502–506. [13] I. Tuzov, D. De Andr´es, and J. C. Ruiz, “Accurate robustness assessment of hdl models through iterative statistical fault injection,” Proceedings - 2018 14th European Dependable Computing Conference, EDCC 2018, pp. 1–8, 2018. [14] S. K. S. Hari, S. V. Adve, H. Naeimi, and P. Ramachandran, “Relyzer: exploiting application-level fault equivalence to analyze application re- siliency to transient faults,” in Proceedings of the Seventeenth Interna- tional Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS XVII. New York, NY, USA: Association for Computing Machinery, 2012, p. 123–134. [15] J. Li and Q. Tan, “Smartinjector: Exploiting intelligent fault injection for sdc rate analysis,” in 2013 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFTS), 2013, pp. 236–242. [16] H. Schirmeier, C. Borchert, and O. Spinczyk, “Rapid fault-space ex- ploration by evolutionary pruning,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8666 LNCS, pp. 17–32, 2014. [17] L. Berrojo et al., “New techniques for speeding-up fault-injection cam- paigns,” in Proceedings 2002 Design, Automation and Test in Europe Conference and Exhibition, 2002, pp. 847–852. [18] J. Na and D. Lee, “Acceleration of simulated fault injection using a checkpoint forwarding technique,” ETRI Journal, vol. 39, no. 4, pp. 605– 613, 2017. [19] B. Tabacaru, M. Chaari, W. Ecker, T. Kruse, and C. Novello, “Efficient checkpointing-based safety-verification flow using compiled-code simu- lation,” in 2016 Euromicro Conference on Digital System Design (DSD), 2016, pp. 364–371. [20] H. Cho, E. Cheng, T. Shepherd, C.-Y. Cher, and S. Mitra, “System- level effects of soft errors in uncore components,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 36, no. 9, pp. 1497–1510, 2017. [21] J. Arlat, A. Costes, Y. Crouzet, J. Laprie, and D. Powell, “Fault injection and dependability evaluation of fault-tolerant systems,” IEEE Transac- tions on Computers, vol. 42, no. 8, pp. 913–923, 1993.