Overview

A fault model in the evidence is the explicit set of fault hypotheses that a generation or testing technique is designed to expose or instantiate. The hypothesis can be hardware-level (e.g., single and double bit flips, stuck-at faults, complex multi-bit faults induced by neutron irradiation), microarchitecture-level (e.g., register, operation, pipeline-path, and inter-operation errors), or interaction-level (e.g., contract-level faults in Systems of Systems). [C1][C2][C3][C4][C8][C9]

Fault models in directed, model-based generation

The MicroTESK survey paper describes a method for directed test program generation that takes detailed microprocessor specifications written in the EXPRESSION language and translates them into an SMV description. The specifications capture the structure of the design (components and interconnections), its behaviour (instruction semantics), and the mapping between structure and behaviour. The key part of this work is a fault model that describes typical errors for registers, individual operations, pipeline paths, and interactions between several operations. For each element of the fault model, a set of concrete properties is generated; each property is covered by a test case constructed by SMV as a counterexample for the negation of the property. The authors state that this technique does not scale well on complex microprocessor designs, and they therefore suggest using a template-based approach as an addition. Both approaches rely on accurate specifications and are best applied in late design stages when the microarchitecture is stable. [C1]

The pipeline-hazards-template paper describes the same EXPRESSION/SMV approach in its related-work section: basing on the specification, a generic structure–behaviour model in SMV is derived; for this model, the test developer defines a fault model; for each fault-model element the negation of the corresponding property is produced; then a counterexample is generated using the SMV model checker. The authors again note that model checking does not scale on complex designs, so a template-based technique is used as an addition. Templates are developed by hand and describe sequences of instructions that create special situations in the microprocessor's behaviour (pipeline hazards in the first place). Generation is performed with the help of the graph model extracted from the specification. [C2]

Fault models in OSM (Operation State Machine) modelling

A related approach specifies the pipeline structure as a state machine, called an Operation State Machine (OSM). The OSM describes control logic at two levels: operational and hardware. At the operational level the "movement" of instructions through pipeline stages is described, with each operation represented by a separate state machine. At the hardware level, hardware resources are modelled using token managers; an OSM changes states by capturing and moving tokens. A pipeline model is a composition of operation state machines and resource state machines. The testing goal is to traverse all transitions of the joint automaton. Like the EXPRESSION/SMV approaches, this is based on accurate specifications and is best applied in late design stages when the microarchitecture is stable. [C3]

Fault models in template-based and combinatorial generation (Genesys-Pro, MicroTESK)

The template-based paper introduces Genesys-Pro (IBM Research), whose generator is composed of two main components: an engine (independent of target architecture) and a model (carrying microprocessor-specific knowledge). A verification engineer develops templates that specify the structure of test programs and the properties they should satisfy; Genesys-Pro transforms templates into executable tests. The template-based technique requires greater manual effort but scales well, in contrast to the model-checking approach. [C3]

The MicroTESK survey discusses other generation techniques whose fault assumptions differ from those of the model-based methods. Random generation is described as the most common technique for producing complex, though unsystematic, test programs; it can detect some high-quality bugs but imposes significant workload. RAVEN (Random Architecture Verification Engine), developed by Obsidian Software Inc. (later acquired by ARM), is cited as one of the most famous random generators; to produce tests it applies randomization while also taking into account information about common microprocessor faults. RAVEN is built upon pre-developed and custom-made modules that can be included in the generator to expand its functionality, although public technical details are reported as unclear. Combinatorial generation systematically enumerates short instruction sequences (2–5 instructions), including situations for individual instructions and dependencies; many microprocessor errata can be detected by such small cases. This technique was implemented in the first version of MicroTESK (ISPRAS), which supports hierarchical decomposition of a test-program generator into iterators (each responsible for its own part) and combinators (combining inner iterators into complex test sequences), and which can also construct test programs with branch instructions by enumerating control-flow graphs and exploring execution traces by bounded depth-first search. Template-based generation treats a template as an abstract representation of a test program, where constraints specify possible values of test situations; this is the core of MicroTESK. [C4]

Fault models in constraint-solving test data generation (HOL-TestGen, VAMP)

In the HOL-TestGen VAMP case study, test data generation is performed by constraint solving and random instantiation. This produces test sequences with coarsely grained memory access; under those conditions the underlying fault model is described as somewhat arcane, with the example of interferences of operations in distant memory areas. If those faults are the target, the paper recommends choosing a denser test method. [C5]

To refine the implicit fault model, additional constraints are introduced to reduce the uniformity domain: bounding the range of addresses used in generated sequences; defining a used-predicate over input sequences that records the set of addresses written by store operations and constrains load operations to that set; or analogous predicates that divide the uniformity domain into interesting sub-domains. These refinements also improve coverage of selected data. [C5]

The case study contrasts implementation-based testing with specification-based testing: stuck-at-fault analysis is described as a white-box mutation technique that uses the structure of the implementation under test — mutators are introduced that capture a fabrication fault model such as broken wires between gates — to construct equivalence-class tests that directly incorporate a fault model. Stuck-at-fault techniques are reported to be effective for medium-size circuits but are also said to be incapable of revealing some design-level flaws, such as a write–read error influenced by byte alignments in memory. [C6]

Although the authors had access to a VAMP gate-level model, the study stayed at the design level of the VAMP machine; the equivalence classes underlying the tests could be refined further by making HOL-TestGen explore byte- or bit-level representations of registers and memory cells, rather than relying on a coarse hypothesis such as one successful write–read of a memory cell standing for all write–reads in that cell. [C7]

Fault models in GPU / DNN reliability evaluation

The DNN-on-GPU paper contrasts software-level fault injection with physical (neutron-beam) fault injection. Software-level fault injection is reported to be a common and fast way to evaluate reliability but to potentially produce unrealistic results because it has limited access to hardware resources and adopts fault models that may be too naive (i.e., single and double bit flips). Physical fault injection with a neutron beam provides realistic error rates but lacks fault-propagation visibility. [C8]

The paper proposes a characterisation of the DNN fault model that combines neutron-beam experiments with software-level fault injection. GPUs running GEMM and DNNs are exposed to beam neutrons to measure their error rate; on DNNs the percentage of critical errors can be up to 61%, and ECC is shown to be ineffective in reducing critical errors. A complementary software-level fault injection is then performed using fault models derived from RTL simulations. With these complex fault models, the YOLOv3 misdetection rate is validated to be very close to the rate measured with beam experiments, which is reported as 8.66× higher than the rate measured with fault injection using only single-bit flips. [C8]

Fault models in Systems of Systems contracts

In the SoS-contract paper, the contractual specifications previously defined for constituent systems of Systems of Systems are extended with explicit fault modelling concepts. The extension introduces fault modelling with respect to an Audio Visual SoS case study from Bang and Olufsen and discusses how it relates to previous work on modelling faults in SoSs, where previous contractual work had not given explicit consideration to faults. [C9]

Synthesis

Across the evidence, a fault model is a test-side abstraction: it enumerates the concrete fault hypotheses the technique can target — whether hardware faults (stuck-at, bit flips, complex neutron-induced patterns), microarchitectural faults (register, operation, pipeline-path, inter-operation errors, pipeline-hazard situations), interaction faults (SoS contract-level), or implicit assumptions arising from how test data is generated (coarse memory access, distant-area interferences). The choice of generation strategy (random, combinatorial, template-based, model-based / model-checker-based, OSM-based, or constraint-solving / specification-based) determines which fault model the technique can actually instantiate and therefore how its coverage must be interpreted. [C1][C2][C3][C4][C5][C6][C7][C8][C9]