Instruction Distillation

Instruction distillation is a technique for reducing the length of instruction sequences used as inputs in RTL-level hardware fuzzing while preserving their usefulness for bug detection and coverage exploration.^[1] In the context of the RTL fuzzer INSTILLER—short for Instruction Distiller—instruction distillation is used to keep fuzzing inputs short because prior CPU fuzzing approaches can generate increasingly long RTL input instruction sequences, and longer inputs are not necessarily more effective for fuzzing.^[1]

Background

Hardware bugs, including CPU bugs, differ from software bugs in that they generally need to be detected before deployment.^[1] RTL fuzzing targets hardware designs at the register-transfer level and attempts to generate inputs that expose mismatches, bugs, or unusual execution behavior before the hardware is manufactured or released.

Previous CPU RTL fuzzing work had several identified limitations. A key problem is that generated RTL input instruction sequences tend to grow in length over the course of fuzzing, which reduces efficiency because longer inputs are not necessarily more effective at finding new bugs.^[1] Instruction distillation directly targets this growth.

INSTILLER and Instruction Distillation

INSTILLER is an RTL fuzzer based on ant colony optimization (ACO).^[1] Its core contribution related to instruction distillation is the use of a variant of ACO called VACO to distill input instructions so that fuzzing inputs remain short and efficient.^[1]

In addition to instruction distillation, INSTILLER addresses two other fuzzing concerns:

1. Realistic interruption and exception modeling — prior work did not simulate realistic interruptions well, so INSTILLER includes mechanisms for inserting interruptions and exceptions when generating fuzzing inputs.^[1]
2. Hardware-based fuzzing improvements — INSTILLER uses hardware-based seed selection and mutation strategies to further improve fuzzing performance.^[1]

The INSTILLER paper was authored by Gen Zhang, Pengfei Wang, Tai Yue, Danjun Liu, Yubei Guo, and Kai Lu, and was published on arXiv in January 2024 with subjects classified under Cryptography and Security (cs.CR) and Software Engineering (cs.SE).^[2][3]

Method

Within INSTILLER, instruction distillation is performed using VACO, a variant of ant colony optimization.^[1] The purpose of the algorithm is to reduce the size of RTL instruction inputs while retaining fuzzing effectiveness. By keeping instruction sequences short, INSTILLER aims to improve execution speed and avoid wasting fuzzing effort on excessively long inputs.^[1]

The available evidence describes VACO as a mechanism that "distills input instructions," but does not provide implementation-level details such as pheromone update rules, cost functions, or selection probabilities.^[1]

Evaluation Results

INSTILLER was evaluated against state-of-the-art RTL fuzzing work on real-world target CPU cores.^[1] Reported results include:

These results suggest that instruction distillation can reduce RTL fuzzing input length substantially while also improving coverage, mismatch detection, and execution speed in the evaluated setting.^[1]

Significance

Instruction distillation is significant because it targets a practical bottleneck in CPU RTL fuzzing: the growth of input instruction sequences over time. By shortening inputs through VACO, INSTILLER improves fuzzing efficiency while maintaining or increasing effectiveness in terms of coverage and mismatch detection.^[1]

References

^[1]: "INSTILLER: Towards Efficient and Realistic RTL Fuzzing," arXiv:2401.15967, 2024. ^[2]: arXiv metadata for INSTILLER paper (authors: Gen Zhang, Pengfei Wang, Tai Yue, Danjun Liu, Yubei Guo, Kai Lu). ^[3]: arXiv subject classification for INSTILLER: Cryptography and Security (cs.CR); Software Engineering (cs.SE).