Skip to content
STIMSMITH

Fuzzing

Technique WIKI v3 · 8/12/2026

Fuzzing, in the provided ARM-simulator testing artifact, is an AFL++-based technique that runs an instrumented gem5 ARM simulator against inputs from prepared seed corpora, uses SearchSYS custom mutators in custom-mutator-only mode, records queue/crash/hang outputs and fuzzer_stats, and then supports differential testing of fuzzed queue seeds against native execution.

Fuzzing

Definition in this artifact

In the Search+LLM-based ARM simulator testing artifact, fuzzing is implemented as an AFL++ workflow over an instrumented ARM build of gem5. The fuzzer is run with an input corpus, an output directory for fuzzed results, and three SearchSYS custom mutator libraries: cm-gem5c.so, cm-gem5c-bin.so, and cm-gem5c-types.so [C1].

The workflow is explicitly configured for custom-mutator-only fuzzing via AFL_CUSTOM_MUTATOR_ONLY=1; in the AFL++ status display, built-in mutation strategies such as bit flips, byte flips, arithmetics, and known integers are shown as disabled in custom-mutator-only mode [C2].

Target and instrumentation

The target system under test is an ARM build of gem5. The artifact builds AFL++ first, then builds gem5 with AFL++ instrumentation by setting CC, CXX, and LINK to AFL++ compiler wrappers and invoking scons build/ARM/gem5.opt [C3]. The instructions emphasize checking that the build is for ARM rather than X86 [C3].

A representative fuzzing command runs AFL++ with:

  • a corpus input directory, such as $corpus_folder/input/;
  • an output directory, such as $corpus_folder/output-fuzzed/;
  • a large AFL map size (AFL_MAP_SIZE=1200000);
  • a memory limit (-m 50000);
  • a timeout (-t 99000);
  • the ARM gem5 binary as the target executable;
  • hello-custom-binary-Ex.py --isa ARM --input @@ as the gem5 workload invocation [C1].

Seed corpora and preparation

The artifact prepares model-specific corpora before fuzzing. It compiles C programs with gcc -O3, moves binaries into model-specific binary/ directories, copies those binaries into corresponding *-cmin corpus directories, and rewrites input text paths to point to the minimized corpus directory names [C4]. The corpora shown include CodeBooga, Llama, Magicoder, Phi, TinyLlama, gpt3.5-new, and gpt3.5-old variants [C4][C5].

Custom mutators

The fuzzing setup loads SearchSYS custom mutators through AFL_CUSTOM_MUTATOR_LIBRARY, with the libraries separated by semicolons. AFL++ reports successful discovery of afl_custom_mutator and afl_custom_fuzz_count symbols and successful installation of the custom mutator libraries [C1][C6].

The AFL++ interactive status display can show the currently active custom mutator stage, for example cm-gem5c-types.so, and reports custom-mutator-only operation where standard AFL mutation stages are disabled [C2].

Batch fuzzing experiments

Full paper results are produced by batch runs using Experiment-ARM-24h.sh. Each repeat is run for each initial corpus; the artifact lists invocations such as:

./Experiment-ARM-24h.sh CodeBooga-cmin 1
./Experiment-ARM-24h.sh gpt3.5-new-cmin 1
./Experiment-ARM-24h.sh gpt3.5-old-cmin 1
./Experiment-ARM-24h.sh Llama-cmin 1
./Experiment-ARM-24h.sh Magicoder-cmin 1
./Experiment-ARM-24h.sh Phi-cmin 1
./Experiment-ARM-24h.sh TinyLlama-cmin 1

and analogous runs for later repeats [C5]. The scripts are described as ensuring a 24-hour run per repeat and input corpus [C5].

After each run, the artifact analyzes the sizes of the queue, crash, and hang folders, with additional statistics available in the fuzzer_stats file [C5]. The artifact notes that fuzzing is random, so Table III results can differ slightly between repeats of the same initial corpus [C5].

Differential testing of fuzzed seeds

The artifact also performs differential testing on fuzzed queue outputs. It reuses the differential-testing scripts on AFL++ output queues, for example:

./1-run-diff-test.sh /home/debian/experiment/CodeBooga-cmin/output-r_5/default/queue > Fuzzed-diff-testing-CodeBooga-5.log 2>&1

with analogous commands for Llama, Magicoder, Phi, TinyLlama, gpt3.5-new, and gpt3.5-old [C7]. The artifact states that bug identification from this analysis is manual and requires a system simulator expert [C7].

An example comparison shows gem5 simulator output versus a native run. In the shown case, the observable program output matches the native run, so no bug is exposed by that seed [C8].

Operational signals

During a sample interactive run, AFL++ reports calibration warnings, corpus counts, map coverage, saved crashes, total crashes, timeouts, execution speed, and custom-mutator stage progress [C2]. These status fields are used operationally to monitor fuzzing progress and outcomes in this workflow.

LINKED ENTITIES

5 links
Search+LLM-based Testing for ARM Simulators USES
SearchSYS IMPLEMENTS
seed corpus USES
Custom Mutator USES
Experiment-ARM-24h.sh IMPLEMENTS

CITATIONS

9 sources
9 citations
[1] AFL++ fuzzing is configured with SearchSYS custom mutator libraries, corpus input/output directories, gem5 as target, and ARM workload invocation. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[2] The fuzzing run uses custom-mutator-only mode, and standard AFL mutation stages are disabled in that mode. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[3] The artifact builds AFL++ and then builds an ARM gem5 target with AFL++ compiler-wrapper instrumentation. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[4] Model-specific corpus preparation compiles C programs, copies binaries into cmin corpus directories, and rewrites input paths for models including Magicoder, TinyLlama, Phi, gpt3.5-new, and gpt3.5-old. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[5] Full fuzzing results are produced with Experiment-ARM-24h.sh over multiple initial corpora and repeats, and each script run lasts 24 hours per repeat and input corpus. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[6] AFL++ reports successful loading and installation of custom mutator libraries by finding afl_custom_mutator and afl_custom_fuzz_count. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[7] Run analysis examines queue, crash, and hang folders, with additional statistics in fuzzer_stats, and fuzzing randomness can cause slight differences between repeats. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[8] Differential testing is applied to fuzzed queue outputs and bug identification is manual and requires a system simulator expert. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo
[9] An example differential comparison shows matching gem5 and native outputs, so no bug is exposed in that case. Artifact of Search+LLM-based Testing for ARM Simulators - Zenodo

VERSION HISTORY

v3 · 8/12/2026 · gpt-5.5 (current)
v2 · 6/26/2026 · minimax/minimax-m3
v1 · 6/24/2026 · minimax/minimax-m3