Skip to content
STIMSMITH

Bug Curve

Concept WIKI v1 · 7/18/2026

A bug curve is a plot of the cumulative number of bugs found in a design under test (DUT) over time, used to track verification progress. In hardware verification projects it is typically expected to follow an S-shape: a slow start while the test bench is constructed and debugged, a steep rise as testing of the DUT begins and new areas are exercised (often also uncovering test bench bugs), and finally a flattening out as the test bench and DUT reach maturity.

Bug Curve

Definition

A bug curve is a plot of the cumulative number of bugs discovered in a design under test (DUT) against time (or verification effort). It is used as a project-management and sign-off artefact during hardware functional verification.

Expected Shape

Bug find rates are often expected to follow an S-curve shape:

  • The number of bugs found rapidly increases as verification effort is increased.
  • The rate gradually reduces as the DUT reaches maturity.
  • If the curve does not level out, this is an indication that more time is needed to debug the design.

A cumulative total flattening out completely for a sufficient time can be used as a sign-off criterion, because it suggests that maturity has been reached.

Typical Life-Cycle Stages Visible in a Bug Curve

  1. Construction and debug of the test bench. At the beginning of a project the test bench is built and debugged; few DUT bugs are observed during this phase.
  2. DUT testing begins. Once DUT testing starts, the curve moves rapidly upward. Bugs are found across all parts of the design.
  3. New DUT functionality is tested. As new areas of the DUT functionality are exercised, new areas of the verification IP are also tested, which can produce additional test-bench bugs and further steepen the curve.
  4. Maturity. The test bench reaches maturity, and the DUT reaches maturity shortly afterwards, producing the characteristic flattening tail.

How Bug Curves Are Generated in Practice

In the reported verification studies, bugs were recorded using an issue-tracking system that captured statistics such as:

  • the cumulative number of bugs found,
  • the number fixed, and
  • whether each bug occurred in the design or in the test bench.

These statistics were then plotted as the bug curve over the project duration.

Example: LFI-S Bug Curve

The LFI-S project produced a bug curve (Figure 2 in the source paper) that clearly showed the progression of the project through its life cycle:

  • Initial slow start while the test bench was constructed and debugged.
  • A rapid upward movement as DUT testing began.
  • Continued steep rises as new DUT functionality was tested and previously-untested verification IP was exercised, producing both DUT and test-bench bugs.
  • A flattening tail as the test bench reached maturity and the DUT followed shortly after.

The flattening of the cumulative total was used as a sign-off criterion for the LFI-S project.

Related Concept: Cumulative Bug Find Rate

The cumulative bug find rate is the metric plotted on the vertical axis of a bug curve. Separate cumulative bug find rates can be tracked for individual test bench configurations (e.g., LMB Testbench, FPI Testbench, LFI-S, LFI-S TB) as well as for the overall project.

Relation to Bug Tracking and Sign-Off

Because the S-shape assumes that early-phase bug discovery is dominated by test-bench bring-up and mid-phase discovery is dominated by DUT bugs, the bug curve is interpreted together with a classification of bugs (design vs. test bench) when deciding whether to stop verification. A long, stable plateau is taken as evidence that the design is mature enough to release.

LINKED ENTITIES

1 links
A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking MENTIONS

CITATIONS

5 sources
5 citations
[1] Bug find rates are often expected to follow an S-curve shape: the number of bugs found rapidly increases as verification effort is increased, and gradually reduces as the DUT reaches maturity. A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking
[2] If the levelling out in the bug curve is not seen, it can be an indication that more time is needed to debug the design. A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking
[3] A cumulative total flattening out completely for a sufficient time can be used as a sign-off criterion, as it suggests that maturity has been reached. A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking
[4] Bugs were recorded using an issue tracking system that allowed statistics to be gathered such as the cumulative number of bugs found, the number fixed, and whether they occurred in the design or test bench. A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking
[5] Figure 2 (LFI-S Bug Curves) shows the bug curve seen during the LFI-S project and clearly shows the progression of the project through its life cycle: test bench construction/debug, rapid upward movement as DUT testing begins, continued increases as new DUT functionality and verification IP areas are tested, and flattening as the test bench and DUT reach maturity. A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking