Combinatorial Coverage

Concept

Combinatorial coverage is a testing and validation methodology that measures the extent to which all possible combinations of input parameter values have been exercised by a test set. Originating in combinatorial interaction testing for software, it has been applied to hardware (post-silicon SoC) validation and to characterize training and test data for machine learning models, including for out-of-distribution detection.

First seen 6/16/2026

Last seen 6/16/2026

Evidence 1 chunks

Wiki v1

WIKI

Combinatorial Coverage

Definition

Combinatorial coverage is a measure of the proportion of all possible combinations of input parameter values that have been exercised by a given test set or data sample. The concept originated in combinatorial interaction testing (CIT) in software testing, where it is used to identify faults by systematically varying parameter combinations. In its simplest form, combinatorial coverage reaches 100% when every possible combination of values for the parameters under test has been applied. For example, a two-input NAND gate's combinatorial coverage reaches 100% when all four possible input combinations (00, 01, 10, 11) are applied.

READ FULL ARTICLE →

NEIGHBORHOOD

No graph connections found for this entity yet. It may appear in future ingestion runs.

explore full graph →

LINKED ENTITIES

1 links

coverage metrics PART_OF Extracted graph relationship

CITATIONS

5 sources

5 citations — click to expand

[1] A two-input NAND gate's combinatorial coverage reaches 100% when all possible four combinations of the input are applied. Unified Coverage Methodology for SoC Post-Silicon Validation

[2] Standardized coverage metrics are an emerging research area in post-silicon validation, and pre-silicon code-based coverage metrics such as statement, branch, and path coverage are not directly applicable because silicon houses hardware logic rather than RTL code. Unified Coverage Methodology for SoC Post-Silicon Validation

[3] Combinatorial interaction testing from software testing can be adapted to systematically select and characterize test and training sets for machine learning models, demonstrated on MNIST, by considering coverage over simple features derived from inputs and outputs rather than model internals. Systematic Training and Testing for Machine Learning Using Combinatorial Interaction Testing

[4] Combinatorial coverage metrics have been explored as an alternative to distribution-based metrics and can correlate with classification error, but their utility is highly dataset-dependent. Metric Learning Improves the Ability of Combinatorial Coverage Metrics to Anticipate Classification Error

[5] Metric learning, a technique for learning latent spaces where data from different classes is further apart, increases the difference between set-difference coverage metrics (SDCCMs) calculated on correctly and incorrectly classified data across six open-source datasets, with paired t-tests validating statistical significance. Metric Learning Improves the Ability of Combinatorial Coverage Metrics to Anticipate Classification Error