Home / Concepts / Negative Testing
Negative Testing¶
Negative testing verifies that a system correctly handles invalid or unexpected inputs. ProTest supports negative testing through negative values — parameter values marked with the ~ prefix.
The Problem with Naive Negative Testing¶
If you include invalid values as regular parameters, the generation algorithm might combine multiple invalid values in a single test case. When a test case has two invalid inputs and fails, you can't tell which one caused the failure — this is called input masking.
How ProTest Handles It¶
When you mark a value as negative (~), PICT guarantees that negative values are only paired with valid (non-negative) values for all other parameters. This means:
- Each negative value appears in test cases where it's the only invalid input
- If the test fails, you know exactly which invalid input caused it
- Valid combination coverage is maintained for non-negative values
Example¶
Parameters:
| Parameter | Values |
|---|---|
| OS | Windows, Linux, macOS, ~InvalidOS |
| Browser | Chrome, Firefox, ~BadBrowser |
PICT will generate test cases like:
| OS | Browser | Type |
|---|---|---|
| Windows | Chrome | Valid |
| Linux | Firefox | Valid |
| ~InvalidOS | Chrome | Negative (only OS is invalid) |
| Windows | ~BadBrowser | Negative (only Browser is invalid) |
It will never generate: ~InvalidOS + ~BadBrowser — that would mask which invalid input triggers a failure.
Marking Values as Negative¶
In the UI¶
On the Parameters tab, check the Negative checkbox for the value in the expanded value editor.
In PICT Text Format¶
Prefix the value with ~:
OS: Windows, Linux, macOS, ~InvalidOS
In .cahtt JSON Format¶
Set "IsNegative": true:
{ "Name": "InvalidOS", "Weight": 1, "IsNegative": true }
Results Display¶
In the Results tab, negative values are highlighted in red text for easy identification. The statistics panel shows the count of negative test cases.