An Unflakable API keyNew tab specified via the
UNFLAKABLE_API_KEY environment variable
Depending on your preference, specify the test suite ID and any desired
configuration options in package.json or a separate file named
unflakable.json (JSON),
unflakable.js/unflakable.cjs (JavaSScript), or unflakable.yml/unflakable.yaml (YAML).
Save the file to the
Jest root directoryNew tab or one of its
ancestors, and commit it to your code repository.
Alternatively, the test suite ID may be specified at runtime via the UNFLAKABLE_SUITE_ID
environment variable. We recommend using this environment
variable if you run
more than one Unflakable test suite in the same repository and would like to share common
configuration options between the test suites.
In addition to the testSuiteId configuration option
(or UNFLAKABLE_SUITE_ID environment variable), the Jest plugin supports the options
below.
Environment variables appear in parentheses for options that may be specified
in either the configuration file or the environment. If both are provided, the
environment variable is used.
The maximum number of times to retry each failed test. The plugin retries each failed test until either it passes or this number of retries is exhausted. Note that the total number of attempts for each test is one greater than the number of retries, including the initial attempt.
Set this value to 0 to disable retries, which will also prevent any tests from automatically being marked as flaky.
Whether the plugin should attempt to determine the current branch and commit hash from the GitNew tab repository containing the tests. If not using Git, or if auto-detection fails, consider setting this to false and providing the optional but recommended UNFLAKABLE_BRANCH and UNFLAKABLE_COMMIT environment variables.
Unflakable test suiteNew tab for which the plugin should track and report test results. If using the same plugin configuration file for multiple test suites, we recommend setting the UNFLAKABLE_SUITE_ID environment variable instead of setting testSuiteId in the configuration file.
Whether to report test results to Unflakable. If set to false, quarantined tests continue to behave as specified by quarantineMode, but test results will not be reported to Unflakable and failures will not cause tests to be marked as flaky or trigger notifications.
Default: Auto-detected from Git (unless gitAutoDetect is false)
Name of the version control (e.g., Git) branch containing the code being tested.
If specified (or auto-detected when gitAutoDetect is enabled), this value is compared against the test suite's stable branches setting to determine whether to update the status of each test (e.g., flaky/quarantined) based on these test results, and whether to trigger notifications. Test results from unstable branches will be visible in the Unflakable web application but will not impact test statuses or trigger notifications.
Default: Auto-detected from Git (unless gitAutoDetect is false)
Git commit hash (e.g., 8682b9e1d521a06e6b8df4eb4f0ff6ee2273f330) or other value that uniquely identifies the commit (version control revision) of the code being tested. If specified (or auto-detected when gitAutoDetect is enabled), this value helps identify test results in the Unflakable web application.
To run tests using the Jest plugin, invoke Jest as you ordinarily would, with the addition of the
UNFLAKABLE_API_KEY environment variable and the following Jest
command-line optionsNew tab:
Jest provides hierarchical test names: the test case name passed to
test()New tab or it()
prefixed with the names of any surrounding
describe()New tab blocks.
Unflakable limits each name component to a maximum of 4096 bytes
(using UTF-8New tab to encode
any Unicode characters). Additionally, each name is limited to a maximum
of eight components, which supports up to seven levels of nested describe()
blocks.
The Jest plugin will run all tests, but Unflakable will not track or quarantine
tests surrounded by more than seven describe() blocks or within any describe()
block that has a name longer than 4096 bytes.
In order to support use cases such as
eslint.RuleTesterNew tab
that often yield long test names, the plugin will truncate the last name component
(i.e., the one passed to test() or it()) to 4096 bytes.
As a result of this
behavior, Unflakable will treat any tests that share the first 4096-byte name prefix
(within the same set of describe() blocks) as a single test case. If a subset
of these tests fail, Unflakable will mark the entire group of tests as flaky. If the
group becomes quarantined, all of the test cases that share this prefix will be
quarantined. To avoid this behavior, we recommend providing human-readable, explicit test
names rather than using large test inputs as test names. For example, eslint.RuleTester
accepts an optional name field for each test case that serves this purpose.
If the quarantineMode configuration option is set
to the non-default value
skip_tests, the Jest plugin skips quarantined tests by internally leveraging Jest's
--testNamePatternNew tab feature. This
feature works by prepending the names of all surrounding
describe()New tab blocks to the test case
name passed to test()New tab or it(),
separating each component with a space character.
For example, the following test will have the combined name “a b c”:
If the test above becomes quarantined and skip_tests is used, the following test will
also be skipped because its name maps to the same string “a b c”:
In addition, Jest treats the testNamePattern regular expression as case insensitive, which
means that any tests whose concatenated names
differ only in letter casing will be skipped if any of the matching tests is quarantined.
If the tests above are skipped, tests with combined names such as “a b C” or
“A B C” will also be skipped.
We recommend using skip_tests only for test suites with naming conventions that avoid
the test name aliasing cases described above.