Exploring Test History

This page provides instructions for exploring your team's test history.

Unflakable displays past test results through the following dimensions:

1. Test suite run history: each time a test suite was run
2. Test suite run: individual test results from a single test suite run
3. Tests: tests that are part of a test suite
4. Test run history: each time a particular test was run

Test suite run history

To view the run history of a test suite:

1. Visit the Test Suites page.
2. Click the Runs () button for the appropriate test suite. If the test suite has never reported results to Unflakable, the button will appear grayed out.

This page displays the following columns for each test suite run:

• Branch: version control branch on which the tests were run, if reported by the plugin. An icon appears to the left of the branch name to indicate whether the branch was stable ( ) or unstable ( ).
• Commit: version control revision (e.g., Git commit hash) on which the tests were run, if reported by the plugin.
• Start Time: Date/time when the test suite run began.
• Duration: Duration of the test suite run in [hours:]minutes:seconds.milliseconds format.
• Pass: Number of tests that passed during this run.
• Fail: Number of tests that failed during this run.
• Flake: Number of tests that exhibited flakiness (i.e., failed, then passed upon retry) during this run.
• Quarantined: Number of tests that failed but were quarantined during this run.

To sort the results, click the heading of the column you would like to use for sorting (e.g., Branch). Click a column heading more than once to reverse the sort order. Note that certain columns may not be used for sorting results.

To filter the results:

1. Click the filter icon ( ) near the upper-left of the results table to open the filter menu.
2. Mouse over the column you would like to use for filtering to open its sub-menu:
   • Branch name: enter a branch name and press ENTER or click the search icon. Note that this will filter case insensitively by prefix (e.g., main will match both main and Main2).
   • Branch stability: select whether to return runs from only stable branches, or only unstable branches.
   • Commit: enter a commit (e.g., Git hash) and press ENTER or click the search icon. Note that this will filter case insensitively by prefix.
   • Start time: to filter based on the date/time when each test suite run began, enter the desired range start and end times. Click the calendar icon ( ) for each range endpoint to open a date/time dialog. To toggle between your local timezone and UTC, click the respective buttons to the right of the date/time range. Press ENTER or click the add filter icon ( ) to add the filter.
   • Results: select whether to return only runs that include failed, flaky, or quarantined tests, or only runs in which all tests passed.

Alternatively, certain filters may be added by clicking on the branch name, commit, or stable/unstable branch icon ( or ) within a result row.

If multiple column filters are added, only runs matching all of the filters (intersection) will be returned. If multiple values are specified for a column (other than Results), runs matching any of the values will be returned (union). For example, adding a branch name filter with the values main and feature and a results filter with “Includes flaky” will return runs with branch names beginning with main or feature that include at least one flaky test result.

To remove a filter, click the corresponding x icon.

Test suite run

To view the results of a single test suite run, click on the corresponding row of the test suite run history.

This page displays the following columns for each individual test executed as part of the test suite run:

• Test Name / File Path: The name of the test, as reported by the corresponding test framework. The file path containing the test is shown underneath the test name.
• Start Time: The start time of the individual test, if reported by the test framework.
• Duration: The duration of the individual test in [hours:]minutes:seconds.milliseconds format, if reported by the test framework.
• Result: The result of the individual test: pass, fail, flake, or quarantined.

To sort the results, click the heading of the column you would like to use for sorting (e.g., Test Name / File Path). Click a column heading more than once to reverse the sort order.

To filter the results:

1. Click the filter icon ( ) near the upper-left of the results table to open the filter menu.
2. Mouse over the column you would like to use for filtering to open its sub-menu:

   • Test Name: enter a test name and press ENTER or click the search icon. Note that this will filter case insensitively by substring (e.g., sync will match both responseSyncTest and response_async_test).

     Certain test frameworks (e.g., Jest) use hierarchical names for organizing tests (e.g., using describe() blocks or class names). This filter works by first concatenating all of the name components with a space between each component, and then checking case insensitively whether the concatenated, space-delimited string contains the specified substring.

   • File Path: select a single file from the drop-down menu, or enter a search string and press ENTER to add all of the matching files to the filter.

   • Result: select the test results to return: fail, flake, pass, or quarantined.

Alternatively, filters may be added by clicking on the test name, file path, or result.

If multiple column filters are added, only tests matching all of the filters (intersection) will be returned. If multiple values are specified for a column, tests matching any of the values will be returned (union). For example, adding a result filter for failed tests and a file path filter with src/a.ts and src/b.ts will return failed tests from either file.

To remove a filter, click the corresponding x icon.

Tests

To view the tests that are part a test suite:

1. Visit the Test Suites page.
2. Click the Tests () button for the appropriate test suite. If the test suite has never reported results to Unflakable, the button will appear grayed out. To jump directly a view filtered by tests with a specific test status, click any of the enabled buttons below the Tests button.

This page displays the following columns for each test:

• Test Name / File Path: The name of the test, as reported by the corresponding test framework. The file path containing the test is shown underneath the test name.
• Status: The current status of the test: passing, failing, flaky, or quarantined.
• Last Run: The last time the test was run on a stable branch. Click on this value to view the test result in the context of the corresponding test suite run.
• Last Fail: The last time the test failed, exhibited flakiness, or was quarantined on a stable branch. Click on this value to view the test result in the context of the corresponding test suite run.

To sort the results, click the heading of the column you would like to use for sorting (e.g., Test Name / File Path). Click a column heading more than once to reverse the sort order.

To filter the results:

1. Click the filter icon ( ) near the upper-left of the results table to open the filter menu.
2. Mouse over the column you would like to use for filtering to open its sub-menu:

   • Test Name: enter a test name and press ENTER or click the search icon. Note that this will filter case insensitively by substring (e.g., sync will match both responseSyncTest and response_async_test).

     Certain test frameworks (e.g., Jest) use hierarchical names for organizing tests (e.g., using describe() blocks or class names). This filter works by first concatenating all of the name components with a space between each component, and then checking case insensitively whether the concatenated, space-delimited string contains the specified substring.

   • File Path: select a single file from the drop-down menu, or enter a search string and press ENTER to add all of the matching files to the filter.

   • Status: select a test status to return only tests with the specified status.

Alternatively, filters may be added by clicking on the test name, file path, or status.

If multiple column filters are added, only tests matching all of the filters (intersection) will be returned. If multiple values are specified for a column, tests matching any of the values will be returned (union). For example, adding a status filter for flaky tests and a file path filter with src/a.ts and src/b.ts will return flaky tests from either file.

To remove a filter, click the corresponding x icon.

Updating a test status

In addition to exploring the tests that are part of a test suite, the Tests page allows users to update the status of each test.

To quarantine a test, click the Quarantine Test icon ( ) near the right side of the row. Note that this icon only appears next to tests that are not currently quarantined.

To remove a test from quarantine and set its status to , click the Remove from Quarantine icon ( ) near the right side of the row. Note that this icon only appears next to tests that are currently quarantined.

To update a test to , click the Update Status to Passing icon ( ) near the right side of the row. Note that this icon only appears next to flaky tests.

Test run history

To view the run history of a single test, click on the corresponding test row.

This page displays the following columns for each test run:

• Branch: version control branch on which the test was run, if reported by the plugin. An icon appears to the left of the branch name to indicate whether the branch was stable ( ) or unstable ( ).
• Commit: version control revision (e.g., Git commit hash) on which the test was run, if reported by the plugin.
• Start Time: The start time of the individual test, if reported by the test framework. Otherwise, this column shows the start time of the corresponding test suite run.
• Duration: The duration of the individual test in [hours:]minutes:seconds.milliseconds format, if reported by the test framework.
• Result: The result of the individual test run: pass, fail, flake, or quarantined.

To sort the results, click the heading of the column you would like to use for sorting (e.g., Branch). Click a column heading more than once to reverse the sort order. Note that certain columns may not be used for sorting results.

To filter the results:

1. Click the filter icon ( ) near the upper-left of the results table to open the filter menu.
2. Mouse over the column you would like to use for filtering to open its sub-menu:
   • Branch name: enter a branch name and press ENTER or click the search icon. Note that this will filter case insensitively by prefix (e.g., main will match both main and Main2).
   • Branch stability: select whether to return runs from only stable branches, or only unstable branches.
   • Start time: to filter based on the date/time when each run began, enter the desired range start and end times. Click the calendar icon ( ) for each range endpoint to open a date/time dialog. To toggle between your local timezone and UTC, click the respective buttons to the right of the date/time range. Press ENTER or click the add filter icon ( ) to add the filter.
   • Result: select the test results to return: fail, flake, pass, or quarantined.

Alternatively, certain filters may be added by clicking on the branch name, stable/unstable branch icon ( or ), or test result within a result row.

If multiple column filters are added, only test runs matching all of the filters (intersection) will be returned. If multiple values are specified for a column, test runs matching any of the values will be returned (union). For example, adding a branch name filter with the value main and a result filter with “fail” and “flake” will return runs with branch names beginning with main that either failed or exhibited flakiness.

To remove a filter, click the corresponding x icon.

Updating a test status

In addition to exploring the history of a single test, the Run History page allows users to update the status of that test.

To quarantine a test, click the Quarantine Test icon ( ) near the upper-right corner of the page. Note that this icon only appears for tests that are not currently quarantined.

To remove a test from quarantine and set its status to , click the Remove from Quarantine icon ( ) near the upper-right corner of the page. Note that this icon only appears for tests that are currently quarantined.

To update a test to , click the Update Status to Passing icon ( ) near the upper-right corner of the page. Note that this icon only appears for flaky tests.