Option to send "Passed" alerts only once

We have tests that run every 5 minutes and sends Slack notifications. I would like an option to send "Passed" alerts only once, not every time the test passes (and then once again after failure). This is a common feature in similar services.

This way we can easily see if a test started working again after failure and failures won't be drowned out by all the "Passed" notifications in Slack.

“How to identify "state change" for the suite when random tests could fail? “

We’re only interested in state changes for individual tests and not the suite itself.

“When should the notification be sent? What ​kind of message do you expect?”

SuiteA (1)

[09:00] Test1 Passed

[09:00] Test2 Passed

[09:00] Test3 Passed

SuiteA (2)

[09:15] Test1 Failed [state change] – Email Alert Sent (Test 1 Failed) “Test1 in SuiteA transitioned at 09:15 from SUCCESS → FAILED.”

[09:15] Test2 Passed

[09:15] Test3 Passed

SuiteA (3)

[09:30] Test1 Failed

[09:30] Test2 Failed [state change] – Email Alert Sent (Test 2 Failed) “Test2 in SuiteA transitioned at 09:30 from SUCCESS → FAILED.”

[09:30] Test3 Passed

SuiteA (4)

[09:45] Test1 Passed [state change] – Email Alert Sent (Test 1 Successful) “Test1 in SuiteA transitioned at 09:45 from FAILED → SUCCESS.

[09:45] Test2 Failed

[09:45] Test3 Passed

SuiteA (5)

[10:00] Test1 Passed

​[10:00] Test2​Passed [state change] – Email Alert Sent (Test 2 Successful) “Test2 in SuiteA transitioned at 10:00 from FAILED → SUCCESS.”

​[10:00] Test3 Passed

If you did want to track suite state and test state, you could always do something like this:

SuiteA (1)

[09:00] Test1 Passed

[09:00] Test2 Passed

[09:00] Test3 Passed

SuiteA (2)

[09:15] Test1 Failed [state change] – Email Alert Sent (Test 1 Failed) “Test1 in SuiteA transitioned at 09:15 from SUCCESS → FAILED.”

[09:15] Test2 Passed

[09:15] Test3 Passed

– Email Alert Sent (SuiteA Failures) “One or more tests in SuiteA failed.”

SuiteA (3)

[09:30] Test1 Failed

[09:30] Test2 Failed [state change] – Email Alert Sent (Test 2 Failed) “Test2 in SuiteA transitioned at 09:30 from SUCCESS → FAILED.”

[09:30] Test3 Passed

SuiteA (4)

[09:45] Test1 Passed [state change] – Email Alert Sent (Test 1 Successful) “Test1 in SuiteA transitioned at 09:45 from FAILED → SUCCESS.

[09:45] Test2 Failed

[09:45] Test3 Passed

SuiteA (5)

[10:00] Test1 Passed

​[10:00] Test2​Passed [state change] – Email Alert Sent (Test 2 Successful) “Test2 in SuiteA transitioned at 10:00 from FAILED → SUCCESS.”

​[10:00] Test3 Passed

– Email Alert Sent (SuiteA Success) “All tests in SuiteA were successful. (3/3)”