Bug Tracking and Closing
Appsurify integrates with Jira and other popular bug tracking tools for each tracking and management of defects.
Project – The Standard Defect Type
Flaky – Tests which fail due to defects of type ‘flaky’ and will not break the build unless otherwise specified.
Invalid Test – Indicates that the test is or was invalid when the test failed.
Outside Scope – Select this defect type when the failure is a consequence of another test failure.
For example, if Login breaks and your Login tests fail and create a defect and all your other tests fail and create a defect which is secondary, then this defect would be marked as ‘Outside Scope,’ essentially saying that we don’t want to prioritize this defect as it is a corresponding failure to the real defect.
Defects will automatically open when tests fail. The type will depend on a number of factors. By default, they are opened as a project defect, but if the model indicates that these failure were flaky the default type will be set to flaky.
The model for determining flakiness looks at a large number of features, such as:
Previous Defects
Failure Messages
Failure Types – fail vs. error vs. assert
Average run time vs when the test failed
History of that test
Area of the change
Defects other than of type, ‘flaky,’ will be closed automatically when all the associated tests (tests that have failed because of this defect) pass on a commit which is a child of the commit which caused the defect.
Defects that are caused by commits are the set of commits which caused the defect.
Defects that are closed by commits are the set of commits which closed the defect.
Last updated
Was this helpful?
Was this helpful?
