Step 3: Connect Tests

Start pushing Test Results

Add Script in CI Pipeline to Connect Tests

Need to add an executable script in the CI to call the TestBrain API with the test conditions.

Requirements

• Required: Python 3.7+

Installation Command

--pip install appsurifyci --upgrade

Available Arguments

Argument	Options
--runtestswithappsurify	Call to Appsurify TestBrain API
--apikey	API Key from Appsurify Instance. Found in top right hand corner drop-down menu. Can be stored as a Secret in CI server secret storage capabilities. Reference the Secret via variable Script execution in the CI configuration.
--url	Url of Appsurify Instance i.e., https://dummy.appsurify.com/
--project	Name of the project
--testsuite	Name of the Test Suite
--branch	Name of the branch the tests are running against
--commit	SHA of the commit (need full SHA)

Recommended Arguments

Argument	Options
--runtemplate	Options - all tests, no tests, prioritized tests. See below for more details.
--testtemplate	Options - mvn, pytest, cypress, mocha, mstest, vstest, opentest...etc. See Integrations for more supported Test Frameworks
--runcommand	Command to execute tests in the target environment, if command is custom to your test suite. Note, --runcommand will override generic --testtemplate at time of final configuration and prioritized test execution.
--report	Location of XML reports created by the test run if this is not the default location for the test type.
--pipeoutput	Additional logging is printed to the screen from the Test Execution runner to help with Debugging and Logs

Test Execution Parameters

--all tests	Will run all tests. Will fail on any failure. Recommended for nightly or weekly test executions. Recommended for Appsurify to Learn
--no tests	Will run no tests. Will fail on any failure. Recommended when starting with Appsurify to just upload data.
--prioritized tests	Requires the Percentage or Time based Argument. Will fail the build on new / re-opened defects. Predictive Test Selection Technology

Example

--runtestswithappsurify --url "https://ABC.appsurify.com" --apikey "api key" --project "project" --testsuite "testsuite" --runtemplate "all tests" --runcommand "mvn" --commit "commit id" --branch "branch" --report "report" --pipeout "true"

Example in CI Pipeline

--runtestwithappsurify --url "https://testbraindemo.appsurify.com" --apikey "NTk6cD1Ua3doV3uNDMdWzz" --project "TestBrainDemoJunit" --testsuite "Selenium" --runtemplate "all tests" --runcommand "mvn" --commit "commit id" --branch "branch" --report "report" --pipeoutput "true"

Confirmation: Test Bind Turns Green

Upon successful Test Connection, in your Summary Page - your Test Bind Icon turns Green.

Push tests as part of CI process to build Model Maturity

Now that you've successfully Connected your Tests, automate them as part of the CI build process to build the Model Maturity wheel. Depending on how many Test Runs per day, this could take ~2-3 weeks. AKA, the more test data / runs - the stronger the results!

Once Model Maturity wheel has hit 50 Test runs or 100%, your Model will be trained over the following weekend, and will be subsequently ready for Predictive Test Selection mode - and your team can begin to enjoy the time and resource savings!

Additional Arguments - For Customization

Argument	Default	Options / Details
--fail	newdefects reopeneddefects	Options: newdefects, reopeneddefects, flakybrokentests, newflaky, reopenedflaky, failedtests, brokentests
--rerun	"false"	whether failed tests should be rerun
--maxrerun	3	the number of times failed tests should be rerun
--failfast	"false"	whether after each set of tests runs to determine if the build has failed (tests will still be rerun if this has been selected)
--executioncommand	" "	command to be executed following completion of the script [[teststorun]] will be replaced with the formatted list of tests to run
--percentage	" "	Percentage based prioritized test selection and execution. RECOMMENDED to start ~30%. Example: --percentage "20" would run top 20% of tests most likely to fail.
--time	" "	Time based prioritized test execution. Example: --time "45s" would run top 45 seconds of tests most likely to fail. Example --time "10m" would run top 10 minutes of tests most likely to fail.

Next: Start Test Prioritization!