Test Runner

This document describes Test Runner configuration file parameters.

LINE_DELAY

Sets the amount of time in seconds before the next line of a script will be executed

Parameter Description Required
Delay Delay in seconds before the next line is executed. A value of 0 means to execute the scripts as fast as possible. True

MONITOR_LIMITS

Log limits events to the Script Runner log file while a script is running

Limits events are always logged by the Command and Telemetry Server but are not put in the Script Runner log without this keyword.

PAUSE_ON_RED

Pause a running script if a red limit occurs

LOAD_UTILITY

Specify a test procedure to run

Procedures will be found automatically in the procedures directory or can be given by a path relative to the COSMOS install directory or by an absolute path. This keyword is used to load the script and execute it line by line by showing the current execution point. Sometimes you want to load utility code that is NOT executed line by line but is executed in the background. For example, a CRC routine or other long running calculation. The Ruby keyword ‘load’ can be used for this purpose, e.g. ‘load utility.rb’. This code will be loaded but not shown when executing.

Parameter Description Required
Filename Name of the test file in quotes True

REQUIRE_UTILITY

DEPRECATED in favor of LOAD_UTILITY

RESULTS_WRITER

Specify a different Ruby file to interpret and print the Test Runner results

The specified Ruby file must define a class which implements the Cosmos::ResultsWriter API

Parameter Description Required
Filename Name of the Ruby file which implements a result writer True
Class Parameters Parameters to pass to the constructor of the results writer False

ALLOW_DEBUG

Whether to allow the user to enable the debug line where the user can enter arbitrary statements

PAUSE_ON_ERROR

Set or clear the pause on error checkbox

If this is checked, Test Runner will pause if the test encounters an error. Otherwise the error will be logged but the script will continue.

Parameter Description Required
Enable Whether to pause when the script encounters an error

Valid Values: TRUE, FALSE
True

CONTINUE_TEST_CASE_AFTER_ERROR

Set or clear the continue test case after error checkbox

If this is checked, Test Runner will continue executing the current test case after encountering an error. Otherwise the test case will stop at the error and the next test case will execute.

Parameter Description Required
Enable Whether to continue the test case when the script encounters an error

Valid Values: TRUE, FALSE
True

ABORT_TESTING_AFTER_ERROR

Set or clear the abort testing after error checkbox

If this is checked, Test Runner will stop executing after the current test case completes (how it completes depends on CONTINUE_TEST_CASE_AFTER_ERROR). Otherwise the next test case will execute.

Parameter Description Required
Enable Whether to continue to the next test case when the script encounters an error

Valid Values: TRUE, FALSE
True

MANUAL

Set the $manual global variable for all executing scripts

The $manual variable can be checked during tests to allow for fully automated tests if it is not set, or for user input if it is set. This capability is completely dependent on user specific code which checks the $manual variable.

Parameter Description Required
Enable Whether to set the $manual global to true

Valid Values: TRUE, FALSE
True

LOOP_TESTING

Set or clear the loop testing checkbox

If this is checked, Test Runner will continue to run whatever level of tests that were initially started. If either “Abort Testing after Error” or “Break Loop after Error” are checked, then the loop testing will stop if an error is encountered. The difference is that the “Abort Testing after Error” will stop testing immediately after the current test case completes. “Break Loop after Error” continues the current loop by executing the remaining suite or group before stopping. In the case of executing a single test case the options effectively do the same thing.

Parameter Description Required
Enable Whether to loop the selected test level

Valid Values: TRUE, FALSE
True

BREAK_LOOP_AFTER_ERROR

Set or clear the break loop after error checkbox

If this is checked, Test Runner continues the current loop by executing the remaining suite or group before stopping.

Parameter Description Required
Enable Whether to break the loop after encountering an error

Valid Values: TRUE, FALSE
True

IGNORE_TEST

Ignore the given test class name when parsing the tests

Parameter Description Required
Test Class Name The test class to ignore when building the list of available tests True

IGNORE_TEST_SUITE

Ignores the given test suite name when parsing the tests

Parameter Description Required
Test Suite Name The test suite to ignore when building the list of available tests True

CREATE_DATA_PACKAGE

Creates a data package of every file created during the test

AUTO_CYCLE_LOGS

Automatically start a new server message log and cmd/tlm logs at the beginning and end of each test. Typically used in combination with CREATE_DATA_PACKAGE.

COLLECT_METADATA

Prompt for Meta Data before starting tests