Adding Custom Output Handlers

Third Party Output Handlers

  • rotest_reportportal
    • Plugin to the amazing Report Portal system, that enables viewing test results and investigating them.
  • rotest-progress
    • Uses tqdm to give you two user-friendly output handlers (tested on Linuex only): full_progress which shows the general progress of the run, and progress which shows the progress of the current component and can be used with other handlers that write to the screen, e.g. logdebug.

How to Make Your Own Output Handler

You can make your own Output Handler, following the next two steps:

  • Inheriting from rotest.core.result.handlers.abstract_handler.AbstractResultHandler, and overriding the relevant methods.

  • Register the above inheriting class as an entrypoint in your setup.py file inside setup():

    entry_points={
        "rotest.result_handlers":
            ["<handler tag, e.g. my_handler> = <import path to the monitor's module>:<monitor class name>"]
    },
    
  • Make sure it’s being installed in the environment by calling

    python setup.py develop
    

For an example, you can refer to rotest_reportportal plugin.

Available Events

The available methods of an output handler:

class rotest.core.result.handlers.abstract_handler.AbstractResultHandler(main_test=None, *args, **kwargs)

Result handler interface.

Defines the required interface for all the result handlers.

main_test

the main test instance (e.g. TestSuite instance or TestFlow instance).

Type:rotest.core.abstract_test.AbstractTest
add_error(test, exception_string)

Called when an error has occurred.

Parameters:
  • test (rotest.core.abstract_test.AbstractTest) – test item instance.
  • exception_string (str) – exception description.
add_expected_failure(test, exception_string)

Called when an expected failure/error occurred.

Parameters:
  • test (rotest.core.abstract_test.AbstractTest) – test item instance.
  • exception_string (str) – exception description.
add_failure(test, exception_string)

Called when an error has occurred.

Parameters:
  • test (rotest.core.abstract_test.AbstractTest) – test item instance.
  • exception_string (str) – exception description.
add_info(test, msg)

Called when a test registers a success message.

Parameters:
  • test (rotest.core.abstract_test.AbstractTest) – test item instance.
  • msg (str) – success message.
add_skip(test, reason)

Called when a test is skipped.

Parameters:
  • test (rotest.core.abstract_test.AbstractTest) – test item instance.
  • reason (str) – reason for skipping the test.
add_success(test)

Called when a test has completed successfully.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
add_unexpected_success(test)

Called when a test was expected to fail, but succeed.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
print_errors(tests_run, errors, skipped, failures, expected_failures, unexpected_successes)

Called by TestRunner after test run.

Parameters:
  • tests_run (number) – count of tests that has been run.
  • errors (list) – error tests details list.
  • skipped (list) – skipped tests details list.
  • failures (list) – failed tests details list.
  • expected_failures (list) – expected-to-fail tests details list.
  • unexpected_successes (list) – unexpected successes tests details list.
setup_finished(test)

Called when the given test finished setting up.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
should_skip(test)

Check if the test should be skipped.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
Returns:skip reason if the test should be skipped, None otherwise.
Return type:str
start_composite(test)

Called when the given TestSuite is about to be run.

Parameters:test (rotest.core.suite.TestSuite) – test item instance.
start_teardown(test)

Called when the given test is starting its teardown.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
start_test(test)

Called when the given test is about to be run.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
start_test_run()

Called once before any tests are executed.

stop_composite(test)

Called when the given TestSuite has been run.

Parameters:test (rotest.core.suite.TestSuite) – test item instance.
stop_test(test)

Called when the given test has been run.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.
stop_test_run()

Called once after all tests are executed.

update_resources(test)

Called once after locking the tests resources.

Parameters:test (rotest.core.abstract_test.AbstractTest) – test item instance.