Requirements for Testing

Note: Test-driven development (TDD) is not linear. You might come back here and make updates to your test cases based on how the development cycle unfolds for each increment.

tests directory

  • All functions or methods must have test cases that are written before you continue with the design-code-test-style-version-control cycle.
  • Testing is separated from solution development

    • The functions/methods we develop are in the src directory of the assignment.

    • The testing functions are in the tests directory of the assignment.

      Note: For the first lab the assignment file structure has only one module. Testing of the module functions is done in main().

Test cases and testing functions

  • The test cases are implemented with testing functions in the tests directory.
  • The docstring specifications of the source functions/methods in the src directory have very important information about what test cases to write.

A test case contains a call to the function/method that is tested.

  • Create and initialize variable (or variables) with values that correspond to particular input(s), mapped to the function/method parameter(s). A test case can test:
    • A “happy path” execution instance, or
    • An “edge case” for special input values that are legal and described in the docstring return comments, or
    • Illege arguments execution instances, described in the raise docstring comments.
  • Create and initialize expected_result variable with the value that represents the correct output you expect from the given input.
  • Call function/method and assign return value to actual_result variable.
  • Create and initialize err_msg with the message that’s outputted IF the test FAILS. Use f-string to compose the error message.
  • Use the assert statement to compare actual_result and expected_result, and issue the error message if the test fails.

assert expected_result == actual_result, err_msg

Each function/method has a suite of tests:

  • Some tests are for the “happy path” execution of the function/method. This means that the input parameters have values that comply with the their specification in the function/method docstring.
  • Some tests are for “edge cases” or “corner cases” cases. This means that the input parameters values comply with the data type of the parameter descriptioh in the docstring, but their values are special, different from the “happy path”.
    • The return comment in the docstring describes what the correct output of the function/method should be for every single edge case.
  • Some tests are for illegal input values. This means that the input parameter values do not comply with the parameter description in the docstring or it is None (no input value was passed through the call)
    • The raise comment in the docstring describes what error must be raise when such value is detected.

PyCharm run-time environment

PyCharm MUST have a correctly set-up run-time environment. This means that:

  • comp801 is the project root directory opened in PyCharm
  • Virtual environment is in .venv directory in the project root
  • A lab or homeowrk directory, for example lab3, has two subdirectories: src and tests
  • You MUST mark the lab or homework directcory as the Source Root. How?
    • Right click on the lab or homework directory, for example lab3
    • Select from the pop-up window “Make directory as … Source Root”
    • This selection tells PyCharm that the PYTHONPATH for running modules in src or for running testing functions in tests is the absolute path of the lab or homework directory, e.g., /Users/mcs/courses/comp801/lab3/
    • Import statements in any of the lab/homeowrk modules are RELATIVE to the PYTHONPATH.
    • For example, to import module lab3 in a testing function in tests, use:

      from src.schedule import Schedule

Running the tests

There are three different ways to run the tests

  1. In PyCharm Terminal
  • Virtual environment must be active
  • Change directory to the lab/homework source root directory
  • Run python -m pytest -v
  • From the Projects pane, select the lab/homework directory, and then the testing function in the tests directory.
    • Testing function opens in the Editor pane.
  • From the main menu, Current File should be shown in the run field shown just efore a green triangle
    • Click the green triangle
  1. From the main menu, run configruation field (shown before the green triangle)
  • Select from the pull-down menu, Edit Configuraiton
  • Run/Debug Configuration dialog opens
    • Select “Add new configuration”, and in the pull-down menu scroll down to find and select “Python”
    • Here, you can configure a specific run-time environemnt for the selected script.

When there is no implementation of the source files, all test cases are supposed to fail by showing assert statement errors: mismatch between the expected and actual results.

If running the tests gives Python syntax errors (NOT pytest assert errors)

  • Debug the tests and fix syntax errors.

Code styling the testing functions

Last step is to fix code styling errors for the testing functionss

  • Examine the erorrs reported in PyCharm Problems, of any, and fix them.
  • Run pycodestyle on all Python files in the PyCharm Terminal
  • Run pylintstyle on all Python files in the PyCharm Terminal
    • Since pylint does not know what the PYTHONPATH is, you need to add the following comments around the import statements (using lab3 testing function example:)
    # pylint: disable=import-error
from src.schedule import Schedule
# pylint: enable=import-error

Version control

A testing function and its test suite represents a unit of development.

Version control - When the implementation of a testing function runs correctly, commit changes associated with this unit of development.