# Create your own tests

In this guide, you will learn how to create your own tests for a given project. You can download the example project used in this guide here.

# Test structure

Before we can make our own tests, we need to understand how a test directory works. You can find all tests within the test/ directory. The test names are defined solely by the directory names.

# driver.py

Test directories will always include a driver.py file. The driver file is the entry point into the test. The example driver.py has the following code:

#!/usr/bin/python3
#####################################################
#############  LEAVE CODE BELOW  ALONE  #############
# Include base directory into path
import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( __file__ ), '..', '..')))

# Import tester
from tester import failtest, passtest, assertequals, runcmd, preparefile, runcmdsafe
python_bin = sys.executable
#############    END UNTOUCHABLE CODE   #############
#####################################################

###################################
# Write your testing script below #
###################################
preparefile('./test.py')
b_stdout, b_stderr, b_exitcode = runcmdsafe(f'{python_bin} ./test.py')

if b_exitcode == 0:
	passtest('')
else:
	failtest(b_stdout.decode('utf-8'))

Everything below the "Write your testing script below" comment box is where the test specific code will go. In this test, it is actually calling another file called test.py and retrieving the stdout, stderr, and exit code. All tests should have this workflow of a driver file executing a test file.

Below that, this specific test's exit code is read from b_exitcode, and if it is 0 the test will pass, otherwise it will fail and return any stdout text using the param b_stdout.decode('utf-8').

# test.py

The test.py includes the following code (with added comments):

# necessary code block for test.py to work properly
import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( __file__ ), '..', '..')))

# import your functions from the assignment
from p0 import add

# test your function
if add(1, 1) == 2:
    exit(0)
else:
    exit(1)

So now we can see that this test is testing the add function from p0.py. It simply calls the function and sees if the intended value is met. If it is, the test exits with exit code 0.

This type of test is the most basic form you can create, but its setup is the scaffold to more involved examples and will help you understand most all tests you come across. If you don't understand a test, you can always ask the assignment giver for more info.

# Creating a test

Since public-test0 from above is in the most basic form, we can copy it to a new directory named anything we want. For this example, we will name it subtract-test.

# driver.py

Depending on how you want to set up your test, you may need to modify the driver.py to your needs. For us, using the exit code is the simplest way to create a test and we will leave it as is. You can use this exact driver.py on most any assignment you are given. You'll just need to modify the file it executes (test.py) depending on what type of code you need to run. For example, here's it changed to a rkt file:

b_stdout, b_stderr, b_exitcode = runcmdsafe(f'racket ./test.rkt')

For now, we can move on to test.py.

# test.py

Since we copied over the other test, we just need to modify it a little to suit our needs of creating a test case for subtract. Below is an example:

import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( __file__ ), '..', '..')))


from p0 import subtract

if subtract(3, 1) == 2:
    exit(0)
else:
    exit(1)

All that was edited here was the import from p0, and the if/else statement to call the subtract function. Within the limits of python, we can do whatever we want here as long as it executes within the timeout period (default 15~ seconds).

# Run your new test

As you may already know, all tests are run from the tester.py in the root directory. To run the individual test, we can specify -t and type ./tester.py -t subtract-test. We can also add -v to output any logs from print statements and errors. -v is probably the most useful thing you can use, because now you can also utilize any debug statements you littered around in your code as well (if you specified to return stdout in the driver.py). Since the tester.py is reactive, we can also do ./tester.py -a and it will include our newly created test when it tries to run all tests from the test/ directory. So, lets do just that:

$ ./tester.py -a
---------------------

Running test: public-test0
        PASSED
---------------------

Running test: subtract-test
        PASSED

===========================
Summary: 2 / 2 tests passed
===========================

And that's it, we now know how to make our own test.

← Autograde Basics