pyam.fixtures package

Submodules

pyam.fixtures.clang module

pytest fixtures to support running C (mock) tests using mocking.

This set of fixtures assumes a single student C file, and a single mock test C file. include path will be set to the cohort test directory, the build directory and the students directory. A file “student.h” is written into build directory to include the students c file in the mock test.

class pyam.fixtures.clang.CLintItem(*k, **kw)[source]

Bases: _pytest.nodes.Item

Pytest.Item subclass to handle each test result item.

config: Config

The pytest config object.

name: str

A unique name within the scope of the parent node.

parent

The parent collector node.

path: Path

Filesystem path where this node was collected from (can be None).

reportinfo()[source]

Short Report info for a lint item

repr_failure(excinfo, style=None)[source]

Called when self.runtest() raises an exception.

runtest()[source]

Execute file

session: Session

The pytest session this node is part of.

class pyam.fixtures.clang.CTestFile(*k, **kw)[source]

Bases: _pytest.nodes.File

A custom file handler class for C unit test files.

ctest_glob

The string from #define PYAM_TEST “glob” - glob to find student file

Type

str

clint_threshold

Numeric threshold value from #define PYAM_LINT theshold,”checks”

Type

int

clint_checks

Lint checks from #define PYAM_LINT threshold,”checks”

Type

str

cflags

List of flags from PYAM_FLAGS to pass to compiler

Type

List[str]

List of link items to pass to compiler

Type

List[str]

timeout

timout in seconds specified using #DEFINE PYAM_TIMEOUT <float>

cohort

Student cohort under for test

Type

pyam.cohort.Cohort

student

student under test

Type

pyam.cohort.Student

test_file_path

first file in student directory matching glob

Type

pathlib.Path

compile_file_path

path to the (students) file under test

Type

pathlib.Path

c_compile(item)[source]

Compile the test for item - its name is set as a command line definition

c_exec(item)[source]

Execute the compiled binary for item.

c_lint(item)[source]

Use clang-tidy to lint the file

cflags: List[str]
clint_checks: str = 'performance-*,readability-*,portability-*'
clint_threshold: int = None
cohort: pyam.cohort.Cohort = None
collect()[source]

Overridden collect method to collect the results from each C Mock unit test executable.

compile_file_path: pathlib.Path
config: Config

The pytest config object.

configure(config)[source]

Set up this CTestFIle from configuration information once it is available

Only done once per file - retrieve cohort and student context and file under test

ctest_glob: str
classmethod from_parent(parent, *, fspath=None, path=None, text='', **kw)[source]

The public constructor.

includes()[source]

The list of include paths to use during compilation

link: List[str]
name: str

A unique name within the scope of the parent node.

parent

The parent collector node.

path: Path

Filesystem path where this node was collected from (can be None).

re_cflags = re.compile('#define\\s+PYAM_CFLAGS\\s+\\"(.+)\\"')
re_clint = re.compile('#define\\s+PYAM_LINT\\s+(\\d+)?(,\\"(.*)\\")?')
re_ctest = re.compile('#define\\s+PYAM_TEST\\s+\\"(.*)\\"')
re_timeout = re.compile('#define\\s+PYAM_TIMEOUT\\s+(.+)')
session: Session

The pytest session this node is part of.

student: pyam.cohort.Student = None
test_file_path: pathlib.Path
text: str
timeout = None
class pyam.fixtures.clang.CTestItem(*k, **kw)[source]

Bases: _pytest.nodes.Item

Pytest.Item subclass to handle each test result item.

config: Config

The pytest config object.

name: str

A unique name within the scope of the parent node.

parent

The parent collector node.

path: Path

Filesystem path where this node was collected from (can be None).

reportinfo()[source]

Short Report info for a test item

repr_failure(excinfo, style=None)[source]

Called when self.runtest() raises an exception.

runtest()[source]

Execute file

session: Session

The pytest session this node is part of.

pyam.fixtures.clang.binary_name(student_c_file, mock_c_file) → str[source]

Fixture: The binary executable name to use for compiled tests - file will be created in build folder

pyam.fixtures.clang.c_binary(student_c_file, mock_c_file, c_compile) → pathlib.Path[source]

Fixture: Path to students executable under test (or None if a Mock test)

pyam.fixtures.clang.c_compile(student, test_path, build_path, mock_c_file, student_c_file, binary_name, compile_flags, compiler)[source]

Fixture: The compile function

The returned function takes the following arguments

Parameters

  • declarations (Sequence[str]) – The list of declarations (defines) to be set for the compilation. Typically only one will be set to select the test in the mock C file.

  • source (Union[Path,str]) – source C file - defaults to mock_c_file or student_c_file

Returns

Location of binary executable

Return type

Path

Raises

cunit.CompilationError – (after printing captured io to stdout)

pyam.fixtures.clang.c_exec(request, c_compile, c_binary)[source]

Fixture: The exec function for this test suite which will compile and execute the mock C tests.

Uses the timeout marker, and if set will stop student programm execution at that time.

The returned function takes the following arguments:

Parameters

  • declarations (Sequence[str]) – The list of delcarations (defines) tobe set for the compilation. Typically only one will be set to select the test in the mock C file.

  • Union[Path (binary) – Path to binary. If not specified source will be compiled first.

  • str] – Path to binary. If not specified source will be compiled first.

  • input – A string that will be fed to standard input or a list of strings can be given - these will be joined with a newline character.

Returns

The stdout from the script as a string.

pyam.fixtures.clang.c_lint(student, test_path, build_path, c_lint_checks)[source]

Fixture: A function to provide lint ouput on students C file

The returned function takes the following arguments:

Parameters

  • source_file (Path) – Path to C source file to be checked

  • man_warnings (int) – Optionally maximum number of warning to accept before test is considered a failure.

pyam.fixtures.clang.c_lint_checks() → str[source]

Fixture: C -lint checks to apply. Overwwite in your test as required

pyam.fixtures.clang.compile_flags() → Sequence[str][source]

Fixture: A list of additional compile flags (style) to use across tests

pyam.fixtures.clang.mock_c_file() → Union[str, pathlib.Path][source]

Fixture: The mock code C file currently under test. Must be set in the test if using mocks.

pyam.fixtures.clang.pytest_collect_file(parent, path) → List[source]

Hook into pytest

These are C files that start with test_ and contain #DEFINE PYAM_TEST fileglob definition where fileglob will match a students file

They may also contain a #DEFINE PYAM_LINT n,expr definition to run a CLANG_TIDY test

Tests are recognised as symbols matching “TEST_[A-Z0-9_]+”

A test timeout may be specified using #DEFINE PYAM_TIMEOUT <float>

Returns

List of tuples of filepath and a list of tests declarations.

pyam.fixtures.clang.pytest_collection_modifyitems(config, items)[source]

ENsure that if we have a test item that the file collector is configured

pyam.fixtures.clang.student_c_file() → Union[str, pathlib.Path][source]

Fixture: for the student C file currently under test. Must be set in the test.

pyam.fixtures.clang.write_header(build_path, student_c_file) → pathlib.Path[source]

Fixture: creates student.h which includes the student_c_file()

Used in Mock C code to include the students file.

pyam.fixtures.common module

Common fixtures for pyAutoMark tests which provid the cohort and student context in which the tests will be run.

pyam.fixtures.common.build_path() → pathlib.Path[source]

Fixture: Directory path for out of source build (executables, libs and object files)

pyam.fixtures.common.cohort(request) → pyam.cohort.Cohort[source]

Fixture: The cohort where student is to be found - specified using –cohort option to pytest

pyam.fixtures.common.compiler(request) → Union[str, pathlib.Path][source]

Fixture: The C Compiler to use

pyam.fixtures.common.pytest_addoption(parser)[source]

Add in pyAutoTest control options for pytest

pyam.fixtures.common.pytest_configure(config)[source]

Add in pyAutoTest markers for pytest

pyam.fixtures.common.solution(cohort)[source]

Fixture The solution “student” in the cohort.

  • if set in configuration

pyam.fixtures.common.student(request, cohort) → pyam.cohort.Student[source]

Fixture: The current student under test - specified using –student option

pyam.fixtures.common.student_files(student)[source]

Fixture: function to find files for student

The returned function takes the following arguments.

Parameters

  • pathname (str) – Unix style pathaname glob (or list of same)

  • containing (str) – An optional regexp to match against file contents

  • recursive (bool) – If true also look in subdirectories

Returns

File Path or list of File Paths found.

If multiple then first found will be given (or list)

Return type

Path (Union[Path,str])

Raises

FileNotFoundError – if no matching file found

pyam.fixtures.common.test_path(cohort) → pathlib.Path[source]

Fixture: Directory path containing test files

pyam.fixtures.python module

Fixtures for testing students Python scripts and functions

exception pyam.fixtures.python.PythonRunError[source]

Bases: Exception

Base Python related errors

exception pyam.fixtures.python.PythonStyleError[source]

Bases: Exception

pylint Score too low

pyam.fixtures.python.module_name(student)[source]

Fixture: Module name under students directory to load for a test. Must be set in the test

pyam.fixtures.python.python_lint(python_file: pathlib.Path, score_threshold: int)[source]

Run pylint on sudents code.

Parameters

  • python_file – the path to the file to be checked

  • score_threshold – the minimum thresold (/10) for the code to past this test.

Raises
pyam.fixtures.python.run_script(student, request)[source]

Fixture: A function to run a students python script.

The function takes the following arguments

Parameters

  • script (str) – The nname of the script in the students directory.

  • stdin (Union[List[str],str]) – A string that will be fed to standard input to the script. Alternatively a list of strings can be given - these will be joined with a newline character.

Raises

PythonRunError – if script fails to run. Error will have stderr output from script execution.

Returns

The stdout from the script as a string.

pyam.fixtures.python.student_module(student, module_name)[source]

Fixture: Returns the student module instance from the module_name() in the students directory.

pyam.fixtures.vhdl module

pyam.fixtures.xlsx module

Test Fixtures to check students xlsx spreadsheet submissions:

Support fixtures:

workbook: students workbook solution_workbook: The workbook for the solution

Configuration fixtures:

student_xlsx_file: relative path for students submission xlsx file

pyam.fixtures.xlsx.solution_workbook(solution, student_xlsx_file) → openpyxl.workbook.workbook.Workbook[source]

Fixture: The exemplar/reference workbook for answers

pyam.fixtures.xlsx.student_xlsx_file(cohort) → str[source]

Fixture: Name of the student workbook. Must be set in the test.

pyam.fixtures.xlsx.workbook(student, student_xlsx_file) → openpyxl.workbook.workbook.Workbook[source]

Fixture: The workbook object under test

Module contents