pyam package

Subpackages

• pyam.cmd package
  • Submodules
  • pyam.cmd.args module
  • pyam.cmd.cohort module
  • pyam.cmd.config module
  • pyam.cmd.extract_downloads module
  • pyam.cmd.find_duplicates module
  • pyam.cmd.generate_template module
  • pyam.cmd.github_push module
  • pyam.cmd.github_retrieve module
  • pyam.cmd.init module
  • pyam.cmd.mark module
  • pyam.cmd.run module
  • pyam.cmd.write_csv module
  • Module contents
• pyam.fixtures package
  • Submodules
  • pyam.fixtures.clang module
  • pyam.fixtures.common module
  • pyam.fixtures.python module
  • pyam.fixtures.vhdl module
  • pyam.fixtures.xlsx module
  • Module contents

Submodules

pyam.cohort module

Classes to provide access to cohort and student information.

Classes

Cohort

representing a specific cohort

Student

representing a student in a cohort

Functions

get_cohort()

Gets a cohort by name (or default cohort)

current_academic_year()

returns a generated default cohort name by current (UK style) academic year

class pyam.cohort.Cohort(name)

Bases: pyam.config_manager.ConfigManager

Class representing a cohort of students with associated tests and reports

name

The cohort name (subdirectory name) (e.g. academic year of study)

Type

str

path

The Path to this cohort

Type

Path

test_path

The Path to the tests for this cohort

Type

Path

report_path

The Path to report directory for this cohort

Type

Path

log

The Logger for cohort.

Type

logging.Logger

start_log_section(title: str) → None

Write a section header in cohort log file

student_columns() → List[tuple]

Return a list of student column information for this cohort configuration

Returns

A list of tuples of regex and column titles suitable for read_csv

students(name: Union[str, None, List[str]] = None) → Union[pyam.cohort.Student, List[pyam.cohort.Student]]

Return student or students from a cohort.

Finds students by full name, student id or username in cohort.

Parameters

name – Name or names to be found in the cohort. May be username, student_id or common name

Returns

If name is not given prodes list of all students in cohort.

If name is a string returns the first matching student found.

If name is a list return list of students corresponding to list of names.

tests() → Dict[str, Dict]

Return dictionary of tests for this cohort indexed by pytest nodeids

If a manifest.json is provided in the test directory then this is the “tests” value from that file. Otherwise the nodeids are collected by pytest and the values fields are empty dictionaies. Future implementations may use the values.

Returns

A dictionary of tests for this cohort indexed by pytest nodeids. Value is a dictionary of test attributes from the test manifest if provided. Currently “description” is used to provide a human readable description and “mark” to provide a numerical mark for this test in the generated template.

class pyam.cohort.Student(cohort: pyam.cohort.Cohort, rec: dict)

Bases: object

A student in a cohort. Initialised from students.csv file in cohort

username

Their username

Type

str

path

The Path to where the students submission resides.

Type

Path

student_id

Their official student id

Type

str

last_name

Their family name

Type

str

first_name

their first name

Type

str

cohort

The Cohort in which they reside

Type

Cohort

course

Possible subcohort course name

Type

str

github_username

Github username if specified

Type

str

check_manifest(files: Optional[list] = None, log: bool = False) → List[str]

Check if student directory contains all files on cohort manifest

Parameters

• files – (optional) List of files to find. If not given use cohort manifest

• log (bool) – If True log missing files in cohort logger

Returns

List of missing files

checkout(until, branch=None)

Checkout last repository for student before given date until or to a specified branch

file(pattern: str, log: Optional[logging.Logger] = None) → Optional[pathlib.Path]

Attempt to find file matching given pathname pattern based on the configuration setting filematch.pattern.

Parameters

• pattern – The file pattern to search for - either an exact path, glob or regexp depending on the configuration setting filematch.pattern

• log – If given use to log if no file found or multiple files found

Returns

The path to the (first) matching file found or None if none found

find_files(pathname: str, containing: Optional[str] = None, recursive: bool = False) → List[pathlib.Path]

Return filtered list of files found in student directory.

If containing is given also filter to files containing this regexp If recursive is true also recurse subdirectories looking for matches

Parameters

• pathname – Unix style pathaname glob

• containing – An optional regexp to match against file contents

• recursive – If true also look in subdirectories

Returns

List of files found - may be empty list

git(*args, **kwargs)

Run git with given args in the student repository.

Return stdout if successful. if log keyword is set Logs action either as info or as an error depending on success

github_lastcommit() → Optional[datetime.datetime]

Return last github commit time if applicable

github_push(files: List[pathlib.Path], subdir=None, reset: bool = True, branch: Optional[str] = None, msg: str = 'Push from pyAutoMark')

Push given set of files into student repository

Parameters

• files – List of files or directories to copy

• subdir – If set - the name of subdirectory in student repository to copy files into

• reset – If True, do a github_retrieve first to ensure we are consistent with student repo

• branch – If set checkout and push files into this branch

If branch is specified original branch checkou in local repository is restored after push

github_retrieve(reset: bool = True, branch=None) → bool

Clone or pull asssessments for this student from their repository.

Returns

Success of retrieval

hash() → int

Return a hash based on students username - this will be first integer of first 8 characters of md5hash of username

name(style: str = 'ref') → str

Return common name formats - default is student id first name, last name

Parameters

style – Style - can be a field from csv file, username or ref(default)

Returns

name string in given style. Default is ref - student username, Lastname and firstname

repository_name() → Optional[str]

Return Github repository name if applicable else False

repository_url() → Optional[str]

Return students github repository url if present else False

pyam.cohort.current_academic_year() → str

Current (by date) academic year to use as a cohort name

This is based on common UK naming where the academic year starts in late September and finished in June with referred assessments are in July/August

Returns

September–December

returns the current year

January–June

return previous year

July-August

returns <previous year>-referred

pyam.cohort.get_cohort(name: str = '2022') → pyam.cohort.Cohort

Return cohort for given name or current default cohort if name=None

Parameters

name – Optional name of cohort to load. If not present will use default - “cohort” field from configuration Or use calculated current_academic_year

Returns

The cohort object

pyam.cohort.list_cohorts() → List[str]

List the cohorts

These are ubdirectories of cohorts which have student.csv and manifest.json files

Returns

List of valid cohort names

pyam.config module

Top level configuration handling for the assessment marking scripts

pyam.config.CONFIG

A ConfigManager with the global configuration data

Type

Config

class pyam.config.Config(location: Optional[pathlib.Path] = None)

Bases: pyam.config_manager.ConfigManager

Global (root) configuration class

root_path

top level Root path for assessments etc

Type

Path

test_paths

Path to tests

Type

Path

cohort_path

Path to cohorts

Type

Path

build_path

Path for out of source build

Type

Path

reports_path

Path for generated reports

Type

Path

log

global Logger

Type

logging.Logger

cohort

The current cohort being processed

Type

Cohort

pyam.config_manager module

Implementation of ConfigManager Class

pyam.config_manager.SCHEMA

A nested dictionary of discriptors for known configuration parameters. For each leaf parameter there should be a dictionary with a “description” and possibly also a “type”

Type

dict

class pyam.config_manager.ConfigManager(config_path: pathlib.Path, domain: str)

Bases: object

Base class for entities which have configuration files

domain

String representing schmea domain

Type

str

config_path

The Path to the configuration file

Type

Path

manifest

The dictionary of loaded configuration parameters

Type

dict

get(index: str, default: Any = []) → Any

Given a configuration index return a value

If default is given return that if not in configurations. If no default is given loop it up in global config - return None if not found

Parameters

• index – An index (may be in ‘.’ format e.g. assessor.username)

• default – Default value or CONFIG is global dictionary is to be searched as well

Returns

The retrieved value

classmethod getconfig(dic: dict, index: str) → Any

Look up nested dictionaries to retrieve a value

Parameters

• dic (dict) – Dictionary

• index (str) – index in ‘.’ format e.g. assessor.username

Raises

KeyError – if item not found

Returns

Value at index

load() → dict

Load configuration from a json file into internal manifest dictionary.

If file doesn’t exist creates a new empty dictionary

Returns

The dictionary of configuration data

classmethod parse_type(value: str, thetype: type)

Given a type from schema parse COnfig value using this type

Class Method

Parameters

• value – The configuration value to be parsed

• thetype – A type to covert value to

store() → None

Store manifest to configuration file, overwriting it.

pyam.config_manager.write_schema_rst(fid)

Write schema out as RST to given file descriptor

Parameters

• fid – A file descriptor to write to

• schema – The schema

pyam.cunit module

C compilation and program execution functions

Typical Usage:

exception pyam.cunit.CompilationError

Bases: Exception

Error in C compilation

exception pyam.cunit.LintError

Bases: Exception

Exceeded maximum number of lint warnings

exception pyam.cunit.RunTimeError

Bases: Exception

Error in binary executation

pyam.cunit.c_compile(binary: Union[pathlib.Path, str], source: Union[pathlib.Path, str], include: Sequence[Union[pathlib.Path, str]] = (), cflags: Sequence[str] = (), declarations: Sequence[str] = (), compiler: str = 'gcc', link: Sequence[str] = ()) → Union[pathlib.Path, str]

Use C compile to compile source files into an executable binary

Parameters

• binary – location for binary executable

• sources – source files to compile

• include – include paths to search for headers

• cflags – additional flags to add during compilation

• declarations – list of compile declarations (-D flags)

• compiler – name of compiler to use

• link – sequence of flags to pass for linkage

Raises

CompilationError – if compiler failed

Returns

Location of binary executable

pyam.cunit.c_exec(binary: Union[pathlib.Path, str], flags: Sequence[str] = (), input: Union[List[str], str] = '', timeout: Optional[Union[float, str]] = None) → True

Execute a binary executable with given flags.

Parameters

• binary – Path to binary executable

• timeout – timeout for process

• input – A string that will be fed to standard input or a list of strings can be given - these will be joined with a newline character.

Returns

From the run

Return type

subprocess.CompletedProcess

pyam.files module

Various file searching utilities for the tests

class pyam.files.PathGlob(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

File Glob Action aimed to works for Windows and Linux

On Windows it treats every value as a potential glob expression to be expanded and adds the results onto the stored values.

On other systems the shell will already have expanded the expressions so just treat as a path

pyam.files.expand_files(filearg)

Given a files value (from args.files) expand the list using glob

This is needed as Windows OS does not expand the file globs before passing to programmes

pyam.files.find_executable(name: str, paths: List[Union[pathlib.Path, str]]) → pathlib.Path

Find path to an executable program.

This will return full path from the system PATH. If not found there it will search the given list of directories (recursively) until it finds an executable with given name.

Parameters

• name – Name of program to be found

• paths – List of additional paths to search

Raises

FileNotFoundError – If no suct executable found

Returns

Path to executable

pyam.files.get_depends(name: Any, depends: List[List]) → List[Any]

Determine the dependencies for a particular (file) name from a dependency graph. Recurses the depends graph, will return dependencies in order.

Parameters

• name (Any) – Item for which dependencies

• depends (List[List]) – A recursive dependency graph

Returns

The ordered list of dependencies for name

pyam.files.read_csv(filename: Union[pathlib.Path, str], columns: Union[List, bool] = []) → List[Dict[str, str]]

Read a csv file.

Warning

The return type of this may change to a NamedTuple in future

Parameters

• filename – csv file to read

• columns – A list of tuples of regex’s to match against csv titles and column names, or if True read column names from first line.

Returns

• If columns is a list - use these to remap names.

• Else if columns is true use first line as column names.

• Else return just as list of lists

pyam.files.set_csv_column(filename: Union[pathlib.Path, str], column_name: str, key_name: str, get_value: Callable[[str], str])

Reads through a csv file overwriting specific named columns

Parameters

• filename – path to the csv file

• column_name – name or regex matching column name to be written to

• key_name – name or regex matching column to use as key

• get_value – a function which, given a key returns a new value if it returns None old value is kept.

pyam.pam module

Toip level pyam command

pyam.pam.main()

Automatically retrieve, mark and provide feedback for digital student submissions

pyam.run_pytest module

Pun pytest in the cohort context.

pyam.run_pytest.run_pytest(cohort: pyam.cohort.Cohort, *extras: Optional[List[str]]) → subprocess.CompletedProcess

Run pytest for pyAutoMark

Parameters

cohort – The cohort context to use

Optional:

extras: extra arguments to send pyttest

Returns

result of run

Return type

subprocess.CompletedProcess

pyam.test_manifest module

pyam.test_manifest.test_manifest(student)

Tests if current student has all files listed in cohort manifest

Module contents