pyretis.inout package¶

The sub-package handles input and output for PyRETIS.

This package is intended for creating various forms of output from the PyRETIS program. It includes writers for simple text-based output and plotters for creating figures. Figures and the text results can be combined into reports, which are handled by the report module.

Package structure¶

Modules¶

__init__.py: Imports from the other modules.
checker.py (pyretis.inout.checker): Functions to check the compatibility and consistency of various input values.
common.py (pyretis.inout.common): Common functions and variables for the input/output. These functions are mainly intended for internal use and are not imported here.
fileio.py (pyretis.inout.fileio): A module which defines a generic file class for PyRETIS output files.
settings.py (pyretis.inout.settings): A module which handles the reading/writing of settings.
simulationio.py (pyretis.inout.simulationio): A module which handles th reading/writing of the main simulations.
restart.py (pyretis.inout.restart): A module which handles restart reading/writing.

Sub-packages¶

analysisio (pyretis.inout.analysisio): Handles the input and output needed for analysis.
formats (pyretis.inout.formats): Handles the input and output of different data formats. This includes the configurations and the internal data formats.
plotting (pyretis.inout.plotting): Handles the plotting needed by the analysis by defining plotting tools, methods and styles.
report (pyretis.inout.report): Generate reports with results from simulations.

Important classes defined in this package¶

Important methods defined in this package¶

generate_report (generate_report()): A function to generate reports from analysis output(s).
parse_settings_file (parse_settings_file()): Method for parsing settings from a given input file.
write_settings_file (write_settings_file()): Method for writing settings from a simulation to a given file.
write_restart_file (write_restart_file()): Method for writing restart information.

Subpackages¶

pyretis.inout.clean module¶

Removal of PyRETIS run artifacts (the pyretis clean command).

This module implements the cleanup that used to live in the per-example Makefile clean targets. It deletes the artifacts a PyRETIS run leaves behind so an example (or any run directory) can be reset to its committed inputs.

What is removed is the union of:

a built-in set of always-generated PyRETIS artifacts (see DEFAULT_FIND_FILES and DEFAULT_FIND_DIRS), and
whatever an optional per-directory CONFIG_NAME file adds.

The per-directory config (TOML) mirrors the variables the old shared common-clean.mk used:

[clean]
use_defaults = true        # apply the built-in defaults (default: true)
find_files = ["*.tpr"]     # extra file-name globs (recursive find -delete)
find_dirs  = ["dump"]      # extra directory names (recursive rm -rf)
rm_paths   = ["lammps/system.data"]  # root-relative rm -rf globs
keep       = ["load/0"]    # never delete these (or their ancestors)

keep protects committed inputs that would otherwise match (for example the committed load/0 .. load/7 initial paths of the validation suite). Anything that is a kept path, lies inside a kept path, or is an ancestor directory of a kept path is left untouched.

pyretis.inout.clean._clean_find_dirs(directory, patterns, protected, dry_run, removed)¶

Delete directories whose name matches one of patterns.

The walk is top-down so a matched directory is removed before its children are visited; dirs is pruned in place to avoid descending into something already scheduled for removal.

pyretis.inout.clean._clean_find_files(directory, patterns, protected, dry_run, removed)¶: Delete files whose name matches one of patterns (recursive).

pyretis.inout.clean._clean_rm_paths(directory, patterns, protected, dry_run, removed)¶: Delete root-relative paths/globs with rm -rf semantics.

pyretis.inout.clean._is_protected(path, protected)¶

Return True if path must not be deleted given protected.

A path is protected when it equals a kept path, lies inside one, or is an ancestor directory of one (so we never delete a parent that still holds something we must keep).

Parameters:

path (string) – The candidate path to delete.
protected (set of str) – Absolute normalised kept paths.

Returns:

boolean – True when the candidate must be left untouched.

pyretis.inout.clean._normalise_keep(directory, keep)¶

Return the set of absolute, normalised kept paths.

Parameters:

directory (string) – The root directory of the clean.
keep (list of str) – The configured keep entries (root-relative, may be globs).

Returns:

set of str – Absolute normalised paths that must be protected.

pyretis.inout.clean._read_toml(path)¶

Parse a TOML file, returning the top-level mapping.

Parameters:: path (string) – Path to the TOML file.
Returns:: dict – The parsed configuration.

pyretis.inout.clean._remove_file(path, dry_run, removed)¶: Delete a file (unless dry-run) and record it.

pyretis.inout.clean._remove_tree(path, dry_run, removed)¶: Delete a file or directory tree (unless dry-run) and record it.

pyretis.inout.clean._under_excluded_dir(path, root)¶

Return True if path lies under an excluded directory name.

Parameters:

path (string) – The path to test.
root (string) – The clean root (its own name is not considered).

Returns:

boolean – True when a path component (below root) is excluded.

pyretis.inout.clean.clean_directory(directory='.', dry_run=False)¶

Remove PyRETIS run artifacts from a directory.

Parameters:

directory (string, optional) – The directory to clean. Defaults to the current directory.
dry_run (boolean, optional) – When True, nothing is deleted; the would-be-removed paths are only collected and returned.

Returns:

list of str – The paths that were removed (or, for a dry run, that would be removed), sorted for a stable report.

pyretis.inout.clean.load_clean_config(directory)¶

Load the cleanup configuration for a directory.

Reads the built-in defaults and merges in the optional CONFIG_NAME file found in directory.

Parameters:: directory (string) – The directory to be cleaned.
Returns:: dict – A configuration with the keys find_files, find_dirs, rm_paths and keep (all lists of strings).

pyretis.inout.common module¶

This file contains common functions for the input/output.

It contains some slave functions that are used in the in/output function of PyRETIS.

Important classes defined here¶

OutputBase (OutputBase): A base class for handling the output.

Important methods defined here¶

check_python_version (check_python_version()): A method that will give warnings when we use older and possibly unsupported Python versions.
create_backup (create_backup()): A function to handle the creation of backups of old files.
make_dirs (make_dirs()): Create directories (for path simulation).
atomic_write (atomic_write()): Write a file atomically and durably (crash-safe).
durable_copy (durable_copy()): Copy a file and flush the copy to stable storage.
fsync_dir (fsync_dir()): Flush a directory entry to disk.
create_empty_ensembles (create_ensembles()): A method to prepare the ensembles inputs in settings
generate_file_name (generate_file_name()): Generate file name for an output task, from settings.

class pyretis.inout.common.OutputBase(formatter)¶

Bases: object

A generic class for handling output.

Variables:

formatter (object like py:class:.OutputFormatter) – The object responsible for formatting output.
target (string) – Determines where the target for the output, for instance “screen” or “file”.
first_write (boolean) – Determines if we have written something yet, or if this is the first write.

__init__(formatter)¶: Create the object and attach a formatter.

__str__()¶: Return basic info.

formatter_info()¶: Return a string with info about the formatter.

output(step, data)¶

Use the formatter to write data to the file.

Parameters:

step (int) – The current step number.
data (list) – The data we are going to output.

target = None¶

abstractmethod write(towrite, end='\n')¶

Write a string to the output defined by this class.

Parameters:

towrite (string) – The string to write.
end (string, optional) – A “terminator” for the given string.

Returns:

status (boolean) – True if we managed to write, False otherwise.

pyretis.inout.common.add_dirname(filename, dirname)¶

Add a directory as a prefix to a filename, i.e. dirname/filename.

Parameters:

filename (string) – The filename.
dirname (string) – The directory we want to prefix. It can be None, in which case we ignore it.

Returns:

out (string) – The path to the resulting file.

pyretis.inout.common.atomic_write(path, write_fn, binary=True, keep_prev=False)¶

Write a file atomically and durably.

The payload is first written to a temporary file in the same directory, flushed to disk with os.fsync and then moved into place with os.replace (an atomic operation on POSIX systems when both files reside on the same filesystem). Finally the parent directory entry is flushed so that the rename itself survives a crash.

This guarantees that path is never observed in a half-written state: after a crash it is either the complete new content or the complete previous content.

Parameters:

path (string) – The file we want to create.
write_fn (callable) – A function that takes a single argument, the open file handle, and writes the payload to it.
binary (boolean, optional) – If True, the temporary file is opened in binary mode, otherwise in (utf-8) text mode.
keep_prev (boolean, optional) – If True, an existing path is copied to path + '.prev' before the replace. This keeps the previous version available even if a crash happens in the (tiny) window around the rename.

Returns:

out (string) – The path that was written.

pyretis.inout.common.check_python_version()¶: Give a warning about old python version(s).

pyretis.inout.common.create_backup(outputfile)¶

Check if a file exist and create backup if requested.

This function will check if the given file name exists and if it does, it will move that file to a new file name such that the given one can be used without overwriting.

Parameters:: outputfile (string) – This is the name of the file we wish to create.
Returns:: out (string) – This string is None if no backup is made, otherwise, it will just say what file was moved (and to where).

Note

No warning is issued here. This is just in case the msg returned here will be part of some more elaborate message.

pyretis.inout.common.create_empty_ensembles(settings)¶

Create missing ensembles in the settings.

Checks the input and allocate it to the right ensemble. In theory inouts shall include all these info, but it is not practical.

Parameters:: settings (dict) – The current input settings.
Returns:: None, but this method might add data to the input settings.

pyretis.inout.common.durable_copy(src, dst)¶

Copy a file and flush the copy to stable storage.

Used for trajectory files (and other binary payloads) that must be on disk before we record them as part of an archived path. Both the copied file and its parent directory entry are flushed with os.fsync.

Parameters:

src (string) – The file to copy.
dst (string) – The destination file name.

pyretis.inout.common.fsync_dir(dirname)¶

Flush a directory entry to disk.

After an atomic rename, the directory entry itself must be flushed for the rename to be durable across a crash (e.g. a power loss). This is best-effort: some filesystems do not allow fsync on a directory handle, and that single situation is the only one we ignore here.

Parameters:: dirname (string) – The directory whose metadata we flush. An empty string is interpreted as the current working directory.

pyretis.inout.common.generate_file_name(basename, directory, settings)¶

Generate file name for an output task, from settings.

Parameters:

basename (string) – The base file name to use.
directory (string) – A directory to output to. Can be None to output to the current working directory.
settings (dict) – The input settings

Returns:

filename (string) – The file name to use.

pyretis.inout.common.make_dirs(dirname)¶

Create directories for path simulations.

This function will create a folder using a specified path. If the path already exists and if it’s a directory, we will do nothing. If the path exists and is a file we will raise an OSError exception here.

Parameters:: dirname (string) – This is the directory to create.
Returns:: out (string) – A string with some info on what this function did. Intended for output.

pyretis.inout.common.name_file(name, extension, path=None)¶

Return a file name by joining a name and an file extension.

This function is used to create file names. It will use os.extsep to create the file names and os.path.join to add a path name if the path is given. The returned file name will be of form (example for posix): path/name.extension.

Parameters:

name (string) – This is the name, without extension, for the file.
extension (string) – The extension to use for the file name.
path (string, optional) – An optional path to add to the file name.

Returns:

out (string) – The resulting file name.

pyretis.inout.fileio module¶

Module defining the base classes for the PyRETIS output.

Important classes defined here¶

FileIO (FileIO): A generic class for handling input & output with files.

Important methods defined here¶

read_some_lines (read_some_lines()): Method to read lines from PyRETIS data files.

class pyretis.inout.fileio.FileIO(filename, file_mode, formatter, backup=True)¶

Bases: OutputBase

A generic class for handling IO with files.

This class defines how PyRETIS stores and reads data. Formatting is handled by an object like OutputFormatter

Variables:

filename (string) – Name (e.g. path) to the file to read or write.
file_mode (string) – Specifies the mode in which the file is opened.
backup (boolean) – Determines the behavior if we want to write to a file that is already existing.
fileh (object like io.IOBase) – The file handle we are interacting with.
last_flush (object like datetime.datetime) – The previous time for flushing to the file.

__del__()¶: Close the file in case the object is deleted.

__enter__()¶: Context manager for opening the file.

__exit__(*args)¶: Context manager for closing the file.

__init__(filename, file_mode, formatter, backup=True)¶

Set up the file object.

Parameters:

filename (string) – The path to the file to open or read.
file_mode (string) – Specifies the mode for opening the file.
formatter (object like py:class:.OutputFormatter) – The object responsible for formatting output.
backup (boolean, optional) – Defines how we handle cases where we write to a file which is already existing.

__iter__()¶: Make it possible to iterate over lines in the file.

__next__()¶: Let the file object handle the iteration.

__str__()¶: Return basic info.

close()¶: Close the file.

flush()¶: Flush file buffers to file.

load()¶: Read blocks or lines from the file.

open()¶: Open a file for reading or writing.

open_file_read()¶

Open a file for reading.

A file that cannot be opened raises. Swallowing the error left fileh as None, and iteration over this object then stops immediately (see __next__()), so an unreadable file was indistinguishable from an empty one. Callers for which a missing file is a legitimate state check for it first, as pyretis.core.path_load._load_energies_for_path() does.

open_file_write()¶

Open a file for writing.

In this method, we also handle the possible backup settings.

output(step, data)¶: Open file before first write.

target = 'file'¶

write(towrite, end='\n')¶

Write a string to the file.

Parameters:

towrite (string) – The string to output to the file.
end (string, optional) – Appended to towrite when writing, can be used to print a new line after the input towrite.

Returns:

status (boolean) – True if we managed to write, False otherwise.

Note

A write that fails on I/O (a full disk, a quota, a failing device) raises. The error used to be logged and absorbed, and since no caller inspects the returned status, the run then carried on and reported success while its output file was left truncated.

pyretis.inout.fileio.read_some_lines(filename, line_parser, block_label='#')¶

Open a file and try to read as many lines as possible.

This method will read a file using the given line_parser. If the given line_parser fails at a line in the file, read_some_lines will stop here. Further, this method will read data in blocks and yield a block when a new block is found. A special string (block_label) is assumed to identify the start of blocks.

Parameters:

filename (string) – This is the name/path of the file to open and read.
line_parser (function, optional) – This is a function which knows how to translate a given line to a desired internal format. If not given, a simple float will be used.
block_label (string, optional) – This string is used to identify blocks.

Yields:

data (list) – The data read from the file, arranged in dicts.

pyretis.inout.restart module¶

This module defines how we write and read restart files.

Important methods defined here¶

read_restart_file (read_restart_file()): A method for reading restart information from a file.
write_restart_file (write_restart_file()): A method for writing the restart file.
write_ensemble_restart (write_ensemble_restart()): A method for writing restart files for path ensembles.

pyretis.inout.restart.load_system_restart(system, info)¶

Restore System state from a restart-info dict.

Equivalent to the soon-to-go System.load_restart_info(info) method on the harness System.

Parameters:

system (object like System) – The system to populate.
info (dict) – The restart info, as produced by system_restart_info().

pyretis.inout.restart.read_restart_file(filename)¶

Read restart info for a simulation.

Parameters:: filename (string) – The file we are going to read from.

pyretis.inout.restart.system_restart_info(system)¶

Collect restart info for a System (free-function form).

Equivalent to the soon-to-go System.restart_info() method on the harness System. Returns the same dict shape; works on any System with the bridge surface (units, temperature, post_setup, order, box, particles).

Parameters:: system (object like System) – The system whose state to serialise.
Returns:: dict – Restart-info dictionary.

pyretis.inout.restart.write_ensemble_restart(ensemble, settings_ens)¶

Write a restart file for a path ensemble.

Parameters:

ensemble (dict) – it contains:
- path_ensemble : object like PathEnsemble The path ensemble we are writing restart info for.
- ` system` : object like System System is used here since we need access to the temperature and to the particle list.
- order_function : object like OrderParameter The class used for calculating the order parameter(s).
- engine : object like EngineBase The engine to use for propagating a path.
settings_ens (dict) – A dictionary with the ensemble settings.

pyretis.inout.restart.write_restart_file(filename, simulation)¶

Write restart info for a simulation.

Parameters:

filename (string) – The file we are going to write to.
simulation (object like Simulation) – A simulation object we will get information from.

pyretis.inout.screen module¶

Module defining the base classes for the PyRETIS output.

Important classes defined here¶

ScreenOutput (FileIO): A generic class for handling output to the screen.

Important constants defined here¶

PROGRESSint: Custom log level (25) between INFO (20) and WARNING (30). Used for user-facing progress messages (green on console).
BANNERint: Custom log level (26) between PROGRESS (25) and WARNING (30). Used for decorative/banner text (cyan on console): logo, version info, references.
REPORTint: Custom log level (29) between SUCCESS (28) and WARNING (30). Used for generated analysis report blocks (plain console text).

class pyretis.inout.screen.ScreenOutput(formatter)¶

Bases: OutputBase

A class for handling output to the screen.

target = 'screen'¶

write(towrite, end=None)¶

Write a string to the file.

Parameters:

towrite (string) – The string to output to the file.
end (string, optional) – Override how the print statements ends.

Returns:

status (boolean) – True if we managed to write, False otherwise.

pyretis.inout.settings module¶

This module handles parsing of input settings.

This module defines the file format for PyRETIS input files.

Important methods defined here¶

parse_settings_file (parse_settings_file()): Method for parsing settings from a given input file.
write_settings_file (write_settings_file()): Method for writing settings from a simulation to a given file.

pyretis.inout.settings.add_default_settings(settings)¶

Add default settings.

Parameters:: settings (dict) – The current input settings.
Returns:: None, but this method might add data to the input settings.

pyretis.inout.settings.add_specific_default_settings(settings)¶

Add specific default settings for each simulation task.

Parameters:: settings (dict) – The current input settings.
Returns:: None, but this method might add data to the input settings.

pyretis.inout.settings.fill_up_tis_and_retis_settings(settings)¶

Make the life of sloppy users easier.

The full input set-up will be here completed.

Parameters:: settings (dict) – The current input settings.
Returns:: None, but this method might add data to the input settings.

pyretis.inout.settings.parse_settings_file(filename, add_default=True)¶

Parse settings from a file name.

Dispatches on the file extension: .toml uses the canonical TOML reader; anything else (typically .rst) falls back to the legacy rst parser.

Parameters:

filename (string) – The file to parse.
add_default (boolean) – If True, we will add default settings as well for keywords not found in the input.

Returns:

settings (dict) – A dictionary with settings for PyRETIS.

pyretis.inout.settings.parse_settings_rst(filename, add_default=True)¶

Parse settings from the legacy .rst PyRETIS input format.

Deprecated since version The: rst input format is deprecated in favour of TOML and will be removed in a future release. This reader still works so existing scripts keep running; to silence the warning, run python -m pyretis.tools.convert_settings YOUR.rst and switch your workflow to the resulting .toml file.

Parameters:

filename (string) – The file to parse.
add_default (boolean) – If True, we will add default settings as well for keywords not found in the input.

Returns:

settings (dict) – A dictionary with settings for PyRETIS.

pyretis.inout.settings.parse_settings_toml(filename, add_default=True)¶

Parse settings from a canonical .toml input file.

The TOML schema mirrors the rst section layout one-to-one: each rst section is a TOML table of the same name; sections that can repeat (potential, collective-variable, ensemble) are TOML arrays-of-tables; hyphenated keys (order-file, trajectory-file …) are emitted with quoted keys.

The returned dict has the same shape as parse_settings_rst(), so downstream PyRETIS code does not need to know which frontend was used.

Parameters:

filename (string) – The TOML file to parse.
add_default (boolean) – If True, we will add default settings as well for keywords not found in the input.

Returns:

settings (dict) – A dictionary with settings for PyRETIS.

pyretis.inout.settings.write_settings_file(settings, outfile, backup=True)¶

Write simulation settings to an output file.

Dispatches on the output extension: .toml writes the canonical TOML schema; anything else writes the legacy rst.

Parameters:

settings (dict) – The dictionary to write.
outfile (string) – The file to create.
backup (boolean, optional) – If True, we will backup existing files with the same file name as the provided file name.

Note

This will currently fail if objects have made it into the supplied settings.

pyretis.inout.settings.write_settings_rst(settings, outfile, backup=True)¶: Write settings to a file in the legacy rst PyRETIS format.

pyretis.inout.settings.write_settings_toml(settings, outfile, backup=True)¶

Write settings to a file in the canonical TOML schema.

The schema mirrors the rst section layout one-to-one. None values are dropped (the parser refills defaults). The decorative heading section is dropped.

pyretis.inout.simulationio module¶

Definition of a class for handling output related to simulations.

Important classes defined here¶

Task (Task): Base class for tasks. This is used by SimulationTask and OutputTask.
OutputTask (OutputTask): A class representing a simulation output task.

Important methods defined here¶

get_task_type (get_task_type()): Do additional handling for a path task.
get_file_mode (get_file_mode()): Determine if we should append or backup existing files.
task_from_settings (task_from_settings()): Create output task from simulation settings.

Important variables defined here¶

OUTPUT_TASKS (OUTPUT_TASKS): A dictionary defining the different output tasks known to PyRETIS.

pyretis.inout.simulationio.OUTPUT_TASKS = {'cross': {'filename': 'cross.txt', 'formatter': <class 'pyretis.inout.formats.cross.CrossFormatter'>, 'result': ('cross',), 'target': 'file', 'when': 'cross-file'}, 'energy': {'filename': 'energy.txt', 'formatter': <class 'pyretis.inout.formats.energy.EnergyFormatter'>, 'result': ('thermo',), 'target': 'file', 'when': 'energy-file'}, 'order': {'filename': 'order.txt', 'formatter': <class 'pyretis.inout.formats.order.OrderFormatter'>, 'result': ('order',), 'target': 'file', 'when': 'order-file'}, 'path-energy': {'filename': 'energy.txt', 'formatter': <class 'pyretis.inout.formats.energy.EnergyPathFormatter'>, 'result': ('path', 'status'), 'target': 'file', 'when': 'energy-file'}, 'path-order': {'filename': 'order.txt', 'formatter': <class 'pyretis.inout.formats.order.OrderPathFormatter'>, 'result': ('path', 'status'), 'target': 'file', 'when': 'order-file'}, 'pathensemble': {'filename': 'pathensemble.txt', 'formatter': <class 'pyretis.inout.formats.pathensemble.PathEnsembleFormatter'>, 'result': ('pathensemble',), 'target': 'file', 'when': 'pathensemble-file'}, 'pathensemble-retis-screen': {'formatter': <class 'pyretis.inout.formats.txt_table.RETISResultFormatter'>, 'result': ('pathensemble',), 'target': 'screen', 'when': 'screen'}, 'pathensemble-screen': {'formatter': <class 'pyretis.inout.formats.txt_table.PathTableFormatter'>, 'result': ('pathensemble',), 'target': 'screen', 'when': 'screen'}, 'thermo-file': {'filename': 'thermo.txt', 'formatter': <class 'pyretis.inout.formats.txt_table.ThermoTableFormatter'>, 'result': ('thermo',), 'target': 'file', 'when': 'energy-file'}, 'thermo-screen': {'formatter': <class 'pyretis.inout.formats.txt_table.ThermoTableFormatter'>, 'result': ('thermo',), 'target': 'screen', 'when': 'screen'}, 'traj-txt': {'filename': 'traj.txt', 'formatter': <class 'pyretis.inout.formats.snapshot.SnapshotFormatter'>, 'result': ('system',), 'target': 'file', 'when': 'trajectory-file'}, 'traj-xyz': {'filename': 'traj.xyz', 'formatter': <class 'pyretis.inout.formats.snapshot.SnapshotFormatter'>, 'result': ('system',), 'target': 'file', 'when': 'trajectory-file'}}¶

Define a set of known output tasks.

The output tasks are defined as dictionaries with the following keys:

targetstring
“file” or “screen”, defines where the task writes to.
filenamestring
A default file name for an output file if writing to a file.
resulttuple of strings
Determines what item from the result dictionary we are outputting.
whenstring
Determines what input setting from the “output” section is used to define the output frequency. Default values are defined by the output section, see: py:mod:pyretis.inout.settings.settings.
formatterobject like OutputFormatter
Selects the formatter for the output.
writerobject like OutputBase
Selects the writer for the output.
settingstuple of strings, optional
A dict with additional settings which can be passed to the formatter if needed. These settings can, for instance, be defined in the output section of the input file.

The writer can be defined explicitly, or via the formatter. If a formatter is given, then the generic FileIO will be used. If no formatter is given, then the writer is assumed to be given.

class pyretis.inout.simulationio.OutputTask(name, result, writer, when)¶

Bases: Task

A base class for simulation output.

This class will handle an output task for a simulation. The output task consists of one object which is responsible for formatting the output data and one object which is responsible for writing that data, for instance to the screen or to a file.

Variables:

target (string) – This string identifies what kind of output we are dealing with. This will typically be either “screen” or “file”.
name (string) – This string identifies the task, it can, for instance, be used to reference the dictionary used to create the writer.
result (tuple of strings) – This string defines the result we are going to output.
writer (object like OutputBase) – This object will handle the actual outputting of the result.
when (dict) – Determines if the task should be executed.

__init__(name, result, writer, when)¶

Initialise the generic output task.

Parameters:

name (string) – This string identifies the task, it can, for instance, be used to reference the dictionary used to create the writer.
result (list of strings) – These strings define the results we are going to output.
writer (object like IOBase) – This object will handle formatting of the actual result and output to screen or to a file.
when (dict) – Determines when the output should be written. Example: {‘every’: 10} will be executed at every 10th step.

__str__()¶: Print information about the output task.

output(simulation_result)¶

Output given results from simulation steps.

This will output the task using the result found in the simulation_result which should be the dictionary returned from a simulation object (e.g. object like Simulation) after a step.

Parameters:: simulation_result (dict) – This is the result from a simulation step.
Returns:: out (boolean) – True if the writer wrote something, False otherwise.

task_dict()¶: Return a dict with info about the task.

class pyretis.inout.simulationio.Task(when)¶

Bases: object

Base representation of a “task”.

A task is just something that is supposed to be executed at a certain point. This class will just set up functionality that is common for output tasks and for simulation tasks.

Variables:: when (dict) – Determines when the task should be executed.

__init__(when)¶

Initialise the task.

Parameters:: when (dict, optional) – Determines if the task should be executed.

execute_now(step)¶

Determine if a task should be executed.

Parameters:: step (dict of ints) – Keys are ‘step’ (current cycle number), ‘start’ cycle number at start ‘stepno’ the number of cycles we have performed so far.
Returns:: out (boolean) – True of the task should be executed.

task_dict()¶: Return basic info about the task.

property when¶: Return the “when” property.

pyretis.inout.simulationio.get_file_mode(settings)¶

Determine if we should append or backup existing files.

This method translates the backup settings into a file mode string. We assume here that the file is opened for writing.

Parameters:: settings (dict) – The simulation settings.
Returns:: file_mode (string) – A string representing the file mode to use.

pyretis.inout.simulationio.get_task_type(task, engine)¶

Do additional handling for a path task.

The path task is special since we do very different things for external paths. The set-up required to do this is handled here.

Parameters:

task (dict) – Settings related to the specific task.
engine (object like EngineBase) – This object is used to determine if we need to do something special for external engines. If no engine is given, we do not do anything special.

Returns:

out (string) – The task type we are going to be creating for.

pyretis.inout.simulationio.task_from_settings(task, settings, directory, engine, progress=False)¶

Create output task from simulation settings.

Parameters:

task (dict) – Settings for creating a task. This dict contains the type and name of the task to create. It can also contain overrides to the default settings in OUTPUT_TASKS.
settings (dict) – Settings for the simulation.
directory (string) – The directory to write output files to.
engine (object like EngineBase) – This object is used to determine if we need to do something special for external engines. If no engine is given, we do not do anything special.
progress (boolean, optional) – For some simulations, the user may select to display a progress bar. We will then just disable the other screen output.

Returns:

out (object like OutputTask) – An output task we can use in the simulation.

pyretis.inout.checker module¶

This module checks that the inputs are meaningful.

Main methods defined here¶

check_ensemble (check_ensemble()): Function to check the ensemble settings.
check_interfaces (check_interfaces()): Function to check engine set-up.
check_for_bullshitt (check_for_bullshitt()): Function to compare nested dicts and lists.
check_engine (check_engine()): Function to check engine set-up.

pyretis.inout.checker.check_engine(settings)¶

Check the engine settings.

Checks that the input engine settings are correct, and automatically determine the ‘internal’ or ‘external’ engine setting.

Parameters:: settings (dict) – The current input settings.

pyretis.inout.checker.check_ensemble(settings)¶

Check that the ensemble input parameters are complete.

Parameters:: settings (dict) – The settings needed to set up the simulation.

pyretis.inout.checker.check_for_bullshitt(settings)¶

Do what is stated.

Just for the input settings.

Parameters:: settings (dict) – The current input settings.

pyretis.inout.checker.check_interfaces(settings)¶

Check that the interfaces are properly defined.

Parameters:: settings (dict) – The current input settings.

pyretis.inout.archive_paths module¶

Central construction of the scheduler’s trajectory-archive paths.

The infinite-swapping scheduler keeps its live accepted trajectories in a per-path archive: one subdirectory per global path number, holding the trajectory files plus the order.txt / energy.txt a resume reloads from. To keep the run’s output a single canonical tree, that archive lives inside the per-ensemble output directories, nested under each path’s birth ensemble (the ens_save_idx fixed when the path is first created and carried, swap-invariant, for its lifetime):

<ens_save_idx>/<subdir>/<path_number>/...

where <subdir> is the [simulation] load_dir name (default accepted) and the path directory holds the order.txt / energy.txt / traj.txt and the trajectory frames flat (no inner accepted/ subfolder). Nesting by the birth ensemble (rather than the current one) keeps a swap a zero-copy bookkeeping change – the swap-invariant anchor is the birth ensemble, so no trajectory file ever moves between ensemble directories.

The per-ensemble archive is the run’s OPERATIONAL store: it holds the live (active) paths at <ensemble>/<load_dir>/<pn>. A path that has been replaced everywhere moves to the per-ensemble LONG-TERM store (LONG_TERM_DIR, <ensemble>/archive/<pn> – the sibling of the operational store under the SAME birth ensemble, so the move is a same-directory rename) via long_term_path_dir(), thinned to the [output] archive_every cadence.

Routing every construction site (writer, eviction mover, restart reader, restart-viability check, initial seeding) through archive_root() / path_dir() gives a single seam. Passing ens_save_idx=None yields the legacy flat <subdir>/<path_number> layout, which the restart reader falls back to for an output.toml written before the per-ensemble nesting (its persisted load_dir is then the old top-level store name).

pyretis.inout.archive_paths.ARCHIVE_SUBDIR = 'accepted'¶: The default name of the per-ensemble trajectory-archive subdirectory ([simulation] load_dir). Single source for the 'accepted' literal; every construction site imports it from here. Runs recorded under a previous default (paths or trajs) keep working: the run file persists its own load_dir, which takes precedence over this default on restart.

pyretis.inout.archive_paths.LONG_TERM_DIR = 'archive'¶: The per-ensemble long-term trajectory store’s subdirectory name. A path that has been replaced in every ensemble is MOVED here from its operational per-ensemble store (<ens>/<load_dir>/<pn> -> <ens>/archive/<pn>, a same-parent rename), thinned to the [output] archive_every cadence. Analysis never needs the moved files (it reads the per-ensemble text output); the long-term store is for the user.

pyretis.inout.archive_paths.archive_root(config, ens_save_idx=None, base='', subdir=None)¶

Return the directory the per-path archive subdirectories live under.

Parameters:

config (dict) – The scheduler config; config['simulation']['load_dir'] names the per-ensemble archive subdirectory (default accepted). A non-default config['output']['data_dir'] re-roots the per-ensemble output directories (see pyretis.inout.pathensemble_output. _ensemble_dirs()), and the nested archive follows it – the archive lives INSIDE those directories. The legacy flat layout predates data_dir support and stays relative to the run directory.
ens_save_idx (int, optional) – The birth ensemble the path archive nests under. None selects the legacy flat layout (just the subdirectory, no ensemble parent).
base (str, optional) – A directory to resolve the (relative) root against – callers that need an absolute archive root pass the run directory (e.g. os.getcwd()); the default leaves the root relative.
subdir (str, optional) – The per-ensemble subdirectory name. None uses the operational store’s load_dir (default accepted); the long-term store passes LONG_TERM_DIR so replaced paths nest as <ensemble>/archive/<pn> beside the live <ensemble>/accepted/<pn>.

Returns:

str – The archive root directory.

pyretis.inout.archive_paths.long_term_path_dir(config, path_number, ens_save_idx=None, base='')¶

Return a path’s directory in the long-term store.

The long-term store is per-ensemble, nested under the path’s birth ensemble exactly like the operational store: a replaced path moves from <ensemble>/<load_dir>/<pn> to <ensemble>/archive/<pn> – a same-directory rename (guaranteed same-filesystem, no cross-device copy). A legacy flat-layout path (ens_save_idx=None) uses the flat archive/<pn> at the run root.

Parameters:

config (dict) – The scheduler config (locates the per-ensemble output dirs; see archive_root()).
path_number (int or str) – The global path number keying the store subdirectory.
ens_save_idx (int, optional) – The path’s birth ensemble; None selects the flat run-root layout.
base (str, optional) – A directory to resolve the (relative) store against – callers that need an absolute location pass the run directory; the default leaves it relative.

Returns:

str – <base>/<ensemble>/archive/<path_number> (or the flat <base>/archive/<path_number> for a legacy path).

pyretis.inout.archive_paths.move_to_long_term(config, path_number, ens_save_idx=None, base='')¶

Move a replaced path from the operational archive to long-term.

Called when a path has been replaced in every ensemble: its whole archive directory (text files + accepted/ trajectory frames) moves from the per-ensemble operational store <ensemble>/<load_dir>/<pn> to the per-ensemble long-term store <ensemble>/archive/<pn> (a same-parent rename), keeping the operational store bounded to the live paths without losing sampled trajectories.

The source is resolved through resolve_path_dir(), so a legacy-flat-layout path moves from its actual location. A missing source (e.g. an in-memory internal-engine path that never wrote trajectory files, or a directory already moved by a replayed cycle after a restart) is a no-op, not an error – the move is bookkeeping, never load-bearing for the sampling. An existing destination (a replayed cycle) is replaced, keeping the operation idempotent.

Parameters:

config (dict) – The scheduler config (locates the operational archive).
path_number (int or str) – The global path number to move.
ens_save_idx (int, optional) – The path’s birth ensemble (see archive_root()).
base (str, optional) – The run directory to resolve both stores against.

Returns:

str or None – The destination directory when the path was moved; None when there was nothing to move.

pyretis.inout.archive_paths.output_ensemble_number(config, ens_save_idx)¶

Map a birth-ensemble slot to its output ensemble number.

The per-ensemble output directories are numbered by route, and the trajectory archive nests under the SAME directory as a slot’s analysis logs (see pyretis.inout.pathensemble_output._ensemble_dirs()): an explore run’s positive ensembles are 1-based (slot j -> j+1, no 000); a single_tis run’s one ensemble is named by its interface number; every other route maps slot j -> j.

Parameters:

config (dict) – The scheduler config; its [simulation] route flags select the numbering.
ens_save_idx (int) – The birth-ensemble slot (0-based coordinator index).

Returns:

int – The output ensemble number to name the archive directory with.

pyretis.inout.archive_paths.path_dir(config, path_number, ens_save_idx=None, base='')¶

Return the archive directory for a single path.

Parameters:

config (dict) – The scheduler config.
path_number (int or str) – The global path number keying the archive subdirectory.
ens_save_idx (int, optional) – The birth ensemble to nest under (see archive_root()).
base (str, optional) – Resolved through archive_root() (see there).

Returns:

str – <archive_root>/<path_number>.

pyretis.inout.archive_paths.resolve_path_dir(config, path_number, ens_save_idx=None, base='')¶

Locate an EXISTING path archive, honouring the legacy flat layout.

path_dir() constructs where a path’s archive belongs under the current (nested, per-ensemble) layout; this function resolves where an existing path actually IS. The two differ for runs whose paths live in the legacy flat top-level <load_dir>/<path_number> store: a user-staged flat load directory (the upstream staging contract) read by a fresh run, or a run recorded before the per-ensemble nesting. Such a legacy run’s paths never move – even after a resume stamps a [current] ens_save_idx for them – so the fallback must apply whenever the nested directory is absent, not only when the birth ensemble is unknown.

Every reader of an existing archive (the restart loader, the restart-viability check, the eviction mover) resolves through this function, so they all agree on the location.

Parameters:

config (dict) – The scheduler config.
path_number (int or str) – The global path number keying the archive subdirectory.
ens_save_idx (int, optional) – The birth ensemble the nested layout files under (see archive_root()). None selects the flat layout directly.
base (str, optional) – Resolved through archive_root() (see there).

Returns:

str – The nested archive directory when it exists (or when neither layout does – so a missing path is reported at its canonical location); the flat legacy directory when only that one exists.

pyretis.inout.scheduler_archive module¶

The infinite-swapping scheduler’s path-archive writer.

This module holds the one genuinely scheduler-specific piece of the old formatter_repex module: the path-data formatters and the path archive used by the infinite-swapping coordinator. Everything else in formatter_repex duplicated the canonical PyRETIS file-IO/formatter code under pyretis.inout.formats, pyretis.inout.common.OutputBase and pyretis.inout.fileio.FileIO; those duplicates have been removed and the base classes are imported from their canonical location.

What is kept here is specific to the infinite-swapping data model and the archive layout the scheduler writes, and is not a duplicate of the classic code:

OrderPathFormatter / EnergyPathFormatter / PathExtFormatter: Path-data formatters whose format reads the infinite-swapping phase-point attributes (phasepoint.order, getattr(phasepoint, key) for the energy terms and phasepoint.config / phasepoint.vel_rev for the trajectory references). The classic path formatters in pyretis.inout.formats instead read phasepoint.particles and reconstruct derived energy terms, so they cannot be reused directly.
SchedulerPathStorage: The infinite-swapping path archive. It writes order.txt, energy.txt, traj.txt and the trajectory frames flat into <dir>/<path_number>/ (no inner accepted/ – it had no sibling rejected/ and only obscured the layout). It is called as output(step, {"path": ..., "dir": ...}) and returns the moved Path. (The retired classic PathStorage wrote a per-cycle accepted/rejected archive instead; this per-path writer is the only trajectory archiver left.)

class pyretis.inout.scheduler_archive.EnergyPathFormatter¶

Bases: EnergyFormatter

A class for formatting energy data for paths (scheduler restart store).

Deliberately a separate class from pyretis.inout.formats.energy.EnergyPathFormatter, not a duplicate to be merged: that one is the ANALYSIS writer and reconstructs the derived etot/temp (via its _dof_kb cache) so a reused/reloaded frame still reports them. This one is the scheduler’s RESTART store – it writes each frame’s energies exactly as the phase point carries them (missing terms stay nan), so a continuation reloads byte-faithful values and does not resurrect a reconstructed number the uninterrupted run never stored. Keep the two apart; do not “dedup” them (same reasoning as the sibling OrderPathFormatter full-precision override above).

__init__() → None¶: Initialise the formatter.

format(step: int, data: Any) → Iterable[str]¶

Format the energy data from a path.

Parameters:

step (int) – The cycle number we are creating output for.
data (Any) – A tuple containing the path (as an object like PathBase) as the first element and a string with the status for the path as the second element.

Yields:

out (str) – The strings to be written.

class pyretis.inout.scheduler_archive.FormattersEntry¶

Bases: TypedDict

To store formatters and output files together.

file: str¶

fmt: OutputFormatter¶

class pyretis.inout.scheduler_archive.OrderPathFormatter¶

Bases: OrderFormatter

A class for formatting order parameter data for paths.

Unlike the analysis order.txt (written by pyretis.inout.formats.order.OrderPathFormatter at the canonical 6-decimal precision for the histogram analysis), this is the scheduler’s RESTART store: pyretis.core.path_load.load_path() reads it back to reconstruct the active path’s phase points on a continuation. Restoring a rounded order parameter makes a reloaded frame’s value differ from the uninterrupted run, so any continuation whose Max-O / Min-O lands on a reloaded frame would diverge in the high-precision pathensemble.txt columns. Persist the order at full float64 round-trip precision (17 significant digits) so a restart reproduces the continuous run byte-for-byte; only the per-frame order column needs it (the integer time column is unchanged).

ORDER_FMT = ['{:>10d}', '{:>26.17e}']¶

__init__() → None¶: Initialise the formatter.

format(step: int, data: List[Any]) → Iterable[str]¶

Format the order parameter data for a path.

Parameters:

step (int) – The cycle number we are creating output for.
data (list) – A tuple on the form (Path, status) where Path is the PathBase to extract order parameters for and status is the string representing the status of the path.

Yields:

out (str) – The formatted order parameters.

class pyretis.inout.scheduler_archive.PathExtFormatter¶

Bases: OutputFormatter

A class for formatting trajectories.

The trajectories are stored as files and this formatter creates a file that includes the location of these files.

This is functionally identical to pyretis.inout.formats.path.PathExtFormatter; it is kept as a small scheduler-local copy so the restart-archive module (pyretis.inout.scheduler_archive) stays self-contained alongside its sibling restart-store formatters (OrderPathFormatter / EnergyPathFormatter above, which genuinely diverge for restart byte-identity). If the two are ever unified, verify the restart round-trip (test/infswap/simulations/test_restart_equivalence.py) still reproduces the continuous run byte-for-byte.

FMT = '{:>10} {:>20s} {:>10} {:>5}'¶

__init__() → None¶: Initialise the PathExtFormatter formatter.

format(step: int, data: List[Any]) → Iterable[str]¶

Format path data for external paths.

Parameters:

step (int) – The current simulation step.
data (list) – A tuple where data[0] is assumed to be the path (as an object like Path) and data[1] a string containing the status of this path.

Yields:

out (str) – The trajectory as references to files.

static parse(line: str) → List[str]¶: Parse the line data by splitting the given text on spaces.

class pyretis.inout.scheduler_archive.SchedulerPathStorage(keep_traj_fnames: list | None = None)¶

Bases: OutputBase

A class for handling storage of external trajectories.

Variables:

target – Determines the target for this output class. Here it will be a file archive (i.e., a directory based collection of files).
formatters (Dict[str, pyretis.inout.scheduler_archive.FormattersEntry]) – This dict contains the formatters for writing path data, with default filenames used for them.
out_dir_fmt – A format to use for creating directories within the archive. This one is applied to the step number for the output.

__init__(keep_traj_fnames: list | None = None)¶

Set up the storage.

Parameters:: keep_traj_fnames (list, optional) – A list of file extensions matched against the source directories of the trajectories; matching files are kept.

Notes

No formatters are passed to the parent class. This is because this class is less flexible and only intended to do one thing: write path data for external trajectories.

__str__() → str¶: Return basic info.

static _move_path(path: InfPath, target_dir: str, keep_traj_fnames: list, prefix: str | None = None) → InfPath¶

Copy a path to a given target directory.

Parameters:

path (InfPath) – The path to copy.
target_dir (str) – The location where we are moving the path to.
keep_traj_fnames (list) – A list of file extensions that are matched against the source directories in which the trajectories are stored. File extensions that match the pattern are also stored.
prefix (str, optional) – A prefix for the file names of copied files.

Returns:

path_copy (InfPath) – A copy of the input path.

formatters: Dict[str, FormattersEntry] = {'energy': {'file': 'energy.txt', 'fmt': <pyretis.inout.scheduler_archive.EnergyPathFormatter object>}, 'order': {'file': 'order.txt', 'fmt': <pyretis.inout.scheduler_archive.OrderPathFormatter object>}, 'traj': {'file': 'traj.txt', 'fmt': <pyretis.inout.scheduler_archive.PathExtFormatter object>}}¶

out_dir_fmt = '{}'¶

output(step: int, data: Any) → InfPath¶

Format the path data and store the path.

Parameters:

step (int) – The current simulation step.
data (Any) – A dictionary containing the path and the directory to write to.

Returns:

path (InfPath) – A copy of the path (moved to the new directory).

output_path_files(step: int, data: List[Any], target_dir: str) → List[Tuple[str, str]]¶

Write the output files for energy, path and order parameter.

Parameters:

step (int) – The current simulation step.
data (list) – A tuple containing:
- The path as an object like Path.
- A string containing the status of this path.
target_dir (str) – The path to where we archive the files.

Returns:

files (list) – The files created as a list of tuples. Each tuple contains:

The full path to the file.
A relative path to the file. The relative path is useful for organizing internally in archives.

target = 'file-archive'¶

write(towrite: str, end: str = '\n') → bool¶: We do not need the write method for this object.

pyretis.inout.scheduler_archive._generate_file_names(path: InfPath, target_dir: str, prefix: str | None = None) → Tuple[List[Tuple[str, int]], Dict[str, str]]¶

Generate new file names for moving or copying paths.

Parameters:

path (InfPath) – The path object we are going to store.
target_dir (str) – The location where we are moving the path to.
prefix (str, optional) – The prefix can be used to prefix the name of the files.

Returns:

out (tuple) – A tuple containing:

A list with new file names.
A dict which defines the unique “source -> destination” for the copy/move operations.

pyretis.inout.config_adapter module¶

Native-config compatibility layer for the infinite-swap coordinator.

This module builds the input half of a configuration-compatibility layer: it translates a canonical RETIS configuration (task = "retis", canonical [simulation] / [tis] / [retis] / [engine] / [orderparameter] sections, canonical [initial-path] method = "kick") into the configuration dictionary the infinite-swap (replica-exchange) coordinator consumes.

The translation is intentionally additive: it does not touch the in-process simulation flow. An opt-in route in pyretisrun calls to_scheduler_config() and then runs the (unchanged) coordinator on the translated dictionary.

The coordinator initiates from pre-generated paths on disk (load_dir); it has no kick-init of its own. So when the canonical config requests method = "kick" this module GENERATES the initial paths by running the kick initiation (pyretis.initiation.initiate_kick()) and serialises them into the load_dir the translated config points at – one accepted path per ensemble, in the coordinator’s external-path file format (traj.txt / order.txt / accepted/traj.xyz).

Notes

The internal engine (class = "Langevin") integrates from in-memory system.pos / system.vel and a system-attached force field, while external engines (gromacs, lammps, turtlemd) read their starting configuration from the file-backed phase points the coordinator’s propagate loop hands them (positions live in traj.xyz, read back by the engine). The internal engine runs fully through the coordinator regardless – EngineBase/internal.py support both; test/integration/test_pyretis_path_sampling.py exercises the full kick-init -> propagate -> per-ensemble-output path on it extensively, including at workers > 1.

TurtleMD is a partial exception: its own config nests particles/box/potential under [engine] ([engine.particles], …), which this module’s kick/load initiation (generate_load_dir() -> pyretis.setup. create_simulation) does not read – that in-process helper expects the top-level [particles]/[box]/[potential] sections the internal engine (and external engines driven the “classic” pyretis way) use instead. A TurtleMD translation (this module’s actual job) is unaffected and fully verified (see test/infswap/simulations/test_quantis_runner.py); only a fresh, kick-initiated canonical-syntax TurtleMD run cannot bootstrap its first load_dir through this route yet. Tracked as MERGE_TODO.md P7.10, not fixed here.

pyretis.inout.pathensemble_output module¶

Native per-ensemble output for the infinite-swap coordinator.

The coordinator internally shares each accepted path across ensembles via a frac vector instead of keeping a distinct scalar-weight path per ensemble. pyretisanalyse and the downstream crossing-probability / rate analysis expect the per-ensemble pathensemble.txt format – this is the only output format the coordinator writes, regardless of which TOML dialect (canonical or legacy-runner) launched the run; the former alternative infswap_data.txt is no longer written (see MERGE_TODO.md; WHAM-style analysis instead reconstructs the same matrix on demand, reconstruct_path_data_matrix() below).

This module bridges the two by emitting, for every ensemble j the accepted path contributed to in the current cycle, one pathensemble.txt row – reusing the canonical pyretis.inout.formats.pathensemble.PathEnsembleFormatter so the byte layout matches the classic output layout exactly. Alongside each row, one order.txt and energy.txt block (the # Cycle: block format of pyretis.inout.formats.order.OrderPathFormatter / pyretis.inout.formats.energy.EnergyPathFormatter) is appended in the same ensemble directory, because the analysis pairs those blocks with the pathensemble.txt rows one-to-one in row order (matched on the cycle number).

The frac -> Weight mapping¶

The crossing-probability analysis (pyretis.analysis.path_analysis.analyse_path_ensemble()) reads the Weight column as a high-acceptance weight: each row’s contribution to the order-parameter histogram is

mc_weight * (1.0 / Weight)

where mc_weight is the Monte-Carlo stationary count (1 for an accepted row, incremented on each following rejected row). The coordinator emits one accepted row per ensemble per cycle, so mc_weight == 1 for every row and the contribution reduces to 1.0 / Weight. To make the analysis histogram reproduce the infinite-swap frac-weighted histogram – where a path contributes its fractional occupancy frac[j] to ensemble j – the coordinator therefore writes

Weight = 1.0 / frac[j] (the per-cycle frac increment)

so that 1.0 / Weight == frac[j]. The Step column carries the global coordinator cycle. This is the empirically-validated realisation of the ratified frac -> Weight cycle decision.

The two folded-in columns (PathNumber, HA-weight)¶

Weight actually encodes ha_factor / frac_inc (see write_pathensemble_data()): for wire-fencing/stone-skipping moves ha_factor is the path’s high-acceptance crossing-count factor for this ensemble (1.0 for the indicator moves sh/wt and the minus ensemble), and frac_inc is this cycle’s fractional-occupancy increment. The combined Weight column is exactly what the classic crossing-probability histogram needs, but it is lossy for WHAM-style analysis (pyretis.analysis.wham_analysis, pyretis.analysis.path_weights): those need frac_inc and ha_factor as two independently-meaningful numbers (each ensemble’s WHAM q-factor weight is the raw, un-unweighted sum of frac_inc, not the sum of frac_inc / ha_factor), and ha_factor cannot be recovered from the combined Weight alone. Separately, the canonical PathEnsembleFormatter columns carry no path-identity column (the retired serial loop never needed one – each ensemble numbers its own paths), so there is no way to tell which accepted path a row belongs to, which a reconstruction needs to group a path’s rows across cycles.

So the coordinator appends those two numbers – the global PathNumber and the raw HA-weight (ha_factor) – as the coordinator row’s two trailing columns, after the 17 canonical columns (which stay byte-identical to the serial flow). frac_inc = ha_factor / Weight then recovers both numbers exactly. This retires the former separate ha_weight.txt sidecar; reconstruct_path_data_matrix() reads the columns directly, falling back to a pre-fold run’s ha_weight.txt only when a parsed row lacks them.

pyretis.inout.pathensemble_output.energy_output_requested(config: Dict[str, Any]) → bool¶

Return True when per-ensemble energy.txt output is requested.

See order_output_requested(); this is the energy-file counterpart (carried as energy_file, default 1).

Parameters:: config (dict) – The coordinator configuration dictionary.
Returns:: boolean – True when per-ensemble energy.txt output is requested.

pyretis.inout.pathensemble_output.init_pathensemble_files(state: Any, overwrite: bool = True) → None¶

Create the per-ensemble directories and file headers.

Each ensemble directory gets a pathensemble.txt with the canonical standard header, plus empty order.txt / energy.txt files when those are requested (the path-data files carry no file-level header in the classic layout – each appended block starts with its own # Cycle: comment), and an ha_weight.txt sidecar (see the module docstring), which is unconditional – unlike order/energy it is small and is what keeps WHAM-style analysis exact, so it is never opt-in.

Parameters:

state (object like pyretis.simulation.repex.InfSwapState) – The coordinator state.
overwrite (bool, optional) – When True (a fresh start, the default) the files are (re)written with a clean header. When False (a restart) only missing files are created and existing ones are kept for appending. The restart path must still (re)create missing files: a run that always restarts (method="restart", e.g. a canonical config seeded from an infswap restart that never wrote per-ensemble output) would otherwise never get its pathensemble.txt and leave the analysis with nothing to read. On a restart the existing files are also trimmed to the committed state.cstep first (see _trim_ensemble_dir_to_cstep()), so an ungraceful crash between the per-cycle output append and the run-file commit cannot make the replay double-count.

pyretis.inout.pathensemble_output.map_move_to_mc_code(move: str) → str¶

Map a coordinator move tag to the pathensemble.txt Mc code.

Parameters:: move (string) – The coordinator’s move tag (generated[0]), e.g. "sh" or "tr".
Returns:: string – The two-character Mc code.

pyretis.inout.pathensemble_output.order_output_requested(config: Dict[str, Any]) → bool¶

Return True when per-ensemble order.txt output is requested.

The input shim carries the [output] order-file interval as order_file (default 1). The analysis pairs one order.txt block with each pathensemble.txt row (see pyretis.analysis.path_analysis.analyse_path_ensemble()), and the coordinator writes a row every cycle a path contributes frac – so the interval reduces to an on (> 0) / off (<= 0) switch here.

Parameters:: config (dict) – The coordinator configuration dictionary.
Returns:: boolean – True when per-ensemble order.txt output is requested.

pyretis.inout.pathensemble_output.reconstruct_path_data_matrix(run_dir: str, nskip: int = 0) → List[List[float]]¶

Rebuild the infswap_data.txt matrix from per-ensemble output.

The canonical route never writes infswap_data.txt; WHAM-style analysis (pyretis.analysis.wham_analysis, pyretis.analysis.path_weights, pyretis.analysis.training_set) instead reconstructs the same matrix from the pathensemble.txt + ha_weight.txt pair in every numbered ensemble directory. For ensemble j and a captured row, frac_inc = ha_factor / Weight recovers the cycle’s fractional occupancy increment exactly (see the module docstring); a path’s Cxy for that ensemble is the sum of frac_inc over every cycle it was live there, and its HA for that ensemble is ha_factor itself (constant across those cycles by construction). Length and Max-O are path-intrinsic, so every row carrying a given path number must agree – a mismatch anywhere is a logic error in the writer, not data to silently average over, hence the loud failures below rather than a best-effort reconciliation.

Row order is not a byte-for-byte reproduction of the live writer’s append order (paths there are written when archived, in roughly-but-not-exactly path-number order under infinite swapping). Rows here are sorted by ascending path number instead. This is immaterial to pyretis.analysis.path_weights.get_path_weights() (and so to interfaces_from_data/infinit, its caller) and to pyretis.analysis.training_set.read_path_data() / pyretis.analysis.combine_data’s per-path remapping – all operate on the row set, not its sequence. It DOES affect the block-error / running-average machinery in pyretis.analysis.wham_analysis.analyse_wham_output() (a chronological proxy, not the original sequence); that estimator is superseded by the crossing-probability analysis once this function is wired in (see MERGE_TODO.md).

Two further, deliberate differences from the historical infswap_data.txt writer (verified empirically against it before it was retired, not assumed):

Still-live paths are excluded, via _read_active_paths() (restart.toml’s [current] active list) – matching the old writer, which only emitted a row once a path was archived, never for an in-progress (not-yet-final) total.
A path that never contributed a non-zero fraction anywhere has no row at all, unlike the old writer (which still wrote an all-"----" row for it on archival). Such a path leaves no trace in any per-cycle output file (frac_inc <= 0.0 rows are never written – see write_pathensemble_data()), so there is nothing to reconstruct it from. This is provably harmless: an all-zero row cannot change any sum, ratio, or weight any consumer computes from the matrix (it contributes exactly 0 everywhere it would appear).

Parameters:

run_dir (string) – The directory holding the numbered ensemble directories (000, 001, …).
nskip (int, optional) – Number of initial paths (by ascending path number) to discard.

Returns:

matrix (list of list of float) – Exactly the shape pyretis.analysis.wham_analysis. read_data_matrix() returns: one row per path, [path_nr, length, max_op, Cxy_0 .. Cxy_{n-1}, HA_0 .. HA_{n-1}].

pyretis.inout.pathensemble_output.write_pathensemble_data(state: Any) → None¶

Append the cycle’s frac-contribution rows per ensemble.

Realises the ratified cycle decision: each accepted path gets a row in every ensemble ``j`` it has a non-zero per-cycle frac increment in. For ensemble j, one row is appended to its pathensemble.txt for every live path that contributed frac to j this cycle. Weight is written as 1.0 / frac_increment (so the analysis, which reads Weight as a 1/HA-weight, recovers frac_increment as the histogram weight – see the module docstring); Step is the global coordinator cycle.

When requested, every appended row is accompanied by one # Cycle: block in the ensemble’s order.txt and energy.txt carrying the path’s per-phase-point order parameters and energies. The classic analysis (analyse_path_ensemble()) consumes these in lockstep – one block per pathensemble.txt row, matched on the cycle number – so the blocks are emitted exactly one per row, in row order. Every row is unconditionally also paired with one (path number, ha_factor) entry in ha_weight.txt (see the module docstring) – the WHAM-side consumers need frac_inc and ha_factor separately, not just their Weight ratio, and path identity to group/match rows, neither of which the canonical pathensemble.txt columns carry.

Parameters:: state (object like pyretis.simulation.repex.InfSwapState) – The coordinator state, after the cycle’s frac increments have been applied (state.last_frac_increment populated, mapping each ensemble index to a list of (path_number, frac) pairs).