# PyRETIS documentation, full text
Published at https://www.pyretis.org/dev/. The index with one line per page is at https://www.pyretis.org/dev/llms.txt.
The API reference is not included here; it is generated from docstrings and is linked from the index.
# About PyRETIS
PyRETIS is a Python library for rare event molecular simulations with emphasis on methods based on transition interface sampling and replica exchange transition interface sampling. The papers describing the PyRETIS program can be found here: https://doi.org/10.1002/jcc.24900 (PyRETIS 1, 2017), here: https://doi.org/10.1002/jcc.26112 (PyRETIS 2, 2020), and here: https://doi.org/10.1002/jcc.27319 (PyRETIS 3, 2024).
The work on PyRETIS was initiated by Titus van Erp and is used in the research and teaching activities in the theoretical chemistry group at the Norwegian University of Science and Technology.
PyRETIS is open source and is released under a GNU Lesser General Public license v2.1+. If you are interested in contributing to the PyRETIS project, please have a look at the developer guide and visit our git repository https://gitlab.com/pyretis/pyretis.
The current PyRETIS version is 4.0.0.dev0. For an overview of the official PyRETIS releases, please visit our git repository: https://gitlab.com/pyretis/pyretis/-/releases.
Since version 2.4, PyRETIS includes PyVisA, a program created to facilitate post-processing and data analysis.
## Contact
For questions about PyRETIS, requests, bug reports, or enquiries about collaborating with us, please contact Enrico Riccardi:
- e-mail: enrico.riccardi@uis.no
- web page: https://www.uis.no/nb/profile/enrico-riccardi
## The PyRETIS team
Head authors & project leaders:
- Titus van Erp
- Enrico Riccardi
Developers:
- Arthur Benigno Weidmann
- Jonas Jaeger
- Dinis O. Abranches
Former Developers:
- An Ghysels
- Daniel Tianhou Zhang
- Ola Aarøen
- Sander Roet
- Anders Lervik
Acknowledgments:
- Wouter Vervust
- Henrik Kiær
- Anastasia Maslechko
- Oda Dahlen
- Christopher Daub
- Mahmoud Moqadam
- César A. Urbina-Blanco
- Jocelyne Vreede
- Magnus Heskestad Waage
- Sudi Jawahery
- Raffaela Cabriolu
## To cite us:
When using PyRETIS, or one of our libraries, please cite us!
Software papers:
- A. Lervik, E. Riccardi and T. S. van Erp, PyRETIS: A well-done, medium-sized python library for rare events, J. Comput. Chem. 38, 2439-2451 (2017).
- E. Riccardi, A. Lervik, S. Roet, O. Aarøen and T. S. van Erp, PyRETIS 2: An improbability drive for rare events, J. Comput. Chem. 41, 370-377 (2020).
- O. Aarøen, H. Kiær and E. Riccardi, PyVisA: Visualization and Analysis of path sampling trajectories, J. Comput. Chem. 42, 435-446 (2021).
- W. Vervust, D. T. Zhang, A. Ghysels, S. Roet, T. S. van Erp and E. Riccardi, PyRETIS 3: Conquering rare and slow events without boundaries, J. Comput. Chem. 45, 1224-1234 (2024).
Studies based on PyRETIS:
In addition to the software papers above, the following published studies use PyRETIS or PyVisA to sample, analyse or visualise rare events. They include work by members of our group and by other authors building on the software.
- T. S. van Erp, M. Moqadam, E. Riccardi and A. Lervik, Analyzing complex reaction mechanisms using path sampling, J. Chem. Theory Comput. 12, 5398-5410 (2016).
- M. Moqadam, E. Riccardi, T. T. Trinh, A. Lervik and T. S. van Erp, Rare event simulations reveal subtle key steps in aqueous silicate condensation, Phys. Chem. Chem. Phys. 19, 13361-13371 (2017).
- E. Riccardi, O. Dahlen and T. S. van Erp, Fast decorrelating Monte Carlo moves for efficient path sampling, J. Phys. Chem. Lett. 8, 4456-4460 (2017).
- M. Moqadam, A. Lervik, E. Riccardi, V. Venkatraman, B. K. Alsberg and T. S. van Erp, Local initiation conditions for water autoionization, Proc. Natl. Acad. Sci. U.S.A. 115, E4569-E4576 (2018).
- E. Riccardi, E. C. van Mastbergen, W. W. Navarre and J. Vreede, Predicting the mechanism and rate of H-NS binding to AT-rich DNA, PLoS Comput. Biol. 15, e1006845 (2019).
- C. D. Daub, E. Riccardi, V. Hänninen and L. Halonen, Path sampling for atmospheric reactions: formic acid catalysed conversion of SO3 + H2O to H2SO4, PeerJ Phys. Chem. 2, e7 (2020).
- E. Riccardi, A. Krämer, T. S. van Erp and A. Ghysels, Permeation rates of oxygen through a lipid bilayer using replica exchange transition interface sampling, J. Phys. Chem. B 125, 193-201 (2021).
- S. Roet, C. D. Daub and E. Riccardi, Chemistrees: Data-Driven Identification of Reaction Pathways via Machine Learning, J. Chem. Theory Comput. 17, 6193-6202 (2021).
- A. Lervik, I.-H. Svenum, Z. Wang, R. Cabriolu, E. Riccardi, S. Andersson and T. S. van Erp, The role of pressure and defects in the wurtzite to rock salt transition in cadmium selenide, Phys. Chem. Chem. Phys. 24, 8378-8386 (2022).
- D. T. Zhang, E. Riccardi and T. S. van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023).
- V. Munizaga and M. L. Falk, The thermodynamic effects of solute on void nucleation in Mg alloys, J. Chem. Phys. 161, 044509 (2024).
- K. Wilke, S. Tao, S. Calero, A. Lervik and T. S. van Erp, NaCl dissociation explored through predictive power path sampling analysis, J. Chem. Theory Comput. 21, 4604-4614 (2025).
- W. Vervust, D. T. Zhang, E. Riccardi, T. S. van Erp and A. Ghysels, Path sampling challenges in large biomolecular systems: RETIS and REPPTIS for ABL-imatinib kinetics, Biophys. J. 124, 3932-3947 (2025).
Additional theory papers from our group:
- T. S. van Erp, D. Moroni and P. G. Bolhuis, A novel path sampling method for the calculation of rate constants, J. Chem. Phys. 118, 7762-7774 (2003).
- T. S. van Erp, Reaction rate calculation by parallel path swapping, Phys. Rev. Lett. 98, 268301 (2007).
- A. Ghysels, S. Roet, S. Davoudi and T. S. van Erp, Exact non-Markovian permeability from rare event simulations, Phys. Rev. Res. 3, 033068 (2021).
- W. Vervust, D. T. Zhang, T. S. van Erp and A. Ghysels, Path sampling with memory reduction and replica exchange to reach long permeation timescales, Biophys. J. 122, 2960-2972 (2023).
- D. T. Zhang, L. Baldauf, S. Roet, A. Lervik and T. S. van Erp, Highly parallelizable path sampling with minimal rejections using asynchronous replica exchange and infinite swaps, Proc. Natl. Acad. Sci. U.S.A. 121, e2318731121 (2024).
---
# PyRETIS License (GNU Lesser General Public License v2.1+)
Copyright (c) 2023 The PyRETIS team
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 2.1 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this program. If not, see .
---
# Documentation Terminology
This page records a few naming and wording conventions for user-facing documentation. The goal is to keep the documentation consistent across the user guide, examples, and API landing pages.
## Project Names
Use the configured substitutions in prose:
- Use |pyretis| for the project name.
- Use |pyvisa| for the visualization and analysis tool.
- Use pyretis run, pyretis analyse, and pyvisa for command names.
Use the literal package or module name when referring to Python imports, for example pyretis.analysis or pyretis.orderparameter.
## Scientific and File Terms
Use these forms in prose:
- order parameter for the concept.
- orderparameter only for the input section, package, or module name.
- Monte Carlo for the method.
- HDF5 for the format, and .hdf5 or .hdf5.zip for extensions.
- reStructuredText for the markup language, and .rst for files.
- Fortran for the language, unless quoting source text that uses another spelling.
## Style Notes
Prefer concise, direct instructions. For example, write “Run the analysis with” instead of “The analysis can be executed using”. Keep command examples in literal blocks and wrap filenames, input keys, and command-line flags in double backticks.
When a page describes a workflow, start with the result the user is trying to get, then give the commands or settings needed to get there. If a step depends on a previous example, link to that example near the start of the page.
---
# Cross-engine validation: water-dimer dissociation
This page records a reproducibility check that the external molecular-dynamics engines driven by PyRETIS reproduce the same rare-event kinetics on one simple molecular system with a shared force field. The system is the dissociation (“splitting”) of a hydrogen-bonded water dimer, with the O–O distance as the order parameter. The case and the scripts below live in examples/validation/engines/water_dimer/.
The point of the test is that GROMACS, LAMMPS and OpenMM integrate the identical potential, so if their rate-constant estimates agree (within statistical error) the PyRETIS machinery that drives them is reproducible across engines.
## The shared system
- Force field: flexible SPC/Fw water (Wu, Tepper and Voth, J. Chem. Phys. 124, 024503, 2006) – harmonic O–H bonds and H–O–H angle, Lennard-Jones on oxygen, point charges. The model is flexible (no SETTLE/SHAKE constraints), so the forces are identical across engines with no constraint-solver divergence.
- Conditions: gas-phase dimer in a 5 nm box with a bare (unshifted) 2 nm cutoff, so every engine evaluates the same 1/r + LJ for all dimer pairs. Langevin dynamics at T = 250 K with matched friction \gamma = 1/ps and a 0.5 fs time step. The temperature was chosen from MD scans: at 300 K the dimer splits in < 10 ps (not a rare event); at 150 K it stays bound for > 500 ps (too rare to converge); at 250 K it dissociates in ~100 ps – a genuine but tractable rare event.
- Order parameter: O–O distance. State A (bound) is O–O < 3.5 Å, state B (dissociated) is O–O > 6.5 Å, with eight interfaces between.
Every engine’s input is generated from a single source, wd_common.py, so each force field is derived from the same numbers.
## The force-field gate
Before any rate is compared, check_forcefield.py evaluates identical configurations – including deliberately strained ones that exercise the bond/angle constants and the factor-of-two prefactor convention that differs between LAMMPS and GROMACS/OpenMM – in all three engines and compares the potential energy:
```pyretis
conda run -n pyretis python check_forcefield.py
```
The engines agree to \le 3\times10^{-5} relative (the residual is the documented difference between the codes’ electrostatic constants). This gate must pass before a rate comparison means anything: a green RETIS run with a mismatched force field would silently compare different physics.
## Engines and samplers
- GROMACS – native PyRETIS RETIS, canonical streaming engine (class = "gromacs" -> GromacsEngine).
- OpenMM – native PyRETIS RETIS, in-process engine.
- LAMMPS – the streaming engine via infinite swapping (class = "lammps"); the rate is obtained from WHAM. This is a different sampler from the other two, so agreement is additionally a cross-sampler check.
## Results
Flux through \lambda_0 and the dissociation rate constant, converted to a common unit (1/ps). RETIS, 3000 cycles per engine.
## Water-dimer dissociation: cross-engine rate comparison (250 K).
engine
sampler
cycles
flux (1/ps)
rate (1/ps)
GROMACS
canonical RETIS
3000
0.161 (8 %)
0.0127 (22 %)
OpenMM
canonical RETIS
3000
0.177 (7 %)
0.0076 (18 %)
LAMMPS
infinite swapping
3000
0.221
0.0185
The bracketed numbers are relative block-error estimates.
Findings:
- The two native engines agree on the flux through \lambda_0 within statistical error (0.161 vs 0.177 /ps, ~1 \sigma), and on the rate within ~2 \sigma. The corresponding escape time (~80–130 ps) matches the direct-MD dissociation timescale (~100 ps), an independent sanity check.
- The streaming LAMMPS engine (a different engine and sampler) reproduces the same flux and rate to within ~1.4x / ~1.5x. The small systematic offset is consistent with the WHAM point-matching estimator and the infinite-swapping sampler; resolving whether it closes fully would need a longer run.
- Overall the three engines agree to within a factor of ~1.4 on the flux and ~2.4 on the rate at 3000 cycles – the level expected for RETIS with ~20 % block errors, and well within an order of magnitude. The force-field gate above guarantees they are integrating the same potential, so the residual spread is statistical / estimator-level, not a physics difference.
## Reproducing the result
On a node with gmx, lmp and openmm (conda env pyretis):
```pyretis
cd examples/validation/engines/water_dimer
python check_forcefield.py # the force-field gate (must pass)
python build_inputs.py # GROMACS + LAMMPS inputs from one source
python compare_engines.py run --steps 3000
python compare_engines.py compare # flux + rate per engine, in 1/ps
```
The streaming-LAMMPS load paths are bootstrapped by lammps_stream/generate_water_loadpaths.py (adapted from generate_H2_loadpaths); compare_engines.py calls it automatically.
The LAMMPS case of the comparison, lammps_stream/, runs the streaming engine (class = "lammps" -> LAMMPSEngine) via infinite swapping. The case directory lammps/ runs the same streaming engine under native PyRETIS RETIS (retis.toml) and is kept for reference. The relaunch engine class = "lammps_steps", which launches LAMMPS once per propagation, is not used here: its kick-initiation is slow on this system, since lambda_0 sits at the bound-state 99th percentile, so the velocity-kick walk needed to reach it requires many LAMMPS starts. GROMACS and OpenMM use canonical RETIS. Because the rate is sampler-independent, mixing samplers is sound – and the agreement between them is an extra cross-sampler validation.
---
# Extending and validating PyRETIS
This page is for developers adding a new Monte Carlo path move or a new engine to PyRETIS, and – just as importantly – checking that the addition reproduces the same physics as the existing code. A new move or a new engine changes how paths are produced, not which path ensemble is sampled, so a correct implementation must give the same crossing probability and rate constant as the established ones, within statistical error.
PyRETIS ships a small reproducibility suite for exactly this check, in examples/validation/. The short test-easy / test-heavy runs in CI cannot afford the cycles needed to drive the rare-event statistics down, so this suite lives in examples/ and is launched manually (on a cluster) when you add or change a move or an engine.
# Table of contents
- Designing a new path move
- Writing a shooting-point selector
- Implementing a new engine
- Validating reproducibility
- Convergence speed across moves
- Adding your move or engine as a case
## Designing a new path move
The unified path moves live in pyretis/core/moves.py. The existing moves are standard shooting (sh) and the sub-trajectory moves Stone Skipping (ss), Web Throwing (wt) and Wire Fencing (wf), each optionally in a high-acceptance variant. Their input keywords (shooting_moves, n_jumps, interface_sour, interface_cap, high_accept, sigma_v) are documented in the TIS input reference, and their output codes in Path types and rejection reasons.
When adding a move:
- Implement it alongside the existing moves, give it a short two-letter move code, and make it selectable through the shooting_moves keyword.
- Preserve (super-)detailed balance: the move must leave the path ensemble’s distribution unchanged. This is the correctness criterion, and it is what the validation suite measures – a move that breaks it will not reproduce the reference rate.
- Document the new move code in Path types and rejection reasons.
- Add a validation case for it (see below) and confirm it agrees with the other moves before relying on it.
## Writing a shooting-point selector
The biased shooting move (shooting_move = "bias", see user-section-tis-biased-shooting) delegates the choice of shooting point to a ShootingPointSelector (pyretis/core/shooting_selection.py). To add your own bias – analytic, density-based, or machine-learned – subclass it and implement a single method:
```
from pyretis.core.shooting_selection import ShootingPointSelector
class MySelector(ShootingPointSelector):
def weights(self, path):
# Return a 1-D numpy array of strictly positive, finite
# weights, ONE PER INTERIOR FRAME (path.phasepoints[1:-1]).
...
```
Contract for weights(path):
- Return exactly path.length - 2 values (interior frames only; the two end points are never shooting points).
- Every weight must be strictly positive and finite. Non-positive or NaN/inf weights raise rather than being silently clamped – a wrong selection weight silently biases the sampled rate, the worst failure mode for this code. (For exp-type weights, guard against underflow: a too-sharp bias that drives a frame to 0 is an error.)
The base class provides sum_weight(path) (the partition function W) and select(path, rgen), which draws one interior frame with probability w_i/W using a single uniform from the per-ensemble random generator. You do not re-implement selection or the acceptance correction; the move applies \min(1, W_\mathrm{old}/W_\mathrm{new}) automatically, so any positive weights is detailed-balance correct.
Load a user selector from the input like any other pluggable class, via a module/class pair in the [shooting-selector] section:
```
[shooting-selector]
module = "my_selectors.py"
class = "MySelector"
# any further keys are passed to MySelector.__init__
```
## Implementing a new engine
How to subclass the engine base classes – EngineBase / MDEngine for an internal engine, or ExternalMDEngine to drive an external program such as GROMACS, LAMMPS or OpenMM – is covered, with templates, in Creating custom engines.
For correctness, the key point is the same as for moves: an engine only changes how the dynamics are generated. On a shared system (same potential / force field, temperature, timestep and thermostat) a correct engine must reproduce the same rate as the others. Make the dynamics genuinely match before treating a rate disagreement as a bug – the rate depends on the initial flux, so a mismatched thermostat or timestep, not the engine code, is the usual culprit.
## Validating reproducibility
The suite in examples/validation/ has two groups, both modelling the same physical system so their rates must agree:
- methods/ – move equivalence on the internal engine: the same 1D double-well sampled with sh / ss / wt / wf (and the high-acceptance and infinite-swapping variants). A direct cross-check of the move implementations.
- engines/ – engine equivalence with a shared force field: the same dissociation run through GROMACS / LAMMPS / OpenMM.
It is driven by a per-machine config, validation.toml – an ordered list of cases with their target cycles, plus a per-machine seed, a reverse_list toggle, and jobs (internal cases run in parallel). The first positional argument selects the action:
```
python run_validation.py # show this usage help
python run_validation.py status # show recorded results (read-only)
python run_validation.py analyze # (re)analyse the runs on disk
python run_validation.py run # run every case, then analyse
python run_validation.py run sh_mt19937 # run just one case
```
cycles is a target total: a case that already has output is continued up to it, otherwise started fresh from seed. Use a different seed on each machine so two runs are independent and can be combined for better statistics. Each case is analysed as soon as it finishes, and the suite maintains a per-case results table in previous_results/ (tracked in git – so git diff of the table is the comparison across validation sessions; running one case updates just that case’s row). The analysis prints a rate comparison (ok / CHECK), the PCG64-vs-MT19937 RNG verdict, and writes the convergence plot rate_vs_cycles.png. See examples/validation/README.rst for the full description.
### Convergence speed across moves
All moves converge to the same rate – that agreement is the validation – but they do not converge at the same speed. The rate_vs_cycles.png plot shows each case’s running rate estimate against cycle number: a curve that flattens onto the common value sooner has converged faster.
This is where the suite doubles as a showcase. The sub-trajectory moves (Stone Skipping, Web Throwing, Wire Fencing 1 2) and the high-acceptance variants are designed to decorrelate successive paths more effectively than plain one-shot shooting, so on this system they typically reach a stable estimate in noticeably fewer cycles – which you can read straight off the plot. Running the methods/ group and looking at the overlaid curves is therefore both a correctness check and a practical guide to which move to reach for.
### Adding your move or engine as a case
- Copy an existing case directory – e.g. methods/wf_mt19937 for a new move, or one of the engines/ cases for a new engine – keeping the same double-well (or shared force field), temperature, timestep and interfaces, so the rate is required to match.
- Point the copy at your new thing (set shooting_moves to your move code, or select your engine).
- Register the case in the manifest in examples/validation/run_validation.py and list it in validation.toml.
- python run_validation.py run and confirm its row in the table agrees within n_sigma once converged. A CHECK usually means not converged yet – raise its cycles and re-analyse before concluding there is a discrepancy.
E. Riccardi, O. Dahlen and T. S. van Erp, Fast decorrelating Monte Carlo moves for efficient path sampling, J. Phys. Chem. Lett. 8, 4456-4460 (2017), https://doi.org/10.1021/acs.jpclett.7b01617
D. T. Zhang, E. Riccardi and T. S. van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023), https://doi.org/10.1063/5.0127249
---
# Developer Guide
PyRETIS is open source and hosted on gitlab. We are happy to include more developers and we want active users. If you wish to contribute to the PyRETIS project, please read through the code guidelines and the short description on contributing given below.
## Guidelines
The guidelines can be summarised as follows:
- Check that your code follows pep8.
- Document your code using NumPy style docstrings
- Use a logger rather than print() in libraries.
- Write comments and docstrings for a reader who arrives cold and who will never see your branch or any earlier version of the file. Everything they say must be a true statement about the code that is there. Three things are therefore never committed.
- History. What the code used to do, what changed, what was broken. No “used to”, “previously”, “no longer”, “now”, “still”, “fixed”, “was wrong”, “the new behaviour”. No commit hashes, no plan or issue labels, no references to who asked for or approved a change, and no pointers to working notes or review documents. One use of “now” stays: the state of the code at that point relative to other code – “the path is now held by the ensemble the swap chose”, or a test stepping through “the folder is now empty” right after the call that emptied it.
- What the code does not do. A reader needs the behaviour, not a tour of the behaviours that were rejected: no “this is not a paste bug”, no “rather than crashing”, no “instead of the old check”. Naming a boundary is fine where the boundary is the rule – “a leg that fails returns BTL above this point” says when a branch applies – but a sentence whose only content is a contrast with something absent from the code does not belong. One use of “rather than failing” / “instead of raising” stays: explaining error handling – a try/except, a caught exception, a fallback taken on error (“a missing file reads as empty rather than raising”).
- Justification by contrast. No “measured rather than assumed”, no “decided rather than guessed”. If a test measures something, it measures it.
- Keep the reasoning and drop the rest. “A two-frame path has no interior frame, so no shooting point can be drawn from it” is useful forever. Where a decision has a scientific source, cite the paper, not the discussion that led to it. The history belongs in the commit message and the issue tracker, which is where someone looking for it will go.
- Such text is meaningless to a user reading the source or the API documentation, it dates immediately, and it makes a scientific codebase read like a changelog of internal disagreements.
- devtools/prose_audit.py finds the phrases of all three kinds, and test/docs/test_comment_rule.py holds the tree to it. A site that keeps a phrase under one of the two exceptions is listed, with the exception that applies, in test/docs/prose_audit_allowed.json.
PyRETIS follows the Python pep8 style guide (see also pep8.org) and new code should be checked with the pep8 style guide checker and pylint:
```pyretis
pycodestyle source_file.py
pylint source_file.py
```
or other tools like PyChecker or pyflakes. NumPy’s imports can be a bit tricky understand so you can help pylint by doing
```pyretis
pylint --extension-pkg-whitelist=numpy source_file.py
```
The PyRETIS project is documented using the NumPy/SciPy documentation standard and contributors are requested to familiarise themselves with this style and use it. For user-facing documentation, also see the documentation terminology guide. Documentation style can be checked with pydocstyle
```pyretis
pydocstyle source_file.py
```
We also try to avoid print() statements in the libraries and reserve such statements for console output from command line scripts/programs. Output, like debug information, warnings etc. can be handled by using a logger. In a library, we typically just import and set up the logger by doing:
```python
import logging
logger = logging.getLogger(__name__) # pylint: disable=C0103
logger.addHandler(logging.NullHandler())
```
We can then use it as follows:
```python
# Report some information
logger.info('Up and running')
logger.debug('Debug info')
logger.warning('This is a warning')
logger.critical('Something critical happened!')
logger.error('An error occurred!')
```
Please note that we do not add any particular handlers here - it is up to the user to define what should happen when a logging event occurs.
### Testing your changes
PyRETIS ships two helper scripts in the repository root that bundle the checks a contribution is expected to pass. Run them from the main directory before you commit and before you open a merge request.
The fast check you run on a normal machine before every commit. It runs the unit-test suites (test/, test/infswap/ and test/integration/) under pytest with coverage, followed by the style and lint checks: pycodestyle, pydocstyle and the project pylint wrapper (devtools/run_linting.py). It is deterministic and finishes in a few minutes.
The slower, end-to-end check you run before pushing changes that can affect numerical results – in particular anything touching an engine, an order parameter or reference data. It chains two driver scripts: the engine reference-data consistency suites (examples/run-tests.sh – internal, GROMACS, PyVisA, CP2K, OpenMM and LAMMPS, each comparing freshly generated output against stored reference data) and a tutorial smoke run (examples/run-tutorials.sh, capped at two seconds per tutorial to confirm each still launches). It is deliberately resilient – both drivers always run, the full output is tee’d to test-heavy.log, and a PASSED/FAILED recap is printed at the end – so it is best run on a cluster.
See testing your installation for the detailed breakdown of what each stage covers and the environment variables that tune the engine scope and tutorial length.
## Contributing to PyRETIS
There is a nice guide on github about contributing to open source projects. In PyRETIS we largely follow this approach and the issue tracker is used for reporting bugs and issues and requesting new features. This section on contributing is based on the description for gitlab.
Before contributing, please read the short guidelines given below for reporting bugs and issues and for requesting new features.
### Bugs and issues
If you find a bug in PyRETIS, please create an issue using the following template:
```restructuredtext
Summary
-------
One sentence summary of the issue: What goes wrong, what did you expect to happen
Steps to reproduce
------------------
Describe how the issue can be reproduced.
Expected behavior
-----------------
Describe what behavior you expected instead of what actually happens.
Relevant logs
-------------
Add logs from the output.
Output of checks
----------------
Be sure that all the tests pass before filing the issue.
Possible fixes
--------------
If you can, link to the line of code that might be responsible for the problem.
```
If you wish to fix the bug yourself, please follow the approach described for merge requests below.
### New features
If you are missing some functionality in PyRETIS you can create a new issue in the issue tracker and label it as a feature proposal. If you do not have rights to add the feature proposal label, you can ask one of the core members of the PyRETIS project to add it for you.
You can also implement the changes you want yourself as described below. We cannot promise that we will automatically include your work in PyRETIS but we are happy to have active users and we will consider your contribution. So, when you are finished with your work please make a merge request.
### Merge requests
The general approach for making your bug-fix or new feature available in the PyRETIS project is as follows:
- Fork 1 PyRETIS into your own personal space.
- Create a new branch and work on your feature.
- Create tests for your feature. Especially, if you are fixing a bug, create a test that shows where the bug is causing PyRETIS to fail and how your code is fixing this.
- Squash all your commits into one and push to your fork.
- Submit a merge request to the master branch using the template given below.
- Wait for review and the following discussion :-).
```restructuredtext
Summary
-------
What does this merge request do?
Justification
-------------
Why is this merge request needed?
Description of the new code
---------------------------
A short description of the new code.
Are there points in the code the reviewer needs to double check?
References
----------
Add references to relevant issue numbers.
```
After submitting the merge request the code will be reviewed 2 by (another) member of the PyRETIS team.
Gitlab documentation on forking: http://doc.gitlab.com/ce/workflow/forking_workflow.html
https://github.com/thoughtbot/guides/tree/master/code-review
---
# Infinite-swapping integration status
The infinite-swapping source-tree merge was completed on 2026-07-06. This page summarises the resulting architecture, the conventions still in force, and where the historical validation records live.
## Goal
PyRETIS has one path-sampling execution path: the scheduler in pyretis/simulation/ drives the unified moves, path/system types, and engine classes. Canonical and runner-style TOML remain separate input schemas, but both feed this scheduler and emit the same per-ensemble output.
Two rules continue to govern development:
- One terminology. PyRETIS uses method-based names throughout: the analysis is WHAM and the sampler is infinite swapping (stem infswap). User-facing documentation and commands use these names consistently.
- Reproducibility first. Results must be deterministic and correct. A reference comparison is only useful if it fails when the physics is wrong (see developer-merge-status-tests).
## What works today
- pyretis run is the single entry point. It runs all path-sampling tasks through the scheduler and enables parallel infinite swapping via [simulation] task = "infinite_swapping" or the [runner] section.
- The WHAM crossing-probability / rate analysis lives in pyretis/analysis/wham_analysis.py (a faithful implementation of the standard weighted-histogram procedure for TIS; an earlier analysis stub was scientifically wrong).
- All tasks share pyretis.core.system_core.System and pyretis.core.path.Path; disk loaders live in pyretis.core.path_load. The random-number generator is PCG64.
## How to configure the scheduler
```bash
# Classical RETIS / TIS, adapted to the scheduler with one worker:
pyretis run -i retis.toml -p
# infinite swapping (multi-worker, infinite-swap + wire fencing):
# the TOML selects it via task = "infinite_swapping" or [runner]
pyretis run -i infswap.toml
```
See examples/validation/ for runnable method and engine comparisons.
## Test policy (green == correct)
- Run ./test-easy.sh (unit + integration + style) before every commit.
- Run ./test-heavy.sh (engine reference suites + tutorials, 40-60 min) before a push – test-easy does not run the example compare.py harnesses, so a reference/default change can pass test-easy and still break test-heavy.
- A comparison must not pass on degenerate input. The shared primitives in pyretis/testing/simulation_comparison.py reject empty data, shape mismatches, NaNs at mismatched positions, and entirely-NaN columns; a quantity an engine does not report must be declared explicitly by name, never accepted merely because both sides contain NaN.
## Resolved issues
- test-gromacs/test-load/test-initialise previously asserted that a loaded run bit-for-bit reproduces a fresh run – that is restart semantics, not load semantics. It has been reframed to load semantics (commit b619a767): it now runs the loaded scenario twice from the same loaded path and initial GROMACS RNG state and compares the two, which is what a load must guarantee (it works and reproduces previously loaded results). Rigorous load-determinism is covered by the internal engine (test-internal/retis-load-sparse), and exact continuation by the restart tests (test-internal/{retis,tis,md,mdflux}-restart, test-gromacs/test-restart).
- The merge-readiness review’s findings (H1–H4, M) are fixed: the load-traj energy comparison is data-driven per ensemble (not a blanket skip); pyretis run restores the completed-restart guard and the descriptive missing-input message; the WHAM lamres default uses the smallest interface gap with a commensurability check; and _wham_pq always returns one crossing probability per interface.
## Known issues
- CP2K loaded frames have no computed energy and declare those unavailable terms per ensemble. The reference is the deterministic conda-forge 2026.1 build (single rank, re-blessed 2026-07-19); other builds run execution-smoke only.
- The former GROMACS 5.1.4 tarballs were re-baselined onto the single current reference build (double-precision 2025.4, re-blessed 2026-07-19), so one executable validates the complete GROMACS reference set.
- External numeric tolerances remain provisional until repeated pinned-engine runs provide measured error envelopes. OpenMM and LAMMPS use additional byte-exact gates on their recorded reference builds.
---
# Method-equivalence validation: the double-well on one engine
This page records a reproducibility check that every sampling strategy and every code path in PyRETIS reproduces the same crossing probability and rate constant on one simple system, and that the result matches an absolute analytical truth rather than only agreeing with itself. The system is a 1D double well sampled on the internal engine; the cases and the driver script live in examples/validation/methods/ and are launched by examples/validation/run_validation.py.
The point of the test is that every sampling strategy, both RNG generations (the legacy MT19937 pin and the shipping PCG64 default), the multi-worker pool, and both rate estimators (the matched crossing-probability report and WHAM) all work over the identical potential, so if their rate estimates agree – and land on the analytical Kramers rate – the sampling machinery is reproducible across strategies, generators and estimators. This complements the cross-engine validation, which checks engine equivalence on a shared force field.
## The shared system
Every method case models the same 1D double well, so they must give the same rate within statistical error:
- Potential: the quartic double well V = a x^4 - b x^2 + c with a = 1 and b = 2 (so the barrier is \Delta V = b^2 / (4 a) = 1), c = 0.
- Dynamics: inertial Langevin at temperature T = 0.07 with a time step dt = 0.025 and friction gamma = 0.3 (reduced units), unit mass. With these numbers \beta\,\Delta V = 14.3, so escape over the barrier is a genuine rare event.
- Order parameter: the particle position, with eight interfaces spanning [-0.99 \ldots 1.0].
The system parameters are not hard-coded into the analysis – they are read back out of each case’s input TOML (read_system_params), so the analytical reference below tracks the actual runs, and every method case is cross-checked to confirm it really models the same well.
## The analytical reference – a known truth
The double well is simple enough to have a closed-form escape rate from Kramers’ theory, so the suite checks the simulations against an absolute truth, not only against each other. analytical_double_well_rate() evaluates it in three increasingly complete forms:
## Analytical double-well escape rate (reduced units).
Estimator
Rate
Meaning
k_TST
2.81e-07
transition-state theory, no recrossing (upper bound)
k_Kramers (spatial)
2.61e-07
moderate/high-friction Kramers prefactor
k_Kramers-MM (truth)
2.52e-07
- Mel’nikov–Meshkov turnover factor (headline)
With \beta\,\Delta V = 14.3 and reduced energy loss \delta = 11.4 the turnover factor is \Upsilon = 0.97 – the system sits firmly in the spatial-diffusion regime, so the Kramers result is accurate to a few percent. Converged cases should land on 2.5e-07 within their statistical error. A case can agree with every other case yet sit several sigma from the analytical rate – a shared systematic that the self-consistency check alone cannot see.
## The methods
The same double well is sampled with every available strategy and checked through both estimators. Every run goes through the single pyretis run entry point; the cases differ in their RNG pin (*_mt19937 requests the legacy generator explicitly, *_pcg64 pins the shipping default, *_default leaves the config unpinned – byte-identical to its *_pcg64 sibling) and in their analysis kind: matched (the point-matching product of per-ensemble crossing probabilities, reported by pyretis analyse) or wham (the WHAM stitch of the same per-ensemble output, e.g. infswap_wham).
## Method cases on the shared double well.
Case
Strategy / what it adds
sh_mt19937
standard shooting, pinned to the legacy MT19937 generator
ss_mt19937
stone skipping, legacy MT19937 pin
wt_mt19937
web throwing, legacy MT19937 pin
wf_mt19937
wire fencing, legacy MT19937 pin
wf_ha_mt19937
wire fencing with high acceptance, MT19937 pin
wf_cap_*
wire fencing with a capped window ([tis] interface_cap)
wt_sour_*
web throwing with a shifted source sub-interface ([tis] interface_sour)
relshoot
RETIS with relative per-ensemble shoot frequencies ([retis] relative_shoots)
*_pcg64
the same samplers pinned to the PCG64 generator (rgen = "pcg64") – the A3.4 RNG-migration validation set
infswap_wham
infinite swapping analysed with WHAM – the cross-check of the second estimator
*_default
the same samplers with NO RNG pin (the shipping default): byte-identical twins of their *_pcg64 siblings, a routing/determinism guard
sh_w3
the default config with a multi-worker pool of three workers (PYRETIS_WORKERS=3) – the parallel-worker route
sh_priority
the default config with priority_shooting = true (group priority; sh_priority_w3 runs it on three workers) – only the allocation of moves changes, so it must reproduce the same rate
bias_gaussian
biased shooting-point selection (Gaussian selector toward the barrier) – the move biases the shooting point but preserves the path ensemble, so it must reproduce the same rate (bias_gaussian_default is its byte-twin, bias_uniform the uniform-selector identity check)
The wf_cap_* / wt_sour_* / relshoot cases (group params) vary non-default TIS knobs that change the sampling but not the physics, so each must still reach the same analytical rate. The wf_convention group (wf_mt19937/wf_ha_mt19937, infswap_wham, wf_default) cross-checks wire fencing across the estimators. On the scheduler’s per-ensemble-output route the wf occupancy is HA-weighted (the compute_weight crossing count drives the swap, as in the infinite-swapping sampler), so the output writer applies the WHAM Cxy/HA unweighting (Weight = compute_weight / frac) to recover the per-ensemble crossing probability – without it the rate over-counts by ~two orders of magnitude; ss_default carries the same treatment for stone skipping. The suite config’s select key (group tags such as wf_convention or params) runs a chosen subset without dropping the rest.
The matched-kind rows (analysed with the standard PyRETIS crossing-probability report) and the wham-kind rows (analysed with WHAM over the identical per-ensemble output) are a direct cross-check of the two estimators. The *_pcg64 cases are the A3.4 step 2 go/no-go: they run the identical samplers with the PCG64 generator instead of the legacy MT19937 and must reproduce the MT19937 rate within statistical error before the default generator is flipped. The *_default twins must stay byte-identical to their *_pcg64 equivalents – a routing/determinism guard.
ss_default (stone skipping under the worker pool) earlier crashed the internal engine’s streaming dump (FileNotFoundError on ss_shoot.xyz) because the move re-pointed a persistent path’s phase point at the transient dump file. That is fixed (stone skipping now dumps a copy), and the case is enabled like the others.
## REPPTIS on two Brownian-dynamics systems
REPPTIS rebuilds the kinetics from partial paths, assuming that a path forgets where it came from once it crosses the next interface. Overdamped (Brownian) dynamics, [engine] high_friction = true, is Markovian in the position, so the assumption holds and REPPTIS must give the exact result. With the Langevin dynamics of the double well above (friction 0.3) the velocity is remembered over a distance of about 0.9, nine interface spacings, and REPPTIS is an approximation there. The group repptis_validation therefore samples two systems with Brownian dynamics, each with a REPPTIS case and its RETIS twin (the inputs differ only in [simulation] task), and each case is compared with its own system’s analytical value:
Cases
System and analytical value
bd_dw_retis, bd_dw_repptis
The double well at k_B T = 0.2 (\beta \Delta V = 5) with D = k_B T/(m\gamma) = 0.2. In one dimension a period in the overall state A runs from the entry into A at \lambda_A to the entry into B at \lambda_B, so the TIS rate is exactly the inverse mean first passage time between them, computed by quadrature from the case parameters: k = 5.49 \times 10^{-3}. It holds at any temperature; at k_B T = 0.2 the Kramers formula in the Smoluchowski limit is 10 % too high.
bd_flat_retis, bd_flat_repptis
A flat membrane with D = 0.01 and h = 0.2, the Brownian test system of Ghysels et al. 1, in permeability mode: P = D/h = 0.05. The reference bin of \tau sits in the middle of [\lambda_{-1}, \lambda_0]; on a flat potential its position does not matter, so a [0^-] ensemble that misses part of the region shows as a wrong permeability.
The status table’s “vs ref” column compares each case with its own system’s value; the flat cases record the permeability in the rate column.
A. Ghysels, S. Roet, S. Davoudi and T. S. van Erp, Exact non-Markovian permeability from rare event simulations, Phys. Rev. Research 3, 033068 (2021), https://doi.org/10.1103/PhysRevResearch.3.033068
## Running it and reading the output
The suite is not a unit test and not a tutorial, and it is not run per change or per release – see when to run it. Short developer runs can be launched manually. The full multi-seed campaign does not run in GitLab CI: it needs on the order of 36 hours, far beyond the one-hour job timeout, so it belongs on a workstation or cluster node you control, typically driven by campaign_run.sh. CI carries only the method_validation_smoke launch check – a single case at ten cycles – which proves the driver still runs end to end but deliberately produces no validation evidence. All modes use a per-run config such as validation.toml – an ordered list of cases with their target cycle counts plus a per-machine seed and a reverse_list toggle. The first positional argument selects the action:
```pyretis
cd examples/validation
# usage helper (also printed when no action is given)
python run_validation.py
# show the recorded results -- READ-ONLY: runs nothing, writes nothing
python run_validation.py status
python run_validation.py status sh_mt19937 # ... for one case
# (re)analyse the runs already on disk and update the results table
python run_validation.py analyze
python run_validation.py analyze sh_mt19937 # ... for one case
# run every case listed in ./validation.toml, then analyse
python run_validation.py run
python run_validation.py run sh_mt19937 # ... for one case
```
One-off overrides, usable with run:
```pyretis
python run_validation.py run --config machineB.toml # different per-machine config
python run_validation.py run --cycles 20000 # override every case's target
python run_validation.py run --seed 2 # this machine's seed
python run_validation.py run --jobs 8 # internal cases in parallel
```
--jobs N runs that many internal-engine (methods/) cases at once; they are independent single-process runs, so this only uses more cores and does not change any result (each case has its own directory and seed). --cycles is a target total, not an increment – a case with existing output is continued up to that count, otherwise it starts fresh from seed. python run_validation.py --write-config validation.toml writes a fresh config template with every case enabled.
Each case is analysed as soon as it finishes – its rate prints on the go – and a combined inventory, convergence plot and comparison follow at the end. The analysis prints a “Rate vs analytical Kramers reference” table (rate, k/k_ref, |d|/sigma, agree?) next to the table against the mean of the cases of the same system (the water-dimer engine cases are compared with their own mean). The persistent summaries land in validation_results.json (one entry per case: rate, relative error, cycles, number of independent runs, the agreement flags agree_analytical and agree – against the analytical rate and against that mean, shown as “vs ref” and “vs mean” – plus provenance – when, at which git commit, and whether the tree was dirty) and the human-readable validation_results.html (the same table with the convergence rate_vs_cycles.png embedded). A CHECK in the comparison almost always means not converged yet – raise that case’s cycles and re-analyse.
See examples/validation/README.rst for the full driver reference, including the two-machine combine-for-statistics workflow.
### When to run it
Seldom, and only for substantial changes. This campaign is not the routine correctness gate and must not be treated as one. Day-to-day confidence that a modification still produces the correct behaviour comes from the ordinary suites:
- the unit and integration tests (test-easy.sh), which pin the algorithms against hand-computed and synthetic oracles;
- the example suites (test-heavy.sh), which compare complete runs byte-for-byte against committed reference output on every engine;
- the tutorial smoke run, which proves every documented workflow still launches.
Those catch a regression far faster, far more cheaply, and far more precisely than a 36-hour statistical campaign can: a golden comparison fails on the first differing digit, whereas a rate estimate has to out-run its own error bars before a change becomes visible at all. A campaign that merely reproduces what the goldens already pin has told you nothing new.
Re-run the campaign when a change can move the sampled distribution itself, so that the goldens are expected to shift and can no longer serve as the reference:
- a new or modified shooting move, swap move, or acceptance rule;
- a change to the random-number generator, or to the order in which random numbers are drawn;
- a change to the scheduler or the coordinator’s ensemble bookkeeping;
- a new or modified rate / crossing-probability estimator, or a change to the high-acceptance reweighting;
- a deliberate re-blessing of the reference output, where a campaign is what distinguishes “the numbers moved for the named reason” from “the numbers moved and we no longer know whether they are right”.
Do not re-run it for refactors that keep the goldens byte-identical, nor for I/O, CLI, packaging, documentation, engine plumbing, new examples, or test-infrastructure work. If test-heavy.sh reproduces every committed reference byte-for-byte, the sampled distribution did not move, and the recorded campaign still describes the current code.
### How the evidence is recorded
The campaign runs off CI, on whatever machines you have. campaign_run.sh takes the seed, the reverse_list flag and a wall-clock budget as arguments, so two or more machines given different seeds sample independently and combine for statistics, and it continues each case to an ever-increasing cycle target; validation_campaign.toml lists the thirteen cases at 20,000 target cycles. The recorded result is the tracked previous_results/validation_results.json, whose per-case rows carry the rate, relative error, cycle count, agreement flags, and the provenance of the host and commit that produced them. That file is the standing evidence: it stays valid until one of the changes listed above invalidates it, which is why the campaign needs to run rarely rather than per release.
What CI contributes is only method_validation_smoke: one case at ten cycles, run on a throwaway copy of examples/validation so it cannot touch the recorded table. It proves the driver still executes end to end after a refactor and nothing more – ten cycles cannot resolve a rate, and the job publishes no artifacts precisely so its numbers can never be mistaken for evidence.
A row is evidence only when validation_results.json records the analytical-rate agreement and confidence information at a full cycle count; job completion alone is not a scientific pass, and neither is a green method_validation_smoke.
---
# Scientific evidence matrix
This matrix records why a numerical result is believed, not merely where its implementation is exercised. DIRECT means a test derives an expected value by hand or from a closed form. INDEPENDENT means a second algorithm or analytical physical model reaches the same result. REGRESSION means a frozen output is useful for detecting change but is not, by itself, evidence that the value is scientifically correct.
# Central numerical outputs and their independent evidence
Result
Implementation
Independent evidence
Status
Common means, block errors, MSD
analysis.analysis
Hand-computed running means, one- and two-point block standard errors, and the squared-lag MSD in test_analysis.py.
DIRECT
MD crossing flux
flux_analysis
Hand-counted crossings per physical time window in test_flux_analysis.py; end-to-end rates are also checked by the double-well campaign.
DIRECT + INDEPENDENT
TIS/RETIS crossing probability
path_analysis.analyse_path_ensemble and match_probabilities
Accepted-path integer ratios and their explicit product in test_path_analysis.py.
DIRECT
RETIS flux and rate
path_analysis.retis_flux and retis_rate
Direct inverse-path-time and product/error-propagation formula checks; the method campaign compares the final rate with Kramers–MM theory.
DIRECT + INDEPENDENT
WHAM crossing probability and rate
wham_analysis
Hand-built unweighted and weighted matrices with exact non-zero crossing probabilities; single-ensemble agreement with point matching; Kramers–MM campaign.
DIRECT + INDEPENDENT
REPPTIS local/global probability
repptis_analysis and _repptis_state
Raw-column occupancy counts feed both a closed-form recursion and a separately constructed Markov matrix in test_repptis_reference_oracle.py.
INDEPENDENT
REPPTIS MSM probability and MFPT
repptis_msm
Symmetric-random-walk closed form, a hand-computed three-state MFPT, stochastic-matrix invariants, and comparison with the upstream implementation.
DIRECT + INDEPENDENT
Path weights and transmission coefficient
path_weights
Equal-weight fixture and explicit 8 reactive / 10 crossings = 0.8 calculation in test_path_weights.py.
DIRECT
Interface placement
interface_placement and interface_estimation
Exact geometric placement on p(x)=0.5^x, plus hand-counted binless crossing curves.
DIRECT
Permeability
path_analysis.perm_calculations
Hand-counted \xi and \tau, explicit permeability product, and quadrature error propagation in test_path_analysis.py.
DIRECT
Energy/order descriptive reports
energy_analysis and order_analysis
Frozen fixture outputs, independently tested common statistics, and the shape-one kinetic gamma law checked against 2\exp(-2E) in test_energy_analysis.py.
DIRECT
Cross-engine rate equivalence
examples/validation/engines/water_dimer
Shared-force-field energy gate, direct-MD timescale, and three-engine rate comparison documented in engine-validation.
INDEPENDENT CAMPAIGN
## Scope boundary
The matrix covers numerical results exposed by the analysis reports and interface tools. Plot layout, file formatting, and training-frame selection have deterministic functional tests, but are not themselves scientific estimators and are outside this oracle inventory.
---
# Trustworthiness workplan
## Purpose
A green test is evidence only when the test demonstrably inspected the quantity it claims to validate. This workplan turns that rule into executable gates. It complements method-validation, engine-validation, and the adversarial comparison audit in docs/test_integrity_audit.md.
## Status terms
Implemented and exercised by an automated test or a reproducible command.
Work is being implemented in the current change set.
The code-side preparation is complete, but reference-grade software, hardware, or a deliberate reference re-baseline is still required.
Required work that has not started.
## Definition of trustworthy
A release is trustworthy only when all of the following are true:
- required unit, integration, and external-engine jobs pass – these are the routine correctness gate, and they are what establishes that a change still behaves correctly;
- the recorded method-validation campaign still applies: it is re-run only when a change can move the sampled distribution (a move, an acceptance rule, the RNG or its draw order, the scheduler, a rate estimator, or a deliberate re-blessing), not per release. Byte-identical engine goldens are the evidence that it has not moved. See method-validation; the CI method_validation_smoke job is a launch check, not evidence;
- no comparison passes after inspecting zero rows, zero paths, zero swaps, an empty archive, or an unexpected all-NaN physical quantity;
- every numerical reference records its generating engine, version, execution mode, command, tolerance rationale, and content digest;
- each central analysis method has an analytical, hand-computed, or independently implemented oracle;
- deterministic and stochastic workflows have separate contracts;
- restart, load, interruption, and parallel shutdown behavior are tested as distinct semantics;
- a file that cannot be read or written stops the run, instead of being absorbed into an empty read or a truncated output that still reports success;
- reading a file gives the same result on every machine, independent of the locale;
- running the tests does not modify the working tree, so no test can quietly rewrite a committed reference or input;
- the release pipeline cannot publish when a required validation was skipped or downgraded to execution-smoke coverage. This covers the per-change gate – the unit, integration and external-engine jobs. The method-validation campaign is deliberately outside it: it needs ~36 h against a 1 h GitLab job timeout and is a rare, change-triggered check rather than a per-release one, so the pipeline neither runs nor verifies it. Judging when it must be re-run is a human call; the criteria are in method-validation.
## Workstreams
### W0. Integrate concurrent review work
Review the concurrent agent’s path-validation, shooting-weight, documentation, and changelog changes without reverting or duplicating them. Focused path-analysis and replica-exchange tests pass.
### W1. Honest engine-suite outcomes
Engine absence must return the conventional skip status (77), and the orchestrator must report VALIDATED, SMOKE, SKIPPED, and FAILED separately.
Add PYRETIS_REQUIRE_VALIDATION=1. In this mode, a skipped or smoke-only required engine is a failing release gate rather than a green result.
Exercise both permissive developer mode and strict release mode with automated driver-contract tests.
### W2. Reference integrity and physical coverage
Add a machine-readable provenance manifest for external-engine references, including deterministic tree digests and explicit tolerance rationale.
Require suites to declare energy terms that are intentionally unavailable. Unexpected all-NaN columns must fail, while declared unavailable columns must be checked to remain all-NaN on both sides.
Report finite OpenMM potential energy from the OpenMM context and regenerate its golden data twice with the pinned OpenMM Reference platform. The two runs were byte-identical, and the release-grade suite now requires the generated files to remain byte-identical.
Establish a run-to-run deterministic CP2K reference build: the conda-forge cp2k 2026.1 package (single rank, OMP_NUM_THREADS=1) reproduces the test-retis-load run byte-identically across clean runs (verified 2026-07-19); the golden was re-blessed on it and the suite validates only on that exact version/revision.
Re-baseline the legacy GROMACS 5.1.4 tarballs onto the single current reference build. The whole GROMACS reference suite now targets one era (double-precision 2025.4 on the development reference box, re-blessed 2026-07-19 from a determinism-checked double run), so strict full-scope validation is satisfiable by one executable.
Keep the test run out of the committed fixtures. Unit tests built engines against tracked input directories, so a run wrote pyretis.mdp and its logs into directories holding committed references, and the example suites left engine scratch in the shared input directory. Tests now work on copies under tmp_path (the example suites keep running in place and sweep with pyretis tools clean, since their goldens sit alongside). A full ./test-easy.sh now leaves the working tree byte-for-byte unchanged, verified by snapshotting every file before and after, so no test can quietly rewrite a reference or an input.
Calibrate each external-engine tolerance from repeated runs on its pinned reference environment. Two strict release-gate runs on 2026-07-27 (PYRETIS_REQUIRE_VALIDATION=1 PYRETIS_GROMACS_SCOPE=full, single rank, OMP_NUM_THREADS=1) both reported six-of-six VALIDATED, and every external-engine golden was reproduced with zero observed deviation across those runs and the 2026-07-19 blessing: the byte-exact GROMACS, LAMMPS, and OpenMM gates and the PyVisA content hash pass only on byte or hash identity, and the pinned CP2K build reproduces its golden byte-identically. The operative tolerance is therefore byte identity (measured envelope 0.0); the retained numeric diagnostics (CP2K 1e-4, OpenMM 1e-5) sit orders of magnitude above the measured envelope, and the provenance manifest records the measurement per set.
### W3. Independent scientific oracles
REPPTIS cutoff occupancy is checked from raw columns with hand-audited counts and two independent probability constructions.
WHAM has hand-built synthetic exact-value tests, including weighted cases.
The method-validation campaign compares rate estimates with the analytical Kramers/Mel’nikov-Meshkov result.
Keep an evidence matrix that maps each public analysis result to at least one independent oracle and names any remaining unsupported result.
### W4. Stochastic and parallel contracts
Single-worker seeded workflows retain byte-reproducibility checks; multi-worker workflows are explicitly completion-ordered and are not falsely required to be byte-identical.
A swap-consistency check that examines no swaps must report zero coverage and fail when the caller explicitly requested swap validation.
Promote the multi-seed method-validation campaign to a scheduled CI job with confidence intervals and stored provenance. The campaign needs ~36 h against a 1 h GitLab job timeout, so it runs off CI via examples/validation/campaign_run.sh and the pipeline carries only a ten-cycle launch check. Stored provenance (host, commit, dirty flag) is in place; the scheduled job and the cross-seed confidence interval are not.
### W5. Failure and recovery behavior
Async worker exception recovery, burst submission, forced cancellation, repeated lifecycle cleanup, SIGTERM unwind, and atomic continuation writes have automated coverage.
Add corruption tests for the reference manifest and explicit failures for missing or changed reference artifacts.
Add controlled disk-write failure injection around run-state persistence and verify that the last durable state remains restartable.
Surface file-I/O failures instead of absorbing them. FileIO used to log a failed open and keep a None handle, which iterates as end-of-file, so an unreadable or missing file was indistinguishable from an empty one; a failed write was logged and reported through a status value no caller reads, leaving a truncated energy.txt or order.txt behind a run that reported success. Both now raise. The path-ensemble reader used by pyretis analyse opened the file itself and swallowed a read error into an empty path set, so an unreadable or mid-read-truncated pathensemble.txt silently produced a wrong crossing probability; it now propagates the error too. Callers for which a missing file is a legitimate state check for it first and say so: a path loaded from sparse or external files carries no energies, and that is reported per path while the simulation continues.
Pin text decoding to UTF-8. The encoding was selected by an inverted test, so every text file was decoded with the platform locale, making a read depend on the machine it ran on. The on-the-fly trajectory reader shared by the streaming LAMMPS and CP2K engines and the LAMMPS dump writer also opened text without an explicit encoding; both now pin UTF-8, so no text open decodes or encodes through the locale. The remaining swallowed I/O error is the directory fsync in inout/common.py, which weakens durability without corrupting data and is documented as such.
### W6. CI and release policy
Run reference-manifest validation and strict Sphinx warnings-as-errors in normal CI.
Make tagged releases depend on the strict validation jobs. Manual developer smoke jobs remain useful, but cannot satisfy a release dependency. The external-engine jobs honour this: on a release tag they switch to allow_failure: false with PYRETIS_REQUIRE_VALIDATION=1. method_validation_smoke is the one job in do_release’s needs that is a launch check rather than a validation job; it is listed to catch a broken driver at release time and is explicitly not counted as method-validation evidence (see the known exception above).
Coverage is collected on CI Python 3.12, separately from the Python 3.14/NumPy combination whose tracer is currently incompatible.
## Execution gates
The intended commands, from least to most expensive, are:
```bash
python -m pyretis.testing.reference_manifest \
examples/tests/reference_provenance.json
sphinx-build -W --keep-going -E -b html docs /tmp/pyretis-docs
./test-easy.sh
./test-heavy.sh
PYRETIS_REQUIRE_VALIDATION=1 ./test-heavy.sh
```
In permissive mode, missing or non-reference engines are reported as SKIPPED or SMOKE and do not by themselves fail the driver. The final command is the release gate and may not pass until every required engine is installed and reference-grade.
## Current blockers and decision points
- The CP2K determinism blocker is resolved (conda-forge 2026.1, single rank, deterministic; golden re-blessed 2026-07-19). Non-reference CP2K builds still run execution-smoke only.
- GROMACS references are unified on a single era (double-precision 2025.4, the development reference box, re-blessed 2026-07-19): the former 5.1.4 archive suite was regenerated on the chosen reference version.
- External-engine tolerances are now measured, not provisional: repeated strict release-gate runs on the pinned reference environment (the 2026-07-19 blessing and two runs on 2026-07-27) reproduced every golden with zero observed deviation, so byte or hash identity is the operative tolerance and the retained numeric diagnostics sit far above the measured 0.0 envelope.
- Reference updates are review events. A digest update must include the engine provenance, generation command, observed error envelope, and reviewer signoff in the merge request.
---
# Example organization
The examples tree has exactly two top-level subfolders: examples/tutorials and examples/tests. Tutorials are for users who want to learn how to set up simulations. Tests are for developers who want to check that generated output still matches reference data. Those goals are related, but the folder layouts inside each branch are allowed to differ.
Area
Purpose
Rule of thumb
tutorials/path_sampling/
TIS, RETIS, REPPTIS, and engine-coupling tutorials.
Each directory should have a short README.rst, one primary input file, and a matching page under docs/examples/.
tutorials/molecular_dynamics/
MD and flux tutorials used to prepare path sampling.
Keep these small and make clear whether they are library-based scripts or pyretis run input-file examples.
tutorials/umbrella_sampling/
Sampling tutorials outside the path-sampling path.
Keep the current topic grouping, but add a README when a folder contains more than one runnable script.
tutorials/extending/
Advanced tutorials for custom engines, force fields, and order parameters.
Treat these as advanced examples and link them from both the examples pages and the developer guide where relevant.
tutorials/from_tests/
User-facing recipes derived from examples/tests fixtures.
Explain how to adapt a tested setup to a new simulation, without requiring users to reproduce reference output.
tutorials/other/
Incubator for examples that do not yet fit a stable topic.
New examples should not stay here once they are documented.
test/
Heavy consistency checks driven by test-heavy.sh.
This folder is special: optimize it for reproducible comparison against reference output, not for narrative readability.
## How tutorials connect to tests
A tutorial does not need to duplicate the exact heavy-test directory. Instead, each documented tutorial should have one of these links to the heavy suite:
- Direct check: the documentation page points to a heavy-test case that runs the same input file or an intentionally shortened version.
- Representative check: the page points to a heavy-test case that exercises the same engine, method, and analysis path.
- Manual / optional check: the page explains which external program or GUI dependency is required and why the example is not part of the default smoke scope.
This avoids turning tutorial folders into brittle CI fixtures while still making every tutorial accountable.
## Tutorial requirements
Every tutorial should contain enough information for a user to adapt the setup:
- State the purpose of the simulation and the method/engine used.
- Identify the files the user is expected to edit first.
- Give the start command and the main files/directories created.
- Explain what a short startup run should look like.
- Add a small Tested by section to each example page. Link to the relevant heavy-test directory or explain the optional dependency.
- Use the heading Tutorial quick start for the command-and-file checklist. This makes the page easy to scan for users and for LLM assistants using the documentation as context.
- Link unusual input keywords to the relevant user-guide section, and link the page to examples-llm-quickstart when adding new setup patterns.
- Move tutorials out of tutorials/other/ once they have a stable docs page.
- Keep generated outputs out of tutorial directories unless they are intentionally used as reference material.
- Add new tutorials to test-heavy.sh by creating a shortened heavy-test fixture under examples/tests/.
## Adding a new example
For a new user-facing tutorial, include:
- a short README.rst with the purpose, requirements, run command, and expected output files;
- a documentation page under docs/examples/ that explains the input choices and links to the runnable files;
- a heavy-test fixture under examples/tests/ or a note explaining why the example is optional/manual;
- a line in example-test-status so the website reports how the example is checked.
This gives contributors a simple contract: if an example is visible in the website, the website also says how it is verified.
## Running an example folder
Each examples/tests/ case ships a run.sh that is both the runnable example and the regression check. It takes an optional flag so the two uses do not get in each other’s way:
Command
What it does
./run.sh
Run the simulation and analysis and leave the results in place so you can inspect them. This is the normal way to try an example.
./run.sh -C
Also compare the output against the committed results/ reference (via pyretis.testing.example_compare).
./run.sh -c
Also clean the generated output afterwards.
./run.sh -cc
Compare and clean – what test-heavy.sh and the per-suite runners use, so the checks run and the tree is left clean.
So a user just runs ./run.sh and keeps the output; the test runners call ./run.sh -cc. The comparison logic is shared in pyretis.testing.example_compare (which selects its checks from the config’s task) rather than duplicated in a per-folder compare.py.
---
# Example test status
This page is the canonical map between tutorial folders, documentation pages, and heavy-test fixtures. Every runnable tutorial should either have a passing check here or a clear reason why it is treated as an optional engine-dependent check. The same information is referenced (but not duplicated) by examples-llm-quickstart and the examples/tutorials/README.rst README, so updating this page keeps the other two in sync.
## Last refreshed
The most recent automated refresh stamp is written by examples/run-tutorials.sh (when invoked with PYRETIS_WRITE_REFRESH_STAMP=1) and is embedded below:
```text
Last refresh: 2026-05-17 00:00 UTC
Tutorials passed: 14 / 14 attempted
Tutorials failed: 0
Tutorials skipped (optional): engine-dependent cases when binaries missing
Runner: examples/run-tutorials.sh
Hard timeout per case: 10s
PyRETIS version: 4.0.0.dev0+4fa4079, Python 3.12.13
```
The internal-engine suite was run on the date above from a temporary copy of the repository so that generated files in the working tree were not modified. The status pills on the individual tutorial pages (“passing”, “engine”, “smoke”, “manual”) reflect the same run.
The repository-level heavy entry point is:
```bash
./test-heavy.sh
```
It is a thin wrapper around examples/run-tests.sh and now also runs the tutorial smoke checks (examples/run-tutorials.sh) at the end. To run the tutorial smoke checks on their own:
```bash
examples/run-tutorials.sh
```
To skip the tutorial smoke checks inside the heavy suite:
```bash
PYRETIS_RUN_TESTS_SKIP_TUTORIALS=1 ./test-heavy.sh
```
This copies tutorial inputs to /tmp, starts each documented command, and intentionally stops long-running tutorials after a timeout. A timeout without errors counts as a successful startup; early crashes, tracebacks, or PyRETIS ERROR/CRITICAL log messages are reported as failures. Optional external-engine and FORTRAN cases are skipped when the required commands are not available unless PYRETIS_TUTORIAL_STRICT=1 is set.
It runs the internal, GROMACS, PyVisA, CP2K, OpenMM, and LAMMPS example-test suites. The GROMACS part defaults to a smoke scope; set PYRETIS_GROMACS_SCOPE=full to run the full GROMACS matrix.
## Tutorial map
The table below is the canonical mapping between tutorial folders under examples/tutorials/, their rendered documentation pages, and the heavy-test fixtures that exercise the same workflow. Tutorial READMEs and the LLM quick start refer to this table rather than repeating it.
The Status column uses four labels:
- passing – covered by a heavy fixture that was passing on the last refresh.
- engine – covered by a heavy fixture but the suite requires an external program (gmx, cp2k, lmp, openmm) that may not be available locally.
- smoke – not yet covered by a dedicated fixture; only the tutorial smoke runner checks that the command starts cleanly.
- manual – requires a GUI, interactive session, or other manual step.
Tutorial folder
Docs page
Heavy-test fixture
Status
path_sampling/internal/1D-double-well/tis
examples-tis-1d
test-internal/tis-multiple, tis-restart
passing
path_sampling/internal/1D-double-well/retis
examples-retis-1d
test-internal/retis, retis-restart, retis-restart-cv, retis-load-sparse
passing
path_sampling/internal/1D-double-well/submoves
examples-submoves-1d
test-internal/retis-ss-wt-wf
passing
path_sampling/internal/1D-double-well/repptis
examples-repptis
test-internal/partial-path, test-gromacs/test-repptis
smoke
path_sampling/internal/1D-double-well/flux
examples-tis-1d-flux
test-internal/mdflux-restart
passing
path_sampling/internal/1D-double-well/tis-multiple
examples-tis-1d
test-internal/tis-multiple
passing
path_sampling/internal/1D-triple-well
examples-retis-triple-well
test-internal/retis-zero-left
passing
path_sampling/internal/2D-hysteresis
examples-retis-2d
test-internal/explorer (representative)
smoke
path_sampling/internal/2D-wca
examples-retis-wca
test-lammps/test-retis (representative)
smoke
path_sampling/lammps/2D-wca
examples-retis-lammps
test-lammps (including test-load-sparse)
engine
path_sampling/lammps/H2/infinite_swapping
examples-infswap
tutorial run (lammps-h2-infinite-swapping)
smoke
path_sampling/gromacs/hydrate
examples-gromacs-hydrate
test-gromacs, test-pyvisa
engine
path_sampling/cp2k/H2/retis
examples-cp2k-hydrogen
test-cp2k
engine
path_sampling/openmm/argon
examples-openmm
test-openmm/test-retis, test-openmm/test-load-sparse
engine
path_sampling/internal/permeability
examples-permeability
tutorial smoke (permeability-retis)
smoke
molecular_dynamics/md_nve/from_settings
examples-md-nve
test-internal/md-restart
passing
molecular_dynamics/md_nve
examples-md-nve-lib
test-internal/md-restart
passing
molecular_dynamics/md_nve/forward_backward
examples-for-back-md
test-internal/compare-internal-with-lammps
passing
umbrella_sampling
examples-mc-umbrella-sampling
(none)
smoke
extending/order-parameter
examples-2d-hysteresis (representative)
test-internal (representative)
smoke
extending/lennard-jones
examples-vvexternal
test-internal/compare-internal-with-lammps
smoke
other/particle_swarm
examples-pso
(none)
smoke
### Tutorials without a heavy check
The following tutorials are currently flagged smoke only. Before a release relies on their numerical output, add a shortened fixture under examples/tests/ and link it back to the corresponding docs page:
- path_sampling/internal/2D-hysteresis
- path_sampling/internal/2D-wca
- path_sampling/internal/permeability
- path_sampling/lammps/H2/infinite_swapping
- umbrella_sampling
- extending/order-parameter
- extending/lennard-jones
- other/particle_swarm
A “Verification status” line at the top of each docs page (see examples-*.rst) mirrors this column so the LLM rule “do not claim an example is fully verified unless it is listed as passing” has a single source of truth.
Suite
Status
Command
Notes
Internal engine examples
Passed 2026-05-17
./test-heavy.sh or examples/tests/test-internal/run-all-internal.sh
Covers pure-Python MD, flux, TIS/RETIS, restart, sparse load, partial paths, explorer, and sub-move style workflows.
LAMMPS examples
Engine available locally
./test-heavy.sh or examples/tests/test-lammps/run-all-lammps.sh
Requires lmp on PATH and a compatible LAMMPS build.
GROMACS examples
Engine available locally
./test-heavy.sh or examples/tests/test-gromacs/run-all-gromacs.sh
Requires gmx and the bundled reference input files.
CP2K examples
Engine available locally
./test-heavy.sh or examples/tests/test-cp2k/run-all-cp2k.sh
Requires cp2k or cp2k.psmp.
OpenMM examples
Validated 2026-07-15
./test-heavy.sh or examples/tests/test-openmm/run-all-openmm.sh
OpenMM 8.5.1 source revision f7fa0c27 on the Reference platform; two clean runs were byte-identical and finite potential energy is checked.
PyVisA analysis examples
Optional GUI/analysis check
./test-heavy.sh or examples/tests/test-pyvisa/run-all-pyvisa.sh
Exercises the analysis helpers used by the PyVisA documentation.
## What the internal suite covers
The internal suite is the main quick check because it does not depend on external molecular-dynamics programs. It should be run before every documentation release and after changes to examples, input parsing, path ensembles, internal engines, or analysis output.
Test directory
Related documentation
What it validates
compare-internal-with-lammps
examples-md
Internal Lennard-Jones MD agrees with saved LAMMPS reference thermodynamic output.
explorer
examples-explore
Loading frames for free-energy landscape exploration, and the structural validity of the explore run: the ensemble directory layout, the path length and order-parameter rules, order.txt in lockstep with pathensemble.txt, and committed (non-loaded) moves.
partial-path
examples-retis-1d
REPPTIS-style partial paths, analysis report generation, and the structural validity of the REPPTIS run.
mdflux-restart
examples-tis-1d-flux
MD-flux runs and restart equivalence.
md-restart
examples-md
NVE restart equivalence for Langevin and Velocity Verlet engines.
retis
examples-retis-1d
A small RETIS run, analysis, and comparison against saved reference data.
retis-load-sparse
examples-retis-1d
Loading sparse paths from trajectories and from individual frames.
retis-restart
examples-retis-1d
RETIS restart behavior and output reproducibility.
retis-restart-cv
initpath-restart-cv
Adding a collective variable to a running simulation: the new column is recorded from the restart on, and the sampling is unchanged relative to a restart without it.
retis-ss-wt-wf
examples-submoves-1d
Stone skipping, web throwing, and wire fencing moves.
tis-multiple
examples-tis-1d
Multiple TIS ensembles and analysis output.
tis-restart
examples-tis-1d
TIS restart behavior and output reproducibility.
## How to refresh this page
Run the quick suite first:
```bash
cd examples/tests/test-internal
./run-all-internal.sh
```
Then run the optional engine suites in an environment where the corresponding programs are installed:
```bash
cd examples/tests/test-lammps
./run-all-lammps.sh
cd ../test-gromacs
./run-all-gromacs.sh
cd ../test-cp2k
./run-all-cp2k.sh
cd ../test-openmm
./run-all-openmm.sh
cd ../test-pyvisa
./run-all-pyvisa.sh
```
When a suite passes, update the status table with the date, platform, PyRETIS version, and any engine version that matters for reproducing the run.
## Documentation build check
The examples pages use literalinclude directives to render input files. To avoid drift between the rendered docs and the runnable tutorials, run the sync helper before building:
```bash
python docs/sync_example_assets.py # report drift only
python docs/sync_example_assets.py --fix # update _static
cd docs && make html
```
The helper copies a small manifest of input files from examples/tutorials/ into docs/_static/examples/. Add new entries to the MANIFEST list when a new tutorial page gains a literalinclude directive.
A missing example asset is usually caught by the Sphinx build before it becomes a broken website page; treat warnings as actionable.
---
# RETIS in a 2D potential
In this example, you will explore a rare event with the Replica Exchange Transition Interface Sampling (RETIS) algorithm.
In this example, we will implement a new potential for PyRETIS and we will create three new order parameters which we will use to investigate a transition between two stable states in the potential. Using the RETIS algorithm, we will compute the rate constant for the transition between the two stable states.
This example is structured as follows:
- Tutorial quick start
- Creating the potential function
- Creating the order parameters
- Creating the x (or y) positional order parameter
- Creating the projection-of-positions order parameter
- Running the RETIS simulation with PyRETIS
- Running the RETIS simulation using x as the order parameter
- Running the RETIS simulation using y as the order parameter
- Running the RETIS simulation using the projection-of-position as the order parameter
- Comparing results from the analysis
- Tested by
Verification status: smoke – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/2D-hysteresis/.
- Edit first: potential.py for the 2D potential, order.py or orderp.py for the order parameter, and one of retis-x/retis.toml, retis-y/retis.toml, or retis-xy/retis.toml.
- Run: pyretis run -i retis.toml -p inside the chosen variant directory.
- Analyse: pyretis analyse -i retis.toml – the analysis reads the same input file as the run.
- Expected output: one RETIS output tree for each order-parameter choice you run, plus analysis reports for comparing rates.
- Related checks: examples/tests/test-internal/explorer/ and examples/tests/test-internal/partial-path/ are representative internal-engine checks; see example-test-status.
## Creating the potential function
We will consider a relatively simple 2D potential where a single particle is moving. The potential energy (V_\text{pot}) is given by
where x and y gives the positions. The potential energy is shown in fig2dpot_ex_retis below.
We begin by creating a new potential function to use with PyRETIS, see the user guide for a short introduction on how custom potential functions can be created. In short, we will now have to define a new class which calculates the potential energy and the corresponding force.
We begin by simply creating a new class for the new potential function. Here, we define the parameters our new potential should accept. We implement the potential slightly more general than the potential given above so that we can change the parameters more easily, if we wish to do so, later. The potential we will be implementing is given by:
where \gamma_1, \gamma_2, \gamma_3, \alpha_1, \alpha_2, \beta_1, \beta_2, x_0 and y_0 are potential parameters.
Next, we define a new method as part of our new potential class which will be responsible for calculating the potential energy and a method responsible for calculating the force. Finally, we add a method that will calculate both the potential and the force at the same time. The full Python code for the new potential class can be found below.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""This is a 2D example potential."""
import logging
import numpy as np
from pyretis.forcefield.potential import PotentialFunction
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
class Hyst2D(PotentialFunction):
r"""Hyst2D(PotentialFunction).
This class defines a 2D dimensional potential with two
stable states.
The potential energy (:math:`V_\text{pot}`) is given by
.. math::
V_\text{pot}(x, y) = \gamma_1 (x^2 + y^2)^2 +
\gamma_2 \exp(\alpha_1 (x - x_0)^2 + \alpha_2 (y - y_0)^2) +
\gamma_3 \exp(\beta_1 (x + x_0)^2 + \beta_2(y + y_0)^2)
where :math:`x` and :math:`y` gives the positions and :math:`\gamma_1`,
:math:`\gamma_2`, :math:`\gamma_3`, :math:`\alpha_1`, :math:`\alpha_2`,
:math:`\beta_1`, :math:`\beta_2`, :math:`x_0` and :math:`y_0` are
potential parameters.
Attributes
----------
params : dict
Contains the parameters. The keys are:
* `gamma1`: The :math:`\gamma_1` parameter for the potential.
* `gamma2`: The :math:`\gamma_2` parameter for the potential.
* `gamma3`: The :math:`\gamma_3` parameter for the potential.
* `alpha1`: The :math:`\alpha_1` parameter for the potential.
* `alpha2`: The :math:`\alpha_2` parameter for the potential.
* `beta1`: The :math:`\beta_1` parameter for the potential.
* `beta2`: The :math:`\beta_2` parameter for the potential.
* `x0`: The :math:`x_0` parameter for the potential.
* `y0`: The :math:`y_0` parameter for the potential.
"""
def __init__(self, desc='2D hysteresis'):
"""Set up the potential.
Parameters
----------
a : float, optional
Parameter for the potential.
b : float, optional
Parameter for the potential.
c : float, optional
Parameter for the potential.
desc : string, optional
Description of the force field.
"""
super().__init__(dim=2, desc=desc)
self.params = {'gamma1': 0.0, 'gamma2': 0.0, 'gamma3': 0.0,
'alpha1': 0.0, 'alpha2': 0.0, 'beta1': 0.0,
'beta2': 0.0, 'x0': 0.0, 'y0': 0.0}
def potential(self, system):
"""Evaluate the potential.
Parameters
----------
system : object like `System`
The system we evaluate the potential for. Here, we
make use of the positions only.
Returns
-------
out : float
The potential energy.
"""
x = system.pos[:, 0] # pylint: disable=invalid-name
y = system.pos[:, 1] # pylint: disable=invalid-name
gam1 = self.params['gamma1']
gam2 = self.params['gamma2']
gam3 = self.params['gamma3']
alf1 = self.params['alpha1']
alf2 = self.params['alpha2']
bet1 = self.params['beta1']
bet2 = self.params['beta2']
x_0 = self.params['x0']
y_0 = self.params['y0']
v_pot = (gam1 * (x**2 + y**2)**2 +
gam2 * np.exp(alf1 * (x - x_0)**2 + alf2 * (y - y_0)**2) +
gam3 * np.exp(bet1 * (x + x_0)**2 + bet2 * (y + y_0)**2))
return v_pot.sum()
def force(self, system):
"""Evaluate forces.
Parameters
----------
system : object like `System`
The system we evaluate the potential for. Here, we
make use of the positions only.
Returns
-------
out[0] : numpy.array
The calculated force.
out[1] : numpy.array
The virial, currently not implemented for this potential.
"""
x = system.pos[:, 0] # pylint: disable=invalid-name
y = system.pos[:, 1] # pylint: disable=invalid-name
gam1 = self.params['gamma1']
gam2 = self.params['gamma2']
gam3 = self.params['gamma3']
alf1 = self.params['alpha1']
alf2 = self.params['alpha2']
bet1 = self.params['beta1']
bet2 = self.params['beta2']
x_0 = self.params['x0']
y_0 = self.params['y0']
term = 4.0 * gam1 * (x**2 + y**2)
exp1 = gam2 * np.exp(alf1 * (x - x_0)**2 + alf2 * (y - y_0)**2)
exp2 = gam3 * np.exp(bet1 * (x + x_0)**2 + bet2 * (y + y_0)**2)
forces = np.zeros_like(system.pos)
forces[:, 0] = -(x * term +
2.0 * alf1 * (x - x_0) * exp1 +
2.0 * bet1 * (x + x_0) * exp2)
forces[:, 1] = -(y * term +
2.0 * alf2 * (y - y_0) * exp1 +
2.0 * bet2 * (y + y_0) * exp2)
virial = np.zeros((self.dim, self.dim)) # just return zeros here
return forces, virial
def potential_and_force(self, system):
"""Evaluate the potential and the force.
Parameters
----------
system : object like `System`
The system we evaluate the potential for. Here, we
make use of the positions only.
Returns
-------
out[0] : float
The potential energy as a float.
out[1] : numpy.array
The force as a numpy.array of the same shape as the
positions in `particles.pos`.
out[2] : numpy.array
The virial, currently not implemented for this potential.
"""
virial = np.zeros((self.dim, self.dim)) # just return zeros here
x = system.pos[:, 0] # pylint: disable=invalid-name
y = system.pos[:, 1] # pylint: disable=invalid-name
gam1 = self.params['gamma1']
gam2 = self.params['gamma2']
gam3 = self.params['gamma3']
alf1 = self.params['alpha1']
alf2 = self.params['alpha2']
bet1 = self.params['beta1']
bet2 = self.params['beta2']
x_0 = self.params['x0']
y_0 = self.params['y0']
term0 = (x**2 + y**2)
exp1 = gam2 * np.exp(alf1 * (x - x_0)**2 + alf2 * (y - y_0)**2)
exp2 = gam3 * np.exp(bet1 * (x + x_0)**2 + bet2 * (y + y_0)**2)
v_pot = gam1 * term0**2 + exp1 + exp2
term = 4.0 * gam1 * term0
forces = np.zeros_like(system.pos)
forces[:, 0] = -(x * term +
2.0 * alf1 * (x - x_0) * exp1 +
2.0 * bet1 * (x + x_0) * exp2)
forces[:, 1] = -(y * term +
2.0 * alf2 * (y - y_0) * exp1 +
2.0 * bet2 * (y + y_0) * exp2)
return v_pot.sum(), forces, virial
```
This new potential function can be included in a PyRETIS simulation by referencing it in the input file as follows (assuming that it has been stored in a file named potential.py):
```rst
Forcefield settings
-------------------
description = 2D hysteresis
Potential
---------
class = Hyst2D
module = potential.py
parameter gamma1 = 1
parameter gamma2 = -10
parameter gamma3 = -10
parameter alpha1 = -30
parameter alpha2 = -3
parameter beta1 = -30
parameter beta2 = -3
parameter x0 = 0.2
parameter y0 = 0.4
```
## Creating the order parameters
In this case, it is not so obvious how to define the order parameter. Three simple possibilities are (see fig2dpot_order for an illustration):
- The x coordinate of the particle, where we include the potential energy in order to define the two stable states.
- The y coordinate of the particle, where we include the potential energy in order to define the two stable states.
- The projection of the distance vector from the position of the particle (x, y) to (-0.2, -0.4) onto the straight line between (-0.2, -0.4) and (0.2, 0.4). Here, we also include the potential energy in order to define the two stable states.
Since we include the potential energy in the order parameter definition, we will have to create customized order parameters. There is a short introduction to how the order parameters are implemented in PyRETIS in the user guide and there is also a more lengthy recipe for creating custom order parameters available.
Here, we will first consider the x (or y) order parameter and then the more involved combination.
### Creating the x (or y) positional order parameter
The order parameter we are now creating will just be the position of the particle with two extra conditions:
- If the position is below a certain value (inter_a) close to the first interface and if the potential energy is above a certain threshold value (energy_a), then the order parameter will not be allowed to come closer to the first interface.
- If the position is closer to the last interface than a certain value (inter_b) and if the potential energy is above a certain threshold value (energy_b), then the order parameter will not be allowed to come closer to the last interface.
Thus, in addition to the position of the particle, we need to consider the 4 additional parameters inter_a, inter_b, energy_a, energy_b. The Python code for this new order parameter is given below:
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""The order parameter for the hysteresis example."""
import logging
from pyretis.orderparameter.orderparameter import OrderParameter
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
class OrderX(OrderParameter):
"""A positional order parameter.
Order parameter for the hysteresis example. In addition to using
the position, we also use the energy to tell if we are in states A/B.
Attributes
----------
index : integer
This is the index of the atom which will be used, i.e.
system.pos[index] will be used.
inter_a : float
An interface such that we are in state A for postions < inter_a.
inter_b : float
An interface such that we are in state B for postions > inter_b.
energy_a : float
An energy such that we are in state A for potential energy < energy_a.
energy_b : float
An energy such that we are in state A for potential energy < energy_b.
dim : integer
This is the dimension of the coordinate to use.
0, 1 or 2 for 'x', 'y' or 'z'.
periodic : boolean
This determines if periodic boundaries should be applied to
the position or not.
"""
def __init__(self, index, inter_a, inter_b, energy_a, energy_b,
dim='x', periodic=False):
"""Initialise the order parameter.
Parameters
----------
index : int
This is the index of the atom we will use the position of.
inter_a : float
An interface such that we are in state A for postions < inter_a.
inter_b : float
An interface such that we are in state B for postions > inter_b.
energy_a : float
An energy such that we are in state A for
potential energy < energy_a.
energy_b : float
An energy such that we are in state A for
potential energy < energy_b.
dim : string
This select what dimension we should consider,
it should equal 'x', 'y' or 'z'.
periodic : boolean, optional
This determines if periodic boundary conditions should be
applied to the position.
"""
txt = 'Position of particle {} (dim: {})'.format(index, dim)
super().__init__(description=txt)
self.inter_a = inter_a
self.inter_b = inter_b
self.energy_a = energy_a
self.energy_b = energy_b
self.periodic = periodic
self.index = index
dims = {'x': 0, 'y': 1, 'z': 2}
try:
self.dim = dims[dim]
except KeyError:
msg = 'Unknown dimension {} requested'.format(dim)
logger.critical(msg)
raise
def calculate(self, system):
"""Calculate the order parameter.
Parameters
----------
system : object like :py:class:`.System`
This object is used for the actual calculation, typically
only `system.pos` and/or `system.vel`
will be used. In some cases `system.forcefield` can also be
used to include specific energies for the order parameter.
Returns
-------
out : float
The order parameter.
"""
particles = system.particles
pos = particles.pos[self.index]
lmb = pos[self.dim]
if lmb < self.inter_a:
if particles.vpot > self.energy_a:
lmb = self.inter_a
elif lmb > self.inter_b:
if particles.vpot > self.energy_b:
lmb = self.inter_b
if self.periodic:
return [system.box.pbc_coordinate_dim(lmb, self.dim)]
return [lmb]
```
This new order parameter can be included in a PyRETIS simulation by adding the following order parameter section (assuming that the order parameter is stored in a file order.py):
```rst
Orderparameter
--------------
class = OrderX
module = order.py
dim = x
index = 0
periodic = False
inter_a = -0.15
inter_b = 0.18
energy_a = -9.0
energy_b = -9.0
```
### Creating the projection-of-positions order parameter
For this order parameter, we will consider the distance from the particle to the stable A state and project the distance vector onto the line joining states A and B. In addition, as in the previously defined order parameters, we will include the energy in the definition of the states.
The Python code for this new order parameter is given below:
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""This file defines the order parameter used for the WCA example."""
import logging
import numpy as np
from pyretis.orderparameter import OrderParameter
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
class OrderXY(OrderParameter):
"""OrderXY(OrderParameter).
This class defines a 2D order parameter such that x + y = constant.
Attributes
----------
index : integer
This selects the particle to use for the order parameter.
inter_a : float
An interface such that we are in state A for postions < inter_a.
inter_b : float
An interface such that we are in state B for postions > inter_b.
energy_a : float
An energy such that we are in state A for potential energy < energy_a.
energy_b : float
An energy such that we are in state A for potential energy < energy_b.
"""
def __init__(self, index, inter_a, inter_b, energy_a, energy_b):
"""Set up the order parameter.
Parameters
----------
index : tuple of ints
This is the indices of the atom we will use the position of.
inter_a : float
An interface such that we are in state A for postions < inter_a.
inter_b : float
An interface such that we are in state B for postions > inter_b.
energy_a : float
An energy such that we are in state A for
potential energy < energy_a.
energy_b : float
An energy such that we are in state A for
potential energy < energy_b.
"""
super().__init__(description='2D->1D projection')
self.index = index
x_1 = 0.2
y_1 = 0.4
x_0 = -0.2
y_0 = -0.4
self.x_0 = x_0
self.y_0 = y_0
self.origin = np.array([x_0, y_0])
self.vec = np.array([x_1 - x_0, y_1 - y_0])
self.vec /= np.sqrt(np.dot(self.vec, self.vec))
self.inter_a = inter_a
self.inter_b = inter_b
self.energy_a = energy_a
self.energy_b = energy_b
def calculate(self, system):
"""Calculate the order parameter.
Here, the order parameter is just the distance between two
particles.
Parameters
----------
system : object like :py:class:`.System`
This object is used for the actual calculation, typically
only `system.pos` and/or `system.vel`
will be used. In some cases `system.forcefield` can also be
used to include specific energies for the order parameter.
Returns
-------
out : float
The order parameter.
"""
pos = system.pos[self.index]
vec = np.array([pos[0] - self.x_0, pos[1] - self.y_0])
proj = np.dot(vec, self.vec)
if proj < self.inter_a:
if system.particles.vpot > self.energy_a:
proj = self.inter_a
elif proj > self.inter_b:
if system.particles.vpot > self.energy_b:
proj = self.inter_b
return [proj]
```
Note that this order parameter is less general than the previous ones, for instance, we make explicit use of the location of the two minimums. If you are feeling adventurous, please try to add these locations as parameters for the order parameter.
This new order parameter can be included in a PyRETIS simulation by adding the following order parameter section (assuming that the order parameter is stored in a file order.py):
```rst
Orderparameter
--------------
class = OrderXY
module = orderp.py
index = 0
inter_a = 0.15
inter_b = 0.75
energy_a = -9.0
energy_b = -9.0
```
## Running the RETIS simulation with PyRETIS
We have now defined the potential and the order parameters we are going to use and we are ready to run the RETIS simulations.
### Running the RETIS simulation using x as the order parameter
In order to run the RETIS simulation, the following steps are suggested:
- Create a new folder, for instance 2D-hysteresis.
- Place the order parameter script in this folder and name it order.py.
- Download the initial configuration initial.xyz and place it in the same directory.
- Create a new sub-directory, retis-x.
- Add the following input script retis.toml to the retis-x folder.
- Run the simulation using (in the retis-x folder):
```pyretis
pyretis run -i retis.toml -p
```
Some examples from the analysis can be found below:
### Running the RETIS simulation using y as the order parameter
In order to run the RETIS simulation, the following steps are suggested:
- Create a new folder, for instance 2D-hysteresis.
- Place the order parameter script in this folder and name it order.py.
- Download the initial configuration initial.xyz and place it in the same directory.
- Create a new sub-directory, retis-y.
- Add the following input script retis.toml to the retis-y folder.
- Run the simulation using (in the retis-y folder):
```pyretis
pyretis run -i retis.toml -p
```
Some examples from the analysis can be found below:
### Running the RETIS simulation using the projection-of-position as the order parameter
In order to run the RETIS simulation, the following steps are suggested:
- Create a new folder, for instance 2D-hysteresis.
- Create a new sub-directory, retis-xy and move into this directory.
- Place the order parameter script in this folder and name it order.py.
- Download the initial configuration initial2.xyz and place it in the same directory and rename it to initial.xyz.
- Add the following input script retis.toml to the retis-xy folder.
- Run the simulation using (still in the retis-xy folder):
```pyretis
pyretis run -i retis.toml -p
```
Some examples from the analysis can be found below:
## Comparing results from the analysis
By using the three different order parameters, we obtain different estimates for the rate constant as summarized in the table below.
## Comparison of the rate constant for the 2D potential
Order parameter
Rate constant / 1e-7
Lower bound
Upper bound
x
7.3
8.3
y
7.7
8.6
xy
7.3
8.3
## Tested by
This tutorial is represented by internal-engine checks that exercise the same kind of custom potential/order-parameter path handling: examples/tests/test-internal/explorer/ and examples/tests/test-internal/partial-path/. Add a shortened 2D-hysteresis fixture under examples/tests/ before using these specific long examples as release regression checks.
---
# Dissociation of Hydrogen with CP2K
In this example, we are going to study dissociation of hydrogen using CP2K. The goal of this example is to show how CP2K can be interfaced with PyRETIS for a relatively simple system.
In this example, we will make use of CP2K 1 for running the dynamics. The example is structured as follows:
- Tutorial quick start
- Creating the PyRETIS input file
- Running the RETIS simulation with PyRETIS
- Tested by
- References
Verification status: engine – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/cp2k/H2/retis/.
- Requirements: a working CP2K executable and the CP2K data files GTH_BASIS_SETS and GTH_POTENTIALS copied into cp2k_input/; see user-section-engine-cp2k.
- Edit first: retis.toml for the [engine] section and cp2k_input/cp2k.inp for the CP2K-side dynamics settings.
- Run: pyretis run -i retis.toml -p.
- Analyse: pyretis analyse -i retis.toml – the analysis reads the same input file as the run.
- Expected output: standard RETIS ensemble folders plus CP2K files managed through the external engine interface.
- Related check: examples/tests/test-cp2k/; see example-test-status.
The order parameter we will consider in this example will just be given by the distance between the two hydrogen atoms.
## Creating the PyRETIS input file
The input file for PyRETIS follows the structure we have used in the previous examples and we will not go into details about all sections here. In the following, we will just describe the section for using the CP2K engine in more detail. The full input file for the RETIS simulation is given at the end of this section.
In order to make use of CP2K, we add the following engine section to the input file:
```toml
[engine]
class = "cp2k"
cp2k = "cp2k"
input_path = "cp2k_input"
timestep = 0.5
subcycles = 1
cp2k_format = "xyz"
extra_files = [
"GTH_BASIS_SETS",
"GTH_POTENTIALS",
]
```
Here, we make use of the following keywords:
- class = cp2k which specifies the engine we will use (here: cp2k).
- cp2k = cp2k which specifies the CP2K executable. On your system, this might be named differently.
- input_path = cp2k_input which specifies the directory where PyRETIS will look for the input files for CP2K. PyRETIS assumes that, at least, the following files are in this folder:
- initial.xyz which contain the initial configuration. This file can be downloaded here: initial.xyz.
- cp2k.inp which contains the settings for the CP2K simulation. This file can be downloaded here: cp2k.inp.
- timestep = 0.5 which is the time step to use in the CP2K simulations.
- subcycles = 1 which is the number of subcycles CP2K will complete before PyRETIS re-calculates the order parameter.
- extra_files = ['GTH_BASIS_SETS', 'GTH_POTENTIALS'] which contain extra files CP2K will need in order to execute. Here, we are using two files to define the basis set and the potential functions. These files are part of the CP2K distribution and can be found in the folder $CP2K_HOME/data/ (or /usr/share/cp2k). PyRETIS expects that all the files you list here are included in the cp2k_input directory.
This specifies and selects the CP2K engine for use with PyRETIS. The full input file for the RETIS simulation is given below:
```toml
[simulation]
task = "retis"
steps = 100
interfaces = [
0.78,
0.8,
0.82,
0.84,
0.86,
0.87,
0.88,
0.9,
0.92,
0.95,
]
[system]
units = "cp2k"
[engine]
class = "cp2k"
cp2k = "cp2k"
input_path = "cp2k_input"
timestep = 0.5
subcycles = 1
cp2k_format = "xyz"
extra_files = [
"GTH_BASIS_SETS",
"GTH_POTENTIALS",
]
[orderparameter]
class = "pyretis_distance"
index = [
0,
1,
]
periodic = true
[output]
archive_every = 50
order-file = 1
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
kick-from = "previous"
[retis]
swapfreq = 0.5
```
## Running the RETIS simulation with PyRETIS
In order to run the RETIS simulation, the following steps are suggested:
- Create a new folder for the simulation named cp2k-hydrogen and enter this directory.
- Create a new file retis.toml with the input settings for the simulation.
- Download the input files given above and store them in a new directory named cp2k_input.
- Copy the extra files GTH_BASIS_SETS and GTH_POTENTIALS from your CP2K distribution and place them in the cp2k_input folder.
- Run the RETIS simulation using:
```pyretis
pyretis run -i retis.toml -p
```
## Tested by
The CP2K engine setup is represented by examples/tests/test-cp2k/. That fixture checks a shorter external-engine run and should be used as the regression target; this page explains how to adapt the same file layout for a user simulation.
## References
The CP2K web-page, https://www.cp2k.org/
---
# Exploring the order parameter (explore task)
This example shows the explore task on the 1D double well. An exploratory run wanders the order-parameter space without enforcing interface conditions, so it is a convenient first step when you do not yet know a good order parameter or where to place the interfaces. The visited order-parameter range and the generated trajectories then guide interface placement for a subsequent TIS/RETIS run, and can be reused as initial paths (see Getting the initial path).
Verification status: smoke – see example-test-status.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-double-well/explore/.
- Edit first: task = "explore" and the interfaces list in the [simulation] section; the order parameter in [orderparameter].
- Run: pyretis run -i explore.toml -p from the tutorial folder.
- Expected output: per-ensemble folders (001, 002, …) with order.txt / energy.txt recording the explored order-parameter range; no crossing-probability analysis (an explore run is for scouting, not rate estimation).
- Related check: examples/tests/test-internal/explorer/; see example-test-status.
## Selecting the explore task
The explore task is requested in the simulation section; the rest of the input is structured like an ordinary TIS run (interfaces, engine, order parameter), and the initial configuration is provided through the usual initial-path section:
```toml
[simulation]
task = "explore"
steps = 2000
interfaces = [
-0.9,
-0.5,
1.0,
]
seed = 0
```
## Running the exploration
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "explore"
steps = 2000
interfaces = [
-0.9,
-0.5,
1.0,
]
seed = 0
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
[box]
periodic = [
false,
]
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
[forcefield]
description = "1D double well"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
[output]
archive_every = 50
energy-file = 100
order-file = 100
[tis]
freq = 0.5
maxlength = 2000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
kick-from = "initial"
```
The initial configuration initial.xyz ships in the tutorial folder. Run with:
```pyretis
pyretis run -i explore.toml -p
```
## Tested by
The explore task is represented by examples/tutorials/path_sampling/internal/1D-double-well/explore/ (in the tutorial smoke suite) and by examples/tests/test-internal/explorer/, whose run is checked for structural validity: the ensemble directory layout, the path length and order-parameter rules, order.txt in lockstep with pathensemble.txt, and committed (non-loaded) moves.
---
# Using heavy checks as tutorials
The directories under examples/tests/ are consistency checks, but they are also useful minimal setups. This page translates them into user-facing recipes. The difference is intent:
- In examples/tests/, the goal is to reproduce reference output.
- In examples/tutorials/, the goal is to understand the input files well enough to adapt them to a new simulation.
The same recipes are shipped next to the code in examples/tutorials/from_tests/README.rst so they stay in sync with the heavy-test folder layout.
## Internal engine
Start from examples/tests/test-internal/ when you want a small, dependency-free template.
Test fixture
What it is useful for
How to turn it into a tutorial setup
retis
Minimal RETIS with the internal Langevin engine.
Copy retis.rst and initial.xyz; then change interfaces, steps, and potential parameters.
tis-multiple
Splitting a multi-ensemble TIS setup into per-ensemble input files.
Start from tis-multiple.rst and inspect the generated tis-001.rst-style files.
md-restart
Restarting MD with Langevin or Velocity Verlet.
Use it to learn which restart files are created and which settings can be overridden.
mdflux-restart
Initial flux and flux restarts.
Use it before designing a TIS/RETIS workflow that needs a flux estimate.
retis-load-sparse
Loading paths from sparse trajectory data.
Use it when your starting trajectories come from another tool.
retis-ss-wt-wf
Stone skipping, web throwing, and wire fencing.
Use it after the 1D RETIS tutorial to understand sub-move choices.
retis-zero-left
Zero-left bounded reactant basin.
Pair it with examples-retis-triple-well to see the zero_left shortcut.
compare-internal-with-lammps
Internal Lennard-Jones MD agreement with LAMMPS reference output.
Use it as a unit-conversion sanity check before swapping engines.
explorer
Loading frames for free-energy landscape exploration.
Use it after the 2D-hysteresis tutorial.
partial-path
REPPTIS-style partial paths.
Use it once you have a working REPPTIS input.
## External engines
External-engine checks are best used as dependency-specific templates. First verify that the external program works in your environment, then copy the input folder and adapt one setting at a time.
Suite
Use it for
First adaptation
test-gromacs
GROMACS command execution, file staging, sparse loading, restart, RETIS, and REPPTIS.
Replace the GROMACS coordinate/topology files, then update the order parameter.
test-lammps
LAMMPS propagation, velocity modification, phase-point dumping, and RETIS.
Replace system.data and lammps.in while keeping order.in simple until propagation works.
test-cp2k
CP2K execution, integration, and loading RETIS results.
Start with the hydrogen input and change only the CP2K input file once the runner is configured.
test-openmm
OpenMM RETIS through the Python engine.
Confirm that openmm imports, then adapt the two-particle setup to your force field.
test-pyvisa
PyVisA post-processing for the GROMACS sparse-load output.
Use after a path-sampling run has produced order, energy, and path ensemble files.
## Running a fixture briefly
For exploration, run a fixture in a temporary copy and stop early once the setup has passed initialization and produced a few cycles. For example:
```bash
cd examples/tests/test-internal/retis
timeout 30s pyretis run -i retis.toml -p
```
For a real tutorial run, remove the timeout and increase the number of cycles only after you have checked the first log and the out.rst file.
---
# Transport of methane in a sI hydrate
In this example, we are going to study the transport of methane in a sI hydrate structure. The initial configuration is shown in fig_gromacs_hydrate.
The process we will investigate is the diffusion of the methane molecule between different cages in the hydrate structure. The order parameter which measures the progress of the methane molecule is defined as the distance traveled by the methane molecule along a vector pointing between the geometric center of selected rings in the hydrate structure (see fig_gromacs_hydrate_order for an illustration).
In this example, we will make use of GROMACS 1 for running the dynamics. The example is structured as follows:
- Tutorial quick start
- Defining the order parameter
- Creating the PyRETIS input file
- Running the RETIS simulation with PyRETIS
- Tested by
- References
Verification status: engine – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/gromacs/hydrate/.
- Requirements: a working GROMACS command such as gmx and the hydrate input files in gromacs_input/; see user-section-engine-gromacs.
- Edit first: retis.toml for the [engine] section and orderp.py for the custom order parameter.
- Run: pyretis run -i retis.toml -p.
- Analyse: pyretis analyse -i retis.toml – the analysis reads the same input file as the run.
- Expected output: standard RETIS ensemble folders plus GROMACS files managed inside the working directories.
- Related checks: examples/tests/test-gromacs/ and examples/tests/test-pyvisa/; see example-test-status.
## Defining the order parameter
The order parameter we make use of is illustrated below.
This is a custom order parameter and we will now create a Python script for calculating it. We refer to the section on creating custom order parameters for more detailed information on how this can be done. Here, we just give the full Python script for calculating the order parameter:
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""This file defines the order parameter used for the hydrate example."""
import logging
import numpy as np
from pyretis.orderparameter import OrderParameter
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
class RingDiffusion(OrderParameter):
"""RingDiffusion(OrderParameter).
This class defines the ring diffusion order parameter for
the hydrate example.
Attributes
----------
name : string
A human readable name for the order parameter
index : integer
This selects the particle to use for the order parameter.
periodic : boolean
This determines if periodic boundaries should be applied to
the position or not.
"""
def __init__(self):
"""Initialise the order parameter.
Parameters
----------
name : string
The name for the order parameter
index : tuple of ints
This is the indices of the atom we will use the position of.
periodic : boolean, optional
This determines if periodic boundary conditions should be
applied to the position.
"""
super().__init__(description='Ring diffusion for hydrate')
self.idx1 = np.array([56, 64, 104, 112, 200, 208], dtype='i4')
# convert to atom index:
self.idx1 *= 4
self.idx1 -= 3
# convert to 0 index:
self.idx1 -= 1
self.idx2 = np.array([56, 64, 72, 80, 104, 112, 136, 152, 168, 176,
200, 208, 232, 248, 264, 272, 296, 304, 328,
336, 344, 352, 360, 368], dtype='i4')
# convert to atom index:
self.idx2 *= 4
self.idx2 -= 3
# convert to 0 index:
self.idx2 -= 1
self.idxd = 1472 # index for diffusing atom
def calculate(self, system):
"""Calculate the order parameter.
Here, the order parameter is just the distance between two
particles.
Parameters
----------
system : object like :py:class:`.System`
This object is used for the actual calculation, typically
only `system.pos` and/or `system.vel` will be used. In some
cases `system.forcefield` can also be used to include
specific energies for the order parameter.
Returns
-------
out : list of floats
The order parameter.
"""
pos = system.pos
resl = 1.0e3
cm1 = np.average(np.rint(pos[self.idx1] * resl) / resl, axis=0)
cm2 = np.average(np.rint(pos[self.idx2] * resl) / resl, axis=0)
cmvec = cm2 - cm1
molvec = np.rint(pos[self.idxd] * resl) / resl
molvec -= cm1
orderp = -np.dot(cmvec, molvec) / np.sqrt(np.dot(cmvec, cmvec))
return [orderp]
if __name__ == '__main__':
testo = RingDiffusion()
print('Idx1', testo.idx1)
print('Idx2', testo.idx2)
```
This order parameter can be used in a simulation in PyRETIS by adding the following order parameter section to the input file:
```rst
Orderparameter
--------------
class = RingDiffusion
module = orderp.py
```
In addition, we can add extra order parameters, for instance the position (z-coordinate) of the diffusing methane molecule:
```rst
Collective-variable
-------------------
class = pyretis_position
index = 1472 # This is the methane molecule
dim = z
```
## Creating the PyRETIS input file
The input file for PyRETIS follows the structure we have used in the previous examples and we will not go into details about all sections here. In the following, we will just describe the section for using the GROMACS engine in more detail. The full input file for the RETIS simulation is given at the end of this section.
In order to make use of GROMACS, we add the following engine section to the input file:
```toml
[engine]
class = "gromacs_steps"
gmx = "gmx"
mdrun = "gmx mdrun"
input_path = "gromacs_input"
timestep = 0.002
subcycles = 5
gmx_format = "g96"
```
Here, we make use of the following keywords:
- class = "gromacs_steps" which specifies the engine we will use (here: gromacs).
- gmx = "gmx" which specifies the GROMACS executable. On your system, this might be named differently.
- mdrun = "gmx mdrun" which specifies the command for running GROMACS. Note that you here can tune how to run GROMACS, for instance by adding a mpirun or something similar if that fits your system.
- input_path = gromacs_input which specifies the directory where PyRETIS will look for the input files for GROMACS. PyRETIS assumes that, at least, the following files are in this folder:
- conf.g96 which contains the initial configuration to use.
- topol.top which contains the topology for GROMACS.
- grompp.mdp which contains simulation settings for GROMACS.
- The files needed for this example can be downloaded as a zip archive here: gromacs_input.zip.
- timestep = 0.002 which is the time step to use in the GROMACS simulations.
- subcycles = 5 which is the number of subcycles GROMACS will complete before PyRETIS re-calculated order parameter. This will also be the frequency by which GROMACS write information (trajectory and energies) to the disk.
This specifies and selects the GROMACS engine for use with PyRETIS. The full input file for the RETIS simulation is given below:
```toml
collective-variable = [
{ class = "pyretis_position", index = 1472, dim = "z" },
]
[simulation]
task = "retis"
steps = 10
interfaces = [
-0.26,
-0.24,
-0.22,
-0.2,
-0.19,
-0.18,
-0.17,
-0.16,
-0.15,
-0.14,
-0.13,
-0.12,
-0.11,
-0.1,
-0.09,
-0.08,
-0.07,
-0.06,
-0.05,
-0.04,
-0.02,
0.0,
0.02,
0.2,
]
[system]
units = "gromacs"
[engine]
class = "gromacs_steps"
gmx = "gmx"
mdrun = "gmx mdrun"
input_path = "gromacs_input"
timestep = 0.002
subcycles = 5
gmx_format = "g96"
[orderparameter]
class = "RingDiffusion"
module = "orderp.py"
[output]
archive_every = 50
order-file = 1
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = true
rescale_energy = false
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
kick-from = "previous"
[retis]
swapfreq = 0.5
```
## Running the RETIS simulation with PyRETIS
In order to run the RETIS simulation, the following steps are suggested:
- Create a new folder for the simulation named gromacs-hydrate and enter this directory.
- Create a new file orderp.py containing the script for calculating the order parameter.
- Create a new file retis.toml with the input settings for the simulation.
- Download the input files (gromacs_input.zip) for GROMACS and store these in a new directory named gromacs_input.
- Run the RETIS simulation using:
```pyretis
pyretis run -i retis.toml -p
```
This will execute the PyRETIS simulation. An example of a reactive trajectory (obtained in the last path ensemble) is shown in fig_gromacs_hydrate_traj.
## Tested by
The external-engine wiring is represented by examples/tests/test-gromacs/. The PyVisA analysis path for external trajectories is represented by examples/tests/test-pyvisa/. These checks are deliberately shorter than a user tutorial run and may require local GROMACS availability.
## References
The GROMACS web-page, http://www.gromacs.org/
---
# Infinite swapping (parallel replica exchange)
This example shows how to run the infinite-swapping sampler – the asynchronous, highly parallel reformulation of replica exchange of Zhang, Baldauf, Roet, Lervik and van Erp 1. Instead of attempting swaps between fixed neighbouring path-ensemble pairs at a synchronisation point, a central coordinator records swap statistics over the running stream of completed trajectories and a pool of workers draws its next task from the resulting distribution. This keeps the wall-clock time scaling close to linearly with the number of workers over a far wider range than synchronous replica exchange allows, which is what makes it the route of choice for large, parallel rate-constant calculations.
The input keywords used here – the [runner] section and the two execution routes – are documented in the runner section reference. The sampler itself is introduced in the moves guide.
Verification status: smoke – see example-test-status.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/turtlemd/1D-double-well/infinite_swapping/ (the internal TurtleMD engine, so no external MD installation is required).
- Edit first: the workers count in the [runner] section and the interfaces / shooting_moves lists in the [simulation] section. Wire Fencing ("wf") pairs naturally with this sampler because it keeps acceptance high on the long paths that aggressive parallel sampling produces.
- Run: bash runner.sh from the tutorial folder – it seeds the starting paths from the bundled load_copy/ and then calls pyretis run -i infswap.toml. A run keeps its live trajectories in a per-ensemble archive (//, load_dir defaulting to accepted); a flat, top-level staged load directory (the historical layout the bundled load_copy/ mirrors) is still read as input via the legacy-layout fallback.
- Analyse: pyretis analyse -i infswap.toml after the run, which combines the per-interface crossing probabilities with the WHAM analysis 2 rather than per-ensemble averaging.
- Expected output: the standard per-ensemble pathensemble.txt / order.txt / energy.txt files (this is the only output format the scheduler writes; there is no native_compat setting any more), so the regular analysis tooling and pyvisa work on an infinite-swapping run exactly as they would on a native one.
## Selecting the route
Every RETIS-family run is driven through the single pyretis run command; PyRETIS reads the input TOML and picks the route:
- An input with a [runner] section – or an explicit [simulation] task = "infinite_swapping" – runs through the infinite-swapping scheduler with the requested number of workers.
- A canonical task = "retis" input with an internal engine is also run through the scheduler, at a single worker, so a plain RETIS input and its parallel infinite-swapping counterpart differ only by the [runner] section.
The minimal addition that turns a single-worker RETIS run into a four-worker infinite-swapping run is therefore just:
```toml
[runner]
workers = 4
files = ["orderp.py"] # custom order-parameter module each worker needs
```
See the runner section reference for the full keyword list and for the PYRETIS_NATIVE_LOOP / PYRETIS_WORKERS environment overrides.
## Other engines
The same input shape drives the external engines. Ready-to-read references ship alongside the TurtleMD example:
- examples/tutorials/path_sampling/gromacs/H2/infinite_swapping and examples/tutorials/path_sampling/gromacs/H2/infinite_swapping_multi_engine – GROMACS (requires gmx on the PATH).
- examples/tutorials/path_sampling/lammps/H2/infinite_swapping – LAMMPS.
- examples/tutorials/path_sampling/cp2k/H2/infinite_swapping – CP2K.
- examples/tutorials/path_sampling/ase/H2/infinite_swapping – an ASE TOML reference.
A minimal programmatic driver that runs the asyncrunner without the CLI is at examples/tutorials/extending/basic_runner.py.
## Tested by
The per-engine infinite-swapping examples (examples/tutorials/path_sampling///infinite_swapping/) are exercised by the integration suite in test/infswap/ and the native-via-scheduler round trip in test/integration/test_via_scheduler.py. The cross-route rate equivalence (in-process loop vs. scheduler) is checked by the validation study under examples/validation/. Use those fixtures as the short regression checks and this page as the user-facing guide.
## References
D. T. Zhang, L. Baldauf, S. Roet, A. Lervik and T. S. van Erp, Highly parallelizable path sampling with minimal rejections using asynchronous replica exchange and infinite swaps, Proc. Natl. Acad. Sci. U.S.A. 121, e2318731121 (2024), https://doi.org/10.1073/pnas.2318731121
S. Kumar, J. M. Rosenberg, D. Bouzida, R. H. Swendsen and P. A. Kollman, The weighted histogram analysis method for free-energy calculations on biomolecules. I. The method, J. Comput. Chem. 13, 1011-1021 (1992), https://doi.org/10.1002/jcc.540130812
---
# Breaking a bond with RETIS and LAMMPS
In this example, you will explore bond breaking in a simple model system with the Replica Exchange Transition Interface Sampling (RETIS) algorithm. This example is similar to the Breaking a bond with RETIS example, but here, we will make use of LAMMPS for running the dynamics.
The system we consider consist of a simple diatomic bistable molecule immersed in a fluid of purely repulsive particles, as illustrated in fig_lammps_wca.
As mentioned in the Breaking a bond with RETIS example, this system was actually considered in the article introducing the TIS method, 1 and we will here largely follow this simulation set-up. We refer to the original article for more details. This example is structured as follows:
- Tutorial quick start
- Defining the system, potential and order parameter
- Preparing input files for LAMMPS
- Generating an initial trajectory
- Running the RETIS simulation
- Setting up the LAMMPS engine
- Loading the initial path
- Executing the RETIS simulation
- Analyzing the RETIS results
- Tested by
- References
Verification status: engine – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/lammps/2D-wca/.
- Requirements: a LAMMPS executable such as lmp or lmp_serial and the files in lammps_input/; see user-guide-lammps.
- Edit first: the lmp command and input_path in retis.toml, then the LAMMPS input files in lammps_input/ if you change the physical model.
- Run: pyretis run -i retis.toml -p.
- Analyse: copy the input, set the effective time step (LAMMPS time step times subcycles) in the copy, then run pyretis analyse -i analyse.toml.
- Expected output: standard RETIS ensemble folders and LAMMPS trajectory/energy files managed by PyRETIS.
- Related check: examples/tests/test-lammps/; see example-test-status.
## Defining the system, potential and order parameter
The system we will consider is a 2D system consisting of 25 particles at a fixed total energy of 1.0 per particle (in reduced units). The number density is set to 0.7. Between two of the particles, the “diatomic molecule”, we have a double well potential, given by
where h = 15, w = 0.5 and r_0 = 2^{1/6}. In addition, we have a Weeks-Chandler-Andersen (WCA) pair interaction between all particles. The potential functions are illustrated in fig_lammps_potential. In this figure, we also show the interfaces we will consider for the RETIS method. These interfaces have been placed at the positions given in the original TIS article. 1
For the order parameter, we choose the distance between the two atoms which are initially bound. We will in the next section describe the input files needed to run this example with LAMMPS.
## Preparing input files for LAMMPS
We will here follow the description given in the LAMMPS user guide. Thus we need to create:
- A LAMMPS input file for performing molecular dynamics: The LAMMPS input file to use with this example can be found here: lammps.in. We will not discuss this input script in detail, but we note the following:
- We set a variable defining the temperature the velocities are drawn at: variable SET_TEMP equal 1.0. When performing the shooting move, we will draw velocities and scale them so that the total energy of the system is 1.0 per particle: the PyRETIS input sets rescale_energy = 1 in [tis], and LAMMPS, with lj units, evaluates the energy per particle.
- We include an external script, defining the double well potential: include "dw-wca.in" This file can be found here: dw-wca.in.
- A file containing initial coordinates: We load the initial coordinates from a file, system.data. This file can be found here: system.data. The box size is set to give a 2D particle density of 0.7 in reduced units.
- A LAMMPS input file for calculating the order parameter: The LAMMPS input file defining the order parameter is in this case rather simple:
```
variable op_1 equal v_delr
```
- The file itself can be found here order.in. The file basically renames an already existing variable, delr, which is used in the potential energy calculation, to op_1, which is the variable name PyRETIS expects for the order parameter.
In total, we now have 4 files we will use to define the LAMMPS part of the PyRETIS simulation:
## LAMMPS input files needed for the example
File description
Filename/download
LAMMPS MD input file
lammps.in
Initial configuration
system.data
Order parameter definition
order.in
Definition of potential functions
dw-wca.in
## Generating an initial trajectory
We need at least two initial trajectories for PyRETIS, which should satisfy some requirements for the order parameter as illustrated in fig_lammps_initial_order_parameter.
The requirements for the initial trajectory in the [0^-] ensemble are that:
- The first and last points of the trajectory should be the only two points where the order parameter is larger than the value of the first interface.
- The trajectory must be more than the first and last points.
These conditions imply that two interface crossing events occur during the trajectory, and that all points other than the first and last points have order parameter values less than the value of the first interface. An example satisfying these conditions is shown in green in fig_lammps_initial_order_parameter.
For an initial trajectory satisfying the requirements for all other ensembles, it is sufficient that:
- The first point of the trajectory should have an order parameter value less than the value of the first interface.
- The last point of the trajectory should have an order parameter value greater than the value of the last interface.
- There must be at least one other point in addition to the first and last points in the trajectory, and these other points should all have order parameter values in between the values of the first and last interfaces.
These conditions imply that we would like to use a reactive trajectory as an initial trajectory. An example reactive trajectory is shown in cyan in fig_lammps_initial_order_parameter.
Because the reactive condition is normally a rare event, we need to run a biased simulation in order to generate the initial reactive trajectory. In this case, because the reactive condition was distance based (distance between two atoms), we are able to use LAMMPS fix spring to pull the atoms apart:
```pyretis
fix pulling_simulation atom1 spring couple atom2 2.0 0.0 0.0 0.0 1.0
```
In this example, atom1 and atom2 are LAMMPS groups referring to the two atoms participating in the double well potential.
## Running the RETIS simulation
After having generated an initial trajectory, we are ready to set up the simulation for PyRETIS. We will here use similar settings to the internal example for the same system, please see the Breaking a bond with RETIS example for a description of these settings. The full input script can be found here retis.toml. In the following, we will highlight the settings that are specific to using LAMMPS, and to loading the initial trajectory we have already generated.
### Setting up the LAMMPS engine
The LAMMPS engine is defined with the following section,
```toml
[engine]
class = "lammps_steps"
lmp = "lmp_serial"
input_path = "lammps_input"
subcycles = 1
extra_files = [
"dw-wca.in",
]
```
These lines assumes that we have stored the LAMMPS input files in a folder named lammps_input and that there is an “extra” file named dw-wca.in needed here. The term “extra” just refers to the fact that this file is in addition to the default files PyRETIS expects to find here. We also specify a specific name for the LAMMPS executable lmp = lmp_serial in this case. Note that this might have to be changed before running the example on your own computer.
### Loading the initial path
Loading the previously obtained initial path can be done with the following section:
```toml
[initial-path]
method = "load"
load_folder = "load"
```
This assumes that the initial path is stored in a folder named load. In this folder, we create a sub-folder for each ensemble, load/000, load/001, …, load/007 and for each of these ensemble directories we include the files needed for the initial path. For a given ensemble this should include the following:
- A file named order.txt with the order parameters for the path.
- A file named traj.txt with information about the initial path: This includes information about the snapshots the initial path consist of. For each snapshot we need to specify the following:
- the step number (“index”) in the path
- the LAMMPS trajectory file that contains the snapshot
- the index in the LAMMPS trajectory file that corresponds to the snapshot
- a flag which tells PyRETIS if the velocities are to be reversed or not before PyRETIS can make use of it.
- The first few lines of such a file might look like:
```pyretis
# Step Filename index vel
0 cross.lammpstrj 1174 -1
1 cross.lammpstrj 1175 -1
2 cross.lammpstrj 1176 -1
3 cross.lammpstrj 1177 -1
```
- Here, the first column lists the step number for the initial path, the second column contains information about what LAMMPS trajectory file to read, the third column tells PyRETIS what index in the given LAMMPS trajectory the snapshot corresponds to and the fourth (and final) column is the velocity flag. A -1 means that PyRETIS can use this snapshot without reversing the velocities, while a 1 means that PyRETIS will have to reverse velocities before using the snapshot.
- A directory named accepted which contains the LAMMPS trajectory file(s) the initial path is made up of.
### Executing the RETIS simulation
Once all the input files have been assembled, the simulation can be executed using pyretis run:
```pyretis
pyretis run -i retis.toml -p
```
assuming that you named the main PyRETIS input file retis.toml.
## Analyzing the RETIS results
After the simulation has finished, or while it is running, you can start analyzing the results. For instance, we can have a look at the accepted paths in the last (007) ensemble. These are stored per path in the operational store 007/accepted/ (and, once replaced, in the per-ensemble long-term store 007/archive/). In figure fig_lammps_rxexample, we show the energies and the order parameter for such an accepted path:
Further, we can run an analysis to obtain the crossing probability and the rate of the transition. This can be done using the pyretis analyse tool, which reads the same kind of input file as the run.
The analysis needs the effective time step – the LAMMPS time step multiplied by the subcycles – so that the rate comes out in the right time units. Rather than editing the input you ran with, copy it and set the effective time step in the copy:
```
cp retis.toml analyse.toml
```
then make sure the [engine] section of analyse.toml carries that value. As a specific example, for a LAMMPS time step of 0.001 and a subcycles setting of 1 in the original retis.toml:
```toml
[engine]
timestep = 0.001
```
The analysis can then be executed using:
```pyretis
pyretis analyse -i analyse.toml
```
This will produce several output files, which are summarized in a folder report in the main directory. Below you can find a table with some sample results and a figure showing the obtained crossing probability.
## Crossing probability, initial flux and the rate constate for an example run of the 2D WCA example with LAMMPS. The quantities are given in reduced units.
Quantity
Value
Relative error
Crossing probability
5.5e-12
24 %
Initial flux
0.1196
1 %
Rate constant
6.6e-13
24 %
## Tested by
The LAMMPS interface is represented by examples/tests/test-lammps/. The heavy check is the correct regression target for executable names, file layout, and order-parameter loading; this page explains the longer user-facing setup.
## References
Titus S. van Erp, Daniele Moroni, and Peter G. Bolhuis, A novel path sampling method for the calculation of rate constants, The Journal of Chemical Physics, 118, pp. 7762-7774, 2003, https://doi.org/10.1063/1.1562614.
---
# Molecular dynamics examples
In this example, we will perform a MD simulation of a Lennard-Jones fluid. The purpose of this example is to illustrate two different ways we can use PyRETIS:
- Running the simulation using an input file and the PyRETIS executable
- Running the simulation by making explicit use of the PyRETIS library
The simulation we set up is rather ordinary and we will start from a regular lattice and simulate the melting at a specified temperature as illustrated in the figure below.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/molecular_dynamics/md_nve/from_settings/ for input-file based MD, or examples/tutorials/molecular_dynamics/md_nve/ for the Python API version.
- Edit first: settings.toml when using pyretis run; edit md_nve.py only when you want to learn the library API.
- Run from an input file: pyretis run -i settings.toml -p.
- Run from Python: python md_nve.py.
- Expected output: energy.txt, thermo.txt, traj.xyz, pyretis.log, and out.toml for the input-file version.
- Related checks: examples/tests/test-internal/md-restart/ and examples/tests/test-internal/compare-internal-with-lammps/; see example-test-status.
## Molecular dynamics using an input file
We can execute PyRETIS by creating an input file and using the PyRETIS application. In order to run this example, merge the code-snippets given below into one file, say md.toml, which then can be executed by:
```pyretis
pyretis run -i md.toml
```
In the first part of the input file, we will specify the type of simulation to run and the number of steps to perform by defining the simulation section:
```toml
[simulation]
task = "md-nve"
steps = 1000
```
In order to produce some actual dynamics, we will have to integrate the equations of motion. The engine to use for this is selected using the engine section:
```toml
[engine]
class = "velocityverlet"
timestep = 0.002
```
and we define the system we are studying using the system section and the particles section: and the particles:
```toml
[system]
temperature = 2.0
units = "lj"
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
generate = "fcc"
repeat = [
3,
3,
3,
]
density = 0.9
[particles.velocity]
generate = "maxwell"
temperature = 2.0
momentum = true
seed = 0
[particles.mass]
Ar = 1.0
```
We also need to specify a force field, which is done by specifying the forcefield section and one or more potential sections. In this case, the force field is set up using a single type of potential function:
```toml
[forcefield]
description = "Lennard Jones test"
[[potential]]
class = "PairLennardJonesCutnp"
shift = true
[potential.parameter.0]
sigma = 1.0
epsilon = 1.0
rcut = 2.5
```
and we modify the output created by specifying the output section:
```toml
[output]
backup = "overwrite"
energy-file = 1
order-file = 10
cross-file = 1
trajectory-file = 10
```
What we are essentially modifying here is the frequency of the output, and we are also telling PyRETIS to overwrite old output files that might be present in the same directory (this is controlled by backup = "overwrite").
After running this example, the following files should have been created:
This file contains energies and temperature as a function of time/step in the MD simulation:
This file contains the same energies as energy.txt and in addition the pressure.
This file contains the trajectory and can be visualised using for instance VMD.
This file contains output messages from the PyRETIS application.
This file contains the input settings you provided as PyRETIS read them. This can be used to check the correctness of the input file.
Plotting the pressure from thermo.txt should then give a figure similar to:
## Molecular dynamics using the PyRETIS library
In this part of the example, we will make explicit use of the PyRETIS library If you want to try this example, you will have to copy the code-snippets given below into a Python script, say md.py, and run it as
```pyretis
python md.py
```
This example is best understood if you have read about the PyRETIS API.
We begin the example by importing from the PyRETIS library:
```python
import numpy as np
from matplotlib import pyplot as plt
from pyretis.core import create_box, Particles, System
from pyretis.simulation import SimulationNVE
from pyretis.engines import VelocityVerlet
from pyretis.tools import generate_lattice
from pyretis.forcefield import ForceField
from pyretis.forcefield.potentials import PairLennardJonesCutnp
```
We also import Numpy and matplotlib here in order to do some plotting. If you want a more fancy plot and have a recent version of matplotlib installed you can try to use one of the many styles, e.g.:
```python
plt.style.use('seaborn-v0_8-colorblind')
```
Next, we create the initial positions and use this to set up a simulation box:
```python
def set_up_system():
"""Set up a system using the PyRETIS library."""
print('Creating box:')
xyz, size = generate_lattice('fcc', [3, 3, 3], density=0.9)
size = np.array(size)
box = create_box(low=size[:, 0], high=size[:, 1])
print(box)
```
We can use the lattice we generated to populate a system with particles. The system contains information about the box, the particles and the temperature:
```python
print('Creating system:')
system = System(units='lj', box=box, temperature=2.0)
system.particles = Particles(dim=3)
for pos in xyz:
system.add_particle(pos, vel=np.zeros_like(pos),
force=np.zeros_like(pos),
mass=1.0, name='Ar', ptype=0)
gen_settings = {'distribution': 'maxwell', 'momentum': True}
system.generate_velocities(**gen_settings)
print(system.particles)
```
In the last lines in the above code, we generated initial velocities, from a Maxwellian distribution such that the total momentum of the system is zero (requested by setting the key 'momentum' to True in the dictionary).
Now, we just have to set-up a force field:
```python
print('Creating force field:')
potentials = [PairLennardJonesCutnp(dim=3, shift=True, mixing='geometric')]
parameters = [{0: {'sigma': 1, 'epsilon': 1, 'rcut': 2.5}}]
ffield = ForceField('Lennard Jones force field',
potential=potentials, params=parameters)
system.forcefield = ffield
print(system.forcefield)
return system
```
select an engine and a simulation to run:
```python
def set_up_simulation(system):
"""Set up the simulation."""
print('Creating simulation:')
engine = VelocityVerlet(0.002)
simulation = SimulationNVE(ensemble={'system': system,
'engine': engine},
settings={},
controls={'steps': 200})
return simulation
```
Now, we are ready to run the simulation and plot the energies as a function of the simulation step:
```python
def run_simulation(simulation):
"""Run the simulation and collect some outputs."""
ekin = []
vpot = []
etot = []
step = []
for result in simulation.run():
if result['cycle']['step'] % 10 == 0:
print('Step:', result['cycle']['step'])
step.append(result['cycle']['step'])
ekin.append(result['thermo']['ekin'])
vpot.append(result['thermo']['vpot'])
etot.append(result['thermo']['etot'])
return step, vpot, ekin, etot
def plot_results(step, vpot, ekin, etot, show=True):
"""Plot some energies."""
fig1 = plt.figure()
ax1 = fig1.add_subplot(111)
ax1.plot(step, ekin, label='Kinetic energy')
ax1.plot(step, etot, label='Total energy')
ax1.plot(step, vpot, label='Potential energy')
ax1.set_xlabel('Step no.')
ax1.set_ylabel('Energy / reduced units')
ax1.legend()
fig1.tight_layout()
fig1.savefig('out.png', dpi=150)
if show:
plt.show()
def main(args):
"""Set up, run simulation and plot some energies."""
system = set_up_system()
simulation = set_up_simulation(system)
step, vpot, ekin, etot = run_simulation(simulation)
plot_results(step, vpot, ekin, etot, show='noplot' not in args)
if __name__ == '__main__':
main(sys.argv[1:])
```
This should result in a plot similar to:
## Tested by
The input-file workflow is represented by examples/tests/test-internal/md-restart/. Internal Lennard-Jones MD output is also checked against saved LAMMPS reference data in examples/tests/test-internal/compare-internal-with-lammps/.
---
# Molecular dynamics in PyRETIS with C or FORTRAN
In this example, we will implement a Lennard-Jones potential in C or FORTRAN and use this to run a MD simulation with PyRETIS for a simple system. The system we consider is a Lennard-Jones mixture starting for a specific configuration as shown below. After running for some steps (which melt the initial configuration), we reverse the velocities and continue the simulation.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/molecular_dynamics/md_nve/forward_backward/ for the pure Python version and examples/tutorials/molecular_dynamics/md_nve/forward_backward/external/ for C/FORTRAN potentials.
- Requirements: a working C compiler for the C variant or f2py/FORTRAN for the FORTRAN variant.
- Edit first: the USE variable in md_forward_backward_ext.py, then the potential module you want to test.
- Build: use python setup.py build_ext --inplace for the C module or make for the FORTRAN module.
- Run: python md_forward_backward.py or python md_forward_backward_ext.py.
- Expected output: traj.xyz and an energy plot showing the forward/backward consistency check.
- Related checks: examples/tests/test-internal/compare-internal-with-lammps/ and the extension examples; see example-test-status.
## Creating a new potential in C or FORTRAN
In this example, we will create a new potential function to use with PyRETIS in C or FORTRAN.
As discussed in the introduction to the library, we need to create a new potential class which PyRETIS will make use of. This new potential class will import routines from a C or FORTRAN library and use these routines to do the actual computation of the forces.
We will in the following show how this can be done, both with FORTRAN and with C. Essentially, we are going to complete the following steps:
- We write the code responsible for the evaluation of the force in an external C or FORTRAN library.
- We compile the external code. Here we do one of the following:
- For C, we create a setup.py which is used to compile.
- For FORTRAN, make use of a Makefile and the program f2py.
- We write a Python module containing a new PotentialFunction sub-class representing the new potential. This class imports the library which we created in steps 1 and 2.
### Writing a new potential function with FORTRAN
We will now create a potential function writing the external library in FORTRAN. This is done in the following steps:
- Step 1. Creating the FORTRAN code
- Step 2. Creating the Makefile and compiling
- Step 3. Creating a new Python class for the potential function
#### Step 1. Creating the FORTRAN code
The FORTRAN code for evaluating the potential and forces is relative simply and we assume that we are handed positions, velocities, and forces as double precision arrays and that we can directly make use of them.
```fortran
! Copyright (c) 2023, PyRETIS Development Team.
! Distributed under the LGPLv2.1+ License. See LICENSE for more info.
module ljfortran
implicit none
private
public :: potential, force, potential_and_force
public :: pbc_dist
contains
function pbc_dist(v, box, ibox) result(dist)
! Function to apply periodic boundaries to a given displacement.
!
! Parameters
! ----------
! v : the displacement to pbc-wrap
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
!
! Returns
! -------
! dist : the pbc-wrapped displacement
implicit none
double precision :: dist
double precision, intent(in) :: v, box, ibox
dist = v - nint(v * ibox) * box
end function pbc_dist
subroutine potential(pos, box, ibox, lj3, lj4, offset, rcut2, ptype, n, d, p, vpot)
! Function to evaluate the Lennard-Jones potential
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj3 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**12
! lj4 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! offset : Potential energy shift
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! vpot : the potential energy
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: box, ibox
double precision, dimension(p, p), intent(in) :: lj3, lj4, offset, rcut2
integer, dimension(n), intent(in) :: ptype
double precision, intent(out) :: vpot
double precision :: rsq, r2inv, r6inv, dx, dy, dz
double precision, dimension(:), pointer :: x, y, z
integer :: i, j, itype, jtype
vpot = 0.0D0
x => pos(1:n,1)
y => pos(1:n,2)
z => pos(1:n,3)
do i=1,n-1
itype = ptype(i) + 1
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv**3
vpot = vpot + r6inv * (lj3(jtype, itype) * r6inv - lj4(jtype, itype))
vpot = vpot - offset(jtype, itype)
end if
end do
end do
end subroutine potential
subroutine force(pos, box, ibox, lj1, lj2, rcut2, ptype, n, d, p, forces, virial)
! Function to evaluate the Lennard-Jones force
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj1 : Lennard-Jones parameters, 48.0 * epsilonij * sigmaij**12
! lj2 : Lennard-Jones parameters, 24.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! forces : the Lennard-Jones forces on the particles
! virial : the virial matrix
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(out) :: forces
double precision, dimension(d, d), intent(out) :: virial
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: box, ibox
double precision, dimension(p, p), intent(in) :: lj1, lj2, rcut2
integer, dimension(n), intent(in) :: ptype
double precision, dimension(:), pointer :: x, y, z, fx, fy, fz
double precision :: rsq, r2inv, r6inv, forcelj, dx, dy, dz
double precision :: forceij_x, forceij_y, forceij_z
integer :: i, j, itype, jtype
x => pos(1:n, 1)
y => pos(1:n,2)
z => pos(1:n,3)
fx => forces(1:n,1)
fy => forces(1:n,2)
fz => forces(1:n,3)
forces = 0.0D0
virial = 0.0D0
do i=1,n-1
itype = ptype(i) + 1
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv*r2inv*r2inv
forcelj = r2inv * r6inv * (lj1(jtype, itype) * r6inv - lj2(jtype, itype))
forceij_x = forcelj * dx
forceij_y = forcelj * dy
forceij_z = forcelj * dz
fx(i) = fx(i) + forceij_x
fy(i) = fy(i) + forceij_y
fz(i) = fz(i) + forceij_z
fx(j) = fx(j) - forceij_x
fy(j) = fy(j) - forceij_y
fz(j) = fz(j) - forceij_z
! accumulate for the virial:
virial(1,1) = virial(1,1) + forceij_x * dx
virial(1,2) = virial(1,2) + forceij_x * dy
virial(1,3) = virial(1,3) + forceij_x * dz
virial(2,1) = virial(2,1) + forceij_y * dx
virial(2,2) = virial(2,2) + forceij_y * dy
virial(2,3) = virial(2,3) + forceij_y * dz
virial(3,1) = virial(3,1) + forceij_z * dx
virial(3,2) = virial(3,2) + forceij_z * dy
virial(3,3) = virial(3,3) + forceij_z * dz
end if
end do
end do
end subroutine force
subroutine potential_and_force(pos, box, ibox, lj1, lj2, lj3, lj4, offset, rcut2, ptype, n, d, p, forces, virial, vpot)
! Function to evaluate the Lennard-Jones force and potential
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj1 : Lennard-Jones parameters, 48.0 * epsilonij * sigmaij**12
! lj2 : Lennard-Jones parameters, 24.0 * epsilonij * sigmaij**6
! lj3 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**12
! lj4 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! offset : Potential energy shift
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! vpot : the potential energy
! forces : the Lennard-Jones forces on the particles
! virial : the virial matrix
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(out) :: forces
double precision, dimension(d, d), intent(out) :: virial
double precision, intent(out) :: vpot
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: ibox, box
double precision, dimension(p, p), intent(in) :: lj1, lj2, lj3, lj4, offset, rcut2
double precision, dimension(:), pointer :: x, y, z, fx, fy, fz
integer, dimension(n), intent(in) :: ptype
double precision :: rsq, r2inv, r6inv, forcelj, dx, dy, dz
double precision :: forceij_x, forceij_y, forceij_z
integer :: i, j, itype, jtype
forces = 0.0D0
virial = 0.0D0
vpot = 0.0D0
x => pos(1:n,1)
y => pos(1:n,2)
z => pos(1:n,3)
fx => forces(1:n,1)
fy => forces(1:n,2)
fz => forces(1:n,3)
do i=1,n-1
itype = ptype(i) + 1
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv*r2inv*r2inv
forcelj = r2inv * r6inv * (lj1(jtype, itype) * r6inv - lj2(jtype, itype))
forceij_x = forcelj * dx
forceij_y = forcelj * dy
forceij_z = forcelj * dz
fx(i) = fx(i) + forceij_x
fy(i) = fy(i) + forceij_y
fz(i) = fz(i) + forceij_z
fx(j) = fx(j) - forceij_x
fy(j) = fy(j) - forceij_y
fz(j) = fz(j) - forceij_z
vpot = vpot + r6inv * (lj3(jtype, itype) * r6inv - lj4(jtype, itype))
vpot = vpot - offset(jtype, itype)
! accumulate for the virial:
virial(1,1) = virial(1,1) + forceij_x * dx
virial(1,2) = virial(1,2) + forceij_x * dy
virial(1,3) = virial(1,3) + forceij_x * dz
virial(2,1) = virial(2,1) + forceij_y * dx
virial(2,2) = virial(2,2) + forceij_y * dy
virial(2,3) = virial(2,3) + forceij_y * dz
virial(3,1) = virial(3,1) + forceij_z * dx
virial(3,2) = virial(3,2) + forceij_z * dy
virial(3,3) = virial(3,3) + forceij_z * dz
end if
end do
end do
end subroutine potential_and_force
end module ljfortran
```
#### Step 2. Creating the Makefile and compiling
In order to compile the FORTRAN code created in the previous step, we make use of f2py.
Note that in some systems you might actually need to use f2py3 (or, for a specific version: f2py3.7 etc.) if you have several versions of Python installed on your system. This is to ensure that the FORTRAN code is compiled with a version that matches the version of Python you are using. You will then have to modify the F2PY = f2py setting in the Makefile.
```make
F2PY = f2py
F2PY_FLAGS = --fcompiler=gfortran
MODULE=ljfortran
all: ${MODULE}.so
${MODULE}.so: ${MODULE}.f90
${F2PY} ${F2PY_FLAGS} -c $< -m ${MODULE}
clean:
find -name \*.so -delete
find -name \*.pyc -delete
find -name __pycache__ -delete
veryclean:
${MAKE} clean
find -name \*energy.dat* -delete
find -name \*thermo.txt* -delete
find -name \*traj.gro* -delete
```
Note that the FORTRAN compiler is specified using --fcompiler=gfortran and other choices can be seen by running:
```pyretis
f2py -c --help-fcompiler
```
Further, the Makefile assumes that the FORTRAN module is named ljfortran.f90.
#### Step 3. Creating a new Python class for the potential function
For the Python class representing the potential function, we import the module we just compiled and make use of the methods defined in that module.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of using a Lennard-Jones potential implemented in FORTRAN."""
import sys
from pathlib import Path
import logging
import numpy as np
from pyretis.forcefield.potentials import PairLennardJonesCut
from pyretis.forcefield.potentials.pairpotentials import (
generate_pair_interactions
)
# Just to handle imports of the library:
sys.path.insert(0, Path(__file__).parent.resolve())
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
try:
from ljfortran import ljfortran
except ImportError:
MSG = ('Could not import external FORTRAN library.'
'\nPlease compile with "make"!')
logger.critical(MSG)
raise ImportError(MSG)
__all__ = ['PairLennardJonesCutF']
class PairLennardJonesCutF(PairLennardJonesCut):
r"""class PairLennardJonesCutF(PairLennardJonesCut).
This class implements as simple Lennard-Jones 6-12 potential which
employs a simple cut-off and can be shifted. The potential energy
(:math:`V_\text{pot}`) is defined in the usual way for an
interacting pair of particles a distance :math:`r` apart,
.. math::
V_\text{pot} = 4 \varepsilon \left( x^{12} - x^{6} \right),
where :math:`x = \sigma/r` and :math:`\varepsilon`
and :math:`\sigma` are the potential parameters. The parameters are
stored as attributes of the potential and we store one set for each
kind of pair interaction. Parameters can be generated with a
specific mixing rule by the force field.
Attributes
----------
params : dict
The parameters for the potential. This dict is assumed to
contain parameters for pairs, i.e. for interactions.
_lj : dict of numpy.array
[1] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``48.0 * epsilon * sigma**12``
[2] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``24.0 * epsilon * sigma**6``
[3] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**12``
[4] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**6``
_offset : numpy.array
Potential values for shifting the potential if requested.
This is the potential evaluated at the cutoff.
_rcut2 : numpy.array
The squared cut-off for each interaction type.
Keys are the pairs (particle types) that may interact.
"""
def __init__(self, dim=3, shift=True, mixing='geometric',
desc='Lennard-Jones pair potential (FORTRAN)'):
"""Set up the Lennard-Jones potential.
Parameters
----------
dim : int
The dimensionality to use.
shift : boolean
Determines if the potential should be shifted or not.
mixing : string
Determines how we should mix potential parameters.
"""
super().__init__(dim=dim, desc=desc, mixing=mixing)
self.ntype = 0
def set_parameters(self, parameters):
"""Update all parameters.
Here, we generate pair interactions, since that is what this
potential actually is using.
Parameters
----------
parameters : dict
The input base parameters.
"""
self.params = {}
pair_param = generate_pair_interactions(parameters, self.mixing)
self.ntype = max(int(np.sqrt(len(pair_param))), 2)
self._lj = {'1': np.zeros((self.ntype, self.ntype)),
'2': np.zeros((self.ntype, self.ntype)),
'3': np.zeros((self.ntype, self.ntype)),
'4': np.zeros((self.ntype, self.ntype))}
self._rcut2 = np.zeros_like(self._lj[1])
self._offset = np.zeros_like(self._lj[1])
for pair in pair_param:
eps_ij = pair_param[pair]['epsilon']
sig_ij = pair_param[pair]['sigma']
rcut = pair_param[pair]['rcut']
self._lj[1][pair] = 48.0 * eps_ij * sig_ij**12
self._lj[2][pair] = 24.0 * eps_ij * sig_ij**6
self._lj[3][pair] = 4.0 * eps_ij * sig_ij**12
self._lj[4][pair] = 4.0 * eps_ij * sig_ij**6
self._rcut2[pair] = rcut**2
vcut = 0.0
if self.shift:
try:
vcut = 4.0 * eps_ij * ((sig_ij / rcut)**12 -
(sig_ij / rcut)**6)
except ZeroDivisionError:
vcut = 0.0
self._offset[pair] = vcut
self.params[pair] = pair_param[pair]
def potential(self, system):
"""Calculate the potential energy for the Lennard-Jones interaction.
Parameters
----------
system : object like :py:class:`.System`
The system we are evaluating the potential in.
Returns
-------
v_pot : float
The potential energy.
"""
particles = system.particles
box = system.box
v_pot = ljfortran.potential(particles.pos,
box.length, box.ilength,
self._lj, self._offset,
self._rcut2, particles.ptype,
particles.npart,
box.dim, self.ntype)
return v_pot
def force(self, system):
"""Calculate the force for the Lennard-Jones interaction.
We also calculate the virial here, since the force
is evaluated.
Parameters
----------
system : object like :py:class:`.System`
The system we are evaluating the force in.
Returns
-------
forces : numpy.array
The forces on the particles.
virial : numpy.array
The virial obtained from the forces.
"""
particles = system.particles
box = system.box
forces, virial = ljfortran.force(particles.pos,
box.length, box.ilength,
self._lj, self._rcut2,
particles.ptype,
particles.npart,
box.dim, self.ntype)
return forces, virial
def potential_and_force(self, system):
"""Calculate potential and force for the Lennard-Jones interaction.
Since the force is evaluated, the virial is also calculated.
Parameters
----------
system : object like :py:class:`.System`
The system we are evaluating the potential and force in.
Note
----
Currently, the virial is only calculated for all the particles.
It is not calculated as a virial per atom. The virial
per atom might be useful to obtain a local pressure or stress,
however, this needs some consideration. Perhaps it's best to
fully implement this as a method of planes or something similar.
Returns
-------
out[0] : float
The potential energy as a float.
out[1] : numpy.array
The force as a numpy.array of the same shape as the
positions in `particles.pos`.
out[2] : numpy.array
The virial, as a symmetric matrix with dimensions
(dim, dim) where dim is given by the box/system dimensions.
"""
particles = system.particles
box = system.box
forces, virial, vpot = ljfortran.potential_and_force(particles.pos,
box.length,
box.ilength,
self._lj,
self._offset,
self._rcut2,
particles.ptype,
particles.npart,
box.dim,
self.ntype)
return vpot, forces, virial
```
### Writing a new potential function with C
We will now create a new potential function by writing the external library in C. This is done by the following steps:
- Step 1. Creating the C code
- Step 2. Creating a setup.py file and compiling
- Step 3. Creating a new Python class for the potential function
#### Step 1. Creating the C code
The C code is more involved, and perhaps cumbersome compared to the FORTRAN code above. We are here explicitly assuming that our system is going to be 3D.
```c
/* Copyright (c) 2023, PyRETIS Development Team.
Distributed under the LGPLv2.1+ License. See LICENSE for more info. */
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include
#include
#include
// Forward function declaration.
static PyObject *potential(PyObject *self, PyObject *args);
static PyObject *force(PyObject *self, PyObject *args);
static PyObject *force_and_pot(PyObject *self, PyObject *args);
// Boilerplate: function list.
static PyMethodDef ljmethods[] = {
{ "potential", potential, METH_VARARGS, "Calculate potential energy."},
{ "force", force, METH_VARARGS, "Calculate force."},
{ "force_and_pot", force_and_pot, METH_VARARGS, "Calculate force and potential energy."},
{ NULL, NULL, 0, NULL } /* Sentinel */
};
static struct PyModuleDef ljc =
{
PyModuleDef_HEAD_INIT,
"ljc",
"Compute Lennard-Jones interactions",
-1,
ljmethods
};
// Boilerplate: Module initialization.
PyMODINIT_FUNC PyInit_ljc(void) {
import_array();
return PyModule_Create(&ljc);
}
static inline npy_float64 pbc_dist(npy_float64 x,
npy_float64 box,
npy_float64 ibox) {
return x - nearbyint(x * ibox) * box;
}
static PyObject *potential(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim, ntype;
npy_int64 itype, loc;
PyArrayObject *py_pos, *py_box, *py_ibox, *py_lj3, *py_lj4, *py_rcut2, *py_offset, *py_ptype;
// Internal variables:
npy_float64 dx, dy, dz, rsq, r2inv, r6inv;
npy_int64 i, xi, yi, zi;
npy_int64 j, xj, yj, zj;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!O!lll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_box,
&PyArray_Type, &py_ibox,
&PyArray_Type, &py_lj3,
&PyArray_Type, &py_lj4,
&PyArray_Type, &py_offset,
&PyArray_Type, &py_rcut2,
&PyArray_Type, &py_ptype,
&npart, &dim, &ntype)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *box = (npy_float64*)PyArray_DATA(py_box);
npy_float64 *ibox = (npy_float64*)PyArray_DATA(py_ibox);
npy_float64 *lj3 = (npy_float64*)PyArray_DATA(py_lj3);
npy_float64 *lj4 = (npy_float64*)PyArray_DATA(py_lj4);
npy_float64 *rcut2 = (npy_float64*)PyArray_DATA(py_rcut2);
npy_float64 *offset = (npy_float64*)PyArray_DATA(py_offset);
npy_int64 *ptype = (npy_int64*)PyArray_DATA(py_ptype);
npy_float64 vpot = 0.0;
// Calculate potential:
for(i = 0; i < npart-1; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
itype = ptype[i];
for (j = i + 1; j < npart; j++){
loc = itype + ntype * ptype[j];
xj = 3*j;
yj = 3*j + 1;
zj = 3*j + 2;
dx = pos[xi] - pos[xj];
dy = pos[yi] - pos[yj];
dz = pos[zi] - pos[zj];
dx = pbc_dist(dx, box[0], ibox[0]);
dy = pbc_dist(dy, box[1], ibox[1]);
dz = pbc_dist(dz, box[2], ibox[2]);
rsq = dx*dx + dy*dy + dz*dz;
if (rsq < rcut2[loc]){
r2inv = 1.0 / rsq;
r6inv = r2inv*r2inv*r2inv;
vpot += r6inv * (lj3[loc] * r6inv - lj4[loc]) - offset[loc];
}
}
}
/* Clean up */
/* Build the output tuple */
PyObject *ret = Py_BuildValue("d", vpot);
return ret;
}
static PyObject *force(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim, ntype;
npy_int64 itype, loc;
PyArrayObject *py_pos, *py_box, *py_ibox, *py_force, *py_virial;
PyArrayObject *py_lj1, *py_lj2, *py_rcut2, *py_ptype;
// Internal variables:
npy_float64 dx, dy, dz, rsq, r2inv, r6inv;
npy_float64 forcelj, fx, fy, fz;
npy_int64 i, xi, yi, zi;
npy_int64 j, xj, yj, zj;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!O!O!lll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_box,
&PyArray_Type, &py_ibox,
&PyArray_Type, &py_lj1,
&PyArray_Type, &py_lj2,
&PyArray_Type, &py_rcut2,
&PyArray_Type, &py_ptype,
&PyArray_Type, &py_force,
&PyArray_Type, &py_virial,
&npart, &dim, &ntype)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *force = (npy_float64*)PyArray_DATA(py_force);
npy_float64 *virial = (npy_float64*)PyArray_DATA(py_virial);
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *box = (npy_float64*)PyArray_DATA(py_box);
npy_float64 *ibox = (npy_float64*)PyArray_DATA(py_ibox);
npy_float64 *lj1 = (npy_float64*)PyArray_DATA(py_lj1);
npy_float64 *lj2 = (npy_float64*)PyArray_DATA(py_lj2);
npy_float64 *rcut2 = (npy_float64*)PyArray_DATA(py_rcut2);
npy_int64 *ptype = (npy_int64*)PyArray_DATA(py_ptype);
// Create array for forces:
//PyObject *ret = PyArray_SimpleNew(dim, test, NPY_DOUBLE);
// Zero virial:
for(i = 0; i < dim*dim; i++) {
virial[i] = 0;
}
// Set all forces to zero.
for(i = 0; i < npart; i++) {
force[3*i] = 0;
force[3*i+1] = 0;
force[3*i+2] = 0;
}
// Calculate forces:
for(i = 0; i < npart-1; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
itype = ptype[i];
for (j = i + 1; j < npart; j++){
loc = itype + ntype * ptype[j];
xj = 3*j;
yj = 3*j + 1;
zj = 3*j + 2;
dx = pos[xi] - pos[xj];
dy = pos[yi] - pos[yj];
dz = pos[zi] - pos[zj];
dx = pbc_dist(dx, box[0], ibox[0]);
dy = pbc_dist(dy, box[1], ibox[1]);
dz = pbc_dist(dz, box[2], ibox[2]);
rsq = dx*dx + dy*dy + dz*dz;
if (rsq < rcut2[loc]){
r2inv = 1.0 / rsq;
r6inv = r2inv*r2inv*r2inv;
forcelj = r2inv * r6inv * (lj1[loc] * r6inv - lj2[loc]);
fx = forcelj * dx;
fy = forcelj * dy;
fz = forcelj * dz;
force[xi] += fx;
force[yi] += fy;
force[zi] += fz;
force[xj] -= fx;
force[yj] -= fy;
force[zj] -= fz;
virial[0] += fx*dx;
virial[1] += fx*dy;
virial[2] += fx*dz;
virial[3] += fy*dx;
virial[4] += fy*dy;
virial[5] += fy*dz;
virial[6] += fz*dx;
virial[7] += fz*dy;
virial[8] += fz*dz;
}
}
}
Py_RETURN_NONE;
}
static PyObject *force_and_pot(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim, ntype;
npy_int64 itype, loc;
PyArrayObject *py_pos, *py_box, *py_ibox, *py_force, *py_virial;
PyArrayObject *py_lj1, *py_lj2, *py_lj3, *py_lj4, *py_rcut2, *py_offset, *py_ptype;
// Internal variables:
npy_float64 dx, dy, dz, rsq, r2inv, r6inv;
npy_float64 forcelj, fx, fy, fz;
npy_int64 i, xi, yi, zi;
npy_int64 j, xj, yj, zj;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!O!O!O!O!O!lll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_box,
&PyArray_Type, &py_ibox,
&PyArray_Type, &py_lj1,
&PyArray_Type, &py_lj2,
&PyArray_Type, &py_lj3,
&PyArray_Type, &py_lj4,
&PyArray_Type, &py_offset,
&PyArray_Type, &py_rcut2,
&PyArray_Type, &py_ptype,
&PyArray_Type, &py_force,
&PyArray_Type, &py_virial,
&npart, &dim, &ntype)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *force = (npy_float64*)PyArray_DATA(py_force);
npy_float64 *virial = (npy_float64*)PyArray_DATA(py_virial);
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *box = (npy_float64*)PyArray_DATA(py_box);
npy_float64 *ibox = (npy_float64*)PyArray_DATA(py_ibox);
npy_float64 *lj1 = (npy_float64*)PyArray_DATA(py_lj1);
npy_float64 *lj2 = (npy_float64*)PyArray_DATA(py_lj2);
npy_float64 *lj3 = (npy_float64*)PyArray_DATA(py_lj3);
npy_float64 *lj4 = (npy_float64*)PyArray_DATA(py_lj4);
npy_float64 *offset = (npy_float64*)PyArray_DATA(py_offset);
npy_float64 *rcut2 = (npy_float64*)PyArray_DATA(py_rcut2);
npy_int64 *ptype = (npy_int64*)PyArray_DATA(py_ptype);
npy_float64 vpot = 0.0;
// Create array for forces:
//PyObject *ret = PyArray_SimpleNew(dim, test, NPY_DOUBLE);
// Zero virial:
for(i = 0; i < dim*dim; i++) {
virial[i] = 0;
}
// Set all forces to zero.
for(i = 0; i < npart; i++) {
force[3*i] = 0;
force[3*i+1] = 0;
force[3*i+2] = 0;
}
// Calculate forces:
for(i = 0; i < npart-1; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
itype = ptype[i];
for (j = i + 1; j < npart; j++){
loc = itype + ntype * ptype[j];
xj = 3*j;
yj = 3*j + 1;
zj = 3*j + 2;
dx = pos[xi] - pos[xj];
dy = pos[yi] - pos[yj];
dz = pos[zi] - pos[zj];
dx = pbc_dist(dx, box[0], ibox[0]);
dy = pbc_dist(dy, box[1], ibox[1]);
dz = pbc_dist(dz, box[2], ibox[2]);
rsq = dx*dx + dy*dy + dz*dz;
if (rsq < rcut2[loc]){
r2inv = 1.0 / rsq;
r6inv = r2inv*r2inv*r2inv;
forcelj = r2inv * r6inv * (lj1[loc] * r6inv - lj2[loc]);
vpot += r6inv * (lj3[loc] * r6inv - lj4[loc]) - offset[loc];
fx = forcelj * dx;
fy = forcelj * dy;
fz = forcelj * dz;
force[xi] += fx;
force[yi] += fy;
force[zi] += fz;
force[xj] -= fx;
force[yj] -= fy;
force[zj] -= fz;
virial[0] += fx*dx;
virial[1] += fx*dy;
virial[2] += fx*dz;
virial[3] += fy*dx;
virial[4] += fy*dy;
virial[5] += fy*dz;
virial[6] += fz*dx;
virial[7] += fz*dy;
virial[8] += fz*dz;
}
}
}
PyObject *ret = Py_BuildValue("d", vpot);
return ret;
}
```
As can be seen in the C-code, there is some boilerplate code and we make use of PyArg_ParseTuple in order to parse the parameters to the function.
#### Step 2. Creating a setup.py file and compiling
The C-code can be compiled using a setup.py file.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""A script for building the C extension."""
from setuptools import (
setup,
Extension,
)
import numpy as np
LJMODULE = Extension(
'ljc',
sources=['ljc.c'],
extra_compile_args=['-Ofast', '-march=native'],
)
setup(
name='PyRETIS Lennard-Jones c extension',
description='C extension for the Lennard-Jones potential.',
ext_modules=[LJMODULE],
include_dirs=[np.get_include()],
)
```
The setup.py file is used to compile via the command
```pyretis
python setup.py build_ext --inplace
```
Here, build_ext is used to tell setup.py to compile the C extension and the --inplace will put the compiled extensions into the directory you have the source code in.
#### Step 3. Creating a new Python class for the potential function
The final step is to create a Python class which is making use of the C-code.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of using a Lennard-Jones potential implemented in C."""
import logging
import sys
from pathlib import Path
import numpy as np
from pyretis.forcefield.potentials import PairLennardJonesCut
from pyretis.forcefield.potentials.pairpotentials import (
generate_pair_interactions
)
# Just to handle imports of the library:
sys.path.insert(0, Path(__file__).parent.resolve())
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
try:
import ljc
except ImportError as exc:
MSG = ('Could not import external C library.'
'\nPlease compile with: "python setup.py build_ext --inplace"')
raise ImportError(MSG) from exc
__all__ = ['PairLennardJonesCutC']
class PairLennardJonesCutC(PairLennardJonesCut):
r"""class PairLennardJonesCutC(PairLennardJonesCut).
This class implements as simple Lennard-Jones 6-12 potential which
employs a simple cut-off and can be shifted. The potential energy
(:math:`V_\text{pot}`) is defined in the usual way for an
interacting pair of particles a distance :math:`r` apart,
.. math::
V_\text{pot} = 4 \varepsilon \left( x^{12} - x^{6} \right),
where :math:`x = \sigma/r` and :math:`\varepsilon`
and :math:`\sigma` are the potential parameters. The parameters are
stored as attributes of the potential and we store one set for each
kind of pair interaction. Parameters can be generated with a
specific mixing rule by the force field.
Attributes
----------
params : dict
The parameters for the potential. This dict is assumed to
contain parameters for pairs, i.e. for interactions.
_lj : dict of numpy.array
[1] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``48.0 * epsilon * sigma**12``
[2] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``24.0 * epsilon * sigma**6``
[3] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**12``
[4] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**6``
_offset : numpy.array
Potential values for shifting the potential if requested.
This is the potential evaluated at the cutoff.
_rcut2 : numpy.array
The squared cut-off for each interaction type.
Keys are the pairs (particle types) that may interact.
"""
def __init__(self, dim=3, shift=True, mixing='geometric',
desc='Lennard-Jones pair potential (C)'):
"""Set up the Lennard-Jones potential.
Parameters
----------
dim : int
The dimensionality to use.
shift : boolean
Determines if the potential should be shifted or not.
mixing : string
Determines how we should mix potential parameters.
"""
super().__init__(dim=dim, desc=desc, mixing=mixing)
self.shift = shift
self.ntype = 0
def set_parameters(self, parameters):
"""Update all parameters.
Here, we generate pair interactions, since that is what this
potential actually is using.
Parameters
----------
parameters : dict
The input base parameters.
"""
self.params = {}
pair_param = generate_pair_interactions(parameters, self.mixing)
self.ntype = max(int(np.sqrt(len(pair_param))), 2)
self._lj = {'1': np.zeros((self.ntype, self.ntype)),
'2': np.zeros((self.ntype, self.ntype)),
'3': np.zeros((self.ntype, self.ntype)),
'4': np.zeros((self.ntype, self.ntype))}
self._rcut2 = np.zeros_like(self._lj[1])
self._offset = np.zeros_like(self._lj[1])
for pair, value in pair_param.items():
eps_ij = value['epsilon']
sig_ij = value['sigma']
rcut = value['rcut']
self._lj[1][pair] = 48.0 * eps_ij * sig_ij**12
self._lj[2][pair] = 24.0 * eps_ij * sig_ij**6
self._lj[3][pair] = 4.0 * eps_ij * sig_ij**12
self._lj[4][pair] = 4.0 * eps_ij * sig_ij**6
self._rcut2[pair] = rcut**2
vcut = 0.0
if self.shift and rcut**22 > 0:
vcut = 4.0 * eps_ij * ((sig_ij / rcut)**12 -
(sig_ij / rcut)**6)
self._offset[pair] = vcut
self.params[pair] = pair_param[pair]
def potential(self, system):
"""Calculate the potential energy for the Lennard-Jones interaction.
Parameters
----------
system : object like :py:class:`.System`
The system we are evaluating the potential in.
Returns
-------
v_pot : float
The potential energy.
"""
particles = system.particles
box = system.box
v_pot = ljc.potential(particles.pos,
box.length, box.ilength,
self._lj, self._offset,
self._rcut2, particles.ptype,
particles.npart,
box.dim, self.ntype)
return v_pot
def force(self, system):
"""Calculate the force for the Lennard-Jones interaction.
We also calculate the virial here, since the force
is evaluated.
Parameters
----------
system : object like :py:class:`.System`
The system we are evaluating the force in.
Returns
-------
forces : numpy.array
The forces on the particles.
virial : numpy.array
The virial obtained from the forces.
"""
particles = system.particles
box = system.box
forces = np.zeros((particles.npart, box.dim))
virial = np.zeros((box.dim, box.dim))
ljc.force(particles.pos,
box.length, box.ilength,
self._lj, self._rcut2,
particles.ptype,
forces, virial,
particles.npart,
box.dim, self.ntype)
return forces, virial
def potential_and_force(self, system):
"""Calculate potential and force for the Lennard-Jones interaction.
Since the force is evaluated, the virial is also calculated.
Parameters
----------
system : object like :py:class:`.System`
The system we are evaluating the potential and force in.
Note
----
Currently, the virial is only calculated for all the particles.
It is not calculated as a virial per atom. The virial
per atom might be useful to obtain a local pressure or stress,
however, this needs some consideration. Perhaps it's best to
fully implement this as a method of planes or something similar.
Returns
-------
out[0] : float
The potential energy as a float.
out[1] : numpy.array
The force as a numpy.array of the same shape as the
positions in `particles.pos`.
out[2] : numpy.array
The virial, as a symmetric matrix with dimensions
(dim, dim) where dim is given by the box/system dimensions.
"""
particles = system.particles
box = system.box
forces = np.zeros((particles.npart, box.dim))
virial = np.zeros((box.dim, box.dim))
vpot = ljc.force_and_pot(particles.pos,
box.length,
box.ilength,
self._lj,
self._offset,
self._rcut2,
particles.ptype,
forces,
virial,
particles.npart,
box.dim,
self.ntype)
return vpot, forces, virial
```
## Running the MD simulation using the PyRETIS library
Below we give a Python script which makes use of the PyRETIS library to run a MD simulation of 2000 steps in the forward and backward directions. It will generate a trajectory (named traj.xyz) and display a plot of the energies (an example is shown in the figure below).
Before running the script you will have to download the initial configuration initial.gro and place it in the same directory as the script. Further, the script makes the following assumptions:
- If you are using FORTRAN (that is if USE = fortran) the script assumes that there is a folder named fortran which contains the compiled library and the Python script named ljpotentialf.py.
- If you are using C (that is if USE = cpython3) the script assumes that there is a folder named cpython3 which contains the compiled library and the Python script named ljpotentialc.py.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""
Example of running a MD NVE simulation.
Here we are running forward and then backward using a potential
written in FORTRAN or C. This can be selected below by the user.
"""
# pylint: disable=invalid-name
import sys
# For plotting:
from matplotlib import pyplot as plt
from matplotlib import gridspec
# PyRETIS imports:
from pyretis.core.units import create_conversion_factors
from pyretis.setup import create_simulation
from pyretis.inout.fileio import FileIO
from pyretis.inout.formats import ThermoTableFormatter
from pyretis.inout.plotting import mpl_set_style
# Define simulation settings:
klass = {'fortran': 'PairLennardJonesCutF',
'cpython3': 'PairLennardJonesCutC'}
module = {'fortran': 'fortran/ljpotentialf.py',
'cpython3': 'cpython3/ljpotentialc.py'}
USE = 'cpython3' # or 'fortran'
settings = {}
settings['simulation'] = {
'task': 'md-nve',
'steps': 2000,
}
settings['system'] = {
'units': 'lj',
'temperature': 2.5,
'dimensions': 3
}
settings['engine'] = {
'class': 'velocityverlet',
'timestep': 0.002
}
settings['output'] = {
'backup': 'overwrite',
'energy-file': 1,
'screen': 10,
'trajectory-file': 1
}
settings['potential'] = [
{'class': klass[USE],
'module': module[USE],
'parameter': {0: {'sigma': 1, 'epsilon': 1, 'factor': 2.5},
1: {'sigma': 1, 'epsilon': 1, 'factor': 2.5}},
'shift': True}
]
settings['particles'] = {
'position': {'input_file': 'initial.gro'},
'velocity': {'generate': 'maxwell', 'momentum': True, 'seed': 0}
}
create_conversion_factors(settings['system']['units'])
simulation_nve = create_simulation(settings)
# Set up extra output:
thermo_file = FileIO('thermo.txt', 'w', ThermoTableFormatter(), backup=False)
thermo_file.open()
store_results = []
# Also create some other outputs:
simulation_nve.set_up_output(settings, progress=False)
# Run the simulation forward:
for result in simulation_nve.run():
stepno = result['cycle']['stepno']
thermo_file.output(stepno, result['thermo'])
result['thermo']['stepno'] = stepno
store_results.append(result['thermo'])
# Run backward:
simulation_nve.ensemble['system'].vel *= -1.0
simulation_nve.extend_cycles(settings['simulation']['steps'] - 1)
for result in simulation_nve.run():
stepno = result['cycle']['stepno']
thermo_file.output(stepno, result['thermo'])
result['thermo']['stepno'] = stepno
store_results.append(result['thermo'])
thermo_file.close()
mpl_set_style() # Load PyRETIS plotting style
step = [res['stepno'] for res in store_results]
pot_e = [res['vpot'] for res in store_results]
kin_e = [res['ekin'] for res in store_results]
tot_e = [res['etot'] for res in store_results]
pressure = [res['press'] for res in store_results]
temp = [res['temp'] for res in store_results]
# Plot some energies:
fig1 = plt.figure()
gs = gridspec.GridSpec(2, 2)
ax1 = fig1.add_subplot(gs[:, 0])
ax1.plot(step, pot_e, label='Potential')
ax1.plot(step, kin_e, label='Kinetic')
ax1.plot(step, tot_e, label='Total')
ax1.set_xlabel('Step no.')
ax1.set_ylabel('Energy per particle')
ax1.legend(loc='center left', prop={'size': 'small'})
ax2 = fig1.add_subplot(gs[0, 1])
ax2.plot(step, temp)
ax2.set_ylabel('Temperature')
ax3 = fig1.add_subplot(gs[1, 1])
ax3.plot(step, pressure)
ax3.set_xlabel('Step no.')
ax3.set_ylabel('Pressure')
fig1.subplots_adjust(bottom=0.12, right=0.95, top=0.95, left=0.12, wspace=0.3)
if 'noplot' not in sys.argv[1:]:
plt.show()
```
Copy this script and store it in a new Python file, say md_forward_backward_py. Before running the script, you will have to set the USE variable in the script to either USE = fortran or USE = cpython3 to select the external library to use. After this has been done, you can execute the script as usual
```pyretis
python md_forward_backward.py
```
This will run the simulation, and when it is done, it should display a figure similar to the one given below. Further, you may wish to inspect the generated trajectory traj.xyz visually.
## Tested by
The internal MD behavior is represented by examples/tests/test-internal/compare-internal-with-lammps/. The compiled-potential pieces are advanced extension tutorials; keep their build instructions in sync with the extension checks before using them as release regression fixtures.
---
# Migration guide: legacy commands and inputs to the unified CLI
PyRETIS 4 unifies every entry point behind a single pyretis command and standardises on TOML input. The older commands and the .rst input format still work, each with a deprecation warning, and are scheduled for removal in PyRETIS 5. This page is the one-stop reference for moving an existing project forward. Nothing here needs to be done in a hurry: the deprecated paths keep running until v5.
Verification status: documentation only.
## Command migration
The single pyretis executable dispatches sub-commands (pyretis.bin.cli); the old standalone executables are kept as deprecated aliases that print a warning and then call the same code.
Legacy command
Replacement
Status
pyretisrun -i input.toml
pyretis run -i input.toml
deprecated
pyretisanalyse -i input.toml
pyretis analyse -i input.toml
deprecated
pyretis clean
pyretis tools clean
deprecated alias
(clean run artefacts)
pyretis tools clean
–
pyretis analyze (US spelling) is accepted as an alias of pyretis analyse. An infinite-swapping run is launched through the same pyretis run command and selected from the input (see the runner section below).
Running a deprecated executable prints, verbatim:
```text
"pyretisrun" is deprecated and will be removed in PyRETIS 5; use "pyretis run" instead.
```
(and the pyretisanalyse analogue). The message is emitted both as a Python DeprecationWarning and, when a run logger is active, at WARNING level so it is visible on screen and in the run log.
## infinite swapping: [runner]
The infinite-swapping (parallel replica-exchange) sampler is not a separate command – pyretis run routes an input to the scheduler when it either sets an infinite-swapping task or carries a [runner] section. The minimal addition that turns a single-worker RETIS input into a parallel run is:
```toml
[runner]
workers = 4
```
The keys, as translated by pyretis.inout.config_adapter.to_scheduler_config(), are:
- workers – the number of parallel MD worker processes (integer, default 1). The environment variable PYRETIS_WORKERS overrides it, for scripts that prefer not to edit the TOML.
- wmdrun – an optional list of per-worker MD-launch command overrides (one string per worker; used by external engines such as GROMACS for MPI/GPU pinning). Absent for the in-process engines.
The runner section reference documents the full behaviour; the infinite-swapping example is the worked walkthrough.
## Input-format migration: .rst to TOML
TOML is the canonical input format; the .rst frontend is deprecated and will be removed in PyRETIS 5. A legacy .rst path-sampling input (a retis, tis, explore, pptis or repptis task) is now translated and run automatically. pyretis run -i retis.rst writes a same-stem retis.toml next to the input, logs a warning, and runs that translated file through the infinite-swapping scheduler:
```text
The legacy input "retis.rst" was translated to "retis.toml" and the
run proceeds from the translated file (the .rst frontend is
deprecated). Use "-i retis.toml" directly next time.
```
An existing retis.toml twin is reused only when it matches the translation; a twin that differs aborts the run with a ValueError (run pyretis run -i retis.toml to use it, or delete it to regenerate). Molecular-dynamics .rst inputs (md, md-flux) are not auto-translated: they still run in place and only emit a deprecation warning pointing at the manual converter.
To convert a .rst ahead of time – or for the inputs that are not auto-translated – run the converter yourself:
```text
python -m pyretis.tools.convert_settings YOUR.rst
```
The converter writes YOUR.toml next to the input and verifies the conversion with a settings round-trip before trusting it. The mapping it applies:
- Each .rst section header becomes a TOML table; the table name is the first word of the header, lower-cased (Simulation -> [simulation]).
- Sections that may appear more than once – collective-variable, ensemble and potential – become TOML arrays-of-tables ([[ensemble]]). The settings parse of a TOML input of a path-sampling task (tis, retis, pptis, repptis, explore) refuses an [[ensemble]] section: the scheduler gives every ensemble the settings of the general sections, so the path-sampling settings go into [tis] and [retis] ([tis] shooting_moves gives one move per ensemble) and the interfaces into [simulation] interfaces. A make-tis-files input keeps its [[ensemble]] sections: it writes the settings of each one into the TIS input of its ensemble, tis-001.toml, ….
- Hyphenated keywords are quoted (order-file stays "order-file"); the decorative heading block is dropped.
## Input-schema migration: legacy-runner TOML to canonical
Some runner-style TOML inputs originally created for infRETIS use a different schema – a [runner] section and the path-sampling parameters under [simulation.tis_set], with no [simulation] task. This schema is deprecated: PyRETIS 4 converts such an input to its canonical form when it reads it, and PyRETIS 5 will not read it. pyretis run, pyretis analyse, the scheduler set-up of a programmatic run (pyretis.bin.pyretisrun.run_infinite_swapping()) and the interface optimiser convert it the same way, and each logs one deprecation notice, at WARNING level and as a DeprecationWarning:
```text
input.toml is an input of the legacy-runner schema ([runner],
[simulation.tis_set], no [simulation] task), which is deprecated:
PyRETIS converts it to the canonical schema when it reads it, and
PyRETIS 5 will not read it. Write its canonical form once with
'python -m pyretis.tools.convert_legacy_schema input.toml' and use
that file (docs/examples/examples-migration.rst gives the mapping).
```
The converter writes the canonical form to a file:
```text
python -m pyretis.tools.convert_legacy_schema input.toml
```
By default the file is converted in place (canonical syntax is still .toml); pass an explicit output path to write elsewhere, --force to overwrite, and --no-validate to skip the check.
The conversion is the same when PyRETIS reads the input and when the converter writes it (pyretis.tools.convert_legacy_schema.convert_legacy_document()). The input is reshaped into the canonical schema and parsed as every canonical input is, so a key that takes no effect on the run is refused, named in the canonical input and in the legacy-runner input, e.g. [tis] nullmoves ([simulation.tis_set] nullmoves of the legacy-runner input) = true. The conversion is then checked: the scheduler configuration of the input is compared with the one the key table builds from the canonical form, and a difference stops the conversion with a RuntimeError that names each setting that differs. The converter then leaves the output path as it was, and a run stops before it writes a file.
A run of a legacy-runner input (pyretis run -i input.toml) records the canonical form, with every default resolved, in output.toml of the run directory, and the scheduler records its state there with it, as for a run of a canonical input: no file the run writes holds the legacy shape. pyretis run -i output.toml resumes the run, pyretis analyse -i output.toml analyses it, and the canonical form with [initial-path] method = "restart" continues it. The run stops with a ValueError, before it writes anything, when the input file is that output.toml: rename the input file and run it under the new name.
The legacy-runner schema has no [initial-path] section: a run of a legacy-runner input starts from the initial paths staged in the run directory under [simulation] load_dir. The canonical form takes the initiation of the canonical input, [initial-path] method = "kick", which its record names. A new run of the canonical form therefore generates its initial paths, and stops before it writes a file when the run directory holds staged paths. A canonical input reads existing paths with [initial-path] method = "load", from a load_folder in one of the layouts of the load method, which recomputes and checks each path before the run.
A state file of the scheduler configuration that an earlier PyRETIS wrote (an output.toml or restart.toml with [simulation.tis_set] and [current]) is not an input, and is read as it is: pyretis run -i refuses it, the canonical input of the run with [initial-path] method = "restart" continues it, and pyretis analyse -i output.toml analyses it, reading its settings under the names of the input through the key table.
The reshaping (pyretis.inout.config_adapter.normalize_legacy_schema()):
legacy-runner
canonical
[simulation.tis_set] (the TIS parameters)
a top-level [tis] table
[simulation.tis_set] lambda_minus_one
[simulation] zero_left
[simulation.tis_set] permeability
[simulation] permeability
[simulation] shooting_moves
[tis] shooting_moves (one move per ensemble)
[simulation] zeroswap
[retis] swapfreq of a retis run; a tis or explore run, which attempts no swap, leaves it out with a warning
[simulation] relative_shoots
[retis] relative_shoots
(topology flags: explore / single_tis)
an inferred [simulation] task (default "retis")
[output] order_file / energy_file
order-file / energy-file (hyphenated)
[simulation] ensemble_engines and the engine sections it names
the same keys and sections (the keyword reference)
the [output] keys of the in-process output (backup, trajectory-file, …)
left out, with a warning
A few shapes are refused by the conversion with a ValueError rather than a silent mistranslation, because no in-repo fixture exercises their translation: the PPTIS/REPPTIS topology flags ([simulation] repptis, pptis_no_minus, pptis_no_zero_plus), whose canonical input is written by hand with task = "pptis" or "repptis", and a bare [simulation] noswap, whose message names the canonical tis task when one samples the same ensembles.
## See also
- The runner section reference – every [runner] keyword.
- The infinite-swapping example – a worked parallel run.
- examples-index – the full example index.
---
# A minimal input file
This is the shortest complete PyRETIS simulation: a RETIS run of a single particle in a double well, written with no optional settings at all. Every line of the input describes the physics of the problem. Everything else – the move set, how the first paths are made, the output frequencies, the path-length cap, the simulation box – is filled in by PyRETIS.
Start here if you want to see what PyRETIS genuinely needs to be told before you meet the full input description.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-double-well/minimal/.
- Edit first: minimal.toml.
- Run: pyretis run -i minimal.toml -p (about a minute).
- Analyse: pyretis analyse -i minimal.toml.
- Expected output: numbered ensemble folders (000, 001, …), pyretis.log, output.toml, and a report/ folder after the analysis.
## The whole input file
```toml
# The smallest input that runs a RETIS simulation. Every setting here
# describes the physics of the problem; everything else -- the move set,
# the initiation, the output frequencies, the path-length cap, the box --
# PyRETIS fills in with defaults (they are echoed to output.toml).
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "retis"
steps = 100
interfaces = [-0.9, -0.8, -0.5, 1.0]
[system]
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
[particles.position]
input_file = "initial.xyz"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
```
That is the complete input. The only other file the run needs is the starting configuration it points at, initial.xyz, which holds one particle at x = -1, the bottom of the left-hand well:
```
1
Ar -1.0 0.0 0.0
```
Run it with:
```pyretis
pyretis run -i minimal.toml -p
pyretis analyse -i minimal.toml
```
Note that the analysis reads the same input file as the run. The output.toml the run writes records that input, with every default resolved, so pyretis analyse -i output.toml gives the same analysis.
## What each section is for
Only a handful of things cannot be guessed, because each one is the problem you are posing:
The energy landscape: here the double well V(x) = a x^4 - b x^2 + c, which has two stable states near x = \pm 1 separated by a barrier at x = 0.
What to run – the retis task, how many Monte Carlo steps, and the interfaces defining the path ensembles. The first and last interface mark the two states; the ones between them slice up the barrier region.
That the system is one-dimensional, and the temperature it is sampled at. The temperature decides how rare the transition is, so it is a physical choice rather than a technical one.
How trajectories are propagated: Langevin dynamics with the given timestep and friction gamma.
Where the system starts, read from initial.xyz.
How progress along the reaction is measured – here simply the position of the particle, which is what the interfaces are placed on.
## What PyRETIS fills in
Everything a longer input would spell out is defaulted. The ones that matter most for this run:
## Settings this input leaves to PyRETIS.
Setting
Default used here
[initial-path] method
kick – the first paths are generated by kicking off the configuration you supply.
[tis] maxlength
20000 steps: a deliberately generous cap on how long a trajectory may become.
[tis] shooting_move
sh – ordinary one-point shooting.
[retis]
The standard swap and null-move frequencies.
[output]
Sensible file-writing frequencies.
[box]
A non-periodic box around the 1D system.
The keyword pages linked above ([tis], [retis], [output], [box], [initial-path]) document each default and when you would want to override it.
Every setting a run actually used is written to output.toml when the run starts, under the names of the input, so after a run you have the fully resolved version of the file you wrote. That is where to look when you want to know what a default resolved to, and what to copy from when you start pinning settings down explicitly.
Two of these defaults are physical rather than cosmetic, so it is worth knowing they are there.
maxlength caps how long a trajectory may become; paths that reach the cap are rejected, and a cap that is too small biases the sampling. That is why the default is generous rather than tight – but for a system with genuinely long paths you should still set it deliberately.
temperature defaults to 1.0 when it is left out. For a rare-event simulation that is almost never what you want, which is why this tutorial sets it explicitly.
This run is deliberately short (100 steps) so it finishes in about a minute. The rate constant it produces is therefore only indicative; raise steps for statistics you would report.
## Where to go next
- RETIS in a 1D potential builds the same simulation the long way, explaining each section as it is added – the natural next step once this run works.
- The input description lists every section and keyword together with its default.
- Analysing the output explains what lands in the report/ folder.
---
# A minimal input file with an external engine
The minimal input file tutorial uses PyRETIS’ own internal integrator, so the input has to describe the system: the potential, the dimensionality, the starting configuration.
With an external engine even that goes away. The engine already owns the system – its particles, force field, box and starting configuration live in the engine’s own input – so PyRETIS only needs to be told three things: what to sample, which engine to drive, and how to measure progress along the reaction.
This tutorial runs RETIS on two argon atoms with OpenMM. The whole simulation is defined by an OpenMM setup module plus the short input below.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/openmm/argon-minimal/.
- Requires: the openmm Python package.
- Edit first: minimal.toml; edit openmm_sim.py to change the physical system.
- Run: pyretis run -i minimal.toml -p (under a minute).
- Analyse: pyretis analyse -i minimal.toml.
## The input file
```toml
# The smallest input that runs a RETIS simulation with an EXTERNAL
# engine. With an external engine the system itself -- particles, force
# field, box, starting configuration -- is defined by the engine's own
# input (here openmm_sim.py), so PyRETIS only needs to be told what to
# sample, which engine to drive, and how to measure progress.
[simulation]
task = "retis"
steps = 20
interfaces = [0.3, 0.35, 0.4]
[engine]
class = "openmm"
openmm_simulation = "simulation"
openmm_module = "openmm_sim.py"
# The OpenMM integrator defines the time step; stating it here lets
# "pyretis analyse" turn the sampled crossings into a rate per unit
# time. The engine cross-checks it against the integrator, so a wrong
# value is an error rather than a silently wrong rate.
timestep = 0.002
[orderparameter]
class = "pyretis_distance"
index = [0, 1]
periodic = false
# The one initiation choice this system needs: build each ensemble's
# first path by kicking off the PREVIOUS ensemble's path. A single kick
# from the starting configuration cannot reach the outermost interface
# here, so the default ("initial") would not find a path for [1^+].
[initial-path]
kick-from = "previous"
```
## Where the system comes from
Nothing in that file describes argon. The system is built by openmm_sim.py, an ordinary OpenMM script that creates a Simulation object – two argon atoms, a Lennard-Jones force, an integrator, and the starting positions and velocities:
```python
# Minimal 2-Argon-atom Lennard-Jones simulation for PyRETIS OpenMM testing.
#
# This module is loaded by the PyRETIS OpenMM engine via the
# openmm_module / openmm_simulation settings in retis.toml.
# It defines `simulation`, an app.Simulation object that PyRETIS uses to
# propagate the system.
#
# The integrator seed is fixed so that results are reproducible across runs.
try:
import openmm as mm
from openmm import app, unit
except ImportError:
import simtk.openmm as mm # legacy namespace
from simtk.openmm import app
from simtk import unit
import numpy as np
# ── System: two Argon atoms ──────────────────────────────────────────────────
system = mm.System()
for _ in range(2):
system.addParticle(39.948 * unit.dalton)
# Lennard-Jones parameters for Ar (no electrostatics)
lj = mm.NonbondedForce()
lj.setNonbondedMethod(mm.NonbondedForce.NoCutoff)
for _ in range(2):
lj.addParticle(0.0,
0.34 * unit.nanometer,
0.997 * unit.kilojoule_per_mole)
system.addForce(lj)
# ── Topology ─────────────────────────────────────────────────────────────────
topology = app.Topology()
chain = topology.addChain()
residue = topology.addResidue('AR', chain)
argon = app.Element.getByAtomicNumber(18)
for atom_name in ('Ar1', 'Ar2'):
topology.addAtom(atom_name, argon, residue)
# ── Integrator & Simulation ──────────────────────────────────────────────────
integrator = mm.LangevinIntegrator(
300 * unit.kelvin,
1.0 / unit.picoseconds,
2.0 * unit.femtoseconds,
)
# Fixed seed ensures deterministic thermal noise during propagation steps.
integrator.setRandomNumberSeed(42)
simulation = app.Simulation(topology, system, integrator)
# Initial positions: atoms placed near the LJ minimum (~0.38 nm)
positions = np.array([[0.0, 0.0, 0.0],
[0.38, 0.0, 0.0]]) * unit.nanometer
simulation.context.setPositions(positions)
simulation.context.setVelocitiesToTemperature(300 * unit.kelvin, 42)
```
The two [engine] keys openmm_module and openmm_simulation are what connect the two: they name the file to import and the variable inside it holding the Simulation. PyRETIS then drives that object, which is why no [system], [particles], [potential], [forcefield] or [box] section appears in the input.
## What each section is for
What to sample: the retis task, how many Monte Carlo steps, and the interfaces – placed on the order parameter below, in the engine’s own units (nm here).
Which engine to drive and where its system comes from. The timestep is the OpenMM integrator’s step size; the engine cross-checks it against the integrator and raises if they disagree, and the analysis needs it to express the rate per unit time.
How progress is measured – here the distance between the two atoms, which is what the interfaces are placed on.
The one initiation choice this system needs. The default builds each ensemble’s first path by kicking off the starting configuration; for these interfaces a single kick cannot reach the outermost one, so kick-from = "previous" builds the ensembles up in turn instead.
Everything else is defaulted exactly as in the internal-engine tutorial: the move set, the path-length cap, the output frequencies, the RETIS swap frequencies.
## Using a different external engine
The same shape applies to the other external engines – GROMACS, LAMMPS and CP2K. Those read their system from an input directory rather than a Python module, so instead of openmm_module they take an input_path pointing at the engine’s own input files. The rest of the input – what to sample, how to measure it – stays exactly as above. See the engine section for the keys each engine takes.
## Where to go next
- A minimal input file – the same idea with PyRETIS’ internal engine, where the input does define the system.
- The engine section documents every engine and its settings.
---
# Running a PyRETIS simulation with OpenMM
In this example, we show the interface between OpenMM and PyRETIS.
Verification status: engine – see example-tutorial-map.
## Tutorial quick start
- Best starting point: the OpenMM fixture under examples/tests/test-openmm/ and the files included below.
- Requirements: openmm importable in Python and an OpenMM Simulation object defined in a Python module.
- Edit first: the openmm_module and openmm_simulation settings in the [engine] section.
- Run: pyretis run -i retis.toml -p.
- Expected output: standard RETIS output folders, with propagation delegated to OpenMM.
- Related check: examples/tests/test-openmm/; see example-test-status.
First, we need to generate a file with an OpenMM Simulation object. We use the online tool OpenMM Script Builder to generate the example with this PDB file containing two water molecules.
```
from openmm import app
import openmm as mm
from openmm import unit
pdb = app.PDBFile('input.pdb')
forcefield = app.ForceField('amber99sbildn.xml', 'tip3p.xml')
system = forcefield.createSystem(pdb.topology, nonbondedMethod=app.PME,
nonbondedCutoff=1.0*unit.nanometers,
constraints=app.HBonds, rigidWater=True,
ewaldErrorTolerance=0.0005)
integrator = mm.LangevinIntegrator(300*unit.kelvin,
1.0/unit.picoseconds,
2.0*unit.femtoseconds)
integrator.setConstraintTolerance(0.00001)
# Check with "python -m simtk.testInstallation" which platforms are available
# Here I use CUDA, but CPU is more commonly available
# platform = mm.Platform.getPlatformByName('CPU')
platform = mm.Platform.getPlatformByName('CUDA')
simulation = app.Simulation(pdb.topology, system, integrator, platform)
simulation.context.setPositions(pdb.positions)
```
With this simulation object, we can now construct the required .toml options in the engine section:
```rst
Engine
------
type = openmm
class = openmm
openmm_simulation = simulation
openmm_module = openmm_sim.py
subcycles = 10
```
Here we say that:
- The engine type is openmm, which tells PyRETIS that it has a special system representation.
- The engine class is also openmm to tell PyRETIS we want to use the OpenMM engine.
- The openmm_module is the module from which we want to load the simulation. Here it is openmm_sim.py.
- The openmm_simulation is the name of the object in that file. In this case it is simulation.
Finally, we give the number of subcycles, which indicates how many MD steps OpenMM performs before PyRETIS requests another frame. This value should be relatively high when using GPUs, as it determines how often PyRETIS communicates with the GPU. However, keep in mind that this also lowers the time resolution of your PyRETIS simulation.
After this, the setup can be run as a regular PyRETIS simulation using pyretis run:
```pyretis
pyretis run -i retis.toml -p
```
## Tested by
OpenMM coverage is tracked through examples/tests/test-openmm/. Developer runs are optional because they depend on a local OpenMM installation; tagged releases require the recorded 8.5.1 source revision and the strict golden comparison. See example-test-status for the current run status before treating this example as verified on a given platform.
---
# Studying permeability with PyRETIS
This example shows how to set up a permeability simulation with PyRETIS. For further details on the derivation of the formulas and description of the Monte Carlo moves, please read and cite the permeability from (RE)TIS paper 1.
This example uses three non-interacting particles on a flat potential. The runnable inputs are bundled under examples/tutorials/path_sampling/internal/permeability/ and consist of retis_perm.toml, flat_potential.py, and initial.xyz.
Verification status: smoke – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/permeability/.
- Edit first: the simulation section for permeability, zero_left, and interfaces; the orderparameter section for the tracked particle and mirror position.
- Run: pyretis run -i retis_perm.toml -p.
- Analyse: pyretis analyse -i retis_perm.toml after the run.
- Expected output: standard RETIS output plus permeability-specific analysis values in report/.
- Related checks: no dedicated heavy-test fixture yet; use this page as the setup reference and example-test-status for current coverage.
## New simulation settings
The simulation section has a few extra options:
```toml
[simulation]
task = "retis"
steps = 50
interfaces = [
-0.1,
0.0,
0.1,
]
zero_left = -0.2
permeability = true
```
Here we have:
- zero_left: Tells PyRETIS that the [0^-] ensemble has a left boundary that is not located at -inf.
- permeability: If True, any path in the [0^-] ensemble that starts and ends at one of the interfaces is accepted. If False, any path that hits the zero_left interface will be rejected, leading to incorrect flux calculations. This option also triggers pyretis analyse to calculate \xi, \frac{\tau}{dz} and the permeability.
The mirror and target-swap moves change the state of the order function: the molecule it tracks and whether its coordinate is mirrored. That state belongs to the path the move produced. Every path records the state it was measured with, each move measures its new frames in the state of the path it continues, and a swap carries the state along with the path into its new ensemble, so all frames of a stored path are measured on one molecule and on one side of the mirror. A path that records no state, such as a loaded initial path, was measured in the configured state: the orderparameter index, unmirrored. The states of the live paths are saved in output.toml under [current] order_state, and a restart gives each path its own state back. No setting is needed for this; the PyRETIS 3 keyword swap_attributes is accepted, has no effect, and logs a warning.
## New TIS settings
The new mirror and target-swap moves add a few options to the tis section:
```toml
mirror_freq = 0.1
target_freq = 0.1
target_indices = [
0,
1,
2,
]
```
Here we have:
- mirror_freq: The probability of attempting the mirror move in the [0^-] ensemble.
- target_freq: The probability of attempting the target-swap move in the [0^-] ensemble.
- target_indices: A list of atom indices. The target swap is only attempted between these atoms. Make sure that the original orderparameter.index is included in this list.
## New order parameter class
This simulation can be run using the new order parameter class pyretis_permeability. This class is a subclass of the pyretis_position order parameter, but alters the output depending on mirror_pos, relative and offset.
```toml
[orderparameter]
class = "pyretis_permeability"
dim = "x"
index = 0
offset = 0
relative = false
mirror_pos = -0.15
```
Here we have:
- dim: The same as for the class Position.
- index: The index of the particle that will be tracked at the start of the simulation. This attribute will be changed by the target-swap move.
- offset: This order parameter adds an offset to the value of compute_s() before wrapping it into the periodic box. This alters the resulting OP value, but all values will fall within the box vectors. If you want to alter the box vectors instead and do not have access to them, you can use pyretis_permeabilityminusoffset, which subtracts the offset after wrapping, before returning the value.
- relative: If True, the output is mapped relative to the box vector (between 0 and 1). Both offset and mirror_pos should be defined relative to this box vector as well.
- mirror_pos: The position of the mirror plane, before applying the offset. For the current implementation, this must be set halfway between the 0-R and 0-L interfaces.
The permeability classes call the function compute_s() before applying the offset and mirror. For the base class this calls the compute function of Position. To use this with your own OP, you can make a subclass of Permeability (the Python class) and override the self.compute_s() function to return your own custom OP before applying the offset and mirroring.
## Output of the new moves
The new moves also lead to some new possible responses in moves.txt, the record of every attempted move. For the mirror move, which is always accepted subject to the constraint on mirror_pos, this is just a new move type called mr.
For the target_swap move:
- A new generated label ts to indicate target-swap.
- A new rejection reason: TSS, which means there are no valid indices to swap to.
- Another new rejection reason TSA, which is a rejection based on the Monte Carlo acceptance.
Another changed response is BTS (backward too short), which is a more common rejection for the [0^-] <-> [0^+] swap. This indicates that the attempted trajectory in [0^-] ended at the L interface, so we do not attempt to extend it into the [0^+] ensemble.
## New analysis options
Adding permeability = true to the simulation settings triggers pyretis analyse to also calculate and plot \xi, \frac{\tau}{dz} and the permeability. This follows the formulas described in the paper 1. For the calculation of \tau to work, a reference region has to be chosen. This is done by adding tau_ref_bin to the Analysis section of the TOML file.
```toml
[analysis]
tau_ref_bin = [
-0.175,
-0.125,
]
```
This value can be changed in retis_perm.toml and the analysis rerun with another reference region. The analysis also plots a 10-bin histogram of the [0^-] region to help the user select a flat histogram region in this space.
## Tested by
The reference run examples/tests/test-internal/permeability/ runs this set-up and compares its output files against a committed reference. test/integration/test_permeability_order_state.py runs the target-swap and mirror moves at a high rate on the input in examples/tests/test-internal/permeability/targetswap-validation/ and checks that every stored path is measured on one molecule and on one side of the mirror, and that a run restarted part-way matches an uninterrupted one.
For slow permeants where even RETIS struggles to reach the permeation timescale, the partial-path variant REPPTIS combines short paths with replica exchange; see the REPPTIS example 2 and its Markov-state-model kinetics analysis 3.
## References
A. Ghysels, S. Roet, S. Davoudi and T. S. van Erp, Exact non-Markovian permeability from rare event simulations, Phys. Rev. Research 3, 033068 (2021), https://doi.org/10.1103/PhysRevResearch.3.033068
W. Vervust, D. T. Zhang, T. S. van Erp and A. Ghysels, Path sampling with memory reduction and replica exchange to reach long permeation timescales, Biophys. J. 122, 2960-2972 (2023), https://doi.org/10.1016/j.bpj.2023.02.021
W. Vervust, E. Wils, S. Safaei, D. T. Zhang and A. Ghysels, Estimating full path lengths and kinetics from partial path transition interface sampling simulations, J. Chem. Theory Comput. (2026), https://doi.org/10.1021/acs.jctc.5c01498 (arXiv:2602.12835).
---
# Particle Swarm Optimization
In this example, we will perform a task that PyRETIS is NOT intended to do. We will optimize a function using a method called particle swarm optimization and the purpose of this example is to illustrate how the PyRETIS library can be used to set up special simulations.
The function we will optimize is the Ackley function which is relatively complex with many local minima as illustrated in the figure below. We will set up our optimization by first creating a new potential and a new engine to do the job for us.
Verification status: smoke – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/other/particle_swarm/.
- Edit first: ackley.py for the potential and psoengine.py for the custom engine; pso_run.py wires them together.
- Run: python pso_run.py.
- Expected output: printed optimization progress and, if plotting is enabled, figures or animation frames for the swarm trajectory.
- Related checks: no dedicated heavy-test fixture yet; this is an advanced API example rather than a standard path-sampling workflow. See example-test-status.
## Creating the Ackley function as a PotentialFunction
Here, we will create the function we will optimize as a PotentialFunction. This can be done by creating a new file ackley.py and adding the following code:
```python
import numpy as np
from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # pylint: disable=unused-import
from pyretis.forcefield import PotentialFunction
TWO_PI = np.pi * 2.0
EXP = np.exp(1)
@np.vectorize
def ackley_potential(x, y): # pylint: disable=invalid-name
"""Evaluate the Ackley function."""
return (-20.0 * np.exp(-0.2*np.sqrt(0.5*(x**2 + y**2))) -
np.exp(0.5 * (np.cos(TWO_PI * x) + np.cos(TWO_PI * y))) +
EXP + 20)
class Ackley(PotentialFunction):
"""A implementation of the Ackley function.
Note that the usage of this potential function differs from
the usual usage for force fields.
"""
def __init__(self):
"""Set up the function."""
super().__init__(dim=2, desc='The Ackley function')
def potential(self, system):
"""Evaluate the potential, note that we return all values."""
xpos = system.pos[:, 0]
ypos = system.pos[:, 1]
pot = ackley_potential(xpos, ypos)
return pot
```
If you add:
```python
def main():
"""Plot the Ackley function."""
xgrid, ygrid = np.meshgrid(np.linspace(-5, 5, 100),
np.linspace(-5, 5, 100))
zgrid = ackley_potential(xgrid, ygrid)
fig = plt.figure()
ax1 = fig.add_subplot(211)
ax2 = fig.add_subplot(212, projection='3d')
ax1.contourf(xgrid, ygrid, zgrid)
ax2.plot_surface(xgrid, ygrid, zgrid, cmap=plt.get_cmap('viridis'))
plt.show()
if __name__ == '__main__':
main()
```
you can also plot the potential by running:
```pyretis
python ackley.py
```
## Creating a custom engine for particle swarm optimization
Here, we will create a new engine for performing the “dynamics” in the particle swarm optimization. The equations of motion are
- For the velocity v_i of particle i:
- where \omega is the co-called inertia weight (a parameter), c_1 and c_2 are acceleration coefficients (parameters), r_1 and r_2 are random numbers drawn from a uniform distribution between 0 and 1, x_i^\ast is particle i’s best estimate of the minimum of the potential and x^\dagger is the global best estimate.
- For the position x_i of particle i:
In both equations, t is the current step and t+1 is the next step. Before updating the positions, the potential energies for the individual particles are obtained and x_i^\ast and x^\dagger are updated.
These equations are similar to the equations used by the MD integrators in PyRETIS, and the engine can be implemented as a sub-class of the pyretis.engines.internal.MDEngine class. Create a new file name psoengine.py and add the following code:
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""A custom engine for particle swarm optimization."""
import numpy as np
from pyretis.engines import MDEngine
class PSOEngine(MDEngine):
"""Perform particle swarm optimization."""
def __init__(self, inertia, accp, accg):
"""Set up the engine.
Parameters
----------
intertia : float
The intertia factor in the velocity equation
of motion.
accp : float
The acceleration for the previous best term. "The congnitive term".
accg : float
The acceleration for the global best term. "The social term".
"""
super().__init__(1, 'Particle Swarm Optimization')
self.inertia = inertia
self.accp = accp
self.accg = accg
self.pbest = None
self.pbest_pot = None
self.gbest = None
self.gbest_pot = None
def integration_step(self, system):
"""Perform one step for the PSO algorithm."""
particles = system.particles
if self.pbest is None:
self.pbest = np.copy(particles.pos)
self.pbest_pot = system.potential()
if self.gbest is None:
pot = system.potential()
idx = np.argmin(pot)
self.gbest = particles.pos[idx]
self.gbest_pot = pot[idx]
rnd1 = np.random.uniform()
rnd2 = np.random.uniform()
particles.vel = (self.inertia * particles.vel +
rnd1 * self.accp * (self.pbest - particles.pos) +
rnd2 * self.accg * (self.gbest - particles.pos))
particles.pos += particles.vel
particles.pos = system.box.pbc_wrap(particles.pos)
pot = system.potential()
# Update global?
idx = np.argmin(pot)
if pot[idx] < self.gbest_pot:
self.gbest_pot = pot[idx]
self.gbest = particles.pos[idx]
# Update for individuals:
idx = np.where(pot < self.pbest_pot)[0]
self.pbest[idx] = np.copy(particles.pos[idx])
self.pbest_pot[idx] = pot[idx]
return self.gbest_pot, self.gbest
```
## Putting it all together and running the optimization
We will now create a simulation for performing the optimization. First, we need to import the new potential function and the new engine we have created:
```python
import numpy as np
from pyretis.core import create_box, Particles, System
from pyretis.simulation import Simulation
from pyretis.forcefield import ForceField
from psoengine import PSOEngine
from ackley import Ackley, ackley_potential
```
We next use this to define a method for setting everything up for us:
```python
NPART = 10
STEPS = 1000
MINX, MAXX = -10, 10
TXT = 'Step: {:5d}: Best: (x, y) = ({:10.3e}, {:10.3e}), pot = {:10.3e}'
def set_up():
"""Just create system and simulation."""
box = create_box(low=[MINX, MINX], high=[MAXX, MAXX],
periodic=[True, True])
print('Created a box:')
print(box)
print(f'Creating system with {NPART} particles')
system = System(units='reduced', box=box)
system.particles = Particles(dim=2)
for _ in range(NPART):
pos = np.random.uniform(low=MINX, high=MAXX, size=(1, 2))
system.add_particle(pos)
ffield = ForceField('Single Ackley function',
potential=[Ackley()])
system.forcefield = ffield
print(f'Force field is:\n{system.forcefield}')
print('Creating simulation:')
engine = PSOEngine(0.7, 1.5, 1.5)
simulation = Simulation(settings=None, controls={'steps': STEPS})
task_integrate = {'func': engine.integration_step,
'args': [system],
'result': 'gbest', 'first': True}
simulation.add_task(task_integrate)
return simulation, system
```
Finally, we can make a method to execute the optimization:
```python
def main():
"""Just run the optimization, no plotting."""
simulation, _ = set_up()
for result in simulation.run():
step = result['cycle']['step']
best = result['gbest']
if step % 10 == 0:
print(TXT.format(step, best[1][0], best[1][1], best[0]))
```
Which is used as follows:
```python
if __name__ == '__main__':
main()
```
Execute the script a couple of time (save the code above in a new file, say pso_run.py) and execute it using:
```pyretis
python pso_run.py
```
## Adding plotting and animation
If you wish, you can also animate the results/optimization process. First modify the imports as follows:
```python
import numpy as np
from pyretis.core import create_box, Particles, System
from pyretis.simulation import Simulation
from pyretis.forcefield import ForceField
from psoengine import PSOEngine
from ackley import Ackley, ackley_potential
from matplotlib import pyplot as plt
from matplotlib import animation, cm
```
And add the following methods:
```python
def evaluate_potential_grid():
"""Evaluate the Ackley potential on a grid."""
X, Y = np.meshgrid(np.linspace(MINX, MAXX, 100),
np.linspace(MINX, MAXX, 100))
Z = ackley_potential(X, Y)
return X, Y, Z
def update_animation(frame, system, simulation, scatter):
"""Update animation."""
patches = []
if not simulation.is_finished() and frame > 0:
results = simulation.step()
best = results['gbest']
if frame % 10 == 0:
print(TXT.format(frame, best[1][0], best[1][1], best[0]))
scatter.set_offsets(system.pos)
patches.append(scatter)
return patches
def main_animation():
"""Run the simulation and update for animation."""
simulation, system = set_up()
fig = plt.figure()
ax1 = fig.add_subplot(111, aspect='equal')
ax1.set_xlim((MINX, MAXX))
ax1.set_ylim((MINX, MAXX))
ax1.axes.get_xaxis().set_visible(False)
ax1.axes.get_yaxis().set_visible(False)
X, Y, pot = evaluate_potential_grid()
ax1.contourf(X, Y, pot, cmap=cm.viridis, zorder=1)
scatter = ax1.scatter(system.pos[:, 0],
system.pos[:, 1], marker='o', s=50,
edgecolor='#262626', facecolor='white')
def init():
"""Just return what to re-draw."""
return [scatter]
# This will run the animation/simulation:
anim = animation.FuncAnimation(fig, update_animation,
frames=STEPS+1,
fargs=[system, simulation, scatter],
repeat=False, interval=30, blit=True,
init_func=init)
plt.show()
return anim
if __name__ == '__main__':
main_animation()
```
## Tested by
The particle-swarm example is a library-extension tutorial and is not currently part of test-heavy.sh. Use it to learn how a custom potential and engine fit into the PyRETIS API; add a small fixture under examples/tests/ before relying on it as a regression check.
---
# PyVisA: Visualization and analysis of path sampling results
In this example, we become familiar with PyVisA. Every example on the website can be used to generate data for the analysis.
PyVisA is composed of two units: a compressor tool and a visualization tool. The visualization tool requires the additional installation of the optional GUI requirements. Both units can be executed directly with the pyvisa command or used as a Python library.
Here, we illustrate the command-line usage of the PyVisA components.
Since clusters might not support GUIs or the related packages, the GUI requirements are not prerequisites for PyRETIS or command-line PyVisA use. To operate the PyVisA GUI visualizer, these packages must therefore be installed manually, for example with pip.
Verification status: engine – see example-tutorial-map.
## Tutorial quick start
- Best starting point: any completed path-sampling tutorial with an out.rst file; the GROMACS hydrate tutorial is a common example.
- Requirements: PyQt5 is required only for the GUI visualizer, not for compression.
- Compress: pyvisa -i retis.toml -cmp.
- Open the GUI: pyvisa -i retis.toml or pyvisa -i data.hdf5.
- Expected output: compressed .hdf5 data and plots selected in the GUI.
- Related check: examples/tests/test-pyvisa/; see example-test-status.
## Compressor
The compressor tool can be executed with:
```pyretis
pyvisa -i -cmp
```
where is the same .toml file used to execute the PyRETIS simulations.
PyVisA will read, for each folder, the energy and the order parameter files. It will check the consistency and integrity of each file and discard corrupted data. The compressor tool will then generate pyvisa_compressed_data.hdf5.zip, where all the input files are condensed, compressed, and organized to simplify the following post-processing operations.
The compressor tool reports file inconsistencies, such as corrupted files or mismatched cycles. This situation might happen when the simulation data have been produced by multiple machines and/or by multiple independent runs.
The source files are not touched by this operation. If the data consistency is very low, a manual check is required before importing the data.
## Remote/headless workflow
On clusters or remote machines where the GUI is not available, run the compressor from the command line and move the compressed file to a machine with the GUI requirements installed:
```pyretis
pyvisa -i -cmp
```
This creates pyvisa_compressed_data.hdf5.zip in the simulation directory. Copy that file to the local machine and open it with:
```pyretis
pyvisa -i pyvisa_compressed_data.hdf5.zip
```
The same remote workflow can be used after recalculating order parameters or collective variables:
```pyretis
pyvisa -i -recalculate
pyvisa -i -cmp
```
For more details on remote use, direct GUI use, and the -data option, see user-guide-pyvisa.
## Visualization
The visualization tool can be executed with:
```pyretis
pyvisa -i
```
where can either be the .toml PyRETIS input file or the compressed file generated by the compressor tool, pyvisa_compressed_data.hdf5.zip by default.
The PyVisA GUI has two main panels: data and plotting.
The plot types are defined by:
- Plot type: select the Matplotlib plot type and the number of dimensions to use. When plotting density maps, path weighting of accepted trajectories is an option.
## Data sub-sets selection
Data selection and manipulation criteria are:
- x, y, z: list of order parameters, cycles and timesteps. Also, y, z allows the selection of the kinetic, potential, and total energy (kinetic+potential).
- Folder: choose an ensemble number or select all of them.
- Paths: accepted (ACC), rejected (REJ), or both (BOTH). See user-guide-path-status-codes for all status codes.
- Cycles: select the minimum and maximum cycle number, where a cycle is a Monte Carlo move, i.e. a trajectory for each ensemble.
- Data shift: shift the x-y data. This can be useful in the case of a cyclic order parameter such as an angle or dihedral.
- Interfaces: toggle interface lines (2D plot) or planes (3D plot) It requires that x is the main order parameter (OP1).
- Reg. line: plot a linear regression line and report its slope, intercept and r–squared values.
- Resolution: number of pixels or grid–points to use for density, surface plot types. For scatter plots, it controls the dot size.
## Plotting
From the GUI, without reloading the data, it is possible to manipulate the picture. The options are:
- Colormap: the colormaps to use for the plot.
- xmin/xmax: minimum and maximum x–values in the plot.
- ymin/ymax: minimum and maximum y–values in the plot.
- Save: save the figure in a .png file.
- Font size, Titles/Axes: the font size of plot titles and axis labels.
- Show titles/labels: display the plot titles and axis labels.
## Data handling
Further options can be accessed from the drop-down panel labeled **File**.
The drop-down menu contains a few options:
- Data Loading: reload the compressed simulation data.
- Data Saving: save the simulation data, for example with the compressor tool, in .hdf5 format.
- Figure save: save the current data selection/picture. Several options are here available. To further facilitate data handling, PyVisA can save the selected data in very different ways. The idea is to minimize user efforts in data manipulation. In all the following selections, the file name is automatically generated to contain all the information for (manually) reconstructing the plot.
- The figure’s data can be saved in:
- Raw format: .txt file. Other visualization software can directly be used (e.g. xmgrace, gnuplot).
- JSON format: .json file. This allows users to directly access the numbers corresponding to a plot and/or load them via the JSON package.
- The figure’s object can be saved in:
- .hdf5: a versatile compressed format that can be loaded also by other programming languages (e.g. R).
- The figure itself can be saved as:
- figure.png: a .png file.
- script.py: a Python program is generated to reproduce the selected plot from the compressed data simply by typing python .
A large variety of plots can thus be generated via PyVisA. The respective data can be saved in different formats to further facilitate post-processing and analysis in different programming languages.
## Tested by
The command-line compression and analysis helpers are represented by examples/tests/test-pyvisa/. GUI behavior can depend on local Qt availability, so use the heavy check as the automated target and this page as the interactive guide.
---
# Post-processing and visualization with PyVisA
In this example, we perform post-processing on the methane hydrate system from one of the previous examples called “Using GROMACS”. All post-processing is done with the PyVisA analysis tab, shown in fig_analysis_tab, and the recalculation tool. We will continue the study of cage-to-cage diffusion within a S1 hydrate and add new collective variables. Then we perform clustering, principal component analysis (PCA), and visualization of the new results. These analysis methods require the scikit-learn and scipy packages.
Verification status: engine – see example-tutorial-map.
## Tutorial quick start
- Best starting point: first complete examples-gromacs-hydrate or another path-sampling tutorial that produces an out.rst file and trajectories.
- Requirements: scikit-learn and scipy for clustering and PCA, plus PyQt5 for the GUI.
- Compress original data: pyvisa -i retis.toml -cmp.
- Recalculate added variables: edit retis.rst and orderp.py, then run pyvisa -i retis.toml -recalculate.
- Expected output: compressed HDF5 data, recalculated collective variables, and clustering/PCA data saved by PyVisA.
- Related check: examples/tests/test-pyvisa/; see example-test-status.
Before we proceed with the analysis, we need to have some simulation results, so please run the example Using GROMACS first.
The example is structured as follows:
## Visualization and compression
When the simulation has finished, the results can be visualized and post-processed. It is always important to save the original data, because the recalculation creates new PyVisA data from the trajectory files of the old simulation. This loses the frames between stored trajectory files. The following steps are suggested:
- Compress the original data into an HDF5 file using:
```pyretis
pyvisa -i retis.toml -cmp
```
- Compress only the order parameter data into an .hdf5.zip file using:
```pyretis
pyvisa -i retis.toml -cmp -oo
```
- Rename these files to standard_simulation.hdf5.zip.
The results can be visualized from the .toml files or from the compressed files. If you want to visualize the data, the following commands can be run:
```pyretis
pyvisa -i
```
```pyretis
pyvisa -i -data
```
where can be either a .toml file or an .hdf5.zip file. If you are using the -data command, then can be all to load all data, a trajectory file, or a simulation directory. For example, the first path ensemble can be loaded by using 000 as .
With PyVisA, you can visualize trajectories from all path ensembles and sort them by status and Monte Carlo move. Try finding and visualizing a reactive pathway by clicking on the plot and pressing “Show trj” in the Analysis tab. This highlights the points that belong to the selected trajectory. You can also customize how the visualization looks. In fig_dens_reactive a reactive pathway has been shown.
## Recalculation of new collective variables
Now that the old data has been stored, we can add two more collective variables to the simulation using PyVisA. The collective variables we introduce are the area of the six-membered ring that the methane jumps through, and the volume of the starting cage. These descriptors will try to capture the breathing of the starting cage. The following lines must be added to the retis.rst file:
```rst
Collective-variable
-------------------
class = AreaAndVolume
module = orderp.py
```
With the collective variable added to retis.rst, we also need to add the Python script for the calculation in the orderp.py file. The scipy package is needed to run this script, so make sure that it is installed. The full script is given here:
```python
import logging
import numpy as np
from scipy.spatial import ConvexHull
from pyretis.orderparameter import OrderParameter
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
class AreaAndVolume(OrderParameter):
"""AreaAndVolume(OrderParameter).
This order parameter calculates the area of the six-membered ring
which the methane molecule jumps through when performing the L6L jump,
and the volume of the starting cage.
Attributes:
----------
periodic : boolean
This determines if periodic boundaries should be applied to
the position or not.
"""
def __init__(self, periodic=True):
"""Initialize the class."""
super().__init__(description="Area of ring and volume "
"of starting cage")
self.periodic = periodic
self.idx1 = np.array([220, 252, 412, 444, 796], dtype='i4')
self.idx2 = np.array([220, 252, 284, 316, 412, 444, 540, 604,
668, 700, 796, 828, 924, 988, 1052, 1084,
1180, 1212, 1308, 1340, 1372, 1404, 1436, 1468],
dtype='i4')
def calculate(self, system):
"""Calculate area and volume."""
pos = system.pos
ar_ring = ConvexHull(pos[self.idx1]).area
vol_cage = ConvexHull(pos[self.idx2]).volume
return [ar_ring, vol_cage]
```
As we also have the z-coordinate of the methane molecule as a collective variable, we also need to add this to our orderp.py file so that the recalculation tool can use it. See the full script for the Position descriptor here:
```python
class Position(OrderParameter):
"""A positional order parameter.
This class defines a very simple order parameter which is just
the position of a given particle.
Attributes
----------
index : integer
This is the index of the atom which will be used, i.e.
``system.pos[index]`` will be used.
dim : integer
This is the dimension of the coordinate to use.
0, 1 or 2 for 'x', 'y' or 'z'.
periodic : boolean
This determines if periodic boundaries should be applied to
the position or not.
"""
def __init__(self, index, dim='x', periodic=False):
"""Initialise the order parameter.
Parameters
----------
index : int
This is the index of the atom we will use the position of.
dim : string
This select what dimension we should consider,
it should equal 'x', 'y' or 'z'.
periodic : boolean, optional
This determines if periodic boundary conditions should be
applied to the position.
"""
txt = 'Position of particle {} (dim: {})'.format(index, dim)
super().__init__(description=txt, velocity=False)
self.periodic = periodic
self.index = index
self.dim = {'x': 0, 'y': 1, 'z': 2}.get(dim, None)
if self.dim is None:
msg = 'Unknown dimension {} requested'.format(dim)
raise ValueError(msg)
def calculate(self, system):
"""Calculate the position order parameter.
Parameters
----------
system : object like :py:class:`.System`
The object containing the positions.
Returns
-------
out : list of floats
The position order parameter.
"""
particles = system.particles
pos = particles.pos[self.index]
lamb = pos[self.dim]
if self.periodic:
lamb = system.box.pbc_coordinate_dim(lamb, self.dim)
return [lamb]
```
Now, to begin the recalculation, we start PyVisA by loading all the data with the retis.rst file with the command:
```pyretis
pyvisa -i retis.toml -recalculate
```
It is important to run PyVisA with the modified retis.rst file so the program knows what has been added. The recalculation can also be started from the PyVisA GUI file menu by selecting “Recalculate data”. The warning shown in fig_recalc_note will appear. Press yes if you have added the post-processing requirements to both retis.rst and orderp.py.
The recalculation will now start. When the procedure is finished, PyVisA loads the new data into the GUI and displays a message that the new data can be visualized.
## Post-processing and clustering
Now that we have the new data, we can use PyVisA to perform clustering and PCA. The following steps can be done in the data exploration:
- Go to the Analysis tab of PyVisA and press the button “Show correlations”. This will produce the correlation matrix. From these results, plot the order parameter and one of the collective variables.
- Select a number of components to use in clustering and from the Analysis-selection, pick an algorithm for clustering and press the Analysis-button. This will produce a cluster plot of the chosen variables. Try to start with k-means, and then try Gaussian mixture and spectral clustering to see if there is a difference between the methods.
- Try to perform a principal component analysis of the results. Begin by selecting 3 components, and PCA as the method, and press the analysis- button. This will produce the loading matrix, the scores plot from the first two components, and the cumulative explained variance. How much variance was retained? Were three components enough? Are there any strong correlations between the principal components and the original descriptors?
If you want to study the results from the principal component analysis further, the data will be stored in an HDF5 file in your simulation directory containing all the simulation data, and the data from the analysis.
## Tested by
The automated coverage for this workflow is represented by examples/tests/test-pyvisa/. The GUI portions remain environment-dependent, while the command-line compression and recalculation paths are the right targets for repeatable checks.
---
# Partial path TIS (REPPTIS) in a 1D potential
This example shows how to run REPPTIS – Replica Exchange Partial Path Transition Interface Sampling – on the 1D double-well benchmark. REPPTIS cuts the sampled trajectories short: instead of requiring a path to connect the two stable states, each path only has to cross from one interface neighbourhood to the next. The short partial paths are then combined with replica exchange between the partial-path ensembles, which makes it possible to reach permeation timescales that ordinary RETIS cannot. The method and its memory-reduction / replica-exchange scheme are described by Vervust, Zhang, van Erp and Ghysels 1.
The closed-form kinetics (mean first passage time, flux and rate) that can be recovered from the short partial paths via a Markov-state-model analysis are derived by Vervust, Wils, Safaei, Zhang and Ghysels 2; REPPTIS for permeation builds on the permeability-from-(RE)TIS formalism of Ghysels, Roet, Davoudi and van Erp 3.
Verification status: smoke – see example-test-status.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-double-well/repptis/.
- Edit first: task = "repptis" and the interfaces list in the [simulation] section; [retis] swapfreq, the probability of a swap with an adjacent ensemble, controls the partial-path replica exchange.
- Run: pyretis run -i repptis.toml -p from the tutorial folder.
- Analyse: pyretis analyse -i repptis.toml after the run has created its output.
- Expected output: the standard per-ensemble folders (000 … 00N) with pathensemble.txt / order.txt, where the partial paths are shorter than the full RETIS paths.
- Related checks: examples/tests/test-internal/partial-path/ (internal Langevin engine) and examples/tests/test-gromacs/test-repptis/ (real GROMACS); see example-test-status.
## Selecting the partial-path task
REPPTIS is requested by setting task = "repptis" in the simulation section. Apart from the task name, the input file is structured exactly like a RETIS input – the same interfaces, [engine], [orderparameter] and [retis] sections apply:
```toml
[simulation]
task = "repptis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
The interfaces list defines the partial-path ensembles in the same way as for RETIS: one [0^{-}] ensemble and one [i^{+}] ensemble per intermediate interface. The difference is in how far each trajectory is propagated – a REPPTIS path is terminated as soon as it reaches a neighbouring interface, rather than being extended all the way to a stable state.
For a permeation setup, combine task = "repptis" with permeability = True and zero_left as shown in the permeability example; the examples/tests/test-internal/partial-path/ fixture runs exactly this combination on a flat 1D potential.
## Running the simulation
Running REPPTIS works the same way as RETIS. Below is the complete input file for the 1D double-well:
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "repptis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
[box]
periodic = [
false,
]
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
[forcefield]
description = "1D double well"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
[output]
archive_every = 50
energy-file = 100
order-file = 100
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
kick-from = "initial"
[retis]
swapfreq = 0.5
```
The initial configuration initial.xyz is included in the tutorial folder. The simulation is executed with:
```pyretis
pyretis run -i repptis.toml -p
```
The -p option displays a progress bar.
## Recovering kinetics from partial paths
Because REPPTIS paths are cut short, the crossing probability, mean first passage time, flux and rate cannot be read off directly the way they are for full RETIS paths. They are instead reconstructed with a Markov-state-model (MSM) analysis of the partial-path ensembles, using the closed formulas of Vervust, Wils, Safaei, Zhang and Ghysels 2. PyRETIS provides that analysis in pyretis.analysis.repptis_msm: the per-ensemble local crossing probabilities (the LML / LMR / RML / RMR path-type fractions) are assembled into a transition matrix with construct_M(), the overall crossing probability follows from global_pcross_msm(), and the mean first passage time – hence the flux and rate – from mfpt_to_first_last_state().
The runnable tutorial msm_kinetics.py in this folder walks through the whole reconstruction. It verifies the exact symmetric limit pcross = 1 / (2 * (N - 1)) and then computes the crossing probability, MFPT and rate for a representative asymmetric model:
```pyretis
python msm_kinetics.py
```
```python
#!/usr/bin/env python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Reconstruct REPPTIS kinetics with the PyRETIS MSM analysis.
A REPPTIS (replica exchange partial path TIS) simulation samples
*short* partial paths, so the overall crossing probability, mean first
passage time (MFPT), flux and rate cannot be read off the path
ensembles directly. They are reconstructed with a Markov-state-model
(MSM) built from the per-ensemble *local* crossing probabilities,
following Vervust, Wils, Safaei, Zhang and Ghysels, J. Chem. Theory
Comput. (2026), https://doi.org/10.1021/acs.jctc.5c01498.
This tutorial demonstrates that analysis with
:py:mod:`pyretis.analysis.repptis_msm`. The starting point is the set
of local crossing probabilities for every partial-path ensemble:
* ``p_mp`` -- LMR: enters from the Left, exits to the Right
(a forward crossing),
* ``p_mm`` -- LML: enters from the Left, returns Left,
* ``p_pm`` -- RML: enters from the Right, exits Left,
* ``p_pp`` -- RMR: enters from the Right, returns Right.
In a real workflow these four numbers per ensemble come from analysing
a REPPTIS run (see the REPPTIS tutorial ``repptis.toml`` in this folder
and ``pyretis.analysis.analyse_repptis_ensemble``). Here we use a
representative set for the 1D double well so the tutorial runs in a
fraction of a second, and we cross-check against the exact symmetric
limit.
Run with::
python msm_kinetics.py
"""
import numpy as np
from pyretis.analysis.repptis_msm import (
construct_M,
construct_tau_vector,
global_pcross_msm,
mfpt_to_first_last_state,
)
def report_crossing_probability(p_mm, p_mp, p_pm, p_pp, n_interfaces):
"""Build the MSM and print the global crossing probability.
Parameters
----------
p_mm, p_mp, p_pm, p_pp : list of float
The per-ensemble local crossing probabilities (length
``n_interfaces - 1``) for the LML, LMR, RML and RMR path types.
n_interfaces : int
The number of interfaces.
Returns
-------
matrix : numpy.ndarray
The MSM transition matrix (reused for the kinetics below).
pcross : float
The global A-to-B crossing probability.
"""
matrix = construct_M(p_mm, p_mp, p_pm, p_pp, n_interfaces)
_, _, y1, _ = global_pcross_msm(matrix)
pcross = float(y1[0, 0])
print(f" global crossing probability P(A->B) = {pcross:.6e}")
return matrix, pcross
def report_kinetics(matrix, pcross, tau_mm, tau_mp, tau_pm, tau_pp,
n_interfaces, flux):
"""Print the rate and the mean first passage time.
The TIS/RETIS rate constant is the product of the initial flux
through the first interface and the overall crossing probability,
``k = flux * pcross``. The mean first passage time (MFPT) is a
separate reconstructed observable, obtained from the MSM together
with the per-ensemble transition times.
Parameters
----------
matrix : numpy.ndarray
The MSM transition matrix from
:py:func:`report_crossing_probability`.
pcross : float
The global A-to-B crossing probability.
tau_mm, tau_mp, tau_pm, tau_pp : list of float
The per-ensemble transition times (length ``n_interfaces``)
for the four path types, in simulation time units.
n_interfaces : int
The number of interfaces.
flux : float
The initial flux through the first interface (crossings per
unit time), as obtained from the ``[0^-]`` flux analysis.
Returns
-------
rate : float
The reconstructed rate constant ``flux * pcross``.
mfpt : float
The reconstructed mean first passage time, in time units.
"""
rate = flux * pcross
print(f" reconstructed rate k = flux * pcross = {rate:.6e}")
n_states = 4 * n_interfaces - 5
tau_vector = construct_tau_vector(
n_interfaces, n_states, tau_mm, tau_mp, tau_pm, tau_pp,
)
# The MFPT machinery splits each visit into a "before", "middle"
# and "after" contribution. For this illustrative example we feed
# the same per-state transition time to each part; a full analysis
# supplies the three measured distributions separately.
_, _, h1, _ = mfpt_to_first_last_state(
matrix, tau_vector, tau_vector, tau_vector,
)
mfpt = float(h1[0, 0])
print(f" mean first passage time (steps) = {mfpt:.6e}")
return rate, mfpt
def main():
"""Run the REPPTIS MSM kinetics demonstration."""
# ----------------------------------------------------------------
# 1) A symmetric, memoryless REPPTIS: every local crossing
# probability is 1/2. The MSM crossing probability then has the
# exact closed form 1 / (2 * (N - 1)), which we verify.
# ----------------------------------------------------------------
n_interfaces = 5
half = [0.5] * (n_interfaces - 1)
print("Symmetric memoryless model (all local probabilities = 0.5):")
_, pcross = report_crossing_probability(
half, half, half, half, n_interfaces,
)
exact = 1.0 / (2.0 * (n_interfaces - 1))
print(f" exact 1/(2*(N-1)) = {exact:.6e}")
assert np.isclose(pcross, exact), "symmetric limit not reproduced"
# ----------------------------------------------------------------
# 2) A representative asymmetric set, as would come from analysing
# a REPPTIS run of the 1D double well (forward crossings get
# harder towards the barrier, so p_mp decreases).
# ----------------------------------------------------------------
p_mp = [0.45, 0.35, 0.30, 0.40] # LMR (forward)
p_mm = [1.0 - value for value in p_mp] # LML (return Left)
p_pp = [0.80, 0.75, 0.70, 0.65] # RMR (return Right)
p_pm = [1.0 - value for value in p_pp] # RML (cross Left)
print("\nRepresentative asymmetric REPPTIS model:")
matrix, pcross = report_crossing_probability(
p_mm, p_mp, p_pm, p_pp, n_interfaces,
)
# ----------------------------------------------------------------
# 3) Combine the crossing probability with the per-ensemble
# transition times and the initial flux to get the rate.
# ----------------------------------------------------------------
tau_mm = [12.0, 14.0, 16.0, 18.0, 20.0]
tau_mp = [40.0, 45.0, 50.0, 55.0, 60.0]
tau_pm = [38.0, 42.0, 46.0, 50.0, 54.0]
tau_pp = [10.0, 11.0, 12.0, 13.0, 14.0]
flux = 1.0e-3 # crossings per step, from the [0^-] flux analysis
print("\nKinetics from the asymmetric model:")
report_kinetics(matrix, pcross, tau_mm, tau_mp, tau_pm, tau_pp,
n_interfaces, flux)
print("\nDone. In a real study the local crossing probabilities "
"and transition\ntimes are measured from a REPPTIS run "
"(see repptis.toml in this folder),\nand the flux from the "
"[0^-] ensemble analysis.")
if __name__ == "__main__":
main()
```
## From a run to the rate (end to end)
repptis_to_kinetics.py closes the loop from sampling to theory: it analyses an actual REPPTIS run for the per-ensemble local crossing probabilities (LML / LMR / RML / RMR, via analyse_repptis_ensemble() and local_crossing_probabilities()) and combines them into the overall crossing probability two independent ways – the closed-form REPPTIS recursion (global_crossing_probabilities_from_local()) and the Markov state model (construct_M() + global_pcross_msm()) – which agree. The full workflow is:
```pyretis
pyretis run -i repptis.toml -p # sample the partial paths
python repptis_to_kinetics.py # reconstruct the kinetics
```
If run without simulation output present, the script falls back to a small set of illustrative local probabilities so it still demonstrates the analysis.
```python
#!/usr/bin/env python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""From a REPPTIS simulation to its kinetics, end to end.
This tutorial closes the loop from *sampling* to *theory*: it takes the
per-ensemble output of a REPPTIS run, extracts the local crossing
probabilities of each ensemble, and combines them into the overall
crossing probability two independent ways -- the closed-form REPPTIS
recursion and the Markov-state-model -- which must agree. It follows the
analysis of Vervust, Wils, Safaei, Zhang and Ghysels, J. Chem. Theory
Comput. (2026), https://doi.org/10.1021/acs.jctc.5c01498.
Workflow
--------
1. Run the REPPTIS simulation in this folder::
pyretis run -i repptis.toml -p
This fills the per-ensemble directories ``000/``, ``001/``, ... with
``pathensemble.txt`` files.
2. Reconstruct the kinetics::
python repptis_to_kinetics.py
If no run output is found, the script falls back to a small set of
*illustrative* local crossing probabilities so that it still
demonstrates the analysis (and runs in the tutorial smoke suite); the
banner makes clear which mode is in effect.
"""
import os
import numpy as np
from pyretis.analysis import (
analyse_repptis_ensemble,
construct_M,
global_crossing_probabilities_from_local,
global_pcross_msm,
)
from pyretis.core.pathensemble import generate_ensemble_name
from pyretis.inout.formats.pathensemble import PathEnsembleFile
from pyretis.inout.settings import parse_settings_file
from pyretis.setup.createsimulation import create_ensembles
INPUT_FILE = "repptis.toml"
ANALYSIS_SETTINGS = {
"ngrid": 1000,
"maxblock": 100,
"bins": 100,
"blockskip": 1,
"skip": 0,
}
# Illustrative per-ensemble local crossing probabilities, used only when
# no run output is present (so the tutorial still demonstrates the
# analysis). Order: LMR (forward), LML (return left), RMR (return
# right), RML (cross left), one entry per [i^+] ensemble.
EXAMPLE_LOCAL_PROBS = {
"LMR": [0.45, 0.35, 0.30, 0.40],
"LML": [0.55, 0.65, 0.70, 0.60],
"RMR": [0.80, 0.75, 0.70, 0.65],
"RML": [0.20, 0.25, 0.30, 0.35],
}
def collect_local_probs_from_run():
"""Analyse each ``[i^+]`` ensemble of a run for its local probs.
Returns
-------
probs : dict or None
Lists of per-ensemble local probabilities keyed ``LMR`` /
``LML`` / ``RMR`` / ``RML``, or None if the run output is
missing.
n_interfaces : int
The number of interfaces (needed to size the MSM).
"""
settings = parse_settings_file(INPUT_FILE)
settings.setdefault("analysis", {}).update(ANALYSIS_SETTINGS)
ensembles = create_ensembles(settings)
n_interfaces = len(settings["simulation"]["interfaces"])
probs = {"LMR": [], "LML": [], "RMR": [], "RML": []}
for ensemble in ensembles:
path_ensemble = ensemble["path_ensemble"]
number = path_ensemble.ensemble_number
if number == 0: # the [0^-] ensemble is not an [i^+-] ensemble
continue
filename = os.path.join(
generate_ensemble_name(number), "pathensemble.txt"
)
if not os.path.isfile(filename):
return None, n_interfaces
ens_file = PathEnsembleFile(
filename, "r",
ensemble_settings={"ensemble_number": number,
"interfaces": path_ensemble.interfaces},
)
result = analyse_repptis_ensemble(
ens_file,
{"tis": {"detect": path_ensemble.interfaces[-1],
"ensemble_number": number},
"analysis": ANALYSIS_SETTINGS},
)
local = result["local_probs"]
for key in ("LMR", "LML", "RMR", "RML"):
probs[key].append(local[key])
return probs, n_interfaces
def report_crossing_probability(probs, n_interfaces):
"""Print the global crossing probability from the local probs.
Computes it twice -- via the closed-form REPPTIS recursion and via
the Markov state model -- and checks the two agree.
Parameters
----------
probs : dict
Per-ensemble local probabilities keyed ``LMR`` / ``LML`` /
``RMR`` / ``RML``.
n_interfaces : int
The number of interfaces.
"""
p_mp, p_mm = probs["LMR"], probs["LML"]
p_pp, p_pm = probs["RMR"], probs["RML"]
print("Per-ensemble local crossing probabilities:")
for i, (mp, mm, pp, pm) in enumerate(zip(p_mp, p_mm, p_pp, p_pm)):
print(f" [{i + 1}^+] LMR={mp:.3f} LML={mm:.3f} "
f"RMR={pp:.3f} RML={pm:.3f}")
recursion = global_crossing_probabilities_from_local(
p_mp, p_mm, p_pp, p_pm
)
if np.any(np.isnan(recursion[0])):
print("\nSome ensemble was not sampled in both directions "
"(nan local probability); collect more REPPTIS cycles "
"before reconstructing the kinetics.")
return
p_cross = recursion[2]
print(f"\nGlobal crossing probability (recursion) = {p_cross[-1]:.6e}")
# The MSM uses N-1 local-prob entries per type.
matrix = construct_M(p_mm[:n_interfaces - 1], p_mp[:n_interfaces - 1],
p_pm[:n_interfaces - 1], p_pp[:n_interfaces - 1],
n_interfaces)
_, _, y1, _ = global_pcross_msm(matrix)
print(f"Global crossing probability (MSM) = {float(y1[0, 0]):.6e}")
print("The two independent estimates agree:",
np.isclose(p_cross[-1], float(y1[0, 0])))
def main():
"""Reconstruct the REPPTIS kinetics from a run (or example probs)."""
probs, n_interfaces = collect_local_probs_from_run()
if probs is None:
print("No REPPTIS run output found in this folder; "
"demonstrating with illustrative local probabilities.")
print("Run 'pyretis run -i repptis.toml -p' first to analyse a "
"real simulation.\n")
probs = EXAMPLE_LOCAL_PROBS
n_interfaces = len(probs["LMR"]) + 1
else:
print("Analysed the REPPTIS run output in this folder.\n")
report_crossing_probability(probs, n_interfaces)
if __name__ == "__main__":
main()
```
## Tested by
The REPPTIS task is represented by examples/tutorials/path_sampling/internal/1D-double-well/repptis/ and checked by examples/tests/test-internal/partial-path/ (internal engine) and examples/tests/test-gromacs/test-repptis/ (GROMACS). The MSM kinetics tutorial msm_kinetics.py runs in the tutorial smoke suite (examples/run-tutorials.sh, exercised by test-heavy) and the underlying analysis is unit-tested in test/analysis/test_repptis_msm.py. Use the fixtures as the short regression checks and this page as the user-facing guide.
## References
W. Vervust, D. T. Zhang, T. S. van Erp and A. Ghysels, Path sampling with memory reduction and replica exchange to reach long permeation timescales, Biophys. J. 122, 2960-2972 (2023), https://doi.org/10.1016/j.bpj.2023.02.021
W. Vervust, E. Wils, S. Safaei, D. T. Zhang and A. Ghysels, Estimating full path lengths and kinetics from partial path transition interface sampling simulations, J. Chem. Theory Comput. (2026), https://doi.org/10.1021/acs.jctc.5c01498 (arXiv:2602.12835).
A. Ghysels, S. Roet, S. Davoudi and T. S. van Erp, Exact non-Markovian permeability from rare event simulations, Phys. Rev. Research 3, 033068 (2021), https://doi.org/10.1103/PhysRevResearch.3.033068
---
# RETIS in a 1D potential
In this example, you will explore a rare event with the Replica Exchange Transition Interface Sampling (RETIS) algorithm.
We will consider a simple 1D potential where a particle is moving. The potential is given by V_{\text{pot}} = x^4 - 2 x^2 where x is the position. By plotting this potential, we see that we have two states (at x=-1 and x=1) separated by a barrier (at x=0):
Using the RETIS algorithm, we will set up the path ensembles, replica exchange moves, and analysis needed to estimate the rate constant for the transition between the two states. This particular problem has been considered before by van Erp 1 2; the focus here is how to adapt the setup for your own rare-event simulations.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-double-well/retis/.
- Edit first: retis.toml for interfaces, move settings, engine settings, and output frequency; edit initial.xyz only when changing the system.
- Run: pyretis run -i retis.toml -p.
- Analyse: pyretis analyse -i retis.toml – the analysis reads the same input file as the run.
- Expected output: numbered ensemble folders, trajectory/order/energy files, pyretis.log, output.toml, and a report/ folder after analysis.
- Related checks: examples/tests/test-internal/retis/, examples/tests/test-internal/retis-restart/, and examples/tests/test-internal/retis-load-sparse/; see example-test-status.
## Creating the PyRETIS input file
We will now create the input file for PyRETIS. We will do this section by section in order to explain the different keywords and settings. The full input file is given at the end of this section.
### Setting up the simulation task
The first thing we will define is the type of simulation we will run. This is done by creating a simulation section.
Here, we are going to do a retis simulation and we will do 20000 steps 2. Since we will be running a path sampling simulation, we will also need to specify the positions of the interfaces we will be using.
```toml
[simulation]
task = "retis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
### Setting up the system
We will now set up the system we are going to consider. Here, we will actually define several PyRETIS input sections:
- The system section which defines the units, dimensions and temperature we are considering:
```toml
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
```
- The box section, but since we are here just considering a single particle in a 1D potential, we will simply use a 1D box without periodic boundaries:
```toml
[box]
periodic = [
false,
]
```
- The particles section which add particles to the system and defines the initial state:
```toml
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
```
- In this case, we will read the initial configuration from a file initial.xyz and velocities are generated from a Maxwell distribution. Further, we specify the mass, particle type and we label the particle as Ar. Note that this does not mean that we are simulating Argon, it is just a label used in the output of trajectories.
- The force field and potential sections which setup up the 1D double well potential:
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[forcefield]
description = "1D double well"
```
### Selecting the engine
Here, we will make use of a stochastic Langevin engine. We set it up by setting the time step, the friction parameter and whether we are in the high friction limit. The seed given is a seed for the random number generator used by the integrator.
```toml
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
```
### TIS specific settings
The TIS settings control how the TIS algorithm is carried out. Here we set that 50 % of the TIS moves should be shooting moves (keyword freq) and we limit all paths to a maximum length of 20 000 steps. Further, sigma_v = -1 selects aimless shooting. We also tell PyRETIS to not set the momentum to zero and to not rescale the energy after drawing new random velocities. We also set allowmaxlength = False which means that for shooting, we determine stochastically the length of new paths based on the length of the path we are shooting from. The given seed is a seed for the random number generator used by the TIS algorithm.
```toml
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
```
### RETIS specific settings
The RETIS section controls the RETIS algorithm. Here swapfreq = 0.5 is the probability that a move picked in the [0^-] or the [0^+] ensemble is the swap of the paths of the two (the zero swap), when the other ensemble is free.
```toml
[retis]
swapfreq = 0.5
```
### Initial path settings
These settings determine how we find the initial path(s). Here, we ask PyRETIS to generate these using the kick method.
```toml
[initial-path]
method = "kick"
kick-from = "initial"
```
### Selecting the order parameter
For this system, we simply define the order parameter as the position of the single particle we are simulating.
```toml
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
```
### Modifying the output
In the Output section, we here keep the trajectory of only every 100th path the run replaces in the archive (archive_every). A value above 0 of energy-file and order-file writes the energies and the order parameter of each path.
```toml
[output]
archive_every = 100
energy-file = 100
order-file = 100
```
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "retis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
[box]
periodic = [
false,
]
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
[forcefield]
description = "1D double well"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
[output]
archive_every = 100
energy-file = 100
order-file = 100
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
kick-from = "initial"
[retis]
swapfreq = 0.5
```
## Running the RETIS simulation
We will now run the RETIS simulation. Create a new directory and place the input file (let’s call it retis.toml) here. Also, download the initial configuration initial.xyz and place it in the same folder. The simulation can then be executed using:
```pyretis
pyretis run -i retis.toml -p
```
The -p option will display a progress bar for your simulation.
## Analysing the results
When the simulation has finished, we can analyse the results. The analysis reads the same input file as the run; the run itself writes output.toml, which records the input it ran with, under the names of the input and with every default included. pyretis analyse -i output.toml reads the input sections from it and gives the same analysis.
How the analysis is performed is controlled by an analysis section, which you can add to the input:
```rst
Analysis settings
-----------------
maxordermsd = -1
ngrid = 1001
report = ['latex', 'rst', 'html']
maxblock = 1000
txt-output = 'txt.gz'
blockskip = 1
plot = {'output': 'png', 'plotter': 'mpl', 'style': 'pyretis'}
bins = 100
skipcross = 1000
```
For a description of these keywords, we refer to the analysis section.
The analysis itself is performed using:
```pyretis
pyretis analyse -i retis.toml
```
This will produce a new folder, report which contains the results from the analysis. If you have latex installed, you can generate a pdf using the file retis_report.tex within the report folder. An example result for the crossing probability is shown below.
## Improving the statistics
We can improve the statistics by running a longer simulation. To continue the simulation in its directory up to 1000000 steps, set method = "restart" in the [initial-path] section and steps = 1000000 in the [simulation] section of retis.toml, run
```pyretis
pyretis run -i retis.toml -p
```
again, and repeat the analysis. The restart continues from the cycle and the paths saved in output.toml and appends the new cycles to the output of each ensemble (see the restart method). Running the input again with method = "kick" in the same directory stops with an error, since its ensemble directories hold the sampled cycles. To start over, use a new directory, or first run pyretis tools clean (see user-guide-clean). Below we show an example for the crossing probability after performing the additional steps
## Tested by
The main workflow is represented by examples/tests/test-internal/retis/. Restart and loading behavior are covered by examples/tests/test-internal/retis-restart/ and examples/tests/test-internal/retis-load-sparse/. These checks are shorter than the tutorial run but exercise the same input parsing, simulation, and analysis path.
## References
Titus S. van Erp, Dynamical Rare Event Simulation Techniques for Equilibrium and Nonequilibrium Systems, Advances in Chemical Physics, 151, pp. 27-60, 2012, https://doi.org/10.1002/9781118309513.ch2
https://arxiv.org/abs/1101.0927
---
# RETIS in a 1D triple-well: the zero_left shortcut
This example shows how a left interface for the [0:math:^-] ensemble (set with the zero_left keyword) can make a RETIS simulation a lot cheaper, and what it does to the flux of a rate calculation.
The motivation is simple. In RETIS the [0:math:^-] ensemble samples the recrossings inside the reactant state. By default, paths in [0:math:^-] can wander arbitrarily far to the left of \lambda_0. In a multi-state landscape this means they can fall into a neighbouring metastable well, where they get trapped and finally rejected at maxlength. zero_left adds a hard left boundary to [0:math:^-] so that paths that would escape into the neighbouring well are rejected on the spot with the cheap 0-L status code (BWI for a trial that starts at zero_left).
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-triple-well/.
- Edit first: compare retis-no-zero-left/retis.toml and retis-zero-left/retis.toml; the key setting is zero_left in the [simulation] section.
- Run: execute pyretis run -i retis.toml -p in each variant directory.
- Analyse: run pyretis analyse -i retis.toml in each completed variant directory, then run python compare.py from the tutorial directory.
- Expected output: two comparable RETIS output trees and a short comparison of [0:math:^-] path cost.
- Related checks: represented by the internal RETIS checks and the tutorial smoke runner; see example-test-status.
## The system
We use the TripleWell potential
with (a, b, c, d) = (1, -3, 2, 0). This gives three minima
- A at x \approx -1.256 (well depth \approx -0.39)
- B at x = 0 (metastable intermediate, V = 0)
- C at x \approx +1.256 (well depth \approx -0.39)
separated by two barriers of height \approx 0.385 above B at x \approx \pm 0.65. From the A side, the same barriers are \approx 0.776 high, so a particle that drops into A spends many integration steps before escaping.
We sample only the B \to C transition. Without a left bound, [0:math:^-] paths that wander into A inflate the sampling cost; with a left bound at x = -0.4 (well inside the B basin) they cannot reach A at all.
## The two input files
The example ships two input files that differ in one line.
Baseline (retis-no-zero-left/retis.toml) – a retis run samples [0:math:^-], here without a left bound:
```rst
Simulation
----------
task = retis
steps = 250
interfaces = [0.05, 0.20, 0.40, 0.55, 0.90]
seed = 0
```
With zero_left (retis-zero-left/retis.toml):
```rst
Simulation
----------
task = retis
steps = 250
interfaces = [0.05, 0.20, 0.40, 0.55, 0.90]
zero_left = -0.4
seed = 0
```
Everything else (interfaces, engine, temperature, seeds, output) is identical. The full input files are bundled with the PyRETIS sources under examples/tutorials/path_sampling/internal/1D-triple-well/.
The triple-well potential itself is specified by:
```rst
Potential
---------
class = TripleWell
a = 1.0
b = -3.0
c = 2.0
d = 0.0
```
## Running the example
Each variant runs in roughly one minute on a typical laptop. From the example directory:
```bash
cd retis-no-zero-left
pyretis run -i retis.toml -p
pyretis analyse -i retis.toml
cd ../retis-zero-left
pyretis run -i retis.toml -p
pyretis analyse -i retis.toml
```
A small helper script then reports the [0:math:^-] ensemble cost for both runs:
```python
"""Compare the [0^-] cost of the two triple-well RETIS runs.
Run after both ``retis-no-zero-left`` and ``retis-zero-left`` have
been simulated and analysed. The script reads ``000/pathensemble.txt``
in each directory and reports the accepted-path statistics that show
the cost of the [0^-] ensemble.
"""
import os
import sys
```
Run it with python compare.py.
## What to look for
In a typical run, the variant with zero_left shows:
- zero FTX / BTX rejections in [0:math:^-] (rejections appear as the much cheaper 0-L and BWI statuses), and fewer MD steps spent in [0:math:^-] (the MD-steps column of 000/moves.txt);
- the same B \to C crossing probabilities in [0:math:^+], [1:math:^+], …, within their statistical errors;
- a higher flux: the [0:math:^-] of a rate calculation holds only the paths that turn back before zero_left, and the flux is computed from them, without the longer excursions it rejects. pyretis analyse warns when more than 1 % of the [0:math:^-] trials were rejected at zero_left; with zero_left = -0.4 a few percent are.
That is the take-away: zero_left keeps the [0:math:^-] out of a neighbouring long-lived basin you are not interested in sampling, and the flux leaves out the paths it stops, so place it where the [0:math:^-] paths seldom reach it.
## Going further
- Move zero_left toward the A-B barrier (e.g. -0.6) and watch the share of rejections at zero_left drop; move it toward the first interface (e.g. -0.2) and watch it grow.
- Add a tilt to the potential (d parameter of TripleWell) to make A deeper and the trapping effect more dramatic.
- Change [retis] swapfreq to see how the share of swaps of [0:math:^-] \leftrightarrow [0:math:^+] changes.
## Tested by
- The tutorial is included in examples/tutorials/path_sampling/internal/1D-triple-well/ and runs every time the tutorial smoke runner (examples/run-tutorials.sh) is invoked, including from the top-level ./test-heavy.sh.
- A dedicated, deterministic regression fixture lives at examples/tests/test-internal/retis-zero-left/. It runs the same two input files at a much smaller size, checks that the 0-L rejections appear in [0:math:^-] (and may propagate to [0:math:^+] through a failed swap-zero) but never in [1:math:^+] or beyond, and that zero_left does not produce FTX/BTX rejections in [0:math:^-]. Run it with:
```
cd examples/tests/test-internal/retis-zero-left
./run.sh
```
---
# Breaking a bond with RETIS
In this example, you will explore bond breaking in a simple model system with the Replica Exchange Transition Interface Sampling (RETIS) algorithm.
The system we consider consist of a simple diatomic bistable molecule immersed in a fluid of purely repulsive particles. This system was actually considered in the article introducing the TIS method, 1 and we will here largely follow this simulation set-up. We refer to the original article for more details.
The potential we use to describe the bond for the diatomic molecule is a double-well potential, similar to the DoubleWellWCA potential implemented in PyRETIS. By tuning the potential parameters, we will consider two cases: one with a high barrier and one with a low barrier.
This example is structured as follows:
- Tutorial quick start
- The high barrier case
- Defining the potential functions and the order parameter
- Running the RETIS simulation
- The low barrier case
- Defining the potential functions and the order parameter
- Running the RETIS simulation
- Tested by
- References
Verification status: smoke – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/2D-wca/.
- Requirements: a compiler toolchain for the C extensions used by the high- and low-barrier examples.
- Edit first: the RETIS input file for the chosen barrier case, then the C/Python order-parameter and force-field modules only if you change the model.
- Build extensions: python setup.py build_ext --inplace in the directory containing the extension module.
- Run: pyretis run -i retis.toml -p inside the chosen barrier variant directory (e.g. high-barrier/retis-c/).
- Expected output: standard RETIS ensemble folders and compiled extension modules next to the input files.
- Related checks: the external LAMMPS version is covered by examples/tests/test-lammps/; internal C-extension coverage is representative rather than one-to-one. See example-test-status.
## The high barrier case
For the low barrier case, we will consider a 2D system consisting of 25 particles and the kinetic energy is scaled so that the energy of the system is 25 in reduced units. 1 Further, the number density of the system is set to 0.7, the barrier height is h = 15, the width parameter is w = 0.5 and r_0 = 2^{1/6}.
### Defining the potential functions and the order parameter
For this example, we implement both the potential functions and the order parameter in C. The code can be downloaded as this zip-archive: high.zip.
In this case, we define the order parameter as the distance between the two atoms in the diatomic molecule.
### Running the RETIS simulation
The script for running the RETIS simulation is given below. Note that before running the simulation with PyRETIS you will have to compile the C code for the order parameter and the potential functions:
```pyretis
python setup.py build_ext --inplace
```
The input script given below assumes that the order parameter and the potential functions are stored in a sub-folder named c-for-python3. The initial configuration can be downloaded here: initial-high.xyz.
```toml
[simulation]
task = "retis"
steps = 1000000
interfaces = [
1.24,
1.34,
1.4,
1.46,
1.52,
1.54,
1.64,
1.74,
]
[system]
units = "reduced"
dimensions = 2
temperature = 1.0
[engine]
class = "VelocityVerlet"
timestep = 0.002
[box]
cell = [
5.976143046671968,
5.976143046671968,
]
periodic = [
true,
true,
]
[particles]
name = [
"B",
"B",
"A",
]
ptype = [
1,
1,
0,
]
[particles.position]
input_file = "../initial.xyz"
[particles.velocity]
scale = 25.0
[particles.mass]
A = 1.0
B = 1.0
[forcefield]
description = "2D Double Well + WCA"
[[potential]]
class = "WCAPotential"
module = "../c-for-python3/wcafunctions.py"
shift = true
dim = 2
[potential.parameter]
sigma = 1.0
epsilon = 1.0
rcut = 1.122462048309373
idxi = 0
idxj = 1
rzero = 1.122462048309373
height = 15.0
width = 0.5
[orderparameter]
class = "WCAOrderParameter"
index = [
0,
1,
]
module = "../c-for-python3/wcafunctions.py"
[output]
archive_every = 5000
energy-file = 100
order-file = 100
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = true
rescale_energy = 25
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
[retis]
swapfreq = 0.5
```
The PyRETIS simulation can be executed in the usual way:
```pyretis
pyretis run -i retis.toml -p
```
Before running the simulation: Ensure that the files and directories (i.e. the initial configuration and the modules referenced) are named correctly with respect to the files you have downloaded for this example.
## The low barrier case
For the low barrier case, we will consider a 2D system consisting of 9 particles and the kinetic energy is scaled so that the energy of the system is 9 in reduced units. 1 Further, the number density of the system is set to 0.6, the barrier height is h = 6, the width parameter is w = 0.25 and r_0 = 2^{1/6}.
### Defining the potential functions and the order parameter
For this example, we implement the potential functions in C. The code can be downloaded as a zip-archive here: low.zip.
For the order parameter, we include a kinetic energy and potential energy 1 in the definition to ensure the stability of the initial and final states. The Python code for the order parameter can be found below.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""This file defines the order parameter used for the WCA example."""
import logging
import numpy as np
from pyretis.orderparameter import OrderParameter
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
class OrderParameterWCAJCP1(OrderParameter):
"""OrderParameterWCAJCP1(OrderParameter).
This class defines a very simple order parameter which is just
the scalar distance between two particles.
Attributes
----------
index : tuple of integers
These are the indices used for the two particles.
`system.pos[index[0]]` and
`system.pos[index[1]]` will be used.
periodic : boolean
This determines if periodic boundaries should be applied to
the position or not.
"""
def __init__(self, index, periodic=True):
"""Set uå the order parameter.
Parameters
----------
index : tuple of ints
This is the indices of the atom we will use the position of.
periodic : boolean, optional
This determines if periodic boundary conditions should be
applied to the position.
"""
pbc = 'Periodic' if periodic else 'Non-periodic'
txt = '{} distance particles {} and {}'.format(pbc, index[0],
index[1])
super().__init__(description=txt)
self.periodic = periodic
self.index = index
def calculate(self, system):
"""Calculate the order parameter.
Here, the order parameter is just the distance between two
particles.
Parameters
----------
system : object like :py:class:`.System`
This object is used for the actual calculation, typically
only `system.pos` and/or `system.vel`
will be used. In some cases `system.forcefield` can also be
used to include specific energies for the order parameter.
Returns
-------
out : list of floats
The order parameter.
"""
# pylint: disable=invalid-name
particles = system.particles
delta = particles.pos[self.index[1]] - particles.pos[self.index[0]]
if self.periodic:
delta = system.box.pbc_dist_coordinate(delta)
r = np.sqrt(np.dot(delta, delta))
dx = delta
dv = particles.vel[self.index[1]] - particles.vel[self.index[0]]
dxdv = np.dot(dx, dv) / r
m1 = particles.mass[self.index[0]]
m2 = particles.mass[self.index[1]]
m = m1 * m2 / (m1 + m2)
potential_func = system.forcefield.potential[0]
if potential_func is None:
return [float(r)]
if r < 1.2:
pot = potential_func.potential_well(system) + 0.5 * m * (dxdv)**2
orderp = 1.19
if pot < 1.5:
orderp = 1.18 - (1.5 - pot) / 0.5 * 0.02
elif r > 1.42:
pot = potential_func.potential_well(system) + 0.5 * m * (dxdv)**2
orderp = 1.43
if pot < 5.0:
orderp = 1.44 + (5.0 - pot) / 0.5 * 0.02
else:
orderp = r
return [float(orderp), dxdv]
```
### Running the RETIS simulation
The script for running the RETIS simulation is given below. Note that before running the simulation with PyRETIS you will have to compile the C code for the potential
```pyretis
python setup.py build_ext --inplace
```
and create the script for the order parameter (the input script given below assumes that the order parameter is stored in a file orderp.py. The initial configuration can be downloaded here: initial-low.xyz.
```toml
[simulation]
task = "retis"
steps = 1000000
interfaces = [
1.2,
1.24,
1.26,
1.32,
1.58,
]
[system]
units = "reduced"
dimensions = 2
temperature = 1.0
[engine]
class = "VelocityVerlet"
timestep = 0.002
[box]
cell = [
3.872983346207417,
3.872983346207417,
]
periodic = [
true,
true,
]
[particles]
name = [
"A",
"A",
"A",
"A",
"A",
"A",
"A",
"B",
"B",
]
ptype = [
0,
0,
0,
0,
0,
0,
0,
1,
1,
]
[particles.position]
input_file = "../initial.xyz"
[particles.velocity]
scale = 9.0
[particles.mass]
A = 1.0
B = 1.0
[forcefield]
description = "2D Double Well + WCA"
[[potential]]
class = "WCAPotential"
module = "../c-for-python3/wcafunctions.py"
shift = true
dim = 2
[potential.parameter]
sigma = 1.0
epsilon = 1.0
rcut = 1.122462048309373
idxi = 7
idxj = 8
rzero = 1.122462048309373
height = 6.0
width = 0.25
[orderparameter]
class = "OrderParameterWCAJCP1"
module = "../c-for-python3/orderp.py"
index = [
7,
8,
]
periodic = true
[output]
archive_every = 5000
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = true
rescale_energy = 9.0
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
[retis]
swapfreq = 0.5
```
The PyRETIS simulation can be executed in the usual way:
```pyretis
pyretis run -i retis.toml -p
```
Before running the simulation: Ensure that the files and directories (i.e. the initial configuration and the modules referenced) are named correctly with respect to the files you have downloaded for this example.
## Tested by
This internal WCA tutorial is represented by the same model family as the LAMMPS heavy checks in examples/tests/test-lammps/ and by the extension tests used for the compiled modules. Add a dedicated shortened fixture under examples/tests/ before relying on the high- or low-barrier internal inputs as release regression checks.
## References
Titus S. van Erp, Daniele Moroni, and Peter G. Bolhuis, A novel path sampling method for the calculation of rate constants, The Journal of Chemical Physics, 118, pp. 7762-7774, 2003, https://doi.org/10.1063/1.1562614.
---
# Subtrajectory moves in a 1D potential
This example shows how to use the subtrajectory monte carlo moves Stone Skipping (SS), Web Throwing (WT) 1 and Wire Fencing (WF) 2 in (Replica Exchange) Transition Interface Sampling simulations for sampling trajectories of a particle in a 1D well. The three moves are sketched out below:
Further details on the 1D potential, how to create the PyRETIS input file and calculating reaction rates using TIS/RETIS are explained by the previous 1D potential TIS example and RETIS example.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-double-well/submoves/.
- Edit first: the shooting_moves, interface_sour, interface_cap, and n_jumps settings in the TIS section.
- Run: pyretis run -i retis.toml -p from the tutorial folder.
- Analyse: pyretis analyse -i retis.toml after the run.
- Expected output: standard RETIS ensemble folders plus move labels for stone skipping, web throwing, and wire fencing in pathensemble.txt.
- Related check: examples/tests/test-internal/retis-ss-wt-wf/; see example-test-status.
## Defining the shooting move
To define the specific shooting moves performed in each of the ensembles in i.e. a RETIS simulation, the number of ensembles needs to be known. This information can be obtained from reading the interfaces variable in the simulation section of the input file:
```toml
[simulation]
task = "retis"
steps = 200
interfaces = [
-0.99,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
Here we have one [0^{-}] ensemble and seven [i^{+}] ensembles for a total of 8 ensembles. Then we can define the specific shooting moves to be performed in each ensemble in the tis section:
```toml
[tis]
freq = 0.0
maxlength = 50000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
shooting_moves = [
"sh",
"sh",
"ss",
"ss",
"wt",
"wt",
"wf",
"wf",
]
interface_sour = -0.8
interface_cap = 0.1
n_jumps = 6
high_accept = true
```
The shooting_moves variable defines the list of shooting moves to be used for all the ensembles. Here we see that [0^{-}] and [0^{+}] performs the shooting move while the other ensembles performs the SS, WT and WF moves. The interface_sour sets the SOUR interface for the WT, while if defined, the interface_cap variable sets the upper value limit of subtrajectories generated by the WF move. The variable n_jumps defines the number of subtrajectories to be generated per move and the bool high_accept determines whether the high acceptance protocol should be used or not. See user-guide-path-move-types for the full list of move type codes and user-guide-path-status-codes for all acceptance and rejection status codes.
## Running the RETIS simulation
Running a RETIS simulation with subtrajectory moves works the same way as running without subtrajectory moves. Below is the complete retis.toml input file.
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "retis"
steps = 200
interfaces = [
-0.99,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.025
gamma = 0.3
high_friction = false
seed = 0
[box]
periodic = [
false,
]
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.mass]
Ar = 1.0
[forcefield]
description = "1D double well"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
[output]
archive_every = 50
order-file = -1
energy-file = -1
[tis]
freq = 0.0
maxlength = 50000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
shooting_moves = [
"sh",
"sh",
"ss",
"ss",
"wt",
"wt",
"wf",
"wf",
]
interface_sour = -0.8
interface_cap = 0.1
n_jumps = 6
high_accept = true
[initial-path]
method = "kick"
kick-from = "initial"
[retis]
swapfreq = 0.5
```
The initial configuration initial.xyz is included in the tutorial folder. The simulation can then be executed using:
```pyretis
pyretis run -i retis.toml -p
```
The -p option will display a progress bar for your simulation.
## Tested by
The subtrajectory move settings are represented by examples/tutorials/path_sampling/internal/1D-double-well/submoves/ and checked by examples/tests/test-internal/retis-ss-wt-wf/. Use the fixture as the short regression check and this page as the user-facing guide to choosing and interpreting the move settings.
## References
E. Riccardi, O. Dahlen and T. S. van Erp, Fast decorrelating Monte Carlo moves for efficient path sampling, J. Phys. Chem. Lett. 8, 4456-4460 (2017), https://doi.org/10.1021/acs.jpclett.7b01617
D. T. Zhang, E. Riccardi and T. S. van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023), https://doi.org/10.1063/5.0127249
---
# TIS in a 1D potential
In this example, you will explore a rare event with the Transition Interface Sampling (TIS) algorithm.
We will consider a simple 1D potential where a particle is moving. The potential is given by V_{\text{pot}} = x^4 - 2 x^2 where x is the position. By plotting this potential, we see that we have two states (at x=-1 and x=1) separated by a barrier (at x=0):
Using the TIS algorithm, we will set up the pieces needed to compute the rate constant for the transition between the two states. This particular problem has been considered before by van Erp 1 2; the focus here is how to build, run, and analyse a similar PyRETIS setup yourself.
Verification status: passing – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/path_sampling/internal/1D-double-well/tis-multiple/ for TIS and examples/tutorials/path_sampling/internal/1D-double-well/flux/ for the initial flux.
- Edit first: tis-multiple.toml, flux.toml, and initial.xyz if you change the system.
- Run the TIS setup: pyretis run -i tis-multiple.toml to create one input file per ensemble, then run the generated files, for example pyretis run -i tis-001.toml -p -f 001.log.
- Run the flux setup: pyretis run -i flux.toml -p.
- Analyse: use pyretis analyse -i tis-multiple.toml for split TIS runs and pyretis analyse -i flux.toml for the flux run.
- Expected output: generated ensemble input files, numbered output directories, generated tis-NNN.toml files, and a report/ folder after analysis.
- Related checks: examples/tests/test-internal/tis-multiple/, examples/tests/test-internal/tis-restart/, and examples/tests/test-internal/mdflux-restart/; see example-test-status.
In order to obtain a rate constant, we will have to obtain both the crossing probability and an initial flux. To obtain to the crossing probability, we need to carry out TIS simulations for each path ensemble we define and in order to obtain the initial flux, we will have to carry out a MD simulation. Below we describe the two kinds of input files we need to create.
## Creating and running the TIS simulations with PyRETIS
We will now create the input file for PyRETIS. We will do this section by section in order to explain the different keywords and settings. The full input file is given at the end of this section.
### Setting up the simulation task
The first thing we will define is the type of simulation we will run. This is done by creating a simulation section.
Here, we request make-tis-files and will do 20000 steps 2. Since we will be running a path sampling simulation, we will also need to specify the positions of the interfaces we will be using.
```toml
[simulation]
task = "make-tis-files"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
### Setting up the system
We will now set up the system we are going to consider. Here, we will actually define several PyRETIS input sections:
- The system section which defines the units, dimensions and temperature we are considering:
```toml
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
```
- The box section, but since we are here just considering a single particle in a 1D potential, we will simply use a 1D box without periodic boundaries:
```toml
[box]
periodic = [
false,
]
```
- The particles section which add particles to the system and defines the initial state:
```toml
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
```
- In this case, we will read the initial configuration from a file initial.xyz and velocities are generated from a Maxwell distribution. Further, we specify the mass, particle type and we label the particle as Ar. Note that this does not mean that we are simulating Argon, it is just a label used in the output of trajectories.
- The force field and potential sections which setup up the 1D double well potential:
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[forcefield]
description = "1D double well"
```
### Selecting the engine
Here, we will make use of a stochastic Langevin engine. We set it up by setting the time step, the friction parameter and whether we are in the high friction limit. The seed given is a seed for the random number generator used by the integrator.
```toml
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
```
### TIS specific settings
The TIS settings control how the TIS algorithm is carried out. Here we set that 50 % of the moves should be shooting moves (keyword freq) and we limit all paths to a maximum length of 20 000 steps. Further, sigma_v = -1 selects aimless shooting. We also tell PyRETIS to not set the momentum to zero and to not rescale the energy after drawing new random velocities. We also set allowmaxlength = False which means that for shooting, we determine (randomly) the length of new paths based on the length of the path we are shooting from. The given seed is a seed for the random number generator used by the TIS algorithm.
```toml
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
```
### Initial path settings
These settings determine how we find the initial path(s). Here, we ask PyRETIS to generate these using the kick method.
```toml
[initial-path]
method = "kick"
```
### Selecting the order parameter
For this system, we simply define the order parameter as the position of the single particle we are simulating.
```toml
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
```
### Modifying the output
To save some disk space, the run keeps the trajectory of only every 100th path it replaces in the archive (archive_every). A value above 0 of energy-file and order-file writes the energies and the order parameter of each path.
```toml
[output]
# Keep roughly 250 archived trajectories per ensemble, whatever
# the run length: without a cadence every replaced path is kept
# and a long run fills the disk.
archive_every = 100
energy-file = 100
order-file = 100
```
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "make-tis-files"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
[system]
units = "reduced"
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
[box]
periodic = [
false,
]
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
[forcefield]
description = "1D double well"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
[output]
# Keep roughly 250 archived trajectories per ensemble, whatever
# the run length: without a cadence every replaced path is kept
# and a long run fills the disk.
archive_every = 100
energy-file = 100
order-file = 100
[tis]
freq = 0.5
maxlength = 20000
allowmaxlength = false
zero_momentum = false
rescale_energy = false
sigma_v = -1
seed = 0
[initial-path]
method = "kick"
```
### Running the TIS simulation(s)
We will now run the TIS simulation. Create a new directory and place the input file (let’s call it tis-multiple.toml) here. Also, download the initial configuration initial.xyz and place it in the same folder. The TIS simulations are executed in two steps:
- Input TIS files are created for each path ensemble. This is done by running:
```pyretis
pyretis run -i tis-multiple.toml
```
- which will create files named tis-001.toml, tis-002.toml and so on. Each one of these files defines a TIS simulation for a specific path ensemble.
- Running the PyRETIS TIS simulations for each path ensemble. The individual TIS simulations can be executed by running:
```pyretis
pyretis run -i tis-001.toml -p -f 001.log
```
- and so on. All these TIS simulations are independent and can, in priciple, be executed in parallel.
The -p option will display a progress bar for your simulation and -f will rename the log file.
### Analysing the TIS results
When all the TIS simulations are finished, analyse the collection with the original tis-multiple.toml input. For a description of the analysis specific keywords, we refer to the analysis section.
The analysis itself is performed using:
```pyretis
pyretis analyse -i tis-multiple.toml
```
This will produce a new folder, report which contains the results from the analysis. If you have latex installed, you can generate a pdf using the file tis_report.tex within the report folder. An example result is shown below.
## Creating and running the initial flux simulation with PyRETIS
The initial flux simulation measure effective crossings with the first interface. In PyRETIS such a simulation is requested by selecting the md-flux task.
This can be done by setting the following simulation settings:
```toml
[simulation]
task = "md-flux"
steps = 1000000
interfaces = [
-0.9,
]
```
The other sections will be similar to the sections already defined for the TIS simulation, however, we do not need to include the TIS section for the md-flux simulation. The full input file is given below.
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "md-flux"
steps = 1000000
interfaces = [
-0.9,
]
[system]
units = "lj"
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
high_friction = false
seed = 0
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.xyz"
[particles.velocity]
generate = "maxwell"
momentum = false
seed = 0
[particles.mass]
Ar = 1.0
[forcefield]
description = "1D double well potential"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
periodic = false
[output]
# Keep roughly 250 archived trajectories per ensemble, whatever
# the run length: without a cadence every replaced path is kept
# and a long run fills the disk.
archive_every = 5000
backup = "overwrite"
energy-file = 1000
order-file = 1000
cross-file = 1
trajectory-file = -1
```
### Running the md-flux simulation
We will now run the md-flux simulation. Create a new directory and place the input file (let’s call it flux.toml) here. Also, download the initial configuration initial.xyz and place it in the same folder. The md-flux simulation can now be executed using:
```pyretis
pyretis run -i flux.toml -p
```
### Analysing the md-flux results
When the md-flux simulation has finished, use flux.toml for analysis. For a description of the analysis specific keywords, we refer to the analysis section.
The analysis itself is performed using:
```pyretis
pyretis analyse -i flux.toml
```
This will produce a new folder, report which contains the results from the analysis. If you have latex installed, you can generate a pdf using the file md_flux_report.tex within the report folder. An example result is shown below.
## Improving the statistics
We can improve the statistics by running a longer simulation, with 1000000 steps in place of 20000. The TIS simulations run in one directory share its output.toml, which holds the state of the simulation run last, so restart can continue only that one. Start the TIS simulations over: in a new directory, or after pyretis tools clean (see user-guide-clean) in this one, set steps = 1000000 in the [simulation] section of tis-multiple.toml, and repeat the two steps that run the TIS simulations, and the analysis. Running one of the TIS inputs again with method = "kick" without cleaning stops with an error, since its ensemble directory holds the sampled cycles. Below we show an example for the crossing probability after 1000000 steps.
## Tested by
The TIS workflow is represented by examples/tests/test-internal/tis-multiple/ and examples/tests/test-internal/tis-restart/. The flux workflow is represented by examples/tests/test-internal/mdflux-restart/. These checks are shorter than a production tutorial run; they verify startup, restart, analysis, and output consistency.
## References
Titus S. van Erp, Dynamical Rare Event Simulation Techniques for Equilibrium and Nonequilibrium Systems, Advances in Chemical Physics, 151, pp. 27-60, 2012, https://doi.org/10.1002/9781118309513.ch2
https://arxiv.org/abs/1101.0927
---
# Umbrella Sampling
This example will simply calculate the free energy profile in a given, known, potential using umbrella sampling. The results we will obtain are shown in the figure below: the potential energy V(x) and the probability density.
Verification status: smoke – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/umbrella_sampling/.
- Edit first: umbrella_sampling_mc.py for umbrella windows, force-field parameters, and the number of Monte Carlo cycles.
- Run: python umbrella_sampling_mc.py.
- Expected output: terminal progress for each umbrella window and a plot of the reconstructed probability/free-energy profile.
- Short check: examples/run-tutorials.sh starts this script with a timeout.
- Related checks: no dedicated heavy-test fixture yet; see example-test-status.
We begin by importing the PyRETIS library and numpy and matplotlib which we will use for some additional numerical methods and for plotting.
```
import numpy as np
from matplotlib import pyplot as plt
from pyretis.core import System, RandomGenerator, Particles
from pyretis.simulation.mc_simulation import UmbrellaWindowSimulation
from pyretis.forcefield import ForceField
from pyretis.forcefield.potentials import DoubleWell, RectangularWell
from pyretis.analysis import histogram, match_all_histograms
```
First, we set up the system by defining the units we will use and adding a particle (labeled as ‘X’) at a specified position.
```
mysystem = System(temperature=500, units='eV/K')
mysystem.particles = Particles(dim=1)
mysystem.add_particle(name='X', pos=np.array([-0.7]))
```
Next we define the force field in terms of potential functions. Here, we create the unbiased potential - a double well potential - and a biased version of the potential where a rectangular well is used. Finally, the biased force field is attached to the system.
```
potential_dw = DoubleWell(a=1, b=1, c=0.02)
potential_rw = RectangularWell()
forcefield = ForceField(desc='Double well', potential=[potential_dw])
forcefield_bias = ForceField(desc='Double well with rectangular bias',
potential=[potential_dw, potential_rw])
mysystem.forcefield = forcefield_bias
```
In order to run the actual simulation, we need to specify some additional settings, like where the umbrella windows should be placed and how many cycles we need to perform:
```
umbrellas = [[-1.0, -0.4], [-0.5, -0.2], [-0.3, 0.0],
[-0.1, 0.2], [0.1, 0.4], [0.3, 0.6], [0.5, 1.0]]
n_umb = len(umbrellas)
MINCYCLES = 1e4 # Number of MC steps to perform.
MAXDX = 0.1 # Maximum allowed displacement in the MC step(s).
```
and we create a random number generator for use in the umbrella simulation:
```
RANDSEED = 1 # Seed for the random number generator:
RGEN = RandomGenerator(seed=RANDSEED)
```
We are now ready to run the simulation. We will do this by looping over the umbrella windows we defined,
```
trajectory, energy = [], [] # For storing trajectories & energies.
for i, umbrella in enumerate(umbrellas):
print('Running umbrealla no: {} of {}. Location: {}'.format(i + 1, n_umb,
umbrella))
# Move rectangular potential to correct place:
params = {'left': umbrella[0], 'right': umbrella[1]}
mysystem.forcefield.update_potential_parameters(potential_rw, params)
mysystem.potential() # Re-calculate potential energy.
over = umbrellas[min(i + 1, n_umb - 1)][0] # Position we must cross.
simulation = UmbrellaWindowSimulation(
ensemble={
'system': mysystem,
'rgen': RGEN
},
settings={
'simulation': {
'task': 'umbrella',
'umbrella': umbrella,
'overlap': over,
'maxdx': MAXDX
}
},
controls={'steps': MINCYCLES}
)
# Also create empy list for storing some data:
traj, ener = [], []
for result in simulation.run():
for pos in mysystem.pos:
traj.append(pos)
ener.append(mysystem.particles.vpot)
trajectory.append(np.array(traj))
energy.append(np.array(ener))
print('Done. Cycles: {}'.format(simulation.cycle['stepno']))
```
The simulation is now done, and we can do the analysis and plot the results. For the analysis we match the histograms:
```
# We can now post-process the simulation output:
BINS = 100
LIM = (-1.1, 1.1)
histograms = [histogram(traj, bins=BINS, limits=LIM) for traj in trajectory]
# Extract the bins (the midpoints) and the bin-width:
bin_x = histograms[0][-1]
dbin = bin_x[1] - bin_x[0]
# We are going to match these histograms:
print('Matching histograms...')
histograms_s, _, hist_avg = match_all_histograms(histograms, umbrellas)
```
And we finally plot the results:
```
print('Plotting matched histograms')
fig = plt.figure()
axs = fig.add_subplot(111)
axs.set_yscale('log')
axs.set_xlabel('Position ($x$)', fontsize='large')
axs.set_ylabel('Matched histograms', fontsize='large')
colors = ['blue', 'green', 'darkviolet', 'brown', 'gray', 'crimson', 'cyan']
for i, histo in enumerate(histograms_s):
axs.bar(bin_x - 0.5 * dbin, histo, dbin, color=colors[i],
alpha=0.6, log=True)
axs.plot(bin_x, hist_avg, lw=7, color='orangered', alpha=0.6,
label='Average after matching')
axs.legend()
plt.xlim((-1.1, 1.1))
print('Plotting the free energy')
fig2 = plt.figure()
ax2 = fig2.add_subplot(111)
XPOT = np.linspace(-2, 2, 1000)
free = -np.log(hist_avg) / mysystem.temperature['beta'] # The free energy.
# Set up unbiased potential for plotting:
VPOT = []
for xi in XPOT:
mysystem.pos = xi
VPOT.append(forcefield.evaluate_potential(mysystem))
VPOT = np.array(VPOT)
free += (VPOT.min() - free.min())
ax2.plot(XPOT, VPOT, 'blue', lw=3, label='Unbiased potential', alpha=0.5)
ax2.plot(bin_x, free, lw=7, alpha=0.5, color='green', label='Free energy')
ax2.set_xlabel('Position ($x$)', fontsize='large')
ax2.set_ylabel('Potential energy ($V(x)$) / eV', fontsize='large')
ax2.legend()
plt.xlim((-1.1, 1.1))
plt.ylim((-0.3, 0.05))
plt.show()
```
## Tested by
This tutorial is covered by the tutorial smoke runner, which verifies that the script starts and performs Monte Carlo cycles with the current PyRETIS installation. Add a shortened fixture under examples/tests/ before treating umbrella sampling as part of the heavy regression suite.
---
# Using C or FORTRAN
In this section, we will show some examples on how C or FORTRAN can be used together with Python and PyRETIS. We show examples on:
- Tutorial quick start
- Creating a new Molecular Dynamics engine with C or FORTRAN
- Writing a new engine with FORTRAN
- Step 1. Creating the FORTRAN code
- Step 2. Creating the Makefile and compiling
- Step 3. Creating a new Python class for the engine
- Writing a new engine with C
- Step 1. Creating the C code
- Step 2. Creating a setup.py file and compiling
- Step 3. Creating a new Python class for the engine
- Comparison of the FORTRAN and C code
- Creating a force field with C or FORTRAN
- Writing a new potential with FORTRAN
- Step 1. Creating the FORTRAN code
- Step 2. Creating the Makefile and compiling
- Step 3. Creating a new Python class for the potential
- Writing a new potential with C
- Step 1. Creating the C-code
- Step 2. Creating a setup.py file and compiling
- Step 3. Creating a new Python class for the potential
- Comparison of the C, FORTRAN and a Numpy implementation
- OpenMP: Parallel evaluation of the force
- Tested by
Verification status: smoke – see example-tutorial-map.
## Tutorial quick start
- Best starting point: examples/tutorials/extending/integrator-vv/ for custom engines and examples/tutorials/extending/lennard-jones/ for custom potentials.
- Requirements: a C compiler for C extensions or f2py/FORTRAN for FORTRAN extensions.
- Edit first: the compiled source file, the build file (setup.py or Makefile), and the Python wrapper class.
- Build: use python setup.py build_ext --inplace for C examples or make for the FORTRAN examples.
- Run: execute the local test or MD script in the chosen tutorial directory, for example python test_vvintegrator.py.
- Expected output: an importable extension module and a Python class that can be referenced from a PyRETIS input file.
- Related checks: extension tests in the tutorial directories and representative internal MD checks; see example-test-status.
## Creating a new Molecular Dynamics engine with C or FORTRAN
In this example, we will create a new molecular dynamics engine using C or FORTRAN. As described in the user guide we have to make a new class and implement a method which performs an integration step. When we now make use of C or FORTRAN in a PyRETIS engine, we essentially just call a method from an external library which we have prepared ourselves (in C or FORTRAN).
In this part, we are here going to create such a library and access it from a piece of Python code. We are going to complete three steps:
- We write the code responsible for the actual integration in an external library.
- We compile the code. Here we do one of the following:
- For C, we create a setup.py which is used to compile.
- For FORTRAN, make use of a Makefile and the program f2py.
- We write a Python module containing a new pyretis.engines.internal.MDEngine sub-class representing the new engine. This class makes use of the code which we created in steps 1 and 2.
### Writing a new engine with FORTRAN
We will now create a new engine by writing the external library in FORTRAN. This is done in the following steps:
- Step 1. Creating the FORTRAN code
- Step 2. Creating the Makefile and compiling
- Step 3. Creating a new Python class for the engine
#### Step 1. Creating the FORTRAN code
The FORTRAN code for creating the new integrator is relatively simple. We can assume that we are handed positions, velocities, and forces as double precision arrays and that we can directly make use of them. Since we need to evaluate the forces during the Velocity Verlet integration, the integration routine is split up in two steps (before/after the evaluation of the forces). The responsibility for updating the forces between these two steps is delegated to the Python script which we will create below.
```fortran
! Copyright (c) 2023, PyRETIS Development Team.
! Distributed under the LGPLv2.1+ License. See LICENSE for more info.
module vvintegrator
implicit none
private
public :: step1, step2
contains
subroutine step1(pos, vel, force, imass, delta_t, half_delta_t, n, d, dpos, dvel)
! Part 1 of the velocity verlet update.
implicit none
integer, intent(in) :: n, d
double precision, dimension(n, d), intent(in) :: pos, vel, force
double precision, dimension(n), intent(in) :: imass
double precision, intent(in) :: delta_t, half_delta_t
double precision, dimension(n, d), intent(out) :: dvel, dpos
integer :: i
dvel = 0.0D0
dpos = 0.0D0
do i=1,d
dvel(:,i) = vel(:,i) + half_delta_t * force(:,i) * imass(:)
dpos(:,i) = pos(:,i) + delta_t * dvel(:,i)
end do
end subroutine step1
subroutine step2(vel, force, imass, half_delta_t, n, d, dvel)
! Part 2 of the velocity verlet update.
implicit none
integer, intent(in) :: n, d
double precision, dimension(n, d), intent(in) :: vel, force
double precision, dimension(n), intent(in) :: imass
double precision, intent(in) :: half_delta_t
double precision, dimension(n, d), intent(out) :: dvel
integer :: i
dvel = 0.0D0
do i=1,d
dvel(:,i) = vel(:,i) + half_delta_t * force(:,i) * imass(:)
end do
end subroutine step2
end module vvintegrator
```
#### Step 2. Creating the Makefile and compiling
In order to compile the FORTRAN code created in the previous step, we make use of f2py.
Note that in some cases you might actually need to use f2py3 (or, for a specific version: f2py3.7 etc.) to compile the FORTRAN code so that it can be used with the version of Python you are using to run this example. You will then have to modify the F2PY = f2py setting in the Makefile
```make
F2PY = f2py
# numpy>=2 f2py uses the meson backend and auto-detects the Fortran
# compiler; the old numpy.distutils --fcompiler flag was removed.
F2PY_FLAGS =
MODULE=vvintegrator
all: ${MODULE}.so
${MODULE}.so: ${MODULE}.f90
${F2PY} ${F2PY_FLAGS} -c $< -m ${MODULE}
clean:
find -name \*.so -delete
find -name \*.pyc -delete
find -name __pycache__ -delete
-rm thermo.txt*
-rm traj.xyz*
-rm energy.txt*
-rm pyretis.log*
-rm pyretis.restart*
-rm out.rst*
```
Note that the FORTRAN compiler is specified using --fcompiler=gfortran and other choices can be seen by running:
```pyretis
f2py -c --help-fcompiler
```
Further, the Makefile assumes that the FORTRAN module is named vvintegrator.f90.
#### Step 3. Creating a new Python class for the engine
For the Python class representing the engine, we just import the module we just compiled and make use of the two methods defined within that module. Note that we here make an explicit call to system.potential_and_force() so that forces are updated between the two “steps” of the Velocity Verlet integration.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of using a integration routine implemented in FORTRAN."""
import logging
import os
import sys
from pyretis.engines import MDEngine
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
try:
from vvintegrator import vvintegrator
except ImportError:
MSG = ('Could not import external FORTRAN library.'
'\nPlease compile with "make"!')
logger.critical(MSG)
raise ImportError(MSG)
__all__ = ['VelocityVerletF']
class VelocityVerletF(MDEngine):
"""VelocityVerletF(MDEngine).
This class defines the Velocity Verlet integrator.
Attributes
----------
timestep : float
The time step.
half_timestep : float
The half of the timestep.
desc : string
Description of the integrator.
"""
def __init__(self, timestep,
desc='The velocity verlet integrator (FORTRAN)'):
"""Set up the Velocity Verlet integrator.
Parameters
----------
timestep : float
The time step.
desc : string
Description of the integrator.
"""
super().__init__(timestep, desc, dynamics='NVE')
self.half_timestep = self.timestep * 0.5
def integration_step(self, system):
"""Velocity Verlet integration, one time step.
Parameters
----------
system : object like :py:class:`.System`
The system to integrate/act on. Assumed to have a particle
list in `system.particles`.
Returns
-------
out : None
Does not return anything, but alters the state of the given
`system`.
"""
particles = system.particles
particles.pos, particles.vel = vvintegrator.step1(
particles.pos,
particles.vel,
particles.force,
particles.imass,
self.timestep,
self.half_timestep
)
system.potential_and_force()
particles.vel = vvintegrator.step2(
particles.vel,
particles.force,
particles.imass,
self.half_timestep
)
```
You can now use the new integrator, for instance by adding the following engine section to the input file:
```toml
[engine]
class = "VelocityVerletF"
timestep = 0.002
module = "vvintegratorf.py"
```
### Writing a new engine with C
We will now create a new engine by writing the external library in C. This is done in the following steps:
- Step 1. Creating the C code
- Step 2. Creating a setup.py file and compiling
- Step 3. Creating a new Python class for the engine
#### Step 1. Creating the C code
The C code is more involved, and perhaps cumbersome compared to the FORTRAN code above. We are here explicitly assuming that our system is going to be 3D and we split the Velocity Verlet integration into two steps as in the FORTRAN code.
```c
/* Copyright (c) 2023, PyRETIS Development Team.
Distributed under the LGPLv2.1+ License. See LICENSE for more info. */
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include
#include
#include
// Forward function declaration.
static PyObject *step1(PyObject *self, PyObject *args);
static PyObject *step2(PyObject *self, PyObject *args);
// Boilerplate: function list.
static PyMethodDef vvmethods[] = {
{ "step1", step1, METH_VARARGS, "Velocity Verlet - update positions and velocity"},
{ "step2", step2, METH_VARARGS, "Velocity Verlet - update velocity"},
{ NULL, NULL, 0, NULL } /* Sentinel */
};
static struct PyModuleDef vvintegrator =
{
PyModuleDef_HEAD_INIT,
"vvintegrator",
"Velocity Verlet Integrator",
-1,
vvmethods
};
// Boilerplate: Module initialization.
PyMODINIT_FUNC PyInit_vvintegrator(void) {
import_array();
return PyModule_Create(&vvintegrator);
}
static PyObject *step1(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim;
npy_float64 delta_t, half_delta_t;
PyArrayObject *py_pos, *py_vel, *py_force, *py_imass;
// Internal variables:
npy_int64 i, xi, yi, zi;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!ddll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_vel,
&PyArray_Type, &py_force,
&PyArray_Type, &py_imass,
&delta_t, &half_delta_t,
&npart, &dim)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *vel = (npy_float64*)PyArray_DATA(py_vel);
npy_float64 *force = (npy_float64*)PyArray_DATA(py_force);
npy_float64 *imass = (npy_float64*)PyArray_DATA(py_imass);
// Update positions and velocity
for(i = 0; i < npart; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
vel[xi] += half_delta_t * force[xi] * imass[i];
vel[yi] += half_delta_t * force[yi] * imass[i];
vel[zi] += half_delta_t * force[zi] * imass[i];
pos[xi] += delta_t * vel[xi];
pos[yi] += delta_t * vel[yi];
pos[zi] += delta_t * vel[zi];
}
Py_RETURN_NONE;
}
static PyObject *step2(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim;
npy_float64 half_delta_t;
PyArrayObject *py_vel, *py_force, *py_imass;
// Internal variables:
npy_int64 i, xi, yi, zi;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!dll",
&PyArray_Type, &py_vel,
&PyArray_Type, &py_force,
&PyArray_Type, &py_imass,
&half_delta_t,
&npart, &dim)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *vel = (npy_float64*)PyArray_DATA(py_vel);
npy_float64 *force = (npy_float64*)PyArray_DATA(py_force);
npy_float64 *imass = (npy_float64*)PyArray_DATA(py_imass);
// Update positions and velocity
for(i = 0; i < npart; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
vel[xi] += half_delta_t * force[xi] * imass[i];
vel[yi] += half_delta_t * force[yi] * imass[i];
vel[zi] += half_delta_t * force[zi] * imass[i];
}
Py_RETURN_NONE;
}
```
As can be seen in the C-code, there is some boilerplate code and we make use of PyArg_ParseTuple in order to parse the parameters to the function.
#### Step 2. Creating a setup.py file and compiling
The C-code can be compiled using a setup.py file.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""A script for building the C Velocity Verlet extension."""
from setuptools import (
setup,
Extension,
)
import numpy as np
CMODULE = Extension(
'vvintegrator',
sources=['vvintegrator.c'],
extra_compile_args=['-Ofast', '-march=native']
)
setup(
name='PyRETIS Velocity Verlet C extension',
description='C extension for the Velocity Verlet integrator.',
ext_modules=[CMODULE],
include_dirs=[np.get_include()],
)
```
The setup.py file is used to compile via the command
```pyretis
python setup.py build_ext --inplace
```
Here, build_ext is used to tell setup.py to compile the C extension and the --inplace will put the compiled extensions into the directory you have the source code in.
#### Step 3. Creating a new Python class for the engine
The final step is to create a Python class which is making use of the C-code.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of using a integration routine implemented in C."""
import logging
import os
import sys
from pyretis.engines import MDEngine
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
try:
import vvintegrator
except ImportError:
MSG = ('Could not import external C library.'
'\nPlease compile with: "python setup.py build_ext --inplace"')
logger.critical(MSG)
raise ImportError(MSG)
__all__ = ['VelocityVerletC']
class VelocityVerletC(MDEngine):
"""VelocityVerletC(MDEngine).
This class defines the Velocity Verlet integrator.
Attributes
----------
timestep : float
The time step.
half_timestep : float
The half of the time step.
desc : string
Description of the integrator.
"""
def __init__(self, timestep,
desc='The velocity verlet integrator (C)'):
"""Set up the Velocity Verlet integrator.
Parameters
----------
timestep : float
The time step.
desc : string
Description of the integrator.
"""
super().__init__(timestep, desc, dynamics='NVE')
self.half_timestep = self.timestep * 0.5
def integration_step(self, system):
"""Velocity Verlet integration, one time step.
Parameters
----------
system : object like :py:class:`.System`
The system to integrate/act on. Assumed to have a particle
list in `system.particles`.
Returns
-------
out : None
Does not return anything, but alters the state of the given
`system`.
"""
particles = system.particles
vvintegrator.step1(
particles.pos,
particles.vel,
particles.force,
particles.imass,
self.timestep,
self.half_timestep,
particles.npart,
particles.dim
)
system.potential_and_force()
vvintegrator.step2(
particles.vel,
particles.force,
particles.imass,
self.half_timestep,
particles.npart,
particles.dim
)
```
You can now use the new integrator, for instance by adding the following engine section to the input file:
```toml
[engine]
class = "VelocityVerletC"
timestep = 0.002
module = "vvintegratorc.py"
```
### Comparison of the FORTRAN and C code
As you may have noticed, the FORTRAN code we have added is somewhat more to the point compared to the C-code we wrote. The C-code could, for instance, be simplified by making use of Cython, and you are encouraged to try this out.
The two Python classes we created are very similar. This means that it is relatively little work to switch between the two libraries from Python’s point of view. This is also evident in the diff of the two modules shown below.
```udiff
--- /builds/pyretis/pyretis/docs/_static/examples/extending-example/engine/vvintegratorf.py
+++ /builds/pyretis/pyretis/docs/_static/examples/extending-example/engine/vvintegratorc.py
@@ -1,6 +1,6 @@
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
-"""Example of using a integration routine implemented in FORTRAN."""
+"""Example of using a integration routine implemented in C."""
import logging
import os
import sys
@@ -9,19 +9,19 @@
logger.addHandler(logging.NullHandler())
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
try:
- from vvintegrator import vvintegrator
+ import vvintegrator
except ImportError:
- MSG = ('Could not import external FORTRAN library.'
- '\nPlease compile with "make"!')
+ MSG = ('Could not import external C library.'
+ '\nPlease compile with: "python setup.py build_ext --inplace"')
logger.critical(MSG)
raise ImportError(MSG)
-__all__ = ['VelocityVerletF']
+__all__ = ['VelocityVerletC']
-class VelocityVerletF(MDEngine):
- """VelocityVerletF(MDEngine).
+class VelocityVerletC(MDEngine):
+ """VelocityVerletC(MDEngine).
This class defines the Velocity Verlet integrator.
@@ -30,14 +30,14 @@
timestep : float
The time step.
half_timestep : float
- The half of the timestep.
+ The half of the time step.
desc : string
Description of the integrator.
"""
def __init__(self, timestep,
- desc='The velocity verlet integrator (FORTRAN)'):
+ desc='The velocity verlet integrator (C)'):
"""Set up the Velocity Verlet integrator.
Parameters
@@ -68,18 +68,22 @@
"""
particles = system.particles
- particles.pos, particles.vel = vvintegrator.step1(
+ vvintegrator.step1(
particles.pos,
particles.vel,
particles.force,
particles.imass,
self.timestep,
- self.half_timestep
+ self.half_timestep,
+ particles.npart,
+ particles.dim
)
system.potential_and_force()
- particles.vel = vvintegrator.step2(
+ vvintegrator.step2(
particles.vel,
particles.force,
particles.imass,
- self.half_timestep
+ self.half_timestep,
+ particles.npart,
+ particles.dim
)
```
## Creating a force field with C or FORTRAN
Evaluation of the forces will typically be the most time-consuming part of your simulation. If you are running PyRETIS with internal engines, there can be a lot to gain by implementing the force evaluation in C or FORTRAN. In this section, we will give a short example of how this can be done. The approach is similar to the approach for the engine, and we will do the following:
- We write the code responsible for the actual evaluation of forces in an external library.
- We compile the code. Here we do one of the following:
- For C, we create a setup.py which is used to compile.
- For FORTRAN, make use of a Makefile and the program f2py.
- We write a python module containing a new PotentialFunction sub-class representing the new engine. This class makes use of the code which we created in steps 1 and 2.
As you may have noticed, in step 3 above, we said that we will sub-class PotentialFunction. This is because a force field is made up of several potential functions, and the actual computations are carried out by the potential functions, and not the force field. So effectively, we will be creating new potential functions to use in a force field. To be concrete, we will implement the Lennard-Jones potential function.
### Writing a new potential with FORTRAN
We will now create a new potential function by writing the external library in FORTRAN. This is done in the following steps:
- Step 1. Creating the FORTRAN code
- Step 2. Creating the Makefile and compiling
- Step 3. Creating a new Python class for the potential
#### Step 1. Creating the FORTRAN code
For the new potential function, we add methods for calculating the potential and the force. In addition, we calculate the virial. Note: Here we also add a method to apply the periodic boundaries. Ideally, we should use the Box object, but in order to not complicate the FORTRAN code, by calling methods from the Python class, we simply make a new FORTRAN method to do this.
```fortran
! Copyright (c) 2023, PyRETIS Development Team.
! Distributed under the LGPLv2.1+ License. See LICENSE for more info.
module ljfortranp
implicit none
private
public :: potential, force, potential_and_force
public :: pbc_dist
contains
function pbc_dist(v, box, ibox) result(dist)
! Function to apply periodic boundaries to a given displacement.
!
! Parameters
! ----------
! v : the displacement to pbc-wrap
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
!
! Returns
! -------
! dist : the pbc-wrapped displacement
implicit none
double precision :: dist
double precision, intent(in) :: v, box, ibox
dist = v - nint(v * ibox) * box
end function pbc_dist
subroutine potential(pos, box, ibox, lj3, lj4, offset, rcut2, ptype, n, d, p, vpot)
! Function to evaluate the Lennard-Jones potential
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj3 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**12
! lj4 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! offset : Potential energy shift
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! vpot : the potential energy
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: box, ibox
double precision, dimension(p, p), intent(in) :: lj3, lj4, offset, rcut2
integer, dimension(n), intent(in) :: ptype
double precision, intent(out) :: vpot
double precision :: rsq, r2inv, r6inv, dx, dy, dz
double precision, dimension(:), pointer :: x, y, z
integer :: i, j, itype, jtype
vpot = 0.0D0
x => pos(1:n,1)
y => pos(1:n,2)
z => pos(1:n,3)
do i=1,n-1
itype = ptype(i) + 1
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv**3
vpot = vpot + r6inv * (lj3(jtype, itype) * r6inv - lj4(jtype, itype))
vpot = vpot - offset(jtype, itype)
end if
end do
end do
end subroutine potential
subroutine force(pos, box, ibox, lj1, lj2, rcut2, ptype, n, d, p, forces, virial)
! Function to evaluate the Lennard-Jones force
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj1 : Lennard-Jones parameters, 48.0 * epsilonij * sigmaij**12
! lj2 : Lennard-Jones parameters, 24.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! forces : the Lennard-Jones forces on the particles
! virial : the virial matrix
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(out) :: forces
double precision, dimension(d, d), intent(out) :: virial
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: box, ibox
double precision, dimension(p, p), intent(in) :: lj1, lj2, rcut2
integer, dimension(n), intent(in) :: ptype
double precision, dimension(:), pointer :: x, y, z, fx, fy, fz
double precision :: rsq, r2inv, r6inv, forcelj, dx, dy, dz
double precision :: forceij_x, forceij_y, forceij_z
integer :: i, j, itype, jtype
x => pos(1:n, 1)
y => pos(1:n,2)
z => pos(1:n,3)
fx => forces(1:n,1)
fy => forces(1:n,2)
fz => forces(1:n,3)
forces = 0.0D0
virial = 0.0D0
do i=1,n-1
itype = ptype(i) + 1
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv*r2inv*r2inv
forcelj = r2inv * r6inv * (lj1(jtype, itype) * r6inv - lj2(jtype, itype))
forceij_x = forcelj * dx
forceij_y = forcelj * dy
forceij_z = forcelj * dz
fx(i) = fx(i) + forceij_x
fy(i) = fy(i) + forceij_y
fz(i) = fz(i) + forceij_z
fx(j) = fx(j) - forceij_x
fy(j) = fy(j) - forceij_y
fz(j) = fz(j) - forceij_z
! accumulate for the virial:
virial(1,1) = virial(1,1) + forceij_x * dx
virial(1,2) = virial(1,2) + forceij_x * dy
virial(1,3) = virial(1,3) + forceij_x * dz
virial(2,1) = virial(2,1) + forceij_y * dx
virial(2,2) = virial(2,2) + forceij_y * dy
virial(2,3) = virial(2,3) + forceij_y * dz
virial(3,1) = virial(3,1) + forceij_z * dx
virial(3,2) = virial(3,2) + forceij_z * dy
virial(3,3) = virial(3,3) + forceij_z * dz
end if
end do
end do
end subroutine force
subroutine potential_and_force(pos, box, ibox, lj1, lj2, lj3, lj4, offset, rcut2, ptype, n, d, p, forces, virial, vpot)
! Function to evaluate the Lennard-Jones force and potential
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj1 : Lennard-Jones parameters, 48.0 * epsilonij * sigmaij**12
! lj2 : Lennard-Jones parameters, 24.0 * epsilonij * sigmaij**6
! lj3 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**12
! lj4 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! offset : Potential energy shift
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! vpot : the potential energy
! forces : the Lennard-Jones forces on the particles
! virial : the virial matrix
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(out) :: forces
double precision, dimension(d, d), intent(out) :: virial
double precision, intent(out) :: vpot
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: ibox, box
double precision, dimension(p, p), intent(in) :: lj1, lj2, lj3, lj4, offset, rcut2
double precision, dimension(:), pointer :: x, y, z, fx, fy, fz
integer, dimension(n), intent(in) :: ptype
double precision :: rsq, r2inv, r6inv, forcelj, dx, dy, dz
double precision :: forceij_x, forceij_y, forceij_z
integer :: i, j, itype, jtype
forces = 0.0D0
virial = 0.0D0
vpot = 0.0D0
x => pos(1:n,1)
y => pos(1:n,2)
z => pos(1:n,3)
fx => forces(1:n,1)
fy => forces(1:n,2)
fz => forces(1:n,3)
do i=1,n-1
itype = ptype(i) + 1
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv*r2inv*r2inv
forcelj = r2inv * r6inv * (lj1(jtype, itype) * r6inv - lj2(jtype, itype))
forceij_x = forcelj * dx
forceij_y = forcelj * dy
forceij_z = forcelj * dz
fx(i) = fx(i) + forceij_x
fy(i) = fy(i) + forceij_y
fz(i) = fz(i) + forceij_z
fx(j) = fx(j) - forceij_x
fy(j) = fy(j) - forceij_y
fz(j) = fz(j) - forceij_z
vpot = vpot + r6inv * (lj3(jtype, itype) * r6inv - lj4(jtype, itype))
vpot = vpot - offset(jtype, itype)
! accumulate for the virial:
virial(1,1) = virial(1,1) + forceij_x * dx
virial(1,2) = virial(1,2) + forceij_x * dy
virial(1,3) = virial(1,3) + forceij_x * dz
virial(2,1) = virial(2,1) + forceij_y * dx
virial(2,2) = virial(2,2) + forceij_y * dy
virial(2,3) = virial(2,3) + forceij_y * dz
virial(3,1) = virial(3,1) + forceij_z * dx
virial(3,2) = virial(3,2) + forceij_z * dy
virial(3,3) = virial(3,3) + forceij_z * dz
end if
end do
end do
end subroutine potential_and_force
end module ljfortranp
```
#### Step 2. Creating the Makefile and compiling
The step is almost identical to the compilation for the external engine. The Makefile is given below, and note here that we are assuming that the FORTRAN code is stored in a module named ljfortranp.f90.
```make
F2PY = f2py
# numpy>=2 f2py uses the meson backend and auto-detects the Fortran
# compiler; the old numpy.distutils --fcompiler flag was removed.
F2PY_FLAGS =
MODULE=ljfortranp
all: ${MODULE}.so
${MODULE}.so: ${MODULE}.f90
${F2PY} ${F2PY_FLAGS} -c $< -m ${MODULE}
clean:
find -name \*.so -delete
find -name \*.pyc -delete
find -name __pycache__ -delete
```
#### Step 3. Creating a new Python class for the potential
For the Python class representing the potential, we just import the module we compiled in the previous step.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of using a Lennard-Jones potential implemented in FORTRAN."""
import logging
import os
import sys
import numpy as np
from pyretis.forcefield.potentials import PairLennardJonesCut
from pyretis.forcefield.potentials.pairpotentials import (
generate_pair_interactions
)
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
try:
from ljfortranp import ljfortranp
except ImportError as exc:
MSG = ('Could not import external FORTRAN library.'
'\nPlease compile with "make"!')
raise ImportError(MSG) from exc
__all__ = ['PairLennardJonesCutFp']
class PairLennardJonesCutFp(PairLennardJonesCut):
r"""class PairLennardJonesCutFp(PairLennardJonesCut).
This class implements as simple Lennard-Jones 6-12 potential which
employs a simple cut-off and can be shifted. The potential energy
(:math:`V_\text{pot}`) is defined in the usual way for an
interacting pair of particles a distance :math:`r` apart,
.. math::
V_\text{pot} = 4 \varepsilon \left( x^{12} - x^{6} \right),
where :math:`x = \sigma/r` and :math:`\varepsilon`
and :math:`\sigma` are the potential parameters. The parameters are
stored as attributes of the potential and we store one set for each
kind of pair interaction. Parameters can be generated with a
specific mixing rule by the force field.
Attributes
----------
params : dict
The parameters for the potential. This dict is assumed to
contain parameters for pairs, i.e. for interactions.
_lj : dict of numpy.array
[1] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``48.0 * epsilon * sigma**12``
[2] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``24.0 * epsilon * sigma**6``
[3] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**12``
[4] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**6``
_offset : numpy.array
Potential values for shifting the potential if requested.
This is the potential evaluated at the cutoff.
_rcut2 : numpy.array
The squared cut-off for each interaction type.
Keys are the pairs (particle types) that may interact.
"""
def __init__(self, dim=3, shift=True, mixing='geometric',
desc='Lennard-Jones pair potential (fortran)'):
"""Set up the Lennard-Jones potential.
Parameters
----------
dim : int
The dimensionality to use.
shift : boolean
Determines if the potential should be shifted or not.
mixing : string
Determines how we should mix potential parameters.
"""
super().__init__(dim=dim, desc=desc, mixing=mixing)
self.shift = shift
self.ntype = 0
def set_parameters(self, parameters):
"""Update all parameters.
Here, we generate pair interactions, since that is what this
potential actually is using.
Parameters
----------
parameters : dict
The input base parameters.
"""
self.params = {}
pair_param = generate_pair_interactions(parameters, self.mixing)
self.ntype = max(int(np.sqrt(len(pair_param))), 2)
self._lj = {'1': np.zeros((self.ntype, self.ntype)),
'2': np.zeros((self.ntype, self.ntype)),
'3': np.zeros((self.ntype, self.ntype)),
'4': np.zeros((self.ntype, self.ntype))}
self._rcut2 = np.zeros_like(self._lj[1])
self._offset = np.zeros_like(self._lj[1])
for pair, value in pair_param.items():
eps_ij = value['epsilon']
sig_ij = value['sigma']
rcut = value['rcut']
self._lj[1][pair] = 48.0 * eps_ij * sig_ij**12
self._lj[2][pair] = 24.0 * eps_ij * sig_ij**6
self._lj[3][pair] = 4.0 * eps_ij * sig_ij**12
self._lj[4][pair] = 4.0 * eps_ij * sig_ij**6
self._rcut2[pair] = rcut**2
vcut = 0.0
if self.shift and not np.isclose(rcut, 0):
vcut = 4.0 * eps_ij * ((sig_ij / rcut)**12 -
(sig_ij / rcut)**6)
self._offset[pair] = vcut
self.params[pair] = pair_param[pair]
def potential(self, system):
"""Calculate the potential energy for the Lennard-Jones interaction.
Parameters
----------
system : object like `System` from `pyretis.core.system`
The system we are evaluating the potential in.
Returns
-------
v_pot : float
The potential energy.
"""
particles = system.particles
box = system.box
v_pot = ljfortranp.potential(
particles.pos,
box.length,
box.ilength,
self._lj,
self._offset,
self._rcut2,
particles.ptype,
particles.npart,
box.dim,
self.ntype
)
return v_pot
def force(self, system):
"""Calculate the force for the Lennard-Jones interaction.
We also calculate the virial here, since the force
is evaluated.
Parameters
----------
system : object like `System` from `pyretis.core.system`
The system we are evaluating the force in.
Returns
-------
forces : numpy.array
The forces on the particles.
virial : numpy.array
The virial obtained from the forces.
"""
particles = system.particles
box = system.box
forces, virial = ljfortranp.force(
particles.pos,
box.length,
box.ilength,
self._lj,
self._rcut2,
particles.ptype,
particles.npart,
box.dim,
self.ntype
)
return forces, virial
def potential_and_force(self, system):
"""Calculate potential and force for the Lennard-Jones interaction.
Since the force is evaluated, the virial is also calculated.
Parameters
----------
system : object like `System` from `pyretis.core.system`
The system we are evaluating the potential and force in.
Note
----
Currently, the virial is only calculated for all the particles.
It is not calculated as a virial per atom. The virial
per atom might be useful to obtain a local pressure or stress,
however, this needs some consideration. Perhaps it's best to
fully implement this as a method of planes or something similar.
Returns
-------
out[0] : float
The potential energy as a float.
out[1] : numpy.array
The force as a numpy.array of the same shape as the
positions in `particles.pos`.
out[2] : numpy.array
The virial, as a symmetric matrix with dimensions
(dim, dim) where dim is given by the box/system dimensions.
"""
particles = system.particles
box = system.box
forces, virial, vpot = ljfortranp.potential_and_force(
particles.pos,
box.length,
box.ilength,
self._lj,
self._offset,
self._rcut2,
particles.ptype,
particles.npart,
box.dim,
self.ntype
)
return vpot, forces, virial
```
You can now use the new potential, for instance by adding the following forcefield and potential sections to the input file:
```toml
[forcefield]
description = "Lennard-Jones evaluated in FORTRAN"
[[potential]]
class = "PairLennardJonesCutFp"
module = "ljpotentialfp.py"
shift = true
dim = 3
mixing = "geometric"
[potential.parameter.0]
sigma = 1
epsilon = 1
factor = 2.5
```
### Writing a new potential with C
We will now create a new potential function by writing the external library in C. This is done in the following steps:
- Step 1. Creating the C-code
- Step 2. Creating a setup.py file and compiling
- Step 3. Creating a new Python class for the potential
#### Step 1. Creating the C-code
We set up the C-code as in the engine example.
```c
/* Copyright (c) 2023, PyRETIS Development Team.
Distributed under the LGPLv2.1+ License. See LICENSE for more info. */
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include
#include
#include
// Forward function declaration.
static PyObject *potential(PyObject *self, PyObject *args);
static PyObject *force(PyObject *self, PyObject *args);
static PyObject *force_and_pot(PyObject *self, PyObject *args);
// Boilerplate: function list.
static PyMethodDef ljmethods[] = {
{ "potential", potential, METH_VARARGS, "Calculate potential energy."},
{ "force", force, METH_VARARGS, "Calculate force."},
{ "force_and_pot", force_and_pot, METH_VARARGS, "Calculate force and potential energy."},
{ NULL, NULL, 0, NULL } /* Sentinel */
};
static struct PyModuleDef ljc =
{
PyModuleDef_HEAD_INIT,
"ljc",
"Compute Lennard-Jones interactions",
-1,
ljmethods
};
// Boilerplate: Module initialization.
PyMODINIT_FUNC PyInit_ljc(void) {
import_array();
return PyModule_Create(&ljc);
}
static inline npy_float64 pbc_dist(npy_float64 x,
npy_float64 box,
npy_float64 ibox) {
return x - nearbyint(x * ibox) * box;
}
static PyObject *potential(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim, ntype;
npy_int64 itype, loc;
PyArrayObject *py_pos, *py_box, *py_ibox, *py_lj3, *py_lj4, *py_rcut2, *py_offset, *py_ptype;
// Internal variables:
npy_float64 dx, dy, dz, rsq, r2inv, r6inv;
npy_int64 i, xi, yi, zi;
npy_int64 j, xj, yj, zj;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!O!lll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_box,
&PyArray_Type, &py_ibox,
&PyArray_Type, &py_lj3,
&PyArray_Type, &py_lj4,
&PyArray_Type, &py_offset,
&PyArray_Type, &py_rcut2,
&PyArray_Type, &py_ptype,
&npart, &dim, &ntype)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *box = (npy_float64*)PyArray_DATA(py_box);
npy_float64 *ibox = (npy_float64*)PyArray_DATA(py_ibox);
npy_float64 *lj3 = (npy_float64*)PyArray_DATA(py_lj3);
npy_float64 *lj4 = (npy_float64*)PyArray_DATA(py_lj4);
npy_float64 *rcut2 = (npy_float64*)PyArray_DATA(py_rcut2);
npy_float64 *offset = (npy_float64*)PyArray_DATA(py_offset);
npy_int64 *ptype = (npy_int64*)PyArray_DATA(py_ptype);
npy_float64 vpot = 0.0;
// Calculate potential:
for(i = 0; i < npart-1; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
itype = ptype[i];
for (j = i + 1; j < npart; j++){
loc = itype + ntype * ptype[j];
xj = 3*j;
yj = 3*j + 1;
zj = 3*j + 2;
dx = pos[xi] - pos[xj];
dy = pos[yi] - pos[yj];
dz = pos[zi] - pos[zj];
dx = pbc_dist(dx, box[0], ibox[0]);
dy = pbc_dist(dy, box[1], ibox[1]);
dz = pbc_dist(dz, box[2], ibox[2]);
rsq = dx*dx + dy*dy + dz*dz;
if (rsq < rcut2[loc]){
r2inv = 1.0 / rsq;
r6inv = r2inv*r2inv*r2inv;
vpot += r6inv * (lj3[loc] * r6inv - lj4[loc]) - offset[loc];
}
}
}
/* Clean up */
/* Build the output tuple */
PyObject *ret = Py_BuildValue("d", vpot);
return ret;
}
static PyObject *force(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim, ntype;
npy_int64 itype, loc;
PyArrayObject *py_pos, *py_box, *py_ibox, *py_force, *py_virial;
PyArrayObject *py_lj1, *py_lj2, *py_rcut2, *py_ptype;
// Internal variables:
npy_float64 dx, dy, dz, rsq, r2inv, r6inv;
npy_float64 forcelj, fx, fy, fz;
npy_int64 i, xi, yi, zi;
npy_int64 j, xj, yj, zj;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!O!O!lll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_box,
&PyArray_Type, &py_ibox,
&PyArray_Type, &py_lj1,
&PyArray_Type, &py_lj2,
&PyArray_Type, &py_rcut2,
&PyArray_Type, &py_ptype,
&PyArray_Type, &py_force,
&PyArray_Type, &py_virial,
&npart, &dim, &ntype)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *force = (npy_float64*)PyArray_DATA(py_force);
npy_float64 *virial = (npy_float64*)PyArray_DATA(py_virial);
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *box = (npy_float64*)PyArray_DATA(py_box);
npy_float64 *ibox = (npy_float64*)PyArray_DATA(py_ibox);
npy_float64 *lj1 = (npy_float64*)PyArray_DATA(py_lj1);
npy_float64 *lj2 = (npy_float64*)PyArray_DATA(py_lj2);
npy_float64 *rcut2 = (npy_float64*)PyArray_DATA(py_rcut2);
npy_int64 *ptype = (npy_int64*)PyArray_DATA(py_ptype);
// Create array for forces:
//PyObject *ret = PyArray_SimpleNew(dim, test, NPY_DOUBLE);
// Zero virial:
for(i = 0; i < dim*dim; i++) {
virial[i] = 0;
}
// Set all forces to zero.
for(i = 0; i < npart; i++) {
force[3*i] = 0;
force[3*i+1] = 0;
force[3*i+2] = 0;
}
// Calculate forces:
for(i = 0; i < npart-1; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
itype = ptype[i];
for (j = i + 1; j < npart; j++){
loc = itype + ntype * ptype[j];
xj = 3*j;
yj = 3*j + 1;
zj = 3*j + 2;
dx = pos[xi] - pos[xj];
dy = pos[yi] - pos[yj];
dz = pos[zi] - pos[zj];
dx = pbc_dist(dx, box[0], ibox[0]);
dy = pbc_dist(dy, box[1], ibox[1]);
dz = pbc_dist(dz, box[2], ibox[2]);
rsq = dx*dx + dy*dy + dz*dz;
if (rsq < rcut2[loc]){
r2inv = 1.0 / rsq;
r6inv = r2inv*r2inv*r2inv;
forcelj = r2inv * r6inv * (lj1[loc] * r6inv - lj2[loc]);
fx = forcelj * dx;
fy = forcelj * dy;
fz = forcelj * dz;
force[xi] += fx;
force[yi] += fy;
force[zi] += fz;
force[xj] -= fx;
force[yj] -= fy;
force[zj] -= fz;
virial[0] += fx*dx;
virial[1] += fx*dy;
virial[2] += fx*dz;
virial[3] += fy*dx;
virial[4] += fy*dy;
virial[5] += fy*dz;
virial[6] += fz*dx;
virial[7] += fz*dy;
virial[8] += fz*dz;
}
}
}
Py_RETURN_NONE;
}
static PyObject *force_and_pot(PyObject *self, PyObject *args) {
// Input variables:
npy_int64 npart, dim, ntype;
npy_int64 itype, loc;
PyArrayObject *py_pos, *py_box, *py_ibox, *py_force, *py_virial;
PyArrayObject *py_lj1, *py_lj2, *py_lj3, *py_lj4, *py_rcut2, *py_offset, *py_ptype;
// Internal variables:
npy_float64 dx, dy, dz, rsq, r2inv, r6inv;
npy_float64 forcelj, fx, fy, fz;
npy_int64 i, xi, yi, zi;
npy_int64 j, xj, yj, zj;
// Parse arguments.
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!O!O!O!O!O!lll",
&PyArray_Type, &py_pos,
&PyArray_Type, &py_box,
&PyArray_Type, &py_ibox,
&PyArray_Type, &py_lj1,
&PyArray_Type, &py_lj2,
&PyArray_Type, &py_lj3,
&PyArray_Type, &py_lj4,
&PyArray_Type, &py_offset,
&PyArray_Type, &py_rcut2,
&PyArray_Type, &py_ptype,
&PyArray_Type, &py_force,
&PyArray_Type, &py_virial,
&npart, &dim, &ntype)){
return NULL;
}
// Get underlying arrays from numpy arrays.
npy_float64 *force = (npy_float64*)PyArray_DATA(py_force);
npy_float64 *virial = (npy_float64*)PyArray_DATA(py_virial);
npy_float64 *pos = (npy_float64*)PyArray_DATA(py_pos);
npy_float64 *box = (npy_float64*)PyArray_DATA(py_box);
npy_float64 *ibox = (npy_float64*)PyArray_DATA(py_ibox);
npy_float64 *lj1 = (npy_float64*)PyArray_DATA(py_lj1);
npy_float64 *lj2 = (npy_float64*)PyArray_DATA(py_lj2);
npy_float64 *lj3 = (npy_float64*)PyArray_DATA(py_lj3);
npy_float64 *lj4 = (npy_float64*)PyArray_DATA(py_lj4);
npy_float64 *offset = (npy_float64*)PyArray_DATA(py_offset);
npy_float64 *rcut2 = (npy_float64*)PyArray_DATA(py_rcut2);
npy_int64 *ptype = (npy_int64*)PyArray_DATA(py_ptype);
npy_float64 vpot = 0.0;
// Create array for forces:
//PyObject *ret = PyArray_SimpleNew(dim, test, NPY_DOUBLE);
// Zero virial:
for(i = 0; i < dim*dim; i++) {
virial[i] = 0;
}
// Set all forces to zero.
for(i = 0; i < npart; i++) {
force[3*i] = 0;
force[3*i+1] = 0;
force[3*i+2] = 0;
}
// Calculate forces:
for(i = 0; i < npart-1; i++){
xi = 3*i;
yi = 3*i + 1;
zi = 3*i + 2;
itype = ptype[i];
for (j = i + 1; j < npart; j++){
loc = itype + ntype * ptype[j];
xj = 3*j;
yj = 3*j + 1;
zj = 3*j + 2;
dx = pos[xi] - pos[xj];
dy = pos[yi] - pos[yj];
dz = pos[zi] - pos[zj];
dx = pbc_dist(dx, box[0], ibox[0]);
dy = pbc_dist(dy, box[1], ibox[1]);
dz = pbc_dist(dz, box[2], ibox[2]);
rsq = dx*dx + dy*dy + dz*dz;
if (rsq < rcut2[loc]){
r2inv = 1.0 / rsq;
r6inv = r2inv*r2inv*r2inv;
forcelj = r2inv * r6inv * (lj1[loc] * r6inv - lj2[loc]);
vpot += r6inv * (lj3[loc] * r6inv - lj4[loc]) - offset[loc];
fx = forcelj * dx;
fy = forcelj * dy;
fz = forcelj * dz;
force[xi] += fx;
force[yi] += fy;
force[zi] += fz;
force[xj] -= fx;
force[yj] -= fy;
force[zj] -= fz;
virial[0] += fx*dx;
virial[1] += fx*dy;
virial[2] += fx*dz;
virial[3] += fy*dx;
virial[4] += fy*dy;
virial[5] += fy*dz;
virial[6] += fz*dx;
virial[7] += fz*dy;
virial[8] += fz*dz;
}
}
}
PyObject *ret = Py_BuildValue("d", vpot);
return ret;
}
```
#### Step 2. Creating a setup.py file and compiling
In order to compile the C-code we just created, we make use of a setup.py file which is almost identical to the one we created for the engine.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""A script for building the C Lennard-Jones extension."""
from setuptools import (
setup,
Extension,
)
import numpy as np
LJMODULE = Extension(
'ljc',
sources=['ljc.c'],
extra_compile_args=["-Ofast", "-march=native"]
)
setup(
name="PyRETIS Lennard-Jones C extension",
description="C extension for the Lennard-Jones potential.",
ext_modules=[LJMODULE],
include_dirs=[np.get_include()],
)
```
As before, the setup.py file is used to compile via the command
```pyretis
python setup.py build_ext --inplace
```
Note that we assume that the C-code is stored in a file named ljc.c.
#### Step 3. Creating a new Python class for the potential
The final step is to create a Python class which is making use of the C-code.
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of using a Lennard-Jones potential implemented in C."""
import logging
import os
import sys
import numpy as np
from pyretis.forcefield.potentials import PairLennardJonesCut
from pyretis.forcefield.potentials.pairpotentials import (
generate_pair_interactions
)
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
logger.addHandler(logging.NullHandler())
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
try:
import ljc
except ImportError as exc:
MSG = ('Could not import external C library.'
'\nPlease compile with: "python setup.py build_ext --inplace"')
raise ImportError(MSG) from exc
__all__ = ['PairLennardJonesCutC']
class PairLennardJonesCutC(PairLennardJonesCut):
r"""class PairLennardJonesCutC(PairLennardJonesCut).
This class implements as simple Lennard-Jones 6-12 potential which
employs a simple cut-off and can be shifted. The potential energy
(:math:`V_\text{pot}`) is defined in the usual way for an
interacting pair of particles a distance :math:`r` apart,
.. math::
V_\text{pot} = 4 \varepsilon \left( x^{12} - x^{6} \right),
where :math:`x = \sigma/r` and :math:`\varepsilon`
and :math:`\sigma` are the potential parameters. The parameters are
stored as attributes of the potential and we store one set for each
kind of pair interaction. Parameters can be generated with a
specific mixing rule by the force field.
Attributes
----------
params : dict
The parameters for the potential. This dict is assumed to
contain parameters for pairs, i.e. for interactions.
_lj : dict of numpy.array
[1] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``48.0 * epsilon * sigma**12``
[2] Lennard-Jones parameters used for calculation of the force.
Keys are the pairs (particle types) that may interact.
Calculated as: ``24.0 * epsilon * sigma**6``
[3] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**12``
[4] Lennard-Jones parameters used for calculation of the potential.
Keys are the pairs (particle types) that may interact.
Calculated as: ``4.0 * epsilon * sigma**6``
_offset : numpy.array
Potential values for shifting the potential if requested.
This is the potential evaluated at the cutoff.
_rcut2 : numpy.array
The squared cut-off for each interaction type.
Keys are the pairs (particle types) that may interact.
"""
def __init__(self, dim=3, shift=True, mixing='geometric',
desc='Lennard-Jones pair potential (C)'):
"""Initialise the Lennard-Jones potential.
Parameters
----------
dim : int
The dimensionality to use.
shift : boolean
Determines if the potential should be shifted or not.
mixing : string
Determines how we should mix potential parameters.
"""
super().__init__(dim=dim, desc=desc, mixing=mixing)
self.shift = shift
self.ntype = 0
def set_parameters(self, parameters):
"""Update all parameters.
Here, we generate pair interactions, since that is what this
potential actually is using.
Parameters
----------
parameters : dict
The input base parameters.
"""
self.params = {}
pair_param = generate_pair_interactions(parameters, self.mixing)
self.ntype = max(int(np.sqrt(len(pair_param))), 2)
self._lj = {'1': np.zeros((self.ntype, self.ntype)),
'2': np.zeros((self.ntype, self.ntype)),
'3': np.zeros((self.ntype, self.ntype)),
'4': np.zeros((self.ntype, self.ntype))}
self._rcut2 = np.zeros_like(self._lj[1])
self._offset = np.zeros_like(self._lj[1])
for pair, value in pair_param.items():
eps_ij = value['epsilon']
sig_ij = value['sigma']
rcut = value['rcut']
self._lj[1][pair] = 48.0 * eps_ij * sig_ij**12
self._lj[2][pair] = 24.0 * eps_ij * sig_ij**6
self._lj[3][pair] = 4.0 * eps_ij * sig_ij**12
self._lj[4][pair] = 4.0 * eps_ij * sig_ij**6
self._rcut2[pair] = rcut**2
vcut = 0.0
if self.shift and not np.isclose(rcut, 0):
vcut = 4.0 * eps_ij * ((sig_ij / rcut)**12 -
(sig_ij / rcut)**6)
self._offset[pair] = vcut
self.params[pair] = pair_param[pair]
def potential(self, system):
"""Calculate the potential energy for the Lennard-Jones interaction.
Parameters
----------
system : object like `System` from `pyretis.core.system`
The system we are evaluating the potential in.
Returns
-------
v_pot : float
The potential energy.
"""
particles = system.particles
box = system.box
v_pot = ljc.potential(
particles.pos,
box.length,
box.ilength,
self._lj,
self._offset,
self._rcut2,
particles.ptype,
particles.npart,
box.dim,
self.ntype
)
return v_pot
def force(self, system):
"""Calculate the force for the Lennard-Jones interaction.
We also calculate the virial here, since the force
is evaluated.
Parameters
----------
system : object like `System` from `pyretis.core.system`
The system we are evaluating the force in.
Returns
-------
forces : numpy.array
The forces on the particles.
virial : numpy.array
The virial obtained from the forces.
"""
particles = system.particles
box = system.box
forces = np.zeros((particles.npart, box.dim))
virial = np.zeros((box.dim, box.dim))
ljc.force(
particles.pos,
box.length,
box.ilength,
self._lj,
self._rcut2,
particles.ptype,
forces,
virial,
particles.npart,
box.dim,
self.ntype
)
return forces, virial
def potential_and_force(self, system):
"""Calculate potential and force for the Lennard-Jones interaction.
Since the force is evaluated, the virial is also calculated.
Parameters
----------
system : object like `System` from `pyretis.core.system`
The system we are evaluating the potential and force in.
Note
----
Currently, the virial is only calculated for all the particles.
It is not calculated as a virial per atom. The virial
per atom might be useful to obtain a local pressure or stress,
however, this needs some consideration. Perhaps it's best to
fully implement this as a method of planes or something similar.
Returns
-------
out[0] : float
The potential energy as a float.
out[1] : numpy.array
The force as a numpy.array of the same shape as the
positions in `particles.pos`.
out[2] : numpy.array
The virial, as a symmetric matrix with dimensions
(dim, dim) where dim is given by the box/system dimensions.
"""
particles = system.particles
box = system.box
forces = np.zeros((particles.npart, box.dim))
virial = np.zeros((box.dim, box.dim))
vpot = ljc.force_and_pot(
particles.pos,
box.length,
box.ilength,
self._lj,
self._offset,
self._rcut2,
particles.ptype,
forces,
virial,
particles.npart,
box.dim,
self.ntype
)
return vpot, forces, virial
```
You can now use the new potential, for instance by adding the following forcefield and potential sections to the input file:
```toml
[forcefield]
description = "Lennard-Jones evaluated in C"
[[potential]]
class = "PairLennardJonesCutC"
module = "ljpotentialc.py"
shift = true
dim = 3
mixing = "geometric"
[potential.parameter.0]
sigma = 1.0
epsilon = 1.0
rcut = 2.5
```
Again, we note that the two Python classes we created (for FORTRAN and C) are very similar. The diff of the two modules shown below.
```udiff
--- /builds/pyretis/pyretis/docs/_static/examples/extending-example/potential/ljpotentialfp.py
+++ /builds/pyretis/pyretis/docs/_static/examples/extending-example/potential/ljpotentialc.py
@@ -1,6 +1,6 @@
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
-"""Example of using a Lennard-Jones potential implemented in FORTRAN."""
+"""Example of using a Lennard-Jones potential implemented in C."""
import logging
import os
import sys
@@ -14,18 +14,18 @@
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
try:
- from ljfortranp import ljfortranp
+ import ljc
except ImportError as exc:
- MSG = ('Could not import external FORTRAN library.'
- '\nPlease compile with "make"!')
+ MSG = ('Could not import external C library.'
+ '\nPlease compile with: "python setup.py build_ext --inplace"')
raise ImportError(MSG) from exc
-__all__ = ['PairLennardJonesCutFp']
-
-
-class PairLennardJonesCutFp(PairLennardJonesCut):
- r"""class PairLennardJonesCutFp(PairLennardJonesCut).
+__all__ = ['PairLennardJonesCutC']
+
+
+class PairLennardJonesCutC(PairLennardJonesCut):
+ r"""class PairLennardJonesCutC(PairLennardJonesCut).
This class implements as simple Lennard-Jones 6-12 potential which
employs a simple cut-off and can be shifted. The potential energy
@@ -70,8 +70,8 @@
"""
def __init__(self, dim=3, shift=True, mixing='geometric',
- desc='Lennard-Jones pair potential (fortran)'):
- """Set up the Lennard-Jones potential.
+ desc='Lennard-Jones pair potential (C)'):
+ """Initialise the Lennard-Jones potential.
Parameters
----------
@@ -140,7 +140,7 @@
"""
particles = system.particles
box = system.box
- v_pot = ljfortranp.potential(
+ v_pot = ljc.potential(
particles.pos,
box.length,
box.ilength,
@@ -175,13 +175,17 @@
"""
particles = system.particles
box = system.box
- forces, virial = ljfortranp.force(
+ forces = np.zeros((particles.npart, box.dim))
+ virial = np.zeros((box.dim, box.dim))
+ ljc.force(
particles.pos,
box.length,
box.ilength,
self._lj,
self._rcut2,
particles.ptype,
+ forces,
+ virial,
particles.npart,
box.dim,
self.ntype
@@ -220,7 +224,9 @@
"""
particles = system.particles
box = system.box
- forces, virial, vpot = ljfortranp.potential_and_force(
+ forces = np.zeros((particles.npart, box.dim))
+ virial = np.zeros((box.dim, box.dim))
+ vpot = ljc.force_and_pot(
particles.pos,
box.length,
box.ilength,
@@ -228,6 +234,8 @@
self._offset,
self._rcut2,
particles.ptype,
+ forces,
+ virial,
particles.npart,
box.dim,
self.ntype
```
### Comparison of the C, FORTRAN and a Numpy implementation
In the figure below, we show some sample timings using the modules you have just created two Python implementations (one is in pure Python, while the other is using Numpy).
### OpenMP: Parallel evaluation of the force
In both the C module and the FORTRAN module, we created for evaluation of the force, it is possible to speed up the force evaluation by, for instance, making use of OpenMP directives. We just give a very brief example below, and leave the rest to you.
```fortran
subroutine potential(pos, box, ibox, lj3, lj4, offset, rcut2, ptype, n, d, p, vpot)
! Function to evaluate the Lennard-Jones potential
!
! Parameters
! ----------
! pos : positions of the particles
! box : the box lengths
! ibox : the inverse box lengths, ibox = 1.0 / box
! lj3 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**12
! lj4 : Lennard-Jones parameters, 4.0 * epsilonij * sigmaij**6
! rcut2 : Squared cut-offs
! offset : Potential energy shift
! ptype : identifier for the particle types
! n : number of particles
! d : dimensionality of the vectors
! p : number of particle types
!
! Returns
! -------
! vpot : the potential energy
implicit none
integer, intent(in) :: n, d, p
double precision, dimension(n, d), target, intent(in) :: pos
double precision, dimension(d), intent(in) :: box, ibox
double precision, dimension(p, p), intent(in) :: lj3, lj4, offset, rcut2
integer, dimension(n), intent(in) :: ptype
double precision, intent(out) :: vpot
double precision :: rsq, r2inv, r6inv, dx, dy, dz
double precision, dimension(:), pointer :: x, y, z
integer :: i, j, itype, jtype
vpot = 0.0D0
x => pos(1:n,1)
y => pos(1:n,2)
z => pos(1:n,3)
do i=1,n-1
itype = ptype(i) + 1
!$omp parallel do private(jtype, dx, dy, dz, rsq, r2inv, r6inv) reduction (+:vpot)
do j=i+1,n
jtype = ptype(j) + 1
dx = x(i) - x(j)
dy = y(i) - y(j)
dz = z(i) - z(j)
dx = pbc_dist(dx, box(1), ibox(1))
dy = pbc_dist(dy, box(2), ibox(2))
dz = pbc_dist(dz, box(3), ibox(3))
rsq = dx*dx + dy*dy + dz*dz
if (rsq < rcut2(jtype, itype)) then
r2inv = 1.0D0 / rsq
r6inv = r2inv**3
vpot = vpot + r6inv * (lj3(jtype, itype) * r6inv - lj4(jtype, itype))
vpot = vpot - offset(jtype, itype)
end if
end do
!$omp end parallel do
end do
end subroutine potential
```
Note that you will have to modify the flags in your Makefile accordingly, for instance, setting:
```make
F2PY_FLAGS = --fcompiler=gfortran --f90flags='-fopenmp' -lgomp
```
## Tested by
The extension tutorials contain local test scripts next to the C and FORTRAN modules. They are also represented by the internal MD checks that exercise the same force and integration contracts. Keep the build commands in this page aligned with those local tests when changing the extension examples.
---
# Examples
Welcome to the PyRETIS examples library. The source distribution keeps user-facing material under examples/tutorials/ and consistency checks under examples/tests/.
The pages are grouped by topic, with a short note on what each tutorial covers so you can pick the one that matches the question you have. If you are new to PyRETIS, start with a minimal input file. If you are adapting the examples with help from an AI agent, start with the Agent-TIS quick start so the agent follows the same folder, command, and verification conventions as the website. The runnable checks behind these tutorials are tracked in example-test-status.
## Start here
- A minimal input file – the shortest input that runs a complete RETIS simulation, and what PyRETIS fills in when you leave everything optional out. The best first read if you have never written a PyRETIS input. Source.
- A minimal input file with an external engine – the same, driving OpenMM: the engine owns the system, so the input shrinks to what to sample, which engine to drive, and how to measure it. Source.
## Potentials
These examples are a good place to start when the main question is which model system to use. They run with the internal PyRETIS engine unless noted otherwise, and each entry links to the matching tutorial source folder in the repository.
- TIS in a 1D double-well potential – the van Erp double-well benchmark and the initial-flux workflow. Source.
- Triple-well RETIS with zero_left – a three-state potential showing how zero_left avoids wasted [0^-] trajectories. Source.
- Hysteresis on a 2D potential – a richer internal landscape where order-parameter choice matters. Source.
- Breaking a bond with RETIS – the WCA bond-breaking model from van Erp, Moroni and Bolhuis (2003). Source.
## Sampling methods
These examples focus on the sampling algorithm or the way initial paths are generated.
- RETIS in a 1D potential – replica exchange between path ensembles for the double-well benchmark. Source.
- Subtrajectory moves on a 1D potential – stone skipping, web throwing and wire fencing from Riccardi, Dahlen and van Erp (2017) and Zhang, Riccardi and van Erp (2023). Source.
- Infinite swapping – the asynchronous, highly parallel replica-exchange sampler of Zhang, Baldauf, Roet, Lervik and van Erp (2024), driven by the [runner] worker pool. Source.
- Partial path TIS (REPPTIS) – replica exchange partial path TIS for long permeation timescales, from Vervust, Zhang, van Erp and Ghysels (2023). Source.
- Exploring the order parameter – the explore task for scouting the order-parameter space and seeding initial paths (see Getting the initial path). Source.
- Umbrella sampling – free-energy estimation alongside the rate-constant workflows. Source.
- Permeability calculation – using permeability = True instead of, or together with, TIS. Source.
- PSO-based initial path optimisation – particle-swarm assisted initialisation. Source.
## External engines
These tutorials replace the internal Langevin engine with one of the standard MD codes, or with compiled extension modules. They require the corresponding engine or compiler to be installed and reachable on your PATH.
- GROMACS hydrate – methane transport in an sI hydrate with a custom Python order parameter. Source.
- CP2K hydrogen – a small ab initio hydrogen bond-breaking setup. Source.
- LAMMPS WCA – the 2D WCA bond-breaking model driven by LAMMPS. Source.
- OpenMM argon – RETIS with OpenMM handling the molecular dynamics. Source.
- Compiled external components – C and FORTRAN extensions for custom integrators, potentials and order parameters. Source.
## Analysis
These examples show how to start, seed, analyse and visualise runs with pyretis run, pyretis analyse and PyVisA.
- Plain MD – input-file (examples-md-nve) and library-based (examples-md-nve-lib) ways to run a Lennard-Jones simulation. Source.
- Forward/backward MD and initial flux – handy when you need to seed initial paths for TIS. Source.
- pyvisa GUI and pyvisa analysis – visualise and post-process your trajectories. Source.
- Transmission coefficient – compute the reactive-flux transmission coefficient from a finished run’s per-path order series. Source.
- Combine data – pool several independent runs’ pathensemble.txt into a single crossing matrix for one WHAM analysis. Source.
## How the examples are organised
The pages below describe the contract between tutorial pages and heavy checks, the rules for adding new examples, the current verification status, and the LLM-oriented quick reference.
- Agent-TIS quick start: PyRETIS for users and AI agents – compact rules for choosing examples, running commands, and avoiding common setup mistakes.
- Migration guide – moving a project off the deprecated pyretisrun/pyretisanalyse commands, the .rst input format, and the legacy-runner TOML schema onto the unified pyretis CLI (all removed in PyRETIS 5).
- Example organization – folder contract, tutorial authoring rules, and how tutorials link to the heavy suite.
- Using heavy checks as tutorials – one-shot table of which test fixture to copy when starting a new setup.
- Example test status – the current verification matrix and per-suite refresh dates.
- Interactive examples – planned notebook front ends for the lightweight tutorials.
The examples/ folder has two subfolders:
- examples/tutorials/ for user-facing setup guides;
- examples/tests/ for heavy consistency checks.
To run all heavy example checks from the repository root:
```bash
./test-heavy.sh
```
To run a single check, cd into its directory and execute its run.sh:
```
cd examples/tests/test-internal/retis
./run.sh
```
run.sh cleans the directory, calls pyretis run and pyretis analyse, then runs compare.py to validate the output against the reference data in results/.
---
# Interactive examples
Interactive examples are a good fit for PyRETIS, but they should be introduced in a way that does not make the normal documentation build fragile. The easiest path is to start with the pure-Python internal examples and keep the external-engine examples as downloadable input sets.
## Recommended first step
Create one notebook for each lightweight internal tutorial:
- TIS in a 1D potential.
- RETIS in a 1D potential.
- Sub-moves on a 1D potential.
- Plain MD and MD-flux setup.
Each notebook should run in a few minutes, generate one or two plots, and link back to the corresponding full documentation page. The source of truth should remain the same input files used by the examples and the regression tests, so the notebook is a guided front end rather than a separate example that can drift.
## Documentation integration
There are two low-maintenance ways to publish these notebooks:
- Notebook links in the examples pages. Store notebooks under examples/notebooks/ and link to them from the relevant pages. This is the smallest change and works immediately.
- Rendered notebooks in Sphinx. Add nbsphinx or myst-nb later and render selected notebooks into the website. This gives a richer website, but the docs build must then decide whether notebooks are executed during CI or rendered from saved outputs.
For a hosted interactive session, Binder is the practical first target: it can install the Python dependencies and run the pure-Python examples without requiring users to install PyRETIS locally. JupyterLite can be considered later for tiny browser-only demos, but it is a tighter fit because PyRETIS uses scientific Python dependencies and some examples need file-system output.
## Suggested implementation plan
- Add examples/notebooks/tis-1d.ipynb and keep it under five visible cells: import, load settings, run, analyse, plot.
- Add a small environment.yml or requirements-notebooks.txt for Binder.
- Add a notebook smoke test that runs the notebook with jupyter nbconvert --execute or pytest --nbmake.
- Link the notebook from examples-tis-1d and this page.
- Repeat for RETIS and sub-moves once the first notebook is stable.
This keeps the interactive work incremental: the website can advertise the direction now, while the normal examples and regression checks stay fully usable without a notebook stack.
---
# Agent-TIS quick start: PyRETIS for users and AI agents
This page is a compact orientation guide for anyone trying to set up a PyRETIS simulation from the documentation. It is intentionally procedural and repetitive so that both users and AI agents (LLM and coding agents) can find the correct pattern quickly. The agent skills and the rules for agents are described in user-guide-agent-tis.
## Start from the right folder
Use examples/tutorials/ when you want to learn or adapt a simulation. Use examples/tests/ when you want a short, validated consistency check.
The two folders have different jobs:
- examples/tutorials/ contains user-facing setups. These may be long and are allowed to be experience-building simulations.
- examples/tests/ contains heavy checks used by test-heavy.sh. These are optimized for reproducible output comparisons, not for narrative readability.
Do not copy generated output folders such as the NNN/ ensemble directories (000/, 001/, each holding generate/ / accepted/ / archive/) or report/ as if they were required source files. Start from the input files, order-parameter modules, initial configurations, and external-engine input folders listed by each tutorial page.
## Canonical command patterns
Most PyRETIS tutorials use one of these command patterns.
Task
Command
What to inspect
Run one input file
pyretis run -i input.toml -p
pyretis.log, output.toml, ensemble folders, trajectory and energy files.
Analyse one completed run
pyretis analyse -i input.toml
report/ and the generated analysis figures/tables.
Split a TIS setup into ensemble inputs
pyretis run -i tis.toml
Generated files such as tis-001.toml, tis-002.toml.
Run one split TIS ensemble
pyretis run -i tis-001.rst -p -f 001.log
The numbered ensemble output and the renamed log file.
Run all tutorial startup checks
examples/run-tutorials.sh
Whether tutorial commands start without missing files, broken imports, PyRETIS errors, or tracebacks. Per-tutorial hard timeout (default 2s standalone, 2m when invoked via test-heavy) is set via PYRETIS_TUTORIAL_TIMEOUT. The path_sampling/internal/1D-triple-well/retis-zero-left tutorial is the canonical default that finishes end-to-end under the 2 m budget (~15-20 s) and only smoke-starts under the 2 s budget.
Run heavy checks
./test-heavy.sh (thin wrapper around examples/run-tests.sh)
Whether bundled examples reproduce their reference output. Heavy checks finish with the tutorial smoke checks; set PYRETIS_RUN_TESTS_SKIP_TUTORIALS=1 to skip them.
## Choose the closest tutorial
The canonical mapping between tutorial folders, docs pages, and heavy-test fixtures lives in example-tutorial-map. Always read that table before recommending an example; it also carries the verification-status column you should cite (“passing”, “engine”, “smoke”, “manual”) rather than inventing one.
Quick pointers for the most common requests:
- First TIS simulation – examples-tis-1d.
- First RETIS simulation – examples-retis-1d.
- Initial flux calculation – examples-tis-1d-flux.
- Sub-trajectory moves (SS / WT / WF) – examples-submoves-1d.
- zero_left for a bounded reactant basin – examples-retis-triple-well.
- Plain molecular dynamics – examples-md.
- External engines – examples-gromacs-hydrate, examples-retis-lammps, examples-cp2k-hydrogen, examples-openmm.
- Custom order parameters / engines / potentials – examples-vvexternal, examples-pso.
## Read input files by section
When adapting an example, keep the PyRETIS section structure intact. The most common sections are:
- Simulation: method, number of cycles, interfaces, and method-specific switches such as permeability or zero_left.
- System and Box: units, temperature, dimensions, and periodicity.
- Particles: initial coordinates, masses, labels, and velocity setup.
- Engine: internal engine or external program coupling.
- TIS and RETIS: shooting, path-length limits, swap behavior, and move frequencies.
- Orderparameter: built-in order parameters or a Python module/class.
- Forcefield and Potential: potential-energy model for internal engines.
- Output and Analysis: write frequency and post-processing options.
When an external engine is used, do not move its input files out of the input_path directory named in the Engine section. The executable name in the input file must match the local installation, for example gmx, lmp, cp2k or a site-specific wrapper.
## Minimal adaptation workflow
- Pick the tutorial with the closest method and engine.
- Copy only the listed input files and support modules to a new working directory.
- Run the tutorial unchanged for a few cycles or with the smoke runner.
- Change one category at a time: system, order parameter, engine, then interface positions.
- Run the changed setup with pyretis run -i ... -p in a new directory, or after pyretis tools clean in this one, and inspect pyretis.log and output.toml before doing a long run. A changed setup is a new simulation, and pyretis run refuses to start one with method = "kick" or "load" where the ensemble directories hold sampled cycles. To add cycles to a run without changing its setup, set method = "restart" and raise steps (see the restart method).
- Analyse only after a run has produced the expected output files.
- Use example-test-status to find a representative check if the adapted setup starts failing.
## Rules for AI agents
When producing PyRETIS instructions or code:
- Prefer an existing tutorial input file over inventing a new one.
- Link to the relevant user-guide section for every non-obvious input section or engine setting.
- State whether a command is a short startup check or a convergence run.
- Keep external-engine examples explicit about required programs and file names.
- Do not claim an example is fully verified unless it is listed as passing in example-test-status.
- If a setup is based on examples/tests/, say that it is a regression fixture and explain how to adapt it for a tutorial.
This guide is deliberately conservative: it should help assistants give correct first steps before trying to optimize or automate a full simulation campaign.
---
# PyRETIS
---
# Infinite swapping in PyRETIS
This page documents PyRETIS’s asynchronous infinite-swapping sampler. It is part of the unified PyRETIS path-sampling scheduler, and the input TOML selects parallel infinite swapping via [simulation] task = "infinite_swapping" or a [runner] section.
For the command and input overview, see README.rst.
## Asynchronous infinite swapping and classical RETIS
Asynchronous infinite swapping extends Replica Exchange Transition Interface Sampling (RETIS) to address two bottlenecks of the classical algorithm:
- Acceptance rate of long subtrajectory moves. Classical RETIS relies on shooting moves whose acceptance falls off sharply as the path length grows. The wire-fencing move replaces the single shooting point with a coordinated multi-point reshape that keeps acceptance high even on long paths.
- Wall-clock cost of replica exchange. The standard REPEX swap is a synchronisation point — every worker must finish its current trajectory before swaps can be evaluated. The asynchronous scheme replaces synchronous swap with an asynchronous infinite-swap scheme: the central coordinator records swap acceptance statistics over a running stream of completed trajectories, lets each worker draw its next task from the resulting distribution, and dispatches work as soon as a worker frees up. Wall-clock scaling with the number of workers stays close to linear over a much wider range than synchronous REPEX allows.
In PyRETIS these two ideas show up as:
- wf shooting moves in [tis] shooting_moves of a canonical input, [simulation] shooting_moves of the scheduler configuration (the wire-fencing implementation), implemented in pyretis/core/moves.py.
- The InfSwapState coordinator in pyretis/simulation/repex.py, together with the asyncio runner in pyretis/simulation/async_runner.py.
- The [runner] section of a canonical input (workers, wmdrun). The deprecated legacy-runner schema, with [runner], [simulation.tis_set] and no [simulation] task (see test/infswap/simulations/data/wf.toml), is converted to its canonical form when PyRETIS reads it.
## When to reach for it
There is a single CLI, pyretis (the run sub-command), and one path-sampling scheduler. The input TOML decides how that scheduler runs.
- Select the infinite-swapping path (TOML with [simulation] task = "infinite_swapping" or a [runner] section) when you need parallel sampling, want infinite-swap REPEX, or want the wf shooting move. The multi-engine pool API (see assign_engines in pyretis/engines/factory.py) is part of this path.
- Use a canonical TOML for single-worker RETIS / TIS runs, custom Python order parameters, or the existing tutorials under examples/tutorials/. Non-path MD and flux tasks use their dedicated in-process runners.
Both selections run through pyretis run and produce the same per-ensemble output format.
## Migration and archival compatibility
PyRETIS translates runner-style TOML inputs created for earlier infRETIS workflows (the legacy-runner schema), as well as older PyRETIS .rst inputs, to the current PyRETIS TOML schema. A legacy-runner input is deprecated: PyRETIS 4 converts it when it reads it, with a deprecation notice, records the canonical form in the output.toml of a run, and PyRETIS 5 will not read it. For a reusable production input, convert it explicitly and retain the converted file with the simulation record:
```bash
python -m pyretis.tools.convert_legacy_schema input.toml
python -m pyretis.tools.convert_settings input.rst
```
The converters validate the translated configuration before writing it, and a conversion when PyRETIS reads a legacy-runner input is validated the same way. The analysis tools also recognise archived infinite-swapping output under its earlier filename, and the state file of the scheduler configuration an earlier PyRETIS wrote, so previously completed simulations remain analysable. The :doc:migration guide gives the mapping between the input forms.
## References
If you use PyRETIS infinite swapping in your research, please cite:
- Zhang, Baldauf, Roet, Lervik, van Erp, Highly parallelizable path sampling with minimal rejections using asynchronous replica exchange and infinite swaps, PNAS 121, e2318731121 (2024). doi:10.1073/pnas.2318731121
For a calculation translated from an earlier implementation, cite this method paper and the PyRETIS software paper below, and report the PyRETIS version and translated input used for the production run.
```bibtex
@article{zhang_highly_2024,
title = {Highly parallelizable path sampling with minimal rejections
using asynchronous replica exchange and infinite swaps},
volume = {121},
url = {https://www.pnas.org/doi/10.1073/pnas.2318731121},
doi = {10.1073/pnas.2318731121},
number = {7},
journal = {Proceedings of the National Academy of Sciences},
author = {Zhang, Daniel T. and Baldauf, Lukas and Roet, Sander
and Lervik, Anders and van Erp, Titus S.},
year = {2024},
}
```
Foundational papers on the methods themselves:
- van Erp, Moroni, Bolhuis, A novel path sampling method for the calculation of rate constants, J. Chem. Phys. 118, 7762 (2003) — the original TIS paper. doi:10.1063/1.1562614
- van Erp, Reaction Rate Calculation by Parallel Path Swapping, Phys. Rev. Lett. 98, 268301 (2007) — the original RETIS paper. doi:10.1103/PhysRevLett.98.268301
- Zhang, Riccardi, van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023) — wire-fencing move. doi:10.1063/5.0127249
- Lervik, Riccardi, van Erp, PyRETIS: A well-done, medium-sized python library for rare events, J. Comput. Chem. 38, 2439 (2017) — original PyRETIS paper. doi:10.1002/jcc.24900
## Examples
Runnable examples for the infinite-swapping path live under examples/tutorials/path_sampling///infinite_swapping/ — each engine (turtlemd, gromacs, lammps, cp2k, ase) has its own sub-directory with an infswap.toml config and a small driver script. The TurtleMD examples need no external MD installation and are the recommended starting point.
For aggregate-observable comparisons against canonical references, see test/infswap/simulations/.
---
# PyRETIS TOML configuration schema
This document lists the keys of the scheduler configuration, grouped by section: the configuration the scheduler builds in memory from a canonical input (task = "retis", [tis], …). The scheduler configuration is set up by pyretis.simulation.setup.setup_config. No file PyRETIS writes holds it: the run file output.toml records the canonical input (see [current] below).
The deprecated legacy-runner schema (a [runner] section, the path-sampling settings under [simulation.tis_set], no [simulation] task) is an input of this shape. PyRETIS 4 converts such an input to its canonical form when it reads it, with a deprecation notice, and PyRETIS 5 will not read it. pyretis run, pyretis analyse, the scheduler set-up of a programmatic run and the interface optimiser convert it the same way (pyretis/tools/convert_legacy_schema.py): each key goes to its input key (the table below), the canonical form is parsed as every canonical input is, and the conversion is checked against the scheduler configuration of the input, a difference stopping it with the settings that differ. python -m pyretis.tools.convert_legacy_schema writes the canonical form to a file once, with the same check.
A canonical input reaches these keys through the declared key table, pyretis.inout.key_table: one entry per scheduler path names the input keys it is read from. The keys whose name or section differ:
Scheduler key
Input key
[simulation.tis_set]
[tis]
[simulation.tis_set] lambda_minus_one
[simulation] zero_left
[simulation.tis_set] permeability
[simulation] permeability
[simulation] shooting_moves
[tis] shooting_moves, else [tis] shooting_move for every ensemble
[simulation] zeroswap
[retis] swapfreq (a repptis run that sets none takes 0.5)
[simulation] relative_shoots
[retis] relative_shoots
[simulation] noswap, repptis, explore, single_tis, pptis_no_minus, pptis_no_zero_plus, repptis_memory
[simulation] task (with zero_left, flux, zero_ensemble, [pptis] memory, [repptis] memory)
[output] order_file, energy_file
[output] order-file, energy-file
A message of pyretis run names a setting by its input key, e.g. [simulation] zero_left. The run file output.toml records the canonical input, not this configuration (see [current] below).
An input key either takes effect on the run or the input is refused: pyretis/inout/key_table.py:NO_EFFECT declares the input keys and sections that take no effect on a run, with the runs they take none on, and the settings parse of a canonical input refuses each one the input gives, naming the key and, for a key whose value another key sets, that key ([simulation] zeroswap names [retis] swapfreq; [simulation] relative_shoots and [tis] relative_shoots name [retis] relative_shoots; [simulation] restart names [initial-path] method = "restart"). The table of these keys is in the input reference (docs/user/section/sections.rst, “Keywords that take no effect on a run”). The canonical form of a legacy-runner input is refused the same way, and the message names each key in the canonical input and in the legacy-runner input, e.g. [tis] nullmoves ([simulation.tis_set] nullmoves of the legacy-runner input) = true. The conversion gives [simulation] zeroswap its input key, [retis] swapfreq of a retis run (a tis or explore run, which attempts no swap, leaves it out with a warning), and leaves out the [output] keys of the in-process output, with a warning.
## [runner]
Key
Type
Default
Description
workers
int
required
Number of parallel REPEX workers
wmdrun
list[str]
[]
Per-worker MD run commands (optional)
## [simulation]
Key
Type
Default
Description
interfaces
list[float]
required
Sorted interface values for TIS ensembles
steps
int
required
Total number of RETIS cycles
shooting_moves
list[str]
required
Move type per interface: "sh" (shooting) or "wf" (wire-fencing)
seed
int
0
RNG seed for reproducibility
load_dir
str
"accepted"
Name of the per-ensemble operational trajectory store (//); a user-staged flat top-level directory of this name is still read as input via the legacy-layout fallback. The name is relative, and the first component of the name, after os.path.normpath, may not be generate, archive, rejected or engine_logs (directories the run writes in each ensemble directory), pyretis-cp2k-wfn (the wavefunction store of the streaming CP2K engine in the run directory), the name of an ensemble directory (000, 001, …), . or ..: such a name stops the run with a TOMLConfigError before it writes a file. A resume reads its active paths from the store its load_dir names and stops with a FileNotFoundError naming each missing traj.txt
zeroswap
float
0.5
Probability of a swap move: in a retis run, the swap of the [0^-] and [0^+] paths when a move picks one of them and the other is free; in a repptis run, a swap with an adjacent ensemble. The input key is [retis] swapfreq
pick_scheme
int
0
Ensemble picking scheme (0 = uniform)
relative_shoots
list[float] or null
null
Relative per-ensemble move-selection frequencies (one weight per ensemble, interface order, [0^-] first). The scheduler weights its per-ensemble pick by these; reallocates sampling effort without biasing the rate. The input key is [retis] relative_shoots
priority_shooting
bool
false
Weights the per-ensemble pick toward the ensembles with fewer submitted moves (weight n_max - n + 1; a zero swap counts for both ensembles); the path is drawn from the chosen ensemble as usual. The counts are kept in [current] moves_submitted over restarts. Cannot be combined with pick_scheme or relative_shoots
ensemble_engines
list[list[str]]
auto
Engine keys per interface
## [simulation.tis_set]
Key
Type
Default
Description
maxlength
int
20000
Maximum allowed path length. The default is a deliberately generous cap so an input need not set it; a run that falls back to it logs that it did, and records the value it used. Set it explicitly for systems with long paths — a cap that is too small biases the sampling
allowmaxlength
bool
false
Accept paths exactly at max length
zero_momentum
bool
false
Reset the centre-of-mass momentum after every velocity draw: shooting and the kicks of the initial-path search, in every engine. GROMACS with velocity_generation = "engine" (gen_vel) and AMS (GenerateVelocities; RandomVelocitiesMethod Gromacs and Exact, the AMS default, while Boltzmann is not checked) remove the net linear momentum of the velocities they generate, so they require true; lammps_steps with velocity_generation = "engine" writes the mom keyword of velocity create from it (mom yes / mom no), and lammps draws the velocities in PyRETIS and resets the momentum as it selects. A continuation with [initial-path] method = "restart" is refused when it draws the shooting velocities differently from the run it continues, and a resume of an output.toml without [current.provenance] velocity_draw_rule (pyretis run -i output.toml) is refused when the settings it records draw differently from the engine of that run (see the restart method in docs/user/section/initial.rst)
rescale_energy
float or bool
false
The total energy the shooting velocities are re-scaled to; false, or a number that is not positive, leaves the energy of the draw. The internal integrators, OpenMM and cp2k_steps re-scale in PyRETIS, and lammps_steps with velocity_generation = "engine" in LAMMPS (to pe + ke as LAMMPS evaluates them, per atom under thermo_modify norm yes); every other engine stops with an error at its first shooting draw for a positive value. The internal integrators and OpenMM also re-scale the kicks of the initial-path search. A continuation and a resume compare it as they compare zero_momentum
n_jumps
int
required
Velocity reassignment jumps per shooting move
quantis
bool
false
Use quantIS variant (per-ensemble engines)
lambda_minus_one
bool/int
false
Create a minus interface below the first
accept_all
bool
false
Accept all paths unconditionally
interface_cap
float
optional
Upper interface cap for flux collection
perm_threshold
int
12
Max ensemble block size for which the exact matrix permanent is computed (larger blocks use the stochastic approximation)
perm_n_samples
int
10000
Sample count for the stochastic permanent approximation used above perm_threshold
exact_perm_only
bool
false
Forbid the stochastic fallback (raise instead of approximating a block larger than perm_threshold)
## [engine]
Multiple engines can be defined as [engine], [engine0], [engine1], etc.
Key
Type
Default
Description
class
str
required
Engine type: turtlemd, cp2k, gromacs, lammps, ase, ams
engine
str
required
Backend name (for logging)
timestep
float
required
MD timestep in engine’s native units
subcycles
int
required
MD steps between two stored frames of a path; an internal integrator of [engine] takes it in the in-process kick initiation as well
temperature
float
required
Simulation temperature
boltzmann
float
required
Boltzmann constant (kB)
input_path
str
external only
Path to external engine input files
cp2k
str
CP2K only
CP2K executable name
gmx
str
GROMACS only
GROMACS executable (e.g. gmx_mpi)
lmp
str
LAMMPS only
LAMMPS executable (e.g. lmp_mpi)
masses
list/str
GROMACS
Particle masses or path to masses.txt
### [engine.integrator]
Key
Type
Description
class
str
Integrator class: LangevinInertia, LangevinOverdamped, VelocityVerlet, Verlet
### [engine.integrator.settings]
Keys vary by integrator. Examples for Langevin:
Key
Type
Description
gamma
float
Friction coefficient
beta
float
Inverse temperature: 1 / (kB * T)
### [engine.potential]
Key
Type
Description
class
str
Potential class: DoubleWell, LennardJones, DoubleWellPair
### [engine.potential.settings]
Keys vary by potential. Examples for DoubleWell:
Key
Type
Description
a
float
Quartic coefficient
b
float
Quadratic coefficient
c
float
Constant offset
### [engine.particles]
Key
Type
Description
mass
list[float]
Mass of each particle
name
list[str]
Name/symbol of each particle
pos
list[list[float]]
Initial position of each particle
### [engine.box]
Key
Type
Description
periodic
list[bool]
Periodicity per dimension
## [orderparameter]
Key
Type
Default
Description
class
str
required
Order parameter class
index
list[int]
required
Particle indices for the OP
periodic
bool
required
Apply PBC to OP calculation
module
str
external
Path to Python file with custom OP class
## [output]
Key
Type
Default
Description
data_dir
str
required
Output directory
screen
int
10
Cycles between two progress reports of the scheduler in the log (0 writes none); the input key [output] screen
pattern
int/bool
false
Print frequency of ensemble distribution
order-file
int
optional
Order parameter output frequency
energy-file
int
optional
Energy output frequency
trajectory-file
int
optional
Read by the in-process tasks (md, md-flux) only; the canonical input of a path-sampling run refuses it, and the conversion of a legacy-runner input leaves it out
backup
str
'append'
Read by the in-process tasks (md, md-flux) only; the canonical input of a path-sampling run refuses it, and the conversion of a legacy-runner input leaves it out
archive_every
int
1
Long-term archive cadence: a superseded path MOVES from its per-ensemble operational store (/accepted/) to the per-ensemble long-term store (/archive/), but only every N-th one is kept (path number a multiple of N), so the archive holds ~1 trajectory per N accepted moves; 1 keeps every one. Bounds disk the way the classic trajectory-file frequency did. (The former delete_old / delete_old_all / keep_maxop_trajs deletion keys are retired.)
keep_traj_fnames
list[str]
[]
Extra trajectory file patterns the archiver carries along when moving a path
log_file
str
'pyretis.log'
Run-log file name; also used by the worker processes, which append to the same file (an explicit pyretis run -f wins for the main process only — set the keyword for parallel runs)
log_mode
str
'append'
Start-of-run treatment of an existing run log: 'append' keeps it growing (a restart continues the same file), 'backup' rotates it to pyretis.log_000… first, 'overwrite' truncates it
engine_log
str/int
'last'
Per-trajectory engine-log retention: each finished trajectory’s engine logs (engine.log/engine.err, stdout.txt/stderr.txt, *.log/*.screen) move from the ensemble generate/ scratch to /engine_logs//, zeroing the live log for the next trajectory; keep 'last' (only the most recent), 'all', or the N most recent. A failed generation leaves its logs in the scratch for inspection
## [current] (scheduler-generated run state)
Do not add this section to the input file you write. When a run starts, pyretis run writes output.toml in the run directory. That generated file is the run record: it contains the sections of the canonical input, under the names of the input and with every default the run takes resolved (the zero-swap probability as [retis] swapfreq), and a [current] section recording the latest restartable scheduler state. It holds none of the settings the input parse derives (the engine type, the input_files the GROMACS and CP2K checkers find, the exe_path of the run directory, the per-ensemble copies of the sections), and none of the keys and sections that take no effect on the run (pyretis/inout/key_table.py:NO_EFFECT): the settings parse refuses each one an input gives, and the record leaves out the value the parse gives such a key (the [simulation] restart it derives from method = "restart", the [engine0] of a run that runs no [engine0] engine, [simulation] rgen, and the rgen of [tis], [engine] and [system] for a run whose initiation kicks no path in process). It records [output] screen. The scheduler builds its configuration from those sections with the key table, when the run starts and when it resumes. pyretis run -i output.toml resumes the run, and pyretis analyse -i output.toml analyses it as its input would. The output.toml of an earlier PyRETIS that holds keys of no effect is read without them (pyretis/inout/run_record.py:load_run): silently when the value is the one the settings parse gives, named in a warning otherwise; [simulation] zeroswap is read as [retis] swapfreq, and [tis] relative_shoots as [retis] relative_shoots when the file sets none there.
The scheduler rewrites output.toml atomically at checkpoints, so it is safe to inspect between updates. Treat it as output, however: changing it by hand can make the stored state inconsistent with trajectories and other run files. To change a simulation, edit the original input and start a new run; to continue an interrupted one, restart from its run directory and let PyRETIS read output.toml.
restart.toml is the older name for the same kind of generated state file. It is read only as a fallback when output.toml is absent, so previously created runs remain restartable. The output.toml and restart.toml of an earlier PyRETIS hold the scheduler configuration of this document, with its [simulation.tis_set] table, and the [current] state. A continuation with [initial-path] method = "restart" reads such a state file and compares its settings with the continuation’s; pyretis run -i refuses it and names that route. pyretis analyse -i reads its settings under the names of the input, through the inverse of the key table (pyretis/inout/key_table.py:canonical_items), and refuses the state file of a tis run of one ensemble, whose analysis counts a crossing at [tis] detect when the input gives [tis] ensemble_number, which the scheduler configuration does not say.
Key
Type
Description
cstep
int
Current RETIS cycle
traj_num
int
Total trajectory count
size
int
Number of ensembles
active
list[int]
Active path indices
rng_state
dict
NumPy Generator state for restart
provenance
dict
Version/env metadata (P7.3)
restarted_from
int
Previous cstep on restart
---
# Agent-TIS: AI agents for PyRETIS path sampling
Agent-TIS is the PyRETIS integration for AI agents: large language model (LLM) agents and coding agents, such as Claude Code, that set up, run and analyse path-sampling simulations (TIS, RETIS, REPPTIS, permeability, infinite swapping) or develop and validate the code. It captures PyRETIS-specific procedural knowledge: method selection, move choice, interface placement, engine validation, and the primary literature that supports those choices. It helps an agent navigate the documentation and execute defined workflows; it does not replace convergence assessment, uncertainty analysis, or scientific judgement by the researcher.
Agent-TIS provides two agent skills, a user skill and a developer skill, together with a compact, machine-readable documentation index, llms.txt. The skills live in the repository. A skill is a folder of agent-agnostic Markdown files: a SKILL.md that holds the rules, the workflow and a table of topic files, and the topic files, which the agent reads when a step of the work needs them. The folder is installed in an agent’s skills directory, and SKILL.md is loaded when its task description applies. The .claude/skills// path is one installation example; the same files can guide any LLM agent that can read local files and use the required tools.
- The user skill — run and analyse simulations
- The developer skill — extend and validate PyRETIS
- Choosing an LLM
- Method references
- Using a skill
## The user skill — run and analyse simulations
skills/pyretis-pathsampling/ guides the configuration, execution, and interpretation of path-sampling calculations. Its SKILL.md holds the rules the agent always follows and a fixed minimal recipe (decision table, a base input per engine, the keys to change, exact commands) that a small or locally hosted model follows step by step. A capable model follows its workflow: selection of a sampling task (TIS / RETIS / REPPTIS / permeability), MD-engine configuration (internal/TurtleMD, GROMACS, LAMMPS, CP2K, or OpenMM), definition of the order parameter and interface ladder, initial-path preparation, execution with pyretis run, and analysis of crossing probabilities and rates with pyretis analyse, reading the topic file of each step:
## Topic files of the user skill
File
Read when
methods.md
choosing the task, the moves and their controls, high acceptance, priority shooting, infinite swapping or permeability
order-parameters.md
defining the order parameter, the states A and B, or extra collective variables
interfaces.md
placing or re-spacing the interfaces, by hand or with pyretis tools init
engines.md
configuring an MD engine
running.md
getting initial paths, running, monitoring, restarting, using several workers
analysis.md
reading the report, WHAM, the REPPTIS MSM, skipping cycles
diagnosis.md
a result looks wrong or a run stops, e.g. a local crossing probability of zero
checking.md
deciding whether to trust a setup, a rate or a speed-up: a known case with an analytical rate, invariance checks, efficiency
references.md
citing the method behind a choice
examples/
notes for a specific system
The skill makes the following method choices explicit:
- RETIS includes the initial-flux contribution and is the standard choice; use plain TIS only when a single ensemble or a custom flux treatment is intended.
- Order-parameter quality and interface placement (target crossing probabilities of approximately 0.2–0.5 per interface) strongly determine sampling efficiency and convergence.
- Subtrajectory moves (Wire Fencing, Stone Skipping, and Web Throwing) are considered for stiff, long, or diffusive barriers, subject to their additional computational cost.
- REPPTIS is appropriate when long-lived intermediates or slow permeation make conventional path ensembles costly.
- Infinite swapping is intended for large parallel calculations and is analysed with WHAM.
Example requests
```text
Set up a RETIS simulation for the 1-D double well, run it, and report
the rate with its statistical error and the crossing-probability curve.
I have a GROMACS system for ligand unbinding. Help me pick an order
parameter and interface ladder, write the retis.toml, and launch RETIS.
My intermediate state is metastable and the permeation is very slow —
is REPPTIS the right method here, and what do I lose vs RETIS?
Analyse this finished run and tell me whether the crossing probabilities
overlap well or whether I need more steps or better-placed interfaces.
Ensemble [3^+] reports a crossing probability of zero. Why, and what
should I change?
```
## The developer skill — extend and validate PyRETIS
skills/pyretis-development/ maps the codebase and records the deterministic-science conventions used in development. Its SKILL.md holds the rules (a failing test is a stop, not a step; comments describe the code that is there; tests for new code come from a separate session; reference data move only with a named cause; the test gates) and the codebase map. In particular, it defines build-version-sensitive engine reference validation: a non-reference engine build is classified as NOT VALIDATED after execution smoke testing rather than as a numerical failure; reference data are re-generated deliberately with OMP_NUM_THREADS=1 using the affected suite’s run.sh generate and checked with run.sh; and the heavy suite can be executed on a cluster through devtools/remote_validate.sh.
## Topic files of the developer skill
File
Read when
style.md
writing or editing code, comments or docstrings
testing.md
running the gates, a test fails, or tests are needed for new code
validation.md
engine reference suites, re-blessing, the cluster, the method-validation suite
engines.md
adding or changing an MD engine
scheduler.md
changing the scheduler, the moves, the output layout, the input schemas or restarts
Example requests
```text
Add a new order-parameter class with a unit test, and run the fast gate.
My LAMMPS reference test fails — is it a real regression or a
build-version mismatch? Fix the version grading if needed.
Re-bless the CP2K references on my local CP2K 2025.2 build and confirm
run-all-cp2k.sh validates rather than smokes.
Add streaming support to the engine, mirroring the GROMACS
streaming/relaunch pair, and verify streaming == relaunch on the mock.
```
## Choosing an LLM
Choose a model by demonstrated capabilities rather than parameter count or brand. Parameter count is not a reliable proxy for tool use, code quality, or scientific reasoning. In both cases, require the agent to retain the input files, commands, software versions, and analysis output needed to audit its result.
## Recommended model capabilities
Skill
Recommended model type
Minimum practical capability
User path-sampling skill
A general-purpose reasoning model that can use local tools, read scientific documentation, and generate TOML and Python reliably.
Follows multi-step instructions, preserves units and file paths, can run shell commands in the project environment, uses a context window of at least 16k tokens, and reports uncertainty rather than inventing a convergence claim.
Developer skill
A software-engineering or coding agent with shell access, repository navigation, patch generation, and test-debugging capability.
Reads a multi-file code change in context (at least 32k tokens), executes tests, interprets failures, and reports the exact validation performed. Prefer a stronger coding model for engine integrations or numerical-reference changes.
For either skill, a model should cite the method references below for scientific claims and should not present an unvalidated run as a converged physical result.
## Method references
The user skill cites the following primary sources rather than attempting to reconstruct the theory, so an agent can attribute method-level claims to the relevant literature.
Foundations
- TIS — T. S. van Erp, D. Moroni, P. G. Bolhuis, A novel path sampling method for the calculation of rate constants, J. Chem. Phys. 118, 7762 (2003), https://doi.org/10.1063/1.1562614
- RETIS — T. S. van Erp, Reaction rate calculation by parallel path swapping, Phys. Rev. Lett. 98, 268301 (2007), https://doi.org/10.1103/PhysRevLett.98.268301
- Transition path sampling — C. Dellago, P. G. Bolhuis, F. S. Csajka, D. Chandler, J. Chem. Phys. 108, 1964 (1998), https://doi.org/10.1063/1.475562
Moves and sampling
- Fast-decorrelating moves (Stone Skipping / Web Throwing) — E. Riccardi, O. Dahlen, T. S. van Erp, J. Phys. Chem. Lett. 8, 4456 (2017), https://doi.org/10.1021/acs.jpclett.7b01617
- Sub-trajectory moves (Wire Fencing) — D. T. Zhang, E. Riccardi, T. S. van Erp, J. Chem. Phys. 158, 024113 (2023), https://doi.org/10.1063/5.0127249
- Infinite swapping / asynchronous replica exchange — D. T. Zhang, L. Baldauf, S. Roet, A. Lervik, T. S. van Erp, Proc. Natl. Acad. Sci. U.S.A. 121, e2318731121 (2024), https://doi.org/10.1073/pnas.2318731121
- WHAM — S. Kumar, J. M. Rosenberg, D. Bouzida, R. H. Swendsen, P. A. Kollman, J. Comput. Chem. 13, 1011 (1992), https://doi.org/10.1002/jcc.540130812
Memory reduction and long timescales
- REPPTIS — W. Vervust, D. T. Zhang, T. S. van Erp, A. Ghysels, Biophys. J. 122, 2960 (2023), https://doi.org/10.1016/j.bpj.2023.02.021
- RETIS + REPPTIS in a large biomolecule (ABL–imatinib) — W. Vervust, D. T. Zhang, E. Riccardi, T. S. van Erp, A. Ghysels, Biophys. J. 124, 3932 (2025), https://doi.org/10.1016/j.bpj.2025.04.020
The PyRETIS software
- A. Lervik, E. Riccardi, T. S. van Erp, J. Comput. Chem. (2017), https://doi.org/10.1002/jcc.24900
- E. Riccardi, A. Lervik, S. Roet, O. Aarøen, T. S. van Erp, J. Comput. Chem. (2019), https://doi.org/10.1002/jcc.26112
- W. Vervust, D. T. Zhang, A. Ghysels, S. Roet, T. S. van Erp, E. Riccardi, J. Comput. Chem. (2024), https://doi.org/10.1002/jcc.27319
See Scientific results for application papers (autoionization of water, DNA binding, atmospheric reactions, …).
## Using a skill
Copy the applicable skill folder (skills/pyretis-pathsampling or skills/pyretis-development) into the agent’s skills directory, with all its files, and work from the project conda environment. In a checkout of the repository, .claude/skills/ links both folders for Claude Code. Both skills start from the smoke test devtools/smoke_test.sh, which checks the environment and the build in a few seconds. The agent may invoke pyretis run / pyretis analyse or the relevant test gates, but examples should always be copied to a scratch directory before execution to preserve the tracked source tree.
---
# The PyRETIS analysis application
The PyRETIS analysis application, pyretis analyse, is used to analyse the results from PyRETIS simulations.
user-guide-output describes the files a run writes and how to read the report this application produces – including how to tell a converged simulation from one that has not sampled enough. user-guide-rate explains how the rate constant itself is put together, and how the routes differ between TIS, RETIS, infinite swapping and the (RE)PPTIS family.
The general syntax for executing is:
```pyretis
pyretis analyse [-h] [-i INPUT] [-V]
```
where INPUT is the input file for the analysis. This file is similar to the input file to the PyRETIS application with some differences:
- It contains the actual number of cycles/steps completed, specified by the endcycle keyword.
- It contains the directory from where the simulation was executed, specified by the exe_path keyword.
- It contains the number of particles in the simulation, specified by the npart keyword.
- It contains settings for the analysis, defined via the analysis section.
Note
The analysis reads the SAME input file the simulation ran from (pyretis analyse -i retis.toml), or the run’s own output.toml (pyretis analyse -i output.toml), which records that input with every default resolved: the analysis takes the input sections from it and the run directory it is run in (see user-guide-rate).
## Input arguments
## Description of input arguments for pyretis analyse
Argument
Description
-h, –help
Show the help message and exit.
-i INPUT, –input INPUT
Location of the input file containing analysis settings.
-V, –version
Show the version number and exit.
-skipb N, –skip-begin N
Discard the first N cycles (equilibration). Optional; without it nothing is discarded.
-skipe N, –skip-end N
Discard the last N cycles, for instance after an interrupted run. Optional; without it nothing is discarded.
## Choosing which cycles to analyse
By default every sampled cycle is analysed. Two optional arguments narrow that window:
```
pyretis analyse -i input.toml -skipb 500
pyretis analyse -i input.toml -skipb 500 -skipe 100
```
The first discards the first 500 cycles – the usual reason being that the simulation had not yet equilibrated. The second additionally discards the last 100, which is what you want when a run was cut short (a wall-time kill mid-cycle) or when you are comparing two runs over the same window.
Both are counted in cycles, they can be used together or on their own, and they override skip_initial_cycles and skip_final_cycles in the input file, so you can try several windows without editing anything. The analysis log states which cycles it dropped.
The chosen window applies to everything the analysis reports: the crossing probabilities, the rate, and the acceptance ratios below.
Independently of the window, each ensemble’s statistics start at its first accepted, non-initialisation path. The paths an ensemble holds before that are the ones initiation put there (ld, ki or is) and they are not samples of the ensemble, so counting them would bias the crossing probability towards whatever the initial path happened to do. An ensemble that has not yet been sampled at all is reported as having no accepted paths to analyse rather than given a number built from its initial path – if you see that, the run was too short to reach that ensemble.
The same holds for a path marked hc: an ensemble held it at a restart that changed [tis] high_accept, so it was accepted under the other acceptance rule. Its rows from the restart on are left out, and the ensemble counts again from its first path accepted after the restart; its rows from before the restart stay in.
The WHAM analysis, and every tool that builds the combined path-data matrix from the per-ensemble output (the training set and the interface optimiser read the same matrix), leaves out the rows of the initialisation paths and of the paths marked hc by the same rule. Its skip counts paths, the initialisation paths included: -skipb 500 discards the first 500 paths of the run by path number, among the paths the ensembles do not hold at the end of the run, and the rows of the initialisation paths among the paths that remain are left out. A run whose initialisation paths all fall within the skipped paths gives the matrix of every row after that skip.
A literal infswap_data.txt or infretis_data.txt in the run directory is read in place of the per-ensemble output. Its rows record no move, so every row of it is kept, the rows of the initialisation paths included; its skip discards its first rows. The merge of several runs (pyretis.analysis.combine_data) writes .txt, combo.txt by default, in this layout, and its interfaces to .toml. Copied into a directory as infswap_data.txt, with .toml as output.toml, the merge is read in this way. The merge reads the paths of each run as the analysis does, and its skip counts them in the same way.
Stone skipping without high acceptance ([tis] high_accept = false) samples each path with weight 1. When the rows of such an ensemble carry the crossing count of each path as HA-weight, and the counts differ from row to row, every weighted average over them is biased, so the analysis leaves those cycles out and its log names them. The run’s output.toml records, as [current] ss_weight_from_cycle, the first cycle whose rows carry the weight of the rule each path was sampled with, and, under [current] ss_weight_before, the move list and layout the earlier rows were written with and the paths held at that cycle. The rows before it are checked, and so is every row of a run whose output.toml does not record it.
Where no weight decides where a path goes (tis, pptis, repptis), only an affected ensemble loses those cycles. In a retis run the weights also decided, through the swaps, where each path went, so every ensemble loses them, up to the last cycle in which a path held at the boundary is still held. The cycles after that start from the paths the earlier weights left behind, as a run starts from its initial paths; give them an equilibration skip (-skipb) of their own if they need one. An ensemble with no cycle left is refused with an error; continuing the run with restart adds cycles that can be analysed. The WHAM analysis, and every tool that reads the combined path-data matrix, sums every cycle of a path, so it refuses such a run.
Two cases leave no trace in the run’s files: a change of [tis] high_accept that output.toml does not record (a state file edited by hand and resumed from), and rows appended after the boundary by a version of PyRETIS that writes the crossing count.
## Acceptance ratios
The report’s Pathensemble data table gives, per ensemble, the fraction of attempted moves that were accepted – separately for the shooting-like moves and for the swapping moves.
These are counted from moves.txt – one line per attempted move: the cycle, the move that ran, and the status it ended in (ACC, or a rejection code such as BWI or NCR; see user-guide-path-types-rejections). It is the only record of a rejection. pathensemble.txt beside it says which path each ensemble holds, so every one of its lines is an accepted path by construction – counting those would report every move as accepted. See user-guide-output-moves for the file’s columns.
A cycle in which two ensembles were picked together is a swap, whatever shooting move those ensembles are configured with, and is recorded in both partners: under the + code in the lower-index one and the - code in the higher. The pair of codes says which swap it was – s+/s- for the zero swap between [0^-] and [0^+], p+/p- for the PPTIS/REPPTIS swap between adjacent ensembles. That is what makes the swap ratio computable at all.
Two further columns report what counting acceptances cannot.
HA eff. is for the high-acceptance moves – wire fencing and stone skipping. Those are built so that a trial is accepted almost every time; what varies is the weight the accepted path carries, its crossing count. A plain acceptance ratio therefore sits near 1.000000 however the sampling is doing, and says nothing. This column reports instead the weight collected per attempt,
which reduces to the ordinary acceptance ratio when every weight is 1. Two ensembles that both accept every wire-fencing trial can differ by a factor of four here, and that difference is the one that matters. The high-acceptance moves are excluded from the shooting column for the same reason; an ensemble that runs only wire fencing shows n/a there.
Exchange answers the question the swap column cannot on most ensembles. Under infinite swapping only the zero swap is an accept/reject move; every other ensemble exchanges by having its occupancy redistributed over the live paths. A cycle in which one path holds the whole ensemble exchanged nothing; a cycle split 0.9 / 0.1 exchanged a tenth. This column averages 1 - (largest share) over the analysed cycles, so 0 means the ensembles are not communicating and larger means more exchange. It is the same quantity a swap acceptance ratio measures, on the routes where no swap is ever proposed.
A ratio is shown as n/a when it cannot be formed: the move was never attempted in that ensemble (a per-interface TIS run attempts no swaps, so its swap ratio is always n/a). A move that was not attempted has no acceptance ratio – it is not a ratio of one.
### Comparing these numbers with PyRETIS 3
The column is called Shoot acc. ratio in both versions but it does not count the same moves, so the two are not directly comparable.
PyRETIS 3 divided everything that was not a swap by its attempts. That denominator included the time-reversal move, and it included the null move, which is accepted by definition. PyRETIS 4 counts only the moves that actually shoot – sh, bias and wt – so it answers “how often does shooting succeed here?” rather than “how often did this ensemble keep a path?”.
The difference is not cosmetic, and it is largest exactly where it misleads most. In a 2000-cycle RETIS run of the 1D double well, the two versions report this for the outermost ensemble [3^+]:
Counted over
[3^+]
Agreement
PyRETIS 3, as reported
0.522
–
PyRETIS 3, null moves removed
0.278
–
PyRETIS 4, as reported (sh only)
0.215
–
PyRETIS 4, counted as sh + tr
0.283
0.006 from PyRETIS 3
PyRETIS 3’s 0.522 is almost twice its own shooting performance: 511 of its attempts in that ensemble were null moves, every one of them accepted. Once those are removed the two versions agree to 0.006. The same holds for the other ensembles, where null moves are rarer and the two numbers were never far apart to begin with.
So if you are reproducing a PyRETIS 3 result, expect this column to drop, most visibly in the outer ensembles – the sampling has not got worse, the ratio has stopped counting moves that could not fail.
The swap column differs for a structural reason rather than a book-keeping one: PyRETIS 3 proposed a discrete swap between every adjacent pair and could report a ratio for each, while under infinite swapping only the zero swap is an accept/reject move. Every other ensemble therefore shows n/a there and reports Exchange instead, as described above.
Older runs are still analysed. Before moves.txt the attempted moves were written into pathensemble.txt itself with a trailing Trial column, and before that into a separate trials.txt; both are read. Only one case genuinely has no answer: output written by the scheduler between those conventions, which holds accepted paths only. That is reported as n/a rather than as the 1.000000 those rows would otherwise produce.
In pathensemble.txt the Mc column is a different thing: it names how the path was originally generated, which for a path that arrived by a swap is the move that created it in its previous ensemble. Read moves.txt for what was attempted here.
The same numbers are logged at the end of a run, broken down per move:
```
Acceptance (attempted trials, per ensemble):
000: 34/50 = 0.6800 [sh: 34/50]
001: 25/50 = 0.5000 [sh: 21/40 tr: 4/10]
```
A very low ratio usually means the interfaces are placed too far apart, or the shooting move is too aggressive for the system.
---
# The PyRETIS application
PyRETIS ships a single command, pyretis, with sub-commands:
```bash
pyretis run -i input.toml # run a simulation
pyretis analyse -i input.toml # analyse the output
pyretis status # per-ensemble sampling summary
pyretis tools init -i input.toml # place the interfaces automatically
pyretis tools clean # remove run artifacts
```
Run pyretis -h for the options of a command; pyretis status is described in user-guide-monitor, and can be run while a simulation is still going. pyretis analyse is described in user-guide-analyse, pyretis tools clean in user-guide-clean, and pyretis tools init in user-guide-tools-init.
Run pyretis with no command (or pyretis -h) to print the logo and this usage summary; pyretis -v (or --version) prints the version.
The standalone pyretisrun and pyretisanalyse commands still work but are deprecated: they print a deprecation warning and will be removed in PyRETIS 5, where only pyretis run and pyretis analyse will be supported. Likewise pyretis clean is a deprecated alias of pyretis tools clean.
The pyretis run sub-command is used to execute simulations defined in input files. The general syntax for executing is:
```pyretis
pyretis run [-h] -i INPUT [-V] [-f LOG_FILE] [-l LOG_LEVEL] [-p]
```
where the different arguments are described in tableappargument and tableloglevel.
## Example use
Here are some examples of the use of the application:
- To run the simulation defined in a file named retis.toml, make a log file for output named retis.log and display a progress bar:
```pyretis
pyretis run -i retis.toml -f retis.log -p
```
- To run the simulation defined in a file named test.toml and display debug messages:
```pyretis
pyretis run -i test.toml -l DEBUG
```
- To run the simulation defined in a file named input.toml, display only critical messages, write to a log file named mylog.log and display a progress bar:
```pyretis
pyretis run -i input.toml -l CRITICAL -p -f mylog.log
```
The long version of the arguments may also be used, for instance, the previous example can also be written more verbose as
```pyretis
pyretis run --input input.toml --log_level CRITICAL --progress --log_file mylog.log
```
## Soft exit
Running simulations can be stopped softly by pressing Ctrl+C. Instead of terminating immediately, PyRETIS creates a file named EXIT in the run directory. The simulation checks for this file between simulation steps, writes restart information, closes output files and then exits.
The same mechanism can be triggered manually by creating the EXIT file in the run directory:
```bash
touch EXIT
```
After a completed soft exit, pyretis run removes the EXIT file automatically so that a subsequent restart does not stop immediately. If EXIT is already present before pyretis run starts, the program will stop and ask you to remove it first.
## Input arguments
## Description of input arguments for PyRETIS.
Argument
Description
-h, –help
Show the help message and exit
-i INPUT, –input INPUT
Location of the input file.
-V, –version
Show the version number and exit.
-f LOG_FILE, –log_file LOG_FILE
Specify location of log file to write.
-l LOG_LEVEL, –log_level LOG_LEVEL
Specify log level for the log file (see tableloglevel).
-p, –progress
Display a progress meter instead of text output. The bar is restart-aware: a restarted run starts it at the resumed cycle, not from zero. It also covers path-sampling runs (the sampling cycles of the scheduler).
## Possible settings for the LOG_LEVEL.
LOG_LEVEL
Description
CRITICAL
Only display critical messages.
ERROR
Display critical and error messages.
WARNING
Display critical and error messages and warnings.
INFO
Display all of the above and some info messages.
DEBUG
Display all of the above and additional debug messages.
PyRETIS also defines three custom levels used internally for console output (they are not intended as -l arguments but are listed here for completeness):
## Custom PyRETIS log levels.
LOG_LEVEL
Value
Description
PROGRESS
25
Progress messages shown on the console (default level).
BANNER
26
Logo and header messages displayed in cyan at startup.
REFERENCE
27
Citation and URL messages displayed at startup/shutdown.
## Placing the interfaces automatically (pyretis tools init)
By default you place the TIS/RETIS interfaces yourself, as the [simulation] interfaces list in the input file. pyretis tools init instead places them automatically, with the iterative infinit driver:
```bash
pyretis tools init -i input.toml
```
Each iteration
- runs a short infinite-swapping simulation with the current interfaces;
- re-estimates the crossing probability with binless WHAM;
- re-places the interfaces so every ensemble carries roughly the same local crossing probability (equal per-ensemble acceptance);
- re-selects the most-decorrelated active paths as the next iteration’s initial paths,
repeating for the configured number of iterations. Each iteration runs in its own step_/ sub-directory and the interface evolution is written to infinit.log; the final interface set is printed at the end. The iterations of a run are numbered from step_0/ whatever [infinit] cstep is, and a run with cstep above 0 starts from the interfaces of its input file and the paths of its starting library. When any step_/ of the run already holds files, left by an earlier run of the driver in the same working directory, the driver stops before its first iteration and names each of these directories; remove them, or use a new working directory.
The input file must contain an [infinit] section (controlling the number of iterations, the target per-ensemble crossing probability, and the interface resolution) and, next to it, a starting path library — one numbered sub-directory per ensemble under the [simulation] load_dir (default load).
```bash
pyretis tools init [-h] -i FILE [-w DIR]
```
- -i, --input — the starting config (.toml) carrying the [infinit] section (required).
- -w, --workdir — directory for the per-iteration step_ sub-directories (default: the current directory).
---
# Cleaning run artifacts
A PyRETIS run leaves a number of generated files and directories behind – log files, the regenerated out.toml / out.rst, restart files, the NNN ensemble directories (each holding its generate scratch, the accepted operational trajectory store and the archive long-term store), the report directory, the multi-run runs directory, Python byte-code caches, and so on. (A flat top-level load directory only exists when it is staged by hand as legacy input, and is no longer a built-in default.) The pyretis tools clean sub-command removes them so a run (or example) directory can be reset to its committed inputs:
```bash
pyretis tools clean # clean the current directory
pyretis tools clean some/dir # clean another directory
pyretis tools clean --dry-run # list what would be removed, delete nothing
```
pyretis tools clean deletes immediately; use --dry-run (or -n) first if you want to preview the list.
pyretis clean (without tools) still works as a deprecated alias: it prints a one-line notice and then behaves identically. Use pyretis tools clean in new scripts.
## What is removed
The removed set is the union of a comprehensive built-in default and an optional per-directory configuration file. The built-in defaults cover the full set of artifacts a routed run generates, so a plain run folder resets with no clean.toml at all:
- files – byte-code / compiled order parameters (*.pyc, *.pyo, *.so), the run file and restart store (output.toml*, out.toml*, out.rst*, infswap.toml, restart.toml*, pyretis.restart, ensemble.restart), the per-run text output (energy.txt*, order.txt*, pathensemble.txt*, trials.txt*, cross.txt*, thermo.txt*, traj.txt*), the logs of both pyretis run and pyretis analyse including their backup copies (pyretis.log*, pyretisanalyse.log*, sim.log, worker[0-9]*.log), the RNG state, the engine scratch of every engine (*.tpr, *.edr, *.cpt, *.lammpstrj, …), and the run-done marker / plots;
- directories – __pycache__, the NNN ensemble directories ([0-9][0-9][0-9], which subsume their generate / accepted / archive / engine_logs sub-stores), the accepted / archive / engine_logs stores and the previous store names (paths / trajs), the retired worker* scratch, the report and runs directories, and the private directories a run removes once it is done with them, left behind by a run that stopped first: pyretis-gromacs-*, the work directory of a GROMACS engine, removed with the engine, and pyretis-load-*, where a load prepares its files, removed when the load ends; and pyretis-cp2k-wfn, where the streaming CP2K engine keeps the wavefunctions its MD segments start from for the whole run;
- the store of the paths of a scheduler run, /, under the [simulation] load_dir a TOML input names (by default accepted), in each ensemble directory of that run: the directories named by ensemble number (000, 001, …, 1000, …) in the directory of the file, or in its [output] data_dir. The TOML files are read as for the load input of the runs (below), before any file is removed, so the output.toml of a run names its store too. The store is removed also where its ensemble directory is left, because it holds a kept path or has a name of four digits. A store reached through a symbolic link, or a load_dir that points out of the ensemble directory, is left, and use_defaults = false leaves every store.
Three mechanisms keep these otherwise-aggressive defaults safe (the clean-safety sweep asserts no git-tracked file is ever removed):
- Excluded reference/input directories. Anything below results, output_data, load, load_copy or an engine input directory (lammps_input, gromacs_input, cp2k_input, openmm_input) is never cleaned. These hold committed reference (golden) output, committed load/NNN initial-path inputs and committed engine input files.
- The load input of the runs. A TOML input file names the paths the run in its directory reads: the [initial-path] load_folder of a classic load, and the store a fresh scheduler run reads its initial paths from ([simulation] load_dir, by default accepted), both relative to the directory of the file. The TOML files in the cleaned directory and in every directory below it are read, so a clean started above a run directory keeps them too. They are left with all their files, whatever the default patterns, find_files, find_dirs or rm_paths match; remove one by hand once you are done with it. A TOML file that cannot be parsed stops the clean before it removes anything. A load folder that only a legacy .rst input names is protected with keep.
- The ``keep`` list (below), for committed inputs that live outside those excluded names – for example a non-standard committed load directory, or the load/0 .. load/7 initial paths of the validation suite.
## Per-directory configuration
Drop a clean.toml next to the files to extend (or replace) the defaults:
```toml
[clean]
use_defaults = true # apply the built-in defaults (default: true)
find_files = ["out.toml*", "*.log"] # extra file-name globs (recursive)
find_dirs = ["report"] # extra directory names (recursive)
rm_paths = ["0*", "lammps/system.data"] # root-relative rm -rf globs
keep = ["load/0", "load/1"] # never delete these (or ancestors)
```
- find_files – file-name globs deleted wherever they occur in the tree (like find . -name PATTERN -delete).
- find_dirs – directory names deleted recursively wherever they occur.
- rm_paths – root-relative paths or globs removed with rm -rf semantics.
- keep – paths that must survive; a kept path, anything inside it, and – since keep declares a committed input – the declaration is honoured even when the clean was started from an ANCESTOR directory (its clean.toml is read during the walk), and its ancestor directories are never removed. Use this to protect committed inputs that would otherwise match (for example the committed load/0 .. load/7 initial paths of the validation suite).
- use_defaults = false disables the built-in defaults so the file is the complete, explicit recipe.
This replaces the per-example Makefile clean targets used in earlier PyRETIS versions.
---
# Creating custom engines
The Engines are used to propagate the equations of motion or alter the state of the system in some other way. In this section, we describe how additional engines can be added to PyRETIS. Here, we make a distinction between two types of engines:
- Internal engines which directly interact with the underlying numpy.arrays representing positions, velocities and forces (see fig-engine-left)
- External engines which are used by PyRETIS to control the execution of external programs. In this case, the state of the particles is represented by a file containing the positions and velocities. As shown in the illustration below (fig-engine-right) , this can be accessed using ParticlesExt.config which is a tuple containing a file name and an index. The interpretation is that the current state of the system can be found in the given file at the given frame/index. Further, the attribute ParticlesExt.vel_rev is used to determine if the velocities in the configuration should be reversed or not. In this case the engine will typically make use of ParticlesExt.set_pos() in order to update the state of the system.
# Table of contents
- Creating a new internal engine
- Example: Creating a new internal LeapFrog integrator
- Step 1 and 2: Sub-classing MDEngine
- Step 3: Adding the integration step method
- Step 4: Making use of the new integrator
- Creating a new external engine
- Implementing methods for a new external engine
- The __init__ method
- The step method
- The _read_configuration method
- The _reverse_velocities method
- The _extract_frame method
- The _propagate_from method
- The modify_velocities method
- References
Once your engine is implemented, validate that it reproduces the same rate on a shared system with the reproducibility suite – see Extending and validating PyRETIS.
## Creating a new internal engine
In order to create a new internal engine, we create a new class for our engine which is sub-classing the generic EngineBase. Often, we wish to create a new integrator for molecular dynamics and then we can simplify our work by sub-classing the more specific MDEngine. The steps you need to complete in order to create a new MDengine is as follows:
- Create a new Python class which is a sub-class of MDEngine
- Write a method to initialise this class, that is a __init__ method. This method will be called when PyRETIS is setting up a new simulation and it will be fed variables from the PyRETIS input file.
- Write a method to perform a single integration step.
- Modify the input script to use the new integrator.
### Example: Creating a new internal LeapFrog integrator
To be specific, we will in the following show the steps needed to create a new MD engine using the Leapfrog scheme 1:
where \mathbf{r} are the positions, \mathbf{v} the velocities, \mathbf{a} the accelerations, \Delta t the time step and t is the current time.
#### Step 1 and 2: Sub-classing MDEngine
In order to define a new class to use with PyRETIS we sub-class the MDEngine and define the __init__ method:
```python
from pyretis.engines import MDEngine
class LeapFrog(MDEngine):
"""Perform Leap Frog integration."""
def __init__(self, timestep):
"""Initiate the engine.
Parameters
----------
timestep : float
Set the time step for the integrator.
"""
super().__init__(timestep, 'Leap Frog integrator', dynamics='NVE')
self.half_timestep = self.timestep * 0.5
self.half_timestep_sq = 0.5 * self.timestep**2
```
#### Step 3: Adding the integration step method
Now, we add the actual integration routine:
```python
def integration_step(self, system):
"""Perform one step for the Leap Frog integrator."""
particles = system.particles
imass = particles.imass
# get current acceleration:
acc_t = particles.force * imass
# update positions:
particles.pos += (self.timestep * particles.vel +
self.half_timestep_sq * acc_t)
# update forces:
system.potential_and_force()
# update acceleration:
acc_t2 = particles.force * imass
# update velocities:
particles.vel += self.half_timestep * (acc_t + acc_t2)
```
#### Step 4: Making use of the new integrator
The new integrator can be used by specifying the following engine section:
```toml
[engine]
class = "LeapFrog"
module = "leapfrog.py"
timestep = 0.002
```
where the keyword module specifies the file where you have stored the new LeapFrog class.
## Creating a new external engine
The external engines are used to interface external programs. In PyRETIS there is a generic ExternalMDEngine which you can make use of in order to set up new custom external engines. Before we give an overview of the methods you will have to implement in order to create a new engine, we list some useful methods which are already present in the generic ExternalMDEngine:
- ExternalMDEngine.execute_command() which can be used to execute system commands.
- Methods to interact with files:
- ExternalMDEngine._movefile()
- ExternalMDEngine._copyfile()
- ExternalMDEngine._removefile()
- ExternalMDEngine._remove_files()
- ExternalMDEngine._modify_input() which can be used to modify simple input files for the external software.
In order to create a new external engine, there are several methods that you need to implement. These methods are described in the next section. Before reading this description, please have a look at the figure below which illustrates the interaction between PyRETIS and the external engine:
### Implementing methods for a new external engine
The following methods must be implemented in the new engine:
- __init__ which does some set-up for the external engine.
- step which is a method for performing a single MD step with the external engine.
- _read_configuration For reading output (configurations) from the external engine. This is used for calculating the order parameter(s).
- _reverse_velocities For reversing velocities in a snapshot. This method will typically make use of the _read_configuration.
- _extract_frame For extracting a single frame from a trajectory.
- _propagate_from The method for propagating the equations of motion using the external engine.
- modify_velocities The method used for generating random velocities for shooting points.
Some examples/templates are shown in the following sections.
#### The __init__ method
The __init__ method sets up the engine and makes it ready for use. This can, for instance, involve checking that the required input files are in place and checking the consistency of the input files.
```python
def __init__(self, exe, input_path, timestep, subcycles):
"""Set up the engine.
Parameters
----------
exe : string
The engine executable, optionally followed by arguments.
input_path : string
Path to the directory containing the engine input files.
timestep : float
Time step used in the external engine.
subcycles : integer
Number of external steps performed per |pyretis| step.
"""
super().__init__('New external engine', timestep, subcycles)
self.exe = shlex.split(exe)
self.input_path = os.path.abspath(input_path)
for name in REQUIRED_FILES:
full = os.path.join(self.input_path, name)
if not os.path.isfile(full):
msg = f'Engine could not find input file "{name}"'
logger.error(msg)
raise ValueError(msg)
self.input_files[name] = full
```
In addition, this is a good point to add a method to execute the external program. Here we will make use of ExternalMDEngine.execute_command() and the ExternalMDEngine.exe_dir property. This property is set internally by PyRETIS and points to a directory which we will use when executing the external engine. As a concrete example, if we are generating a path for ensemble [1^+], this will point to the sub-directory generate of the ensemble-directory 002. Here is a short template for executing external commands:
```python
def _run_program(self, deffnm):
"""Execute the external engine in ``self.exe_dir``.
Parameters
----------
deffnm : string
Base name used for the output files of this run.
Returns
-------
out : dict
Mapping ``{kind: absolute path}`` for the files produced.
"""
self.execute_command(self.exe, cwd=self.exe_dir, inputs=None)
out_files = {}
for kind, template in OUTPUT_FILES.items():
full = os.path.join(self.exe_dir, template.format(deffnm))
if os.path.isfile(full):
out_files[kind] = full
return out_files
```
#### The step method
The step method is used to execute a single step with the external software. This method is used by the PyRETIS engine in the following way:
```python
conf = self.step(system, 'output-file')
```
That is, the method takes in two parameters: a system object and a string and returns a single parameter. This single parameter is just a string which contains the output configuration created after executing the external program. Here is a short template for the step method:
```python
def step(self, system, name):
"""Perform a single MD step with the external engine.
Parameters
----------
system : object like :py:class:`.System`
The system to advance.
name : string
Base name used for the output files of this step.
Returns
-------
conf : string
Path of the output configuration.
"""
self.dump_frame(system)
out_files = self._run_program(name)
conf = os.path.join(self.exe_dir, f'{name}.{self.ext}')
ekin, vpot = read_energies(out_files['energy'])
system.particles.set_pos((conf, None))
system.particles.set_vel(False)
system.particles.ekin = ekin[-1]
system.particles.vpot = vpot[-1]
return conf
```
#### The _read_configuration method
The _read_configuration method is used to get the actual positions and velocities from a configuration file. As a short template:
```python
@staticmethod
def _read_configuration(filename):
"""Return ``(box, xyz, vel)`` for a configuration file.
Used by :py:meth:`.ExternalMDEngine.calculate_order`.
"""
return read_configuration(filename)
```
#### The _reverse_velocities method
The _reverse_velocities method is used to reverse velocities for a single configuration.
```python
@staticmethod
def _reverse_velocities(filename, outfile):
"""Write ``outfile`` as ``filename`` with reversed velocities."""
box, xyz, vel = read_configuration(filename)
write_configuration(outfile, xyz, -vel, box=box)
```
#### The _extract_frame method
The _extract_frame method is used to extract a single snapshot from a trajectory file.
```python
def _extract_frame(self, traj_file, idx, out_file):
"""Extract frame ``idx`` of ``traj_file`` into ``out_file``.
Used by :py:meth:`.ExternalMDEngine.dump_config` when dumping
from a trajectory file. The implementation is engine specific;
for common formats one of ``mdtraj``, ``MDAnalysis`` or
``ase`` can do this in a few lines.
"""
raise NotImplementedError
```
#### The _propagate_from method
The _propagate_from method is used to propagate the equations of motion using the external engine. In some cases, this can make use of the step method, in other cases this is really distinct (e.g. in GROMACS this would correspond to extending a simulation).
```python
def _propagate_from(self, name, path, ensemble, msg_file, reverse=False):
"""Propagate using the external engine.
Called from :py:meth:`.ExternalMDEngine.propagate` after the
initial configuration (and any required velocity reversal) has
been written to disk.
Parameters
----------
name : string
Base name for the trajectory we are generating.
path : object like :py:class:`.PathBase`
Path object to fill with phase-space points.
ensemble : dict
Contains ``system``, ``order_function`` and ``interfaces``.
msg_file : object like :py:class:`.FileIO`
File used to log propagation progress.
reverse : boolean, optional
If True, the system is propagated backward in time.
Returns
-------
success : boolean
True if an acceptable path was generated.
status : string
Human-readable description of the outcome.
"""
status = f'propagating with new engine (reverse = {reverse})'
system = ensemble['system']
left, _, right = ensemble['interfaces']
traj_file = os.path.join(self.exe_dir, f'{name}.{self.ext}')
msg_file.write(f'# Trajectory file is: {traj_file}')
order = self.calculate_order(ensemble)
success = False
msg_file.write('# Step order parameter cv1 cv2 ...')
out_files = {}
for i in range(path.maxlen):
msg_file.write(f'{i} {" ".join(str(o) for o in order)}')
snapshot = {
'order': order,
'config': (traj_file, i),
'vel_rev': reverse,
}
phase_point = self.snapshot_to_system(system, snapshot)
status, success, stop, _ = self.add_to_path(
path, phase_point, left, right,
)
if stop:
logger.debug('Propagation ended at step %i: %s', i, status)
break
# Advance the engine and refresh the order parameter:
out_files = self._run_program(name)
system.particles.set_pos((traj_file, i + 1))
system.particles.set_vel(reverse)
order = self.calculate_order(ensemble)
msg_file.flush()
msg_file.write('# Propagation done.')
# If the engine writes energies for the whole run, hand them
# to the path object here, e.g.:
# ekin, vpot = read_energies(out_files['energy'])
# path.update_energies(ekin, vpot)
return success, status
```
#### The modify_velocities method
This method is used to draw random velocities, typically when performing the shooting move. The generic modify_velocities method is quite involved, but for most cases is sufficient to only implement aimless shooting. Here is a template:
```python
def modify_velocities(self, ensemble, vel_settings=None):
"""Draw new velocities for the current configuration.
Only aimless shooting is supported by this template. Soft
velocity perturbations and energy re-scaling can be added in
the same fashion as in :py:mod:`pyretis.engines.cp2k`.
Parameters
----------
ensemble : dict
Contains ``system`` and ``rgen``.
vel_settings : dict, optional
Velocity generation options:
* ``sigma_v`` - negative for aimless shooting; otherwise
a standard deviation for soft perturbations.
* ``zero_momentum`` - if True, reset the linear momentum;
a missing key means False, the ``[tis]`` default.
* ``rescale`` - if a positive float, rescale the total
energy to this value.
Returns
-------
dek : float
Change in kinetic energy.
kin_new : float
New kinetic energy.
"""
rgen = ensemble['rgen']
system = ensemble['system']
particles = system.particles
rescale = vel_settings.get('rescale') if vel_settings else None
if rescale is not None and rescale is not False and rescale > 0:
msg = 'New external engine does not support energy rescale!'
logger.error(msg)
raise NotImplementedError(msg)
if not is_aimless_velocity_setting(vel_settings):
msg = 'New external engine only supports aimless shooting!'
logger.error(msg)
raise NotImplementedError(msg)
pos = self.dump_frame(system)
box, xyz, vel = read_configuration(pos)
particles.vel = vel
kin_old = calculate_kinetic_energy(particles)[0]
new_vel, _ = rgen.draw_maxwellian_velocities(system)
particles.vel = new_vel
if is_zero_momentum_setting(vel_settings):
reset_momentum(particles)
kin_new = calculate_kinetic_energy(particles)[0]
conf_out = os.path.join(self.exe_dir, f'genvel.{self.ext}')
write_configuration(conf_out, xyz, particles.vel, box=box)
particles.set_pos((conf_out, None))
particles.set_vel(False)
particles.ekin = kin_new
dek = float('inf') if not kin_old else kin_new - kin_old
return dek, kin_new
```
## References
https://en.wikipedia.org/wiki/Leapfrog_integration
---
# Some common PyRETIS errors
In this section, we describe some of the most common PyRETIS warning or error messages and how these errors can be resolved.
If your problem is not covered here, or you want to report a bug or make a request, please get in touch.
## Simulation warnings/error messages
## Error: SyntaxError: invalid syntax
This is either a plain bug in the PyRETIS source code or it can be caused by using Python 2.x. If the bug is related to something like this:
```python
class RandomGeneratorBase(metaclass=ABCMeta):
^
```
This error is most likely caused by using Python 2.
Fix: Use Python 3, or report this as a bug.
## Error: Unknown engine XXX
An unknown engine was requested in the input.
Fix: Specify an engine in the PyRETIS input (see the engine section).
## Warning: These settings are not used by "X" and are being IGNORED
A keyword was given in a section that names a class – engine, orderparameter, collective-variable, potential – and the chosen class does not take it. Almost always a misspelling:
```
[orderparameter]
class = "pyretis_position"
dimm = "x" # <- 'dim', so this is ignored
```
These sections cannot be checked against a fixed keyword list, because what is accepted depends on which class you select. So the check happens where the class is known, when the object is built.
This matters because the run continues. An ignored keyword leaves the class default in place, so the simulation does not fail – it quietly samples something other than what you asked for.
Fix: correct the spelling, or remove the keyword if the class genuinely does not use it. The warning names the class, so its accepted keywords can be looked up in the input reference.
Keywords the framework itself consumes or injects – class, module, name, exe_path, type, rgen, seed, input_files, input_path, cp2k_format, parameter, and the engine input-file overrides conf, input_o, topology, index, template – are never reported. Note the flip side: a keyword on that list is silently accepted even by a class that ignores it (an [engine] seed on an engine that draws no random numbers does nothing, and nothing says so).
Keywords misspelled in the sections with a FIXED keyword list (such as simulation) are a hard error rather than a warning, listing everything unrecognised at once.
## Error: The main order parameter changed
A restart was given a different first order parameter than the one the loaded paths were sampled with. The first order parameter is the progress coordinate: it defines the interfaces and drives the sampling, so paths sampled with a different one do not belong to the ensembles this run describes.
PyRETIS detects this by recomputing the order parameter from the stored frames and comparing it against the value recorded when the path was sampled. The message names the frame, both values and the tolerance used.
Fix: restore the original [orderparameter] section. If you genuinely want to sample a different reaction coordinate, that is a new simulation, not a restart.
Note that collective variables may be added or removed freely on a restart — only the first order parameter is fixed. See adding collective variables when you restart.
## Warning: The order parameter produces N value(s) but the paths being loaded carry M
The input gained (or lost) a collective-variable section since the run being resumed. This is supported: the loaded paths are re-read from their trajectory files so the added collective variable is computed for them too, and every ensemble writes one consistent set of columns from here on.
Fix: none needed if the change was deliberate. If it was not, you have probably restarted in the wrong directory, or edited the input between runs by accident.
Be aware that order.txt is appended to, so the blocks written by the earlier run keep the old column count; only frames sampled from the restart onwards carry the new collective variable.
## Warning: No energy.txt for the path loaded from "X"
A path was loaded from a directory that holds coordinates but no energies — the normal case for a sparse load, where you supply only the trajectory.
Fix: none needed. Engines that can recompute the per-frame energies from the stored frames (the internal integrators) do so, and the energies appear in the output as usual. An external engine reports energy through its own files and cannot recover them after the fact, so for those the terms are written as nan.
## Warning: No random seed given. Will just use "0"
If a random generator is needed and the seed value has not been set, PyRETIS will use a seed equal to 0. This message is just to remind the user that not setting a seed is the same as setting it to zero.
Fix: Are you ok with this seed? If not, change it in the input settings.
## Error: XXX settings not found: YYY/ValueError('Please update settings!')
For simulation type XXX the required settings YYY was not found in the input. Please read the manual describing the simulation you wish to perform in order to see what simulation settings you will need to set.
Fix: Update your input settings.
## Plotting warnings/error messages
## Warning: UserWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.
PyRETIS supports different versions of matplotlib. In the most recent matplotlib versions, the color cycle is set using axes.prop_cycle, while axes.color_cycle was used in previous versions. Currently PyRETIS will try to set axes.color_cycle as the most recent version of matplotlib will accept this, albeit with some complaining (i.e. the warning message given above).
Fix: N/A. Note that this behaviour might change in the future if support for older matplotlib versions is dropped from PyRETIS.
## Warning: Using Matplotlib version < 1.4.0, please upgrade it!'
We recommend using a more recent matplotlib version than 1.4. These versions support styles for plotting which makes it easier to customize the figures for the reports.
Fix: Upgrade matplotlib.
## Overriding color cycle
When plotting with matplotlib, the selection of colors is typically handled by matplotlib and the selected plotting style. When plotting a lot of data series in the same plot, we might run out of distinct colors. PyRETIS will then create some ad-hoc colors so that the series can be distinguished. Note that these colors may not be good enough for colorblindness or gray-scale images. Here, we recommend writing your own plotting routine or updating the color cycle settings in the plot settings file.
Fix: Make a custom plot or modify the PyRETIS matplotlib style.
## Analysis warnings/error messages
## Warning: No accepted paths to analyse in ensemble N / Skipping ensemble [N^+] in probability matching
An ensemble has not accepted a single path. This is normal early in a run, or for the outermost ensembles of a short one, and it is also what you get when -skipb/-skipe discard the whole record.
Fix: none needed — sample more cycles. The analysis and the report both continue:
- The ensemble still gets its figure panels. They are drawn empty, with the axes and the ensemble name in place and “No accepted paths in this ensemble” across the middle, so an empty ensemble is visibly different from one that was never set up.
- It still gets its row in every table, with the numbers it has none of shown as n/a.
- Quantities that need it are reported as unavailable rather than computed from nothing — the [0^-] flux length, and the permeability, which needs \xi and \tau from that ensemble.
If no ensemble accepted anything, the report is still produced; the matched-probability figure then says so.
## Ignored unknown mc move: XXX
This warning is given when the analysis encounters a path that was not generated by any of the known (to PyRETIS) Monte Carlo moves. These paths are NOT included when obtaining path lengths for histograms in the analysis.
Fix: If this happens for your own modified version of PyRETIS, double check the path analysis routines and the method generating the path.
## The number of paths stored in path ensemble does not correspond to the number of paths seen by the path ensemble! Consider re-running the analysis using the path ensemble file!
The analysis can be run on either path ensemble objects or on path ensemble files. The path ensemble object stores a finite number of paths (or rather representation of paths) which means that the path ensemble does not necessarily store (in memory) all the paths – some will be stored in a path ensemble file when the path ensemble object is “full”. This warning means that this has happened and that it will be more correct to run the analysis using the path ensemble file.
Fix: Run the analysis using the path ensemble file as the source (see the section describing the analysis tool).
---
# CECAM workshop: A single particle in a 1D potential
This is a short user guide for running the 1D potential example with the RETIS algorithm for the CECAM school “Multiscale Simulations of Soft Matter with Hands-On Tutorials on ESPResSo++ and VOTCA”, Schloss Waldthausen in Mainz, October 10 to 13, 2016.
# Table of Contents
- Introduction and installation
- Running the example script
- Modifying the RETIS simulation
- References
## Introduction and installation
In this example, we will use the RETIS algorithm to investigate the transition between two states for a particle moving in a 1D potential. The potential is given by V_{\text{pot}} = a x^4 - b (x-c)^2 where x is the position and a, b and c are potential parameters.
The script for running a RETIS simulation can be downloaded here: retis_movie.py. But before we can run this script, we need to install the PyRETIS library and matplotlib:
- The PyRETIS library is distributed in the Python Package Index and can be installed using pip:1
```bash
pip install pyretis
```
- If you want to install the library system-wide, you will need super-user access (typically a sudo will do). If you don’t have super-user access, pip works well with virtualenv and we refer to the virtualenv user guide for more information about setting this up. 2
- Note: If you have a previous installation of PyRETIS (the newest release is version 4.0.0.dev0), it can be upgraded using
```bash
pip install --upgrade pyretis
```
- Matplotlib can also be installed using pip. However, for the best performance we recommend that you follow a guide specific for your operative system. Please see the matplotlib documentation. 3 If you are sure that all matplotlib requirements are satisfied, you can install it directly using pip:
```bash
pip install matplotlib
```
After installing the PyRETIS library, please check that is has been properly set up by running the following command (at the command line):
```bash
python -c 'import pyretis; print(pyretis.__version__)'
```
which should print out the version of your installed PyRETIS library.
## Running the example script
Download the example script retis_movie.py to a location on your computer. Now this script can be executed by running
```bash
python retis_movie.py
```
which should display an animation similar to the image shown below.
The bulk of this script handles the plotting, and we will not go into details on how matplotlib is used to plot the result. We will in the following show highlight some changes you can do to modify the RETIS simulation.
If you complete the full 20000 cycles, you can compare your results with the previously reported data of van Erp. 4 5
## Modifying the RETIS simulation
The dictionary SETTINGS in the retis_movie.py script defines the simulation. Changing the values in this dictionary will modify the simulation.
Here are some examples:
- Changing the number of steps:
- The number of steps is changed by changing the dictionary for SETTINGS['simulation']. From 20000 steps
```python
SETTINGS['simulation'] = {'task': 'retis',
'steps': 20000}
```
- to just 100:
```python
SETTINGS['simulation'] = {'task': 'retis',
'steps': 100}
```
- Changing the interfaces:
- The interfaces are defined in a list:
```python
SETTINGS['interfaces'] = [-0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, 1.0]
```
- We can, for instance, see what happens if we just use two interfaces:
```python
SETTINGS['interfaces'] = [-0.9, 1.0]
```
- Changing the temperature:
- The temperature is defined by:
```python
SETTINGS['system'] = {'units': 'lj', 'temperature': 0.07}
```
- and can be changed by giving it a different value:
```python
SETTINGS['system'] = {'units': 'lj', 'temperature': 0.21}
```
- Changing the potential parameters:
- The potential parameters is defined by:
```python
SETTINGS['potential'] = [{'a': 1.0, 'b': 2.0, 'c': 0.0,
'class': 'DoubleWell'}]
```
- We can change the parameters directly, e.g.:
```python
SETTINGS['potential'] = [{'a': 0.5, 'b': 1.0, 'c': 0.0,
'class': 'DoubleWell'}]
```
- Note: It is a good idea to plot the potential if you change the parameters. This will allow you to check the position of the interfaces.
## References
The pip user documentation, https://pip.pypa.io/en/stable
The virtualenv user guide, https://virtualenv.pypa.io/en/stable/userguide/
The matplotlib installation instructions, http://matplotlib.org/users/installing.html
Titus S. van Erp, Dynamical Rare Event Simulation Techniques for Equilibrium and Nonequilibrium Systems, Advances in Chemical Physics, 151, pp. 27-60, 2012, https://doi.org/10.1002/9781118309513.ch2
https://arxiv.org/abs/1101.0927
---
# Molecular modelling: Introduction to RETIS
# Table of Contents
- Introduction
- Installing PyRETIS and running the exercise
- Installing PyRETIS
- Running the exercise
- Part 1 - Introduction
- Part 2 - Improving the efficiency
- References
## Introduction
In this exercise, you will explore a rare event with the Replica Exchange Transition Interface Sampling (RETIS) algorithm.
We will consider a simple 1D potential where a particle is moving. The potential is given by V_{\text{pot}} = x^4 - 2 \times x^2 where x is the position. By plotting this potential, we see that we have two states (at x=-1 and x=1) separated by a barrier (at x=0):
Using the RETIS algorithm, we will compute the rate constant for the transition between the two states. We will also try to position the RETIS interfaces to maximise the efficiency of our calculation. The exercise is structured as follows:
- Installing PyRETIS and running the exercise. Short version:
- Install PyRETIS (pip install pyretis)
- Install matplotlib (pip install matplotlib)
- Download retis.py
- Download retis_movie.py
- Part 1 - Introduction
- Part 2 - Improving the efficiency
- References
How to pass:
- From Part 2 - Improving the efficiency, show the teaching assistant the results from your best simulation. Here, “best” is the simulation with the fewest force evaluations and lowest uncertainties for a fixed number of steps as specified in the exercise. The results to show the teaching assistant are the interfaces you have used, the number of force evaluations, the initial flux, the crossing probability and rate constant with error estimates.
## Installing PyRETIS and running the exercise
### Installing PyRETIS
In this example, we will make use of the PyRETIS library for carrying out the RETIS simulations and we will use matplotlib for plotting some results. These two libraries will have to be installed before we start:
- The PyRETIS library is distributed in the Python Package Index and can be installed using pip. 1 Typically, pip is included when you install Python, but if this is not the case you can install it manually. This can be done by following the installation pip installation instructions:
- Download the get-pip.py script from the pip web-page.
- Install pip using the terminal:
```bash
python get-pip.py
```
- or
```bash
python get-pip.py --user
```
- The --user option will allow you to install pip locally for your own user when you do not have super-user access to the computer you are working on.
- As already mentioned, the PyRETIS library is distributed in the Python Package Index and can be installed using pip 1 and the following command:
```bash
pip install pyretis
```
- Note that this command might require you to have super-user access. If this is not the case, you can try to install PyRETIS locally for your own user using a virtual environment or with pip locally:
```bash
pip install pyretis --user
```
- For information about setting up a virtual environment, we refer to the virtualenv user guide. 2
- Note: If you have a previous installation of PyRETIS (the newest released version is: 4.0.0.dev0), it can be upgraded using
```bash
pip install --upgrade pyretis
```
- Matplotlib can also be installed using pip. However, for the best performance we recommend that you follow a guide specific for your operative system. Please see the matplotlib documentation. 3 If you are sure that all matplotlib requirements are satisfied, you can install it directly using pip:
```bash
pip install matplotlib
```
### Running the exercise
We will use two Python scripts in this example to run the RETIS simulation. One will display an animation on the fly, while the other script will just output some text results.
Download the example script retis_movie.py to a location on your computer. This script can be executed by running the command
```bash
python retis_movie.py
```
in your terminal. This should display an animation similar to the image shown below.
The bulk of this script handles the plotting, and we will not go into details on how matplotlib is used to plot the result. We will rather focus on the RETIS algorithm and how we can use it to calculate rate constants. But before we do that, let us also try the text-based script. This script will give the same output as the retis_movie.py script, but it will not display the animation and will thus run slightly faster. Download the example script retis.py to a location on your computer and execute it by running the command
```bash
python retis.py > name_of_file.txt
```
which should display a progress bar showing estimating the time left for your simulation. In addition, text output from the simulation is written to the file name_of_file.txt:
The RETIS method which is used in this exercise aim to calculate the rate constant k_{\text{AB}} for the transition between the two states shown in fig1dpot. This is accomplished by generating reactive trajectories connecting the reactant state and the product state. We define these two states using two interfaces (the vertical lines in fig1dpot) which we label \lambda_\text{A} and \lambda_\text{B} respectively. That is, left of \lambda_\text{A} corresponds to the reactant state and right of \lambda_\text{B} corresponds to the product state. Since the probability for the actual crossing can be very small, we introduce several more interfaces which help us obtain the crossing probability. These interfaces define the so-called path ensembles, labeled [0^-], [0^+], [1^+], \ldots and so on as shown in figretistxt. The path ensemble [i^+] contain trajectories that start at \lambda_\text{A}, then cross an intermediate interface \lambda_i before reaching \lambda_\text{B} or returning to \lambda_\text{A}. As you may have noticed, there is a special path ensemble labelled [0^-] which contains paths that start at \lambda_\text{A}, then go in the opposite direction (i.e. to the left of \lambda_\text{A}) before returning to \lambda_\text{A}. This ensemble is used to calculate how often we cross \lambda_\text{A} and head towards the product state at \lambda_\text{B}. Now, if we know how often we cross \lambda_\text{A} and head for the product state and the probability of reaching this state given that we crossed \lambda_\text{A} then the rate constant can be obtained according to
where f_{\text{A}} is the initial flux and P_{\text{A}} (i+1|i) are intermediate crossing probabilities, obtained using the different path ensembles. The initial flux is obtained using the path lengths in ensembles [0^-] and [0^+] as
where \langle t_{\text{path}}^{[k]} \rangle denotes the average path length in ensemble [k]. The intermediate probability P_{\text{A}} (i+1|i) is the probability of path reaching the next (i+1) interface given that it originated in \lambda_\text{A}, ended in \lambda_\text{A} or \lambda_\text{B} and had at least one crossing with \lambda_\text{i}. This probability can be obtained as the fraction of paths in the [i^+] ensemble that cross \lambda_{i+1}. It is these probabilities that are labelled P_\text{cross} in figretistxt.
The optimal number of path ensembles to use in a RETIS simulation is a trade-off between not having to do too many path simulations and not having too small crossing probabilities (the P_\text{cross}-values in figretistxt). By making some basic assumptions on the path lengths etc. one can establish a theoretical optimum which should be obtained when all crossing probabilities are around 0.2 (the derivation of this can be found in Ref. 6). However, the assumptions are never completely valid in realistic systems. Hence, the optimum is presumably close to having all probabilities equal to 0.2 but not exactly (but somewhere in the range 0.1 till 0.9).
As stated above, the RETIS method generate reactive trajectories connecting the initial and final state using the moves shown in figretistxt and described in tabmoves. The outcome of each attempted RETIS move is given a three letter abbreviation as described in tababbrev.
### Abbreviations for the RETIS moves
Abbreviation
Description
swap
A RETIS swapping move.
nullmove
Just accepting the last accepted path once again.
tis (sh)
A TIS shooting move
tis (tr)
A TIS time-reversal move
### Abbreviations for the RETIS statuses
Abbreviation
Description
ACC
The path has been accepted.
BWI
When integrating backward in time for a shooting move we arrived at the right interface and not the left one, i.e. we reached the Wrong Interface.
BTL
When integrating backward in time for a shooting move, the backward trajectory will have a maximum length determined by the RETIS algorithm in order to obey detailed balance. This maximum varies each time: it is the old path length divided by a random number between 0 and 1. BTL means that we did not reach any of the outer most interfaces before we exceeded this maximum path length.
BTX
We also have a maximum length for trajectories in order to limit the memory the trajectories use. BTX means that the trajectory length exceeded this limit before we reached an interface. This limit implies that we ignore very long trajectories and, therefore, it will result in a systematic error. The number of BTX or FTX occurrences can be reduced by increasing the parameter maxlength in the input script.
FTL
Similar to BTL in the Forward direction.
FTX
Similar to BTX in the Forward direction.
NCR
No crossing with the interface which has to be crossed in order to be part of the specific path ensemble. This can happen when we attempt swapping move or when the shooting move goes backward and forward in time to the left interface without making sufficient progress along the reaction coordinate.
The tables above list the most common codes encountered in this exercise. For a complete reference of all move types and path status codes used by PyRETIS, see user-guide-path-types-rejections.
Now, hopefully, you are able to execute the two example scripts we will be using.
Before we start with the exercise we give a few examples on how you can change the input scripts in order to modify your simulation. Typically, this involves making changes in the SETTINGS dictionary defined in the Python scripts. In fact the RETIS simulation you are running is defined with this Python dictionary. In this exercise, we will just change a few of these settings and we give some examples here:
- In order to run a longer simulation you need to change the steps keyword. For instance from 150 steps to 20000. If you run the simulation for 20000 steps you can compare your results with the previously reported data of van Erp. 4 5 The required change is from:
```python
SETTINGS['simulation'] = {'task': 'retis',
'steps': 150,
'interfaces': INTERFACES}
```
- to:
```python
SETTINGS['simulation'] = {'task': 'retis',
'steps': 20000,
'interfaces': INTERFACES}
```
- The number of interfaces to use in the RETIS algorithm is defined in a list:
```python
INTERFACES = [-0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, 1.0]
```
- We can, for instance, see what happens if we just use fewer interfaces:
```python
INTERFACES = [-0.9, -0.8, -0.6, -0.4, -0.3, 1.0]
```
- We can also change settings for the RETIS algorithm itself:
- The settings for the RETIS method is split into two parts, one is RETIS specific and the other is TIS specific. The former controls swapping while the latter the shooting. In this exercise, we will just change the frequencies of the different moves. The percentage of swapping are controlled by the swapfreq keyword of the RETIS settings:
```python
# RETIS specific settings:
SETTINGS['retis'] = {'swapfreq': 0.5,
'nullmoves': True,
'swapsimul': True}
```
- Here, 50 % (a fraction of 0.5) will be swapping moves. What about the remaining 50% of the moves? These will be TIS moves and here we have two options - shooting or time reversal. The relative frequency of these moves is determined by the freq keyword in the TIS specific settings.
```python
# TIS specific settings:
SETTINGS['tis'] = {'freq': 0.5,
'maxlength': 20000,
'allowmaxlength': False,
'sigma_v': -1,
'seed': 0,
'zero_momentum': False,
'rescale_energy': False}
```
- Here, 50% will be shooting moves and 50% will be time-reversal moves. Note that this is as a percentage of the TIS moves, so in total, we will have 50% swapping, 25% shooting and 25% time reversal.
- What happens with a swapfreq equal to 0.8 and a freq equal to 0.6? We will have 80% swapping moves, 12% (0.2\times0.6) time reversal moves and 8% (0.2 \times (1-0.6)) shooting moves.
## Part 1 - Introduction
We will warm up with a few questions about the RETIS method and we will run a few short simulations to see how the method works in practice.
- What “problem” are we solving with the RETIS method? Why don’t we just run a brute-force simulation?
- How would you (in just a few sentences) describe what happens when you run a RETIS simulation. Some keywords to get you started: swapping, shooting, time-reversal, path ensembles, rate, crossing probability… Here you can also try to explain what is displayed in the animation shown by running retis_movie.py.
- In the RETIS method, we have three main moves which generate new paths. These are the shooting, time-reversal and swapping moves. How do these moves generate new paths? Which move is the most computationally demanding? Which one is the least computationally demanding?
- Let us check your answers to the previous question by performing two short RETIS simulations with the retis_movie.py script using different swapping frequencies. One of the outputs from the retis_movie.py script is the number of force evaluations which is a measure of the computational cost. Compare the total number of force evaluations in these two cases:
- Set steps to 1000 and swapfreq to 0.2 and run a RETIS simulation using:
```bash
python retis_movie.py > swap-0.2.txt
```
- Set steps to 1000 and swapfreq to 0.8 and run a RETIS simulation using:
```bash
python retis_movie.py > swap-0.8.txt
```
- In which of these two cases are the number of force evaluations greatest? Does this agree with your answer to question 3? What happens if you increase the freq keyword? Is this as expected? Note that the total number of force evaluations can be found at the end of the text output from the script.
- Why do we use so many path ensembles in our RETIS simulation? Explore what happens when you change the number of interfaces in the retis_movie.py script. For instance, reduce the number of interfaces to just two:
```python
INTERFACES = [-0.9, 1.0]
```
- and set the number of steps to 4000. Do you expect this to change the total number of force evaluations dramatically, compared to a case with 8 interfaces running for 1000 steps? Compare with a simulation using the interfaces
```python
INTERFACES = [-0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, 1.0]
```
- for 1000 steps. In these two simulations you can also set
```python
PCROSS_LOG = True
```
- to show the crossing probabilities on a log scale.
- The swapping move is the defining move for the RETIS algorithm and is essentially the move that separates it from the TIS algorithm. What is the purpose of the swapping move? What will happen if we remove it? Investigate this using the retis_movie.py script and setting the frequency of swapping moves to 0. Here, you can run 1000 steps and compare with the output from the previous question. Use the same 8 interfaces and set PCROSS_LOG = TRUE.
## Part 2 - Improving the efficiency
In the last part in the previous section, you ran several very short RETIS simulation. Here we will run some longer simulations and we will see how efficiently we can run the RETIS simulations. For instance, if we set the interfaces to
```python
INTERFACES = [-0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, 1.0]
```
steps to 2000 and the frequencies to 0.5, the output will be similar to:
```python
# Flux: 0.285125 +- 0.00956114
# Crossing probability: 8.10682e-07 +-4.9152e-07
# K_AB: 2.31146e-07 +- 1.40359e-07
```
with a total number of force evaluations equal to about 3.9 million. As you can see there are rather large uncertainties, because 2000 is in fact a small number of steps. In this part of the exercise, we will see if we can improve the situation and our goal is to lower (if possible) the number of force evaluations and the uncertainty. Here, you can proceed as you like, but to get you started here are some suggestions:
- First, consider how you would compare two simulations. Say that we have the following results:
- A simulation with 10^6 force evaluations and an error (relative) in the rate constant of 10%.
- A simulation with 2\times 10^6 force evaluations and an error (relative) in the rate constant of 5%.
- Which one of these two simulations would you say performs better?
- Run first some short simulations (say 100-200 steps) where you test out different positions of interfaces and also a different number of interfaces. Here you should make use of both the retis_movie.py and retis.py scripts to investigate how your changes influence the results.
- After having found a set of interfaces you are happy with, run a longer simulation (2000 steps) with the retis.py script and report your results to the teaching assistant. How do your results compare to the results given above? Are you able to perform better? Note: It will take around 7-8 minutes to complete the 2000 cycles.
## References
The pip user documentation, https://pip.pypa.io/en/stable
The virtualenv user guide, https://virtualenv.pypa.io/en/stable/userguide/
The matplotlib installation instructions, http://matplotlib.org/users/installing.html
Titus S. van Erp, Dynamical Rare Event Simulation Techniques for Equilibrium and Nonequilibrium Systems, Advances in Chemical Physics, 151, pp. 27-60, 2012, https://doi.org/10.1002/9781118309513.ch2
https://arxiv.org/abs/1101.0927
T. S. van Erp and P. G. Bolhuis, Elaborating transition interface sampling methods, J. Comput. Phys. 205, 157-181 (2005), https://doi.org/10.1016/j.jcp.2004.11.003
---
# PyRETIS exercises
Here, we list some exercises in which PyRETIS has been used to demonstrate some different simulation methods.
---
# User Guide
The guide is ordered as a workflow: first the background and entry points, then how to prepare and run simulations, then analysis, visualisation and reference material.
---
# Getting the initial path
Before a TIS, RETIS or REPPTIS simulation can start, every path ensemble needs one valid initial path — a trajectory that already satisfies the ensemble’s interface conditions. Generating these starting paths is a step in its own right, and PyRETIS offers four routes, summarised here and configured through the initial-path section.
Which route to use:
- No trajectory yet, simple system → kick.
- No trajectory yet, hard-to-reach barrier → explore first to map the order parameter, then kick or load.
- You already have a trajectory (MD or a previous run) → load (and its sparse variants).
- Continuing an interrupted run → restart (see the reference).
## Kick: MC + MD generation from a single configuration
The kick method builds initial paths from scratch, needing only a single starting configuration. For each ensemble it randomises the velocities (a Monte-Carlo kick), integrates the dynamics (MD) one step, and accepts the move when it gets closer to the ensemble’s middle interface — repeating until a crossing is found, then shooting forward and backward to complete the path.
```toml
[initial-path]
method = "kick"
kick-from = "previous"
```
The kick-from option controls how multi-ensemble runs are seeded:
- kick-from = "initial" (default) — every ensemble is kicked from the same user configuration. Simple, but ensembles whose interface lies far from the start may need many iterations.
- kick-from = "previous" (recommended for RETIS) — only the first ensemble starts from the user configuration; each later ensemble is kicked from the frame of the previously generated path closest to (and left of) the next interface, propagating the seed through interface space.
Every engine can kick-initiate a path, including ones with no usable single MD step (CP2K, LAMMPS streaming, ASE, TurtleMD): they search for the crossing by advancing propagate() one step at a time instead. With kick-from = "initial", setting kick-parallel = true kicks every ensemble in parallel (one job per worker) instead of sequentially in one process — validated for TurtleMD so far; internal engines should keep kick-parallel = false for now.
The full keyword list is in the kick reference. Every standard 1D tutorial uses kick, e.g. TIS in a 1D potential and RETIS in a 1D potential.
## Explore: mapping the order parameter first
When a good order parameter or a sensible interface placement is not yet known, the explore task runs an exploratory simulation that wanders the order-parameter space without enforcing interface conditions. Its output (the visited order-parameter range and the trajectories) helps you place interfaces sensibly before committing to a TIS/RETIS run, and the generated trajectories can themselves be loaded as initial paths.
```toml
[simulation]
task = "explore"
[initial-path]
method = "kick"
```
See the explore tutorial for a runnable 1D double-well example.
## Load: reusing an existing trajectory
The load method initialises the ensembles from trajectory data you already have — a molecular-dynamics run or a previous (P)RETIS simulation — instead of generating paths from scratch. PyRETIS distributes the frames across the ensemble subfolders, recomputes the order parameter from the atomic positions (so you can change the order parameter definition without regenerating trajectories), and validates / cleans each path.
```toml
[initial-path]
method = "load"
load_folder = "load"
```
Two folder layouts are supported — a single shared trajectory placed directly in load_folder, or per-ensemble 000/, 001/, … subfolders. With load_and_kick = True the loaded frames become a seed pool: if they cannot form a valid path, PyRETIS falls back to a kick from the nearest frame. The layouts and all keywords are documented in the load reference.
## Load (sparse): loading only the frames you kept
You do not need the full, densely sampled trajectory on disk. A sparse load reconstructs each path from a small set of stored frames plus a traj.txt index, with PyRETIS recomputing the order parameter and filling in the path bookkeeping. This is the common case when reusing trajectories where only selected frames were saved. Two runnable references with the internal engine ship under the test suite:
- load-traj — a shared trajectory referenced from traj.txt.
- load-frames — individual per-ensemble frame files.
Both run with pyretis run -i retis.toml -p exactly like the load method above; they differ only in how the kept frames are laid out.
## Restart: continue an interrupted run
restart does not construct new paths at all — it resumes a simulation from the pyretis.restart state file a previous run wrote, picking the sampling up exactly where it stopped (the same active paths, random-number state and cycle counter). Use it to extend a finished run to more cycles, or to recover one that was interrupted:
```toml
[initial-path]
method = "restart"
```
flexible_restart = True relaxes the consistency checks when the restart settings have changed (for example a different cycle target). The full keyword list is in the restart reference.
### Adding collective variables when you restart
A simulation often runs for a while before you realise there is another quantity you would like recorded. You do not have to start over: add a collective-variable section to the input (its keywords are the same as an orderparameter section) and restart as usual.
```toml
[orderparameter]
class = "pyretis_position" # unchanged -- see the warning below
dim = "x"
index = 0
[[collective-variable]] # new
class = "pyretis_distance"
index = [0, 1]
[initial-path]
method = "restart"
```
The paths being resumed were sampled without the new column, so on startup PyRETIS re-reads their frames and computes the added collective variable for each one. The trajectories are still on disk and a collective variable is a deterministic function of a frame, so the values are recovered exactly rather than guessed, and from the first new cycle every ensemble writes one consistent set of columns. You will see a warning naming the old and new column counts.
Two things are worth knowing.
The history keeps its old width. order.txt is appended to, so the blocks written before the restart still have the old number of columns. Only frames sampled from the restart onwards carry the new collective variable. If you want a file with one uniform width, restart into a fresh directory.
The sampling is unaffected. Adding a collective variable is an observation, not a change to the simulation: the progress coordinate that the interfaces are defined on is kept exactly as it was sampled, so the continuation follows the very same trajectory it would have without the new column. (The one exception is a velocity-dependent collective variable, such as pyretis_velocity. It forces the order parameter to be re-evaluated whenever a path is time-reversed, which re-derives the values at full precision instead of the six decimals order.txt stores. In our validation runs the moves, acceptances and paths were identical; in principle a borderline acceptance could flip, since the full-precision value may differ from the stored one exactly at an interface. Reported order values can differ in the ninth significant digit.)
Collective variables can be removed on a restart in the same way; the dropped columns simply stop being written.
The first order parameter is the progress coordinate: it defines the interfaces and drives the sampling. It must stay the one the loaded paths were sampled with. PyRETIS verifies this by recomputing it from the frames, and refuses to start if it has moved — continuing would mix two different reaction coordinates in a single data set. See the error description.
The complete keyword reference for every method (including restart and flexible_restart) is in the initial-path section of the input documentation.
---
# Running PyRETIS
PyRETIS is executed using the pyretis run application and a PyRETIS input file:
```pyretis
pyretis run -i INPUT
```
where INPUT is the input file. This will produce output files which can be analysed using the pyretis analyse application.
In the following, we describe the syntax for the PyRETIS input file.
## The PyRETIS input file
PyRETIS simulations can be set up and run with a simple input file. The input file defines a simulation by setting values for keywords which are organised into sections. Here we will discuss the following:
- Structure of the input file
- Formatting of keywords
- Comments
- Summary
- How much you actually have to write
- The sections in the input file
### Structure of the input file
The input file is written in TOML. It is organised into sections, written as a name in square brackets, and each section sets keywords with keyword = value:
```toml
[simulation]
task = "md-nve"
steps = 100
```
This sets the two keywords task and steps of the simulation section. A section runs until the next section name, so the order of the keywords inside it is free, and the order of the sections themselves is free as well.
Some sections are nested, written as a dotted name:
```toml
[particles.position]
input_file = "initial.xyz"
```
### Formatting of keywords
Section names and keywords are lower case, and the file is case-sensitive: [Simulation] and TASK are refused, with a message naming the section or keyword it could not place.
Each value carries its type. Text is quoted, numbers are written plainly, booleans are true and false, and a list is written in square brackets:
```toml
[tis]
maxlength = 20000
sigma_v = -1
high_accept = true
shooting_moves = [
"sh",
"sh",
"wf",
]
```
A value that is itself a set of named settings is written as a table, either inline or as its own section:
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
```
Because a quoted value is taken exactly as written, the names of external Python modules and classes, and the names of files, keep their case:
```toml
[engine]
class = "MyExternalClass"
module = "filename.py"
```
PyRETIS expects the file filename.py under exactly that name. The same holds for a unit you define yourself, where "m" and "M" are different units:
```toml
[unit-system]
length = [1.0, "m"]
```
### Comments
A # starts a comment, and the rest of the line is ignored. A comment can stand on its own line or follow a value:
```toml
# The settings that define the simulation.
[simulation]
task = "md-nve" # not TIS this time
steps = 100
[system]
temperature = 1.0
```
### Summary
- The input file is TOML. It is organised into [sections] whose keywords are set with keyword = value.
- Section names and keywords are lower case, and the file is case-sensitive.
- Text is quoted, booleans are true and false, and lists are written in square brackets.
- Comments are marked with a #.
### How much you actually have to write
Most of the sections below are optional. PyRETIS only needs to be told the things that define your problem – the potential, what to run, the system, how it is propagated, where it starts, and how progress is measured – and supplies defaults for the rest, including the move set, the initiation, the output frequencies, the path-length cap and the simulation box.
Twelve keywords, across five sections, plus the potential, are enough to run a complete RETIS simulation. This is the whole input file:
```toml
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "retis"
steps = 100
interfaces = [-0.9, -0.8, -0.5, 1.0]
[system]
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
[particles.position]
input_file = "initial.xyz"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
```
Each of the twelve states something only you can state:
- simulation – task what to run, steps for how long, and interfaces where the interfaces that define the path ensembles lie.
- system – dimensions and temperature.
- engine – class how the dynamics are propagated, with the settings that engine needs, here timestep and the Langevin friction gamma.
- particles – input_file, the configuration the run starts from.
- orderparameter – class, dim and index: what progress is measured along.
The potential above it is the energy landscape. Everything else has a default: the move set, the initiation, the output frequencies, the path-length cap and the box.
A minimal input file walks through this same file and lists what gets defaulted. When a run starts it writes out the fully resolved settings, so you can always see what every default resolved to for that run.
### The sections in the input file
The following sections are recognised by PyRETIS:
- simulation: For defining the simulation we are going to run.
- system: For defining system properties.
- box: For defining a simulation box.
- particles: For defining the initial state of particles.
- forcefield: For defining a forcefield.
- potential: For defining potential functions to use in the force field.
- engine: For defining the simulation engine.
- orderparameter: For defining the order parameter.
- retis: For defining settings for a RETIS simulation.
- repptis: For defining settings for a PPTIS or REPPTIS simulation.
- initial-path: For defining settings to initialize a RETIS simulation.
- tis: For defining settings for a TIS simulation.
- output: For defining output settings.
- unit-system: For defining custom unit systems.
In addition, there are analysis specific settings which can be set by making use of the following section(s):
- analysis: For defining an analysis.
---
# Obtaining and installing PyRETIS
You are reading the documentation for PyRETIS 4 (v4.0.0.dev), a development version that is not yet released on PyPI. To use the version these docs describe, install PyRETIS from source with git: clone the repository and check out the develop branch. The step-by-step instructions – including the development requirements – are given in Installing a development version later on this page. If you want the latest stable release (PyRETIS 3) instead, install it directly with pip install pyretis.
PyRETIS and its requirements can be installed using pip or using conda as described below. We also describe how it can be installed directly from the source code using git.
## Requirements for PyRETIS
In order to run PyRETIS, several Python libraries are needed, for instance SciPy, NumPy, and matplotlib (see also the information on installing the SciPy Stack). A list of the requirements can be found in the file requirements.txt in the source code directory. These packages can be installed by:
```pyretis
python -m pip install -r requirements.txt
```
after downloading the requirements.txt file. This should be automatically done if you are installing PyRETIS using pip/conda. Notes: (1) the analysis and visualization package PyVisA requires PyQt5 and graphviz, which must be installed separately as described below.
## Prerequisites
PyRETIS requires Python 3.11 or higher. Please make sure that you have an updated version of Python installed on your system. PyRETIS depends on several other Python packages, however, these should be installed automatically as part of the installation of PyRETIS. The requirements list is shown above.
## Installing via pip
The latest version of PyRETIS can be installed using pip with the following command:
```pyretis
python -m pip install git+https://gitlab.com/pyretis/pyretis.git
```
A direct installation of PyRETIS 4 (stable release):
```pyretis
python -m pip install pyretis
```
PyRETIS offers an analysis and visualization tool, named PyVisA. Its GUI requires PyQt5, mdtraj, scikit-learn, and graphviz. To install them via pip:
```pyretis
python -m pip install pyqt5 mdtraj scikit-learn graphviz
```
Note: Please make sure that you are using a Python environment of version 3.11 or newer. On some systems, Python 2 may still be the default version and pip might actually try to install for Python 2. In this case, use pip3 in the command above.
## Installing via conda
PyRETIS can be installed using conda with the following commands:
```pyretis
conda create --name pyretis python=3.12
conda activate pyretis
conda install pyretis -c conda-forge
```
PyRETIS offers an analysis and visualization tool, named PyVisA. Its GUI requires PyQt5, mdtraj, scikit-learn, and graphviz to be executed. To install them via conda:
```pyretis
conda install pyqt mdtraj scikit-learn python-graphviz -c conda-forge
```
Note: Since PyRETIS will only work with Python 3.11 or newer, please make sure that you are using an environment with a recent version of Python.
## Optional: Installing from the PyRETIS git repository
Previous versions and the latest (possibly unstable) sources can be obtained using git:
```pyretis
git clone git@gitlab.com:pyretis/pyretis.git
```
or,
```pyretis
git clone https://gitlab.com/pyretis/pyretis.git
```
After cloning the repository, PyRETIS can be installed via pip (after navigating to the main source directory):
```pyretis
pip install .
```
Note: If you want to be able to make modifications locally and to apply them, you can install the package with the -e option. PyRETIS will be executed with the files present in the folder you created.
To be able to run PyVisA’s GUI, you need to install pyqt5, mdtraj, scikit-learn, and graphviz:
```pyretis
python -m pip install pyqt5 mdtraj scikit-learn graphviz
```
## Optional: Installing a development version from the PyRETIS git repository
After cloning the repository as described above, check out the development branch you are interested in, e.g.:
```pyretis
git checkout develop
```
Then install the development requirements (these are defined in the file requirements-dev.txt, note that they include pyqt5, which is not supported in some environments):
```pyretis
pip install -r requirements-dev.txt
```
Finally, install PyRETIS using:
```pyretis
pip install -e .
```
## Testing your installation
After installing from the PyRETIS source, your installation can be tested by running the unit-test suite from the main directory:
```pyretis
python -m pytest test
```
To also run style and lint checks alongside the unit tests, use the provided helper script:
```pyretis
./test-easy.sh
```
To exercise the bundled engine examples (internal, GROMACS, PyVisA, CP2K, OpenMM, LAMMPS) end-to-end, run:
```pyretis
./test-heavy.sh
```
The example suite has a longer runtime than the unit tests because it covers two complementary parts, both chained together by test-heavy.sh:
- Consistency tests (examples/run-tests.sh) – the engine reference-data suites (internal, GROMACS, PyVisA, CP2K, OpenMM and LAMMPS). Each regenerates output and compares it against pre-computed reference data, so the checks are deterministic and re-runnable. The GROMACS sub-suite runs its complete matrix of 14 cases; set PYRETIS_GROMACS_SCOPE=smoke to run the quick 6-case subset.
- Tutorials (examples/run-tutorials.sh) – every documented tutorial, run to confirm that its commands still work. Their length is controlled by PYRETIS_TUTORIAL_TIMEOUT: test-heavy.sh sets it to 2s so each tutorial only has to start cleanly (a short smoke check), whereas running the driver with the variable unset – its default – lets every tutorial run to completion at full length.
---
# Introduction to PyRETIS and rare event methods
PyRETIS is a computational library for performing molecular simulations of rare events with a focus on transition interface sampling (TIS) 1 and replica exchange transition interface sampling (RETIS) 2.
Rare events are called rare because they happen at time or length scales much longer than we are able to simulate through brute-force simulations. Raindrops form every day, but compared to the motion of water molecules, raindrop formation is rare, and simulating it through brute-force simulations would take forever and a day.
Rare event methods aim to sample these rare events inaccessible through brute-force simulation.
## Transition Interface Sampling
To explain the basic concepts, we consider an illustrative example: The transition between two stable states, from the reactant (labeled A) to the product (labeled B) as illustrated in fig_intro_pot.
These two states are defined by a progress coordinate (or order parameter), \lambda, where state A is for \lambda \leq \lambda_\text{A} and state B for \lambda \geq \lambda_\text{B}. The central quantity calculated in TIS/RETIS simulations is the rate constant k_\text{AB} for the transition \text{A} \to \text{B} which can be expressed as:
and we see that it contains two parts:
- The initial flux f_\text{A} which measures how often trajectories start off at the foot of the reaction barrier from the reaction side \lambda_\text{A}. In TIS this is obtained from a molecular dynamics (MD) simulation which in PyRETIS is requested by running a md-flux simulation. Note that this is not needed for the RETIS method as explained.
- The crossing probability \mathcal{P}_{\text{A}} (\lambda_\text{B} \vert \lambda_\text{A}) of reaching \lambda_\text{B} before \lambda_\text{A} given that \lambda_\text{A} has just been crossed. In PyRETIS this can be accomplished by running a tis or retis simulation.
As this is a rare event, the crossing probability is extremely small and nearly impossible to compute in brute-force simulations. Transition interface sampling divides the region between A and B into sub-regions using interfaces, denoted \lambda_i (see fig_intro_tis). The first interface, \lambda_0 is positioned at the interface defining state A (\lambda_A), and the next interface, \lambda_1 is put at \lambda_1 > \lambda_0 such that the probability of reaching \lambda_1 from \lambda_0 is no longer extremely small. We can continue in this fashion and place N more interfaces until we reach \lambda_{N} = \lambda_\text{B}. What we effectively have done with these N+1 interfaces is to split up the computation of the extremely small crossing probability into the computation of many, not-so small, crossing probabilities:
Here, the intermediate crossing probabilities, P_{\text{A}} (\lambda_{i+1}|\lambda_{i}), are formally defined as the probability of a path crossing \lambda_{i+1} given that it originated from \lambda_\text{A}, ended in \lambda_\text{A} or \lambda_\text{B}, and had at least one crossing with \lambda_i in the past.
The interfaces we have placed define the so-called path ensembles. A path ensemble comprises all possible trajectories that start at the foot of reaction barrier from the reactant side (\lambda_\text{A}), end in it or at the product region (\lambda_\text{B}) and having reached a certain threshold value (\lambda_i) between the start point and the final point. This path ensemble is labelled as \left[i^{+} \right]. The probabilities, \mathcal{P}_{\text{A}} (\lambda_{i+1} \vert \lambda_{i}) can then be obtained as the fraction of paths in the \left[i^+\right] ensemble that also cross \lambda_{i+1}.
What is left now, is to have an efficient way of generating trajectories for the various path ensembles. This is in fact done by making use of a selection of Monte Carlo (MC) moves. For TIS 1 we choose between two moves:
- The shooting move which is adapted from the transition path sampling (TPS) shooting algorithm 3 4 to allow variable trajectory length. In this move, we generate a new trajectory from an existing trajectory one by:
- Picking randomly one of the discrete MD steps in the present trajectory.
- Modifying the velocities of this phase point (e.g. by randomly drawing new velocities from a Maxwellian distribution).
- Generating a new trajectory from this new phase point by integrating (i.e. running MD simulations) forward and backward in time until A or B is reached. The new trajectory is then obtained by merging the backward and forward trajectories.
- The new trajectory is accepted as part of [i^+] only if all the following criteria are satisfied:
- A detailed balance condition for the energy and path length. 1
- It starts at \lambda_A
- It has at least one crossing with \lambda_i before ending in A or B.
- The shooting move gives a much higher chance to generate a valid trajectory at each trial compared to simply starting from a random phase point within the reactant well.
- The time reversal move which generates trajectories by simply changing the time direction of a path. 1
These two moves are illustrated in fig_intro_retis_moves. See user-guide-path-move-types for a complete list of all move type codes used in PyRETIS output and input files.
## Replica Exchange Transition Interface Sampling
The RETIS method is similar to TIS and employs both the shooting and time reversal moves. In addition, RETIS makes use of the swapping move and defines a new ensemble [0^-] which consist of trajectories that explore the reactant state (see the illustration in fig_intro_retis).
The swapping move acts between different path simulations. If two simulations generate simultaneously two paths that are valid for each other’s path ensemble, these two paths can be swapped. The swapping moves increase with negligible extra computational cost the number of accepted paths in the ensembles and decrease significantly the correlations between the consecutive paths within the same ensemble. All the moves used for generating trajectories are illustrated in fig_intro_retis_moves.
There is one notable exception where the swapping move is more computationally demanding: the swap between the [0^+] and [0^-] ensembles requires MD simulations. 2
In a TIS simulation, as explained above, we have to perform an extra simulation in order to calculate the initial flux. In RETIS, this initial flux can directly be obtained by:
where \left \langle t_{\rm path}^{[0^+]} \right \rangle is the average path length in the [0^+] ensemble and \left \langle t_{\rm path}^{[0^-]}\right \rangle the average path length in the [0^-] ensemble.
In PyRETIS, RETIS simulations are requested by setting the simulation task to RETIS.
### References
T. S. van Erp, D. Moroni and P. G. Bolhuis, A novel path sampling method for the calculation of rate constants, J. Chem. Phys. 118, 7762-7774 (2003), https://doi.org/10.1063/1.1562614
T. S. van Erp, Reaction rate calculation by parallel path swapping, Phys. Rev. Lett. 98, 268301 (2007), https://doi.org/10.1103/PhysRevLett.98.268301
C. Dellago, P. G. Bolhuis, F. S. Csajka, and D. Chandler, Transition path sampling and the calculation of rate constants, J. Chem. Phys. 108, 1964-1977 (1998), https://doi.org/10.1063/1.475562
P. G. Bolhuis, D. Chandler, C. Dellago and P. L. Geissler, Transition path sampling: Throwing ropes over rough mountain passes, in the dark, Annu. Rev. Phys. Chem. 53, 291-318 (2002), https://doi.org/10.1146/annurev.physchem.53.082301.113146
---
# Introduction to the PyRETIS library
This page gives a compact map of the main classes and functions in the PyRETIS library. For the full method-level description, see the API documentation.
The PyRETIS library is organized into sub-packages that work together when a simulation is prepared, run and analysed. For a RETIS simulation, the main objects appear in this order:
- Define the System under study, including Particles, Box and ForceField objects when needed.
- Define an OrderParameter, which evaluates the progress coordinate from a System object.
- Store trajectories as Path objects and organize them in PathEnsemble objects.
- Run the simulation with SimulationRETIS or another Simulation class, using an EngineBase-like engine to propagate the system.
- Use the setup, initiation, I/O and analysis sub-packages to construct objects from input settings, initialize path ensembles, write output and analyse simulation results.
# Table of Contents
- The core and force field sub-packages
- The System class
- The Particles class
- The Box class
- The ForceField class
- The Path and PathEnsemble classes
- The order parameter sub-package
- The OrderParameter class
- The simulation and engines sub-packages
- The Simulation class
- The EngineBase class
- The setup sub-package
- The initiation sub-package
- The I/O (inout) sub-package
- The analysis sub-package
- The tools sub-package
- Some examples of using the PyRETIS library
- Reversing a trajectory
- Recalculating order parameters
## The core and force field sub-packages
The pyretis.core library defines the central methods and classes used to represent systems, paths and path ensembles. Together with pyretis.forcefield, it provides:
- System class which defines the system we are investigating. This class is actually composed of several other objects, which we will also discuss here:
- Particles: A class which represents the particles.
- Box: A class which represents the simulation box.
- ForceField: A class representing the force field. (Note: This is defined within pyretis.forcefield).
- Path class which defines paths/trajectories.
- PathEnsemble class which defines path ensembles.
Below is an illustration of how some of these classes interact. The classes (shown as boxes in this figure) will be discussed more in the following.
### The System class
The System class defines the system we are investigating. It will typically contain particles, a simulation box and a force field. This class exposes important parts we can interact with, in particular, the particles.
Example of creation:
```python
from pyretis.core import System
new_system = System(temperature=0.8, units='lj')
```
This will create an empty system with a set temperature equal to 0.8 in lj units (lj refers to Lennard-Jones units). It is also possible to specify a box here if needed:
```python
new_system = System(temperature=0.8, units='lj', box=mybox)
```
where mybox can be created as described below.
Particles can be added by first creating an object as described below. A short example:
```python
from pyretis.core import System, Particles
new_system = System(temperature=0.8, units='lj')
new_system.particles = Particles()
new_system.add_particle([0.0, 0.0, 0.0], mass=1.0, name='Ar', ptype=0)
```
Here, we are setting the System.particles attribute and using the System.add_particle() to add a particle with a given position, mass, name and type.
### The Particles class
The Particles class represents a collection of particles and in many ways it can be viewed as a particle list. Internally in PyRETIS, the particle list is one of the most important classes. The positions, velocities and forces are accessed through an instance of this class using the class attributes Particles.pos, Particles.vel and Particles.force. Actually, there is an additional particle class within PyRETIS which is called ParticlesExt. This class is used when PyRETIS is using external engines. It is very similar to the Particles class but it has, in addition, a ParticlesExt.config which can be used to reference files which hold the current configuration of the particles.
Here are some examples of interacting with the Particles class, using Particles.add_particle() to add some particles. Actually, the system will make use of this method if you are calling System.add_particle().
```python
from pyretis.core import Particles
part = Particles(dim=3)
pos = [0.0, 1.0, 0.0]
vel = [0.0, 0.0, 0.0]
force = [0.0, 0.0, 0.0]
part.add_particle(pos, vel, force, mass=1.0, name='Ar', ptype=0)
print(part.pos)
pos = [1.0, 0.0, 0.0]
part.add_particle(pos, vel, force, mass=1.0, name='Ar', ptype=0)
print(part.pos)
```
Here, we can add names to particles using the keyword name and we can also specify a particle type using ptype. The name can be used to identify/tag specific particles and is used for output purposes. Internally, the particle type is more important: The particle type can be used to specify parameters for pair interactions which is computed by the force field.
When we initialize an instance of Particles, we define the dimensionality using the dim keyword parameter.
### The Box class
The Box class defines a simulation box. It is useful in simulations where we wish to have periodic boundaries. Typically, we do not interact much with the box beyond creating it. Boxes are created by passing an (optional) cell argument which is a list of floats of form [lengthx, lengthy, lengthz]. If more than three floats are given, we assume that these represent a flattened version of the box matrix of the form: [xx, yy, zz, xy, xz, yx, yz, xz, zy]. At the same time, periodicity can be specified with the keyword periodic which is a list of boolean values that determine if a dimension is periodic or not. The default is periodic in all directions.
Some examples:
```python
from pyretis.core import create_box
box1 = create_box()
print(box1)
box2 = create_box(cell=[10, 10, 10])
print(box2)
box3 = create_box(low=[0, -10, 10], high=[10, 10, 20],
periodic=[True, True, False])
print(box3)
```
### The ForceField class
The ForceField class is used to define force fields. A force field is just a list of functions (and parameters) which can be used to obtain the force and potential energy. In general, the force field expects that its constituent potential functions support calling three functions, which means that the potential functions must be slightly more complex than just simple functions; they need to be classes which subclass the PotentialFunction class.
If we, for the sake of an example, let an instance of the ForceField class have a constituent potential function named func, then PyRETIS will assume that it can call:
- func.potential(system) to obtain the potential energy.
- func.force(system) to obtain the forces and the virial.
- func.potential_and_force(system) to obtain the potential energy, forces and the virial. Typically, this can be done by just calling func.potential(system) and func.force(system).
Notice that all these functions should only take in a System as the only parameter.
Let’s see an example of how we can set up a potential function (or class) and add it to a force field.
First, define the potential function using:
```python
from matplotlib import pyplot as plt
import numpy as np
from pyretis.core import System, Particles
from pyretis.forcefield import ForceField
from pyretis.forcefield.potential import PotentialFunction
class Harmonic1D(PotentialFunction):
"""A 1D harmonic potential function."""
def __init__(self):
"""Set up the potential."""
super().__init__(dim=1, desc='1D harmonic potential')
self.eq_pos = 0.0 # equilibrium position
self.k_force = 1.0 # force constant
def potential(self, system):
"""Calculate potential energy."""
pos = system.pos # Get positions from the particle list.
vpot = 0.5 * self.k_force * (pos - self.eq_pos)**2
return vpot.sum()
def force(self, system):
"""Calculate the force."""
pos = system.pos # Get positions from the particle list.
virial = None # We are lazy and do not calculate the virial here.
forces = -self.k_force * (pos - self.eq_pos)
return forces, virial
def potential_and_force(self, system):
"""Calculate force and potential."""
pot = self.potential(system)
force, virial = self.force(system)
return pot, force, virial
```
Using what we have already discussed above about the System and Particles we can make a plot of the potential by adding:
```python
if __name__ == '__main__':
# Create empty force field:
forcefield = ForceField(desc='1D Harmonic potential')
pot = Harmonic1D() # create potential
# Add the potential function to the force field:
forcefield.add_potential(pot)
system = System()
system.particles = Particles(dim=1)
system.add_particle(pos=np.zeros(1))
# Do some plotting, first calculate the potential as some locations:
vpotentials = []
positions = np.linspace(-2.5, 2.5, 100)
for xi in positions:
system.pos = xi
vpotentials.append(forcefield.evaluate_potential(system))
# And plot it using matplotlib:
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.plot(positions, vpotentials, lw=3)
plt.show()
```
### The Path and PathEnsemble classes
These two classes are representations of paths and path ensembles. Paths are essentially trajectories and path ensembles are collections of such paths. The PathEnsemble stores some information about all paths in the attribute PathEnsemble.paths, but it will not store the full trajectory (positions, velocities, etc.). It will, however, keep a reference to the last accepted path in PathEnsemble.last_path. This Path object can, for instance, be inspected by using the Path.phasepoints attribute which is a list of system snapshots.
We give here a short example on how you can interact with a path. A complicating factor here is that we need to create a random generator to use with the Path. This is done by using the RandomGenerator class and the reader is referred to the API documentation for information about this.
```python
import numpy as np
from pyretis.core.system import System
from pyretis.core.particles import Particles
from pyretis.core.path import Path
from pyretis.core.random_gen import RandomGenerator
path = Path(rgen=RandomGenerator(seed=0)) # Create empty path.
# Add some phase points to the path:
for i in range(10):
phasepoint = System()
phasepoint.order = [i]
phasepoint.particles = Particles(dim=3)
phasepoint.add_particle(np.zeros(3), vel=np.zeros(3))
phasepoint.vpot = i
phasepoint.ekin = i
path.append(phasepoint)
# Loop over the phase points in the path:
print('Looping forward:')
for i, phasepoint in enumerate(path.phasepoints):
print('Point {}. Order parameter = {}'.format(i, phasepoint.order))
# Loop over the phase points in the path:
print('Looping backward:')
for phasepoint in reversed(path.phasepoints):
print('Order parameter = {}'.format(phasepoint.order))
# Get some randomly chosen shooting points:
print('Generating shooting points:')
for i in range(10):
point, idx = path.get_shooting_point()
print(
'Shooting point {}: index = {}, order = {}'.format(i, idx, point.order)
)
```
## The order parameter sub-package
The pyretis.orderparameter package defines order parameters to use for path sampling simulation. In PyRETIS, such order parameters are assumed to be functions of a System only. Here, we give some examples of how a generic order parameter can be used. For information on how custom order parameters can be created, we refer to the detailed description in the user guide.
### The OrderParameter class
The most important piece of the OrderParameter class is the actual calculation of the order parameter. This should be defined in a method like OrderParameter.calculate(). Here, it is assumed that the order parameter takes in an object like System. Since this is described elsewhere we will here just describe the usage of:
- OrderParameter.calculate() which is used to calculate the order parameters.
- CompositeOrderParameter which can be used to combine several collective variables (e.g. when you are interested in additional order parameters).
First, we create an order parameter. For simplicity we use the pre-defined Position class:
```python
from pyretis.core import System, Particles
from pyretis.orderparameter import Position
position = Position(0, dim='x')
```
Next, we can calculate the order parameter as follows (the system we set up here is just for testing):
```python
# Define a simple system for testing:
system = System()
system.particles = Particles(dim=3)
system.add_particle([1.0, 2.0, 3.0], mass=1.0, name='Ar', ptype=0)
print('Order =', position.calculate(system))
```
Several order parameters can be combined by creating a CompositeOrderParameter. Below is an example of how this can be used:
```python
# Copyright (c) 2026, PyRETIS Development Team.
# Distributed under the LGPLv2.1+ License. See LICENSE for more info.
"""Example of interaction with a Composite order parameter."""
import numpy as np
from pyretis.core import System, Particles
from pyretis.orderparameter import (
OrderParameter,
Position,
CompositeOrderParameter,
)
# Create a new, empty, order parameter:
order_parameter = CompositeOrderParameter()
# Add a position order parameter:
order_parameter.add_orderparameter(Position(0, dim='x'))
# Add a custom order parameter:
position_y = OrderParameter(description='Position along y-axis')
def collective_y(system):
"""Position along y-axis."""
return [system.pos[0][1]]
position_y.calculate = collective_y
order_parameter.add_orderparameter(position_y)
# Add another order parameter:
position_cos = OrderParameter(description='Cosine of z-coordinate')
def collective_cos(system):
"""Additional collective variable."""
return [np.cos(np.pi * system.pos[0][2])]
position_cos.calculate = collective_cos
order_parameter.add_orderparameter(position_cos)
# Create a system for testing the new order parameter:
i_sys = System()
i_sys.particles = Particles(dim=3)
i_sys.add_particle([1.0, 2.0, 3.0], mass=1.0, name='Ar', ptype=0)
print('Order =', order_parameter.calculate(i_sys))
```
The first order parameter returned from calculate() is taken as the progress coordinate used in path sampling simulations.
## The simulation and engines sub-packages
The pyretis.simulation sub-package defines classes which are used to set up and define different types of simulations. Typically, such simulations will need to interact with and change the state of a given System. This interaction is carried out by a particular engine object which behaves like EngineBase from the pyretis.engines sub-package. The interaction between these classes is illustrated in the figure below:
In this section, we will not give a complete example on how to create a new simulation class or a new engine class. We refer the reader to the examples, in particular:
- The section of the user guide which describes how new external engines can be interfaced.
- The particle swarm optimization example in which a custom potential and engine is created. Further, in this example a customized simulation is set up by making use of Simulation.add_task()
- The example showing how C or Fortran can be used to create a custom Velocity Verlet engine.
### The Simulation class
The Simulation class defines a simulation we can run. A simulation will typically act on a System and alter its state. We will here just describe the generic base class Simulation and we refer the reader to the extended pyretis API documentation for information about specific simulation classes, for instance, the SimulationRETIS class. The most commonly used methods from the Simulation are:
- run() which will run a simulation and for each step it will yield a dictionary with results.
- step() which will run a single step from the simulation and return a dictionary with results.
- is_finished() which will return True if the simulation has ended.
- add_task() which can be used to add simulation tasks to a generic simulation.
Example of creation:
```python
from pyretis.simulation import Simulation
new_simulation = Simulation(startcycle=0, endcycle=100)
for step in new_simulation.run():
print(step['cycle']['stepno'])
```
The code block above will create a generic simulation object and run it. This simulation is not doing anything useful, it will only increment the current simulation step number from the given startcycle to the given endcycle. In order to do something more productive, we can attach tasks to the simulation. This can be done as follows:
```python
from pyretis.core import System
from pyretis.simulation import Simulation
def function(sys):
"""Return the set temperature for the system."""
return sys.temperature
simulation = Simulation(settings={}, controls={'steps': 10})
system = System(temperature=300)
my_task = {'func': function,
'args': [system],
'first': True,
'result': 'temperature'}
simulation.add_task(my_task)
for result in simulation.run():
step = result['cycle']['step']
temp = result['temperature']['set']
print(step, temp)
```
As you can see, a new task is added by defining it as a dictionary:
```python
my_task = {'func': function,
'args': [system],
'first': True,
'result': 'temperature'}
```
The following keywords are used:
- func which is a reference to a function to use.
- args which is a list of arguments that should be given to the function.
- first whether the task should be executed on the first simulation step (i.e. step 0).
- result a string which labels the result in the dictionary returned by the methods Simulation.run() or Simulation.step().
Typically, when creating a custom simulation, you will rewrite the methods Simulation.run() and Simulation.step() to fit the custom simulation you are going to perform, rather than adding tasks. However, for interactive work, short examples etc., the Simulation.add_task() can be useful.
### The EngineBase class
The EngineBase is a base class defining a generic engine. In PyRETIS engines are either internal or external. External engines, e.g. ExternalMDEngine, interface with external programs while internal engines, e.g. MDEngine interact directly with a System object. Creating an external engine may be somewhat involved depending on the program you wish to interface. A description on how to create new external engines can be found in the user guide.
## The setup sub-package
The setup library contains methods to construct the main objects of PyRETIS from settings and restart information.
The pyretis.setup sub-package handles the creation of objects from simulation settings. See the pyretis API documentation for more information about this sub-package.
## The initiation sub-package
The pyretis.initiation sub-package contains methods to initialize path ensembles. We refer the reader to the pyretis API documentation for more information about this sub-package.
## The I/O (inout) sub-package
The pyretis.inout sub-package contains methods which PyRETIS uses to read input and present output. This sub-package is relatively large and is made up of the following sub-packages:
- pyretis.inout.analysisio which handles the input and output needed for analysis.
- pyretis.inout.formats for formatting and presenting text-based output.
- pyretis.inout.plotting which handles plotting of figures. It defines simple things like colors etc. for plotting. It also defines functions which can be used for specific plotting by the analysis and report tools.
- pyretis.inout.report which is used to generate reports with results from different simulations.
Again, we refer to the pyretis API documentation for more information about these sub-packages.
## The analysis sub-package
The pyretis.analysis sub-package defines tools which are used in the analysis of simulation output. It defines methods for running predefined analysis tasks, e.g. retis_flux(), but also generic analysis methods, e.g. running_average(). Here, we refer the reader to the pyretis API documentation for more information about these methods.
## The tools sub-package
The tools sub-package can be used to generate initial structures for a simulation. In this sub-package, the function generate_lattice() is defined and it supports the creation of the following lattices where the shorthand keywords (sc, sq etc.) are used to select a specific lattice:
- sc: A simple cubic lattice.
- sq: Square lattice (2D) with one atom in the unit cell.
- sq2: Square lattice with two atoms in the unit cell.
- bcc: Body-centered cubic lattice.
- fcc: Face-centered cubic lattice.
- hcp: Hexagonal close-packed lattice.
- diamond: Diamond structure.
A lattice is generated in the following way:
```python
from pyretis.tools import generate_lattice
xyz, size = generate_lattice('diamond', [1, 1, 1], lcon=3.57)
```
Here, the first parameter selects the lattice type, the second parameter sets the number of repetitions in each direction and the optional parameter lcon is the lattice constant. The returned values are xyz – the coordinates – and size which is the bounding box of the generated structure. This variable can be used to define a simulation box. It is also possible to specify a number density:
```python
from pyretis.tools import generate_lattice
xyz, size = generate_lattice('diamond', [1, 1, 1], density=1)
```
If the density is specified, the lattice constant lcon is deduced:
where n is the number of particles in the unit cell, \rho the specified number density and d the dimensionality. If we wish to save a generated lattice, this can be done as follows
```python
from pyretis.tools import generate_lattice
from pyretis.inout.formats.xyz import write_xyz_file
xyz, size = generate_lattice('diamond', [3, 3, 3], lcon=3.57)
write_xyz_file('diamond.xyz', xyz, names=['C']*len(xyz), header='Diamond')
xyz, size = generate_lattice('hcp', [3, 3, 3], lcon=2.5)
name = ['A', 'B'] * (len(xyz) // 2)
write_xyz_file('hcp.xyz', xyz, names=name, header='HCP test')
xyz, size = generate_lattice('sq2', [3, 3], lcon=1.0)
write_xyz_file('sq2.xyz', xyz, header='sq2 test')
```
## Some examples of using the PyRETIS library
Here, we show some examples of how we can perform some common tasks using the PyRETIS library.
### Reversing a trajectory
PyRETIS will not reverse backward trajectories during a simulation if you are using an external engine. For visualization purposes, it is very helpful to reverse these trajectories before viewing them. This can be accomplished with the PyRETIS library as follows:
- For GROMACS .trr trajectories:
```python
from pyretis.inout.formats.gromacs import reverse_trr
reverse_trr('trajB.trr', 'rev-trajB.trr')
```
- which will read the trajectory trajB.trr and store it as rev-trajB.trr.
- For .xyz trajectories:
```python
from pyretis.inout.formats.xyz import reverse_xyz_file
reverse_xyz_file('trajB.xyz', 'rev-trajB.xyz')
```
- which will read the trajectory trajB.xyz and store it as rev-trajB.xyz.
### Recalculating order parameters
If you have an existing trajectory and wish to calculate order parameters for each step in the trajectory, this can be accomplished by making use of the pyretis.tools.recalculate_order module. For example, if you have a trajectory named traj.trr and an order parameter defined in an input file retis.toml this can be done as follows:
```python
from pyretis.inout.formats.order import OrderFile
from pyretis.inout.settings import parse_settings_file
from pyretis.inout.setup import create_orderparameter
from pyretis.tools.recalculate_order import recalculate_order
settings = parse_settings_file('retis.toml')
order_parameter = create_orderparameter(settings)
options = {'reverse': False, 'maxidx': None, 'minidx': None}
order = recalculate_order(order_parameter, 'traj.trr', options)
with OrderFile('order.txt', 'w') as outfile:
outfile.write('# Order parameters')
for step, data in enumerate(order):
print(step, data)
outfile.output(step, data)
```
This will create a new file order.txt with the re-calculated order parameters. The keyword reverse can be used to tell PyRETIS that you are looking at a time-reversed trajectory. The keyword minidx gives the frame number from which we start calculating (None is equal to reading from the first frame) and the keyword maxidx gives the last frame number we will read (setting it to None will read until the end of the trajectory file).
---
# Running LAMMPS with PyRETIS
This page describes how to use the LAMMPS Molecular Dynamics Simulator with PyRETIS.
## Standard LAMMPS input
The LAMMPS-PyRETIS interface is designed to use LAMMPS’s internal ability to perform on-the-fly calculations. PyRETIS expects a LAMMPS input file to be present in the folder from which it is executed. This input file should contain all the information LAMMPS needs to run a simulation on your system, including but not limited to:
- Initial coordinates and molecular topology, with force field coefficients if needed, in a LAMMPS data file. This file can be read into LAMMPS using the read_data command.
- Information about units, bonded and non-bonded force field coefficients, and long-range electrostatics methods, if applicable.
- A LAMMPS fix that will integrate the motion of the atoms in the system.
The LAMMPS input file should contain everything that is needed to run a normal MD simulation except for the run command.
## PyRETIS requirements for LAMMPS input
In addition to the standard input requirements for a LAMMPS simulation, there are additional requirements for running LAMMPS with PyRETIS.
- Mandatory format for ``class = lammps_steps`` with ``velocity_generation = engine``: LAMMPS draws the shooting velocities with velocity create, at the target temperature of the RETIS simulations, which must be set to a LAMMPS equal-style variable called SET_TEMP. An example is shown below:
```pyretis
variable SET_TEMP equal 300.0
```
- In the above example, the system temperature is set to 300.0 K (assuming real LAMMPS units). class = lammps, and class = lammps_steps with velocity_generation = maxwell, draw the velocities in PyRETIS, at the temperature of the engine section, with the constants of LAMMPS real units, and require units real in lammps.in (see velocity_generation).
- Masses in the data file for ``class = lammps``, and for ``class = lammps_steps`` with ``velocity_generation = maxwell``: PyRETIS takes the masses of the atoms from the Masses section of the LAMMPS data file, so lammps.in has no mass command, which sets masses in LAMMPS in place of those of the data file. A mass command in lammps.in stops the run with an error: for class = lammps when the engine is set up, and for class = lammps_steps at the draw of a configuration held in a data file; its draw of a frame of a dump reads the data file that LAMMPS writes for the frame, with the masses of the mass commands (see atom_style).
- Mandatory format with ``order_mode = lammps``, the default of ``class = lammps_steps``: The order parameter(s) that PyRETIS tracks should be declared as LAMMPS equal-style variable(s). Multiple order parameters can be defined, but the first order parameter, op_1, will be used by PyRETIS to determine interface crossings.
```pyretis
variable op_1 equal c_1
variable op_2 equal c_2
```
- In the above example, two order parameters will be tracked and output by LAMMPS to a file called order.txt. The first column of order.txt is the LAMMPS step, the second column is op_1, and the third column is op_2. Only op_1 is used by LAMMPS to determine crossing of the first or last interface and stop the simulation.
- The above example refers to LAMMPS compute commands, referenced by c_1 and c_2, that return scalar values. By combining LAMMPS compute and LAMMPS variable commands, many calculations can be performed internally by LAMMPS. LAMMPS can also call Python scripts for more calculations. For clearer input files, order parameter calculations can be scripted in a separate file, order.in, and included in the main LAMMPS input script as follows:
```pyretis
include order.in
```
- This mode corresponds to order_mode = lammps in the PyRETIS engine section, the default of class = lammps_steps.
## Order parameters evaluated by PyRETIS
The LAMMPS engine can also evaluate order parameters and collective variables with the normal PyRETIS orderparameter and collective-variable sections. Select this behavior with:
```pyretis
Engine
------
class = lammps
lmp = lmp
input_path = lammps_input
subcycles = 1
temperature = 300.0
order_mode = pyretis
```
In this mode, LAMMPS is used to propagate the dynamics and write trajectory frames. PyRETIS then reads those frames, reconstructs positions and velocities, and calls the configured PyRETIS order-parameter object. The first returned value is still the main order parameter used for interface decisions; any additional returned values or configured collective variables are stored as extra order/CV columns.
For example:
```pyretis
Orderparameter
--------------
class = pyretis_distance
index = [0, 1]
periodic = True
Collective-variable
-------------------
class = pyretis_position
index = 0
dim = x
```
When order_mode = pyretis is used, the LAMMPS input no longer needs to define op_1 in order.in. If your lammps.in still contains include order.in, the file must still exist or be listed as an extra_files dependency, because LAMMPS itself will try to read it.
The tradeoff is that LAMMPS cannot stop internally on v_op_1 in this mode. PyRETIS therefore runs LAMMPS up to the requested path length and truncates the resulting path after evaluating the Python order parameter. The traditional order_mode = lammps mode can stop inside LAMMPS as soon as v_op_1 crosses an interface.
## Restrictions
There are some restrictions for the LAMMPS-PyRETIS interface.
- Motion should be propagated using NVE dynamics, as NVT/NPT dynamics have not been shown to be accurately time-reversible in practice.
- We recommend that the SHAKE algorithm not be used, because in practice it produces non-time reversible results due to the iterative nature of the algorithm.
- Order parameters must not depend on the direction of velocity components.
## Notes
The subcycle length, defined in the PyRETIS input file, refers to how often LAMMPS will print to the log file and the order.txt file.
---
# Monitoring a running simulation
A path-sampling run is long, and the useful question during it is not “what is the rate” – that is the last thing to converge – but “is this run doing anything sensible?” A run that is sampling badly looks wrong within a few hundred cycles, in ways that are visible long before any rate is trustworthy.
Everything on this page can be done while the simulation is still running. PyRETIS writes its per-ensemble output incrementally and appends to it, so the output on disk is always a complete record of the cycles finished so far. Nothing here modifies the run, and nothing here needs the run to stop.
user-guide-output describes the files these commands read; user-guide-rate explains the rate itself, and user-guide-clean how to reset a directory afterwards.
# Contents
- pyretis status: what each ensemble sampled
- Reading the table
- What to look for
- Options
- Analysing a run that has not finished
- Visualising a run that has not finished
- Resetting a directory afterwards
## pyretis status: what each ensemble sampled
Run it in the directory the simulation is running in:
```bash
pyretis status
```
It reads the per-ensemble output in place, computes nothing expensive – no crossing probabilities, no matching, no rate – and prints what the sampler has actually been doing:
```none
Per-ensemble status
Ens Cycles Trials Acc Rej Acc% Shoot Swap HA eff
----- ------------- -------- -------- -------- ------ ------ ------ -------
000 1-25 8 7 1 87.5 0.750 1.000 n/a
001 1-25 5 5 0 100.0 1.000 1.000 n/a
002 1-25 11 7 4 63.6 0.636 n/a n/a
003 1-25 5 5 0 100.0 1.000 n/a n/a
Moves attempted, per ensemble
000: s+: 4/4 (1.000) sh: 3/4 (0.750)
001: s-: 4/4 (1.000) sh: 1/1 (1.000)
002: sh: 7/11 (0.636)
003: sh: 5/5 (1.000)
* a high-acceptance move: read its HA eff., not its ratio.
Why trials were rejected
000: BTL: 1
002: NCR: 4
The codes are explained in the documentation: path types and
rejections.
```
Because it is cheap and read-only, it is safe to run repeatedly against a live run – for instance under watch:
```bash
watch -n 60 pyretis status --no-moves
```
### Reading the table
Column
Meaning
Ens
The ensemble directory (000, 001, …).
Cycles
The first and last cycle this ensemble has recorded.
Trials
Monte Carlo moves attempted in this ensemble.
Acc / Rej
How many of those were accepted, and how many rejected.
Acc%
The overall acceptance percentage over all move types.
Shoot
Acceptance ratio of the shooting-like moves alone, excluding the high-acceptance moves (which would drag it towards 1).
Swap
Acceptance ratio of the swapping moves. n/a where no swap was ever proposed – a per-interface TIS run, for instance.
HA eff
For wire fencing and stone skipping: the weight collected per attempt. Those moves accept almost every trial, so their acceptance ratio is near 1 whatever the sampling is doing; the weight is what varies and what matters.
A ratio is n/a when the move was never attempted in that ensemble. A move that was not attempted has no acceptance ratio – it is not a ratio of one.
### What to look for
- A very low shooting ratio in one ensemble usually means its interfaces are too far apart, or the shooting move is too aggressive for the system.
- A dominant rejection code is the most useful single diagnostic on the page. Many NCR (no crossing) means trials are not reaching the interface; the codes are listed in user-guide-path-types-rejections.
- ``Swap`` at ``n/a`` in a RETIS run means the ensembles are not exchanging, which defeats the point of RETIS.
- Ensembles with very different ``Cycles`` in an infinite-swapping run are expected – occupancy is spread over ensembles – but a stalled ensemble is not.
### Options
Argument
Description
-d DIR, --directory DIR
The directory the simulation ran in (default: the current one).
-skipb N, --skip-begin N
Discard the first N cycles, as for pyretis analyse.
-skipe N, --skip-end N
Discard the last N cycles.
--no-moves
Leave out the per-move breakdown.
--no-rejections
Leave out the rejection breakdown.
--json
Write the status as JSON, for further processing.
The --json form is the one to build on if you are watching many runs at once:
```bash
pyretis status --json | python -c \
"import json,sys; [print(e['ensemble'], e['rejections']) \
for e in json.load(sys.stdin)]"
```
Old runs are handled too. pyretis status reads the trial records through the same reader the analysis uses, which knows every layout PyRETIS has written – moves.txt, attempts recorded inside pathensemble.txt, or a separate trials.txt.
## Analysing a run that has not finished
pyretis analyse can be run against a live simulation:
```bash
pyretis analyse -i retis.toml
```
The analysis reads the output files as they stand and reports on the cycles completed so far. It does not touch the running simulation, and the simulation does not wait for it.
Two things are worth knowing:
- The report is stamped with the cycle count (_report_cycles-NNNNNNNNN.rst), so analysing repeatedly during a run leaves a series of reports rather than overwriting one. That series is itself informative: the rate should stop moving.
- A rate from an unconverged run is not wrong so much as premature. Use -skipb to drop the equilibration, and read user-guide-output-converged before quoting anything.
The cheapest useful check during a run is whether the rate is still drifting. Analyse twice, a few hundred cycles apart, and compare – if the second is outside the first’s error bar, the run is not converged.
## Visualising a run that has not finished
PyVisA can likewise be pointed at a running simulation:
```bash
pyvisa -i retis.toml
```
It reads the same output files and plots what is there. Re-loading picks up the cycles finished since. This is the fastest way to see the shape of a problem – a crossing-probability curve with a kink at one interface, or a path-length distribution with a tail that should not be there.
## Resetting a directory afterwards
When a run is finished with, pyretis tools clean removes the generated files and leaves the committed inputs:
```bash
pyretis tools clean --dry-run # list what would be removed
pyretis tools clean # remove it
```
Always use --dry-run first in a directory that holds anything you staged by hand. The full description, including what counts as an artifact and what is protected, is in user-guide-clean.
---
# Monte Carlo moves
In TIS and RETIS simulations, every Monte Carlo cycle proposes a new trajectory (a path) from the current one using a move, and then accepts or rejects it. A move only changes how a trial path is generated – it never changes which path ensemble is being sampled – so every move listed here targets the exact same distribution and may be combined freely without biasing the result.
This page explains what each move does. The compact two-letter code that PyRETIS writes for every cycle (sh, tr, s+ …) is collected, together with the rejection codes, in user-guide-path-types-rejections. Which move is used in each ensemble is controlled by the shooting_moves keyword of the [TIS] section.
Added a new move, or changed an existing one? A correct move must reproduce the same crossing probability and rate constant as the established ones, within statistical error. Validate it with the reproducibility suite described in Extending and validating PyRETIS.
## Basic moves
These are the standard TIS/RETIS moves and are sufficient for most simulations.
### Shooting (sh)
The workhorse of path sampling. A shooting point is picked at random among the phase-space points of the current accepted path, its momenta are modified, and the equations of motion are integrated forward and backward from that point to assemble a new trial path. The trial is accepted only if it still satisfies the ensemble’s start/end conditions (i.e. it crosses the relevant interface).
How the momenta are modified is set by the sigma_v keyword:
- Aimless shooting (sigma_v negative, the default): the velocities at the shooting point are redrawn from the Maxwell–Boltzmann distribution at the system temperature.
- Non-aimless shooting (sigma_v zero or positive): a Gaussian perturbation of width sigma_v (scaled per particle) is added to the current velocities and the momentum change is accepted or rejected with the Metropolis criterion.
### Biased shooting (bias)
Standard shooting picks the shooting point uniformly. The biased move picks it non-uniformly – biased toward “interesting” frames such as the barrier top – to converge a rate in fewer MD steps, without changing the sampled path ensemble: the crossing probability and the rate are statistically identical. A per-frame weight w(x) (set by a [shooting-selector]) gives the selection probability w_i/W, and a valid trial path is accepted with the exact correction \min(1, W_\mathrm{old}/W_\mathrm{new}) (aimless only). It runs on both the in-process loop and the infinite-swapping scheduler. See Biased shooting-point selection for the theory, the selectors, and a worked TOML example; contrast with the non-rigorous legacy exp heuristic.
### Time-reversal (tr)
A cheap move that needs no new integration: the current accepted path is reversed in time – the order of the phase points is inverted and the momenta are flipped – to produce the trial path. For deterministic dynamics the reversed trajectory is itself a valid trajectory, so time-reversal is typically interleaved with shooting to help decorrelate the sampled paths.
### Replica swapping (s+ / s-)
The defining move of RETIS. Two neighbouring path ensembles attempt to exchange their accepted paths: a path from [i^+] is offered to [(i+1)^+] and vice versa. Because a path accepted in the higher-index ensemble already crosses the lower interface, one direction is always acceptable by construction; the other is accepted when the incoming path crosses the neighbour’s interface. No integration is needed except for the [0^-]/[0^+] pair, where the equations of motion are integrated. The codes s+ and s- record whether the incoming path came from the higher- or lower-index ensemble. The probability of a swap is set by the swapfreq keyword of the [RETIS] section.
## Advanced moves: sub-trajectory moves
Stone Skipping (ss), Web Throwing (wt) 1 and Wire Fencing (wf) 2 build a new path from several short sub-trajectories (“jumps”) instead of a single long shot. They are designed to improve sampling efficiency and path decorrelation for stiff or diffusive barriers where ordinary shooting struggles. The number of sub-trajectories is set by n_jumps, and high_accept selects the high-acceptance variant of each move. Multiresolution Wire Fencing (mwf) runs the Wire Fencing scheme at two engine resolutions, and is described below.
### Stone Skipping (ss)
Generates the new path through a sequence of short shooting steps that “skip” off the interface, reusing the accepted sub-paths. See 1 for the algorithm and acceptance rule.
### Web Throwing (wt)
A sub-trajectory move closely related to Stone Skipping, also introduced in 1.
### Wire Fencing (wf)
A sub-trajectory move 2 that bounds its sub-paths with an additional cap interface, placed above the ensemble interface and set by the interface_cap keyword. The cap has to lie beyond the interface defining the ensemble; choosing it poorly can hurt sampling efficiency.
### Multiresolution Wire Fencing (mwf)
Wire Fencing run at two engine resolutions. The intermediate sub-paths of each set are shot with the reduced subcycle count set by mwf_subcycle_small, and the final sub-path of the set is shot with the engine’s own subcycles. The surviving sub-path is extended to a full path and accepted with the same sub-trajectory acceptance rule as the moves above. mwf_nsubpath sets how many sub-paths a set holds, and n_jumps how many sets are generated.
A hands-on walkthrough of configuring these three moves is given in the sub-trajectory moves tutorial.
## Infinite swapping and parallel workers
For large, parallel simulations PyRETIS offers an infinite-swapping sampler 3 – an asynchronous, highly parallel reformulation of the replica-exchange move. It removes the main scaling bottleneck of classical RETIS, where a replica swap is a synchronisation point: every worker must finish its current trajectory before the next round of swaps can be evaluated.
Instead of attempting swaps between fixed neighbouring pairs, the infinite-swapping scheme keeps a central coordinator that records swap acceptance statistics over the running stream of completed trajectories. Each worker then draws its next task from the resulting distribution and is handed new work as soon as it frees up. As a result the wall-clock time keeps scaling close to linearly with the number of workers over a far wider range than synchronous replica exchange allows. The Wire Fencing move described above pairs naturally with this sampler, since it keeps acceptance high on the long paths that aggressive parallel sampling tends to produce.
This path is selected entirely from the input TOML – a [runner] section, or task = "infinite_swapping" under [simulation] – and runs through the same pyretis run command as every other simulation; the number of parallel workers is a setting in that file. The runner section reference documents the [runner] keywords, the two execution routes and the environment overrides, and the infinite-swapping tutorial walks through a runnable example.
Because the trajectories are no longer collected from fixed, independent ensembles, the per-interface crossing probabilities and the final rate constant are combined with the Weighted Histogram Analysis Method (WHAM) 4 rather than by simple per-ensemble averaging. See the analysis guide for how to run the WHAM crossing-probability analysis.
## Other moves
These appear mainly in specific algorithms or during path initiation; all their codes are listed in user-guide-path-move-types.
- Target swap (ts) – builds a new path by swapping toward an allowed target index, looping forward and backward along the old trajectory (integrating where required).
- Mirror (mr) – mirrors the order parameter in place and recomputes it along the path; always accepted.
- Null move (00) – keeps the current path unchanged; used internally.
- Initiation moves – how the very first path of a run is produced: kick initiation (ki), initial shoot (is), and loading an existing path from external files, either unformatted/binary (ld) or formatted/text (re).
## Configuring moves
The moves are selected and tuned through the [TIS] and [RETIS] input sections. The most relevant keywords are:
Keyword
Controls
shooting_moves
The move used in each ensemble ('sh', 'ss', 'wt', 'wf').
sigma_v
Aimless vs. non-aimless shooting and the velocity-perturbation width.
n_jumps
Number of sub-trajectories for Stone Skipping, Web Throwing and Wire Fencing.
high_accept
Selects the high-acceptance variant of the sub-trajectory moves.
interface_cap
The cap interface used by Wire Fencing.
swapfreq
The probability of a swap move.
## References
E. Riccardi, O. Dahlen and T. S. van Erp, Fast decorrelating Monte Carlo moves for efficient path sampling, J. Phys. Chem. Lett. 8, 4456-4460 (2017), https://doi.org/10.1021/acs.jpclett.7b01617
D. T. Zhang, E. Riccardi and T. S. van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023), https://doi.org/10.1063/5.0127249
D. T. Zhang, L. Baldauf, S. Roet, A. Lervik and T. S. van Erp, Highly parallelizable path sampling with minimal rejections using asynchronous replica exchange and infinite swaps, Proc. Natl. Acad. Sci. U.S.A. 121, e2318731121 (2024), https://doi.org/10.1073/pnas.2318731121
S. Kumar, J. M. Rosenberg, D. Bouzida, R. H. Swendsen and P. A. Kollman, The weighted histogram analysis method for free-energy calculations on biomolecules. I. The method, J. Comput. Chem. 13, 1011-1021 (1992), https://doi.org/10.1002/jcc.540130812
---
# Creating custom order parameters
Often, you will find the need to create custom order parameters for your path sampling simulation(s). In PyRETIS, you can create new order parameters by making use of the generic OrderParameter class defined in the library. Technically speaking, you will have to sub-class OrderParameter and implement a method (like OrderParameter.calculate()) which actually calculates the order parameter. In addition, you can define additional collective variables to be calculated.
The short version of what we need to do in order to create a new order parameter is:
- Create a new Python class which is a sub-class of OrderParameter
- Write a method to initialise this class, that is the __init__ method. This method will be called when PyRETIS is setting up a new simulation and it will be fed variables from the PyRETIS input file.
- Write a method to calculate the order parameter. This should take in a System object as its argument.
- Setting up the input file to use the new order parameter.
## Example: Creating a new order parameter
Let us see how this can be done in practice. To be concrete, we will consider an order parameter defined as follows: The distance between a particular particle and a plane positioned somewhere along the x-axis.
### Step 1 and 2: Sub-classing OrderParameter
In order to define a new class to use with PyRETIS, we import numpy and the OrderParameter class:
```python
import numpy as np
from pyretis.orderparameter import OrderParameter
```
And we define a class, representing our new order parameter:
```python
class PlaneDistanceX(OrderParameter):
"""A positional order parameter.
This class defines a very simple order parameter which is
the distance from a plane for a given particle.
"""
def __init__(self, index, plane_position):
"""Initialise the order parameter.
Parameters
----------
index : integer
Selects the particle to use.
plane_position : float
The location of the plane, along the x-axis.
"""
txt = 'Distance from particle {} to the plane at {}'.format(
index,
plane_position)
super().__init__(description=txt)
self.index = index
self.plane_position = plane_position
```
Here, we are initialising the class by storing two variables index and plane_position which identifies the particle we will consider and the location of the plane. In addition, we add some more information when we initialise the parent class (the line calling super().__init__). This is simply following the convention defined by OrderParameter.
If the order parameter you are defining depends explicitly on the velocity (i.e. the direction so that it is not symmetric under time reversal) you should mark the order parameter as being velocity dependent. This can be done by setting the velocity argument to True while initiating the order parameter:
```python
super().__init__(description='Description', velocity=True)
```
### Step 3: Creating a method for calculating the order parameter
Next, we will write a method for actually calculating the order parameter. In order to do this, we need to know how we can interact with the System object, and this might be a good point to read the introduction to the API. In particular, we will mostly be interacting with the Particles class which we can access using System.particles.
Typically, our calculation of the order parameter will correspond to one of the two following cases:
- We want to access the positions of the particles directly and calculate the order parameter. Here, just a new method to the PlaneDistanceX class we are creating as follows:
```python
def calculate(self, system):
"""Calculate the order parameter."""
pos = system.pos[self.index]
dist = -np.abs(pos[0] - self.plane_position)
return [dist]
```
- Here, we are making use of the index and position of the plane which we stored previously. Further, we are using the System.particles object in order to access the positions. Finally, we are returning the order parameter. In this particular case, we return the negative of the distance, so that the order parameter will increase when we are approaching the place.
- We are using an external engine (e.g. GROMACS) and we just want to reference the file containing the current configuration, and pass this file reference to another external library or program.
- Here, we will make use of some external tools to obtain the order parameter, for simplicity, let us here make use of the mdtraj library. First, at the top of your file, add the import of this library:
```python
import mdtraj
```
- In order to access the file containing the configuration, we make use of the ParticlesExt.config attribute. Add the following method to the PlaneDistanceX class:
```python
def calculate(self, system):
"""Calculate the order parameter."""
filename = system.particles.config[0]
traj = mdtraj.load(filename)
return some_function_to_obtain_order(traj)
```
- Note that the method some_function_to_obtain_order has not been defined here and you will have to define his method yourself in order to complete the example.
### Step 4: Making use of the new order parameter
We make use of the new order parameter by adding the Orderparameter section to the input file:
```toml
[orderparameter]
class = "PlaneDistanceX"
module = "orderparameter1.py"
index = 0
plane_position = 1.0
```
As you can see, we specify the following:
- The name of the order parameter class.
- The name of the file where we have stored the new class.
- The value for the index parameter of the order parameter.
- The value for the plane_position parameter of the order parameter.
---
# Published applications of PyRETIS
This collection highlights peer-reviewed applications of PyRETIS across molecular kinetics, condensed-phase chemistry, biomolecular transitions, and method development. Each study couples a physical system with a path-sampling strategy and an associated publication.
Where available, Source files lead to runnable inputs: a retis.rst for PyRETIS 3, the corresponding retis.toml for PyRETIS 4, and a README describing requirements and execution. Large solvated and ab initio systems are intended for HPC environments and require system-specific convergence studies.
Biomolecular kinetics
ABL-imatinib kinetics with RETIS and REPPTIS.
RETIS and REPPTIS were used to characterise the ABL-imatinib transition at atomistic resolution, assessing path-sampling performance and rare-event kinetic estimates in a large biomolecular system.
Source files Package to be recovered and mirrored.
Publication W. Vervust, D. T. Zhang, E. Riccardi, T. S. van Erp and A. Ghysels, Path sampling challenges in large biomolecular systems: RETIS and REPPTIS for ABL-imatinib kinetics, Biophys. J. 124, 3932-3947 (2025).
Method development
Memory reduction using partial path ensembles.
Partial path ensembles and replica exchange were examined as a route to reduce path-storage requirements when long-lived metastable states govern the reaction dynamics.
Source files Repository
Publication W. Vervust, D. T. Zhang, T. S. van Erp and A. Ghysels, Path sampling with memory reduction and replica exchange to reach long permeation timescales, Biophys. J. 122, 2960-2972 (2023).
Condensed-phase dynamics
Rupture of a thin liquid film.
Wire Fencing and Stone Skipping subtrajectory Monte Carlo moves were applied to dodecane-film rupture in water with GROMACS, providing a condensed-phase benchmark for enhanced path sampling.
Source files Download inputs
Publication D. T. Zhang, E. Riccardi and T. S. van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023).
Ab initio kinetics
Redox reaction of a diruthenium complex.
The subtrajectory moves were evaluated for an ab initio CP2K model of a diruthenium aqua complex, where the high cost of each force evaluation makes efficient path generation essential.
Source files Download inputs
Publication D. T. Zhang, E. Riccardi and T. S. van Erp, Enhanced path sampling using subtrajectory Monte Carlo moves, J. Chem. Phys. 158, 024113 (2023).
Data-driven analysis
ChemiTrees for reaction-path interpretation.
Decision-tree analysis was used as a post-processing workflow for path-sampling trajectories of proton exchange in formic acid-water clusters, identifying geometric descriptors that distinguish reactive and unreactive paths.
Source files Package to be recovered and mirrored.
Publication S. Roet, C. D. Daub and E. Riccardi, Chemistrees: data driven identification of reaction pathways via machine learning, J. Chem. Theory Comput. 17, 6193-6202 (2021).
Proton-transfer dynamics
Proton transfer in a water trimer.
Proton transfer in a polarizable water trimer provides a compact reference system for examining path-sampling trajectories and the analysis and visualisation workflows in PyVisA.
Source files Download inputs
Publication O. Aarøen, H. Kiær and E. Riccardi, PyVisA: Visualization and Analysis of path sampling trajectories, J. Comput. Chem. 42, 435-446 (2021).
Atmospheric chemistry
Formic acid catalysed formation of sulfuric acid.
The formic-acid-catalysed conversion of sulfur trioxide and water to sulfuric acid was investigated to resolve competing mechanisms and determine their temperature-dependent rate constants.
Source files Download inputs
Publication C. D. Daub, E. Riccardi, V. Hänninen and L. Halonen, Path sampling for atmospheric reactions: formic acid catalysed conversion of SO3 + H2O to H2SO4, PeerJ Phys. Chem. 2, e7 (2020).
Biomolecular recognition
Predicting the mechanism and rate of H-NS binding to AT-rich DNA.
H-NS adsorption to AT-rich DNA was studied at atomistic resolution with GROMACS. Metadynamics identified metastable basins, and RETIS was used to calculate transition pathways and kinetic rates.
Source files Download inputs
Publication E. Riccardi, E. C. van Mastbergen, W. W. Navarre and J. Vreede, Predicting the mechanism and rate of H-NS binding to AT-rich DNA, PLoS Comput. Biol. 15, e1006845 (2019).
Aqueous reaction dynamics
Autoionization of water.
Born–Oppenheimer DFT simulations combined with RETIS were used to resolve water autoionization pathways and quantify their kinetics. Machine-learning analysis assessed the discriminatory power of the candidate order parameters.
Source files Download inputs
Publication M. Moqadam, A. Lervik, E. Riccardi, V. Venkatraman, B. K. Alsberg and T. S. van Erp, Local initiation conditions for water autoionization, Proc. Natl. Acad. Sci. U.S.A. 115, E4569-E4576 (2018).
Protein conformational dynamics
Conformational study of Cyclophilin - A.
Atomistic GROMACS simulations were used to sample structural rearrangements of cyclophilin A and estimate their transition rates.
Source files Download inputs
Publication Manuscript in preparation.
---
# Simulation outputs
A PyRETIS run leaves behind a directory of plain-text files, and pyretis analyse turns those into a report of tables and figures. This page explains what each file contains and how to read the report – in particular, how to tell a converged simulation from one that has not sampled enough.
# Contents
- What a run writes
- pathensemble.txt – what each ensemble held
- Coming from PyRETIS 3
- moves.txt – what the sampler attempted
- Navigating it
- order.txt and energy.txt – along the path
- cross.txt – crossings in a flux run
- The trajectory stores
- Reading the report
- The figures
- Is it converged?
- A worked example: the same simulation, short and long
- Too short: 200 cycles
- Long enough: 20000 cycles
- A second system: water-dimer dissociation with GROMACS
- The health check
## What a run writes
A path-sampling run produces one numbered directory per path ensemble, plus a few files at the top level:
```none
retis.toml the input you supplied
output.toml the input (defaults resolved) + the restart state
pyretis.log the run log (see the [output] log keywords)
000/ the [0^-] ensemble
001/ the [0^+] ensemble
002/ 003/ ... the [1^+], [2^+], ... ensembles
report/ written by "pyretis analyse", not by the run
```
and inside each ensemble directory:
```none
pathensemble.txt which paths this ensemble HELD, and with what weight
moves.txt every move this ensemble ATTEMPTED, and how it ended
order.txt the order parameter along each held path
energy.txt the energies along each held path
accepted// the trajectory files of the live paths
archive// trajectories kept for the long term
```
Those first two files answer two different questions, and it is worth being clear about which is which from the start:
File
Answers
pathensemble.txt
What was this ensemble holding? One line per path per cycle, with the share of the ensemble that path carried. The crossing probability – and so the rate – is computed from this.
moves.txt
What did the sampler try, and did it work? One line per attempted move, accepted or rejected. The acceptance ratios are counted from this, and it is the only record of a rejection.
Neither can be derived from the other. A rejected move never becomes a held path, so it appears only in moves.txt; and under infinite swapping a path can arrive in an ensemble with no move attempted there at all, so it appears only in pathensemble.txt.
The numbering is by slot: 000 is the minus ensemble [0^-], 001 is [0^+], and so on outwards. A run without a minus ensemble (single-ensemble TIS) names its one directory after its interface instead. A pptis run without zero_left or flux numbers only the windows it builds: 000 is [0^+] with zero_ensemble = true and [1^+] without it, and pyretis analyse labels each directory by that role.
order.txt and energy.txt are written only when the corresponding [output] keyword asks for them. Everything else in the list is always produced.
## pathensemble.txt – what each ensemble held
This is the central file: it records the path the ensemble was holding, cycle by cycle, and it is what the crossing-probability analysis reads.
Every line is an occupancy record – which path the ensemble held this cycle, and what share of the ensemble it carried. One path produces one line per cycle it survives, so there are always at least as many lines as cycles. Under infinite swapping there are usually many more: a cycle can spread an ensemble over several live paths at once, each holding a fraction (26839 lines after 20000 cycles in the worked example below).
When a cycle appears more than once, the ensemble was holding more than one path that cycle, and their shares – the FracInc column – sum to one:
```none
# Step ... Weight PathNumber HA-weight FracInc
11 ... 2.000000000e+00 14 1.0 0.5
11 ... 2.000000000e+00 8 1.0 0.5
```
Attempted moves are not in this file. They are in moves.txt beside it.
```none
# Step No.-acc No.-shoot l m r Length Acc Mc Min-O ...
1 1 0 L M L 68 ACC ld -9.001880000e-01 ...
3 3 1 L M L 252 ACC sh -9.003640815e-01 ...
```
The columns are:
Column
Meaning
Step
The simulation cycle this row describes.
No.-acc
Running count of accepted paths in this ensemble: the initial path, then every accepted trial.
No.-shoot
Running count of accepted shooting trials in this ensemble. Both counters advance per attempted move, never per row, so at cycle N No.-shoot is at most N and No.-acc at most N + 1.
l m r
Which interfaces the path touches: whether it starts (l), crosses the middle (m) and ends (r) on the left (L) or right (R) side. L M R is a path that crossed and went on to the next state; L M L came back.
Length
Number of phase points in the path.
Acc
The status: ACC, or a rejection code. See user-guide-path-types-rejections.
Mc
The move that generated this path – sh shooting, tr time reversal, ld loaded, ki kicked, and so on.
The two swaps are different moves and are labelled differently: s+/s- is the zero swap between [0^-] and [0^+] (the RETIS one, or the QuanTIS one when quantis = true), and p+/p- is the PPTIS/REPPTIS swap between adjacent ensembles. A RETIS swap of whole paths costs no MD; the zero swap and the PPTIS swap both integrate new trajectory, and the PPTIS one integrates a different amount each cycle. What each attempt integrated is the MD-steps column of moves.txt.
A move that ran no dynamics at all cannot turn an initialisation path into a sampled one, so its product keeps the source path’s code (ld, ki or is) instead of its own: a time reversal of a loaded path is that same path read backwards, a mirror is it seen from the other side, and a swap or target swap that only copied frames is it rearranged. A restart that changes [tis] high_accept stamps hc on each path an ensemble holds instead: it was accepted under the acceptance rule the continuation no longer runs. Those four codes are what holds a path out of the production statistics. The test for the first three is whether the move integrated any frame for the product; hc clears once the ensemble accepts a path under the new rule.
Min-O, Max-O
The smallest and largest order-parameter value along the path. Max-O is the one that decides whether the path reached the next interface, so it is the number the crossing probability is built from.
Idx-Min, Idx-Max
The indices of the frames where those extrema occur.
O-shoot
The order parameter at the shooting point. Wire fencing shoots each of its jumps from its own point and records the last one. Multiple wire fencing (mwf) shoots from many points and writes 9.0e+03 here, which the analysis does not read as a shooting point.
Idx-sh, Idx-shN
For sh, the index of the shooting point in the old and in the new path. The other moves use them for their own counts: stone skipping (ss) its one-step crossing tries, web throwing (wt) its accepted segments, wire fencing (wf) its usable segments and multiple wire fencing (mwf) its successful sets, each followed by the path length.
Weight
The statistical weight of the row: HA-weight divided by FracInc.
PathNumber
The run-wide identifier of the path, so a path can be followed across cycles and ensembles.
HA-weight
The high-acceptance weight, for the moves that use one.
FracInc
The share of the ensemble this row’s path carried this cycle. The shares of the rows of one cycle sum to one, and a row whose HA-weight is zero carries its share here.
The last three columns are written by the scheduler and are absent from files produced by much older versions; everything before them is the long-standing format.
Every line in this file is an accepted path, because the file says what the ensemble held and a rejected move never becomes a held path. Counting these lines therefore tells you nothing about the acceptance rate – it would report every move as accepted. The rejections are in moves.txt.
This also explains a path that repeats over several cycles: when a move is rejected the ensemble keeps what it had, and that path is recorded again next cycle. Those repeats are not noise – they carry real statistical weight, since a path held for five cycles counts five times. The rejection that caused them is recorded once, in moves.txt.
Two lines sharing a cycle are not a repeat: that is one ensemble holding two paths at once, each with its share.
Mc here is how the held path was originally generated, which for a path that arrived by a swap is the move that created it in its previous ensemble. For what was attempted here, read moves.txt.
### Coming from PyRETIS 3
The first 17 columns are the PyRETIS 3 columns, in the same byte layout, and PathNumber, HA-weight and FracInc follow them. A reader that stops after Weight parses the file. What a row means is different, and that decides what can be computed from it:
PyRETIS 3
PyRETIS 4
a row is
the move attempted in that cycle, rejections included
a path the ensemble held in that cycle; always ACC
rows per cycle
one
one per path held, each with its share in FracInc
Weight
the high-acceptance weight
HA-weight / FracInc
Mc
the move attempted in this ensemble
the move that generated the held path, possibly in another ensemble and in an earlier cycle
Infinite swapping lets an ensemble hold several paths at once, which one row per attempted move cannot express. Weight is defined so that the PyRETIS 3 estimator – each row contributes 1 / Weight – builds the occupancy-weighted histogram: PyRETIS 3’s crossing-probability analysis run on these files gives PyRETIS 4’s crossing probabilities once the rows of initialisation paths (ld, ki, is, hc) are left out, as PyRETIS 4 leaves them out. PyRETIS 3 starts an ensemble’s statistics at its first accepted shooting move, and so reports nothing for an ensemble whose accepted paths all arrived by swaps.
Read from moves.txt instead:
- acceptance ratios – every row here is accepted, so counting rows reports every move as accepted;
- per-move statistics – Mc is how the held path was made, not what this ensemble attempted;
- MD cost – MD-steps records what each attempt integrated, the rejected ones included.
Skip cycles by Step, not by rows: a cycle can have several rows. pyretis analyse -skipb / -skipe count cycles.
## moves.txt – what the sampler attempted
One line per attempted Monte Carlo move, accepted or rejected. This is the only record of a rejection, and what the acceptance ratios in both pyretis analyse and pyretis status are counted from.
```none
# Step Move Status Length Min-O Max-O O-shoot Idx-sh With ->Path MD-steps
1 sh ACC 17 -1.095714675e-01 -7.417634393e-02 -8.921102800e-02 1 - 3 16
2 s- ACC 4 -1.029370000e-01 -9.127384900e-02 - - 000 5 2
4 tr ACC 4 -1.029370000e-01 -9.127384900e-02 - - - 7 0
5 sh BTL 11 -9.277306100e-02 -6.659124237e-02 -9.277306100e-02 1 - - 10
12 tr BWI 114 -1.106760990e-01 1.024782825e-01 - - - - 0
```
Read line by line it is the story of the run: cycle 1 shot and was accepted, producing path 3; cycle 2 swapped with ensemble 000; cycle 5 shot from order parameter -0.0928 and was rejected for being too long backwards (BTL), so no path was produced – but it still integrated 10 frames of MD, which MD-steps records.
Column
Meaning
Step
The cycle the move was attempted in.
Move
The move that actually ran: sh shooting, wf wire fencing, ss stone skipping, wt web throwing, tr time reversal, mr mirror, ts target swap, s+/s- the zero swap between [0^-] and [0^+], p+/p- the PPTIS/REPPTIS swap with the neighbour above/below, ld a loaded path.
Status
ACC, or the code saying why it was rejected. The codes are listed in user-guide-path-types-rejections.
Length
Length of the trial path, in frames.
Min-O / Max-O
The lowest and highest order parameter the trial reached. For a rejection these say how close it got.
O-shoot
The order parameter of the shooting point. Written for the shooting-like moves only (sh, ss, wt, wf, bias); - for the others. Wire fencing records the point of its last jump.
Idx-sh
For sh and bias, the frame the shooting point was taken from; for ss the one-step crossing tries, for wt the accepted segments, and for wf the usable segments.
With
The ensemble this move swapped with; - when it was not a swap.
->Path
The path the move produced, or - when it was rejected – a rejected move produces no path.
MD-steps
The frames this attempt integrated in this ensemble: its trial, accepted or rejected, and every sub-trajectory the move integrated and discarded, such as the segments of wire fencing and stone skipping. The starting frame of each propagation is handed over rather than integrated, so it is not counted. A move that runs no dynamics – time reversal, mirror – records 0. For a swap, each partner’s line records what its own leg integrated. This is the MD the ensemble spent, and what the path-length statistics of pyretis analyse are built from.
A column that does not apply to a move is written -, never 0. A zero in O-shoot would read as “the shooting point was at zero”, which for a swap or a time reversal is not imprecise but false. Read - as “this move has no such quantity”.
Because a swap touches two ensembles, it is recorded in both: under the + code (s+ or p+) in the lower-index partner and the - code in the higher one, each naming the other under With. The same event, seen from each side.
### Navigating it
The file is plain text with fixed columns, so the ordinary tools work. The questions worth asking, and how:
Question
Command
Acceptance rate per move
awk '!/^#/{n[$2]++; if($3=="ACC") a[$2]++} END{for(m in n) print m, a[m]"/"n[m]}' moves.txt
Why are moves rejected?
awk '!/^#/ && $3!="ACC" {print $3}' moves.txt | sort | uniq -c | sort -rn
Is the sampler stuck?
awk '!/^#/{print $3}' moves.txt | uniq -c – long runs of one code stand out
Which shooting points failed?
awk '!/^#/ && $2=="sh" && $3!="ACC" {print $7}' moves.txt
Are swaps happening?
grep -c ' s[+-] ' moves.txt
What made path 144?
awk '!/^#/ && $10==144' moves.txt
Everything across all ensembles
awk '!/^#/ && $3!="ACC" {print FILENAME, $3}' 0*/moves.txt
The fourth of those is worth dwelling on. The distribution of shooting points that failed is the sharpest diagnostic for a badly placed interface: if the rejections cluster near one end of the path, the shooting is being asked to do something the interface spacing makes unlikely. Simply running pyretis status gives the summary; this gives the detail behind it.
Older runs are read too. Before this split, the attempted moves were written into pathensemble.txt itself with a trailing Trial column (0 occupancy, 1 attempt), and before that into a separate trials.txt. All three layouts still analyse, so a run made with any version can be re-analysed with this one.
## order.txt and energy.txt – along the path
Both files are a sequence of blocks, one per pathensemble.txt row, in the same order and labelled with the same cycle:
```none
# Cycle: 1, status: ACC, move: ('ld', nan, 0, 0)
# Time Orderp
0 -0.900188
1 -0.899927
```
Time is the frame index along the path, not a physical time. energy.txt has the same block structure with potential, kinetic, total energy and temperature columns; a term the engine does not report is written as nan rather than guessed.
Because the blocks are paired one-to-one with the rows of pathensemble.txt, the two files can be read side by side – that is how the analysis reconstructs the order parameter of a given cycle.
## cross.txt – crossings in a flux run
A flux simulation (task = "md-flux") writes three columns: the step, the interface number, and the direction of the crossing (+ or -). The interfaces are numbered from 1, not from 0.
## The trajectory stores
Trajectories are kept in two tiers. accepted// holds the paths the ensembles are currently working with. When a path is replaced it moves to archive//. The default is archive_every = 1: every replaced path is kept, so a long external-engine run stores every trajectory it ever generated. Raise [output] archive_every to keep only every n-th replaced path when disk space matters.
## Reading the report
pyretis analyse -i retis.toml writes report/ containing the same report in several formats (reStructuredText, HTML, LaTeX/PDF) plus the figures as PNG and the underlying numbers as .txt.gz. The headline result is at the top:
```none
* The crossing probability:
P_cross = 0.000000000e+00 ± nan %
* The initial flux (unit: 1/reduced):
f_A = 0.417660720 ± 6.431378533 %
* The rate constant (unit: 1/reduced):
k_AB = 0.000000000e+00 ± nan %
```
The rate constant is the product of the flux out of the initial state and the crossing probability, so an unusable value in either one makes the rate unusable (user-guide-rate derives that product and shows where each factor is measured). The example above is taken from a deliberately short run and shows exactly that: see user-guide-output-converged.
Three tables follow.
Interfaces simply restates the interface positions the analysis used, so a report is self-contained. Check it first if numbers look unexpected – it is where a mismatched input file shows up.
Crossing probabilities gives, per ensemble, the probability that a path reaching that interface goes on to reach the next one, with its error estimate. These are the numbers whose product is the overall crossing probability. Individual values in a healthy simulation are typically a few tens of percent; a value of exactly 1.000000 with zero error, or exactly 0, means that ensemble sampled nothing informative.
Pathensemble data reports, per ensemble, the number of analysed cycles, four measures of how the sampling is doing, and the average path length. user-guide-analyse-acceptance explains them in full; in short:
- Shoot acc. ratio – the fraction of ordinary shooting trials that were accepted.
- Swap acc. ratio – the same for swapping moves. It reads n/a for the outer ensembles by design, because only the zero swap is an accept/reject move; the rest exchange through the infinite-swapping weights.
- HA eff. – the weight a high-acceptance move (wire fencing, stone skipping) collects per attempt. Those moves are built to be accepted, so counting acceptances says nothing; the weight does.
- Exchange – how much of the ensemble’s occupancy is handed to another path per cycle. This is what answers, for every ensemble, the question the swap column can only answer for the zero swap.
A value shown as n/a means the move was never attempted, not that it was never accepted.
### The figures
Per ensemble, named by its directory number:
Figure
What it shows, and what to look for
NNN_pcross
The crossing probability as a function of the order parameter. It should fall smoothly and monotonically from 1 at the ensemble’s own interface. Steps and flat plateaus mean few distinct paths contributed – the histogram is being carried by a handful of trajectories.
NNN_prun
The running average of the crossing probability against cycle number. This is the primary convergence diagnostic. It should start noisy and flatten out. If it is still drifting at the right edge, the run has not converged, whatever the quoted error says.
NNN_perror
Block-error analysis: the estimated error against block length. Correlated data makes the estimate grow with block length until it reaches a plateau; the plateau value is the honest error. If the curve is still rising at the largest block, the run is too short for its own error estimate to be trusted.
NNN_lpath
Two distributions. The first, of the accepted path lengths: a long tail towards the maximum length means trajectories are struggling to terminate, and if paths pile up at maxlength they are being truncated, which biases the sampling. The second, of the MD each attempted move integrated in this ensemble – the MD-steps of moves.txt, rejected trials included – is what the sampling of this ensemble cost.
NNN_shoots
Where the shooting points were taken and which of them were accepted. Accepted shots concentrated in a narrow region mean the ensemble is exploring poorly.
And for the simulation as a whole:
Figure
What it shows
total-probability
Every ensemble’s crossing probability curve on one axis, before matching.
matched-probability
The single curve obtained by matching those together. This is the overall crossing probability, and it should decay smoothly over orders of magnitude. A kink at an interface means neighbouring ensembles disagree where they overlap – usually too few paths, or interfaces spaced too widely.
overall-prun
The running average of the overall result. Read it exactly like NNN_prun: flat means converged.
overall-err
The block-error analysis of the overall result.
## Is it converged?
No single number answers this. Work through the following, in order – each is a way for a run to be wrong that the others do not catch:
- Did the outermost ensemble ever reach the final state? If the overall crossing probability is exactly 0 and the rate is 0 with a nan error, the answer is no, and nothing else in the report matters yet. This is the normal appearance of a run that is simply too short.
- Has the running average flattened? Look at overall-prun and the per-ensemble NNN_prun. A curve still trending at the right edge means the estimate is still moving; the run needs more cycles.
- The two below are the same ensemble of the same system, at 200 and at 20000 cycles. On the left the estimate sits at zero until cycle 150 – nothing had crossed yet – and is then climbing steeply when the run ends: whatever number that run reports is the value the curve happened to have reached. On the right it swings wildly for the first few thousand cycles and then settles onto 0.242 and stays there. That settling is what “converged” looks like.
- Has the block-error curve reached a plateau? In overall-err and NNN_perror, a curve still rising at the largest block length means the quoted error is an underestimate – the samples are more correlated than the run is long.
- Is the matched probability smooth? Kinks at interface positions in matched-probability mean neighbouring ensembles do not agree in their overlap region. A healthy one falls smoothly through orders of magnitude, with the interfaces (dashed) spread evenly along the decay – here from 1 to about 4 \times 10^{-7}, the crossing probability the run reports. It flattens past the last interface because nothing beyond it was sampled.
- Are the per-ensemble numbers sane? Crossing probabilities of exactly 0 or exactly 1, acceptance ratios near zero, or path-length distributions piled up against maxlength all point at a badly configured ensemble rather than at a merely short run.
If a run fails only test 2 or 3, it needs more cycles. If it fails 4 or 5, more cycles will not help on their own: the interfaces are probably placed too far apart, or the move set does not suit the system.
## A worked example: the same simulation, short and long
The clearest way to learn the difference is to read one system’s report twice. Both runs below are the shipped 1D double-well RETIS example, unchanged except for [simulation] steps.
### Too short: 200 cycles
```none
* The crossing probability:
P_cross = 0.000000000e+00 ± nan %
* The initial flux (unit: 1/reduced):
f_A = 0.417660720 ± 6.431378533 %
* The rate constant (unit: 1/reduced):
k_AB = 0.000000000e+00 ± nan %
```
This looks alarming and is merely short. Nothing is broken: the flux is measured perfectly well, because it only needs the innermost ensemble. But in 200 cycles no path ever reached the outermost interface, so the crossing probability is exactly zero, and the rate – their product – with it. The nan is the honest report of an error that cannot be formed from zero successes, not a numerical fault.
The tell that this is a short run and not a broken one is that everything else looks healthy: the per-ensemble table of the same run has shooting acceptance ratios between 0.5 and 0.9, and sensible path lengths. Test 1 fails; tests 4 and 5 pass.
### Long enough: 20000 cycles
```none
* The crossing probability:
P_cross = 3.908476360e-07 ± 53.824318315 %
* The initial flux (unit: 1/reduced):
f_A = 0.271443162 ± 2.832354761 %
* The rate constant (unit: 1/reduced):
k_AB = 1.060929183e-07 ± 53.898789185 %
```
Now there is a rate. Note what did not change much: the flux, which was already well determined after 200 cycles (2.8 % error here). All the difficulty is in the crossing probability, and it is worth seeing why:
```none
| Ensemble | Pcross | Error | Rel. error (%) |
| [0^+] | 0.244074 | 0.017055 | 6.987693 |
| [1^+] | 0.155206 | 0.018709 | 12.054258 |
| [2^+] | 0.124862 | 0.020380 | 16.322180 |
| [3^+] | 0.124981 | 0.022884 | 18.309615 |
| [4^+] | 0.106953 | 0.027190 | 25.422839 |
| [5^+] | 0.188040 | 0.031766 | 16.893095 |
| [6^+] | 0.032874 | 0.011243 | 34.198995 |
```
Each ensemble measures a conditional probability of order 0.1, none of them alarming on its own. The overall crossing probability is their product – which is how a rate of 10^{-7} is obtained from simulations that never wait 10^{7} steps for anything – and the relative errors compound along the chain, from 7 % at the innermost interface to 34 % at the outermost, giving the 54 % on the final result.
So this run has converged in the sense that matters least and most. It has converged enough to produce a rate, which the 200-cycle run could not; it has not converged to a precise one. If you need better than a factor of two, this system needs more cycles, more interfaces in the outer region, or both. Reading the error is not optional.
### A second system: water-dimer dissociation with GROMACS
The double well above is a one-particle toy. The same report on a real MD engine – the shipped water-dimer RETIS example, GROMACS at 250 K, 2000 cycles, order parameter the O–O distance – reads:
```none
* The crossing probability:
P_cross = 0.116483662 ± 33.757375544 %
* The initial flux (unit: 1/gromacs):
f_A = 0.155524126 ± 24.403500782 %
* The rate constant (unit: 1/gromacs):
k_AB = 1.811601973e-02 ± 41.654426584 %
```
The striking difference is the size of the answer: a crossing probability of 0.12 against the double well’s 4 \times 10^{-7}. Breaking a water dimer is not a rare event on this order parameter – it is barely activated – so the per-ensemble probabilities run the other way:
```none
| Ensemble | Pcross | Error | Rel. error (%) |
| [0^+] | 0.428929 | 0.121031 | 28.216993 |
| [1^+] | 0.558257 | 0.073001 | 13.076596 |
| [2^+] | 0.722222 | 0.081805 | 11.326792 |
| [3^+] | 0.882557 | 0.043049 | 4.877726 |
| [4^+] | 0.878774 | 0.030353 | 3.453983 |
| [5^+] | 0.929374 | 0.019230 | 2.069189 |
| [6^+] | 0.934468 | 0.018839 | 2.016036 |
```
Put that beside the double well’s curve above and the difference is the whole story: this one spans well under two orders of magnitude, and most of the drop happens between the first two interfaces. They rise outward, 0.43 to 0.93, where the double well’s fell. Once the dimer is stretched past the inner interfaces it comes apart almost every time, so the outer ensembles are easy and the whole difficulty sits at the first interface – which is also where the largest relative error is. Read that as a design signal: this interface set is finer than this system needs at the outside and too coarse at the inside.
The acceptance table tells the same story from the sampling side:
```none
| Ensemble | TIS cycles | Shoot acc. ratio | Swap acc. ratio | Exchange |
| [0^-] | 1998 | 0.617647 | 0.978632 | 0.000000 |
| [0^+] | 7074 | 0.689922 | 0.978632 | 0.242415 |
| [1^+] | 9798 | 0.600000 | n/a | 0.445224 |
| [2^+] | 10054 | 0.460630 | n/a | 0.597915 |
| [3^+] | 10394 | 0.351852 | n/a | 0.711665 |
| [4^+] | 9870 | 0.274510 | n/a | 0.738265 |
| [5^+] | 9178 | 0.178862 | n/a | 0.758949 |
| [6^+] | 8636 | 0.093496 | n/a | 0.757857 |
```
Shooting acceptance falls steeply outward, 0.69 down to 0.09 – an external engine on a floppy molecule is far harder to shoot in than a Langevin particle in a smooth well, where the same column only fell to 0.64. An acceptance below 0.1 in the outermost ensemble is the number to act on: it says most of the work spent there is thrown away, and that the interfaces are too widely spaced at the outside.
Note also that Exchange rises outward here (0.24 to 0.76) while acceptance falls. That is the infinite-swapping machinery doing its job: where shooting struggles, the ensembles lean more on exchanging occupancy with their neighbours.
The engine unit shows in the report (1/gromacs), because the rate carries whatever time unit the engine works in. Compare rates only after converting to a physical unit.
### The health check
The per-ensemble table of the long run is what a well-behaved RETIS simulation looks like:
```none
| Ensemble | TIS cycles | Shoot acc. ratio | Swap acc. ratio | Avg. path length |
| [0^-] | 19998 | 0.808360 | 1.000000 | 1.5051e+03 |
| [0^+] | 26839 | 0.918100 | 1.000000 | 340.941287 |
| [1^+] | 29581 | 0.810756 | n/a | 507.382453 |
| [2^+] | 27511 | 0.718249 | n/a | 608.313264 |
| [3^+] | 26428 | 0.686131 | n/a | 713.262891 |
| [4^+] | 26173 | 0.682670 | n/a | 819.305071 |
| [5^+] | 26499 | 0.671490 | n/a | 954.391600 |
| [6^+] | 24253 | 0.636364 | n/a | 1.1479e+03 |
```
Two trends are the signature of a healthy run, and both are visible:
- Acceptance falls outwards, 0.92 down to 0.64. Outer ensembles ask more of a trajectory, so more trials are rejected. A flat column – and especially a column of exactly 1.000000 – would mean the rejections are not being counted, not that the sampling is perfect.
- Paths grow longer outwards, 341 up to 1148 frames, because a path has further to travel before it can return. Lengths that stop growing, or that pile up at maxlength, mean trajectories are being truncated.
The swap column reads n/a from [1^+] outwards by design, as explained above: only the zero swap is an accept/reject move.
---
# Path types and rejection reasons
During a PyRETIS simulation, each Monte Carlo cycle generates (or loads) a path and records two short codes in the output:
- a move type (how the path was generated), and
- a status code (whether the path was accepted, and if not, why).
Both codes appear in the per-ensemble output files and in the simulation log, e.g.:
```
# Cycle: 42, status: ACC, move: sh
```
The tables below list every code that PyRETIS can produce.
Adding a new move? Validate that it reproduces the reference rate with the reproducibility suite – see Extending and validating PyRETIS.
## Move types
The move type describes how the new path was generated. For an explanation of what each move actually does – the basic moves, the sub-trajectory moves and infinite swapping – see user-guide-moves.
Code
Short name
Description
sh
Shoot
Path generated with a shooting move: a phase-space point is selected on the current path, momenta are perturbed, and new forward and backward trajectories are integrated. A path made by biased shooting is written sh as well; moves.txt records its attempt as bias.
ss
Stone skipping
Path generated by the Stone Skipping algorithm, a multi-step variant of shooting that reuses short sub-paths.
wt
Web Throwing
Path generated by the Web Throwing move.
wf
Wire Fencing
Path generated by the Wire Fencing move.
mw
Multiple wire fencing
Path generated by multiple wire fencing (mwf). The Mc column holds two characters, so pathensemble.txt writes the move as mw; moves.txt records it as mwf. The move shoots its segments from many points, so its O-shoot holds a marker value, not a shooting point.
ts
Target swap
Path generated by a target swap move.
tr
Time-reversal
Path generated by time-reversal: the current accepted path is reversed in time to produce the trial path.
s+
Swap from +
Path produced by the zero swap between [0^-] and [0^+] (the RETIS one, or the QuanTIS one when quantis = true), recorded in [0^-]: the incoming path came from the + (higher-index) ensemble.
s-
Swap from −
The same zero swap, recorded in [0^+]: the incoming path came from the - (lower-index) ensemble.
p+
PPTIS swap from +
Path produced by the PPTIS/REPPTIS swap between adjacent ensembles, recorded in the lower-index partner: the incoming path came from the + (higher-index) ensemble.
p-
PPTIS swap from −
The same PPTIS/REPPTIS swap, recorded in the higher-index partner: the incoming path came from the - (lower-index) ensemble.
mr
Mirror
Path generated by a mirror move.
hc
Held across rule change
A path an ensemble held at a restart that changed [tis] high_accept. It was accepted under the other acceptance rule, so, like an initialisation path, it is left out of the statistics until it is replaced.
ki
Kick initiation
Path generated by integrating the equations of motion after kicking (re-drawing) the initial momenta; used during path initiation.
is
Initial shoot
Path generated by a shooting move applied during the initial Stone Skipping preparation step, before the actual SS move.
ld
Load (unformatted)
Path loaded from unformatted (binary) external file(s).
rf
Load (formatted)
Path loaded from formatted (text) external file(s).
re
Restart
Path restored from a restart simulation.
Unlike ki, ld and is, a re path (and an rf one) carries full, trusted phase points, so it is not treated as an initialisation path: the sampling continues normally and the path counts towards the production statistics. It contributes nothing to the generated-path-length distribution, though, because restoring it ran no dynamics.
00
Null move
Path produced by a null move: the current path is kept without any modification (used internally).
## Path status codes
The status code explains why the trial path was accepted or, if rejected, what condition was violated.
Acceptance
Code
Meaning
ACC
The trial path was accepted and becomes the new current path.
Trajectory-length rejections
These rejections occur when the generated trajectory is too long or too short to satisfy the detailed-balance or max-length conditions.
Code
Meaning
BTL
Backward trajectory too long — the backward segment exceeded the limit imposed by the detailed-balance condition.
BTX
Backward trajectory too long — the backward segment exceeded the hard maxlength cut-off.
BTS
Backward trajectory too short — the backward segment ended before reaching a required interface.
FTL
Forward trajectory too long — the forward segment exceeded the limit imposed by the detailed-balance condition.
FTX
Forward trajectory too long — the forward segment exceeded the hard maxlength cut-off.
FTS
Forward trajectory too short — the forward segment ended before reaching a required interface.
XSS
Stone Skipping sub-path too long — an internal sub-path within the Stone Skipping algorithm exceeded the allowed length.
Interface / crossing rejections
Code
Meaning
NCR
No crossing with the middle interface — the trial path did not cross the required central interface of the ensemble.
BWI
Backward trajectory ends at wrong interface — the backward segment terminated at an interface other than the one required.
EWI
Initial path ends at wrong interface — used during path loading/initiation: the loaded path ends at an interface that is not compatible with the ensemble.
SWI
Initial path starts at wrong interface — used during path loading/initiation: the loaded path starts at an interface that is not compatible with the ensemble.
0-L
{0−} path ends at left interface — for paths in the {0-} ensemble the path ended at the left (A-state) interface instead of crossing to the right.
NSS
No one-step crossing in Stone Skipping — the Stone Skipping algorithm could not find a valid one-step interface crossing.
Momentum / energy rejections
Code
Meaning
MCR
Momenta change rejection — the new momenta drawn during the shooting move were rejected (e.g. by the Metropolis criterion on the kinetic energy).
KOB
Shooting point outside of boundaries — the shooting point lies below the left interface or above the right interface of the ensemble. Both propagation legs start from the shooting point and would stop on it, so the trial is rejected before any propagation. The point tested is the one after the momentum kick, which moves a velocity-dependent order parameter, or the crossing frame that the initial shooting move (is) of Stone Skipping shoots from, which in the {0+} ensemble lies below the left interface for one frame of every crossing.
Detailed-balance / super-detailed-balance rejections
These rejections enforce the (super) detailed-balance conditions for advanced moves.
Code
Meaning
HAS
High-acceptance swap rejection — the high-acceptance criterion for Stone Skipping / Wire Fencing detailed balance was not satisfied.
SSA
Stone Skipping super-detailed-balance rejection — the Stone Skipping move was rejected by the super-detailed-balance acceptance test.
WFA
Wire Fencing super-detailed-balance rejection — the Wire Fencing move was rejected by the super-detailed-balance acceptance test.
WTA
Web Throwing super-detailed-balance rejection — the Web Throwing move was rejected by the super-detailed-balance acceptance test.
SAR
Selection-bias acceptance rejection — a valid biased shooting trial path was rejected by the selection-bias Metropolis test \min(1, W_\mathrm{old}/W_\mathrm{new}).
Target-swap rejections
Code
Meaning
TSS
Target swap selection rejection — the target-swap move was rejected at the selection step (no suitable target ensemble was found).
TSA
Target swap detailed-balance rejection — the target-swap trial was rejected by the detailed-balance acceptance criterion.
REPPTIS-specific rejections
Code
Meaning
SWD
REPPTIS swap — wrong propagation direction — in a REPPTIS swap move the trial trajectory propagated in the wrong direction relative to the interface.
SWH
REPPTIS swap — first-half extension rejected — the first half of the REPPTIS swap extension was rejected.
Other
Code
Meaning
NSG
No suitable segments — the path contains no segments that can be used as shooting points.
EXP
Exploration path — the path was generated during an exploration phase and is not counted as a regular MC step.
---
# The PyVisA application
PyVisA is the analysis and visualization tool for PyRETIS simulations. It consists of two components:
- A compressor that reads raw simulation output, checks consistency, and stores the data in pyvisa_compressed_data.hdf5.zip.
- A visualizer (GUI) that loads compressed or raw data and produces interactive plots, cluster analyses, and PCA results. The GUI has additional optional requirements, which must be installed separately (see installation).
The compressor and recalculation tools can run without a graphical environment. This is useful on remote clusters and other machines where Qt or an X display is not available. A common workflow is to prepare the compact PyVisA data on the remote machine and inspect it later on a local workstation with the GUI.
## ChemiTrees and post-analysis
ChemiTrees are a post-analysis workflow for converting path-sampling trajectories into chemically interpretable decision trees. Instead of selecting a mechanism only by visual inspection, trajectory descriptors are classified into reactive and unreactive paths and the resulting tree identifies the geometric conditions that separate the two classes.
In the PyVisA workflow this belongs after compression and, when needed, recalculation of order parameters or collective variables: create the compact HDF5 data set, add the descriptors that matter for the system, and then use the statistical-analysis routines to build decision-tree, clustering or PCA views of the trajectory ensemble. The current API entry point for the tree analysis is pyretis.pyvisa.statistical_methods.decision_tree.
The ChemiTrees publication is listed in our-results. The corresponding reproduction input files are not yet mirrored on this website; they should be added there once the original data package has been recovered.
The general syntax is:
```pyretis
pyvisa [-h] [-i INPUT] [-V] [-cmp] [-data DATA] [-recalculate] [-oo] [-p] [-w N]
```
where the arguments are described in tableappargument_pyvisa.
Flags can be freely combined: -recalculate and -cmp may be given together to run recalculation followed by compression in one step. If neither is set, the GUI is launched.
## Choosing a workflow
The most common PyVisA workflows are:
- Remote or headless use: run the compressor and, if needed, recalculation from the command line. This only writes files and does not open the GUI.
- Local GUI use: install the GUI requirements and open either the compressed PyVisA file or the original simulation input file.
- Remote GUI use: install the GUI requirements on the remote machine and run PyVisA through an environment that can show graphical applications, such as X11 forwarding or a remote desktop session.
## Remote or headless usage
On a remote machine, first go to the completed simulation directory. This is the directory containing the original PyRETIS input file and the numbered ensemble folders such as 000, 001, and 002.
To compress the simulation output without opening the GUI, run:
```pyretis
pyvisa -i retis.toml -cmp
```
This creates pyvisa_compressed_data.hdf5.zip in the current directory. The original simulation output is not modified.
If the remote machine should only package order parameter data, use:
```pyretis
pyvisa -i retis.toml -cmp -oo
```
The -oo option skips the energy files and can be useful when only the order parameter and collective variable data are needed.
To recalculate order parameters or collective variables after changing retis.rst or orderp.py, run:
```pyretis
pyvisa -i retis.toml -recalculate
```
The recalculation uses the trajectory files found in the simulation directory and writes updated order.txt data. To limit the recalculation to one trajectory file or a directory of trajectory files, use -data:
```pyretis
pyvisa -i retis.toml -recalculate -data path/to/traj.xyz
pyvisa -i retis.toml -recalculate -data path/to/trajectory_folder
```
After recalculation, run the compressor again if you want a new compact file for transfer or later visualization:
```pyretis
pyvisa -i retis.toml -cmp
```
The remote environment needs the Python packages required to read the trajectory files and execute the order parameter code used by the simulation. The GUI dependencies are only needed when a PyVisA window will be opened.
## Local GUI usage
Copy the compressed file from the remote machine to a local workstation where the PyVisA GUI requirements are installed:
```pyretis
scp user@remote.host:/path/to/simulation/pyvisa_compressed_data.hdf5.zip .
```
Then open it locally:
```pyretis
pyvisa -i pyvisa_compressed_data.hdf5.zip
```
You can also open the GUI directly from a simulation directory:
```pyretis
pyvisa -i retis.toml
```
or load a specific ensemble or data source:
```pyretis
pyvisa -i retis.toml -data 000
```
## Direct remote GUI usage
Direct remote GUI usage is possible when the remote environment can show graphical applications. Install the GUI requirements on the remote machine, connect with display forwarding or a remote desktop session, and run PyVisA from the simulation directory:
```pyretis
pyvisa -i retis.toml
```
For large simulations this can be slower than the headless workflow, because the GUI must read the simulation data interactively over the remote session. Compressing remotely and opening the compressed file locally is usually the simpler workflow.
## Example use
Compressor
- Compress raw simulation output into pyvisa_compressed_data.hdf5.zip:
```pyretis
pyvisa -i retis.toml -cmp
```
- Compress using only order parameter files (faster, smaller output):
```pyretis
pyvisa -i retis.toml -cmp -oo
```
Recalculate
- Recalculate order parameters and collective variables:
```pyretis
pyvisa -i retis.toml -recalculate
```
- Recalculate for a single ensemble only:
```pyretis
pyvisa -i retis.toml -recalculate -data 000
```
- Recalculate with a progress bar and 4 parallel workers:
```pyretis
pyvisa -i retis.toml -recalculate -p -w 4
```
Combined flags
- Recalculate and then compress in one step:
```pyretis
pyvisa -i retis.toml -recalculate -cmp
```
- Recalculate a single ensemble, then compress with order files only:
```pyretis
pyvisa -i retis.toml -recalculate -data 000 -cmp -oo
```
Visualization GUI
- Open the visualization GUI from a compressed file:
```pyretis
pyvisa -i pyvisa_compressed_data.hdf5.zip
```
- Open the GUI from the original input file, loading a specific ensemble:
```pyretis
pyvisa -i retis.toml -data 000
```
- Open the GUI loading all data with order files only (faster):
```pyretis
pyvisa -i retis.toml -oo
```
For a guided walkthrough of these features, see examples-pyvisa and examples-pyvisa-analysis.
## Input arguments
## Description of input arguments for PyVisA.
Argument
Description
-h, --help
Show the help message and exit.
-i INPUT, --input INPUT
Location of the input file (.toml, .hdf5, or .hdf5.zip).
-V, --version
Show the version number and exit.
-cmp, --pyvisa_compressor
Run the compressor tool to generate pyvisa_compressed_data.hdf5.zip from raw simulation output.
-data DATA, --pyvisa-data DATA
Select a subset of data to load. Use all for all data, or a folder name (e.g. 000) for a single ensemble.
-recalculate, --pyvisa-recalculate
Recalculate order parameter and collective variable data using the current orderp.py.
-oo, --only_order
Use only data from order.txt files when compressing (faster, skips energy data).
-p, --progress
Show progress bars during recalculation.
-w N, --workers N
Number of parallel worker processes for recalculation (default: all CPU cores).
## Visualisation and Analysis with PyVisA
PyVisA
Compress, inspect and visualise path-sampling trajectories from the tutorial suite.
Tutorial source
PyVisA analysis
Add collective variables, run PCA and cluster trajectories from the methane-hydrate example.
Tutorial source
ChemiTrees
Use decision-tree post-analysis to identify molecular descriptors that distinguish reactive and unreactive path-sampling trajectories.
Paper
---
# Quickstart
After installing PyRETIS, you can run simulations by using a text based input file or by using the PyRETIS library explicitly.
The former approach is perhaps the simplest, but you will first have to learn how to create input files and this is explained in the input description.
The latter option is more involved, but you are then given more freedom in defining, running and interacting with a simulation. In order to make use of the library, you will have to read about the structure of the PyRETIS library in the introduction to the library and in the detailed reference section.
A more extensive overview can be found in the full user guide. Since version 2.4, PyVisA is automatically installed with PyRETIS. Here, we report some examples showing the use of PyRETIS and PyVisA.
## Quickstart workflow
Your first run takes one input file and two commands. The file below is a complete RETIS simulation of a single particle in a double well – it is the minimal input file tutorial, and it is short because PyRETIS only needs to be told the physics and fills in everything else.
- Install PyRETIS in a Python 3.11+ environment:
```bash
python -m pip install pyretis
```
- Check that the command-line tools are available:
```bash
pyretis run --version
pyretis analyse --version
```
- Create a directory with two files. The first is the input, minimal.toml:
```toml
# The smallest input that runs a RETIS simulation. Every setting here
# describes the physics of the problem; everything else -- the move set,
# the initiation, the output frequencies, the path-length cap, the box --
# PyRETIS fills in with defaults (they are echoed to output.toml).
potential = [
{ class = "DoubleWell", a = 1.0, b = 2.0, c = 0.0 },
]
[simulation]
task = "retis"
steps = 100
interfaces = [-0.9, -0.8, -0.5, 1.0]
[system]
dimensions = 1
temperature = 0.07
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
[particles.position]
input_file = "initial.xyz"
[orderparameter]
class = "pyretis_position"
dim = "x"
index = 0
```
- The second is the starting configuration it points at, initial.xyz – one particle at x = -1:
```
1
Ar -1.0 0.0 0.0
```
- Both ship with PyRETIS under examples/tutorials/path_sampling/internal/1D-double-well/minimal/, so you can copy that folder instead of typing them.
- Run the simulation (about a minute):
```bash
pyretis run -i minimal.toml -p
```
- Analyse it. The analysis reads the same input file as the run:
```bash
pyretis analyse -i minimal.toml
```
After a successful run you have a pyretis.log, one numbered folder per path ensemble (000, 001, …), an output.toml recording the input the run actually used – under the names of the input, with every default it filled in – and the state to continue from, and, after the analysis, a report/ folder with the crossing probabilities and the rate.
## What that run actually does
A path-sampling run has two phases, and the log shows them in order.
1. Initiation. Before anything can be sampled, every path ensemble needs one trajectory that already belongs to it – a path that starts and ends in the right places and crosses that ensemble’s interface. The run generates these from the starting configuration you supplied (method = "kick", the default): it gives the system a random kick, integrates forwards and backwards until the trajectory terminates, and keeps it if it satisfies the ensemble. You will see one Found initial path for [0^-], [0^+], [1^+] … line per ensemble. This phase produces no statistics; it only gets each ensemble a valid starting point.
2. Sampling. Then the actual Monte Carlo begins, and it is what steps counts. Each step picks an ensemble and one of the paths it holds, proposes a new trajectory from that path – usually by shooting: pick a point on the path, draw new velocities there, and integrate outwards to get a fresh trajectory – and accepts or rejects it. RETIS lets the ensembles share their trajectories: each trajectory counts for every ensemble it qualifies for (infinite swapping), and a swap move exchanges paths between [0^-] and [0^+]. That sharing is what lets the outer ensembles be reached at all. The ensemble’s pathensemble.txt records the paths it holds, cycle by cycle, and moves.txt beside it every trial that was attempted, accepted or rejected. The log line for each step names the move, the ensemble and the outcome (ACC, or a rejection code with its reason).
The point of the whole procedure is the rare event: each ensemble measures how often a trajectory that reached its interface goes on to reach the next one. Those conditional probabilities multiply into the crossing probability of the full transition, which – combined with the flux out of the initial state – gives the rate constant that pyretis analyse reports. user-guide-rate works that identity through, and says where each factor comes from in a run.
Because sampling is a Monte Carlo procedure, its numbers are only as good as the number of steps: the 100-step run above is enough to see the machinery work end to end, not to quote a rate.
## Continuing a run
To sample further, restart from the output.toml the run wrote:
```bash
pyretis run -i output.toml -p
```
output.toml reads as an input: raise [simulation] steps in it (see below) and run it. The output.toml of a run made with an earlier PyRETIS, which holds the scheduler configuration (a [simulation.tis_set] table), is refused by -i; continue that run from its input with [initial-path] method = "restart" (see the restart method).
One thing to know before you do: ``steps`` is the total number of cycles for the run, not the number to add. A run that has completed 100 steps and is restarted with steps = 100 still has nothing to do, and stops immediately – so raise steps to the new total (steps = 500 continues to cycle 500, adding 400). The log states which case you are in, either
```none
Restarting from step 100; sampling continues up to the total of 500
steps set in [simulation] steps (400 cycles to go).
```
or, when the target has already been reached, a warning naming the number of completed steps and telling you to raise it.
From here, the minimal-input tutorial explains what each section of that file is for and what got defaulted, and RETIS in a 1D potential builds the same simulation the long way, one setting at a time.
---
# Calculating the rate
Everything a path-sampling run does is in service of one number: the rate constant k_{AB} at which the system leaves state A for state B. This page explains how that number is put together – the identity the methods rest on, where each factor comes from in a PyRETIS run, and how the routes differ between TIS, RETIS, infinite swapping and the (RE)PPTIS family.
user-guide-output explains the files these numbers are computed from and how to read the report; user-guide-analyse is the reference for the application itself.
# Contents
- In one command
- The identity behind every method
- Where each factor comes from in a run
- The initial flux
- The crossing probability
- The rate
- How the methods differ
- Two estimators over the same data
- Running the analysis
- Is the number trustworthy?
## In one command
Point the analysis at the same input file the simulation used, from the directory the simulation ran in:
```bash
pyretis analyse -i retis.toml
```
That reads the per-ensemble output in place, prints the headline numbers to the screen and to pyretisanalyse.log, and writes a full report:
```none
report/
_report_cycles-NNNNNNNNN.rst the report, reStructuredText
_report_cycles-NNNNNNNNN.html the same, for a browser
_report_cycles-NNNNNNNNN.pdf the same, if pdflatex is present
*.png every figure
*.txt.gz the numbers behind each figure
```
The cycle count is in the file name, so analysing the same run again after more sampling does not overwrite the earlier report. Which formats are written is set by report, and where they go by report-dir.
The screen output ends with the three numbers this page is about:
```none
* The crossing probability:
P_cross = 3.908476360e-07 ± 53.824318315 %
* The initial flux (unit: 1/reduced):
f_A = 0.271443162 ± 2.832354761 %
* The rate constant (unit: 1/reduced):
k_AB = 1.060929183e-07 ± 53.898789185 %
```
The rest of this page explains where each of those comes from, and what to do when one of them looks wrong.
## The identity behind every method
A rare transition is rare because the system almost never gets from A to B. Waiting for it is hopeless; the trick every method here uses is to factorise the event into pieces that are each common enough to measure.
Put an ordered set of interfaces \lambda_0 < \lambda_1 < \dots < \lambda_n between the two states, with \lambda_0 on the edge of A and \lambda_n on the edge of B. Then
where
- f_A is the initial flux: how often, per unit time, the system leaves A through \lambda_0 at all. This is a frequent event, so plain molecular dynamics measures it.
- P_A(\lambda_n | \lambda_0) is the crossing probability: given that a trajectory has just crossed \lambda_0 on its way out, the probability it reaches \lambda_n before returning to A. This is the rare part.
The crossing probability is itself factorised, which is the whole point:
Each factor is the probability of getting from one interface to the next – a conditional probability of order 0.1, which ordinary sampling can measure. Their product can be 10^{-7} or smaller without any single simulation ever having to wait that long. Each ensemble [i^+] samples one factor.
That is why a rate of 10^{-7} comes out of runs of a few thousand cycles, and also why the errors multiply: see the worked example in user-guide-output-worked-example, where seven factors with 7–34 % relative error each give a rate with 54 %.
## Where each factor comes from in a run
### The initial flux
The flux is measured from the two innermost ensembles, [0^-] and [0^+], which between them cover a full round trip in and out of A. PyRETIS computes it as the reciprocal of the mean time for that round trip:
where \langle L \rangle is the mean accepted path length in frames and \Delta t the time between frames (the engine time step times subcycles). The two subtracted frames are the crossing points shared with the interface, counted once rather than twice.
A run has one \Delta t for all its engines. The ensembles of an engine pool ([simulation] ensemble_engines) and the [0^-] of a quantis run, with its [engine0], take their MD from different engine sections, and under infinite swapping a path held by one ensemble can have been generated by the engine of another ensemble; a zero swap builds a path from segments of two engines. Path lengths in frames convert to time with one time per step only, so a run whose engine sections have different times per step is refused before it starts. The analysis takes \Delta t from [engine]: timestep times subcycles, with the timestep of the lammps.in of a LAMMPS engine that leaves it out. This is the \Delta t of every engine section of the run, for the flux of the matched analysis and of the WHAM analysis alike. When the time step of [engine] cannot be read, the WHAM analysis reports its rate per step and the matched analysis asks for [engine] timestep.
This is the cheap half of the calculation: the flux is usually well determined long before the crossing probability is, which is why a short run reports a sensible f_A next to a useless P_A.
A pure task = "md-flux" run measures only this, and by a different route: it counts effective crossings of each interface during plain MD and divides by time. There the time in state A is cut into windows of skipcross steps and the flux computed in each, which is what gives the estimate an error bar. The window changes the error, not the flux.
### The crossing probability
Each ensemble’s own crossing probability curve is a histogram over the highest order parameter its accepted paths reached. Turning the set of curves into one number is called matching: each ensemble’s curve is scaled so that it continues its neighbour’s where they overlap, and the product of the per-ensemble factors is accumulated as the curves are joined. The result is the matched-probability figure, and its value at the outermost interface is P_A(\lambda_n|\lambda_0).
### The rate
The relative errors add in quadrature, so the rate is never more precise than its worse factor – in practice always the crossing probability.
The rate carries the engine’s time unit: a report from an external engine says 1/gromacs or similar. Convert before comparing.
## How the methods differ
All of them sample paths and all of them end at k = f_A P_A. They differ in which ensembles they define and how the per-ensemble probabilities are combined.
task
What it samples
How the rate is formed
md-flux
Plain MD in A, recording crossings of each interface (cross.txt).
Gives f_A only – no crossing probability, so no rate. Use it to get the flux cheaply, or to place interfaces.
tis
With two, or four or more interfaces: one path ensemble per interface, the [0^-] included, each sampled independently. With three interfaces: the one ensemble of those interfaces.
With the [0^-]: the flux, the crossing probability and the rate, formed as for retis. A single TIS run (three interfaces) has no [0^-] ensemble and so no flux: it reports a crossing probability and needs the flux from a separate md-flux run.
retis
The same ensembles, [0^-] included, sampled together, with swap moves exchanging paths between neighbours.
Flux and crossing probability come from the one run, so it reports the rate directly. This is the usual choice.
infinite_swapping
The same physics as RETIS, but instead of accepting or rejecting a swap the coordinator spreads each path’s occupancy over the ensembles it is valid in.
Identical sampling target, so the same matched estimator applies. A second estimator (WHAM) is also available – see below.
pptis / repptis
Partial paths: each ensemble samples only between neighbouring interfaces rather than all the way back to A, which keeps path lengths short in long or diffusive systems.
The local probabilities are assembled into a Markov state model over the interfaces and the global crossing probability is obtained from it (global_pcross_msm()), rather than by a plain product. The same MSM also yields mean first-passage times.
Whichever task is written in the input, the sampling goes through one code path – the infinite-swapping coordinator. What the task selects is the ensemble layout, the move set and the estimator.
## Two estimators over the same data
For a run that produced per-ensemble output, two independent estimators can be applied to it, chosen with the method keyword:
The point-matching product described above – the classic RETIS estimator, with block-error analysis.
Weighted-histogram analysis over the same paths, which reweights all ensembles together instead of matching them pairwise.
Run each and report both.
They are different estimators of the same physical rate, so on a converged run they agree within their statistical error. Disagreement beyond that is a signal – usually that the run is not as converged as the error bars suggest. Running method = "both" is a cheap cross-check, since both read the same files.
## Running the analysis
```bash
pyretis analyse -i input.toml
```
The input is the same file the simulation used. The analysis reads the per-ensemble output in place and writes report/.
The output.toml the run writes records the input of the run, under the names of the input and with every default resolved, and the [current] state. pyretis analyse -i output.toml reads the input sections from it and gives the analysis of the input; the output.toml of a run of a legacy-runner input records its canonical form. The output.toml of a run made with an earlier PyRETIS, a legacy-runner run with an engine pool among them, holds the scheduler configuration instead. pyretis analyse -i output.toml reads its settings under the names of the input, through the key table, and a setting it does not hold, the [analysis] section among them, takes its default, which a warning says. The state file of a tis run of one ensemble is refused: the analysis of that ensemble counts a crossing at [tis] detect when the input gives [tis] ensemble_number, and the scheduler configuration holds neither [tis] detect nor whether the input gave the number. Analyse such a run with its input file. The analysis of a tis run of several ensembles counts the crossings of each ensemble at the interface after its middle one, and reads its state file as the state file of any other run.
The arguments are listed in tableappargument_analyse; the two worth knowing are -skipb and -skipe, which choose the cycle window without editing the input file:
```bash
pyretis analyse -i input.toml -skipb 500 # drop equilibration
pyretis analyse -i input.toml -skipb 500 -skipe 100 # and a cut-short tail
```
Everything else is set in the [analysis] section of the input file. The keywords that change the result rather than the presentation are:
Keyword
Effect on the rate
method
Which estimator(s) to use: matched, wham or both.
skip_initial_cycles
Cycles discarded at the start; -skipb overrides it.
skip_final_cycles
Cycles discarded at the end; -skipe overrides it.
maxblock, blockskip
The block lengths used for the error estimate. They do not change the rate, but they do change its quoted error – and the plateau of the block-error curve is what makes that error trustworthy.
ngrid, bins
Resolution of the crossing-probability grid and of the histograms. Too coarse a grid visibly steps the matched curve.
A run analysed twice with different windows gives two legitimate answers over two different sets of cycles; the log states which cycles were used, and the report is stamped with the cycle count.
## Is the number trustworthy?
The rate is the most-processed quantity the code produces, so it is the last thing to converge and the first to mislead. Before quoting one:
- Work through user-guide-output-converged – especially whether the running average has flattened and the block-error curve has reached its plateau.
- Check the flux and the crossing probability separately. They fail differently: the flux is usually fine early, so a suspicious rate is nearly always a crossing-probability problem.
- Look at the matched-probability curve for kinks at the interfaces. A kink means neighbouring ensembles disagree where they overlap, and the product across that seam is not to be trusted.
- If in doubt, run method = "both" and compare the two estimators.
PyRETIS ships a validation suite (examples/validation/) that runs this whole chain on a one-dimensional double well whose rate is known analytically from Kramers’ theory, and checks that the sampled rate lands on it. It is the reference for “does the machinery still give the right answer” – see that directory’s README.rst.
---
# The Analysis section
The analysis section defines parameters for the analysis.
```toml
[analysis]
method = "crossing"
skip_initial_cycles = 0
maxblock = 1000
blockskip = 1
bins = 100
ngrid = 1001
report = [
"latex",
"rst",
"html",
]
maxordermsd = 100
report-dir = "report"
txt-output = "txt.gz"
skipcross = 1000
[analysis.plot]
plotter = "mpl"
output = "png"
style = "pyretis"
```
In a TOML input file the same settings are written as a [analysis] table, for example:
```toml
[analysis]
method = "wham"
skip_initial_cycles = 100
ngrid = 1001
bins = 100
```
## Choosing the rate-estimation method
PyRETIS can estimate the crossing probability and the rate constant with two complementary methods, selected with the method keyword. Both read the sampled paths from the same simulation; they differ in how those paths are combined:
- matched (the default; crossing is accepted as the historical spelling) – the matched / point-matching analysis. It reads the per-ensemble pathensemble.txt files and matches the per-ensemble crossing-probability curves at the interfaces. This is the classic PyRETIS analysis and is available for every path-sampling task (tis, retis, pptis, repptis); the PPTIS/REPPTIS memory-kernel analyses build on its per-ensemble local probabilities.
- wham – the weighted-histogram (WHAM) analysis. It reweights the shared paths across all ensembles in a single histogram, built from the same per-ensemble output every scheduler run writes (a literal infswap_data.txt, if one is present – from an older run, or from merging several runs’ path data – is read directly instead). Use it when you have an infinite-swapping run.
- both – run the two analyses and report each. When a run’s data is unusable for one method, that method is run and the other is skipped with a warning (method = "wham" instead fails loudly if no usable path data is found).
The two methods are different estimators of the same physical rate, so on a converged simulation they agree within their statistical error. Comparing them with method = "both" is a useful cross-check.
Regardless of the method, the skip_initial_cycles and skip_final_cycles keywords discard the same cycles at the start and at the end of the sampling before the statistics are accumulated, so the two estimates are computed over the same production window. On the command line they are pyretis analyse -skipb N -skipe M; given neither, everything is kept.
## Keywords for the Analysis section
For the analysis section the following keywords can be set:
## Supported keywords for the analysis section.
Keyword
Description
method
Select the rate-estimation method.
skip_initial_cycles
Initial cycles to skip (both methods).
skip_final_cycles
Trailing cycles to skip (both methods).
bins
Set the number of bins to use when creating histograms.
blockskip
Selects certain block lengths to skip in the in the block error analysis
maxblock
Set maximum length of blocks to consider in the block error analysis.
maxordermsd
Set the maximum number of time origins to consider when calculating a mean square displacement.
ngrid
Number of points used for calculating the crossing probability.
plot
Settings related to plotting of results.
report
Define output format(s) for the report.
report-dir
Defines the directory where the analysis results should be written.
skipcross
Set time window for initial flux calculation.
txt-output
Select format for text-based output.
skip
Initial cycles to skip.
tau_ref_bin
Set the reference region of the permeability analysis.
### Keyword method
The method keyword selects which rate-estimation method pyretis analyse runs. See Choosing the rate-estimation method for guidance. Valid choices are:
- matched – the matched (point-matching) analysis of the per-ensemble pathensemble.txt files (the classic PyRETIS analysis; crossing is accepted as the historical spelling).
- wham – the weighted-histogram (WHAM) analysis of an infinite-swapping run’s shared-path data (built from the per-ensemble output, or a literal infswap_data.txt if one is present).
- both – run both analyses and report each.
The default value is: method = matched
### Keyword skip_initial_cycles
The skip_initial_cycles keyword sets the number of initial (equilibration) cycles to discard before the statistics are accumulated. It applies to both the crossing and the wham methods, so the two estimates are computed over the same production window. When it is left at 0, the legacy skip keyword (if set) is used instead.
The default value is: skip_initial_cycles = 0
pyretis analyse -skipb N sets this on the command line, overriding the input file. That is usually what you want: how much equilibration to drop is a question you answer after looking at the run, without editing (and re-saving) the input.
### Keyword skip_final_cycles
The counterpart of skip_initial_cycles: the number of cycles to discard at the end of the sampling. This is what you reach for when a run was stopped part-way – a wall-time kill, a crashed node – and its final cycles are not to be trusted, or when you want to compare two runs over the same number of cycles.
pyretis analyse -skipe N sets it on the command line.
Nothing is discarded.
### Keyword bins
The bins keyword defines the number of bins to use when histograms are created.
The default value is: bins = 100
### Keyword blockskip
The blockskip keyword can be used to skip certain block lengths in the block error analysis. That is:
- blockskip = 1 will consider all blocks up to the value given in the keyword maxblock
- blockskip = n will consider every n’th block up to the value given in the keyword maxblock, That is, it will use block lengths equal to 1, 1 + n, 1 + 2*n and so on.
The default value is: blockskip = 1
### Keyword maxblock
The maxblock keyword defines the maximum length of the blocks to consider for the block error analysis. If a negative number is set, PyRETIS will set maxblock equal to (1/2) * length of the data we are analysing. This is also the maximum length that PyRETIS will consider, even if the input maxblock is set higher.
The default value is: maxblock = 1000
### Keyword maxordermsd
The maxordermsd keyword defines the maximum number of time origins to consider when calculating a mean square displacement. If this is set to a value < 0, PyRETIS will internally set maxordermsd to (1/5) * length of the data for which the mean square displacement will be obtained.
The default value is: maxordermsd = -1
### Keyword ngrid
The ngrid keyword defines the number of points used for the order parameter when PyRETIS is calculating the crossing probability as a function of the order parameter.
The default value is: ngrid = 1001
### Keyword plot
The plot keyword defines how plotting should be carried out. It is given as a dictionary, where the following keys can be used:
- 'plotter': Which selects the kind of plotter to use. Currently, PyRETIS only implements a plotter using matplotlib which is selected by setting plot = {'plotter': 'mpl'}.
- 'output': Which selects the output format for plots. This can be set to any of the supported output formats by the plotter. Often this will depend on your specific system, but common choices are 'png', 'pdf' and 'svg'.
- 'style': Selects a style for creating plots. Here you can, for instance, create your own matplotlib style or use one of the built-in styles. Please see http://matplotlib.org/users/style_sheets.html for more information.
The default value is: plot = {'plotter': 'mpl', 'style': 'pyretis', 'output': 'png'}
### Keyword report
The report keyword defines the output format for the report. This is provided as a list of strings, where each item in the list correspond to a specific format:
- 'latex' selects latex output.
- 'rst' selects output in reStructuredText.
- 'html' selects html output.
The default value is: report = ['latex', 'rst', 'html']
### Keyword report-dir
The report-dir keyword defines where the analysis results should be written.
The default value is a sub-folder named report within the directory where the analysis program was executed.
### Keyword skipcross
The length, in simulation steps, of the time window the initial flux is averaged over in an md-flux analysis.
The name is historical and misleading: nothing is skipped. The keyword is a window length.
The flux is the number of effective crossings of an interface per unit time. Measuring it once over the whole run would give a single number with no way to put an error on it, so the time the system spends in the overall A state is instead cut into windows of skipcross steps, and the flux is computed in each. That series is what the running-average plot and the block-error estimate are built from.
It does not change the flux itself. The mean over the windows is the total crossing count divided by the total time however the windows are drawn; only the spread of the series changes, and with it the error estimate:
skipcross
windows
mean flux
spread
500
200
0.50000
0.313
1000
100
0.50000
0.215
5000
20
0.50000
0.067
20000
5
0.50000
0.039
(the same 500 synthetic crossings over 100000 steps, binned four ways.)
So choose it for the statistics, not for the answer. Too small and each window holds a handful of crossings, so the series is noisy; too large and there are too few windows for a meaningful block analysis. A useful target is enough windows to block over – a few tens – with several crossings in each.
If the run does not contain even one full window, the analysis stops with an explicit error rather than producing an empty flux array: reduce skipcross or run longer.
This keyword affects only the md-flux route. A TIS/RETIS run takes its flux from the [0^-] and [0^+] path lengths instead (see user-guide-rate-in-code), and ignores it.
The default value is: skipcross = 1000
### Keyword txt-output
The txt-output keyword defines the format for how text-based output should be written. Valid choices are:
- txt for writing plain text files.
- txt.gz for creating gzipped files.
The default value is: txt-output = txt.gz
### Keyword skip
The number of initial cycles left out of the analysis. skip_initial_cycles takes precedence over this keyword when both are set.
The default value is 0.
### Keyword tau_ref_bin
The order-parameter bin the permeability analysis takes as its reference region when it computes \tau, given as the two edges of the bin. See examples-permeability for a worked example.
The default value is [].
---
# The Box section
The box section defines the simulation box. This is useful, for instance, if periodic boundary conditions are needed.
```toml
[box]
cell = [
10,
10,
10,
]
periodic = [
true,
true,
false,
]
```
## Keywords for the Box section
For the Box section the following keywords can be set:
## Keywords for the Box section
Keyword
Description
cell
Which defines the cell parameters of the simulation box.
low
Which defines the lower boundaries of the box.
high
Which defines the upper boundaries of the box.
periodic
Which determines if the simulation box has periodic boundaries.
### Keyword cell
The cell keyword defines the length of the simulation box in each dimension. This is given by a list of numbers:
```toml
[box]
cell = "[10, 10, float('inf')]"
```
This will define a simulation box with limits starting from 0 and up to the specified length. In case you want to explicitly define the boundaries you can do this by making use of the keywords low and high described below. Note that if more than three values are given, then the created box will be triclinic. The box is then represented by a matrix:
- If 6 values are given:
- If 9 values are given:
If the cell is not given in the box section, then the cell will be set to infinite in all directions. However, if present, the cell found in the input configuration will be used.
### Keyword low
Determines the lower boundaries for the simulation box.
```toml
[box]
low = [
0,
-1,
0,
]
```
The default setting is 0 for each dimension.
### Keyword high
Determines the upper boundaries for the simulation box.
```toml
[box]
high = [
10,
-10,
100,
]
```
The default setting is low + cell.
### Keyword periodic
Determines if the boundaries of the simulation box are periodic or not. This is defined by giving a list of boolean values:
```toml
[box]
periodic = [
true,
true,
true,
]
```
or for a 2D case:
```toml
[box]
periodic = [
true,
false,
]
```
The default setting is periodic (True) for all directions.
---
# The Engine section
The engine section specifies the engine to use for the dynamics.
```toml
[engine]
class = "VelocityVerlet"
timestep = 0.002
```
In PyRETIS, the engines are used as numerical integrators for Newton’s equations of motion. The different engines need different settings and this is specified in detail below for the different types.
A path sampling input may give further engine sections, each with a class and the settings of its engine as below: [engine0], the engine of the [0^-] ensemble of a quantis run, and the numbered sections [engine1], [engine2], … of an engine pool, which [simulation] ensemble_engines assigns to the ensembles. The engine of such a section runs with the settings of that section only: an internal integrator takes its timestep and subcycles from it. Every engine section of a run, [engine] and the [engine0] of a quantis run included, has the same time per step, timestep times subcycles: under infinite swapping a path held by one ensemble can have been generated by the engine of another ensemble, a zero swap builds a path from segments of two engines, and the flux adds the lengths of the [0^-] and [0^+] paths, so the path lengths, counted in steps, convert to time with one time per step only. The engines may differ in every other setting. A run whose engine sections have different times per step is refused before it starts (see ensemble_engines). An [engine0] section in a run in which no ensemble runs it is refused, and so is an rgen of [engine0] or of a numbered engine section other than "pcg64": the scheduler hands the engine of each move a stream of numpy’s PCG64 generator spawned from [simulation] seed. The rgen of [engine] is the generator of the engine of the in-process kick initiation, and a run that kicks no path in process refuses a value other than "pcg64" (see the keywords that take no effect).
An internal integrator (Verlet, Velocity Verlet, Langevin) stores a frame of a path every subcycles integration steps of its section (1 when the section sets none), in a path the scheduler streams and in a path the in-process kick initiation generates with the engine of [engine], so the initial paths of the kick have the time per frame of the paths of the run. An md, md-nve or md-flux run stores a step after each integration step of an internal integrator, so such a run refuses an [engine] subcycles other than 1 for it (see the keywords that take no effect).
The types of engine are:
# Supported engines in PyRETIS
Engine
Description
Verlet
Internal engine, integrate using the Verlet scheme.
Langevin
Internal engine, stochastic dynamics.
Velocity Verlet
Internal engine, integration using the Velocity verlet scheme.
GROMACS
External engine, using GROMACS.
CP2K
External engine, using CP2K.
LAMMPS
External engine, using LAMMPS.
AMS
External engine, using AMS (proprietary; see note).
User-defined
Internal/External engine, user-defined.
## The Verlet engine
The Verlet engine integrates the equations of motion according to the Verlet scheme. NB: Only MD is allowed for this scheme. The engine is selected by specifying the class Verlet and the time step:
```toml
[engine]
class = "Verlet"
timestep = 0.002
```
### Keywords for the Verlet engine
For the Verlet engine, the following keywords can be set:
### Keywords for the Verlet engine.
Keyword
Description
class
Selects the Verlet engine.
timestep
Defines the time step.
#### Keyword class
The class selects the Verlet engine and it should be set to Verlet.
#### Keyword timestep
The timestep keyword defines the time step for the engine in internal units.
Not any. This keyword must be specified.
## The Velocity Verlet engine
The Velocity Verlet engine integrates the equations of motion according to the Velocity Verlet scheme. The engine is selected by specifying the class VelocityVerlet and the time step:
```toml
[engine]
class = "VelocityVerlet"
timestep = 0.002
```
### Keywords for the Velocity Verlet engine
For the Velocity Verlet engine, the following keywords can be set:
### Keywords for the Velocity Verlet engine.
Keyword
Description
class
Selects the Velocity Verlet engine.
timestep
Defines the time step.
#### Keyword class
The class selects the Velocity Verlet engine and it should be set to VelocityVerlet.
#### Keyword timestep
The timestep keyword defines the time step for the engine in internal units.
Not any. This keyword must be specified.
## The Langevin engine
The engine is selected by specifying the class Langevin. This is a stochastic (Brownian) integrator and a description of the implementation can be found in e.g. 1. The integrator is fully specified as follows:
```toml
[engine]
class = "Langevin"
timestep = 0.002
gamma = 0.3
seed = 0
high_friction = false
```
### Keywords for the Langevin engine
For the Langevin engine, the following keywords can be set:
### Keywords for the Langevin engine.
Keyword
Description
class
Selects the Langevin engine.
timestep
Which defines the time step.
gamma
Which defines the the friction parameter.
seed
Which defines a seed to use for the random number generator.
high_friction
Which determines if we are in the high friction limit or not.
#### Keyword class
The class selects the Langevin engine and it should be set to Langevin.
#### Keyword timestep
The timestep keyword defines the time step for the engine in internal units.
Not any. This keyword must be specified.
#### Keyword gamma
This is the gamma parameter (\gamma, see the integration scheme) for the Langevin integrator.
Not any. This keyword must be specified.
#### Keyword seed
The seed value is an integer that is used to seed the random number generator used for the stochastic integration.
The default seed is zero: seed = 0
#### Keyword high_friction
This keyword determines how the equations of motion are integrated. This is a boolean value and thus it can be set to True or False:
- If high_friction is True, we are in the high friction limit and the equations of motion are integrated according to
- where \mathbf{f}(t) is the deterministic force and the displacements (\delta \mathbf{r}) are drawn from a normal distribution,
- If high_friction is False, we are in the low friction limit and the equations of motion are integrated according to
- where,
- and c_0 = \text{e}^{-\gamma \Delta t}, c_1 = (1 - c_0) / ( \gamma \Delta t) and c_2 = (1 - c_1) / ( \gamma \Delta t). In this case, \delta \mathbf{r} and \delta \mathbf{v} are obtained as stochastic variables.
The default setting is high_friction = False
## The GROMACS engine
The GROMACS engine will use GROMACS 2 in order to integrate the equations of motion. The engine is selected by specifying the class gromacs with some additional keywords:
```toml
[engine]
class = "gromacs"
gmx = "gmx"
mdrun = "gmx mdrun"
input_path = "gromacs_input"
timestep = 0.002
subcycles = 3
maxwarn = 0
write_vel = true
write_force = false
gmx_format = "g96"
```
Engine naming: the bare gromacs engine is the streaming engine – it reads the GROMACS output on the fly and runs faster than relaunching GROMACS for every order-parameter point. The slower relaunch-per-step engine is selected as gromacs_steps. The former name gromacs2 is a deprecated alias of gromacs (it still resolves to the streaming engine but emits a discontinuation warning). LAMMPS follows the same convention: lammps is the streaming engine and lammps_steps the relaunch engine, which launches LAMMPS once per propagation and reads its trajectory when the LAMMPS run has ended.
### Keywords for the GROMACS engine
For the GROMACS engine, the following keywords can be set:
### Keywords for the GROMACS engine.
Keyword
Description
class
Selects the GROMACS engine.
gmx
Defines the command used to execute the GROMACS gmx program.
mdrun
Defines the command used to execute the GROMACS mdrun program.
input_path
Defines the folder containing the input to GROMACS.
timestep
Which defines the time step.
subcycles
Which defines the number of steps GROMACS will execute before we calculate order parameters, i.e. the frequency of output from GROMACS.
maxwarn
Which sets the -maxwarn option of GROMACS grompp.
write_vel
Determines if velocities should be written by GROMACS or not.
write_force
Determines if forces should be written by GROMACS or not.
gmx_format
Can be used to select the format used for GROMACS configurations.
domain_decomp
Can be toggled on if domain decomposition is used in GROMACS.
velocity_generation
Selects how shooting velocities are drawn (the engine’s own generator or a Python Maxwell draw).
#### Keyword class
The class selects the GROMACS engine and it should be set to gromacs.
gromacs is the streaming engine: it launches mdrun once per propagated path segment and reads the trajectory frames on the fly, stopping the run as soon as the path leaves the ensemble. This is the fast, recommended choice. The historical alias gromacs2 selects the same engine and is kept for backward compatibility.
For the slower relaunch-per-step engine — a fresh mdrun launch for every order-parameter point — set class = gromacs_steps. The two engines integrate the same dynamics (see examples/tests/test-gromacs/test-integrate/); prefer gromacs unless you specifically need the per-step relaunch behaviour. (Before this release the bare name gromacs selected the relaunch engine; it now selects the streaming engine — update old inputs to gromacs_steps if they relied on the previous behaviour.)
#### Keyword gmx
This keyword sets the command PyRETIS will use for executing the GROMACS gmx program. This can be used if you have different versions of GROMACS installed, for instance, a single precision build named gmx and a double precision build named gmx_d.
Not any. This keyword must be specified.
#### Keyword mdrun
This keyword sets the command PyRETIS will use for executing the GROMACS mdrun program. This can be used to execute an MPI compiled version of mdrun, by, for instance, setting: mdrun = mpiexec_mpt mdrun_mpi. Note that this command is specific to the system you are using.
Not any. This keyword must be specified.
#### Keyword input_path
This keyword sets the directory where PyRETIS will look for input files to use with GROMACS. If, for instance, input_path = gromacs-input is set, then PyRETIS will look in the folder gromacs-input relative to the directory PyRETIS is executed in.
Further, in the folder specified by input_path the following files must be present:
- conf.g96: The initial configuration.
- grompp.mdp: The simulation settings to use. Note that PyRETIS will alter the frequency of output and time step to match the setting given in the input to PyRETIS using the timestep and subcycles keywords. Effectively, PyRETIS will use grompp.mdp as a template and create a pyretis.mdp, which is used for the GROMACS simulations.
- topol.top: The topology for your system.
PyRETIS only reads this folder. The files it derives from the input – pyretis.mdp, the .tpr built from it and, when only one of conf.g96 and conf.gro is staged, its conversion to the other – are written to a directory pyretis-gromacs-* of the engine’s own in the directory PyRETIS is executed in, which is removed when the run ends.
Not any. This keyword must be specified.
#### Keyword timestep
The timestep keyword defines the time step for the engine in internal units, which in this case will be gromacs units.
Not any. This keyword must be specified.
#### Keyword subcycles
The subcycles keyword defines the frequency of output from GROMACS and thus the frequency by which the order parameter(s) are obtained.
Not any. This keyword must be specified.
#### Keyword maxwarn
The maxwarn keyword set the maximum number of warnings the GROMACS grompp command will ignore.
The default value is maxwarn = 0.
#### Keyword write_vel
The write_vel keyword determines if GROMACS should write velocities when integrating the equations of motion. This can safely be set to False if your order parameter does not depend on the velocities.
The default value is write_vel = True.
#### Keyword write_force
The write_force keyword determines if GROMACS should write forces when integrating the equations of motion. Typically, the forces are not needed unless your order parameter depends on them.
The default value is write_force = False.
#### Keyword gmx_format
The gmx_format keyword specifies the format PyRETIS writes for GROMACS configurations. It can be set to gro or g96, and it is optional.
Both formats are always read: the engine takes the format from the file it is given, so a loaded path, a staged initial configuration or a frame from an external tool may be in either, whatever this keyword says.
g96 is the default because it is the one that does not lose precision. A gro file stores three decimals (1e-3 nm) against g96’s nine, and that truncation is enough to change a cutoff- or count-based order parameter – so a configuration written to gro and read back can report a different order parameter than the frame it came from. External compatibility is not lost: a .gro copy is regenerated from the staged g96 for tools that need one.
The default value is gmx_format = g96.
#### Keyword domain_decomp
The domain_decomp keyword will execute gmx trjconv -pbc whole ... to a frame from which the system is propagated. This was implemented as some GROMACS versions did not handle bonds over periodic boundaries correctly.
The default value is domain_decomp = False.
#### Keyword velocity_generation
The velocity_generation keyword selects how the shooting velocities are drawn for a shooting move. With engine (the default) the velocities are produced by GROMACS itself, with a zero-step gen_vel run; gen_vel removes the centre-of-mass motion, so this mode requires zero_momentum = True. With maxwell PyRETIS draws them from a Maxwell-Boltzmann distribution (the draw_maxwellian_velocities helper, sharing the same reproducible random stream as the rest of the move), and the momentum is reset as zero_momentum selects. maxwell additionally requires a temperature and the particle masses, and the g96 configuration format. Neither mode re-scales the energy of the drawn velocities, so a positive rescale_energy stops the run with an error before the draw.
The default value is velocity_generation = engine.
## The CP2K engine
The CP2K engine will use CP2K 3 in order to integrate the equations of motion. The engine is selected by specifying the class cp2k with some additional keywords:
```toml
[engine]
class = "cp2k"
cp2k = "cp2k"
input_path = "cp2k-input"
timestep = 0.002
subcycles = 5
```
Engine naming: as for GROMACS and LAMMPS, the bare cp2k engine is now the continuous engine – it runs CP2K as a single persistent process for a whole propagation segment and reads its trajectory on the fly (warm-starting the wavefunction from a .wfn restart), which is faster than relaunching CP2K for every order-parameter point. The wavefunctions are kept in the directory pyretis-cp2k-wfn of the run directory, shared by the initiation, the workers and a restart of the run; the input_path folder is only read. When the run directory holds a pyretis-cp2k-wfn of an earlier run, a new run stops, and names the directory to remove: pyretis run before its initiation, a run of a legacy-runner input before it starts, the interface optimizer before its first step when a step directory of the run holds one, and a scheduler run of initial paths staged flat in /, or in a directory whose ensemble directories hold a pathensemble.txt, before its load. A scheduler run started from the Python interface, with no path staged flat and no pathensemble.txt in its ensemble directories, may follow an initiation run from Python in the same directory, and starts from the store it finds there (see load_dir). The slower relaunch-per-step engine, which chains the run through previous.restart / previous.wfn files, is selected as cp2k_steps.
### Keywords for the CP2K engine
For the CP2K engine, the following keywords can be set:
### Keywords for the CP2K engine.
Keyword
Description
class
Selects the CP2K engine.
cp2k
Defines the command used to execute CP2K.
input_path
Defines the location of input files to use for CP2K.
timestep
Which defines the time step.
subcycles
Which defines the number of steps CP2K will execute before we calculate order parameter(s).
#### Keyword class
The class selects the CP2K engine and it should be set to cp2k.
#### Keyword cp2k
The command used for executing CP2K.
Not any. This keyword must be specified.
#### Keyword input_path
This keyword sets the directory where PyRETIS will look for input files to use with CP2K. If for instance input_path = cp2k-input is set, then PyRETIS will look in the folder cp2k-input relative to the directory PyRETIS is executed in.
Further, in the folder specified by input_path the following files must be present:
- initial.xyz: A file with the initial configuration.
- cp2k.inp: A CP2K input file, defining a MD simulation.
Not any. This keyword must be specified.
#### Keyword timestep
The timestep keyword defines the time step for the engine in internal units.
Not any. This keyword must be specified.
#### Keyword subcycles
The subcycles defines the frequency of the output from CP2K and thus the frequency by which the order parameter(s) are obtained.
Not any. This keyword must be specified.
## The LAMMPS engine
The LAMMPS engine will use LAMMPS 4 in order to integrate the equations of motion. A description of how to use the LAMMPS engine with PyRETIS can be found in the user guide on LAMMPS.
The engine is selected by specifying the class lammps with some additional keywords:
```toml
[engine]
class = "lammps"
lmp = "lmp"
input_path = "lammps_input"
subcycles = 1
temperature = 300.0
extra_files = [
"potential.in",
]
```
### Keywords for the LAMMPS engine
For the LAMMPS engine, the following keywords can be set:
### Keywords for the LAMMPS engine.
Keyword
Description
class
Selects the LAMMPS engine.
lmp
Defines the command used to execute LAMMPS.
input_path
Defines the location of input files to use for LAMMPS.
subcycles
Which defines the number of steps LAMMPS will execute before we calculate order parameter(s).
extra_files
Defines the location of input files to use for LAMMPS.
order_mode
Selects whether order parameters are evaluated by LAMMPS or by PyRETIS.
velocity_generation
Selects how shooting velocities are drawn.
temperature
Sets the temperature of the velocities PyRETIS draws.
atom_style
Defines how atom rows in a LAMMPS data file are interpreted by PyRETIS.
#### Keyword class
The class selects the LAMMPS engine and it should be set to lammps or lammps_steps.
class = lammps selects the streaming engine (pyretis.engines.lammps.LAMMPSEngine), analogous to the GROMACS gromacs engine: a propagation runs one LAMMPS process, PyRETIS reads its dump trajectory as LAMMPS writes it and stops the process once the path is complete. class = lammps_steps selects the relaunch engine (pyretis.engines.lammps_steps.LAMMPSEngineSteps), which reads the trajectory of a propagation once the LAMMPS run has ended. The two engines draw the shooting velocities differently; see velocity_generation.
#### Keyword lmp
The command used for executing LAMMPS.
Not any. This keyword must be specified.
#### Keyword input_path
This keyword sets the directory where PyRETIS will look for input files to use with LAMMPS. If for instance input_path = lammps_input is set, then PyRETIS will look in the folder lammps_input relative to the directory PyRETIS is executed in.
Further, in the folder specified by input_path the following files must be present:
- system.data: A LAMMPS data file with the initial configuration. class = lammps reads its Atoms and Masses sections when it is set up, and its box when it draws the velocities, or evaluates the order parameter, of the configuration held in it, as atom_style describes.
- lammps.in: A LAMMPS input file, defining a MD simulation.
- order.in: A LAMMPS input file, defining the order parameter calculation. This file is required when order_mode = lammps and optional when order_mode = pyretis.
Not any. This keyword must be specified.
#### Keyword subcycles
The subcycles defines the frequency of the output from LAMMPS, and thus the frequency by which the order parameter(s) are obtained.
Not any. This keyword must be specified.
#### Keyword extra
The extra_files defines additional files that might be needed to execute LAMMPS. For instance, may the user refer to several additional LAMMPS script in the given lammps.in file. This keyword is to make PyRETIS aware of these additional files.
None.
#### Keyword order_mode
The order_mode keyword selects how order parameters and collective variables are evaluated for the LAMMPS engine.
With order_mode = lammps, LAMMPS variables defined in order.in are printed by LAMMPS, and v_op_1 is used by LAMMPS halt fixes to stop path propagation at the interfaces.
order_mode = pyretis uses LAMMPS only for propagation. PyRETIS reads the dumped LAMMPS trajectory frames and evaluates the normal orderparameter and collective-variable sections in Python. In this mode, LAMMPS cannot halt directly on v_op_1; PyRETIS truncates the generated path after evaluating the order parameter.
pyretis for class = lammps and lammps for class = lammps_steps.
#### Keyword velocity_generation
The velocity_generation keyword selects how the shooting velocities are drawn for a shooting move.
With class = lammps_steps and engine (the default), LAMMPS produces the velocities itself, with velocity create, and PyRETIS writes its mom keyword from zero_momentum: mom yes for zero_momentum = True and mom no for zero_momentum = False. With maxwell, PyRETIS draws them from a Maxwell-Boltzmann distribution (the draw_maxwellian_velocities helper, sharing the same reproducible random stream as the rest of the move) and resets the momentum as zero_momentum selects. With engine, a positive rescale_energy makes LAMMPS re-scale the created velocities so that pe + ke, as LAMMPS evaluates them, equals rescale_energy (per atom under thermo_modify norm yes, the LAMMPS default for lj units), and keeps them, with a WARNING, for a pe above rescale_energy; the maxwell draw does not re-scale, and a positive rescale_energy stops the run with an error before the draw.
With class = lammps, PyRETIS draws the velocities in both modes, from a Maxwell-Boltzmann distribution at temperature on the same random stream, and resets the momentum as zero_momentum selects, so the two values draw the same velocities. The velocities of a configuration held in a LAMMPS data file, as the system.data a run starts from, are drawn as class = lammps_steps with maxwell draws them, with the positions, image flags, masses and box of the data file (see atom_style). For a 3D system, the positions, velocities and kinetic energy of that draw are those of the draw of a dump frame of the same configuration; the frame that the draw of a dump frame writes has the image flags 0 0 0. The draws leave the energy as drawn: a positive rescale_energy stops the run with an error before the draw.
The draws of PyRETIS, those of class = lammps and of class = lammps_steps with maxwell, use the constants of LAMMPS real units. They require a temperature and the units real command in lammps.in:
- Without a temperature, maxwell stops the run with an error when the engine is set up, and class = lammps with engine at its first draw.
- A units command of lammps.in that sets other units stops the run with an error at the first draw. Without [system] units, PyRETIS takes the units of the run from that command, and the LAMMPS units that PyRETIS does not define, cgs, nano and micro, stop the run earlier, when the ensembles are set up.
- PyRETIS reads the units command of lammps.in itself, not of a file it includes, so units real goes in lammps.in. LAMMPS runs a lammps.in without a units command in lj units, unless a file it includes sets the units. Without [system] units, PyRETIS takes the units of the run from the units command of lammps.in, and a lammps.in without one stops the run with an error when the settings are read; with [system] units set, it stops the run with an error at the first draw.
The default value is velocity_generation = engine.
#### Keyword temperature
The temperature keyword sets the temperature, in K, of the Maxwell-Boltzmann distribution that PyRETIS draws the shooting velocities from. Every velocity draw of class = lammps requires it. Without it, velocity_generation = maxwell stops the run with an error when the engine is set up, for both engines, and class = lammps with velocity_generation = engine at its first draw. With class = lammps_steps and velocity_generation = engine, LAMMPS draws the velocities at the SET_TEMP variable of lammps.in (see the user guide on LAMMPS).
Not set.
#### Keyword atom_style
The atom_style keyword tells PyRETIS how columns in the LAMMPS data file’s Atoms section are laid out. With it, PyRETIS reads the atom types of the data file, and so the masses, and the positions of a configuration held in a data file. The rows are read in the style of the first of these that is given:
- the atom_style keyword;
- the # style annotation of the Atoms header, as in Atoms # full, which a data file written by LAMMPS carries;
- the atom_style command of lammps.in;
- atomic, the style LAMMPS uses when its input sets none.
PyRETIS reads the atom_style command of lammps.in itself, not of a file it includes. An atom_style command whose style is a LAMMPS variable, as atom_style ${style}, is not a source, since PyRETIS does not evaluate LAMMPS variables: the rows are read in the style of the atom_style keyword or of the annotation, and without either of them the run stops with an error that asks for the atom_style keyword. The supported styles, and the columns of their rows as LAMMPS read_data reads them, are:
- atomic: atom-ID atom-type x y z;
- charge: atom-ID atom-type q x y z;
- molecular, bond and angle: atom-ID molecule-ID atom-type x y z;
- full: atom-ID molecule-ID atom-type q x y z.
Each row can end with the three image flags. The styles molecular, bond and angle have the same rows, so any two of them agree. A style with the suffix of an accelerator package, as full/kk or full/kk/device of KOKKOS, has the rows of the style without the suffix and agrees with it; write_data of a LAMMPS run with -sf kk can write such a style in the Atoms annotation.
PyRETIS reads a data file as LAMMPS read_data lays it out. The header is the lines below the title line up to the first line that does not start with a number. It gives N atoms, N atom types and the box bounds, a line lo hi xlo xhi, lo hi ylo yhi or lo hi zlo zhi for each axis, in any order of the lines; an axis whose line the header leaves out, as the z axis of a 2D system often is, has the bounds -0.5 0.5, the default of LAMMPS. A header line is read, as LAMMPS reads it, by the keywords at their positions in the line, and the words after them are ignored, so 0 10 xlo xhi extra gives x the bounds 0 10; the other lines of the header, as N bonds, are not read. The rows of the Masses and Atoms sections start at the second line below the keyword line of the section, whatever the line between them holds, one row for each atom type or atom of the header, and the keyword line of the next section can follow the last row with no blank line between them. LAMMPS reads a keyword line together with the line below it, and a keyword line that is the last line of the file ends the read, so PyRETIS does not read that line either. PyRETIS takes the masses of the atoms from the Masses section. These stop the run with an error:
- a style that is not supported, as sphere or hybrid, from any of the three sources;
- two of the three sources that are given and name styles with different rows, as an Atoms # atomic annotation and atom_style full in lammps.in; the error names both;
- a header line that has a keyword of these lines and is neither one of them nor another header line of LAMMPS, such as 0 40 zlo zhj or 0 10 xhi xlo, which LAMMPS stops at with Unknown identifier in data file; a number of atoms or atom types that is not written in digits; a bound, or a number of a line of a triclinic box, that is not a number; a hi bound that is not above its lo bound; the error names the file and the line;
- a blank line or a comment among the rows of a Masses or Atoms section, a section that ends with the file before its last row, and a line below the rows that starts with a number, as a row beyond the N atoms or N atom types of the header does: LAMMPS reads that line as the keyword line of a section and stops at it, unless it is the last line of the file, which LAMMPS leaves out; the error names the file and the line;
- an Atoms row whose number of columns fits the style neither without nor with the three image flags (7 or 10 columns for full), or is not the number of columns of the first row, as LAMMPS read_data requires, so either every row ends with the image flags or none does; an Atoms row whose atom ID, atom type or image flags are not integers, as the atom type 1.0, whose atom ID is not positive, or that repeats the atom ID of an earlier row; the error names the file and the row; a second Atoms section;
- a Masses row that is not an integer atom type and a positive mass, or whose atom type is not one of the N atom types of the header, numbered from 1; a Masses section that gives an atom type twice, or not the atom type of an atom; the error names the file;
- with class = lammps, a mass command in lammps.in, which sets masses in LAMMPS in place of those of the Masses section; with class = lammps_steps and velocity_generation = maxwell, such a command for the draw of a configuration held in a data file. The error names the command and asks for the masses in the Masses section. PyRETIS reads the mass commands of lammps.in itself, not of a file it includes.
With class = lammps, these errors come when the engine is set up, as it reads the masses from the system.data of input_path, or when it draws the velocities, or evaluates the order parameter, of a configuration held in another data file; the draw reads the data file before it writes any file. class = lammps_steps reads a data file only in the draw of velocity_generation = maxwell, and the errors come at the first draw. That draw reads the data file conf.data of the run directory: for a configuration held in a data file, a copy of that file, and for a frame of a dump, the data file that LAMMPS write_data writes for the frame, whose Masses section holds the masses that LAMMPS has, those of the mass commands of lammps.in among them. A mass command therefore stops the draw of a configuration held in a data file only, before the file is copied, and the errors of the data file come after it is written. With velocity_generation = engine the engine does not read the data file.
The velocity draw of a configuration held in a data file, of both engines, its order parameter, of class = lammps, and every draw of class = lammps_steps with velocity_generation = maxwell read the box of the data file too, as an orthogonal box from its lo and hi bounds. These stop them with an error that names the file and the line:
- a data file of a triclinic box, whose header has an xy xz yz line, even with zero tilt factors, or an avec, bvec, cvec or abc origin line of a general triclinic box, with or without words after its keywords. write_data writes the xy xz yz line for a frame of a dump in a triclinic box, so every draw of class = lammps_steps with velocity_generation = maxwell in a triclinic box stops;
- a header with two lines for the bounds of one axis, of which LAMMPS read_data takes the last.
The set-up of class = lammps reads no box of system.data, so a run from a system.data of a triclinic box sets up, and stops at the first draw or order parameter of a configuration held in a data file.
With class = lammps, a configuration names its data file by a path, absolute or relative to the directory PyRETIS runs in, or by a bare file name. A bare name that is not a file of the directory PyRETIS runs in is the file of that name in the run directory, and, without one, the system.data of input_path when it has its name. A configuration whose data file is none of these stops the run with an error that names the file. The propagation of both engines, which copies a configuration held in a data file to the system.data of the run directory, reads a path in the same way, and a bare name as the file of that name in the run directory.
The rows of charge and those of molecular, bond and angle have the same number of columns, 6 or 9, and the positions in the same columns, so the number of columns cannot tell them apart. Read as molecular rows, charge rows give their charge as the atom type, and only a charge that is not an integer stops the run; read as charge rows, molecular rows give their molecule ID as the atom type. For these rows the style must be right.
When a file-backed initial LAMMPS trajectory contains wrapped coordinates (x/y/z) and unwrapped coordinates (xu/yu/zu), but omits image flags, PyRETIS derives ix/iy/iz while extracting the selected timestep into a canonical one-frame starting dump. The source trajectory remains unchanged and is read only up to the referenced timestep. For triclinic dumps, image flags must currently be provided explicitly.
Not set: the style comes from the data file or lammps.in, as listed above.
## The AMS engine
The AMS engine drives the Amsterdam Modeling Suite (AMS) to integrate the equations of motion. Unlike the other external engines it does not launch a command-line executable per step; it controls a persistent AMS process through the scm.plams AMSWorker Python API, reading the run definition (geometry, time step, thermostat) from an ams.inp in the engine’s input_path.
AMS is proprietary software (SCM – Software for Chemistry & Materials) and its scm.plams Python binding must be licensed and installed separately; PyRETIS cannot install it. A mock-engine test (test/engines/mockams.py with test/engines/test_ams_mock.py) injects fake scm.plams modules so the engine’s construction and MD-step logic run without an AMS installation. The tests of the velocities AMS draws (test/infswap/engines/test_velocity_functions.py, test/engines/test_ams_draw_momentum.py) need a licensed AMS and are skipped without one. There is no numerically-validated AMS reference suite. The avatar above is a generic PyRETIS-drawn molecule, not the SCM/AMS logo.
The engine is selected by specifying the class ams with some additional keywords:
```toml
[engine]
class = "ams"
input_path = "ams_input"
timestep = 0.5
subcycles = 1
```
AMS draws the shooting and kick velocities with its own generator (GenerateVelocities), and PyRETIS uses them as drawn. With RandomVelocitiesMethod = Gromacs and Exact, the AMS default, the draws have zero net linear momentum, so the AMS engine requires zero_momentum = True; zero_momentum = False, the default, stops the run with an error before the draw. The AMS engine draws the velocities of aimless shooting, selected by a negative sigma_v, and leaves their energy as drawn: a zero or positive sigma_v, and a positive rescale_energy, stop the run with an error before the draw.
The SCM manual states the removal of the net momentum for RandomVelocitiesMethod = Gromacs, and draws of Exact, the AMS default from AMS2023.1, have zero total momentum. For Boltzmann, the method of the AMS velocity tests (test/infswap/engines/ams_inp/ams.inp), it has not been checked: the check needs a licensed AMS (AMSBIN set, NSCM=1).
### Keywords for the AMS engine
For the AMS engine, the following keywords can be set:
### Keywords for the AMS engine.
Keyword
Description
class
Selects the AMS engine (class = ams).
input_path
The directory holding ams.inp (and the geometry it references).
timestep
The MD time step; it must match the timestep in ams.inp (a mismatch is rejected).
subcycles
The number of AMS MD steps run before each order- parameter evaluation.
#### Keyword class
Selects the AMS engine; set it to ams.
#### Keyword input_path
The directory that holds the ams.inp AMS input file (and the geometry file it references).
#### Keyword timestep
The MD time step. It must agree with the time step declared in ams.inp; PyRETIS raises an error on a mismatch.
#### Keyword subcycles
The number of AMS MD steps taken between order-parameter evaluations.
## User-defined engines
It is also possible to extend PyRETIS with user-defined engines, written in for instance Python, FORTRAN or C. User-defined engines are specified in Python modules that PyRETIS can load and they are typically specified as follows:
```toml
[engine]
class = "VelocityVerletF"
module = "vvengineF.py"
timestep = 0.002
```
### Keywords for user-defined engines
Typically, the user-defined engines requires that, at least, the following keywords are set:
### Keywords for user-defined engines.
Keyword
Description
class
Selects the engine.
module
Defines the Python file where the engine class is defined.
timestep
Defines the timestep for the engine.
#### Keyword class
This keyword selects the engine and it should be set to the class name as it is defined in the given module.
#### Keyword module
This keyword specified the location of the file containing the user-defined class. This file must be accessible by PyRETIS.
Not any. This keyword must be specified.
#### Keyword timestep
The timestep keyword defines the time step for the engine in internal units.
Not any. This keyword must be specified.
#### Additional keywords
In addition, user-defined keywords can be specified, e.g.:
```toml
[engine]
class = "VelocityVerletF"
module = "vvintegratorf.py"
timestep = 0.002
keyword = 100
otherkeyword = 31
```
## References
M. P. Allen and D. J. Tildesley, Computer Simulation of Liquids, 1989, Oxford University Press.
http://www.gromacs.org
https://www.cp2k.org/
https://lammps.sandia.gov/
---
# The Forcefield section
The forcefield section defines the force field to use for the simulation.
```toml
[forcefield]
description = "Lennard Jones test"
```
The actual potential functions to use are explicitly defined using the potential section.
## Keywords for the Forcefield section
For the Forcefield section the following keywords can be set:
## Keywords for the Forcefield section.
Keyword
Description
description
Text that labels the force field.
### Keyword description
description The description keyword is a text that labels the force field. This is only used when printing out information about the force field.
```toml
[forcefield]
description = "my new force field"
```
The default value for the description is the text: 'Generic force field'
---
# The initial-path section
The initial-path section specifies how the initial path for a TIS/RETIS simulation should be generated.
The whole section is optional: an input that leaves it out generates its initial paths with kick, which needs nothing beyond the starting configuration the input already supplies. The chosen initiation is always written to the log, so a run records how its first paths were made. Give the section when you want load or restart instead, or to tune the kick.
```toml
[initial-path]
method = "kick"
kick-from = "previous"
```
The different methods may require different keywords and this is described below for the following supported methods:
# Supported initiation methods in PyRETIS
Method name
Description
kick
For initiating new trajectories from given initial configuration(s).
load
For loading previously generated frames/trajectories
restart
For restarting PyRETIS simulations.
Choosing a method. Use restart whenever a pyretis.restart file is available — it is the most faithful continuation and requires no extra setup. Use load when you have a molecular dynamics trajectory or a previous run whose restart files are unavailable or whose settings need to change beyond what a restart allows; PyRETIS handles the distribution of frames across ensembles automatically. Use kick when no prior trajectory exists and paths must be generated from scratch.
## The kick method
The kick method generates initial paths for all path ensembles from scratch, requiring only a single starting configuration from the user. It is recommended when no prior trajectory is available.
For each ensemble, the method searches for a crossing of the ensemble’s middle interface \lambda_i starting from a given phase point x:
- Velocities are randomised (e.g. drawn from a Maxwell–Boltzmann distribution).
- The equations of motion are integrated one step forward. This gives a new phase point y.
- If y is closer to \lambda_i than x, the update x \leftarrow y is accepted and the steps are repeated. Otherwise, the steps are repeated from x unchanged.
When a crossing is found, two phase points are available: one immediately before the interface and one immediately after. A full path is then generated by integrating forward from the point after the interface and backward from the point before it.
Efficient ensemble population with kick-from = previous. With kick-from = initial every ensemble is kicked from the same user-provided configuration, which may require many iterations for ensembles whose interface lies far from the starting point. With kick-from = previous (recommended for RETIS) only the first ensemble is kicked from the provided configuration. For each subsequent ensemble PyRETIS automatically selects the frame from the just-generated path that lies closest to the next interface and is still to the left of it, then kicks from that frame. This propagates the starting point progressively through interface space so that the kicking algorithm is always seeded close to the target interface, greatly reducing the number of iterations needed per ensemble and requiring no additional input from the user.
### Keywords for the kick method
For the kick method, the following keywords can be set:
### Keywords for the kick method.
Keyword
Description
method
Selects the kick method.
kick-from
Modify how the initial point for the kick method is picked.
kick-parallel
Kick every ensemble in parallel, one job per worker, instead of sequentially in one process.
#### Keyword method
The method keyword selects the kick initialization.
#### Keyword kick-from
The kick-from keyword specifies what initial point we use for the kick method. This setting is used when several path ensembles are initiated at the same time. It can be set to:
- kick-from = initial: In which we will kick from the initial configuration provided. That is, all path ensembles are initiated from the same starting point.
- kick-from = previous: In which we kick from the point on the previous path which is closest to the next interface and still left of this interface. The first path ensemble will be initiated using the provided initial configuration for the particles.
The default value is kick-from = initial.
#### Keyword kick-parallel
When kick-parallel = True (and kick-from = initial, the default), every ensemble is kicked in parallel — one job per ensemble, dispatched across the simulation’s worker pool — instead of sequentially in a single process. This only changes how the kick search is run, not the search itself or the resulting paths’ validity.
kick-from = previous always seeds each ensemble from the previous ensemble’s just-kicked path, so it is inherently sequential; kick-parallel is ignored when combined with it.
Engine coverage for the kick search itself (independent of kick-parallel): every engine can now generate a kick-initiated path, including ones without a usable single MD step (CP2K, LAMMPS streaming, ASE, TurtleMD) — they search for the middle-interface crossing by advancing one step of propagate() at a time rather than calling step() directly. kick-parallel itself has been validated for TurtleMD; internal engines (Langevin, Verlet, velocity-Verlet, …) still kick correctly, but only on the sequential route, so kick-parallel = True is not yet recommended for them.
The default value is kick-parallel = False.
## The load method
The load method initialises path ensembles from existing trajectory data, avoiding the need to generate paths from scratch via kicking. It is the recommended starting point whenever a prior molecular dynamics trajectory or a previous (P)RETIS run is available.
### How ensembles are populated
PyRETIS maps each path ensemble to a numbered subfolder inside load_folder (000, 001, 002, …, the names of the ensemble directories of the run, see user-guide-output-layout). A pptis run without zero_left or flux numbers only the windows it builds, so it reads its first window, the [0^+] or the [1^+], from 000. The load folder is input and is only read: its files are linked or copied into a directory pyretis-load-* in the directory PyRETIS runs in, the files a load prepares (traj.txt, order.txt and the frames of each ensemble in NNN/accepted/) are written there, and the paths are read from there. The directory is removed when the load ends. A run that stops during the load leaves it in the run directory; remove that pyretis-load-* directory by hand. In pyretis-load-* the .txt files of the load folder are copies, and its other files, the frames, are hard links where the filesystem allows, which take no disk space of their own, and copies elsewhere. A scheduler run that reads initial paths staged flat, in /, keeps a copy of their files that takes as much disk space as the staged files (see load_dir). Messages about the files of the load name them in the load folder, and pyretis tools clean leaves the load folder a TOML input names in place. For each ensemble in turn the following steps are applied:
- Collect frames. Trajectory or configuration files are read from the load folder (see user-section-initial-load-layouts below for the two supported layouts).
- Recompute the order parameter. The order parameter is always recomputed from atomic positions using the order parameter function defined in the simulation input. If an order.txt file is present it is used as-is; if it is absent it is generated. This design means users can change the order parameter definition without regenerating their trajectory files.
- Validate and clean the path. Only frames relevant to the ensemble are kept; the rest are discarded (see user-section-initial-load-cleaning below).
- Fall back to kick if needed. If no valid path can be assembled from the loaded data and load_and_kick = True, PyRETIS automatically kicks from the frame nearest to the ensemble’s middle interface.
### Path validation and cleaning
After collecting frames PyRETIS checks whether the assembled sequence constitutes a formally valid path for the target ensemble. In the [0^-] of a rate calculation with zero_left, a valid path starts and ends at the first interface, so a loaded path that reaches zero_left is recovered like any other. A recovered path is a seed: its interior frames are the shooting points of the first moves, and the analysis leaves it out. If the sequence is not a valid path, the following recovery steps are attempted in order:
- Frames are sorted by order parameter and the validation is repeated.
- If sorting alone is insufficient, the sequence is mirrored to produce an L-to-L or R-to-R path — a valid bookkeeping form that satisfies the ensemble definition.
- Frames outside the ensemble’s relevant region are discarded, keeping:
- the two frames immediately before and after the first interface, and
- all frames between the interface defining the ensemble and the last interface.
- If this second group is empty, the frame with the highest order parameter is retained so that the load_and_kick fallback has a useful seed.
If all three steps fail and load_and_kick = False, PyRETIS raises an error.
### Load folder layouts
Two layouts are supported.
Layout 1 — unformatted (single shared trajectory). Place one or more trajectory or configuration files directly in load_folder (accepted formats: .xyz, .gro, .trr, .g96, plain-text .txt). PyRETIS will, in the directory where it prepares the load:
- distribute the files to each ensemble subfolder,
- generate traj.txt (frame index list) automatically,
- recompute and write order.txt using the order parameter function.
Only the frames relevant to each ensemble survive path cleaning. Use this layout when you have a single long trajectory and want PyRETIS to handle the per-ensemble distribution automatically with minimal user effort.
Layout 2 — formatted (per-ensemble subfolders). Create subfolders 000, 001, 002, … inside load_folder and place the relevant files for each ensemble there. If traj.txt or order.txt are absent they are regenerated automatically.
For internal engines (positions and velocities stored in PyRETIS format):
File
Notes
traj.txt
Required. Full path (positions and velocities).
order.txt
Optional. Recomputed from atomic positions if absent.
energy.txt
Optional. Energies are silently ignored if absent.
For external engines (GROMACS, CP2K, …):
File / folder
Notes
traj.txt
Required. Frame index file referencing MD files.
accepted/
Required. Actual MD trajectory files (.trr, .gro, .xtc, .g96, .xyz, …) referenced in traj.txt.
order.txt
Optional. Recomputed from atomic positions if absent.
energy.txt
Optional. Energies are silently ignored if absent.
Use Layout 2 when different ensembles should start from different trajectories, or when you want to resume from formatted output of a previous (P)RETIS run with changed settings (and no restart file is available).
Order parameters are always recomputed from atomic positions using the order parameter function defined in the simulation input. Placing an order.txt file in the load folder is a performance shortcut, not a requirement. If it is absent or outdated, PyRETIS regenerates it automatically. This allows users to change the order parameter definition without regenerating trajectory files.
### Keywords for the load method
For the load method, the following keywords can be set:
### Keywords for the load method.
Keyword
Description
method
Selects the load method.
load_and_kick
Fall back to kick initialisation when loaded data cannot form a valid path.
load_folder
Directory containing the trajectory data to load.
#### Keyword method
The method keyword selects the load initialization.
#### Keyword load_and_kick
When load_and_kick = True, the loaded trajectory data is used as a pool of candidate frames rather than as a ready-made path. If the loaded frames cannot be assembled into a valid path for the target ensemble — even after sorting and mirroring — PyRETIS:
- Selects the frame from the loaded data that is closest to the ensemble’s middle interface.
- Uses this frame as the starting configuration for a standard kick initialisation (two-way shooting).
- The resulting kicked path fully replaces the loaded frames; no loaded frames are retained in the final path.
This option is particularly useful when the available trajectory does not cross the relevant interface, or when you want a freshly generated, unbiased initial path seeded from a physically relevant region of phase space.
If load_and_kick = False (the default) and no valid path can be assembled from the loaded data, PyRETIS raises an error.
The default value is load_and_kick = False.
#### Keyword load_folder
The load_folder keyword specifies the directory containing trajectory data for load initialisation. See user-section-initial-load-layouts for the two supported layouts (unformatted single trajectory or formatted per-ensemble subfolders).
The default value is load_folder = load.
## The restart method
The restart method is used to continue an existing simulation. It is not a vehicle for changing the simulation itself — for that, use load.
A path-sampling run (tis, retis, explore, pptis, repptis) started with pyretis run keeps its state in the run directory’s output.toml, and a restart continues from there. The settings of the continuation are taken from the new input file; only the running state — the cycle counter, the random-generator state, the occupancy weights and the paths each ensemble holds — comes from output.toml.
output.toml records the input of the run: its sections under the names of the input, with every default the run takes resolved (the zero-swap probability as [retis] swapfreq), and the [current] state. It holds none of the settings the input parse derives (the engine type, the input_files the GROMACS and CP2K checkers find, the exe_path of the run directory, the per-ensemble copies of the sections), and none of the keys and sections that take no effect on the run (the keywords that take no effect): the input of a path-sampling run gives none, as the settings parse refuses each one, and the record leaves out the value the parse gives such a key, such as the [simulation] restart it derives from method = "restart", the [engine0] of a run that runs no [engine0] engine, [simulation] rgen, and the rgen of [tis], [engine] and [system] for a run whose initiation kicks no path in process, a continuation among them. It records [output] screen, the cycles between two progress reports of the scheduler. pyretis run -i output.toml resumes the run with the settings it records, and pyretis analyse -i output.toml analyses it as its input would.
An output.toml written by an earlier PyRETIS can hold keys that took no effect on its run, such as [retis] nullmoves or [output] trajectory-file. The run file still continues and analyses: such a key is left out when the file is read, silently when its value is the one the settings parse gives, and named in a warning otherwise. A value that earlier PyRETIS took as another input key is read under that key: [simulation] zeroswap as [retis] swapfreq, which it won over, and [tis] relative_shoots as [retis] relative_shoots when the file sets none there. pyretis run -i output.toml then writes the record of this version, with the defaults the file does not hold, such as [output] screen.
An output.toml or restart.toml written by an earlier PyRETIS holds the configuration of the scheduler instead, with its [simulation.tis_set] table. A restart continues from it as from the output.toml of this version: the continuation compares its settings with the ones that state file records, reading each where the scheduler of that version read it, and names each setting by its input key. pyretis run -i refuses such a state file, and the message names this route: set method = "restart" in the input of the run and run it in the run directory. pyretis analyse -i reads the settings of such a state file under the names of the input, through the key table, except for the state file of a tis run of one ensemble, whose analysis counts a crossing at [tis] detect when its input gives [tis] ensemble_number, which the state file does not say (see user-guide-rate). The continuation writes output.toml in the form of this version.
kick and load start a new simulation, which writes the output files of its ensemble directories afresh. When one of those directories holds a pathensemble.txt with data rows, the record of cycles a simulation has sampled, pyretis run stops with a ValueError before initiation and leaves the ensemble directories and output.toml as they were. This holds for every value of [output] backup. To add cycles to that simulation, set method = "restart" and raise [simulation] steps to the new total. To start a new simulation, use a new directory, or first clean the run directory with pyretis tools clean or remove the ensemble directories the message names. A scheduler run that starts a new simulation from initial paths already in the run directory, such as a run of a legacy-runner input, stops before its load in the same case, and when a pathensemble.txt holds the header alone, with a FileExistsError that names the ensemble directories. A run of a legacy-runner input stops before it writes anything, with a ValueError, when the run directory holds the state file of a simulation, an output.toml or the legacy restart.toml with a [current] section. pyretis run -i output.toml continues the simulation of an output.toml that records the input of the run; a state file of an earlier PyRETIS is continued by the input converted to the canonical schema (python -m pyretis.tools.convert_legacy_schema) with method = "restart" (see load_dir).
The legacy-runner schema ([runner], [simulation.tis_set], no [simulation] task) is deprecated: PyRETIS 4 converts such an input to its canonical form when it reads it, with a deprecation notice, and PyRETIS 5 will not read it (see the migration guide). The schema has no [initial-path] section: a run of a legacy-runner input starts from the initial paths staged in the run directory, and records its canonical form, every default resolved, in output.toml, as a run of a canonical input does, the pool of ensemble_engines and the engine sections it names among them. pyretis run -i output.toml resumes that run, and the canonical form with method = "restart" continues it. The canonical form takes the initiation of the canonical input, method = "kick", which the record names: a new run of the canonical form generates its initial paths, and stops before it writes a file when the run directory holds initial paths staged flat.
Before continuing, the restart compares the new input with the one that wrote output.toml and refuses, with a ValueError and without touching the saved state, when:
- the number of ensembles differs;
- any interface position differs;
- the ensemble layout differs: the ensembles [simulation] task builds, the [0^-] window that zero_left bounds, a permeability run, for task = "pptis", whether zero_left or flux adds a [0^-] window and zero_ensemble a [0^+] window, and, for task = "pptis" and "repptis", the ensemble windows that [pptis] memory and [repptis] memory set;
- any of these settings differs: [tis] maxlength, [tis] allowmaxlength, [tis] enforce_must_cross_m, [system] temperature, the engine temperature, timestep and subcycles, any setting of the orderparameter section (its class, every setting the class reads, the module it is read from and its name), and the GROMACS exchange format gmx_format;
- the continuation draws the shooting velocities from another distribution (see below).
The refusal names each setting by its input key, e.g. [simulation] zero_left, and says what it fixes and the value of each run.
maxlength and allowmaxlength enter the acceptance probability of the shooting move, and enforce_must_cross_m decides which paths are members of an ensemble, so changing one of them would continue the chain under a different acceptance rule. The ensemble layout decides which ensembles exist and which paths each of them accepts, so the paths the ensembles hold were accepted in the recorded layout. flux and zero_ensemble change the ensembles of a pptis run only, so the restart compares them for that task. The memory sets the window [lambda_{i-memory}, lambda_i, lambda_{i+memory}] of each ensemble of a pptis or repptis run. The time grid and the order parameter fix what the saved paths mean: a particle index, a dimension or a periodicity that differs defines another reaction coordinate, as a different class does.
The temperature sets the distribution the ensembles sample. [system] temperature is compared for every run, and the internal integrators run at it. The engine temperature is compared for the engines that take one: TurtleMD and ASE run at it; GROMACS, CP2K and LAMMPS draw Maxwell velocities at it, and GROMACS also couples at it, through the ref-t it writes into the mdp. Where no engine temperature is given, these three thermostat at the temperature of their own input files, which the resume leaves to be compared by hand.
The engine timestep and subcycles apply to every engine: each engine stores a frame every subcycles integration steps. The gmx_format applies to the GROMACS engines and to a module-provided engine, whose class is imported only when the run starts. A setting that applies to neither run’s engine is skipped. The internal integrators (Langevin, VelocityVerlet, Verlet) and OpenMM run with subcycles = 1 when the input leaves it out, and the restart compares it as 1; the GROMACS, LAMMPS, CP2K, AMS, ASE and TurtleMD engines require it. A setting that applies and cannot be compared is reported as UNKNOWN in a WARNING, with the reason, and the restart proceeds: the previous run did not record it (an archive written before the setting was persisted, or a value its engine took from its own input), the continuation does not record it, neither run records it, or it applies to the engine of one run only. LAMMPS and OpenMM take the time step from their own input (the LAMMPS input file, the OpenMM integrator); it is recorded, and cross-checked by the engine, when [engine] timestep states it as well, and the warning about a continuation that leaves it out says so. For any other setting that neither run records, the warning asks for the two inputs to be compared by hand.
[tis] zero_momentum and [tis] rescale_energy select the distribution of the shooting velocities, and the paths the ensembles hold were shot with the draws of the run that wrote output.toml. The restart compares those draws with the draws of the continuation: whether every draw has zero total linear momentum, and the energy the drawn velocities are re-scaled to. The continuation draws as its input selects, and so does the run that wrote an output.toml whose [current.provenance] holds velocity_draw_rule = 1. An output.toml without that key records the draws of the engine the run used:
- the internal integrators, OpenMM and cp2k_steps keep the linear momentum of every draw;
- GROMACS and lammps_steps with velocity_generation = "engine" (gen_vel, and velocity create with the LAMMPS default mom yes) draw with zero total linear momentum;
- the streaming cp2k engine draws with zero total linear momentum unless the recorded zero_momentum is false;
- AMS draws with zero total linear momentum (GenerateVelocities removes the net linear momentum with RandomVelocitiesMethod Gromacs and Exact, the AMS default; Boltzmann is not checked);
- TurtleMD, ASE, the streaming LAMMPS engines, and GROMACS and lammps_steps with velocity_generation = "maxwell" follow the recorded zero_momentum, a missing key being false;
- no engine of PyRETIS re-scales the energy of a shooting draw;
- for an engine imported from a module, the momentum reset, and a positive recorded rescale_energy, are reported as UNKNOWN in a WARNING and the restart proceeds: a module engine can inherit the modify_velocities of an engine of PyRETIS, whose momentum reset depends on the PyRETIS version that wrote the output.toml.
A continuation that draws differently is refused with a ValueError that names the engine, how each of the two runs draws, and, where one exists, the zero_momentum and rescale_energy that draw as the run being continued did. For example, the output.toml of a GROMACS run with velocity_generation = "engine" without the key continues with zero_momentum = true, the draw of gen_vel, and is refused with zero_momentum = false.
pyretis run -i output.toml resumes with the settings output.toml records. For an output.toml without velocity_draw_rule it compares the draws those settings select with the draws of the engine the run used, as listed above, and refuses a resume that draws differently with a ValueError that names the [tis] zero_momentum and rescale_energy that draw as that run did; setting them in the [tis] section of output.toml lets the resume proceed. Every run records velocity_draw_rule in the output.toml it writes.
Every other setting is taken from the new input file as written, and applies from the resumed cycle on.
[tis] high_accept may change on a restart too, but it selects the acceptance rule of stone skipping, and the paths the ensembles hold at that point were accepted under the other rule. They are therefore left out of the statistics: each is marked with the initialisation code hc, the change is recorded under [current] high_accept_changes in output.toml, and each ensemble counts again from its first path accepted after the restart. A WARNING reports how many paths were set aside.
A restart also records, as [current] ss_weight_from_cycle in output.toml, the first cycle it writes, unless the state already holds one, and with it, under [current] ss_weight_before, the move list and layout of the run being continued and the paths it holds. The analysis reads them to tell which rows carry the stone-skipping weight of the rule each path was sampled with; see the analysis guide.
[simulation] priority_shooting may change on a restart as well. While it is on, the moves submitted to each ensemble are recorded as [current] moves_submitted in output.toml and continue over the restart; a restart that turns it on without that record starts from the attempted moves in each ensemble’s moves.txt. See priority_shooting.
To continue although one of the checked settings changed, or the draws differ, set allow_setting_change = true in the [simulation] section (of the input, or of output.toml for pyretis run -i output.toml). The check is then skipped and a WARNING says that the samples before and after the restart were not drawn under the same settings. It does not lift the ensemble and interface checks: to move, add or remove interfaces, start a new simulation from the saved paths with load.
### Keywords for the restart method
For the restart method, the following keywords can be set:
### Keywords for the restart method.
Keyword
Description
method
Selects the restart method.
flexible-restart
Read by create_simulation only; pyretis run ignores it.
#### Keyword method
The method keyword selects the restart initialization. The continuation reads the run directory’s output.toml (or the legacy restart.toml), whichever PyRETIS version wrote it, so run it in the directory of the run it continues.
### The restart lists of create_simulation
pyretis.setup.create_simulation() reads a pyretis.restart file when restart is set in its simulation settings, takes the settings from that file, and compares the new input against the two lists below while it builds the simulation object. A path-sampling simulation object built this way cannot be run: its cycles belong to the scheduler, so for a path-sampling run the checks in the restart method are the ones that apply.
#### Implicit parameter overrides
For create_simulation, a curated set of parameters can be changed in the new input file. Whenever a value differs from the one stored in the restart file, PyRETIS logs a WARNING and applies the new value; all other settings from the restart file are preserved unchanged.
The parameters that can be overridden in this way are (also exposed in code as pyretis.inout.settings.RESTART_OVERRIDE_KEYWORDS):
- simulation: steps, seed, priority_shooting
- output: screen, backup, trajectory-file, energy-file, order-file, cross-file, restart-file, pathensemble-file, log_file, log_mode, engine_log
- tis: freq, sigma_v, shooting_move, shooting_moves, high_accept, n_jumps, mirror_freq, target_freq, target_indices
- retis: swapfreq
These cover parameter tuning (step count, random seed, move frequencies, output paths) that does not alter the statistical validity of the accumulated data. For anything beyond this list — including changing interface positions or the number of interfaces — see user-section-initial-restart-forbidden and user-section-initial-flexible-restart.
#### Forbidden keywords (topology lock)
The following keywords describe the topology of the simulation — what is being sampled and how — and changing them would silently reinterpret the already-saved paths against a different state space. In create_simulation, a restart that finds any disagreement on one of these keys between the new input and the restart file aborts with a ValueError:
- simulation: interfaces, task, zero_left, zero_ensemble, flux, permeability
- tis: maxlength, allowmaxlength, enforce_must_cross_m
maxlength and allowmaxlength enter the acceptance probability of the shooting move, and enforce_must_cross_m decides which paths count as members of an ensemble. A restart that changed one of them would continue the same chain under a different acceptance rule, so the saved paths and the new ones would not be samples of one distribution.
The full list is exposed in code as pyretis.inout.settings.RESTART_FORBIDDEN_KEYWORDS.
To change any of these you have two correct options:
- Start a new simulation from saved trajectories — use load. This is the safest route whenever the new topology means the saved paths must be re-validated against fresh ensemble definitions.
- Reuse the saved paths but rewrite the topology — set flexible_restart = True. This bypasses the topology lock and takes the entire new input file as the settings; the user is then responsible for keeping detailed balance. See user-section-initial-flexible-restart.
Practical examples that require one of these two routes (a plain restart will refuse):
- changing one or more interface positions,
- adding or removing interfaces (and the corresponding ensembles),
- changing zero_ensemble, zero_left or flux (which of the [0^-] and [0^+] a run builds follows its task; only pptis reads zero_left, flux and zero_ensemble for that, and zero_left bounds the [0^-] of every task that builds one),
- switching task between tis / retis / repptis / explore,
- enabling or disabling permeability.
#### Keyword flexible_restart
Read only by create_simulation, as described above; a restart through pyretis run ignores it. If True, all settings are taken from the new input file instead of from the restart file. Use this when structural changes to the simulation are needed, for example:
- changing interface positions or their number,
- adding or removing path ensembles.
In this mode no WARNING is emitted for changed values, and no whitelist is enforced — the new input is used as-is.
When settings are changed under flexible_restart, the user is responsible for ensuring that the new setup is consistent with detailed balance. If it is not, the pathensemble.txt files should be discarded before continuing.
If some interfaces are removed, the ensemble folders must be relabeled manually. PyRETIS expects a continuous numbering scheme, e.g. 000/, 001/, 002/, …
The default value is flexible_restart = False.
---
# The orderparameter section
The orderparameter section defines which order parameter to use for a path sampling simulation.
```toml
[orderparameter]
class = "pyretis_distance"
index = [
0,
1,
]
periodic = false
```
Every order parameter that ships with PyRETIS has a name beginning pyretis_: pyretis_position, pyretis_distance, and so on. The bare names are not accepted.
The prefix exists to keep your order parameters apart from ours. A class called Position or Distance is exactly what you are likely to write for your own system, and without the prefix PyRETIS would quietly hand you the built-in instead of your class – a simulation that runs perfectly and measures the wrong thing. Using a bare name now stops the run with an error naming the built-in you may have meant.
Your own order parameters keep whatever name you like; they are selected by naming the file they live in with a module key, and are never confused with a built-in.
Different order parameters require different settings and this is described in detail below for the different types:
- Order parameter Position
- Keywords for order parameter Position
- Keyword class
- Keyword index
- Keyword dim
- Keyword periodic
- Order parameter Distance
- Keywords for order parameter Distance
- Keyword class
- Keyword index
- Keyword periodic
- User-defined order parameters
- Keywords for user-defined order parameters
- Keyword class
- Keyword module
## Order parameter Position
This order parameter is defined as the position of a single particle in a specified dimension. It is defined by specifying the class pyretis_position, selecting the index of the particle to use, dimension to consider and whether to consider periodic boundaries:
```toml
[orderparameter]
class = "pyretis_position"
index = 0
dim = "x"
periodic = false
```
### Keywords for order parameter Position
The following keywords can be set for order parameter Position:
### Keywords for the Position order parameter.
Keyword
Description
class
Selects the Position order parameter.
index
Selects the particle to use.
dim
Selects the dimension to consider for the position.
periodic
Determines if periodic boundaries should be used.
#### Keyword class
Selects the order parameter, should be set to pyretis_position.
#### Keyword index
The index selects the particle to use. The particles are numbered sequentially, from 0, in the sequence they appear in the input to PyRETIS.
Not any. This keyword must be specified.
#### Keyword dim
The dimension to consider for the position. This can be one of:
- x: For the x-coordinate of the particle
- y: For the y-coordinate of the particle
- z: For the z-coordinate of the particle
Not any. This keyword must be specified.
#### Keyword periodic
This keyword determines if we should apply periodic boundaries to the position or not. If it is set to True periodic boundaries will be applied.
The default is: periodic = False
## Order parameter Distance
This order parameter is defined as the distance between two particles. It is defined by selecting the class pyretis_distance, selecting the indices of the particles to consider and then determining if periodic boundaries should be used or not:
```toml
[orderparameter]
class = "pyretis_distance"
index = [
10,
11,
]
periodic = true
```
### Keywords for order parameter Distance
The following keywords can be set for order parameter Distance:
### Keywords for the Distance order parameter.
Keyword
Description
class
Selects the Distance order parameter.
index
Selects the particles to use for the distance.
periodic
Determines if periodic boundaries should be applied.
#### Keyword class
Selects the order parameter, should be set to pyretis_distance.
#### Keyword index
The index selects the particles to use. The particles are numbered sequentially, from 0, in the sequence they appear in the input to PyRETIS. This index is selected by specifying the indices as a Python tuple: index = (0, 1)
Not any. This keyword must be specified.
#### Keyword periodic
This keyword determines if we should apply periodic boundaries to the distance or not. If it is set to True periodic boundaries will be applied.
The default is: periodic = False.
## User-defined order parameters
You can also define custom order parameters to use with PyRETIS. Such order parameters should subclass the generic OrderParameter class as described in the user guide.
```toml
[orderparameter]
class = "WCAJCP1"
module = "../orderp/orderp.py"
index = [
7,
8,
]
periodic = true
```
### Keywords for user-defined order parameters
At least the following keywords can be set for user-defined order parameters:
### Keywords for user-defined order parameters.
Keyword
Description
class
Selects a Python class to use as the order parameter.
module
Defines the module where the Python class can be located.
In addition, user-defined keywords can be specified, e.g.:
```toml
[orderparameter]
class = "MyOrderParameter"
module = "order.py"
setting1 = "input"
setting3 = 123.456
setting2 = true
```
In the example the file ´orderp.py´ shall contain the class ´MyOrderParameter´. The class can use the PyRETIS system object, which contains the various information to compute the order parameter(s) (position, velocities, etc). Furthermore, when using an external engine, the name of the file where the latest frame is stored is included in system.particles.config[0]. The index of the frame can be found in system.particles.config[1].
#### Keyword class
This keyword selects the order parameter and it should be set to the class name of the custom order parameter class defined in the given module.
#### Keyword module
This keyword specified the location of the file containing the user-defined class for the order parameter. This file must be accessible by PyRETIS.
Not any. This keyword must be specified.
---
# The Output section
The output section defines settings for how PyRETIS should create output files.
```toml
[output]
order-file = 1
energy-file = 1
screen = 10
```
The following keywords can be set for the output section:
# Keywords for output to files.
Keyword
Description
bakcup
Determines how we deal with already existing files.
prefix
Add a prefix to output files.
energy-file
Set frequency of output to the energy file.
cross-file
Set frequency of output of crossing data.
order-file
Set frequency of output of order parameter data.
pathensemble-file
Set frequency of output of path ensemble data.
restart-file
Set frequency of restart files overwriting.
trajectory-file
Set frequency of output of trajectory data.
# Keywords for output to the screen.
Keyword
Description
screen
Set frequency of output to the screen.
data_dir
Re-root the per-ensemble output directories.
keep_rejected_status
Keep the trials rejected with the listed status codes.
## Keywords controlling output to files
For the keywords controlling the frequency of the output to files, e.g. energy-file, writing of such output can be disabled by setting a frequency of 0 or less.
Note that PyRETIS will create a set of files for each path ensemble in a RETIS (or TIS) simulation.
The keywords bakcup, prefix, cross-file, pathensemble-file, restart-file and trajectory-file set the output of the in-process tasks (md, md-nve, md-flux). A path-sampling run (tis, retis, explore, pptis, repptis), and make-tis-files, refuses them: the scheduler writes its own output files, and the in-process initiation of the initial paths writes the same files whatever their values (see the keywords that take no effect).
### Keyword backup
Determines how we deal with already existing files. This can be set to one of the following:
- backup = append: Output will be appended to already existing files.
- backup = overwrite: Output will overwrite already existing files.
- backup = backup: A backup of existing files will be created. Existing files will be renamed with a suffix _XXX where XXX is a number.
The keyword applies to the output files of the md, md-nve and md-flux tasks, and a path-sampling run (tis, retis, explore, pptis, repptis) refuses it: a restart appends to the ensemble output of such a run, and a new simulation started with kick or load stops with an error when that output holds sampled cycles. See the restart method for how to add cycles or start over.
The default is backup = append.
### Keyword prefix
The prefix can be used to add names to the output files, that is prefix = TEXT will prepend TEXT to output files created by PyRETIS. This can, for instance, be used to organize your output.
The default is None, that is: no prefix is added by default.
### Keyword energy-file
Determines how often we should write to the energy file. energy-file = N will make PyRETIS write to the energy file every Nth step.
The default is energy-file = 1.
### Keyword cross-file
Determines how often PyRETIS will try to write to the crossing file. The crossing file will only be written to if an actual crossing has been detected. Note that this only applies to simulations for determining the initial flux.
cross-file = N will make PyRETIS write to the crossing file every Nth step. Normally, this keyword should be set to 1. This does not mean that we will write to the crossing file every step, but only that we will check if we can write new crossing data every step. The file will only be written to if a crossing has been made.
The default is cross-file = 1.
### Keyword order-file
Determines how often we will write out order parameters to the order file.
order-file = N will make PyRETIS write to the order file every Nth step.
The default is order-file = 1.
### Keyword restart-file
Determines how often we will write out the restart file.
restart-file = N will make PyRETIS overwrite the restart files every Nth step. The restart files are: pyretis.restart in the main folder and ensemble.restart in the ensemble’s pathfolders for path ensemble simulations.
The default is restart-file = 1.
### Keyword pathensemble-file
Determines how often we will write out path data for path ensembles. This file contains the basic information about the paths (length, maximum/minimum order parameter, what kind of move we perform, was the path accepted etc.) and is the most important file for the analysis. The move type codes and path status (acceptance or rejection reason) codes recorded in this file are explained in user-guide-path-types-rejections.
Every row is an occupancy record – which path the ensemble held that cycle – and is what the crossing-probability analysis reads. After the canonical set come three extra trailing columns: the global PathNumber, the raw high-acceptance weight HA-weight (ha_factor) and FracInc, the ensemble’s share of that path in the cycle. The WHAM analysis reads them directly. The attempted moves, accepted or rejected, which the acceptance ratios are counted from, are written to moves.txt beside this file. Output written in the earlier layouts (a separate ha_weight.txt, attempted moves as Trial = 1 rows of this file, or a separate trials.txt) is still read. See user-guide-output-pathensemble.
Normally, the frequency of output should be set to 1, but setting pathensemble-file = N will make PyRETIS write to the order file every Nth step.
Setting a value different from 1 is not accounted for in the analysis and is not recommended.
The default is pathensemble-file = 1.
### Keyword trajectory-file
Determines how often we should write trajectory data to a trajectory output file.
Setting this to a small value (say 1) is not recommended when using external engines as this may quickly use up all the space on your hard drive. Setting trajectory-file = N means that PyRETIS will store a trajectory every Nth step for each path ensemble in a RETIS/TIS simulation. You are advised to investigate typical sizes for the trajectories before setting this frequency output.
The default is trajectory-file = 100.
## Keywords controlling on-disk trajectory storage
A path-ensemble run keeps its trajectories in a two-tier, per-ensemble store: the operational store /accepted// holds the live accepted paths, and the long-term store /archive// holds paths that have since been replaced. Trial moves are generated in a transient /generate/ scratch directory. Paths are keyed by their global path number and evicted in first-in-first-out order (the eviction queue is persisted, so a restart prunes identically). The following [output] keys control what happens to a superseded path and how much disk the run uses.
A replaced path always moves from the operational store /accepted/ to the long-term store /archive/ (a cheap same-parent rename). By default every replaced path is kept there, so a long run accumulates one trajectory per accepted move and can use a lot of disk. Use archive_every to keep only a periodic sample of them – this is the way to bound disk usage. (The former delete_old / delete_old_all / keep_maxop_trajs deletion keys are retired; a config still setting them is rejected.)
How often a replaced path is kept in the long-term store, restoring the classic PyRETIS trajectory-saving cadence for the two-tier store. archive_every = N keeps only every N-th replaced path in /archive/ (those whose global path number is a multiple of N) and removes the rest, so the archive holds roughly one trajectory per N accepted moves instead of every one. The operational store (the live paths) is unaffected, so sampling and restart are unchanged — only the retained trajectory history is thinned. This is the way to bound disk usage while keeping a representative sample.
The default is archive_every = 1 (keep every replaced path).
Extra trajectory file-name patterns the archiver carries along when a path moves between the stores (for example a topology or restart file that must survive with the trajectory).
The default is an empty list.
## Keywords controlling the log files
A run writes two kinds of logs, on two different file structures:
- The run log (pyretis.log): one file for the whole simulation, written by PyRETIS itself. It keeps growing with the simulation — by default a restart appends to it, so one file tells the whole story of the run.
- The engine logs: the external engine’s own output (the engine.log / engine.err command capture, the streaming engines’ stdout.txt / stderr.txt, and per-run files such as a LAMMPS .log / .screen pair). These are per trajectory: the live log is zeroed every time a new trajectory is generated, and each finished trajectory’s logs are stored in a numbered folder /engine_logs// next to the ensemble’s generate/ scratch. The engine_log keyword decides how many of those per-trajectory folders are kept. When a trajectory generation fails, its logs are left in the scratch directory for inspection.
The name of the run log. The name is also carried to the worker processes of a parallel run, so all of them append to the same file. An explicit pyretis run -f overrides this keyword for the main process only — in a parallel run, set the keyword (not just -f) so the workers follow the same name.
The default is log_file = 'pyretis.log'.
What happens to an existing run log when a run starts:
- 'append': keep the existing log and append to it — the log grows with the simulation and a restart continues the same file.
- 'backup': rotate the existing log aside first (to pyretis.log_000, _001, …), the historical behaviour.
- 'overwrite': truncate the existing log.
The default is log_mode = 'append'.
How many per-trajectory engine-log folders (/engine_logs//) are kept per ensemble:
- 'last': keep only the most recent trajectory’s logs.
- 'all': keep every trajectory’s logs (mind the disk usage of a long run).
- a positive integer N: keep the N most recent.
The retention never affects the sampled paths — the logs are a purely human-inspectable record. The scratch directories are per ensemble (they replaced the old per-worker directories), so the engine-log store is per ensemble too.
The default is engine_log = 'last'.
## Keywords controlling output to the screen
Note that if you run the PyRETIS application and choose to use the progress bar, that is:
```pyretis
pyretis run -i input.toml -p
```
then writing of results to the screen will be disabled.
### Keyword screen
Determines how often output from a simulation should be written to the screen during a simulation. The scheduler of a path-sampling run writes its progress report (the moves it submits and the paths they give) to the log every screen cycles, and writes none with screen = 0; the run file output.toml records the value the run takes.
The default is screen = 10.
### Keyword data_dir
Places the per-ensemble output directories under this directory. The per-path archive follows them, since it lives inside those directories.
No default; the ensembles are written in the run directory.
### Keyword keep_rejected_status
The rejection status codes whose trials are kept. A rejected trial is otherwise discarded, and keeping the interesting ones makes the rejected trajectory available for inspection afterwards. ACC is not a rejection and is refused here. See user-guide-path-types-rejections for the codes.
No default; no rejected trial is kept.
---
# The Particles section
Particles are added to a simulation in the particles section:
```toml
[particles]
name = [
"Ar",
]
ptype = [
0,
]
[particles.position]
input_file = "initial.gro"
[particles.velocity]
generate = "maxwell"
temperature = 2.0
momentum = true
seed = 0
[particles.mass]
Ar = 1.0
```
In this example, the particles are read from an input file called 'initial.gro' and the velocities are set according to a Maxwell distribution with a temperature of 2.0 and the velocities are initiate so that the total momentum is set to zero. Further, the mass of particles with name 'Ar' is set to 1.0 and all particles are named 'Ar' and assigned a particle ptype of 0.
## Keywords for the Particles section
The following keywords can be specified for the Particles section:
## Keywords for the Particles section.
Keyword
Description
position
Defines how initial positions are set.
velocity
Defines how initial velocities are set.
mass
Defines the masses for the particles.
name
Defines names for the particles.
ptype
Defines the known particle types.
npart
Defines the number of particles (analysis specific).
type
Select how the trajectories are held.
### Keyword position
The position keyword is used to set the initial positions for a simulation. It is also used to define the particles we will use in the simulation. The initial positions can be:
- Generated on a lattice,
```toml
[particles.position]
generate = "fcc"
repeat = [
3,
3,
3,
]
lcon = 1.0
```
- When generating on a lattice, the following settings must be provided:
- 'generate' selects what kind of lattice to generate, valid choices are:
- 'sc': A simple cubic lattice.
- 'sq': A square lattice (2D) with one atom in the unit cell.
- 'sq2': A square lattice (2D) with two atoms in the unit cell.
- 'bcc': A body-centered cubic lattice.
- 'fcc': A face-centered cubic lattice.
- 'hcp': A hexagonal close-packed lattice.
- 'diamond': A diamond-like structure.
- 'repeat' select the number of repetitions of the unit lattice in the spatial directions.
- 'lcon' or 'density' which specify the size of the lattice:
- 'lcon': Specifies the lattice constant.
- 'density': Specifies a particle density.
- The available lattices are further described in the description of the generate_lattice command from the API.
- Or read from a file,
```toml
[particles.position]
input_file = "initial.gro"
```
- When reading initial positions from a input file, the following settings must be provided:
- 'input_file' which specifies the input file to open.
- PyRETIS will also attempt to read velocities from the input file when the format supports them. This includes the extended xyz format (the 6-column variant with three velocity columns vx vy vz), as well as GROMACS gro and g96 files. Velocities read this way are assumed to be in the units associated with the file format (Å/fs for xyz, nm/ps for gro and g96) and are converted to the system’s internal units on read. The resulting numbers may therefore look very different from the values written in the file, especially when the system uses reduced units such as lj.
- Velocities read from the file can be used directly, so the velocity keyword is optional. If velocity = {'generate': ...} is also specified together with an input_file that provided velocities, the generated velocities override those read from the file and a warning is logged.
The specified input_file name is case-sensitive.
More examples:
```toml
# Read a file from the current directory:
[particles.position]
input_file = "initial.gro"
```
```toml
# Read a file from another directory:
[particles.position]
input_file = "/path/to/file/initial.gro"
```
```toml
# Generate an fcc lattice with 4 * 5**3 = 500 particles and
# a lattice constant of 1 in reduced units:
[particles.position]
generate = "fcc"
repeat = [
5,
5,
5,
]
lcon = 1.0
```
```toml
# Generate a bcc lattice with 2 * 5**3 = 250 particles and
# a reduced density of 0.9:
[particles.position]
generate = "bcc"
repeat = [
5,
5,
5,
]
density = 0.9
```
### Keyword velocity
The velocity keyword is used to set initial velocities for the particles according to a specified temperature. It is only needed when the position source does not already provide velocities, or when the user wants to override velocities read from an input file. If specified together with an input file that contains velocities, the values produced by velocity take precedence and a warning is logged.
Example:
```toml
[particles.velocity]
generate = "maxwell"
temperature = 2.0
momentum = true
seed = 0
```
The following settings must be provided:
- 'generate' which specifies the distribution to draw random velocities from. Currently, the only recognised distribution is a Maxwell distribution, which is selected by setting the value 'maxwell', as shown in the example above.
- 'temperature' which specifies the desired temperature in internal units. If this is not provided, the value set by the input keyword temperature is used.
- 'momentum' which specifies if the resulting momentum should be zero or not. The default value is True which will generate velocities so that the total momentum is zero.
- 'seed' which can be used to specify a seed for the random number generator. If this value is not given a 0 will be used.
### Keyword mass
The mass keyword sets the masses for different particles. The provided values are assumed to be in internal units. If masses are not set, PyRETIS will try to guess them from the periodic system, however, this might fail unless the names you have used for the particles correspond to names used in the periodic system.
Example 1:
```toml
# define masses for particles with name "Ar"
[particles.mass]
Ar = 1.0
```
Example 2:
```toml
# define masses for particles with name "Ar", "Kr" and "big"
[particles.mass]
Ar = 1.0
Kr = 2.098
big = 10.0
```
### Keyword name
The name keyword is used give particles a name. This can, for instance, be used to label specific particles and, when used together with the mass keyword, it can be used to control the masses assigned to the different particles. The particle names are also used when writing output configurations.
Example:
```toml
[particles]
name = [
"Ar",
]
```
The input value is a list with the particle names in the order they have been generated/read from a file by the position keyword. If the name list is too short, the last item in the list will be repeated.
This means that,
```toml
[particles]
name = [
"Ar",
"Kr",
]
```
will name one particle (the first) 'Ar' and the rest of the particles will be named 'Kr'.
### Keyword ptype
The ptype keyword is used to specify the particle types for the different particles. The particle type is used to distinguish particles, for instance when calculating pair interactions.
The input value is a list with the particle types in the order they have been generated/read from a file by the position keyword. If the ptype list is too short, the last item in the list will be repeated.
Example 1:
```toml
[particles]
ptype = [
0,
1,
]
name = [
"Ar",
"Kr",
]
```
This can be used to define a simulation where we have particles of two different ptypes with different masses.
Example 2:
```toml
[particles]
ptype = [
0,
]
name = [
"Ar",
"Ar2",
"Ar",
]
```
This can be used to define a simulation where we have only particles of the same ptype, however, one of the particles have a different name than the others and can, for instance, be assigned a different mass.
### Keyword npart
The npart keyword specifies the number of particles in total in the simulation. This keyword is used by the analysis application, and does not need to be set in the input. The PyRETIS application will automatically update this keyword.
None, the PyRETIS application will automatically update this keyword.
### Keyword type
Selects the path ensemble the simulation builds. internal keeps the trajectories in memory, and external keeps them as the files the engine writes.
The default value is "internal".
---
# The Potential section
The potential section specifies a single potential function to add to the force field.
```toml
potential = [
{ class = "DoubleWell", parameter = { a = 1.0, b = 2.0, c = 0.0 } },
]
```
In order to add several potential functions to a force field, several Potential sections can be added:
```toml
[[potential]]
class = "PairLennardJonesCutnp"
shift = true
dim = 2
mixing = "geometric"
[potential.parameter.0]
sigma = 1.0
epsilon = 1.0
factor = 1.12246205
[potential.parameter.1]
sigma = 1.0
epsilon = 1.0
factor = 1.12246205
[[potential]]
class = "DoubleWellWCA"
dim = 2
[potential.parameter]
rzero = 1.122462048309373
height = 15.0
width = 0.5
types = [
[
1,
1,
],
]
```
As the different potentials typically require different settings, detailed information can be found below for specific potentials:
- The DoubleWell potential
- Keywords for the DoubleWell potential
- Keyword class
- Keyword parameter
- The RectangularWell potential
- Keywords for the RectangularWell potential
- Keyword class
- Keyword parameter
- The PairLennardJonesCutnp potential
- Keywords for the PairLennardJonesCutnp potential
- Keyword class
- Keyword dim
- Keyword shift
- Keyword mixing
- Keyword parameter
- The DoubleWellWCA potential
- Keywords for the DoubleWellWCA potential
- Keyword class
- Keyword dim
- Keyword parameter
- User-defined potential functions
- Keywords for user-defined potential functions
- Keyword class
- Keyword module
## The DoubleWell potential
This class defines a one-dimensional double well potential. The potential energy (V_\text{pot}) is given by
where x is the position and a, b and c are parameters for the potential.
```toml
potential = [
{ class = "DoubleWell", parameter = { a = 1.0, b = 2.0, c = 0.0 } },
]
```
### Keywords for the DoubleWell potential
The following keywords can be specified for the DoubleWell potential:
### Keywords for the DoubleWell potential.
Keyword
Description
class
Selects the DoubleWell potential.
parameter
Sets the parameters for the potential.
#### Keyword class
This keyword selects the potential and should be set to: DoubleWell.
#### Keyword parameter
For the DoubleWell potential, three parameters can be specified:
The a parameter in the potential energy function given above.
The default is: parameter a = 1.0
The b parameter in the potential energy function given above.
The default is: parameter b = 1.0
The c parameter in the potential energy function given above.
The default is: parameter c = 0.0
## The RectangularWell potential
This class defines a one-dimensional rectangular well potential. The potential energy is zero within the potential well and infinite outside. The well is defined with a left and right boundary.
```toml
potential = [
{ class = "RectangularWell", parameter = { left = -1.0, right = 1.0 } },
]
```
### Keywords for the RectangularWell potential
The following keywords can be specified for the RectangularWell potential:
### Keywords for the RectangularWell potential.
Keyword
Description
class
Selects the RectangularWell potential.
parameter
Sets the parameters for the potential.
#### Keyword class
This keyword selects the potential and should be set to: RectangularWell.
#### Keyword parameter
For the RectangularWell potential, two parameters can be set.
The left border for the potential well function. The potential energy is 0 for positions x such that \text{left} < x < \text{right}.
The default parameter is: parameter left = 0.0
The right border for the potential well function. The potential energy is 0 for positions x such that \text{left} < x < \text{right}.
The default parameter is: parameter right = 1.0
## The PairLennardJonesCutnp potential
This class implements as simple Lennard-Jones 6-12 potential which employs a simple cut-off and can be shifted. The potential energy (V_\text{pot}) is defined in the usual way for an interacting pair (i,j) of particles a distance r apart,
where x = \sigma_{ij}/r and \varepsilon_{ij} and \sigma_{ij} are the potential parameters.
Here is an example for defining parameters for a mixture of 3 particle types with geometric mixing.
```toml
[[potential]]
class = "PairLennardJonesCutnp"
shift = true
dim = 3
mixing = "geometric"
[potential.parameter.0]
sigma = 1.0
epsilon = 1.0
rcut = 2.5
[potential.parameter.1]
sigma = 1.2
epsilon = 1.1
rcut = 2.5
[potential.parameter.2]
sigma = 1.4
epsilon = 0.9
rcut = 2.5
```
### Keywords for the PairLennardJonesCutnp potential
The following keywords can be specified for the PairLennardJonesCutnp potential:
### Keywords for the PairLennardJonesCutnp potential.
Keyword
Description
class
Selects the potential.
dim
Set the number of dimensions to consider.
shift
Determines if the potential should be shifted.
mixing
Selects the mixing rules for generating parameters for cross interactions.
parameter
Sets the parameters for the potential.
#### Keyword class
This keyword selects the potential and should be set to: PairLennardJonesCutnp.
#### Keyword dim
Sets the dimensionality for the potential. Should be 1, 2, or 3 for 1D, 2D or 3D, respectively.
The default value is dim = 3.
#### Keyword shift
Determines if the potential should be shifted or not. This should be either True (enables shifting) or False (disables shifting). If shifting is True, the following value will be added to the potential (for a given (i, j) pair):
where x_{\text{c}} = \sigma_{ij} / r_{\text{c}, ij} and r_{\text{c}, ij} is the cut-off.
The default value is shift = True.
#### Keyword mixing
Determines how mixing parameters for the potential should be determined. The supported mixing rules are:
- mixing = geometric:
- \epsilon_{ij} = \sqrt{\epsilon_{i} \times \epsilon_{j}}
- \sigma_{ij} = \sqrt{\sigma_{i} \times \sigma_{j}}
- r_{\text{c},ij} = \sqrt{r_{\text{c},i} \times r_{\text{c},j}}
- mixing = arithmetic:
- \epsilon_{ij} = \sqrt{\epsilon_{i} \times \epsilon_{j}}
- \sigma_{ij} = \frac{\sigma_{i} \times \sigma_{j}}{2}
- r_{\text{c},ij} = \frac{r_{\text{c},i} \times r_{\text{c},j}}{2}
- mixing = Sixthpower:
- \epsilon_{ij} = 2 \sqrt{\epsilon_{i} \times \epsilon_{j}} \frac{\sigma_i^3 \times \sigma_j^3}{\sigma_i^6 + \sigma_j^6}
- \sigma_{ij} = \left( \frac{\sigma_{i}^6 \times \sigma_{j}^6}{2} \right)^{1/6}
- r_{\text{c},ij} = \left(\frac{r_{\text{c},i}^6 \times r_{\text{c},j}^6}{2}\right)^{1/6}
The default value is mixing = geometric.
#### Keyword parameter
The following parameters can be set for the PairLennardJonesCutnp potential:
The parameters for the PairLennardJonesCutnp potentials are given for each particle type on the form:
```toml
[[potential]]
class = "PairLennardJonesCutnp"
[potential.parameter.0]
sigma = 1.0
epsilon = 1.0
rcut = 2.5
```
Where i is the particle type, identified as an integer. Here, sigma is \sigma_i, epsilon is \varepsilon_i and rcut is r_{\text{c},i}.
Note that several parameters can be given in this way and that they will be combined according to the selected mixing rule.
## The DoubleWellWCA potential
This class defines a n-dimensional Double Well potential. The potential energy (V_\text{pot}) for a pair of particles separated by a distance r is given by,
where h gives the ‘height’ of the potential, r_0 the minimum and w the width.
```toml
[[potential]]
class = "DoubleWellWCA"
dim = 2
[potential.parameter]
rzero = 1.122462048309373
height = 6.0
width = 0.25
types = [
[
1,
1,
],
]
```
### Keywords for the DoubleWellWCA potential
The following keywords can be specified for the DoubleWellWCA potential:
### Keywords for the DoubleWellWCA potential
Keyword
Description
class
Selects the potential.
dim
Sets the number of dimensions to consider.
parameter
Sets the parameters for the potential.
#### Keyword class
Selects the potential, should be set to: DoubleWellWCA.
#### Keyword dim
Sets the dimensionality for the potential. Should be 1, 2, or 3 for 1D, 2D or 3D, respectively.
The default value is dim = 3.
#### Keyword parameter
The following parameters can be set for the DoubleWellWCA potential:
This is the r_0 parameter in the potential energy function given above.
The default is rzero = 0
This is the h parameter in the potential energy function given above.
The default is height = 0
This is the w parameter in the potential energy function given above.
The default is width = 0
This parameter determines for which pair of particles the interaction should act on. If this is not set, it will turn on the interaction for all particles. For example, if types = [(1,2)] the interaction will be turned on between particles of type 1 and type 2 only. (See the particle section for a description about the particle type.) Note that several types can be specified at the same time: types = [(1, 2), (0, 1), (0, 2)].
The default is activated for all pairs.
## User-defined potential functions
User-defined potential functions are selected as follows:
```toml
[[potential]]
class = "PotentialName"
module = "potential.py"
keyword1 = "value"
[potential.parameter]
p = 123.0
```
Here, the user can both define custom keywords and parameters. However, the class and the module must always be specified in this case.
### Keywords for user-defined potential functions
At least, the following keywords can be specified for user-defined potential functions:
### Keywords for user.defined potential functions.
Keyword
Description
class
Selects the Python class which implements the potential.
module
Defines the file where the Python class with the potential can be found.
#### Keyword class
This keyword selects the potential function and it should be set to the class name of the custom potential function class defined in the given module.
#### Keyword module
This keyword specified the location of the file containing the user-defined class for the potential function. This file must be accessible by PyRETIS.
Not any. This keyword must be specified.
---
# The REPPTIS and PPTIS sections
A partial-path simulation reads these settings from the section named after its task: task = "repptis" from [repptis] and task = "pptis" from [pptis]. Both sections carry the same keyword. The TIS section also has to be defined for a partial-path simulation.
```toml
[simulation]
task = "repptis"
[repptis]
memory = 1
```
The shooting_moves keyword states which moves a partial-path task accepts.
## Keywords for the REPPTIS and PPTIS sections
The following keyword can be set for either section:
## Keywords for the REPPTIS and PPTIS sections.
Keyword
Description
memory
Set how far the local window of each positive ensemble reaches.
### Keyword memory
Sets the local interface window each positive ensemble samples in. With memory = m, ensemble i samples the window [\lambda_{i-m}, \lambda_i, \lambda_{i+m}], and the paths it holds are the partial paths that window bounds. The analysis rebuilds the kinetics from the crossing probabilities of these local windows.
The default value is 1.
---
# The RETIS section
The RETIS section specifies settings for RE(PP)TIS simulations. Note that the section for TIS also needs to be defined for a RE(PP)TIS simulation. A keyword of this section that takes no effect on the run is refused (see the keywords that take no effect).
```toml
[retis]
swapfreq = 0.5
```
## Keywords for the RETIS section
The following keywords can be set for the RETIS section:
## Keywords for the RETIS section.
Keyword
Description
nullmoves
Refused: no part of a path-sampling run reads it.
relative_shoots
Selects and defines relative shooting.
priority_shooting
Prioritize the ensembles with fewer moves.
rgen
Refused unless "pcg64", the generator of the scheduler.
seed
Refused: [simulation] seed seeds the scheduler.
swapfreq
The probability of a swap move.
swapsimul
Refused: no part of a path-sampling run reads it.
### Keyword nullmoves
No part of a path-sampling run reads this keyword, so a path-sampling run and make-tis-files refuse it. The infinite-swapping scheduler moves the ensembles of a move and leaves the paths of every other ensemble in place.
### Keyword relative_shoots
This keyword selects and defines relative shooting. Relative shooting means that the ensembles are shot in with unequal frequencies. This is selected by specifying the relative frequency for each ensemble in a list, e.g. relative_shoots = [0.1, 0.1, 0.2, 0.6] for four ensembles (one weight per ensemble, in interface order — the [0^-] minus ensemble first when it is present), where these numbers give the relative probability of the move being performed in that ensemble. The infinite-swapping scheduler that drives all path-sampling runs weights its per-ensemble move selection by these frequencies, so a heavier ensemble is worked on more often. The weights are state-independent, so every ensemble still samples its own path distribution and the computed rate is unchanged — only the allocation of sampling effort across the ensembles changes. The weights set only which ensemble’s move is chosen; the path the move starts from is drawn from the paths that ensemble holds, by their occupancy, as without the keyword.
The weights need not sum to one (they are normalised internally); they must be non-negative with a positive sum. If relative_shoots = None (the default when the key is omitted) the ensembles are selected with the sampler’s own, roughly uniform, frequencies. relative_shoots cannot be combined with priority_shooting.
[retis] relative_shoots is the one keyword of the weights: an input that sets relative_shoots in [tis] or [simulation] is refused, with a message that names [retis] relative_shoots. A run that picks no ensemble, a tis run of one ensemble or a task that runs no move, refuses it.
relative_shoots = None (no relative shooting).
### Keyword priority_shooting
Set this keyword in the simulation section.
If True, the scheduler weights the choice of each move’s ensemble toward the ensembles that have received fewer moves. A move is a pair: the ensemble whose rules it follows (its interfaces and its acceptance) and one of the paths that ensemble holds, drawn by its infinite-swap occupancy. priority_shooting changes only the first choice: an ensemble that has received n moves, while the most-sampled ensemble has received n_max, is chosen with weight n_max - n + 1. The ensembles that lag are shot more often until all have received the same number of moves; level ensembles are equally likely. The path is then drawn from the chosen ensemble as usual.
A move counts when it is submitted, accepted or not, and a zero swap counts for both of its ensembles. The weights therefore depend on the moves made so far, and not on the current paths or on how long the moves take to finish with several workers, so every move still leaves the sampled distribution unchanged: only the allocation of moves, and with it the statistical error of each ensemble, changes.
The counts are kept in output.toml as [current] moves_submitted, so the ensembles stay level over a series of short jobs, each one a restart of the previous one. A restart of a run that did not record them starts from the attempted moves in each ensemble’s moves.txt. priority_shooting cannot be combined with relative_shoots or pick_scheme.
The default value is priority_shooting = False.
### Keyword rgen
The scheduler draws every random number of a path-sampling run from streams of numpy’s PCG64 generator spawned from [simulation] seed, so a path-sampling run refuses any value of this keyword other than "pcg64".
The default value is rgen = None.
### Keyword seed
The scheduler spawns its random streams from [simulation] seed, and the in-process kick initiation seeds its generators from [tis] seed, so a path-sampling run refuses this keyword, with a message that names [simulation] seed.
The default value is seed = None.
### Keyword swapfreq
The probability of a swap move, the one keyword of it. In a retis run, a move picked in the [0^-] or the [0^+] ensemble is, with this probability, the swap of the paths of the two (the zero swap) when the other ensemble is free; in a repptis run, a move is, with this probability, a swap with an adjacent ensemble. A tis, pptis or explore run attempts no swap, and refuses the keyword. An input that sets [simulation] zeroswap is refused, with a message that names [retis] swapfreq; the run file output.toml records the probability as [retis] swapfreq.
The default value is swapfreq = 0.5 for a retis and a repptis run.
### Keyword swapsimul
No part of a path-sampling run reads this keyword, so a path-sampling run and make-tis-files refuse it.
---
# The Runner section (infinite swapping)
The [runner] section configures the infinite-swapping sampler – the asynchronous, parallel reformulation of replica exchange described in the moves guide. It is only used by the infinite-swapping route; the in-process simulation flow ignores it.
This page documents how the two PyRETIS execution routes are selected from the input, the keywords of the [runner] section, and the environment variables that override the routing. For a worked example see the infinite-swapping tutorial and the runnable inputs under the per-engine examples/tutorials/path_sampling///infinite_swapping/ cases.
The infinite-swapping inputs use the TOML input format (pyretis run -i infswap.toml), not the classic .rst input. A complete annotated input is shipped at examples/tutorials/path_sampling/turtlemd/1D-double-well/infinite_swapping/infswap.toml.
## The two execution routes
Every path-sampling run is driven through the single pyretis run command, and every one of them runs on the infinite-swapping scheduler: a central coordinator hands work to a pool of parallel workers. That holds for a TOML input with task = "tis", "retis", "explore", "pptis" or "repptis", with an explicit infinite-swapping task (task = "infinite_swapping" or "infswap"), or with a [runner] section, and for every engine. Without a [runner] section the scheduler runs one worker.
An input that asks for a combination the scheduler does not support is refused with an error; it is not run another way. From Python, a path-sampling run is driven the same way, with pyretis.simulation.scheduler.scheduler_cycles(). The worker count can be overridden with the environment variables below.
## Keywords of the runner section
```toml
[runner]
workers = 4
```
This requests four parallel workers.
## Keywords for the runner section.
Keyword
Description
workers
Number of parallel workers in the scheduler pool.
files
Files pyretis tools init stages for each step.
wmdrun
Per-worker external-engine MD run commands.
### Keyword workers
The number of parallel workers the scheduler drives. Each worker propagates one trajectory at a time and is handed new work as soon as it frees up. The coordinator takes the finished moves back one at a time and, where the engine can, recomputes the energies of every new path from its stored frames, so the wall-clock time falls close to linearly with the number of workers only while the coordinator spends little time on a move against the time a move takes; with the internal engines it spends about as long, which limits the gain from more workers. workers should not exceed the number of path ensembles (a worker per ensemble is the natural maximum).
A worker is an independent Python process with its own molecular- dynamics engine. The coordinator chooses an ensemble and one of the paths it holds, gives that trial to a free worker, and incorporates the returned accepted or rejected path. It is therefore closer to a laboratory assistant completing one trajectory assignment at a time than to a thread splitting a single trajectory. For an external engine, each worker launches its own LAMMPS / GROMACS / CP2K process; any MPI ranks or OpenMP threads used inside that engine are a separate level of parallelism and count in addition to the worker pool.
With pyretis run -p, the Monte Carlo bar reports completed sampling cycles while its live postfix reports the most recently advancing worker, ensemble, propagation leg and leg length, for example worker 2 | ensemble 004 | forward leg length 17 | cap <= 2000. maxlength is displayed as a cap rather than a percentage because a path normally stops at an interface before reaching it and regular variable-length shooting may draw a still smaller stochastic cap.
With several workers, the scheduler takes the finished moves back one at a time, in the order they finished. While it treats one move (taking in its accepted path or keeping the old one, updating the occupancies, writing its output and picking the next move), a move that finishes meanwhile waits with its ensembles locked. The time at which its paths return to the free ensembles then depends on the durations of the moves and, through them, on the paths, and the sampled occupancies deviate from the product measure. In small models solved exactly, the deviation is small when the coordinator spends little time on a move against the time a move takes, largest when the two times are comparable, and smaller again when the coordinator is much slower. The internal-engine tutorials run near a ratio of 1, the two turtlemd infinite-swapping tutorials at about 0.06 and 0.2. With two workers at most one finished move waits while the coordinator treats another; with three or more, several can. A run with one worker has none of this.
The default is workers = 1 (a single-worker run, which reproduces the canonical RETIS loop).
### Keyword files
Read only by the automatic interface placement, pyretis tools init, which copies these files, resolved relative to the input file, into the directory of each of its short runs. The most common entry is a custom order-parameter module referenced from the orderparameter section, e.g. files = ["orderp.py"]. pyretis run refuses an input that carries this keyword.
The default is an empty list (no auxiliary files are copied).
### Keyword wmdrun
For external MD engines (GROMACS / LAMMPS / CP2K), the per-worker command used to launch the engine. The list has one entry per worker and is indexed by the worker’s pin, so different workers can run on different devices or with different launch wrappers. It is not used by the internal in-process engines.
The default is unset (the engine’s own run command is used).
## Environment overrides
A few environment variables override the routing decision without editing the input file. They are reversible escape hatches, primarily for debugging and for plain-vs-scheduler reference comparisons:
## Environment variables controlling the route.
Variable
Effect
PYRETIS_WORKERS=N
Number of workers for the canonical retis scheduler route (default 1; the pre-rename spelling PYRETIS_NATIVE_WORKERS is still honoured).
The permeability mirror / target swap moves route through the scheduler by default (at n_workers = 1); no opt-in is needed any more.
## Output and analysis
The scheduler always writes the per-ensemble pathensemble.txt / order.txt / energy.txt files (in the numbered ensemble directories under the output data directory), so the standard pyretis analyse tooling and pyvisa can analyse a scheduler run exactly as they would a plain one. There is no native_compat setting any more (a retired legacy key) – this is the only output format.
Because the trajectories are shared fractionally across ensembles rather than collected from fixed, independent ones, the per-interface crossing probabilities and the final rate constant can also be combined with the Weighted Histogram Analysis Method (WHAM) rather than by simple per-ensemble averaging. WHAM reads the same information from the per-ensemble output (reconstructed on demand – the scheduler no longer writes a separate infswap_data.txt). See the analysis guide for how to run the WHAM crossing-probability analysis.
---
# Input file sections
The PyRETIS input file described in detail in the user guide. The short version is:
- The input file is TOML. It is organised into sections whose keywords are given values:
```toml
[section]
keyword = "value"
```
- Comments are marked with a #.
- Section names and keywords are lower case, and the file is case-sensitive.
Below, we list the different sections that you can make use of in order to define your simulation:
# Input sections for defining simulations.
Section
Usage
simulation
For defining the simulation we are going to run.
system
For defining system properties.
box
For defining a simulation box.
particles
For defining the initial state of particles.
forcefield
For defining a forcefield.
potential
For defining potential functions to use in the force field.
engine
For defining the simulation engine.
orderparameter
For defining the order parameter.
retis
For defining settings for a RETIS simulation.
repptis
For defining settings for a PPTIS or REPPTIS simulation.
tis
For defining settings for a TIS simulation.
runner
For configuring the infinite-swapping scheduler (parallel workers).
initial-path
For defining how the initial path is generated.
output
For defining output settings.
unit-system
For defining custom unit systems.
In addition, an analysis can be defined using:
# Input sections for defining an analysis.
Section
Usage
analysis
For defining an analysis.
## Keywords that take no effect on a run
Every keyword an input gives either takes effect on the run, or the input is refused before the run starts, with a message that names each such keyword and its value, says why it takes no effect, and names the keyword to set instead when another keyword sets what it would set. A keyword can take effect on one task and none on another: the [output] keywords of the in-process output are read by the md tasks and refused by a path-sampling run, whose scheduler writes its own output files. The rule is declared in one place, pyretis.inout.key_table.NO_EFFECT, and the settings parse of every input route applies it: a TOML input, and a legacy .rst input, which pyretis run checks before it routes it (a path-sampling one to its TOML twin). A value the table accepts states what the run does (rgen = "pcg64", [engine] subcycles = 1): the run takes it as it takes an input without the keyword. A run file (output.toml) written by an earlier PyRETIS that holds such a keyword continues and analyses: the keyword is left out when the file is read, silently when its value is the one the settings parse gives or an accepted one, and named in a warning otherwise.
In the table, a path-sampling run is a run of the task retis, repptis, tis, pptis or explore, and the refusal applies to make-tis-files as well, which writes the input of a tis run of each ensemble.
## The keywords a run refuses, and the runs it refuses them on.
Keyword
Refused on
Set instead
[simulation] zeroswap
Every task.
[retis] swapfreq
[simulation] relative_shoots, [tis] relative_shoots
Every task.
[retis] relative_shoots
[retis] relative_shoots
Every task but a path-sampling run that picks the ensemble of its moves: a tis run of one ensemble refuses it.
[retis] swapfreq
A tis, pptis or explore run and make-tis-files, which attempt no swap.
[retis] nullmoves, [retis] swapsimul, [tis] nullmoves
A path-sampling run.
[retis] seed
A path-sampling run.
[simulation] seed
[simulation] rgen, [retis] rgen
A path-sampling run, unless the value is "pcg64", the generator of the scheduler.
[tis] rgen, [engine] rgen, [system] rgen
A path-sampling run that kicks no path in process: a load or a restart initiation, and a parallel kick (kick-parallel = true); unless the value is "pcg64".
[engine0] rgen and the rgen of a numbered engine section of a pool
A path-sampling run, unless the value is "pcg64".
[engine0]
Every run in which no ensemble runs it: an ensemble runs it when its list in [simulation] ensemble_engines names it, and, without ensemble_engines, the [0^-] ensemble of a run with [tis] quantis = true.
[simulation] restart
A path-sampling run.
[initial-path] method = "restart"
[simulation] startcycle, endcycle, umbrella, overlap, maxdx, mincycle
A path-sampling run.
[simulation] flux
A path-sampling run other than pptis.
[simulation] zero_ensemble
A path-sampling run other than pptis; make-tis-files reads it.
[tis] ensemble_number
A path-sampling run other than tis.
[tis] detect
A path-sampling run other than a tis run with [tis] ensemble_number.
[output] backup, cross-file, pathensemble-file, prefix, restart-file, trajectory-file
A path-sampling run.
[engine] subcycles
An md, md-nve or md-flux run with an internal integrator (Verlet, Velocity Verlet, Langevin), which stores a step after each integration step; unless the value is 1.
The run file output.toml of a run records none of these keywords.
The rgen keyword of [simulation], [system], [tis], [engine], [engine0], [retis], [particles.velocity] and of a numbered engine section names a random generator: "pcg64" (numpy’s PCG64), "rgen" (numpy’s legacy MT19937), "mock", "rgen-borg" or "mock-borg". An input whose rgen names no generator is refused on every task, with the values that name one.
## Notation for describing keywords
Each keyword entry is shown as:
```text
keyword = DATA-TYPE
```
where keyword is the input-file key and DATA-TYPE is the kind of value accepted by that key. The paragraph below each keyword explains what it controls. When a keyword has a default value, the default is listed directly below the description.
Example:
```text
task = string
Selects the simulation task to run.
Default:
md
```
The data types used in the keyword reference are listed below.
## The different data types encountered in PyRETIS.
DATA-TYPE
Description
Example
string
A string of characters, i.e. text.
task = retis
integer
An integer.
steps = 100
float
A floating-point number
timestep = 0.002
boolean
A boolean value (True or False).
shift = True
dictionary
A Python dictionary.
mass = {'Ar': 1.0}
list
A Python list.
interfaces = [0.1, 0.2, 0.3]
tuple
A Python tuple.
index = (7,8)
None
This represents an optional value
---
# The simulation section
The simulation section defines and selects the simulation PyRETIS will run.
```toml
[simulation]
task = "retis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
For this section, the keyword task specifies the type of simulation to run and this will also determine the keyword that can be set in this section. Further, a specific task might also require additional sections to be defined as detailed in the table below:
# Possible values for the task keyword
Task
Description
Keywords
Required sections
retis
A replica exchange transition interface sampling simulation.
task, steps, interfaces
tis, retis
tis
A transition interface sampling simulation.
task, steps, interfaces, ensemble_number, detect
tis
repptis
A replica exchange partial path transition interface sampling simulation.
task, steps, interfaces
tis
pptis
A partial path transition interface sampling simulation.
task, steps, interfaces
tis
explore
A free energy surface exploration simulation.
task, steps, interfaces
tis
md-flux
A MD FLUX simulation.
task, steps, interfaces
md-nve
A MD NVE simulation.
task, steps
Three further values are accepted. md runs a molecular dynamics simulation and make-tis-files writes the input files for single TIS simulations and exits, both taking task and steps. umbrellawindow runs one window of an umbrella sampling simulation; its keywords are given in its own section.
Since the different tasks may require different keywords, these are described below individually. In addition, there are some settings which typically are used in relation to the analysis, or when extending simulations. These are described in the section on common keywords.
- The retis task
- Keywords for the retis task
- Keyword steps
- Keyword interfaces
- Keyword priority_shooting
- The tis task
- Keywords for the tis task
- Keyword steps
- Keyword interfaces
- Keyword priority_shooting
- Keyword ensemble_number
- Keyword detect
- The repptis task
- Keywords for the repptis task
- Keyword steps
- Keyword interfaces
- Keyword priority_shooting
- The pptis task
- Keywords for the pptis task
- Keyword steps
- Keyword interfaces
- Keyword priority_shooting
- The explore task
- Keywords for the explore task
- Keyword steps
- Keyword interfaces
- The md-flux task
- Keywords for the md-flux task
- Keyword steps
- Keyword interfaces
- The md-nve task
- Keywords for the md-nve task
- Keyword steps
- The umbrellawindow task
- Keywords for the umbrellawindow task
- Keyword umbrella
- Keyword overlap
- Keyword maxdx
- Keyword mincycle
- Common keywords
- Keyword task
- Keyword endcycle
- Keyword exe_path
- Keyword restart
- Keyword startcycle
- Path sampling keywords
- Keyword flux
- Keyword zero_left
- Keyword zero_ensemble
- Keyword permeability
- Keyword zeroswap
- Keyword pick_scheme
- Keyword ensemble_engines
- Keyword load_dir
## The retis task
The retis task defines a replica exchange transition interface sampling simulation.
```toml
[simulation]
task = "retis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
### Keywords for the retis task
For the retis task, the following keywords can be set:
### Keywords for the retis task.
Keyword
Description
steps
The number of total retis steps (cycles)
interfaces
The location of the interfaces to consider.
priority_shooting
Prioritize the ensembles with fewer moves.
In addition, this task requires that the sections TIS and RETIS are defined.
#### Keyword steps
The steps keyword defines the number of RETIS cycles to perform. Note that it indicates the goal/final number of the simulation cycles.
Not any, this keyword must be specified.
#### Keyword interfaces
The interfaces keyword specifies the interfaces to use in the path simulation.
Not any, this keyword must be specified.
interfaces is topology-defining: it is locked once the simulation is started. A restart that supplies a different list aborts with a ValueError — use load or flexible_restart to change the interface set.
#### Keyword priority_shooting
If True, the scheduler weights the choice of each move’s ensemble toward the ensembles that have received fewer moves. A move is a pair: the ensemble whose rules it follows (its interfaces and its acceptance) and one of the paths that ensemble holds, drawn by its infinite-swap occupancy. priority_shooting changes only the first choice: an ensemble that has received n moves, while the most-sampled ensemble has received n_max, is chosen with weight n_max - n + 1. The ensembles that lag are shot more often until all have received the same number of moves; level ensembles are equally likely. The path is then drawn from the chosen ensemble as usual.
A move counts when it is submitted, accepted or not, and a zero swap counts for both of its ensembles. The weights therefore depend on the moves made so far, and not on the current paths or on how long the moves take to finish with several workers, so every move still leaves the sampled distribution unchanged: only the allocation of moves, and with it the statistical error of each ensemble, changes.
The counts are kept in output.toml as [current] moves_submitted, so the ensembles stay level over a series of short jobs, each one a restart of the previous one. A restart of a run that did not record them starts from the attempted moves in each ensemble’s moves.txt. priority_shooting cannot be combined with relative_shoots or pick_scheme.
The default value is priority_shooting = False.
## The tis task
The tis task defines a transition interface sampling simulation.
```toml
[simulation]
task = "tis"
steps = 20000
interfaces = [
-0.9,
-0.8,
-0.7,
-0.6,
-0.5,
-0.4,
-0.3,
1.0,
]
```
```toml
[simulation]
task = "tis"
steps = 20000
interfaces = [
-0.9,
-0.9,
1.0,
]
[tis]
detect = -0.8
ensemble_number = 1
```
### Keywords for the tis task
For the tis task, the following keywords can be set:
### Keywords for the tis task.
Keyword
Description
steps
The total number of steps to perform.
interfaces
The interfaces defining the TIS ensemble.
priority_shooting
Prioritize the ensembles with fewer moves.
ensemble_number
For defining the ensemble number considered.
detect
The interface used for detecting successful paths.
maxlegth
The maximum number of step allowed for a paths.
In addition, this task requires that the section TIS is defined.
#### Keyword steps
The steps keyword defines the number of TIS cycles to perform.
Not any, this keyword must be specified.
#### Keyword interfaces
The interfaces keyword specifies the interfaces to use in the path simulation. Three interfaces are the left, middle and right interfaces of one ensemble, which a single TIS simulation samples. Two, or four or more interfaces define the RETIS ensembles [0^-], [0^+], [1^+], … of the interfaces, which are sampled without swaps; pyretis run refuses such an input with ensemble_number, which names one ensemble. For such a run, pyretis analyse with the default [analysis] method (matched) runs the RETIS analysis and writes the RETIS report (retis_report), as for a retis run: the crossing probability of each [i^+] ensemble, the matched crossing probability, the initial flux from the path lengths of the [0^-] and the [0^+], and the rate constant. The run attempts no swaps, so the report gives no swap acceptance ratio (n/a). With method = "wham" it writes wham_analysis.txt, with the point-matching and WHAM crossing probabilities, the initial flux and the point-matching and WHAM rate constants, and with method = "both" it writes the two.
Not any, this keyword must be specified.
interfaces is topology-defining: it is locked once the simulation is started. A restart that supplies a different list aborts with a ValueError — use load or flexible_restart to change the interface set.
#### Keyword priority_shooting
If True, the scheduler weights the choice of each move’s ensemble toward the ensembles that have received fewer moves. A move is a pair: the ensemble whose rules it follows (its interfaces and its acceptance) and one of the paths that ensemble holds, drawn by its infinite-swap occupancy. priority_shooting changes only the first choice: an ensemble that has received n moves, while the most-sampled ensemble has received n_max, is chosen with weight n_max - n + 1. The ensembles that lag are shot more often until all have received the same number of moves; level ensembles are equally likely. The path is then drawn from the chosen ensemble as usual.
A move counts when it is submitted, accepted or not, and a zero swap counts for both of its ensembles. The weights therefore depend on the moves made so far, and not on the current paths or on how long the moves take to finish with several workers, so every move still leaves the sampled distribution unchanged: only the allocation of moves, and with it the statistical error of each ensemble, changes.
The counts are kept in output.toml as [current] moves_submitted, so the ensembles stay level over a series of short jobs, each one a restart of the previous one. A restart of a run that did not record them starts from the attempted moves in each ensemble’s moves.txt. priority_shooting cannot be combined with relative_shoots or pick_scheme.
The default value is priority_shooting = False.
#### Keyword ensemble_number
The number of the one ensemble a single TIS simulation (a tis task with three interfaces) samples: ensemble_number = 1 names the [0^+], 2 the [1^+] and so on. The number names the ensemble and its directory, and the three interfaces set its window. pyretis run refuses a number below 1: 0 names the [0^-], whose paths start on the right, and a negative number names no ensemble.
A tis input with two, or four or more interfaces and this keyword, as PyRETIS 3 wrote a single TIS input, is read as the one ensemble of that number, and pyretis analyse reads the output in its directory as that ensemble. pyretis run refuses such an input, because it samples every ensemble of those interfaces (interfaces): to sample one [i^+] ensemble, give its three interfaces, as make-tis-files writes them, and sample the [0^-] with task = "retis".
make-tis-files writes the number of each ensemble into the single TIS input it writes for it. The retis, repptis, pptis and explore tasks sample every ensemble they build from their interfaces, and make-tis-files writes an input for each: these tasks refuse the keyword, which takes no effect on them (see the keywords that take no effect). A value that is not an integer, such as true or 1.0, is refused for every task.
The default value is ensemble_number = 2.
#### Keyword detect
The interface at which the analysis of the one ensemble of a tis run with ensemble_number counts a crossing. Every other path-sampling run, and make-tis-files, refuses the keyword: the analysis of each of its ensembles counts a crossing at the interface after the middle one of the ensemble, and make-tis-files writes that interface into the input of each ensemble.
Not any.
## The repptis task
The repptis task defines a replica exchange partial path transition interface sampling simulation. Paths between adjacent pptis ensembles can be swapped using the replica exchange move.
```toml
[simulation]
task = "repptis"
steps = 20000
interfaces = [
-0.5,
-0.3,
0,
0.3,
0.5,
]
```
### Keywords for the repptis task
For the repptis task, the following keywords can be set:
### Keywords for the repptis task.
Keyword
Description
steps
The number of total repptis steps (cycles)
interfaces
The location of the interfaces to consider.
priority_shooting
Prioritize the ensembles with fewer moves.
In addition, this task requires that the sections TIS and RETIS are defined.
(RE)PPTIS simulations take almost all of their keywords from the TIS and RETIS sections, because almost all of the keywords are the same. The repptis section carries the one keyword of its own, memory.
#### Keyword steps
The steps keyword defines the number of REPPTIS cycles to perform. Note that it indicates the goal/final number of the simulation cycles.
Not any, this keyword must be specified.
#### Keyword interfaces
The interfaces keyword specifies the interfaces to use in the path simulation.
Not any, this keyword must be specified.
#### Keyword priority_shooting
If True, the scheduler weights the choice of each move’s ensemble toward the ensembles that have received fewer moves. A move is a pair: the ensemble whose rules it follows (its interfaces and its acceptance) and one of the paths that ensemble holds, drawn by its infinite-swap occupancy. priority_shooting changes only the first choice: an ensemble that has received n moves, while the most-sampled ensemble has received n_max, is chosen with weight n_max - n + 1. The ensembles that lag are shot more often until all have received the same number of moves; level ensembles are equally likely. The path is then drawn from the chosen ensemble as usual.
A move counts when it is submitted, accepted or not, and a zero swap counts for both of its ensembles. The weights therefore depend on the moves made so far, and not on the current paths or on how long the moves take to finish with several workers, so every move still leaves the sampled distribution unchanged: only the allocation of moves, and with it the statistical error of each ensemble, changes.
The counts are kept in output.toml as [current] moves_submitted, so the ensembles stay level over a series of short jobs, each one a restart of the previous one. A restart of a run that did not record them starts from the attempted moves in each ensemble’s moves.txt. priority_shooting cannot be combined with relative_shoots or pick_scheme.
The default value is priority_shooting = False.
## The pptis task
The pptis task defines a partial path transition interface sampling simulation. PPTIS ensembles are shorter than TIS ensembles, cutting the paths’ memory requirements. While for (RE)TIS the paths of [i^+] need to start from state A, cross interface i, and either commit to state B or return to state A, PPTIS paths of [i^+-] only need to start and end at interfaces i-1 or i+1, having crossed interface i.
```toml
[simulation]
task = "pptis"
steps = 20000
interfaces = [
-0.5,
-0.3,
0,
0.3,
0.5,
]
```
### Keywords for the pptis task
For the pptis task, the following keywords can be set:
### Keywords for the pptis task.
Keyword
Description
steps
The total number of steps to perform.
interfaces
The interfaces defining the PPTIS ensemble.
priority_shooting
Prioritize the ensembles with fewer moves.
In addition, this task requires that the section TIS is defined.
PPTIS simulations take their other keywords from the TIS section. The pptis section carries memory.
#### Keyword steps
The steps keyword defines the number of PPTIS cycles to perform.
Not any, this keyword must be specified.
#### Keyword interfaces
The interfaces keyword specifies the interfaces to use in the path simulation. If the number of interfaces given is 3 or less, a single PPTIS simulation will be performed, otherwise, input files for several single PPTIS simulations will be written. These simulations can then be run manually.
Not any, this keyword must be specified.
#### Keyword priority_shooting
If True, the scheduler weights the choice of each move’s ensemble toward the ensembles that have received fewer moves. A move is a pair: the ensemble whose rules it follows (its interfaces and its acceptance) and one of the paths that ensemble holds, drawn by its infinite-swap occupancy. priority_shooting changes only the first choice: an ensemble that has received n moves, while the most-sampled ensemble has received n_max, is chosen with weight n_max - n + 1. The ensembles that lag are shot more often until all have received the same number of moves; level ensembles are equally likely. The path is then drawn from the chosen ensemble as usual.
A move counts when it is submitted, accepted or not, and a zero swap counts for both of its ensembles. The weights therefore depend on the moves made so far, and not on the current paths or on how long the moves take to finish with several workers, so every move still leaves the sampled distribution unchanged: only the allocation of moves, and with it the statistical error of each ensemble, changes.
The counts are kept in output.toml as [current] moves_submitted, so the ensembles stay level over a series of short jobs, each one a restart of the previous one. A restart of a run that did not record them starts from the attempted moves in each ensemble’s moves.txt. priority_shooting cannot be combined with relative_shoots or pick_scheme.
The default value is priority_shooting = False.
## The explore task
The explore task is a path sampling simulation designed to map the free energy landscape of a transition without computing rates. It is most useful before a production TIS/RETIS run, to locate local minima, check that paths can cross all interfaces, and generate a pool of initial paths to seed those simulations.
During an explore run every generated trajectory is accepted unconditionally (path status EXP). This means the simulation does not satisfy detailed balance and must not be used to compute crossing probabilities or rate constants.
Two tis-section settings are overridden automatically regardless of what the input file specifies:
- freq is forced to 0 — only shooting moves are performed, no time-reversal moves.
- allowmaxlength is forced to True — paths that reach maxlegth are treated as valid and accepted.
Setting a short maxlength in the tis section therefore controls how quickly the simulation restarts from a new shooting point and hence how densely the landscape is sampled.
```toml
[simulation]
task = "explore"
steps = 5000
interfaces = [
-1.9,
-0.7,
-0.5,
0.3,
]
[tis]
freq = 0.5
maxlength = 100
allowmaxlength = false
```
freq and allowmaxlength in the tis section are shown here for completeness but are ignored at run time — explore always sets them to 0 and True respectively.
### Keywords for the explore task
For the explore task, the following keywords can be set:
### Keywords for the explore task.
Keyword
Description
steps
The total number of shooting steps to perform.
interfaces
The interfaces defining the exploration region.
maxlegth
Maximum path length; shorter values restart the trajectory more often and improve landscape coverage.
In addition, this task requires that the section TIS is defined.
#### Keyword steps
The steps keyword defines the number of shooting cycles to perform.
Not any, this keyword must be specified.
#### Keyword interfaces
The interfaces keyword specifies the interfaces that define the exploration region. The first interface acts as the left boundary and the last interface as the right boundary. The path sampler will attempt to generate trajectories that span this region.
The number of interfaces determines the number of path ensembles created: N interfaces yield N - 1 ensembles, analogous to a TIS setup, so an explore task needs two interfaces or more. A larger number of intermediate interfaces gives finer spatial resolution but requires more initial paths.
Not any, this keyword must be specified.
explore is intended as a preparatory step. Once a satisfactory set of paths has been collected, switch task to tis, retis, pptis, or repptis and restart from the saved paths to begin a production run. See the initial-path section for how to load frames from a previous explore run.
The explorer example test in examples-from-tests demonstrates a complete explore run on the 1D double-well potential.
## The md-flux task
The md-flux task is a molecular dynamics simulation for determining the initial flux for a TIS path simulation.
```toml
[simulation]
task = "md-flux"
steps = 10000000
interfaces = [
-0.9,
]
```
### Keywords for the md-flux task
For the md-flux task, the following keywords can be specified:
### Keywords for the md-flux task
Keyword
Description
steps
Defines the number of steps to carry out.
interfaces
Defines the interfaces to consider for the flux simulation.
#### Keyword steps
The steps keyword specifies the number of MD steps to perform.
Not any, this keyword must be specified.
#### Keyword interfaces
The interfaces keyword specifies for which interfaces the initial flux should be obtained. This can be given as a list of floats.
Not any, this keyword must be specified.
## The md-nve task
The md-nve task is a NVE molecular dynamics simulation.
```toml
[simulation]
task = "md-nve"
steps = 10000000
```
### Keywords for the md-nve task
The following keywords can be specified for the md-nve task:
### Keywords for the md-nve task
Keyword
Description
steps
The number of overall steps to consider for the simulation.
#### Keyword steps
The steps keyword specifies the number of MD steps to perform.
Not any, this keyword must be specified.
## The umbrellawindow task
The umbrellawindow task runs one window of an umbrella sampling simulation as a Monte Carlo walk. umbrella gives the window it covers and maxdx bounds a single trial move. The window finishes once it has run mincycle cycles and every particle has passed overlap, so several windows joined together cover the whole order-parameter range.
```toml
[simulation]
task = "umbrellawindow"
umbrella = [-1.0, -0.4]
overlap = -0.5
maxdx = 0.1
mincycle = 10000
```
### Keywords for the umbrellawindow task
All four keywords have to be set for this task.
#### Keyword umbrella
The two order-parameter values of the window this simulation covers. It is recorded with the simulation and written to the restart file. A path-sampling run refuses it, as it does overlap, maxdx and mincycle: they take no effect on it.
Not any, this keyword must be specified.
#### Keyword overlap
Every particle has to pass this order-parameter value before the window is allowed to finish, which is what makes consecutive windows overlap.
Not any, this keyword must be specified.
#### Keyword maxdx
The largest displacement one Monte Carlo trial move may make.
Not any, this keyword must be specified.
#### Keyword mincycle
The number of cycles the window runs before it is allowed to finish.
Not any, this keyword must be specified.
## Common keywords
The following keywords are common to all simulation tasks:
## Common keywords
Keyword
Description
task
Selects the simulation to run.
endcycle
Specifies the cycle step the simulation ended.
exe_path
Specifies the directory from where the simulation was executed.
restart
Specifies the restart file to use.
startcycle
Specifies the cycle step the simulation should start at.
### Keyword task
Selects the simulation to run. The table of possible values at the top of this page lists the tasks, the keywords each one takes and the sections each one requires.
The default value is "md".
### Keyword endcycle
The endcycle keyword specifies the cycle number at which the simulation ended. If the simulation was stopped before the specified number of steps were reached, the endcycle will be difference from the steps keyword. This keyword will be set and updated by PyRETIS and written to the processed input file of an in-process task. A path-sampling run refuses it, as it does startcycle: the scheduler counts the cycles of a run in the [current] state of output.toml.
Not any, note that this keyword will be updated and modified by PyRETIS as part of the output process. This keyword is only used by the analysis program.
### Keyword exe_path
The exe_path keyword specifies the location from where the simulation was executed. This will be set and updated by PyRETIS.
Not any, note that this keyword will be updated and modified by PyRETIS as part of the output process. This keyword does not need to be set in the input file by the user since PyRETIS will update this setting automatically.
### Keyword restart
The restart keyword specifies the path to the restart file of an in-process task (md, …) to use for restarting/continuing it. To be used, it requires that method keyword in the initial-path section is set to restart. A path-sampling run reads no restart file: [initial-path] method = "restart" continues it from the output.toml of its run directory, so it refuses this keyword, with a message that names that method.
The default value is pyretis.restart.
### Keyword startcycle
The startcycle keyword specifies the cycle number the simulation starts at. This can be used, for instance, when extending a simulation to tell PyRETIS that the simulation should start at a specified step number. A path-sampling run refuses it: the scheduler counts the cycles of a run in the [current] state of output.toml.
The default value is 0.
## Path sampling keywords
These keywords shape the ensembles a path sampling task samples, and what the analysis reports for them.
## Path sampling keywords
Keyword
Description
flux
Sample the initial flux.
zero_left
Place the left boundary of the minus ensemble.
zero_ensemble
Select whether the plus-zero ensemble is built.
permeability
Report the permeability.
zeroswap
Refused: [retis] swapfreq sets the swap probability.
pick_scheme
Weight the scheduler’s choice of ensemble.
load_dir
Name the directory initial paths are loaded from.
ensemble_engines
Name the engine sections each ensemble runs with.
### Keyword flux
Builds the [0^-] ensemble of a pptis task, as zero_left does, and pyretis analyse reports the flux through the first interface. A retis or repptis task always samples the [0^-] and the [0^+], whose path lengths give the flux. A tis task with two, or four or more interfaces samples them as well, and pyretis analyse reports its flux and its rate as for a retis task (interfaces). A single TIS run (a tis task with three interfaces) and an explore task sample no [0^-]. For all of these, and make-tis-files, whose single TIS inputs sample no [0^-], flux takes no effect on the ensembles, and they refuse it, whatever its value (see the keywords that take no effect).
The default value is false for pptis.
### Keyword zero_left
The order-parameter value bounding the [0^-] ensemble on the left: the interfaces of the ensemble then run from this value to the first interface. Without it the ensemble reaches to minus infinity on the left. In a pptis task, setting it builds the [0^-]. A retis or repptis task, and a tis task with two, or four or more interfaces, build the [0^-] in any case, and zero_left bounds it. A single TIS run and an explore task sample no [0^-], and zero_left has no effect there. A make-tis-files task refuses it.
In a run that measures the permeability (permeability), the [0^-] is the [0^-'] ensemble, whose paths start and end at either interface, and the analysis corrects the flux for the paths that end at zero_left (the factor \xi). In a rate calculation, the [0^-] takes its paths from the first interface only: a trial path that reaches zero_left is rejected (0-L, or BWI when it starts there, as a shooting move’s backward part that reaches it does), and the flux is computed from the paths that turn back before it. Those are the shorter excursions into state A, so the flux comes out too high when many trials reach zero_left; pyretis analyse warns when more than 1 % of the [0^-] trials were rejected there. Place zero_left where the [0^-] paths seldom reach it, or measure the permeability.
No default.
### Keyword zero_ensemble
Selects whether a pptis task builds the [0^+] ensemble, and whether make-tis-files writes an input for it. Setting it to false there leaves that ensemble out, and the sampling starts at [1^+]. A retis or repptis task, a tis task with two, or four or more interfaces and an explore task always sample the [0^+]. A single TIS run samples the one ensemble of its three interfaces. For all of these, zero_ensemble takes no effect on the ensembles, and they refuse it, whatever its value.
A pptis task with a [0^-] ensemble (zero_left or flux) needs the [0^+] as well, because the flux combines the two, so such an input without zero_ensemble = true is refused. With zero_ensemble = true it measures the flux and the rate, and a zero_left sets the left interface of its [0^-]. A pptis task without either ensemble reports the crossing probability P(\lambda_N \mid \lambda_1) of its body ensembles, and no flux or rate.
The default value is false for pptis and make-tis-files; the windows of a pptis task are then all body windows unless this keyword asks for the zero ensemble.
### Keyword permeability
Reports the permeability: pyretis analyse also computes \xi, \frac{\tau}{dz} and the permeability, and the middle interface of the [0^-] ensemble is placed midway between zero_left and the first interface.
No default.
### Keyword zeroswap
[retis] swapfreq is the one keyword of the swap probability, so every task refuses [simulation] zeroswap, with a message that names [retis] swapfreq. A run file output.toml of an earlier PyRETIS that holds it is read with its value as [retis] swapfreq, which it won over.
### Keyword pick_scheme
An exponent applied to the ensemble weights when the scheduler picks which ensemble to move next, which biases the choice towards the outer ensembles. It cannot be combined with priority_shooting.
The default value is 0, which leaves the choice unweighted.
### Keyword ensemble_engines
An engine pool: one list of engine sections per interface, as [tis] shooting_moves gives one move per interface. The scheduler runs the MD of each ensemble with the engines of its list, and a shooting or wire-fencing move with the first one. A list names [engine], [engine0] and the numbered engine sections [engine1], [engine2], … of the input, each of which gives a class and the keyword arguments of its engine, as [engine] does. The engine of a section runs with the settings of that section and takes none from [engine]: an internal integrator (Langevin, VelocityVerlet, Verlet) takes timestep and subcycles from its own section, and stores every integration step when the section leaves subcycles out. The input is refused when a list names a section it does not define with a class, when it gives another number of lists than interfaces, and when no list names one of its numbered sections. [engine] is the engine the in-process kick initiation generates every initial path with; the parallel kick phase ([initial-path] kick-parallel = true) generates the initial path of each ensemble with the first engine of its list. A continuation keeps the engine sections of each ensemble, the class of each engine section other than [engine], and the locked engine settings (timestep, subcycles, temperature and gmx_format) of every engine section an ensemble runs with, each compared on the value the engine runs with: the value of the section, or the default of the engine when the section leaves the setting out. When no ensemble runs with [engine], a continuation keeps its timestep and subcycles only: the analysis converts the path lengths of the run to time with them, and a continuation kicks no path.
Every engine section of the run has one time per step, timestep times subcycles, with subcycles 1 when a section leaves it out: [engine], each section the lists name, and the [engine0] of a quantis run. Under infinite swapping a path held by one ensemble can have been generated by the engine of another ensemble, and a zero swap builds a path from segments of two engines, so the path lengths, counted in steps, convert to time with one time per step only. The engines may differ in every other setting, the split of the time per step into timestep and subcycles among them. A LAMMPS section that leaves timestep out has the timestep of its lammps.in. When two sections have different times per step, the run is refused when its input is read, before an engine runs, and the message names each section and its time per step. A section whose time per step cannot be read from the input, an OpenMM engine or an engine imported from a module without timestep in its section, is compared with no other section; when no section’s time per step can be read, the run is not refused. The analysis converts the path lengths with the time per step of [engine], the time per step of every section (see the initial flux).
Every run records subcycles_rule = 1 in the [current] provenance of its output.toml. A state file without it was written by a run whose internal integrators all streamed with the subcycles of [engine]. A continuation with [initial-path] method = "restart" compares the subcycles of another section that holds an internal integrator as that value, and its refusal says so; pyretis run -i output.toml of such a state is refused when an internal integrator of another section would stream with other subcycles, and the refusal names the value that resumes as that run streamed, which the section of output.toml can state. For a section with an engine imported from a module whose subcycles is not that value, the comparison is reported as UNKNOWN in a warning.
```toml
[simulation]
task = "retis"
interfaces = [0.345, 0.3625, 1.200]
ensemble_engines = [["engine0"], ["engine1"], ["engine1", "engine2"]]
```
No default: every ensemble runs with [engine], and the [0^-] ensemble of a quantis run with [engine0].
### Keyword load_dir
Names the per-ensemble directory the initial paths are read from, which is also the name of the store the accepted paths are kept in.
A path staged flat, in / of the run directory, is input and is only read. When at least one initial path is staged there, a fresh run reads every initial path from these directories and keeps a copy of each in the per-ensemble store, //. The copy holds files of its own, written byte for byte, and the run reads the frames of the path from it; the order values a load evaluates are written into the copy. The copy takes as much disk space as the staged files, and the log line of each copy gives its size in bytes. Rewriting a staged file afterwards, in place or under a new name, leaves the copy as it was. The copy holds the files of the staged directory and of its accepted/ subdirectory, so a staged traj.txt that names a frame file anywhere else stops the run with a ValueError before the load copies anything. A restart reads the path from the copy, as does a run resumed from the output.toml written before its first cycle completed. Such a fresh run stops before it reads or writes anything, and names the directories, when an initial path has no staged directory, or when the run directory holds, left by an earlier run: a directory of the per-ensemble store for one of the initial paths, a pathensemble.txt in one of its ensemble directories, or the CP2K wavefunction store pyretis-cp2k-wfn.
With no initial path staged flat, a fresh run reads its initial paths from the per-ensemble store: the paths pyretis run generates there from [initial-path], the paths the interface optimizer seeds there for each of its steps, and paths a user places there. The run reads these paths in place: their directories become the run’s own, the order values the load evaluates are written into their order.txt, and the run may move one it has replaced to /archive/ or remove it. pyretis run stops before its initiation when the run directory holds a staged initial path, a directory of the per-ensemble store for one of its initial paths, or a pyretis-cp2k-wfn. A run of a legacy-runner input runs no engine before its load. It stops before it starts when the run directory holds the state file of a simulation, an output.toml or restart.toml with a [current] section, or a pyretis-cp2k-wfn. pyretis run -i output.toml continues the simulation of an output.toml that records the input of the run, and a state file of an earlier PyRETIS, with a [simulation.tis_set] table, is continued by a canonical input of the run with method = “restart”. The run of a legacy-runner input records the canonical form of the input in output.toml of the run directory, and stops as well when its input file is that output.toml; rename such an input file. The interface optimizer stops before its first step when a step directory of the run holds any file. Every fresh run other than one started by pyretis run stops before its load when one of its ensemble directories holds a pathensemble.txt, with data rows (see the restart method) or with the header alone, which the scheduler writes once the load of a run has read its initial paths; it names those directories and a pyretis-cp2k-wfn beside them. pyretis run stops before its initiation when a pathensemble.txt holds data rows, and writes a pathensemble.txt with the header alone afresh.
A load, a restart included, reads the energies of a path from the energy.txt beside its traj.txt, one row per frame. An energy.txt with another number of rows stops the run with a ValueError that names the file; remove it, or replace it with the energies of the path.
The paths an earlier run left in the per-ensemble store before its load wrote the ensemble output are laid out as the paths a user places there: pyretis run leaves the paths it generated there when it stops between its initiation and its load. With no output.toml of that run in the run directory, a fresh run of a legacy-runner input, or a scheduler run started from the Python interface (pyretis.bin.pyretisrun.run_infinite_swapping()), with no path staged flat reads them as its initial paths. A scheduler run started from the Python interface may follow an initiation run from Python in the same directory, and starts its CP2K SCF from a pyretis-cp2k-wfn it finds there. Remove those directories, or use a new run directory, before such a run. pyretis tools clean leaves the flat store that a TOML input names in place.
The default value is "accepted".
---
# The System section
The system section defines some properties of the system.
```toml
[system]
units = "reduced"
dimensions = 2
temperature = 1.0
```
## Keywords for the System section
The following keywords can be set for the System section:
## Keywords for the System section
Keyword
Description
dimensions
Specify the dimensionality of the system.
temperature
Specify a set temperature for the system.
units
Sepcify the unit system to use.
### Keyword dimensions
Specify the dimensionality of the system. Should be 1, 2, or 3 for 1D, 2D or 3D, respectively.
The default value is dimensions = 3.
### Keyword temperature
Specify a set temperature for the system. This temperature, T, is used:
- to obtain \beta = 1/(k_\text{B} \times T) where k_\text{B} is the Boltzmann constant for the system.
- as a target temperature for velocity generation if no temperature is given in the velocity keyword of the particles section.
- as the target temperature for the Python Maxwell-Boltzmann draw when an external engine sets velocity_generation = maxwell (otherwise the external engine draws its own velocities; this is the default).
The default value is temperature = 1.0.
### Keyword units
The units keyword defines the system of units to use by PyRETIS. Currently, the following system of units are defined by PyRETIS (see the definitions of unit systems):
- lj: A Lennard-Jones type of (reduced) units (based on Argon 1).
- real: A system of units similar to the LAMMPS 2 unit real.
- metal: A system of units similar to the LAMMPS 2 unit metal.
- au: Atomic units. 3
- electron: A system of units similar to the LAMMPS 2 unit electron.
- si: A system of units similar to the LAMMPS 2 unit si.
- gromacs: A system of units similar to the units used by GROMACS. 4
- ase: eV, Å and fs, the energy and length units of ASE and the unit of the time step of the ASE engine.
The system of units should be one of the systems defined by PyRETIS listed above. Alternatively, you can define your own unit system by making use of a special unit-system section in combination with the units keyword.
The default is the unit system of the engine: lj for the internal integrators, gromacs for GROMACS, openmm for OpenMM, cp2k for CP2K, reduced for TurtleMD, ase for ASE, and the units command of the LAMMPS input for LAMMPS.
## References
L. A. Rowley, D. Nicholson and N. G. Parsonage, Monte Carlo grand canonical ensemble calculation in a gas-liquid transition region for 12-6 Argon, J. Comput. Phys. 17, 401-414 (1975), https://doi.org/10.1016/0021-9991(75)90042-X
The LAMMPS manual, http://lammps.sandia.gov/doc/units.html
https://en.wikipedia.org/wiki/Atomic_units
The GROMACS manual, version 5.1.1, tables 2.1 and 2.2 on page. 8, http://manual.gromacs.org/documentation/5.1.1/manual-5.1.1.pdf
---
# The TIS section
The TIS section defines settings for (PP)TIS and (PP)RETIS simulations.
```toml
[tis]
allowmaxlength = false
freq = 0.5
high_accept = true
maxlength = 20000
n_jumps = 8
rgen = "rgen"
rescale_energy = false
seed = 1
shooting_moves = [
"sh",
]
sigma_v = -1
zero_momentum = false
```
## Keywords for the TIS section
The following keywords can be set for the TIS section:
## Keywords for the TIS section, commonly set
Keyword
Description
allowmaxlength
Select whether shooting moves may use the full maxlength cap directly.
freq
Define how often time reversal moves are performed.
high_accept
Set the Stone Skipping version to use.
maxlength
Set the maximum length of the paths generated.
n_jumps
Set the number of jumps for SS, WT and WF moves.
rgen
Set the random number generator to use.
seed
Set a seed for the random number generator.
rescale_energy
Selects re-scaling of velocities.
shooting_moves
Set the MC shooting moves in each ensemble.
sigma_v
Set the velocity-perturbation width, or select aimless shooting when negative.
zero_momentum
Specify whether momentum should be set to zero after each velocity draw.
The remaining keywords of the section tune a particular move, the initiation or the infinite-swapping scheduler:
## Keywords for the TIS section, tuning
Keyword
Description
enforce_must_cross_m
Require a trial to cross the middle interface in the ensembles that define one.
interface_sour
Sets the position of SOUR for Web Throwing.
mwf_subcycle_small
Set the subcycle count the intermediate mwf sub-paths are shot with.
mwf_subcycle_small_by_ensemble
Override mwf_subcycle_small for named ensembles.
mwf_nsubpath
Set how many sub-paths one mwf set holds.
shooting_move
Set one MC shooting move for every ensemble.
aimless
Deprecated; the sign of sigma_v selects aimless shooting.
kick_maxtries
Bound the MD advances one kick may spend.
initial_path_maxtries
Bound the attempts spent generating an initial path.
initial_path_fix_maxtries
Bound the attempts spent repairing an initial path.
mirror_freq
Set how often the mirror move is performed.
target_freq
Set how often the target swap move is performed.
target_indices
List the order-parameter indices a target swap may switch to.
perm_threshold
Set the largest block the permanent is computed exactly for.
perm_n_samples
Set the sample count of the stochastic permanent.
exact_perm_only
Require the exact permanent.
quantis
Select the scheduler’s quantis zero-ensemble move.
accept_all
Skip the Metropolis acceptance of the quantis move.
interface_cap
Set the cap value for subpaths generated by the WF move.
### Keyword allowmaxlength
Controls the length cap used when generating trial paths with shooting moves. If True, the trial path may use the full value given by keyword maxlength as its hard cap. If False, PyRETIS first draws a stochastic cap based on the length of the current path and then also limits it by maxlength.
This keyword does not force all paths to have the same length. The propagation still stops when the path reaches the required interface, so paths may be shorter than maxlength. The False setting is the default for regular path sampling because the random cap is part of the detailed-balance treatment for variable-length shooting paths. During some initialization and sub-path steps, PyRETIS may set this option internally to True.
The default is allowmaxlength = False.
### Keyword freq
This defines how often time reversal moves are performed: E.g. if freq = 0.4 then 40% of the TIS moves are time reversal. Note that if you are running a RETIS simulation, then the percentage of TIS moves will be modified by the percentage of swapping moves, e.g. the percentage of shooting moves will then be given by (1 - swapfreq) \times (1 - freq).
No default, this keyword must be specified.
### Keyword interface_sour
This defines the position of the SOUR interface for Web Throwing. It HAS to be smaller then the interface defining the ensemble where WT is used. Note that an improper selection of interface_sour will hinder the sampling efficiency.
No default, this keyword must be specified if WT is used.
### Keyword high_accept
Selects the acceptance rule of stone skipping (ss). With high_accept = True a trial path is accepted and carries a weight, which the swaps and the analysis apply (the HA-weight column of pathensemble.txt); with high_accept = False stone skipping uses Metropolis acceptance on the crossing count. High acceptance is the more efficient of the two. Wire fencing (wf) always uses high acceptance and does not read this keyword; web throwing (wt) keeps its own segment-ratio acceptance, and standard shooting (sh) never reads it. Note that the current implementation, due to detailed balance, DOES NOT ALLOW to have multiple shooting methods in the same ensemble.
The same keyword and default apply to the deprecated legacy-runner schema, as high_accept in [simulation.tis_set], which its conversion to the canonical schema places in [tis]. A run records the value it samples with in the [tis] section of its output.toml; a resumed run keeps it.
The default is high_accept = True.
### Keyword quantis
Selects the infinite-swapping scheduler’s “quantis” zero-ensemble move: the [0^-]/[0^+] pair propagates from an independently configured [engine0] section (same keys as [engine]) instead of the default [engine]. Only meaningful for a scheduler-routed run; a no-op otherwise. An [engine0] section in a run in which no ensemble runs it is refused. [engine0] has the time per step of [engine], timestep times subcycles, and a run whose two sections differ is refused before it starts (see the engine section).
The default is quantis = False.
### Keyword accept_all
When quantis is set, skip the energy-based Metropolis acceptance rule for the resulting zero swap and accept it unconditionally. Only meaningful together with quantis; a no-op otherwise.
The default is accept_all = False.
### Keyword maxlength
This determines the maximum length of the paths generated. Ideally, no paths should be longer than this value. A value too restrictive can invalidate detailed balance.
A path may hold maxlength frames: a trajectory that crosses an interface on its last allowed frame is complete, and one that has not crossed by then is rejected (BTX or FTX). The shooting and swap moves build their paths under this ceiling, and loading or restarting admits paths of at most maxlength frames.
The default is maxlength = 20000, a deliberately generous cap so that an input which does not mention path lengths still samples correctly. Because a cap that is too small biases the sampling, a run that falls back to this default says so in its log, and the value it used is recorded in the resolved settings file the run writes. Set it explicitly for systems whose paths are expected to be long.
### Keyword n_jumps
The number of jumps for Stone Skipping, Wire Fencing and of web for Web Throwing.
No default for Stone Skipping and Web Throwing: this keyword must be specified if either is used, and an input that uses them without it is refused. Wire Fencing and Multiresolution Wire Fencing take 2 when the keyword is not set, which the run record output.toml of such a run writes as n_jumps = 2.
### Keyword rgen
The random number generator the in-process kick initiation draws the velocities of its kicks with: "pcg64" (numpy’s PCG64), "rgen" (numpy’s legacy MT19937), or a generator for tests ("mock", and the shared "rgen-borg" and "mock-borg"). The scheduler draws every move from streams of numpy’s PCG64 generator spawned from [simulation] seed, so a run that kicks no path in process (a load or a restart initiation, or a parallel kick) refuses a value other than "pcg64" (see the keywords that take no effect).
The default is rgen = "pcg64".
### Keyword rescale_energy
If this keyword is set to a positive number, then the velocities are re-scaled so that the total energy is equal to the given number. This is useful for performing NVE simulations. If the keyword is set to False, or to a number that is not positive, then the energies will not be re-scaled.
Every shooting move re-scales the velocities it draws: the shooting moves of every ensemble, including the shots of wire fencing and biased shooting and the one-step jumps of stone skipping, and the shots of the initiation. The internal integrators and OpenMM also re-scale each kick of the search for a crossing with the middle interface in the initiation; the kicks of the other engines keep the energy they are drawn with.
The internal integrators, OpenMM and cp2k_steps re-scale in PyRETIS; cp2k_steps needs the potential energy of the shooting point and stops with an error for a point that does not carry one. lammps_steps with velocity_generation = engine re-scales in LAMMPS, which evaluates the energy as its thermo keywords pe and ke do: per atom under thermo_modify norm yes, the LAMMPS default for lj units. GROMACS, AMS, TurtleMD, ASE, the streaming cp2k and lammps engines, and lammps_steps with velocity_generation = maxwell do not re-scale: a positive value stops the run with an error at the first shooting move, before any velocities are drawn.
A shooting point whose potential energy is above rescale_energy cannot be re-scaled to it: the internal integrators, OpenMM, cp2k_steps and lammps_steps log a WARNING and keep the energy of the draw. lammps_steps compares the pe of LAMMPS with rescale_energy in LAMMPS, and PyRETIS reads the outcome back from the generate_vel log of the engine.
A continuation with method = “restart” is refused when it re-scales the shooting velocities differently from the run it continues; see the draws a restart compares.
The default value is rescale_energy = False.
### Keyword seed
This integer is a seed for the random number generator used in the TIS algorithm (e.g. when selecting a shooting point).
The default is seed = 0.
### Keyword shooting_moves
This list contains the flags that determines the shooting move to be used for each ensemble. Usable moves are 'sh' for shooting, 'ss' for Stone Skipping, 'wt' for Web Throwing, 'wf' for Wire Fencing, 'mwf' for Multiresolution Wire Fencing, which shoots the intermediate wire-fencing sub-paths at a reduced subcycle count set by mwf_subcycle_small, and 'bias' for biased shooting-point selection. See user-guide-path-move-types for the full list of move type codes and their descriptions.
A partial-path task (task = "pptis" or task = "repptis") refuses the subtrajectory moves 'ss', 'wt', 'wf' and 'mwf'. Those moves build one complete path out of subpaths and weight it as a complete path. PPTIS and REPPTIS sample partial paths, each carrying weight 1, and rebuild the kinetics from the local crossing probabilities. Use 'sh' for a partial-path task.
The default value is [].
### Keyword enforce_must_cross_m
Requires a trial path to cross the middle interface before it counts as a member of an ensemble whose definition names that interface. Ensembles that define no middle interface are unaffected by this keyword.
The requirement holds when the keyword is absent, so a configuration carries a line for it to sample the permissive way: enforce_must_cross_m = false accepts a trial in those ensembles without the middle crossing.
The default value is true.
### Keyword mwf_subcycle_small
The number of engine subcycles the intermediate sub-paths of the Multiresolution Wire Fencing move are shot with. The final sub-path of each set uses the engine’s own subcycles. It has to be a positive integer.
One fifth of the engine’s subcycles, and at least 1.
### Keyword mwf_subcycle_small_by_ensemble
Maps an ensemble name to the subcycle count to use for that ensemble, taking precedence over mwf_subcycle_small there. An ensemble the dictionary does not name keeps mwf_subcycle_small.
No default.
### Keyword mwf_nsubpath
The number of sub-paths one Multiresolution Wire Fencing set holds. The number of sets is n_jumps.
The default value is 3.
### Keyword shooting_move
Sets the shooting move used in every ensemble, taking the move codes of shooting_moves. A per-ensemble shooting_moves list takes precedence over this keyword.
The default value is "sh".
### Keyword aimless
Deprecated. The sign of sigma_v selects aimless shooting in every engine; an aimless line is removed from the settings, with a notice when it agrees with sigma_v and a warning when it disagrees.
No default.
### Keyword kick_maxtries
Bounds the number of MD advances a single kick may spend reaching a valid shooting point. The engine stops with an error when the bound is reached.
The default value is 100000.
### Keyword initial_path_maxtries
Bounds the number of attempts the kick initiation may spend generating an initial path.
The default value is 1000.
### Keyword initial_path_fix_maxtries
Bounds the number of attempts the kick initiation may spend repairing an initial path.
The default value is 1000.
### Keyword mirror_freq
Sets how often the mirror move (mr) is performed. A run that uses it requires workers = 1. An accepted mirror move changes which side of the mirror the order parameter measures, and the path it produces carries that state into every ensemble it is swapped to.
The default value is 0, which does not use the move.
### Keyword target_freq
Sets how often the target swap move (ts) is performed. A run that uses it requires workers = 1. An accepted target swap changes which molecule the order parameter tracks, and the path it produces carries that state into every ensemble it is swapped to.
The default value is 0, which does not use the move.
### Keyword target_indices
The order-parameter target indices the target swap move may switch the order function to. The index the run starts from has to appear in this list.
The default value is [].
### Keyword perm_threshold
The largest ensemble block the infinite-swapping permanent is computed exactly for. A larger block is approximated stochastically, with perm_n_samples samples.
The default value is 12.
### Keyword perm_n_samples
The number of samples the stochastic permanent approximation draws for a block larger than perm_threshold.
The default value is 10000.
### Keyword exact_perm_only
Requires the exact permanent: a block larger than perm_threshold stops the run with an error instead of being approximated stochastically.
The default value is false.
### Keyword interface_cap
This defines the position of the cap interface for Wire Fencing. It HAS to be larger than the interface defining the ensemble where WF is used. Note that an inproper selection of interface_cap can hinder the sampling efficiency.
No default.
### Keyword sigma_v
Controls whether shooting is aimless and, for non-aimless shooting, sets the width of the velocity perturbation. If sigma_v is negative, PyRETIS uses aimless shooting: new velocities are drawn from the Maxwellian distribution using the system temperature and masses. The default sigma_v = -1 therefore selects aimless shooting.
If sigma_v is zero or positive, PyRETIS uses non-aimless shooting. The value is multiplied by sqrt(inverse mass) for each particle and used as the standard deviation for Gaussian velocity increments added to the current velocities. These momentum changes are then accepted or rejected with the Metropolis criterion.
The internal integrators, OpenMM and cp2k_steps draw both kinds of shooting. GROMACS, LAMMPS, AMS, TurtleMD, ASE and the streaming cp2k engine draw the velocities of aimless shooting only: with a zero or positive sigma_v a shooting move stops the run with an error before any velocities are drawn. OpenMM stops in the same way for a System with constraints: the Metropolis criterion uses the kinetic energy of every velocity component, including the components along the constraints that the integrator removes, and this breaks detailed balance for the Maxwellian distribution of the constrained velocities. OpenMM evaluates the criterion with the kinetic energy in kJ/mol and the inverse temperature in mol/kJ.
The default value is sigma_v = -1.
### Keyword zero_momentum
If this keyword is set to True, then the linear momentum of the system is set to zero after random velocities are drawn: for a shooting move (including the shots of wire fencing and the one-step jumps of stone skipping), and for every kick of the search for a crossing with the middle interface in the initiation. If False, the drawn velocities are used as drawn. The keyword has this meaning in every engine. Where the MD program draws the velocities itself, the setting is handed to it or checked against it:
- LAMMPS with class = lammps_steps and velocity_generation = engine draws with velocity create, whose mom keyword PyRETIS writes from it: mom yes for True and mom no for False. With class = lammps, PyRETIS draws the velocities in both velocity_generation modes and resets the momentum as this keyword selects.
- GROMACS with velocity_generation = engine draws with gen_vel, which removes the centre-of-mass motion, so it requires zero_momentum = True.
- AMS draws the velocities with GenerateVelocities, and PyRETIS uses them as drawn. With RandomVelocitiesMethod = Gromacs and Exact, the AMS default, the draws have zero net linear momentum (for Boltzmann this is not checked), so it requires zero_momentum = True.
GROMACS with velocity_generation = engine, and AMS, stop the run with an error before the first velocity draw for zero_momentum = False.
The paths the ensembles hold were shot with velocities drawn under this keyword. A continuation with method = “restart” is refused when its shooting velocities are drawn from another distribution than those of the run it continues, and pyretis run -i output.toml is refused for an output.toml that does not record its draws when the settings it records draw differently from the engine of that run; see the draws a restart compares.
The default value is zero_momentum = False.
## Biased shooting-point selection
The standard shooting move picks the shooting point uniformly among the interior frames of the current path. Setting shooting_move = "bias" (or listing 'bias' for an ensemble in shooting_moves) instead picks it non-uniformly, biased toward “interesting” frames – for example near the barrier top – which can reduce the number of MD steps needed to converge a rate.
Crucially, biased selection does not change the sampled path ensemble: the crossing probability and the rate are statistically identical to the uniform move. Only the sampling efficiency changes. This is guaranteed by an exact acceptance correction.
### How it works
A selector assigns a strictly positive weight w(x) to each interior frame. The shooting point is drawn with probability w_i / W, where W = \sum_j w_j is the sum over the interior frames of the current path. The trial path is then integrated to completion (up to maxlength; the stochastic length cap of the uniform move is not used), and a valid trial path is accepted with the Metropolis probability
where W_\mathrm{old} and W_\mathrm{new} are the interior weight sums of the old and the trial path. The weight of the shooting point itself cancels, because it is the same physical frame shared by both paths. With constant weights this reduces to the standard flexible-length two-way-shooting acceptance, so the scheme is correct by construction. A valid trial path rejected by this test is recorded with the status code SAR (selection-bias acceptance rejection).
This is distinct from the legacy 'exp' shooting heuristic, which is not detailed-balance corrected; prefer 'bias' when rigor matters.
### Restrictions
- Aimless only. Biased shooting requires aimless velocities (sigma_v < 0, the default), so that \min(1, W_\mathrm{old}/W_\mathrm{new}) is the sole acceptance correction. A non-negative sigma_v is rejected at set-up time.
- It biases the per-ensemble shooting move; swapping moves are unaffected.
### Where it runs
Biased shooting runs on the infinite-swapping scheduler – the single execution route for every path-sampling task – so you select it the same way for any run. A scheduler worker runs the biased move respecting the [0^-]/[0^+]/[i^+] ensemble structure, and because the move preserves the path ensemble the crossing-probability / WHAM / rate analysis is unchanged. Per-ensemble moves are set with shooting_moves (the [0^-] ensemble must use "sh"), e.g. shooting_moves = ["sh", "bias", "bias", ...].
A single [shooting-selector] section is shared by every biased ensemble.
### The [shooting-selector] section
When shooting_move = "bias" is used, a shooting-selector section selects and parameterises the weight function. Like the engine and orderparameter sections it accepts either a built-in class or a user module/class pair (see extending PyRETIS for writing your own).
Built-in selectors:
- gaussian (GaussianOrderSelector) – w(x) = \exp\!\big(-\alpha\,(\lambda(x) - \lambda_0)^2\big), a bias toward the order-parameter value lambda0 with sharpness alpha.
- uniform (UniformSelector) – constant weights; reproduces the standard uniform move through the explicit-acceptance scheme (useful as a reference/validation).
Example (TOML):
```
[simulation]
shooting_move = "bias"
[tis]
sigma_v = -1 # must be < 0 (aimless)
[shooting-selector]
class = "gaussian"
alpha = 50.0
lambda0 = 0.0 # order-parameter value to bias toward
```
---
# The unit-system section
The unit-system section is used in combination with the units keyword from the system section for defining custom system of units. When defining a custom unit system, the units keyword is used to select the new system and the unit-system section is defining the base units.
```toml
[system]
units = "new-system"
[unit-system]
name = "new-system"
length = [
1.0,
"bohr",
]
mass = [
9.81e-31,
"kg",
]
energy = [
1.0,
"kcal/mol",
]
charge = "e"
```
Note that the unit-system section is optional.
## Keywords for the unit-system section
For the unit-system section the scale for each dimension is specified using the keywords:
## Keywords for the unit-system section
Keyword
Description
length
Specify the scale and base unit for the length dimension.
mass
Specify the scale and base unit for the mass dimension.
energy
Specify the scale and base unit for the energy dimension.
charge
Specify the base unit for the charge dimension
### Keyword length
The length keyword specifies the scale and base unit for the length dimension. This is done on the form:
```toml
[unit-system]
length = [1.0, "baseunit"] # [scale, base unit]
```
For an overview of the possible base units, please see the description in the manual section describing units in PyRETIS. Also, note that the 'baseunit' string is case-sensitive!
None. This keyword must be specified if the unit-system section is used.
### Keyword mass
The mass keyword specifies the scale and base unit for the mass dimension. This is done on the form:
```toml
[unit-system]
mass = [1.0, "baseunit"] # [scale, base unit]
```
For an overview of the possible base units, please see the description in the manual section describing units in PyRETIS. Also, note that the 'baseunit' string is case-sensitive!
None. This keyword must be specified if the unit-system section is used.
### Keyword energy
The energy keyword specifies the scale and base unit for the energy dimension. This is done on the form:
```toml
[unit-system]
energy = [1.0, "baseunit"] # [scale, base unit]
```
For an overview of the possible base units, please see the description in the manual section describing units in PyRETIS. Also, note that the 'baseunit' string is case-sensitive!
None. This keyword must be specified if the unit-system section is used.
### Keyword charge
The charge keyword specifies base unit for the charge dimension. This is done on the form:
```toml
[unit-system]
charge = "baseunit"
```
For an overview of the possible base units, please see the description in the manual section describing units in PyRETIS. Also, note that the 'baseunit' string is case-sensitive!
None. This keyword must be specified if the unit-system section is used.
---
# Tutorials
In addition to reading the documentation, we have prepared several examples to highlight the usage and capabilities of PyRETIS. The cards below point to the user-facing tutorial folders that ship with the source tree. Larger published-result packages that regenerate figures from validation studies are collected on the results page. The complete tutorial list is in the example section.
The examples below are grouped by what you are most likely to change: the potential, the sampling method, the external dynamics engine, or the molecular dynamics workflow. Each card links to the rendered documentation and to the matching source folder under examples/tutorials/.
## Start here
A minimal input file
The shortest input that runs a complete RETIS simulation – only the physics, everything else defaulted.
Tutorial source
Minimal, external engine
The same, driving OpenMM: the engine owns the system, so the input is only what to sample and how to measure it.
Tutorial source
## Potentials
1D double well
The smallest TIS/RETIS model: one particle in the van Erp double-well benchmark.
Tutorial source
1D triple well
RETIS on a three-state landscape, including the zero_left shortcut for the [0-] ensemble.
Tutorial source
2D hysteresis
A two-dimensional internal potential where the best order parameter is deliberately less obvious.
Tutorial source
2D WCA bond
Bond breaking in the WCA model from van Erp, Moroni and Bolhuis (2003), with low- and high-barrier variants.
Tutorial source
## Sampling methods
TIS and flux
Initial flux plus TIS crossing probabilities for the 1D double-well system.
Tutorial source
RETIS
Replica exchange between path ensembles for the same double-well benchmark.
Tutorial source
Infinite swapping
Asynchronous, parallel replica exchange driven by the [runner] worker pool – the route for large rate-constant runs.
Tutorial source
Subtrajectory moves
Stone skipping, web throwing and wire fencing from Riccardi et al. (2017) and Zhang et al. (2023).
Tutorial source
Umbrella sampling
Monte Carlo umbrella sampling for free-energy estimation alongside the rate-constant tutorials.
Tutorial source
Permeability
A path-sampling setup using permeability = True for membrane-style transport calculations.
Tutorial source
Particle swarm
Particle-swarm optimisation for finding useful initial paths before path sampling.
Tutorial source
REPPTIS
Replica-exchange partial-path TIS on the 1D double well, with the Markov-state-model kinetics analysis.
Tutorial source
Exploring the landscape
Free-energy landscape exploration to generate a first valid initial path before path sampling.
Tutorial source
## External engines
GROMACS
Methane transport in an sI hydrate structure with a custom Python order parameter.
Tutorial source
LAMMPS
The 2D WCA bond-breaking model driven by LAMMPS instead of the internal engine.
Tutorial source
CP2K
A small hydrogen bond-breaking example using CP2K for the dynamics.
Tutorial source
OpenMM
RETIS for an argon model with OpenMM providing the molecular dynamics.
Tutorial source
Compiled extensions
C and FORTRAN extensions for custom integrators, potentials and order parameters.
Tutorial source
## Molecular dynamics
pyretis run MD
Plain molecular dynamics from an input file and from the PyRETIS Python API.
Tutorial source
Flux paths
Forward and backward MD utilities for generating path-sampling starting points.
Tutorial source
---
# PyRETIS units
PyRETIS simulations are always carried out with a consistent set of units. A system of units is defined by specifying the energy scale, E, the length scale, L, and the mass scale, M. This implicitly defines the time scale, T, as
The charges are typically specified in units of electron charges or in Coulombs. In PyRETIS the charges are internally scaled with the factor \frac{1}{\sqrt{4 \pi \varepsilon_0}} (where the units of \varepsilon_0 is consistent with the defined E, L and M). This means that the Coulomb constant is implicitly contained within the charges in the internal computations.
In PyRETIS, several systems of units are defined and you may also define your own units. The choice of a system of units will define the units used for input parameters. The different systems of units defined in PyRETIS are described in the following section and the creation of custom systems are also described. In the PyRETIS library, the units are defined in the pyretis.core.units module. The defined constants can be displayed by,
```python
from pyretis.core.units import CONSTANTS
print('All constants':)
print(CONSTANTS)
print('The Boltzmann constant:')
print(CONSTANTS['kB'])
```
## System of units in PyRETIS
The systems of units that are defined in PyRETIS are:
- lj: A Lennard-Jones type of units.
- real: A system of units similar to the LAMMPS 1 unit real.
- metal: A system of units similar to the LAMMPS 1 unit metal.
- au: Atomic units. 2
- electron: A system of units similar to the LAMMPS 1 unit electron.
- si: A system of units similar to the LAMMPS 1 unit si.
- gromacs: A system of units similar to the units used by GROMACS. 3
- ase: The energy and length units of ASE, eV and Å, and the femtosecond, the unit of the time step of the ASE engine, as the time unit. It is the default unit system of the ASE engine: the beta of the analysis is in 1/eV, and the fluxes and rates are per fs. The velocities the ASE engine hands to an order parameter are ASE’s, in Å per ASE time unit (10.18 fs).
The different bases for the different units are given in the table of definitions below.
The system of units is defined by setting the units keyword to any of these units, e.g.
```python
units = real
# or perhaps:
units = metal
# or even Lennard-Jones units:
units = lj
```
Setting the unit system will define the units used for input to PyRETIS. The units for the different systems are given in the table of input units below. Note that the time unit is determined by the energy, length and mass scale and that the input time step will be assumed to be given in internal units. Conversion factors to more familiar units are given in the table on time units.
If the initial configuration is given as a file (for instance in xyz format), then the system of units will be used to convert the configuration. That is, PyRETIS will assume that the configuration is given in the units specified by the file format (e.g. Ångström for positions in a xyz file). The coordinates read from initial configuration fill will be converted internally to the selected system of units.
## Input units for different systems
Unit system
Energy
Length
Mass
lj
reduced
reduced
reduced
real
kcal/mol
Å
g/mol
gromacs
kJ/mol
nm
g/mol
metal
eV
Å
g/mol
au
hartree
bohr
electron mass
electron
hartree
bohr
g/mol (amu)
si
J
m
kg
ase
eV
Å
eV fs²/Ų
For the lj system, the energy, length, mass, etc. are all given in reduced units. However, it still might be useful (for instance for trajectory output) to convert to other units. By default, the scales for the Lennard-Jones system of units corresponds to values for Argon. 4 These default units can be overridden explicitly by defining the length scale, mass scale and energy scale:
```python
units = lj
units-base = {'length': (3.405, 'A'), 'mass': (39.948, 'g/mol'),
'energy': (119.8, 'kB')}
```
However, again note that this will only influence how we interpret input and output configurations: all other output units for this system will still be in reduced Lennard-Jones units. The syntax for overriding and creating your own units are further described in the next section.
## Defining units for energy systems
Unit system
Energy unit
Length unit
Mass unit
lj
0.238 kcal/mol
0.3405 nm
39.948 g/mol
real
1 kcal/mol
1 Å
1 g/mol
gromacs
1 kJ/mol
1 nm
1 g/mol
metal
1 eV
1 Å
1 g/mol
au
1 hartree
1 bohr
9.10938291e-31 kg
electron
1 hartree
1 bohr
1 g/mol
si
1 J
1 m
1 kg
ase
1 eV
1 Å
1 eV fs²/Ų
These units are also used for the input and define the time unit. The time units are shown in the table below. Further, all system of units expect an input temperature in Kelvin (K) and all systems, except for si, expects a charge in units of electron charges. The si system uses here Coulomb as the unit of charge. The time units for the different energy systems are given in the table below.
## Time units and velocity conversions for energy systems
Unit system
Time unit
lj
2.15634977232 ps
real
48.8882129084 fs
gromacs
1.0 ps
metal
10.1805056505 fs
au
0.0241888423521 fs
electron
1.03274987345 fs
si
1.0 s
ase
1.0 fs
The interpretation here is that if you are for instance using the system real and would like to have a time step equal to 0.5 fs, then the input time step should be 0.5 fs / 48.8882129084 fs which is approximately 0.010227. Another example: If you are using the system metal and set your time step to 0.1, then the time step will be approximately 1.018 fs.
## Defining your own system of units
Defining your own system of units is basically just a matter of choosing the energy, length and mass scales. Typically, this is done by setting the units keyword and defining the base units in the section Unit-system:
```toml
[system]
units = "new-system"
[unit-system]
name = "new-system"
length = [
1.0,
"bohr",
]
mass = [
9.81e-31,
"kg",
]
energy = [
1.0,
"kcal/mol",
]
charge = "e"
```
Note again that this will influence how the input parameters are interpreted and that the input configuration files will be converted to the internal unit system. In the example above, a configuration read from a xyz-file will be converted from Ångström to Bohr for internal calculations. The syntax for specifying a new scale for a dimension is of form
```python
'dimension' = (value, unit)
```
Where the value is a number defining the scale and unit a text string which defines the unit to use. This text string is case-sensitive. For each dimension, PyRETIS defines a set of “known” units to choose from:
- A: Ångström.
- nm: Nanometre.
- bohr: Bohr radius.
- m: Meter.
- kcal: Kilocalorie.
- kcal/mol: Kilocalorie per mol.
- J: Joule.
- J/mol: Joule per mol.
- kJ/mol: Kilojoule per mol.
- eV: Electronvolt.
- hartree: Hartree (atomic unit of energy).
- g/mol: Grams per mol, numerically equal to the atomic mass unit.
- g: Gram.
- kg: Kilogram.
For the charge setting, there it is currently only possible to define the unit as either e (for the electron charge) or C (for Coulomb).
## References
The LAMMPS manual, http://lammps.sandia.gov/doc/units.html
https://en.wikipedia.org/wiki/Atomic_units
The GROMACS manual, tables 2.1 and 2.2 on page. 8, http://manual.gromacs.org/documentation/5.1.1/manual-5.1.1.pdf
L. A. Rowley, D. Nicholson and N. G. Parsonage, Monte Carlo grand canonical ensemble calculation in a gas-liquid transition region for 12-6 Argon, J. Comput. Phys. 17, 401-414 (1975), https://doi.org/10.1016/0021-9991(75)90042-X