.. _example-test-status:

###################
Example test status
###################

This page is the **canonical map** between tutorial folders,
documentation pages, and heavy-test fixtures. Every runnable
tutorial should either have a passing check here or a clear reason
why it is treated as an optional engine-dependent check. The same
information is referenced (but not duplicated) by
:ref:`examples-llm-quickstart` and the
:file:`examples/tutorials/README.rst` README, so updating this page
keeps the other two in sync.

Last refreshed
==============

The most recent automated refresh stamp is written by
:file:`examples/run-tutorials.sh` (when invoked with
``PYRETIS_WRITE_REFRESH_STAMP=1``) and is embedded below:

.. literalinclude:: /_static/examples/tutorials-last-refresh.txt
   :language: text

The internal-engine suite was run on the date above from a temporary
copy of the repository so that generated files in the working tree
were not modified. The status pills on the individual tutorial pages
("passing", "engine", "smoke", "manual") reflect the same run.

The repository-level heavy entry point is:

.. code-block:: bash

   ./test-heavy.sh

It is a thin wrapper around :file:`examples/run-tests.sh` and now also
runs the tutorial smoke checks (:file:`examples/run-tutorials.sh`) at
the end. To run the tutorial smoke checks on their own:

.. code-block:: bash

   examples/run-tutorials.sh

To skip the tutorial smoke checks inside the heavy suite:

.. code-block:: bash

   PYRETIS_RUN_TESTS_SKIP_TUTORIALS=1 ./test-heavy.sh

This copies tutorial inputs to ``/tmp``, starts each documented command,
and intentionally stops long-running tutorials after a timeout. A timeout
without errors counts as a successful startup; early crashes, tracebacks,
or PyRETIS ``ERROR``/``CRITICAL`` log messages are reported as failures.
Optional external-engine and FORTRAN cases are skipped when the required
commands are not available unless ``PYRETIS_TUTORIAL_STRICT=1`` is set.

It runs the internal, GROMACS, PyVisA, CP2K, OpenMM, and LAMMPS
example-test suites. The GROMACS part defaults to a smoke scope; set
``PYRETIS_GROMACS_SCOPE=full`` to run the full GROMACS matrix.

.. _example-tutorial-map:

Tutorial map
============

The table below is the canonical mapping between tutorial folders
under :file:`examples/tutorials/`, their rendered documentation
pages, and the heavy-test fixtures that exercise the same workflow.
Tutorial READMEs and the LLM quick start refer to this table rather
than repeating it.

The **Status** column uses four labels:

* ``passing`` -- covered by a heavy fixture that was passing on the
  last refresh.
* ``engine`` -- covered by a heavy fixture but the suite requires an
  external program (``gmx``, ``cp2k``, ``lmp``, ``openmm``) that may
  not be available locally.
* ``smoke`` -- not yet covered by a dedicated fixture; only the
  tutorial smoke runner checks that the command starts cleanly.
* ``manual`` -- requires a GUI, interactive session, or other manual
  step.

.. list-table::
   :header-rows: 1
   :widths: 28 24 32 16

   * - Tutorial folder
     - Docs page
     - Heavy-test fixture
     - Status
   * - ``path_sampling/1D-double-well/tis``
     - :ref:`examples-tis-1d`
     - ``test-internal/tis-multiple``, ``tis-restart``
     - passing
   * - ``path_sampling/1D-double-well/retis``
     - :ref:`examples-retis-1d`
     - ``test-internal/retis``, ``retis-restart``,
       ``retis-load-sparse``
     - passing
   * - ``path_sampling/1D-double-well/submoves``
     - :ref:`examples-submoves-1d`
     - ``test-internal/retis-ss-wt-wf``
     - passing
   * - ``path_sampling/1D-double-well/repptis``
     - :ref:`examples-repptis`
     - ``test-internal/partial-path``,
       ``test-gromacs/test-repptis``
     - smoke
   * - ``path_sampling/1D-double-well/flux``
     - :ref:`examples-tis-1d-flux`
     - ``test-internal/mdflux-restart``
     - passing
   * - ``path_sampling/1D-double-well/tis-multiple``
     - :ref:`examples-tis-1d`
     - ``test-internal/tis-multiple``
     - passing
   * - ``path_sampling/1D-triple-well``
     - :ref:`examples-retis-triple-well`
     - ``test-internal/retis-zero-left``
     - passing
   * - ``path_sampling/2D-hysteresis``
     - :ref:`examples-retis-2d`
     - ``test-internal/explorer`` (representative)
     - smoke
   * - ``path_sampling/2D-wca``
     - :ref:`examples-retis-wca`
     - ``test-lammps/test-retis`` (representative)
     - smoke
   * - ``path_sampling/2D-wca-lammps``
     - :ref:`examples-retis-lammps`
     - ``test-lammps``
     - engine
   * - ``path_sampling/gromacs-hydrate``
     - :ref:`examples-gromacs-hydrate`
     - ``test-gromacs``, ``test-pyvisa``
     - engine
   * - ``path_sampling/cp2k-hydrogen``
     - :ref:`examples-cp2k-hydrogen`
     - ``test-cp2k``
     - engine
   * - ``path_sampling/openmm-argon``
     - :ref:`examples-openmm`
     - ``test-openmm/test-retis``
     - engine
   * - ``path_sampling/permeability``
     - :ref:`examples-permeability`
     - tutorial smoke (``permeability-retis``)
     - smoke
   * - ``molecular_dynamics/md_nve/from_settings``
     - :ref:`examples-md-nve`
     - ``test-internal/md-restart``
     - passing
   * - ``molecular_dynamics/md_nve``
     - :ref:`examples-md-nve-lib`
     - ``test-internal/md-restart``
     - passing
   * - ``molecular_dynamics/md_nve/forward_backward``
     - :ref:`examples-for-back-md`
     - ``test-internal/compare-internal-with-lammps``
     - passing
   * - ``umbrella_sampling``
     - :ref:`examples-mc-umbrella-sampling`
     - (none)
     - smoke
   * - ``extending/order-parameter``
     - :ref:`examples-2d-hysteresis <examples-retis-2d>`
       (representative)
     - ``test-internal`` (representative)
     - smoke
   * - ``extending/lennard-jones``
     - :ref:`examples-vvexternal`
     - ``test-internal/compare-internal-with-lammps``
     - smoke
   * - ``other/particle_swarm``
     - :ref:`examples-pso`
     - (none)
     - smoke

Tutorials without a heavy check
-------------------------------

The following tutorials are currently flagged ``smoke`` only. Before
a release relies on their numerical output, add a shortened fixture
under :file:`examples/tests/` and link it back to the corresponding
docs page:

* ``path_sampling/2D-hysteresis``
* ``path_sampling/2D-wca``
* ``path_sampling/permeability``
* ``umbrella_sampling``
* ``extending/order-parameter``
* ``extending/lennard-jones``
* ``other/particle_swarm``

A "Verification status" line at the top of each docs page (see
``examples-*.rst``) mirrors this column so the LLM rule "do not
claim an example is fully verified unless it is listed as passing"
has a single source of truth.

.. list-table::
   :header-rows: 1
   :widths: 24 18 18 40

   * - Suite
     - Status
     - Command
     - Notes
   * - Internal engine examples
     - Passed 2026-05-17
     - ``./test-heavy.sh`` or ``examples/tests/test-internal/run-all-internal.sh``
     - Covers pure-Python MD, flux, TIS/RETIS, restart, sparse load,
       partial paths, explorer, and sub-move style workflows.
   * - LAMMPS examples
     - Engine available locally
     - ``./test-heavy.sh`` or ``examples/tests/test-lammps/run-all-lammps.sh``
     - Requires ``lmp`` on ``PATH`` and a compatible LAMMPS build.
   * - GROMACS examples
     - Engine available locally
     - ``./test-heavy.sh`` or ``examples/tests/test-gromacs/run-all-gromacs.sh``
     - Requires ``gmx`` and the bundled reference input files.
   * - CP2K examples
     - Engine available locally
     - ``./test-heavy.sh`` or ``examples/tests/test-cp2k/run-all-cp2k.sh``
     - Requires ``cp2k`` or ``cp2k.psmp``.
   * - OpenMM examples
     - Attempted 2026-05-17
     - ``./test-heavy.sh`` or ``examples/tests/test-openmm/run-all-openmm.sh``
     - ``openmm`` was importable locally, but the temp-copy run did not
       complete within the time allowed for this documentation pass.
   * - PyVisA analysis examples
     - Optional GUI/analysis check
     - ``./test-heavy.sh`` or ``examples/tests/test-pyvisa/run-all-pyvisa.sh``
     - Exercises the analysis helpers used by the PyVisA documentation.

What the internal suite covers
==============================

The internal suite is the main quick check because it does not depend
on external molecular-dynamics programs. It should be run before every
documentation release and after changes to examples, input parsing,
path ensembles, internal engines, or analysis output.

.. list-table::
   :header-rows: 1
   :widths: 26 30 44

   * - Test directory
     - Related documentation
     - What it validates
   * - ``compare-internal-with-lammps``
     - :ref:`examples-md`
     - Internal Lennard-Jones MD agrees with saved LAMMPS reference
       thermodynamic output.
   * - ``explorer``
     - :ref:`examples-tis-1d`
     - Loading frames for free-energy landscape exploration and
       checking path, order, energy, and swap consistency.
   * - ``partial-path``
     - :ref:`examples-retis-1d`
     - REPPTIS-style partial paths, analysis report generation, and
       trajectory consistency checks.
   * - ``mdflux-restart``
     - :ref:`examples-tis-1d-flux`
     - MD-flux runs and restart equivalence.
   * - ``md-restart``
     - :ref:`examples-md`
     - NVE restart equivalence for Langevin and Velocity Verlet engines.
   * - ``retis``
     - :ref:`examples-retis-1d`
     - A small RETIS run, analysis, and comparison against saved
       reference data.
   * - ``retis-load-sparse``
     - :ref:`examples-retis-1d`
     - Loading sparse paths from trajectories and from individual
       frames.
   * - ``retis-restart``
     - :ref:`examples-retis-1d`
     - RETIS restart behavior and output reproducibility.
   * - ``retis-ss-wt-wf``
     - :ref:`examples-submoves-1d`
     - Stone skipping, web throwing, and wire fencing moves.
   * - ``tis-multiple``
     - :ref:`examples-tis-1d`
     - Multiple TIS ensembles and analysis output.
   * - ``tis-restart``
     - :ref:`examples-tis-1d`
     - TIS restart behavior and output reproducibility.

How to refresh this page
========================

Run the quick suite first:

.. code-block:: bash

   cd examples/tests/test-internal
   ./run-all-internal.sh

Then run the optional engine suites in an environment where the
corresponding programs are installed:

.. code-block:: bash

   cd examples/tests/test-lammps
   ./run-all-lammps.sh

   cd ../test-gromacs
   ./run-all-gromacs.sh

   cd ../test-cp2k
   ./run-all-cp2k.sh

   cd ../test-openmm
   ./run-all-openmm.sh

   cd ../test-pyvisa
   ./run-all-pyvisa.sh

When a suite passes, update the status table with the date, platform,
|pyretis| version, and any engine version that matters for reproducing
the run.

Documentation build check
=========================

The examples pages use ``literalinclude`` directives to render input
files. To avoid drift between the rendered docs and the runnable
tutorials, run the sync helper before building:

.. code-block:: bash

   python docs/sync_example_assets.py        # report drift only
   python docs/sync_example_assets.py --fix  # update _static
   cd docs && make html

The helper copies a small manifest of input files from
:file:`examples/tutorials/` into :file:`docs/_static/examples/`.
Add new entries to the ``MANIFEST`` list when a new tutorial page
gains a ``literalinclude`` directive.

A missing example asset is usually caught by the Sphinx build before
it becomes a broken website page; treat warnings as actionable.