Experiment

class Experiment

A collection of optimization trial records stored as one OMMX Artifact.

An Experiment owns shared attachments and a sequence of closed Run records. Each Run represents one trial, with scalar parameters, attachments, and zero or more Solve or Sampling records. Solving, transforming, or evaluating a problem does not require an Experiment.

Record two candidates and read them back

This complete example evaluates two supplied candidates without a solver. The shared problem belongs to the Experiment; each Run stores its own condition and result. A temporary Local Registry keeps the example separate from your saved data. Use Experiment(image_name) instead when you want a persistent local record that can later be opened by load().

>>> from ommx import Instance
>>> from ommx.experiment import Experiment
>>> problem = Instance.maximize()
>>> x = problem.new_binary("x")
>>> y = problem.new_binary("y")
>>> problem.objective = 3 * x + 2 * y
>>> _ = problem.add_constraint(x + y <= 1)
>>> with Experiment.with_temp_local_registry() as experiment:
...     experiment.log_instance("problem", problem)
...     for label, candidate in [
...         ("choose-x", {x.id: 1, y.id: 0}),
...         ("choose-y", {x.id: 0, y.id: 1}),
...     ]:
...         with experiment.run() as run:
...             solution = problem.evaluate(candidate)
...             run.log_parameter("candidate", label)
...             run.log_parameter("objective", solution.objective)
...             run.log_parameter("feasible", solution.feasible)
...             run.log_solution("evaluation", solution)
>>> experiment.run_parameters_df()[["candidate", "objective", "feasible"]].to_dict(
...     "records"
... )
[{'candidate': 'choose-x', 'objective': 3.0, 'feasible': True}, {'candidate': 'choose-y', 'objective': 2.0, 'feasible': True}]
>>> [
...     (run.run_id, run.status, run.get_solution("evaluation").objective)
...     for run in experiment.runs
... ]
[(0, 'finished', 3.0), (1, 'finished', 2.0)]

The table contains only values explicitly passed to Run.log_parameter(). A JSON attachment, an Adapter option, and an in-memory Python variable do not automatically become table columns. The saved Solutions also retain their candidate values. They record evaluation, not a proof of optimality.

Save the committed record to a new .ommx file. The following reader runs in a new Python process with an empty Local Registry; it needs only that file, not experiment, problem, or the sender’s registry. It reads the stored results without rerunning either a solver or the candidate evaluation.

>>> import os
>>> from pathlib import Path
>>> import subprocess
>>> import sys
>>> from tempfile import TemporaryDirectory
>>> reader = """
... import sys
... from ommx.experiment import Experiment
... loaded = Experiment.import_archive(sys.argv[1])
... problem = loaded.get_instance("problem")
... assert len(problem.decision_variables) == 2
... for run in loaded.runs:
...     result = run.get_solution("evaluation")
...     print(run.run_id, run.status, result.objective, result.feasible)
... """
>>> with TemporaryDirectory() as directory:
...     archive = Path(directory) / "candidate-trials.ommx"
...     experiment.save(archive)
...     result = subprocess.run(
...         [sys.executable, "-c", reader, str(archive)],
...         env={
...             **os.environ,
...             "OMMX_LOCAL_REGISTRY_ROOT": str(Path(directory) / "receiver"),
...         },
...         check=True,
...         capture_output=True,
...         text=True,
...     )
>>> print(result.stdout, end="")
0 finished 3.0 True
1 finished 2.0 True

save() does not overwrite an existing archive. Use a new path on each export. import_archive() imports the file into the reader’s Local Registry and opens a read-only Experiment. Reading stored results does not establish that the original computation can be reproduced: preserve required source data, code, settings, and environment information explicitly for that purpose.

Choose a record’s scope

Record	What it holds	Read it through
Experiment	Shared attachments and closed Runs	attachment_names, typed getters, runs
Run	Scalar comparison parameters, trial attachments, Solve/Sampling records	run_parameters_df(), SealedRun
Solve	Input Instance, Adapter class/options, and a Solution on success	Solve
Sampling	Input Instance, Adapter class/options, and the complete SampleSet on success	Sampling

Run.log_solve() and Run.log_sample() execute an Adapter and associate its input with its output. The stored Instance is the one supplied to the logging call. An Adapter’s internally prepared copy is not automatically saved. Policy choices, transformation weights, backend settings made through direct model access, Python variables, and package/OS versions are not comprehensively collected. Log additional inputs or explanations when later interpretation needs them. Run.open_solve() supports direct backend-model access and explicit option recording.

The finished, failed, and interrupted statuses describe how recording scopes ended. They do not establish feasibility or optimality. A successful Sampling may contain no feasible candidate. A finished Run may contain a failed Solve if its exception was handled inside the Run. Read candidate feasibility and objectives from the Solution/SampleSet, and backend termination/bounds/gaps from available Adapter diagnostics.

Attachments

Use experiment-level attachments for common inputs and run-level attachments for trial-specific data. Names are local to that scope: two Runs can each have an attachment called evaluation. Standard methods include log_json, log_instance, log_parametric_instance, log_solution, and log_sample_set. log_file() gives a complete file-and-model attachment example; it copies bytes at logging time rather than retaining a link to the source file.

Typed getters reject a mismatched media type. get_attachment() dispatches on media type, and get_blob() returns bytes. compression="zstd" is available for JSON, codec, raw-byte, and file logging: getters and file export return decompressed content and the original logical media type. OMMX marks storage compression with reserved metadata, including when the logical media type itself ends in +zstd. Use log_with_codec() only for types requiring a custom encoding; concrete codecs belong with the package that owns the type.

Commit, recover, or fork

New Experiments are editable (unsealed). Close each Run as a context manager; commit() requires every open Run to be closed. Successful commit writes a config and manifest, publishes the chosen name in the Local Registry, and makes this object read-only. with Experiment(...) commits on normal exit. In a Notebook, keep the Experiment open across cells, inspect closed Runs, add the next trial, and commit explicitly when finished.

Logging writes payload bytes immediately. Payload storage alone does not make a recoverable Experiment. By default, closing a Run also attempts to publish a best-effort draft checkpoint containing closed Runs. AutosavePolicy controls that schedule. A process that dies with an open Run may have written bytes, but that Run’s partial state is not a recoverable closed Run. Runs closed after the latest checkpoint are also absent from that recovery point.

Exceptional exit from an Experiment context tries to publish a failed or interrupted checkpoint without advancing its successful image name; KeyboardInterrupt uses interrupted. The exception propagates. Exceptions outside a non-context-managed Experiment are not automatically captured. Successful commit removes that Experiment’s checkpoint.

• Use restore_from_checkpoint() to resume editing the latest saved closed Runs after interruption; this does not resume a backend’s search state.

• Use fork() to add trials while preserving an already committed parent.

• Use list_experiments() for catalog and annotation filtering, and list_experiment_checkpoints() to discover recovery points.

An omitted image name creates an anonymous local name containing .ommx.local. It can be exported as a file, but must be renamed to a remote registry reference before push(). Name/tag references can later point to different contents; the Artifact manifest digest identifies an immutable record. File exchange and remote registry distribution are alternatives, not requirements for recording. ommx.artifact.Artifact documents the common storage/identity model.

Errors

Attachment lookup by an absent name raises KeyError. Invalid caller-provided image references, autosave values, media types, or JSON inputs raise ValueError. Registry, archive, storage, and lifecycle-state failures raise RuntimeError. Exceptions raised by Python codecs, Adapters, tracing hooks, and data libraries pass through unchanged.

__enter__() → Experiment

__exit__(exc_type: Optional[Any] = None, exc_value: Optional[Any] = None, traceback: Optional[Any] = None) → bool

__new__(image_name: Optional[str] = None, *, store_trace: bool = False) → Experiment

Start a new Experiment in the default local registry.

If image_name is omitted, OMMX generates an anonymous local Experiment name. Pass an OCI image reference such as "example.com/team/experiment:tag" when the experiment should be loaded later by name. The image reference is a mutable local registry alias for the committed Artifact; the Artifact manifest digest remains the immutable identity of the committed contents.

Set store_trace=True to store traces for Run context managers created from this Experiment. The Experiment itself does not need to be used as a context manager.

__repr__() → str

attachment_media_type(name: str) → str

Original media type of an experiment-level attachment.

Storage compression suffixes are hidden from this logical media type.

commit() → Artifact

Commit this unsealed Experiment into the local registry.

All open runs must be closed before committing. The returned Artifact can be saved as a .ommx archive or passed to Experiment.from_artifact. After commit, this object becomes a read-only view of the committed Experiment. A successful commit publishes the requested image reference and removes any local checkpoint for that Experiment when present.

fork(image_name: Optional[str] = None, *, store_trace: bool = False) → Experiment

Fork this committed Experiment into a new unsealed child Experiment.

The parent is unchanged. Existing attachments, Runs, Solves, Samplings, and Run parameters are carried into the child. The child reuses unchanged payload bytes in the Local Registry, and its committed manifest records the parent’s manifest as OCI subject. An uncommitted parent raises an error.

If image_name is omitted, the child receives an anonymous local name. The returned Experiment can be a context manager. Set store_trace=True to store traces for Runs created from that child. The Experiment itself need not be a context manager.

This complete example needs no solver or remote registry:

>>> from ommx.experiment import Experiment
>>> with Experiment.with_temp_local_registry("example.com/fork:parent") as parent:
...     parent.log_json("dataset", {"name": "demo"})
...     with parent.run() as run:
...         run.log_parameter("trial", "baseline")
>>> with parent.fork("example.com/fork:child") as child:
...     with child.run() as run:
...         run.log_parameter("trial", "additional")
>>> len(parent.runs), len(child.runs)
(1, 2)
>>> child.get_json("dataset")
{'name': 'demo'}
>>> child.run_parameters_df()["trial"].to_list()
['baseline', 'additional']

A parent opened with Experiment.load() or Experiment.import_archive() can be forked in the same way. A fork starts new recording; it does not resume a solver’s internal search state.

Experiment.save() and Experiment.push() share the entire child, including inherited Runs and attachments. The child contains their payload layers, so reading those records does not require separately opening the parent Experiment.

from_artifact(artifact: Artifact) → Experiment

Interpret an already-open Artifact as a finished, failed, or interrupted Experiment.

This is the usual entry point after importing or receiving an OMMX Artifact handle. The artifact must contain an Experiment config.

get_attachment(name: str) → Any

Read an experiment-level attachment by name.

The returned Python object is decoded from the attachment media type: JSON attachments become normal Python objects, OMMX instance-like attachments become the corresponding ommx objects, and unknown media types are returned as raw bytes.

get_blob(name: str) → bytes

Read decompressed bytes of an experiment-level attachment by name.

get_instance(name: str) → Instance

Read an Instance experiment-level attachment by name.

get_json(name: str) → Any

Read a JSON experiment-level attachment by name.

Raises an error if the attachment exists but its media type is not application/json.

get_parametric_instance(name: str) → ParametricInstance

Read a ParametricInstance experiment-level attachment by name.

get_sample_set(name: str) → SampleSet

Read a SampleSet experiment-level attachment by name.

get_solution(name: str) → Solution

Read a Solution experiment-level attachment by name.

get_with_codec(codec: type[AttachmentCodec[T]], name: str) → T

Read an experiment-level attachment by name and decode it with a codec.

The codec class must provide media_type, encode(value) -> bytes, and decode(bytes) -> object. This method validates the stored media type against the codec before decoding.

import_archive(path: str | PathLike | Path) → Experiment

Import a finished, failed, or interrupted Experiment Artifact from a .ommx OCI archive file (or an OCI Image Layout directory).

The archive is imported into the default Local Registry, matching import_archive(), and then interpreted as an Experiment. The imported artifact must contain an Experiment config.

load(image_name: str) → Experiment

Load a committed Experiment Artifact by image reference.

If the image is not found in the default Local Registry, OMMX tries to pull it from the remote registry, matching load(). The loaded artifact must contain an Experiment config. Use Experiment(...) to create a new unsealed experiment.

Raises RemoteArtifactNotFoundError when the exact remote reference does not exist. Other remote access failures raise subclasses of RemoteArtifactError.

log_attachment(name: str, media_type: str, bytes: bytes, *, compression: Literal["none", "zstd"] = 'none') → None

Attach arbitrary bytes with an explicit OCI media type in the experiment space.

The name is stored as attachment metadata and is intended for humans. The bytes are stored as a layer in the committed artifact. Set compression="zstd" to compress the stored layer transparently.

log_file(name: str, path: str | PathLike | Path, media_type: Optional[str] = None, *, filename: Optional[str] = None, compression: Literal["none", "zstd"] = 'none') → None

Attach an existing filesystem file in the experiment space.

File bytes are streamed into the Local Registry immediately. If media_type is omitted, OMMX infers it from the contents and falls back to application/octet-stream. Only a basename for export is stored, not the source path. filename overrides that basename. compression="zstd" compresses the stored layer transparently.

This example records common input data and a file at Experiment scope, and a candidate evaluation at Run scope. It then retrieves the typed attachments and exports the original file bytes. Changing the source file after logging does not alter the stored attachment.

>>> from pathlib import Path
>>> from tempfile import TemporaryDirectory
>>> from ommx import Instance
>>> from ommx.experiment import Experiment
>>> problem = Instance.minimize()
>>> x = problem.new_binary("x")
>>> problem.objective = x
>>> with TemporaryDirectory() as directory:
...     source = Path(directory) / "notes.txt"
...     _ = source.write_bytes(b"candidate evaluated locally\n")
...     with Experiment.with_temp_local_registry() as experiment:
...         experiment.log_json("source-data", {"description": "one binary variable"})
...         experiment.log_instance("problem", problem)
...         experiment.log_file(
...             "notes", source, media_type="text/plain", compression="zstd"
...         )
...         with experiment.run() as run:
...             run.log_solution("evaluation", problem.evaluate({x.id: 0}))
...     _ = source.write_bytes(b"changed later\n")
...     assert experiment.get_blob("notes") == b"candidate evaluated locally\n"
...     output = Path(directory) / "restored-notes.txt"
...     _ = experiment.write_attachment("notes", output)
...     assert output.read_bytes() == b"candidate evaluated locally\n"
>>> sorted(experiment.attachment_names)
['notes', 'problem', 'source-data']
>>> experiment.get_json("source-data")
{'description': 'one binary variable'}
>>> saved = experiment.runs[0].get_solution("evaluation")
>>> experiment.get_instance("problem").evaluate(saved.state).objective == saved.objective
True

write_attachment rejects an existing file unless overwrite=True is explicit. Use get_blob() for a binary-file consumer, for example io.BytesIO(experiment.get_blob("notes")). Typed getters reject mismatched media types; get_attachment() uses the media type to choose a standard decoder. The Experiment example shows saving and reopening all these records without the original Python objects.

log_instance(name: str, instance: Instance) → None

Attach an Instance in the experiment space.

log_json(name: str, value: Any, *, compression: Literal["none", "zstd"] = 'none') → None

Attach a JSON-serializable value in the experiment space.

The value is encoded with Python’s json.dumps and stored with media type application/json. Set compression="zstd" to compress the encoded JSON transparently.

log_parametric_instance(name: str, pi: ParametricInstance) → None

Attach an ParametricInstance in the experiment space.

log_sample_set(name: str, sample_set: SampleSet) → None

Attach a SampleSet in the experiment space.

log_solution(name: str, solution: Solution) → None

Attach a Solution in the experiment space.

log_with_codec(codec: type[AttachmentCodec[T]], name: str, value: T, *, compression: Literal["none", "zstd"] = 'none') → None

Encode a Python object with an attachment codec and attach it in Experiment scope.

A codec supplies media_type, encode(value) -> bytes, and decode(bytes) -> object, as described by ommx.experiment.attachments.AttachmentCodec. OMMX owns that protocol; a concrete codec should live in the package that owns the payload type. compression="zstd" compresses the encoded bytes transparently.

Prefer the typed logging methods for supported OMMX/JSON values and log_file() for an existing file. Use a codec when the Python object needs its own encoding. For example, this codec stores a small application-owned coordinate value:

>>> import json
>>> from ommx.experiment import Experiment
>>> class PointCodec:
...     media_type = "application/vnd.example.point+json"
...
...     @staticmethod
...     def encode(value):
...         return json.dumps({"x": value[0], "y": value[1]}).encode("utf-8")
...
...     @staticmethod
...     def decode(data):
...         point = json.loads(data)
...         return (point["x"], point["y"])
>>> with Experiment.with_temp_local_registry() as experiment:
...     experiment.log_with_codec(PointCodec, "location", (2, 3), compression="zstd")
>>> experiment.get_with_codec(PointCodec, "location")
(2, 3)
>>> experiment.attachment_media_type("location") == PointCodec.media_type
True

The reader needs the matching codec and its provider package. For an external modeling object, use the owning package’s public serialization API and media type/version contract; OMMX does not infer its meaning or guarantee compatibility between provider versions. get_with_codec() validates the attachment’s logical media type before decoding. Exceptions raised by a codec pass through unchanged.

Run.log_with_codec() and SealedRun.get_with_codec() provide the same protocol for a payload associated with one trial.

push() → None

Push this finished, failed, or interrupted Experiment Artifact remotely.

Use rename(...) first when an anonymous or local-only experiment should be published under a remote container image reference.

Raises an error if the Experiment has not been committed yet.

rename(image_name: str) → None

Rename this Experiment to another local registry image reference.

Before commit, this changes the image reference that commit() will publish. After commit, it publishes the same Artifact manifest under image_name and updates this handle to use the new name. The previous name remains as an alias in the Local Registry.

restore_from_checkpoint(image_name: str) → Experiment

Restore an unsealed Experiment from its latest checkpoint.

Pass the original requested Experiment image name, not the generated internal checkpoint ref. Accepted checkpoint statuses are draft, failed, and interrupted. The result is editable and uses the original requested image name recorded in checkpoint metadata. Discover recovery points with list_experiment_checkpoints() and pass their requested_image_name here.

A new process must use the same Local Registry as the process that recorded the checkpoint. If OMMX_LOCAL_REGISTRY_ROOT was set, give the reader the same value. This self-contained example uses two separate processes and a temporary registry. The first process closes one Run without committing the Experiment; the default autosave policy writes the recovery point. The second process restores that closed Run, adds another, and commits.

>>> import os
>>> from pathlib import Path
>>> import subprocess
>>> import sys
>>> from tempfile import TemporaryDirectory
>>> writer = """
... from ommx.experiment import Experiment
... experiment = Experiment("example.com/checkpoint:demo")
... with experiment.run() as run:
...     run.log_parameter("trial", "first")
...     run.log_json("result", {"value": 3})
... """
>>> reader = """
... from ommx.experiment import Experiment, list_experiment_checkpoints
... name = "example.com/checkpoint:demo"
... checkpoints = list_experiment_checkpoints(name)
... assert len(checkpoints) == 1 and checkpoints[0].status == "draft"
... restored = Experiment.restore_from_checkpoint(checkpoints[0].requested_image_name)
... assert restored.runs[0].get_json("result") == {"value": 3}
... with restored.run() as run:
...     run.log_parameter("trial", "second")
...     run.log_json("result", {"value": 2})
... restored.commit()
... assert list_experiment_checkpoints(name) == []
... print(Experiment.load(name).run_parameters_df()["trial"].to_list())
... """
>>> with TemporaryDirectory() as directory:
...     env = {**os.environ, "OMMX_LOCAL_REGISTRY_ROOT": str(Path(directory) / "registry")}
...     _ = subprocess.run([sys.executable, "-c", writer], env=env, check=True)
...     result = subprocess.run(
...         [sys.executable, "-c", reader],
...         env=env,
...         check=True,
...         capture_output=True,
...         text=True,
...     )
>>> print(result.stdout, end="")
['first', 'second']

Only Runs included in the latest saved checkpoint are recovered. Open Runs are not included; if autosave was throttled, some later closed Runs can also be absent. This operation neither retries failed calls nor resumes a solver’s internal search. Experiment.commit() publishes the original name and removes its checkpoint. A restored session used as a context manager can publish a new failed/interrupted checkpoint if it ends exceptionally again.

AutosavePolicy belongs to a session and is not persisted. Reapply a nondefault policy to the restored session if needed.

run() → Run

Start a new Run in this unsealed Experiment.

The returned Run must be closed before commit(). Use it as a context manager to close it automatically on normal or exceptional exit. This complete example uses an isolated temporary Local Registry:

>>> from ommx.experiment import Experiment
>>> with Experiment.with_temp_local_registry() as experiment:
...     with experiment.run() as run:
...         run.log_parameter("capacity", 47)
>>> experiment.run_parameters_df()["capacity"].to_list()
[47]

Closing a Run records its status as "finished", "failed", or "interrupted". It also publishes a best-effort draft checkpoint when the Experiment’s autosave policy is due. Payloads written by an open Run before it is closed are stored in the Local Registry but are not recoverable through a checkpoint until a later checkpoint includes that Run.

run_parameters_df() → DataFrame

Wide DataFrame of run parameters, indexed by run_id.

Run parameters are scalar values logged with Run.log_parameter. Closed runs with no parameters are still present as index rows. Adapter options recorded by Run.log_solve are solve metadata and do not appear in this table.

save(path: str | PathLike | Path) → None

Save this finished, failed, or interrupted Experiment Artifact as a .ommx archive.

The archive is an exchange-format export of the registry-resident Experiment Artifact. Loading the archive back via Experiment.import_archive() reimports it into the SQLite Local Registry under the same image name.

Raises an error if the Experiment has not been committed yet.

set_annotation(key: str, value: str) → None

Set a manifest annotation on this unsealed Experiment.

OMMX-owned annotation keys are reserved. Use reverse-DNS style keys such as "com.example.project" for caller-owned metadata.

set_autosave_policy(policy: AutosavePolicy) → None

Set the rolling draft checkpoint policy for this unsealed Experiment.

Changing the policy resets its schedule at the current closed-Run count. The policy affects only draft autosaves after Run close; exception-driven failed or interrupted checkpoints are still written.

with_temp_local_registry(image_name: Optional[str] = None, *, store_trace: bool = False) → Experiment

Start a new Experiment backed by a temporary Local Registry.

The temporary registry is kept alive by the returned Experiment and by Artifacts / loaded Experiments derived from it. This is useful for examples and tests because it does not write entries into the process-wide default local registry.

Set store_trace=True to store traces for Run context managers created from this Experiment. The Experiment itself does not need to be used as a context manager.

write_attachment(name: str, path: str | PathLike | Path, *, overwrite: bool = False) → Path

Write an experiment-level attachment to a filesystem path.

If path names an existing directory, the attachment filename stored by log_file is used inside that directory. Otherwise path is treated as the destination file path. Compressed attachments are decompressed while exporting.

property annotations: dict[str, str]

Read-only property.

Manifest annotations that will be written, or were written, to this Experiment artifact.

property artifact: Artifact

Read-only property.

Immutable OMMX Artifact for this finished, failed, or interrupted Experiment.

Raises an error if the Experiment has not been committed yet.

property attachment_names: list[str]

Read-only property.

Names of experiment-level attachments.

property image_name: str

Read-only property.

OCI image reference used to store this Experiment in a local registry.

property lifecycle_reason: Optional[str]

Read-only property.

Concise exception type and message for a failed or interrupted Experiment.

This is lifecycle metadata, not solver diagnostics. It is None when no reason was recorded.

property runs: list[SealedRun]

Read-only property.

Closed runs in insertion order.

property status: Optional[str]

Read-only property.

Experiment config status for a committed Experiment.

Returns None for an unsealed Experiment.