Skip to content

Chart Generation

This guide covers how to write chart code for ExpOps. For configuration details, see the Probe paths and Chart Dependencies sections below.

Component Charts

The expops-matplotlib library provides BarChart and Histogram chart components that run as pipeline processes — no Python chart code required. If you only need standard bar charts or histograms, start there.

The rest of this page covers custom @chart() functions, which give you full control over matplotlib output.

Static Charts

Static charts generate PNG image files that are saved to disk.

Configuration: Chart entrypoints are configured in project_config.yaml under reporting.static_entrypoint. See the Probe paths and Chart Dependencies sections below for details.

Chart Functions

Functions decorated with @chart() generate visualizations. Chart functions have strict requirements:

Required Function Signature

Every static chart function MUST:

  1. Accept metrics as the first parameter (Dict[str, Any])
  2. Accept ctx as the second parameter (ChartContext), for metrics and optional context
  3. Use plt.savefig() to save figures (the process runs with cwd set to the output directory; all files written there are synced to artifacts)
from expops.reporting import chart, ChartContext
from typing import Dict, Any
import matplotlib.pyplot as plt

@chart()
def plot_metrics(metrics: Dict[str, Any], ctx: ChartContext) -> None:
    """
    Chart function signature requirements:
    - metrics: Dict containing metrics from probe_paths (REQUIRED)
    - ctx: ChartContext for metrics and run context (REQUIRED)
    - Returns: None (void function)
    """
    # Access metrics directly from the metrics dict
    # Keys are probe_path keys or resolved paths (see Probe paths section)
    train_metrics = metrics.get('train', {})
    eval_metrics = metrics.get('eval', {})

    # Extract specific metric values
    train_acc = train_metrics.get('accuracy', {})
    eval_acc = eval_metrics.get('accuracy', {})

    # Generate matplotlib plot
    fig, ax = plt.subplots(figsize=(10, 6))
    # ... plotting code ...

    # Use plt.savefig() with a relative path; cwd is the process output dir
    plt.savefig('plot_metrics.png', dpi=150)
    plt.close(fig)

Note: The function name should match the chart name defined in project_config.yaml, or be registered via the @chart() decorator.

Metrics Access

Metrics are passed directly to chart functions via the metrics parameter:

  • Metrics structure: The metrics dict is keyed by your probe_path keys (when an XPath matches one node) or by resolved probe path strings (when an XPath matches multiple nodes). See Probe paths for how keys are determined.
  • Access pattern: metrics.get('key', {}) returns metrics for that key (either a config key or a resolved probe path)
  • Metric values: Each probe path contains metrics logged from that process/step
  • Step-based metrics: Metrics logged with step= parameter are stored as dicts like {"1": value1, "2": value2, ...}

Example:

Given this config: 

reporting:
  charts:
    - name: "my_chart"
      probe_paths:
        train: "//*[@name='train_model']"
        eval: "//*[@name='evaluate_model']"

The chart function receives: 

@chart()
def my_chart(metrics: Dict[str, Any], ctx: ChartContext) -> None:
    import matplotlib.pyplot as plt
    # metrics['train'] contains metrics from train_model process
    train_data = metrics.get('train', {})

    # metrics['eval'] contains metrics from evaluate_model process
    eval_data = metrics.get('eval', {})

    # Access specific metrics (will be a dict)
    train_acc = train_data.get('accuracy', {})
    eval_acc = eval_data.get('accuracy', {})

Probe paths

Probe paths must be XPath selectors: strings that look like XPath (e.g. start with // and contain @ and [). They are evaluated against an internal pipeline tree and may resolve to one or many concrete probe paths.

XPath format and pipeline structure

The pipeline is represented as a tree: root pipeline, then nested process elements with @name, optional @partition (data split, e.g. p1, p2) and @seed (seed value, e.g. 41, 42), and optional child step elements with @name. XPath is evaluated over this tree (standard lxml). Use // for descendant-or-self, / for path steps, * for any element, and [@name="..."] for attribute predicates.

Common patterns:

Goal XPath pattern
Process by name //*[@name='process_name']
Process + step //*[@name='process_name']/*[@name='step_name'] or //*[@name='step_name'] if the step name is unique among process names
Specific partition/seed //*[@partition='p1']/*[@seed='41']/*[@name='process_name']
Any partition/seed //*[@partition]/*[@seed]/*[@name='process_name']

How keys map to chart metrics

  • One XPath match: The config key is preserved (e.g. train → metrics['train']).
  • Multiple XPath matches: Each resolved probe path becomes a key. The key is the canonical XPath-style identifier for that process/step (e.g. "//*[@partition='p1']/*[@seed='41']/*[@name='nn_training_a']/step[@name='train_and_evaluate_nn_classifier']"). Chart code can iterate over keys or use prefix/grouping logic to aggregate across partitions or seeds.
  • Literal path: Single key as in config (e.g. train: "train_model" → metrics['train']).

Output

Static charts produce image files (PNG) saved under the unified artifacts layout: 

.<project_id>/artifacts/<version_hash>/<encoded_probe_path>/<chart_name>.png
In GCS: gs://<bucket>/<project_id>/artifacts/<version_hash>/<encoded_probe_path>/<chart_name>.png

Dynamic Charts

Dynamic charts provide real-time, interactive visualizations.

Configuration: Dynamic charts are defined as pipeline processes in project_config.yaml (under experiment.parameters.pipeline.processes). Each dynamic chart process must have:

  • code - a unified code reference that points to your JS chart script and function (e.g. code: "reporting_js.nn_losses"), where reporting_js is defined under scripts: at the top of the config
  • chart_type: "dynamic"
  • probe_paths - same XPath semantics as static charts (see Probe paths below)

Example

Config (excerpt from premier-league): register the JS script under scripts, then add a process with chart_type: "dynamic":

# Under experiment.parameters.pipeline.processes:
        - name: "nn_losses"
          code: "reporting_js.nn_losses"
          environment: "premier-league-env-reporting"
          chart_type: "dynamic"
          probe_paths: ...

Client-side (subscribe to metrics and update a Chart.js chart):

// Subscribe to metrics
subscribeToMetrics((metrics) => {
    // Update Chart.js chart
    chart.data.datasets[0].data = metrics;
    chart.update();
});