Bluesky should move the scan plan, not the safety system

The NSLS-II experimental floor during an April 2023 open house. Bluesky was developed at this facility, but the scene is a useful corrective to the word software: a scan plan ultimately meets detectors, motion hardware, cabling, shielding, and interlocks that should not all move layers at once.[6]

A beamline that already takes data has at least two systems hiding inside it. One protects and moves the physical machine: EPICS IOCs, controller logic, limit switches, shutters, interlocks, hardware triggers, and engineering screens. The other describes the experiment: which motors to position, which detector to trigger, what metadata to record, how to repeat a scan, and what to do after an interruption.

Bluesky is a strong candidate for the second system. It represents an experimental procedure as a Python generator called a plan and executes that plan through the RunEngine. Hardware appears to the plan as Python objects, commonly modeled with Ophyd; data and metadata leave the RunEngine as a live stream that callbacks can plot, transform, store, or forward.[1][3] This separation makes procedures reusable and inspectable without requiring a new control system underneath.

That last clause is the migration boundary. Adopting Bluesky should not begin by replacing every process variable, rewriting proven interlocks in Python, or assuming that a generator can provide deterministic synchronization. Begin above EPICS. Preserve the layer that already knows how to protect the equipment, then move one experimental procedure at a time into an explicit device model and plan.

The facility evidence favors that narrow route. At BESSY II, engineers kept heterogeneous EPICS controls in place, created Ophyd classes around them, converted SPEC macros into Bluesky plans, and used simulated hardware before pointing the same interfaces at real devices.[4] Diamond Light Source later deployed its Bluesky-based Athena services beside the existing GDA platform, routing selected operations through either path while it proved the new stack in user experiments.[5] Neither case treated “migration” as a single switch.

Image context: the cover photograph looks across the NSLS-II experimental floor, the facility where Bluesky originated. The beamline enclosures and service runs make the layering visible: orchestration software can change quickly; the physical system it commands cannot be treated as disposable.[6]

Freeze one working scan as a contract

Do not start with the longest macro or the most temperamental detector. Choose one low-risk, representative step scan whose present behavior is understood by both a beamline scientist and a controls engineer. Capture what it does before translating how it is written.

The baseline should include the commanded positions and readbacks, settling rules, detector exposure and trigger order, motor and detector timeouts, pre- and post-scan state, file names, operator prompts, metadata fields, output format, and the behavior of stop, abort, and hardware failure. Save a known input and the resulting data. Record the versions of the IOC databases, motor controllers, detector firmware, legacy macros, and analysis tool that produced it.

Acceptance is not source-code resemblance. The new plan passes when it reaches the same intended positions, obtains the scientifically required readings, records enough context to interpret them, leaves equipment in the agreed state, and fails in a way operators can recover. A small numeric matrix makes the comparison honest: point count, endpoint tolerance, maximum readback error, expected exposure count, missing-frame count, timestamp skew, cleanup state, and time to a usable dataset.

Keep the old path runnable while this contract is being established. Diamond's I22 deployment is a useful model: Athena and the existing GDA system ran alongside each other, and requests could be directed to either while the new orchestration path was tested.[5] Side-by-side operation is not indecision when the rollback target is a known-good scientific procedure.

Build device behavior before translating the macro

The quickest way to make an unmaintainable Bluesky deployment is to expose a loose bag of EPICS PVs directly to every plan. Ophyd's job is to group related signals into a device with a coherent name and behavioral interface. A read-only beam-position object, for example, can collect two PVs without leaking their prefixes into scan logic:

from ophyd import Component, Device, EpicsSignalRO

class BeamPosition(Device):
    x = Component(EpicsSignalRO, "X")
    y = Component(EpicsSignalRO, "Y")

The suffixes here are illustrative; the production class must reflect the facility's actual records and naming. More important than the syntax is the contract around it. A movable device needs a completion condition, timeout behavior, a meaningful stop() implementation, readback, limits, and configuration signals. A detector needs staging, triggering, completion, data description, and asset handling that agree with the real acquisition path.

BESSY II discovered why this work comes first. Its beamline motors did not use the standard EPICS Motor Record, and interfaces varied even among devices of the same type. The team wrote bespoke Ophyd classes, then used integration tests that attempted to move every motor; those tests found bugs that ordinary abstraction tests would have missed.[4] A polished scan plan cannot compensate for a device object that declares success before motion finishes or stops the wrong PV.

Use three test layers. First, exercise plan structure with simulated devices and no connection to the beamline. Second, connect device classes to a simulator or digital twin using a separate PV prefix. Third, commission each class against real hardware with bounded moves, independent readback, and a controls engineer present. At BESSY II, switching an Ophyd interface from simulated to real equipment could be as small as changing its EPICS prefix, which made scarce commissioning time more useful.[4]

Pin this device layer as deliberately as any other control dependency. The current Ophyd documentation marks its v2 API provisional and notes that v1 and v2 devices can share a RunEngine.[2] That makes gradual evaluation possible, but it is also a warning against combining a beamline cutover with an unbounded API-generation rewrite. Choose the generation and versions for the pilot, record them, and change them only through the same acceptance suite.

Port procedure into plans, not RunEngine calls inside helpers

Once the devices are credible, translate the scan at the level of intent. Bluesky already supplies plans such as count, scan, rel_scan, and grid_scan. A facility-specific procedure can compose those plans with yield from, add metadata, and wrap cleanup without taking control away from the RunEngine.[1]

That detail matters. The Bluesky tutorial explicitly warns against hiding RE(...) inside loops or helper functions; nested entries make interruption and error handling unpredictable. The helper should itself be a plan, yielding messages to one RunEngine that owns the procedure from start to finish.[1] Likewise, cleanup should use the plan-preprocessor mechanisms intended for it, such as finalize_wrapper or the more targeted contingency_wrapper, rather than a normal Python finally block that assumes hardware commands will always execute as expected.[1]

Start by translating the visible sequence: configure, stage, move, trigger, read, unstage, restore. Then expose the information that the old macro carried implicitly—sample identity, proposal, detector settings, calibration reference, scan purpose, and the versions of the plan and device configuration. Bluesky can apply baseline readings before and after every run and can send its data stream to multiple callbacks, so metadata capture and live display do not have to be reimplemented inside each plan.[1]

BESSY II reported that common SPEC macros such as ascan had close Bluesky equivalents and that even bespoke setup macros were relatively straightforward to convert. Its first deployment nevertheless exported data back to .spec files for use in PyMca, preserving a familiar analysis path while acquisition changed.[4] That is a sound migration tactic: move orchestration first, and give downstream users a compatibility lane until the new document and storage path has proved itself.

Keep protection and deterministic timing below the RunEngine

Bluesky's interruption model is useful but easy to oversell. On pause, the RunEngine tells touched devices to stop; on resume, it rewinds to a checkpoint and repeats instructions as needed. Its own tutorial says that this stop behavior is a convenience, not a replacement for proper equipment protection, and warns that time-evolving experiments cannot necessarily be paused and resumed safely.[1]

Therefore shutters, collision avoidance, hard travel limits, vacuum protection, over-temperature trips, and other equipment-protection logic should remain in the IOC, PLC, controller, or hardwired layer that can act without a healthy Python process. A plan may check permissives and refuse to start. It may request a safe transition. It should not become the only place where a dangerous transition is prevented.

Timing deserves the same boundary. In a step scan, “move completed, then trigger and read” may be the scientific contract. In a fly scan, detector frames, encoders, shutters, and sample-environment events may need a shared hardware clock or trigger line. BESSY II's deployment paper notes that grouping PVs into an Ophyd device does not make their values update synchronously; that guarantee has to come from below the IOC. Its continuous scan could use RunEngine polling only while the monochromator moved slowly enough for the read-time difference to be insignificant, with hardware triggering identified as the route when polling was no longer adequate.[4]

Make this a hard acceptance gate. For every continuous path, record the trigger source, clock owner, expected frame count, timestamp authority, maximum acceptable skew, overflow behavior, and what happens when a detector or writer falls behind. Compare frames against encoder positions under representative speed and data volume. If a run looks smooth but cannot prove which position produced a frame, it has not passed.

Diamond's phased deployment shows the complementary pattern. Its Athena service used Bluesky plans and Ophyd devices for timed acquisition while dedicated hardware handled synchronized triggering; subsequent work targeted more complex external-trigger cases.[5] Bluesky expressed the experimental sequence. It did not pretend the Python scheduler was the timing source.

Treat the document stream as a public interface

A migration can move every motor correctly and still lose the scientific record. Before user data depends on the new path, define which metadata and data documents constitute a complete run, where large detector assets live, how they are linked, and which consumer is authoritative for durable storage.

The RunEngine publishes a live stream to callbacks for visualization, processing, message queues, files, and databases.[1] That fan-out makes failure isolation possible, but a callback running in the acquisition process is not isolated merely because it is called a callback. Move nonessential consumers across a tested queue or transport before assuming that an e-log exporter or live plotter can fail independently. In every topology, a green plot is not evidence that the durable writer committed the run.

Test the stream with deliberate failures. Restart a visualization client during a scan. Make a nonessential callback raise an error. Fill a staging filesystem in a lab environment. Interrupt the link between asset metadata and detector files. Confirm that the run's start and completion state, unique identifier, baseline readings, configuration, and external asset references remain internally consistent—or that the acquisition stops with a clearly incomplete record. BESSY II separated image files from searchable metadata and used the stored run identifier to retrieve them through Databroker; it also moved secondary consumers away from the critical measurement path.[4]

Promote by evidence, one lane at a time

A practical rollout has four stages:

Inventory: one scan contract, one owner for each device class, a dependency lock, and a tested legacy rollback.
Simulation: plan inspection, digital-twin execution, injected timeouts, interruption tests, and validation of emitted metadata without beam.
Commissioning: bounded real moves, detector staging, stop behavior, asset writing, and timing measurements during protected machine time.
User pilot: one agreed technique or shift through the new path, with the old system available and a written cutback trigger.

For a single laboratory instrument, one scientist and one controls engineer may own all four stages. A shared facility needs separate owners for device abstractions, plans, control-system safety, storage, authentication, deployment, and support during beam time. Queueing the RunEngine as a service can isolate acquisition from a crashed notebook or display, but it also introduces permissions, network, deployment, and recovery work that should be adopted only when the operating team can own it.[4][5]

Stop promotion if a required device has no trustworthy completion or stop semantics, if a critical interlock exists only in a Python plan, if fly-scan alignment cannot be measured, if a completed run can lose its durable assets, or if operators cannot return to the legacy path inside the agreed recovery window. Those are migration failures, even when the demo produces a beautiful live plot.

The right first Bluesky success is deliberately modest: the same safe scan, expressed as a testable plan, executed through honest device objects, and recorded with richer context. Once that lane repeats, more procedures can move upward. The interlocks and clocks stay where their guarantees are strongest.

cronfeed.work