Timing Model

This page records the design decisions behind NeuroConv’s temporal alignment surface, interface.alignment. The user guide (Temporal Alignment) explains how to align a conversion; this page explains why the surface has the shape it does, which is what you need when adding alignment to a new modality.

The model

The primitive is the time-bearing object: anything an interface writes that carries a time coordinate relative to session_start_time. An ElectricalSeries is one, so is each PoseEstimationSeries of a pose interface, each EventsTable of an events interface, and a trials table. A Device, an electrodes table and a Skeleton are not.

What makes it the right primitive is how little it claims. It says this object has times, and nothing more: not which clock they came from, not which other objects share that clock. That is the weakest statement that still supports every operation the surface offers, and weakest is what you want here, because anything stronger would have to be true of the data and the framework has no way to check it.

The alternatives all claim more. A timebase, a clock or a timeline names a group of objects sharing one basis, so building on one means the framework has to know which objects belong together. That is a fact about how the rig was wired rather than anything present in the files: a converter holding an Intan recording, pose estimation and a behavior camera has three such groups and nothing in the data distinguishes them. With time-bearing objects the question never arises. Two objects are co-timed because you gave them times that agree, which is a description of what you did rather than a claim the framework makes on your behalf.

The alignment state is two scalar offsets and, optionally, one replacement array, applied at write:

output = base + object_offset + interface_offset

base is the object’s default times, the ones its source recorded or, where it records none, the ones its header gives, or a replacement array once set_times or remap_times has provided one. interface_offset is written by shift_times and is shared by every object the interface names. object_offset is written by start_at and belongs to one object. Both default to 0.0, the identity, so an interface that is never aligned writes the times its source recorded. The offsets are stored rather than folded into the times, and the source times are never mutated, so the original timing stays recoverable and nothing is read from the source until someone asks for times.

Two axes, two operations

Every operation on the surface sits on two axes. Scope: does it move the whole interface rigidly, or one object relative to its siblings? Arithmetic: is the number it takes relative, a delta added to where the object is, or absolute, a statement of where the object should be? Whether the operation stores a scalar or materialises every sample time follows from those two.

scope	relative, a delta	absolute, a position
whole interface	shift_times(delta)	empty
one object	empty	start_at(t), and set_times(times) when the times themselves are known

shift_times is interface-wide and relative because the number it takes is a fact about a device’s clock: this rig’s clock reads 3.2 seconds behind the reference, so everything it wrote moves by 3.2 seconds, including objects that were placed individually, which keep their relative positions. An interface reads one source from one acquisition system, so one clock offset is the right granularity.

start_at is per-object and absolute because the number it takes is a fact about one object: this file began 12.5 seconds into the session. It arises where the source records nothing about where its pieces sit, a camera or a microphone triggered once per trial, and there the user knows a position per file and never a delta. It stores one scalar, so placing an hour of audio costs nothing, where expressing the same fact through set_times would build 158 million sample times to carry one number and would force the series to be written as a timestamps array rather than a rate.

The empty cells are empty because nobody has a number to put in them. A per-object shift would need a delta for one file among its siblings on the same device, which no workflow produces, and it would accumulate on a re-run, which is the failure start_at exists to remove. An interface-wide absolute setter would write into the same offset shift_times accumulates into and silently discard an earlier shift, since it has nothing to store itself against. The per-object form escapes that because its scalar is separate from the interface’s.

Composition, not inheritance

Interfaces hold the alignment component, as interface.alignment, rather than inheriting a base class that provides the methods directly. The base class alternative is BaseTemporalAlignmentInterface, whose contract is get_timestamps() -> np.ndarray and set_aligned_timestamps(np.ndarray).

Holding it wins on four counts.

It promises nothing about shape. The inherited contract is one array per interface, and an events interface has no such array, since its times live per event type. Satisfying it means fabricating one, which is how a dict-returning get_timestamps ends up violating its own annotation. A held component makes no shape promise at all.

It gives a converter a capability check. A converter can align its interfaces if each one has an alignment, and can name the one that does not. That is a plain attribute test rather than duck-typing individual methods.

It keeps one implementation. Every modality shares the same component, so a fix or an addition lands once instead of being reimplemented per base class, and the modalities cannot drift apart in behaviour.

It leaves the interface’s own surface alone. An interface class is already large, and half a dozen alignment methods at the top level would bury get_metadata and add_to_nwbfile among them. Under a namespace they stay together and are discoverable as a unit.

Everything a user passes or reads is in file time

get_times returns the times the file will carry, and set_times writes exactly the times it is given, so calling the getter and handing the result straight back to the setter changes nothing.

There were two coherent readings of set_times and this is the one that was chosen. Under the other, the argument is the object’s times inside the interface and the file receives times + offset; that version composes with a shift and commutes with it, but the file then quietly disagrees with the numbers the caller passed, and the disagreement is only discoverable by reading the file back or by knowing about a scalar the caller never typed, since the offset is often applied by a converter rather than by the user. The chosen reading puts the surprise in the API instead: whatever get_times reports is what will be written, always.

A consequence worth stating plainly, because it looks like a bug and is not: a shift and a set do not commute. shift_times(2.0) followed by set_times(v) writes v, while set_times(v) followed by shift_times(2.0) writes v + 2.0. This is assignment against increment, the same asymmetry as x = 10 against x += 2, and it follows from set_times being absolute and shift_times being relative.

start_at is absolute in the same way, and the cleanest statement of how the four operations compose is as functions on T, the times an object will be written on:

shift_times(d):      T' = T + d
start_at(t):         T' = T + (t - T[0])
set_times(A):        T' = A
remap_times(L, R):   T' = interp(T, L, R)

Composition is function composition, so order matters exactly as it does for functions. start_at twice is a no-op. set_times or remap_times after start_at supersede it, since they define the times outright, and start_at after either moves the given times rigidly. A later shift_times moves everything. In storage, start_at writes object_offset = t - base[0] - interface_offset, the two array writers reset object_offset to zero, and the start is read back as base[0] + object_offset + interface_offset rather than stored, so there is no second copy of it to fall out of step. start_at needs base[0] without materialising the array, which is what the optional native start an interface can register alongside its native times is for; absent it, the first native time is read.

remap_times reads and writes in the same frame, which is what fixes where its arguments live: it interpolates the times as they currently stand, so local_sync_times is on the timeline get_times reports and carries a shift already applied, while reference_sync_times is on the clock being aligned to. The two arrays pair up positionally, so a pulse only one system recorded has to be dropped from the other as well.

How many objects an interface names

An interface registers its time-bearing objects, and how many it names follows from where their placement comes from. Where the source records how its segments sit, the interface names one object and its times carry the layout. A multi-segment electrophysiology recording is the case: each segment’s start is in the file and the extractor already exposes it, and the segment count is a reader setting, a gap tolerance, rather than a fact about the session, so it cannot carry a public address. Where only the user can say where a piece sits, the interface names one object per piece. Trialised external video and per-trial audio files are the cases: the files record nothing about their position, so each is placed with start_at. Files an acquisition system split at a size limit are neither. They are one continuous recording, and the extractor concatenates them before the interface sees them.