Recovery metrics#

Functions to score an analysis pipeline’s output against the ground truth. See the benchmarking guide for how they fit together.

Spatial matching#

minisim.hungarian_match(A_est, A_true, *, metric='iou', energy_frac=0.9, shift=None)[source]#

Optimally pair estimated spatial footprints to true ones by overlap.

Forms the pairwise footprint-similarity matrix under metric and runs scipy.optimize.linear_sum_assignment() to find the assignment maximizing total similarity. Pairs with zero similarity are dropped from the returned Match.pairing.

Parameters:

  • A_est – Footprint stacks (n, height, width), non-negative. Negative values (if any) are clipped to zero before scoring.

  • A_true – Footprint stacks (n, height, width), non-negative. Negative values (if any) are clipped to zero before scoring.

  • metric (str) – Footprint similarity (see SIMILARITY_METRICS): "iou" (binary energy-mask Jaccard, the default), "cosine", or "weighted_jaccard". The two weighted metrics compare the intensity profile, so pixel weights - not just which pixels are lit - drive the match.

  • energy_frac (float) – For metric="iou" only: fraction of each footprint’s energy its binary mask retains, in (0, 1].

  • shift (tuple[float, float] | str | None) – Global translation applied to A_est before scoring, to absorb a uniform offset between the estimate’s (motion-corrected) frame and the true frame. None applies nothing; a (dy, dx) tuple applies that known shift (rounded to whole pixels - derive it from the motion trajectories); "auto" estimates one by aligning the intensity-weighted centroids of the two footprint stacks. The applied integer shift is recorded on Match.shift.

Return type:

Match

class minisim.Match(similarity_matrix, pairing, metric='iou', shift=(0, 0))[source]#

Bases: object

The result of pairing estimated footprints to true ones by spatial overlap.

pairing is the optimal one-to-one assignment (maximizing total similarity) with pure non-overlapping pairs dropped, so it is safe to feed straight into the temporal metrics. The threshold-dependent quality summaries (recall(), precision()) count only pairs whose similarity clears threshold.

metric records which similarity was used (see SIMILARITY_METRICS) and shift the global (dy, dx) translation applied to the estimate before scoring ((0, 0) when none). iou_matrix / mean_iou remain as aliases for the generic similarity_matrix / mean_similarity so existing callers keep working; they hold the chosen metric, not necessarily IoU.

Empty denominators (no estimated or no true cells, no matched pairs) report 0.0 rather than nan - convenient for assert metric >= bound tests.

Parameters:
property iou_matrix: ndarray#

Alias for similarity_matrix (holds the chosen metric).

matched_pairs(threshold=0.5)[source]#

The assigned pairs whose similarity is at least threshold (true positives).

Parameters:

threshold (float)

Return type:

list[tuple[int, int]]

recall(threshold=0.5)[source]#

True positives over the number of true cells (0.0 if there are none).

Parameters:

threshold (float)

Return type:

float

precision(threshold=0.5)[source]#

True positives over the number of estimated cells (0.0 if there are none).

Parameters:

threshold (float)

Return type:

float

property mean_similarity: float#

Mean similarity over the matched (positive-overlap) pairs (0.0 if none).

property mean_iou: float#

Alias for mean_similarity (the chosen metric, not always IoU).

Temporal scores#

minisim.trace_pearson(C_est, C_true, pairing)[source]#

Per-matched-pair Pearson correlation between estimated and true traces.

Returns one correlation per (est_idx, true_idx) in pairing (a constant trace has undefined correlation and yields nan). C_est/C_true are (unit, frame).

Return type:

ndarray

minisim.activity_similarity(S_est, S_true, pairing)[source]#

Scale-aware recovery of the deconvolved activity over the matched units.

For each matched pair the estimated activity S_est[i] is compared to the true activity S_true[j] ((unit, frame) arrays) without binarizing: the deconvolved estimate is a continuous activity rate, related to the truth by an unknown non-negative amplitude factor, so the comparison must be invariant to that scale. Per pair this returns (see ActivityScore):

  • the Pearson correlation (scale- and offset-invariant shape match);

  • the recovered non-negative scale alpha minimizing || S_true - (alpha * S_est + beta) || (the best amplitude map, alpha >= 0);

  • the variance_explained by that fit, 1 - SS_res / SS_tot.

A constant estimate yields scale = 0 and variance_explained = 0 (it explains nothing); a constant truth yields nan for both correlation and variance explained (there is nothing to explain). This is the deconvolution scorecard CaLab uses (per-cell amplitude alpha plus proportion of variance explained), not a spike-timing precision/recall.

Return type:

ActivityScore

class minisim.ActivityScore(correlation, scale, variance_explained)[source]#

Bases: NamedTuple

Per-matched-pair recovery of the deconvolved activity (see activity_similarity()).

The deconvolved estimate (CNMF/minian S) is not a spike train: it is a non-negative estimate of neural activity rate, scaled by an unknown factor that maps activity to calcium-kernel amplitude. So each pair is scored without binarizing and without assuming a common scale:

  • correlation - Pearson r between estimated and true activity (scale- and offset-invariant; the shape match). nan for a constant trace.

  • scale - the recovered non-negative amplitude factor alpha that best maps the estimate onto the true activity (the unknown gain, made explicit).

  • variance_explained - proportion of the true activity’s variance the non-negatively scaled estimate accounts for, in (-inf, 1]. nan when the true activity is constant.

Parameters:
correlation: ndarray#

Alias for field number 0

scale: ndarray#

Alias for field number 1

variance_explained: ndarray#

Alias for field number 2

Field and motion#

minisim.field_pearson(est, true)[source]#

Pearson correlation between two 2-D fields (vignette, leakage), flattened.

Scale- and offset-invariant, so it scores the shape of the recovered field rather than its absolute level. Returns nan if either field is constant.

Return type:

float

minisim.shift_rmse(shifts_est, shifts_true, *, correction=False, align=False)[source]#

Root-mean-square error (pixels) between two (frame, 2) shift trajectories.

RMSE over all frames and both axes. The two arrays must share a sign convention: a motion-correction estimate is the negation of the applied GroundTruth.shifts, so pass correction=True to negate shifts_est first (the common case, since a motion-correction pipeline emits the shift it would apply to undo the motion); leave it False to compare two trajectories already in the same convention.

They must also share an origin, and that is the catch across pipelines: a correction trajectory is relative to whatever template the pipeline registered to, so it can carry an arbitrary constant per-axis offset versus minisim’s zero-shift reference. align=True removes the best constant offset (the per-axis mean residual) before the RMSE, scoring how well the motion was tracked rather than which frame was called zero. Leave it False to hold the pipeline to the absolute origin too.

Parameters:
Return type:

float

minisim.global_shift_from_trajectories(shifts_est, shifts_true, *, correction=True)[source]#

The constant (dy, dx) offset between two motion trajectories, in pixels.

The estimated correction and the true applied motion track the same brain movement, so their difference is (up to noise) a constant: the offset between the pipeline’s registration template and minisim’s zero-shift reference. That constant is exactly the global translation between the estimated and true footprint frames, so it can be read off the trajectories rather than searched for. Returns the per-axis mean difference rounded to whole pixels, ready to pass as hungarian_match()’s shift. correction=True negates the estimate first (the usual convention, matching shift_rmse()).

Parameters:

correction (bool)

Return type:

tuple[int, int]