Guide

Fill gaps in irregular time-series data: 5 methods and when each breaks

Missing observations, frozen measurements and irregular sampling are different problems. Choose a method for the specific signal and gap; retain the source data and record every proposed reconstruction.

First: what kind of gap is it?

A few missing samples in a fast, smooth channel are very different from a multi-minute sensor freeze during a transient. Fixing them the same way is how bad interpolation ends up in a report. And irregular timestamps are their own problem — a method that counts rows treats a 1-second gap the same as a 10-minute one.

Handle irregular timestamps first

These are limited offline examples, not the NLSYS engine. Download the tested example functions

import pandas as pd
from engineering_examples import short_gap_linear

df = pd.read_csv("run.csv")
# This export contract uses elapsed seconds, not dates or nanoseconds.
t = pd.to_numeric(df["elapsed_seconds"], errors="raise").to_numpy()
x = pd.to_numeric(df["measurement"], errors="raise").to_numpy()
segments = df["segment_id"].to_numpy()
# Do not resample by default. Choose a grid only when the task requires it.

1. Forward-fill

# Forward-fill is a modeling assumption: the previous value remains valid.
# Use only for known held states, with an explicit maximum elapsed age.
# Do not apply an unlimited ffill() to arbitrary missing measurements.

Breaks when the signal is moving. Forward-fill draws a staircase and invents a plateau that was never measured — fine for a slow set-point, wrong for a ramp.

2. Linear interpolation

candidate, imputed = short_gap_linear(
    t, x, segments, max_gap_seconds=2.0
)
# Internal gaps exceeding the budget or crossing a segment remain missing.
# The original x is unchanged; review candidate and imputed together.

Breaks when the gap spans a peak or fast transient — it cuts the corner and reports a straight line through what may have been the most important part.

3. Spline / polynomial

# A spline is a candidate model, not evidence that missing data are known.
# Evaluate on held-out observed intervals before using it for a real gap.
# Do not let a default interpolator bridge arbitrary long gaps or boundaries.

Breaks when the gap is wide: splines overshoot near edges and invent wiggles that look like real dynamics. Smoothness is not accuracy.

4. Correlated-channel reconstruction

A contemporaneous related channel may provide additional information for reconstruction. Learn the relationship only from available observations and test it on deliberately masked known intervals. Correlation alone does not establish accuracy during a gap or a changed regime.

Breaks when no correlated channel exists, or the relationship itself changed during the gap.

5. Leave it flagged — do not fabricate

Sometimes the honest answer is to mark the gap and not fill it, so downstream analysis knows the data is absent rather than invented. For anything feeding a certification or a paper, a flagged gap beats a plausible fabrication.

Decision guide

FAQ

How do I fill missing values in a time series in Python?

Keep the original time grid unless resampling is needed. Use declared units, explicit continuity and elapsed-time limits; retain a reconstruction mask and leave unsupported gaps missing.

What is the best way to handle irregular timestamps?

Declare whether time is numeric elapsed time or a timestamp, and parse it accordingly. Check reversals and repeats before interpolation. Resampling is optional and changes the data grid; its interval must follow the task.

Should I always fill gaps?

No. Over a transient or a wide gap, reconstructing from a correlated channel or simply flagging the gap is more honest than inventing values that interpolation makes look real.