Guide

Remove noise from sensor data: which filter, and when

If you record from a DAQ, a load cell, a thermocouple or an IMU, you have noisy channels. Here are the methods engineers actually reach for to remove noise from sensor data in Python, with runnable code and — more usefully — the exact failure mode of each, so you pick the right one instead of the first one.

Moving average and low-pass (Butterworth)

The cheapest denoiser is a moving average; its principled cousin is a Butterworth low-pass. You choose a cutoff frequency and everything faster is attenuated.

import numpy as np
from scipy.signal import butter, filtfilt

fs = 1000.0              # sample rate, Hz
cutoff = 20.0            # keep everything below 20 Hz
b, a = butter(4, cutoff/(fs/2))
clean = filtfilt(b, a, x)   # filtfilt = zero phase lag (offline)

Where it fails. A single cutoff is only right for one part of the record. Pick it for the quiet stretch and you smear the fast transients; pick it for the transients and you keep the noise. On non-stationary signals (a chirp, a load step, a vibration whose character changes) a low-pass can actually add error at high SNR — you would have been better off leaving the signal alone. A causal moving average also adds lag; filtfilt cancels the lag but only works offline on a full record.

Savitzky-Golay

Savitzky-Golay fits a low-order polynomial in a sliding window, so it smooths while preserving peak height and width far better than a moving average — which is why it is a lab favourite for spectra, chromatograms and voltammograms.

from scipy.signal import savgol_filter
clean = savgol_filter(x, window_length=51, polyorder=3)  # window odd, order < window

Where it fails. You still hand-pick window length and polynomial order, and the right pair depends on the local signal. Too wide a window flattens real features; too narrow keeps the noise. The moment the signal’s character changes mid-record, one setting is wrong somewhere.

Wavelet denoising

Wavelet shrinkage (e.g. VisuShrink) transforms the signal, thresholds small coefficients as noise, and inverts. It handles sharp features and some non-stationarity better than a fixed low-pass.

# pip install PyWavelets
import pywt, numpy as np
coeffs = pywt.wavedec(x, "db4", level=4)
thr = np.std(coeffs[-1]) * np.sqrt(2*np.log(len(x)))
coeffs = [coeffs[0]] + [pywt.threshold(c, thr, "soft") for c in coeffs[1:]]
clean = pywt.waverec(coeffs, "db4")

Where it fails. Wavelet and threshold choice matter, artefacts (ringing near discontinuities) appear if you over-threshold, and it is easy to get subtly wrong without checking the residual.

Kalman filter

A Kalman filter is optimal — if you have a state-space model of the system and its noise covariances. It is excellent for tracking and sensor fusion (an IMU on a moving platform, a battery equivalent circuit).

Where it fails. For most bench data you do not have a model, and building one per channel is a project. A poorly specified model quietly biases the estimate. Kalman is the right tool for known dynamics and the wrong tool for a generic noisy trace.

Quick reference

Method	Preserves peaks	Needs a model	Best for
Moving average / Butterworth	No	No	stationary, mildly noisy
Savitzky-Golay	Yes	No	peaks, spectra, voltammograms
Wavelet shrinkage	Mostly	No	sharp features, some non-stationarity
Kalman	n/a	Yes	tracking with known dynamics

The common thread

Every classical method asks you to choose a parameter — a cutoff, a window, a wavelet, a model — that is only correct for part of the record. On real, non-stationary sensor data that is the whole problem. The honest fixes are to segment the record and tune per segment, or use a self-tuning method that adapts per block. Whatever you use, test it on an already-clean stretch first: a good denoiser leaves a clean signal essentially untouched. If your filter visibly changes a clean segment, it is over-smoothing and will erase real dynamics elsewhere.

FAQ

How do I remove noise from sensor data in Python?

Start with scipy.signal: a Butterworth low-pass (butter + filtfilt) for stationary data, or savgol_filter to preserve peaks. The hard part is choosing the cutoff or window, because one setting is only right for part of a non-stationary record.

Which is better, Savitzky-Golay or a moving average?

Savitzky-Golay preserves peak height and width far better because it fits a local polynomial, so prefer it for spectra and peaks. A moving average is simpler but flattens features.

Why does my filter make the signal worse?

On non-stationary signals a fixed cutoff or window that suits the quiet part smears the transients and can add error at high SNR. Test the filter on an already-clean segment; if it changes that, it is over-smoothing.