nlsys — Benchmarks: every number, with its protocol

How to read this page

Six terms, once

dB (RMSE-reduction) — how much error the filter removed vs the noisy input; higher is better. Rule of thumb: +6 dB cuts the error roughly in half; a negative number means the filter made the signal worse than doing nothing.

SNR — signal-to-noise ratio of the input: 30 dB is nearly clean, 0 dB is noise as loud as the signal.

on / flt / OFF — the three outputs of the same engine at three latency budgets: on = zero-lag (sees past + current sample only), flt = 80 samples of delay, OFF = offline batch that sees the whole signal.

NRMSE & free-run — model prediction error as a share of the signal's own scale, measured in free-run: the compiled model runs on its own outputs over a held-out tail it has never seen — no teacher forcing, errors compound honestly.

Settling time & effort — how many seconds a controller needs to reach the target and stay, and the total "push" it spends getting there (less = cheaper, gentler on actuators). The oracle is the same controller handed the true equations — the ceiling no controller can beat.

Constraint violation & ms/step — how far a hard limit was crossed (0.000 = never), and the compute one control step costs.

1 · Signal filtration

Eight signal classes, six noise levels, four classical baselines

Denoising on N=2048 samples per run. Metric: dB RMSE-reduction vs the noisy input. Classical baselines: Butterworth (BtFF), Kalman-AR1, Savitzky-Golay, wavelet VisuShrink. The flt output is aligned (lag ≤ 160). Our three outputs are in bold; red means the method added error.

AR(1) slow-linear (fs=100)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	-20.6	-2.5	-22.3	-20.5	-6.7	+0.1	+0.1
20 dB	-10.6	-4.3	-12.3	-11.0	-1.1	+0.3	+0.4
10 dB	-1.1	-1.8	-2.6	-2.4	+1.4	+2.2	+2.6
5 dB	+3.5	+0.6	+2.1	+1.6	+2.0	+4.3	+4.6
1 dB	+6.3	+3.2	+5.5	+4.9	+2.5	+5.5	+6.5
0 dB	+7.1	+3.8	+6.2	+5.9	+2.6	+6.5	+7.3

HeaviSine (fs=100)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	+4.7	+0.7	+3.4	+9.7	-6.7	+8.0	+10.9
20 dB	+9.3	+3.1	+9.2	+10.1	+1.6	+8.1	+12.1
10 dB	+10.0	+5.5	+10.7	+11.2	+3.8	+8.0	+13.4
5 dB	+10.6	+6.1	+11.3	+11.9	+3.7	+8.7	+15.6
1 dB	+11.2	+7.7	+11.8	+12.6	+3.7	+10.4	+15.7
0 dB	+10.4	+7.4	+10.9	+11.4	+3.5	+8.7	+13.9

Doppler non-stationary (fs=1000)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	+11.8	+0.6	+12.6	+13.3	-1.7	+7.6	+6.2
20 dB	+10.6	+3.0	+11.1	+11.8	+4.0	+7.8	+9.5
10 dB	+10.1	+4.7	+10.9	+11.4	+2.7	+9.3	+13.4
5 dB	+10.6	+5.3	+11.4	+12.0	+3.0	+9.2	+14.1
1 dB	+11.2	+5.7	+11.8	+12.6	+3.2	+9.5	+7.9
0 dB	+10.5	+6.9	+10.9	+11.4	+3.2	+9.2	+13.9

Blocks piecewise-const (fs=100)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	-10.8	+0.2	-12.4	+0.7	-5.0	+8.7	+11.2
20 dB	-1.1	+1.2	-2.7	+2.9	+0.9	+9.3	+10.8
10 dB	+6.7	+3.1	+6.0	+6.5	+2.2	+7.9	+9.9
5 dB	+9.0	+4.2	+8.9	+8.9	+2.8	+8.8	+10.4
1 dB	+10.3	+5.4	+10.4	+10.8	+3.1	+9.2	+10.5
0 dB	+10.0	+6.3	+10.1	+10.3	+3.1	+8.7	+11.9

Bumps spikes (fs=100)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	-15.9	-0.3	-18.6	+0.3	-0.7	+4.3	+4.2
20 dB	-6.0	-0.9	-8.7	+2.0	+0.8	+5.8	+6.3
10 dB	+2.9	+0.2	+0.8	+3.1	+1.6	+7.1	+7.8
5 dB	+6.5	+1.5	+5.1	+5.0	+2.2	+6.9	+8.2
1 dB	+9.2	+3.3	+7.9	+7.6	+2.6	+8.7	+9.9
0 dB	+9.0	+4.6	+8.3	+7.5	+2.6	+8.8	+10.6

Van der Pol nonlinear (fs=100)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	+10.3	+0.5	+9.4	+4.8	-0.0	+5.0	+10.5
20 dB	+10.4	+2.7	+10.8	+8.5	+2.5	+7.2	+7.8
10 dB	+10.1	+4.1	+10.9	+11.0	+2.6	+8.0	+12.0
5 dB	+10.6	+5.0	+11.4	+11.8	+3.0	+8.9	+12.9
1 dB	+11.2	+5.4	+11.8	+12.5	+3.2	+9.7	+13.6
0 dB	+10.5	+6.7	+10.9	+11.4	+3.1	+5.1	+13.1

VIB-3 multi-tone (fs=1000)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	-25.2	-6.5	-25.1	-26.5	+5.3	+8.3	+12.5
20 dB	-15.3	-9.3	-15.3	-16.7	+4.7	+8.2	+12.8
10 dB	-5.4	-5.4	-5.3	-6.7	+3.0	+8.2	+12.8
5 dB	-0.8	-2.7	-0.7	-2.0	+3.0	+8.2	+14.1
1 dB	+2.8	+0.1	+3.0	+1.9	+2.8	+7.9	+14.4
0 dB	+3.8	+1.3	+4.0	+2.9	+2.6	+7.1	+9.6

Chirp non-stationary (fs=1000)

SNR	Butterworth	Kalman-AR1	Sav-Golay	Wavelet	on · 0 lag	flt · 80 smp	OFF · batch
30 dB	-29.6	-7.5	-30.0	-17.9	-12.2	+0.2	+0.0
20 dB	-19.7	-11.7	-20.1	-12.1	-2.7	+5.3	+0.0
10 dB	-9.7	-8.1	-10.1	-7.4	-0.2	+4.9	+0.2
5 dB	-4.9	-4.3	-5.3	-4.4	+0.7	+6.7	+2.6
1 dB	-1.0	-0.8	-1.4	-1.0	+1.5	+7.1	+3.9

Where classics break: on multi-tone vibration (VIB-3) and moving spectra (Chirp) every classical baseline goes negative at high SNR — it amplifies error — while flt stays strongly positive. On slow, near-linear signals at high SNR there is simply little noise to remove, and the honest answer is to leave the signal alone — which is what the next table shows.

Do-no-harm on near-clean input

RMSE ×1e3 after filtering an almost clean signal; the input itself = 1, so 1 is the floor — anything above it is damage.

near-clean signal	input	on	flt	OFF
AR(1)	1	62	1	1
HeaviSine	1	18	1	1
Doppler	1	52	1	1
Bumps	1	16	1	1

flt and OFF leave a clean signal essentially untouched: a principled record/replay SURE criterion backs the denoiser off to the raw signal when there is nothing to remove. on is a zero-lag predictor — it over-models clean input by construction, which is why it is the mode for hard real-time consumers, not for archiving.

The latency–quality frontier

Jump	What you pay	What you gain (avg, 30–5 dB)
on → flt	80 samples of latency	+6.1 dB — always positive (+0.8…+14.7)
flt → OFF	whole-signal context	+1.9 dB — small; negative on non-stationary signals

Most of the achievable denoising is bought by the first 80 samples of latency. flt beats even the offline batch on non-stationary signals (Doppler at 30 dB: +7.6 vs +6.2; Chirp at 20 dB: +5.3 vs 0.0) — streaming per-block adaptation tracks a moving spectrum that a single global method choice cannot. OFF wins on globally structured signals (VIB-3, HeaviSine, Van der Pol), where whole-signal SVD captures the global periodicity.

These are the full tables behind the excerpt on the Filtration page.

2 · Model compiler

Free-run on real measured systems — as published

The table below is the one on the Analytics page, consolidated here. Each case is a real measured system from a public nonlinear system-identification benchmark suite; the compiled model is evaluated in free-run on a held-out tail. NRMSE ratio is compiler ÷ best stable classical baseline on the same dataset — below 1 means the compiler won.

Benchmark case	Compiler — free-run NRMSE	Best classical baseline — NRMSE · method	Ratio (ours ÷ baseline)
PUB-001	0.0204	0.0547 · lightweight nonlinear state-space	0.37×	WIN
PUB-002	0.0253	0.0959 · lightweight nonlinear state-space	0.26×	WIN
PUB-004	0.2168	0.3174 · EDMD/DMD	0.68×	WIN
PUB-005	0.0561	0.5577 · EDMD/DMD	0.10×	WIN
PUB-006	0.2583	0.2838 · RLS/Kalman tracker	0.91×	WIN
PUB-008	0.1103	0.1794 · EDMD/DMD	0.61×	WIN

PUB-001…PUB-008 are public case identifiers from the community benchmark set. Want the protocol details for a specific case? Write us — and if you send your own dataset, we run it and publish both numbers, win or lose.

3 · Control

Identified online — measured against the oracle

Canonical benchmark plant: the Duffing double-well, stabilized at its unstable origin. No model supplied — and the identified equation row comes out exact: x1 +1.000 · x2 −0.300 · x1³ −1.000 · u +1.000 (true: +1, −0.3, −1, +1). Because the identified model is exact, the data-driven SDRE is optimal — not a heuristic.

Stabilization region (|u| ≤ 2.0)

Controller	Max stabilizable \|x₀\|
nlsys SDRE — identified online	4.0
Oracle — SDRE on the true model	4.0 — identical to ours
LQR (linearized)	4.0
PID (tuned)	3.0 — smallest region

Efficiency from x₀ = [2.0, 0]

Controller	Settling time	Control effort
nlsys SDRE — identified online	2.11 s	2.36
Oracle — SDRE on the true model	2.11 s	2.36 — identical to ours
LQR (linearized)	1.95 s	3.96 — 1.7× the effort
PID (tuned)	4.47 s	9.67 — 4× the effort, slowest

Hard constraint x₂ ≥ −2.0 — the full table

Method	min x₂	Violation	Cost	Effort	ms / step
SDRE (ours)	−2.182	0.182 — soft	19.18	3.08	1.04 — 15× faster than MPC, 46× than NMPC
Linear MPC	−2.085	0.085	20.82	5.58	16.11
NMPC on our identified model	−2.000	0.000 — exact	21.00	5.05	47.98

The model is the asset; SDRE and NMPC both consume it — trading compute for a hard-constraint guarantee. This is the full version of the tables on the Control page.

4 · MIMO

A hidden channel, recovered from coupling alone

System: rigid-body Euler rotation — x1' = α·x2x3, x2' = β·x1x3, x3' = γ·x1x2 — pure inter-channel nonlinearity. Only [x1, x2] are measured; x3 is hidden. The identified model is exact (+1 · −1.5 · +0.5, B = I).

Hidden-channel estimation: a 3-state estimator on 2 sensors reconstructs the unmeasured x3 to MSE 0.00004 — final true value +0.367 vs estimate +0.369. The unmeasured channel is recovered from cross-coupling alone.

Hard floor on x₂ — violation vs floor (|u| ≤ 2.5)

x₂ floor	our SDRE	linear MPC
−0.85	0.048	0.052
−0.70	0.198	0.102
−0.55	0.348	0.182

Compute & cost at floor −0.70

Method	ms / step	Cost	min x₂
our MIMO SDRE	1.11 — 23× cheaper than MPC, 84× than NMPC	11.51	−0.898
linear MIMO MPC	25.74	11.38	−0.802
multivariable NMPC	93.16	11.52	−0.700

MIMO SDRE drives ‖x‖ from 1.74 → 0 on a system that, uncontrolled, conserves ‖x‖ forever. Linear MPC leaks the constraint for a structural reason: its linearization at the origin drops the cross-channel coupling entirely — only the identified nonlinear model lets MPC/NMPC honour the hard floor.

Honest limits

What these numbers are — and what they are not

Canonical plants and standard signal classes, not your factory floor: benchmarked, not field-proven. Every control and MIMO result is produced with no supplied model — the plant is identified online from the stream, exactly the way the product works on your channels. Numbers change only with a versioned engine release.

Skeptical? Send us your case — we run it and publish both numbers, win or lose. Or check it yourself, today, on your own signals:

Try free — 5 channels, no card

Every number, with its protocol