Setup Performance Index (SPI): How We Compute It
Data sources, detection engine, backtest simulation, and every assumption baked into the numbers you see on our pages.
The Setup Performance Index (SPI) is a rolling record of how every setup our detector finds has resolved historically. For each combination of coin, timeframe, setup type, direction, and management style, we run an event-driven simulation on real Coinbase candles going back as far as each coin's history allows. Detection is causal, with no look-ahead: a setup is only entered from the bar its defining conditions actually closed on, so nothing after entry can change whether it fired. When we removed look-ahead from our own detectors, most grains dropped to a D, and that honest picture is what we publish. Each row reports win rate, expectancy in R, hold time, reversal rate, maximum favorable and adverse excursion, projected annual gain and max drawdown at 1, 2, and 5 percent risk per trade, and an A-to-D grade blending edge, drawdown and reward-vs-dip. Every metric is net of a 0.12 percent round-trip fee model. Rows are surfaced only once the grain has at least 20 instances. Strategies that graduate into our published, live-traded library are tested harder still: held out of sample, re-run across the least correlated coins, and passed through a random-entry timing null (a permutation test) that strips out setups whose apparent edge is really just market drift or geometry. The ones that survive are then paper traded live, and we compute a separate live A-to-D grade from their real forward trades. Backtest numbers are historical simulation, not a forecast; the live grade is the honest forward proof, and we weight it above any backtest.
What SPI is
The Setup Performance Index is a per-(coin, timeframe, setup type, direction, management mode) record of how instances of that setup have resolved on real historical price action. Every learn article, every setup card, and every backtest table on Botsfolio reads its numbers from the same underlying SPI store. The index is versioned by the exact detection code that produced it, so when we improve a detector, historical instances get re-scored under the new rules.
Data sources
All price data comes from Coinbase Advanced Trade public candles, fetched from the unauthenticated market-data endpoint (api.coinbase.com/api/v3/brokerage/market/products/{id}/candles). This is real settled trade data from a regulated US exchange, not synthetic. We pull hourly candles for the six-month rolling window and daily candles for the multi-year backtest window. Timeframes not natively served by Coinbase (4H, 8H, 12H, 1W) are derived by standard OHLCV resampling from the finer native series, with epoch-aligned bucket boundaries.
No third-party price feeds, no synthetic data, no user-submitted signals. Every SPI row can be independently reconstructed from the raw Coinbase candle series.
Detection engine
Setups are detected by pure functions running on closed candles only. Once a candle closes, the analysis engine runs a dependency-ordered detector suite over the recent series and emits structural objects: order blocks, fair value gaps, liquidity sweeps, breaker blocks, mitigation blocks, break-and-retest, wave signals, and points of interest.
Every object has an immutable formation anchor and a mutable lifecycle status that tracks its reaction as new candles close: active, tested, mitigated, or invalidated. Objects never repaint. If a wick pierces a level intra-candle but the close is on the other side, the level's status only changes based on the closed candle.
For zone-based setups (order blocks, FVGs, POIs), we compute a refined level inside the wider zone by clustering the wick tips of surrounding candles that have already tested the zone. The refined level is typically 5 to 15 percent of the zone width and is the actual trigger price used in the backtest. Sizing off the wide zone would systematically underestimate the trader's position size at their real intended stop.
No look-ahead: causal detection
The most common way a backtest lies is look-ahead bias: confirming a level or a zone with candles that had not yet closed at the moment the setup would have been entered. It inflates every number, because the detector is quietly borrowing from the future.
Our detection is causal. A setup is only actionable from the first bar at which every condition that defines it has already closed: a zone is not treated as an order block until the bar that establishes it closes, and a sweep-and-reclaim is not actionable until the reclaim bar closes. The simulation enters no earlier than that bar, and nothing that happens afterward can change whether or when the setup fired.
This is the single most important choice on this page. When we removed look-ahead from our own detectors, most setup grains that had looked strong collapsed to a D grade. That honest result is the one we publish, rather than the inflated numbers look-ahead would produce.
Backtest simulation
For each detected instance, we run a walk-forward simulation from the moment the setup becomes actionable until one of three exit conditions fires:
- The invalidation level is reached (loss).
- The first target is reached (partial or full win, depending on management mode).
MAX_BARS = 60candles have passed without either (time stop, resolved at the running unrealized R).
Two management modes are simulated for every instance:
- Book early: partial exit at +1R, remainder trailed with structural stop moves.
- Book at target: hold to the full first target with no partials.
Both modes are reported side by side because they represent different trader profiles and neither is universally better. We publish the numbers for both without steering.
Cost model: COST_PCT = 0.0012 (0.12 percent round-trip fees). This is on the high end of retail perp fee schedules and errs conservative. Real fees on higher tiers or spot-only exchanges are usually lower. Every expectancy R and annual gain figure has fees already deducted.
R:R cap: RR_CAP = 5. Winners exceeding a 5R payout are capped at 5R in the sample. This prevents rare outlier winners from distorting mean expectancy.
What SPI measures
Each SPI row exposes:
- N: total historical instances at this grain
- Win rate: percentage of instances that finished with positive R
- Expectancy R: mean net R per instance after fees, R:R cap applied
- Avg hold: mean bars held from entry to exit
- Median MFE R: median maximum favorable excursion in R (best point during the trade)
- Median MAE R: median maximum adverse excursion in R (worst point during the trade)
- Reversal rate: percentage of instances that reached +1R at some point and then finished at or below breakeven
- Annual gain (%) at 1%, 2%, 5% risk: projected annual return if the same strategy were traded continuously at each risk-per-trade level, net of the 0.12 percent round-trip fee applied to every trade
- Max drawdown (%) at 1%, 2%, 5% risk: worst peak-to-trough equity decline over the sample, net of the same fee
Setup grade (A to D)
Alongside the raw numbers, each setup grain carries a single A-to-D grade so a strong record is legible at a glance. The grade blends three axes, all computed from the SPI row:
- Edge (45 percent) — expectancy in R per trade, net of fees. Does it make money at all.
- Drawdown (30 percent) — worst peak-to-trough equity decline at 2 percent risk. Could a real trader actually hold it through the losing stretches.
- Reward versus dip (25 percent) — median favorable excursion against median adverse excursion. How much heat you sit through before it works.
Two hard floors force a D regardless of the rest: non-positive expectancy, or a drawdown of 55 percent or more. A grain with fewer than 20 instances is marked provisional, and full confidence is reserved for 100 or more. The grade is descriptive: it summarizes a historical record. It is not a recommendation and not a forecast.
What SPI does NOT measure
- Future returns. Past performance in backtest does not forecast future performance. Regime shifts, liquidity changes, and market structure evolution all affect forward outcomes.
- Real fill quality. The simulation assumes the trigger price fills. Real orders may fill worse due to slippage, especially on lower-liquidity coins during high-volatility events.
- Exchange-specific execution risk. Latency, downtime, and margin call mechanics vary by venue and are not modeled.
- Weekend and gap risk. Crypto trades 24 by 7 but liquidity is thinner on weekends. Position risk during these windows may exceed the modeled invalidation distance.
Sample thresholds
A row is only surfaced publicly once its sample size crosses MIN_SAMPLE = 20. Below that, we display the row greyed out with a caveat, or fall back to a wider scope (for example, aggregating across long and short when a directional split is thin). This is stricter than most public backtest sites and deliberately so. A 60 percent win rate on 8 instances is directionally interesting but statistically noisy. We label it as such.
Validating user-built strategies
When you describe your own strategy in plain words, it is put through a stiffer test than a single backtest, because a strategy that only looks good on one coin over one window is usually noise:
- Out-of-sample holdout. The history is split by time (roughly 70/30). The strategy is measured on the earlier slice and then re-graded on the later slice it never saw. If the edge does not survive the held-out slice, we say so plainly.
- Cross-coin generalization. The same rules are re-run on two of the least-correlated major coins. A real edge tends to hold on more than one market; one that works only on the coin it was fit to reads as coin-specific, not a broad edge.
- Walk-forward consistency. Returns are broken into sequential folds to check the edge is spread across the sample rather than carried by one lucky stretch.
The random-entry timing null
A strategy can look profitable for a boring reason: the coin simply drifted in its favor while it happened to be in a trade. To rule that out, we run a random-entry timing null, a permutation test. We keep the exact trade geometry (the same stop, target and hold logic) but fire it at random times across the same history, many times over, to build the distribution of returns you would get from timing alone. A strategy passes only if its real edge beats that random-timing distribution at the 5 percent significance level.
Because our backtest grade is optimistic by construction (it selects the best-performing exit from the whole management menu), this timing null is the gate that keeps overfit and geometry-only winners out. It is the single test that most separates a real, repeatable edge from a lucky-looking curve.
The published library: only survivors are shown
Not every backtested setup earns a place in our published, live-traded library. A strategy is published only when it clears every gate at once:
- A backtest grade of A, B or C on the setup grade rubric above.
- At least 30 instances in the sample.
- Cross-coin generalization: the same rules still hold on the least correlated major coins, not only the one they were fit to.
- Out-of-sample survival: the edge holds on the later slice of history the strategy was never measured on.
- Beats the random-entry timing null at the 5 percent level.
A published strategy that later stops clearing these gates on two consecutive scans is retired, so the library reflects current edge rather than a one-time result. This is why we lean on these grades: each gate targets a different way a backtest can mislead (fitting to one coin, a lucky sample, an edge that fades out of sample), so a number that is merely inflated or lucky is unlikely to clear all of them at once.
Live grading: the forward proof
Everything above is still simulation on history. The test that outranks all of it is live paper trading. The moment a strategy is published, our agent paper trades it forward in real time, on candles it was never measured against, with fees and slippage applied.
From those real forward trades we compute a separate live grade (A to D), using the same rubric as the backtest grade but on the strategy's own live results. A strategy needs at least 25 closed paper trades before it earns a live letter; until then its live grade reads building. Every setup card can therefore carry two grades side by side: the backtest grade (what history says) and the live grade (what is actually happening forward).
We weight the live grade above any backtest. A backtest can be overfit to the past; a forward paper record on candles the strategy never saw cannot be fit to them. It is the strongest evidence we have that an edge is real, though a short live record can still be luck, which is why it needs 25 closed trades before it earns a letter.
How often it updates
SPI is recomputed as new candles close and new setup outcomes resolve. The rolling backtest window advances with every daily maintenance run. The API and the pages that consume it always show the current computed state, timestamped.
Limitations we take seriously
- Survivorship in the coin list. We backtest coins that currently trade on Coinbase Advanced Trade. Delisted coins are not in the sample. This biases results slightly upward.
- Regime dependence. The sample window includes multiple market regimes (bull, bear, chop). Aggregate SPI blends across them. Individual future periods may deviate from aggregate.
- Detector evolution. When we improve a detector (better refined-level clustering, tighter reaction thresholds), we re-score historical instances under the new rules. This means SPI values can shift as our methodology improves. Every material change is logged in the changelog below.
Not investment advice
Botsfolio publishes market structure observations and historical simulation results. We do not provide investment advice, do not issue trade signals, and do not recommend that any specific trade be taken or avoided. Every reader is responsible for their own trading decisions.
Changelog
- 2026-09-30: Documented the random-entry timing-null permutation test, the full published-library gate (grade A-C, 30-instance floor, cross-coin generalization, out-of-sample holdout, timing null), and live grading: the separate live A-to-D grade computed from real forward paper trades and weighted above any backtest.
- 2026-09-20 — Documented causal (no-look-ahead) detection, the A-to-D setup grade, deeper multi-regime history, and the out-of-sample plus cross-coin validation applied to user-built strategies.
- 2026-08-18 — Initial methodology publication.
Related
Educational analysis, not financial advice. Past performance does not predict future results.