How we test
An independent lab has to show its work. Here's exactly how every bot is evaluated — the same protocol, every time.
We are not a bot seller, a signals service, or a developer-for-hire. We are an independent testing laboratory — think of a used-car inspection or a UL certification. We have no financial stake in whether your bot passes, which is the entire point: the verdict means something because we don't profit from it being positive.
The four-stage protocol
1 · Code audit (can it trade at all?)
We read the strategy's source and replay its logic candle by candle. This surfaces the defects that make trading impossible — contradictory rules, state that never advances, take-profit targets placed on the wrong side of entry. These are bugs no amount of parameter tuning can fix, and they're invisible in a vendor's marketing.
2 · Multi-year backtest (did it ever have an edge?)
We replay the strategy across years of real market data — through multiple bull and bear regimes, not a flattering recent window. We report win rate, average result per trade, drawdowns, and an honest comparison against simply buying and holding the same asset.
3 · Fee & slippage reality (do costs kill it?)
Every backtest is run twice: once at zero cost to isolate the raw signal, and once with realistic exchange fees and slippage. A strategy that looks profitable on paper but bleeds out once real costs apply is a common — and fatal — failure mode we make explicit.
4 · Robustness grid (is it luck or a real plateau?)
We sweep the strategy's key parameters across a grid. A genuine edge holds up across a broad range of settings; a curve-fit one works only at the single combination its author happened to pick. We also flag when trade count is too low to distinguish skill from chance.
The verdict scale
Every report ends in one of four standardized classifications. Same scale for every bot, so the verdict is legible and comparable.
| Classification | What it means |
|---|---|
| Mechanically broken | The code cannot place valid trades — defects or contradictory logic. No tuning fixes it. |
| No historical edge | It trades correctly, but showed no profitable edge over multi-year data. |
| Edge with caveats | A real edge exists, but it's fragile — fee-sensitive, regime-dependent, or thinly traded. |
| Validated | Holds up across the full test battery, costs included. |
What a verdict is — and isn't. We report what your bot did against historical data and how it's built. We never predict future profits, and a "validated" result is not a promise of returns. Past performance does not predict future results. This is educational analysis of software you own, not investment advice.
What we can and can't test
Best results: readable source code (Python especially) on crypto markets — a full code audit plus backtest. Limited: a compiled or black-box bot can't have its code audited, but we can backtest its stated rules. Out of scope: we don't fix, optimize, or tune bots, and we never need or accept your exchange API keys — if anyone asks for your live keys to "check" a bot, walk away.
How we handle your code
Your submitted code and files are deleted from our systems within 7 days of report delivery — on an automated schedule, not our memory. We never reuse submissions to build our own bots and never share your code with anyone; we keep only the anonymized verdict, never your source. Any public teardown is fully anonymized. We're a validation service, not a strategy shop.