Guide · Systematic

Trading system development: the engineering lifecycle

Getting an idea to live capital is the build. That path, from a falsifiable hypothesis through the rules, the risk layer and the paper bridge to the first live fill, is covered in the build pipeline guide, and this page does not repeat it. This page begins where that one ends: you have a working system, and now you must operate it professionally, for months, while it decays, while the market changes underneath it, and while a live corporate action or an exchange rule threatens to break it. The reader who survives is the one who ran the thing like an engineer rather than a punter.

The lifecycle is a loop, not a line

The single most useful shift is to stop picturing system development as a project that finishes. A system is never done; it is either in service, under revision, or retired. The stages are the familiar ones from software engineering, mapped onto trading. Research forms a hypothesis about who is on the wrong side of a trade and why. Specification writes that down as unambiguous rules. Implementation turns the rules into a single source of truth, code or a written checklist. Validation tries to disprove the edge on data the rules never saw. Deployment ramps live capital in steps. Monitoring watches live behaviour against the validated numbers. Retirement ends the system on purpose and files the evidence.

What makes it a loop is the feedback. Monitoring does not just observe; when live behaviour drifts past a threshold it pushes the system back into research and revision. And validation is a gate, not a rubber stamp: a change that fails it is returned, not deployed. Drawn honestly, the arrows go both ways.

The specification is a versioned document

The centre of the whole discipline is one idea borrowed straight from software: the rules of the system are a document under version control, and behaviour never changes without the version changing with it. Version one is the spec you validated and deployed. The moment you want to alter a parameter, a filter, an execution assumption or the universe, you do not edit the running system; you write a change proposal, it is re-validated as a candidate, and only if it clears the gates does it become version two.

The rule that makes this real is no silent edits. It sounds bureaucratic until you see what a silent edit is: a quiet tweak made after seeing live or backtested results you did not like. That is not tuning; it is in-sample contamination. Every parameter you nudge to fit outcomes you have already observed borrows information from the future and inflates your expectations, which is precisely the overfitting bias that backtesting integrity is built to defend against. A versioned spec forces every such edit through the same out-of-sample gate the original had to pass, so the system cannot be optimised into a fantasy one convenient nudge at a time.

Implementation discipline: one source of truth

Between the spec and the market sits the implementation, and this is where clean systems and fragile ones part company. Three habits carry most of the weight. First, one source of truth for the rules: the parameters live in exactly one place, whether that is a config block or a written checklist, never copied into two files that can drift apart. Second, configuration over code edits: a value you expect to review, a stop distance, a threshold, a risk fraction, is a named setting you change deliberately, not a number buried in logic that someone patches at 9:20 in the morning. The point is that a change is an event you can see, date and reverse, which is the same instinct as the versioned spec expressed in code.

Third, and most important in production, event-log every order decision with its inputs. For each order, record the timestamp, the instrument, the signal values that fired it, the spec version in force, the intended price and size, and then the actual fill and any error. The standard to hold yourself to is replay: from the log alone, you can reconstruct why any live trade happened and check it against the spec that was live at that instant. When a trade looks wrong, replay tells you in minutes whether the system misbehaved or the market simply moved, and that distinction is the difference between a fix and a panic.

The reason this pays is that a live system fails in ways a backtest never shows you: a data feed hiccups, an order is rejected for margin, a fill comes back partial, a timestamp arrives out of order. None of these are strategy errors, and none of them appear in a clean historical simulation, yet each can corrupt a live position if it passes unnoticed. A log that captures the inputs beside the outcome lets you separate a flaw in the idea from a flaw in the plumbing, which are fixed in completely different places. Without it, every anomaly looks the same, and the temptation is to blame the strategy and start editing rules, the exact silent-edit trap the versioned spec exists to prevent.

Validation gates, and the demotion path

A gate is a promotion condition: a set of things that must be true before a system, or a new version, is allowed to advance to the next stage. Gates are what stop a promising backtest from walking straight onto live capital. Each gate has an evidence bar, something you can point to, not a feeling that it looks ready.

The gate most people skip is the last one. A promotion path without a demotion path is how systems quietly ruin accounts: they are ramped up on good results and then never stepped down when the results turn, because nobody wrote the rule for stepping down. Decide the demotion trigger in numbers, before you go live, for the same reason you decide a stop before you enter: a drawdown is exactly when your judgement is worst. A stated trigger reached means cut size and review, not rationalise and continue.

Two properties make a gate trustworthy. It must be pre-committed, written before the evidence arrives, so it cannot be softened in the moment to let a favourite system through; and it must be measurable, expressed as a number you can check rather than a judgement you can talk yourself past. A gate that reads "looks robust" is not a gate; a gate that reads "at least a stated minimum sample of out-of-sample trades, expectancy still positive after fees and a widened slippage assumption" is one, because it can only be answered yes or no. The out-of-sample requirement in particular is non-negotiable: a system tuned and tested on the same data has, in effect, seen the answer sheet, and its validated numbers describe the past rather than predict the future. The gate exists to force a verdict on data the rules never touched.

Monitoring in production: the four dashboards

Once live, the system generates data faster than intuition can read it, so you watch a small number of instruments that each answer one question. Four cover almost everything that matters, and each is read the same way: a healthy band you expect normal behaviour to stay inside, and an alert threshold that, when crossed, means act rather than watch.

Operating a system in India

The lifecycle above is universal; running it in the Indian market adds three concrete operational duties that a generic guide will not mention.

Automation controls. If any part of the system places orders automatically through a broker API, it falls under SEBI's framework for retail participation in algorithmic trading, issued on 4 February 2025 with implementation extended to 1 October 2025. In outline: every algo order must carry a unique algo identifier so the exchange has a clean audit trail; access runs only through broker-registered algos over whitelisted static IP addresses with two-factor authentication, not open or rotating access; a retail-built strategy exceeding ten orders per second must be registered with the exchange through the broker; and a kill switch must be able to halt the strategy, with exchanges able to disable specific algo identifiers. The engineering consequence is direct: the order-logging discipline you already keep for replay is most of what the audit trail and the kill switch require.

Live corporate actions. A bonus issue, a stock split or a dividend changes the price series on the ex-date, and in production that change arrives while the system is live. The unadjusted price gaps down mechanically, and a naive system can read that gap as a real move, fire a spurious signal, or miscompute a stop. Handling corporate actions in production is the live twin of the data adjustment and survivorship bias you defend against in a backtest: get it wrong on the day and the live system trades a phantom. Know which of your instruments have actions pending, and how your data feed and your rules treat the adjustment.

Expiry-calendar awareness. Any system on futures or options must respect the expiry schedule. Contracts expire on fixed dates, liquidity migrates from the near contract to the next, and a system that does not roll deliberately will find itself holding an illiquid or expiring instrument. The expiry calendar is a hard input to an F&O system, not a detail, and it belongs in the specification alongside the entry and the stop.

Retirement: systems die, so archive the body

Every system has a finite life, and pretending otherwise is how a decayed one keeps trading long past its edge. Systems die of two forces. Regime change: the behaviour the edge harvested, a pattern of panic, a structural flow, a slow diffusion of news, simply stops happening, because the market that produced it has changed. Crowding: enough participants find and trade the same signal that the reason someone was on the other side at a good price disappears, and the edge is competed away. Both show up first as expectancy drift on the dashboard, then, if ignored, as a drawdown.

Retirement is a deliberate stage, reached when the demotion criteria have fired and re-validation cannot restore the edge. Do it properly: stop the system, and archive the full specification history and the production log. That archive is not housekeeping; it is the input to a post-mortem that asks why the edge decayed, what the market was doing when it did, and which of your assumptions failed first. A retired system with its evidence intact makes the next system smarter. One that is quietly switched off and forgotten throws away the most expensive lesson you paid for. The discipline that carried the system through its life, versioned rules, honest gates, a replayable log, is exactly what turns its death into a starting point, and that upstream way of thinking is what the method we teach is built around.