Guide · Risk measurement · Updated 2026-09-19

Standard deviation is a second moment, and the size of the worst day is not in it

Everything on this page was computed from the exchange's own daily index close files rather than quoted, and the method is stated so it can be redone. No regulatory claim is made anywhere on it, for a reason given in the Sources block.

A second moment, and the thing a second moment cannot see

The standard deviation of a series is the square root of the average squared distance from its mean. That is the whole definition, and the whole limitation. Squared distance is a smooth, symmetric weighting that treats a run of ordinary sessions and one enormous one as interchangeable so long as the squares add up the same way. It is a summary of the middle, which is what it was built to be.

The mistake is not using it. The mistake is the step that usually follows, in which a standard deviation is multiplied by two, or three, and the result is described as a worst case. That multiplication only means something if you have supplied a distribution, and the distribution almost always supplied by default is the normal one. Under a normal, the second moment determines every other moment, so the standard deviation really does tell you everything. Under anything else, it does not, and the gap opens fastest exactly where the money is.

Here is the mechanism in its smallest form, computed exactly rather than asserted. Take two distributions. The first is a normal with a standard deviation of one. The second is a mixture: on 98 per cent of sessions it draws from a normal of standard deviation 0.822, and on the remaining 2 per cent from a normal of standard deviation 4.110. That mixture has a standard deviation of exactly one as well. The two distributions are indistinguishable on the only statistic most risk notes report.

They are not remotely alike at the edge. The mixture has excess kurtosis 15.5 and puts a five standard deviation session at roughly one in 223 sessions, about once every 0.9 years. The normal puts the same event at one in 1,744,278 sessions, about once every 7,091 years. Same second moment; a factor of roughly 7,807 in the thing you care about. This pair is an illustrative construction, not a measurement, and it exists to make one point: the standard deviation did not lie to you, it simply was never being asked the question.

What the record holds, and how it was measured

The data is the exchange archive's daily all index close file, one per session, read directly. 3,385 sessions from 2013-01-01 to 2026-09-18 carry a close for the broad fifty share index, giving 3,373 daily log returns, about 13.7 years. Index names changed in late 2015 and the series is stitched across that renaming. Two integrity guards run before any figure is produced: no calendar year from 2013 to 2025 may hold fewer than 235 sessions, and no gap between consecutive sessions may exceed seven calendar days, since a missing block would masquerade as one enormous return and would poison every statistic on this page. The largest gap present is 6 days.

The daily standard deviation over the whole record is 1.017 per cent, which annualises to 16.1 per cent. Skewness is -1.15, so the left side is longer than the right. Excess kurtosis is 17.6, against zero for a normal. Those two numbers are the conventional summary and, taken alone, they are close to useless: kurtosis is a fourth power average, which means it can be large for reasons that have nothing to do with how often anything happens. So the rest of this page counts events instead.

The ladder, where the error stops being an approximation

Standardise every session by the mean and standard deviation of the whole record, then simply count how many sessions land beyond each threshold, and compare that count against the number a normal distribution would produce in a sample of the same size. No estimation, no fitting, no assumption beyond the arithmetic of the normal itself.

Read the last two columns down the page. At two standard deviations the normal is close enough, and slightly conservative. At three the record holds 4.8 times what a normal predicts. At four it holds 80 times. At five, 5,688 times. At six, over a million times. At seven the ratio is a number with nine digits in front of the decimal point.

That progression is the argument, and it is worth restating in units a reader can hold. A normal says a four standard deviation session arrives about once every 64 years. This record, spanning 13.7 years, contains 17 of them. To accumulate that many under a normal you would wait roughly 1,091 years. A normal says a five standard deviation session arrives about once every 7,091 years, and this record contains 11. The normal assumption is not slightly optimistic about the tail. It is wrong by a factor that grows without limit as you move outward, which is the defining property of comparing a distribution with exponentially decaying tails against one whose tails decay far more slowly.

The worst session in the record, 2020-03-23, closed -12.98 per cent lower, which is 13.72 standard deviations. Under a normal that has a probability of roughly ten to the minus 42, which is to say one session in ten to the power 42, or about ten to the power 40 years. The universe has existed for something on the order of ten to the power ten years. That is not a rhetorical flourish, it is what the arithmetic returns, and it is the clearest possible statement that the model and the event belong to different worlds. The largest rise, on 2020-04-07, was 8.76 per cent, or 8.22 standard deviations, which a normal puts at one in ten to the power 16 sessions.

The middle is thinner than normal, not fatter

This is the result that most surprises people who expect a fat tailed series to be uniformly wilder, and it is worth pausing on because it corrects a common mental picture. Count the sessions in every band, not just the extreme one.

Only two bands exceed what a normal predicts, and they are the two ends. The quietest band holds 1.31 times its normal share; the band beyond three standard deviations holds 4.83 times. Every band in between is under what a normal predicts, bottoming at 0.62. The median session moved 0.499 standard deviations against a normal's 0.674, so half the record is materially quieter than the model. At the ninety ninth percentile the relationship inverts: 3.28 against 2.58.

That is what a fat tail is. Not extra movement spread evenly, but calm traded for catastrophe. The practical consequence is that a normal model does not feel wrong in ordinary use. It feels conservative, because on most days it over predicts the movement you actually get. The bill arrives in one place, rarely, and all at once. A measurement of the concentration says the same thing: the single largest session accounts for 5.58 per cent of the total squared return in the whole record, and the largest one per cent of sessions account for 28.4 per cent of it. A normal draw of the same size gives 0.49 per cent and 7.9 per cent.

One session did most of the work, and the statistic knows it

Before treating 17.6 as a property of Indian equities, find out where it comes from.

Removing one session out of 3,373 takes the excess kurtosis from 17.6 to 8.3. Removing ten takes it to 2.7. The exceedance count barely moves across the same operation, staying between 44 and 47, which is the first practical reason to prefer counts to moments: a count is not dominated by its largest member and a fourth moment is.

A delete one jackknife over the full record puts the standard error of the excess kurtosis at about 9.4, which against a point estimate of 17.6 gives an ordinary interval of roughly -1 to 36. The point estimate is not two standard errors from zero. That does not mean the tail is not fat, since the exceedance counts settle that question on their own terms. It means the specific statistic most often quoted for fat tails is, on thirteen years of daily data, barely distinguishable from its null.

Through 2019 the record said 3.1, which is a mildly fat tail and would have supported a calm risk note. One session in March 2020 took it to 20.8, where it has broadly stayed. Nothing about the market changed on that date in a way any earlier statistic had captured; the sample simply acquired the observation it had been missing. Anyone who had measured the tail in 2019 and written it down as a fixed property of the series would have been carrying a number too small by a factor of six, and would have had seven years of data supporting it.

The tail you measure depends on the window you measured it over

Everything so far standardised by one number covering thirteen years. That choice is doing far more work than it appears to. A single standard deviation estimated across calm and stressed periods together is an average of two different market states, and mixing states manufactures kurtosis on its own, entirely separately from whether either state has a fat tail.

The test is to standardise each session by the standard deviation of the sessions immediately before it, using only information available at the time, and to repeat that for several window lengths on exactly the same set of sessions so that nothing but the estimator changes.

Excess kurtosis runs from 1.9 to 23.7 across that table, a factor of 12.6, on the same sessions, with no change to the data whatsoever. The count of sessions beyond four standard deviations runs from 9 to 17. One of those statistics is measuring the market and the other is mostly measuring the analyst's choice of window.

The honest reading has two halves and both matter. First, most of the headline fatness is volatility clustering seen through a long lens: track the standard deviation locally and excess kurtosis collapses from 21.9 to 1.9. Second, it does not go away. Even with the shortest window, where the estimate is following the market state closely, 9 sessions still land beyond four standard deviations against 0.17 predicted, and 2 beyond five against 0.0015. A residual tail survives after the clustering has been removed by construction. Anyone claiming fat tails are purely an artefact of changing volatility has to explain those.

The extreme days arrive together, which is the second failure

A normal assumption smuggles in two claims, not one. The first is the shape of each session's distribution. The second, quieter and more damaging, is that sessions are independent draws. The first failure has been measured above. The second is easier to measure and worse in its consequences.

The direction of tomorrow is very close to unpredictable from today: the correlation between one session's return and the next is -0.0044. The size of tomorrow is a different matter entirely. The correlation between one session's absolute move and the next is +0.232, and it is still +0.185 ten sessions later. Extreme sessions do not sprinkle themselves across the calendar. They come in bursts.

The sharpest way to measure that is to hold the distribution exactly fixed and vary only the arrangement. Take the same 3,373 returns, reshuffle them at random, and ask the same question of each reshuffle. Every reshuffle has identical mean, standard deviation, skewness, kurtosis and exceedance count, by construction, because it contains exactly the same numbers. The only thing destroyed is the sequence.

The record packs 7 three sigma sessions into a single ten session window, between 2020-03-09 and 2020-03-23. Across 20,000 reshuffles of the identical returns the average maximum was 2.20 and the largest ever reached was 4. Not one reshuffle out of 20,000 produced what the record contains. The same test on the worst ten session cumulative move gives the same verdict: the record's worst stretch was -30.75 per cent, the typical reshuffled worst was -14.78 per cent, and the worst of 20,000 reshuffles reached only -28.97 per cent.

That is a clean separation. Fat tails are a statement about the distribution of one session. Clustering is a statement about the arrangement of many, and it is a separate, independently measurable fact that no amount of fixing the marginal distribution will address.

Why the second failure is the one that matters under leverage

A single bad session is survivable by a position that was sized for it. A run of them is a different problem, because losses compound against a position that has not been given the chance to recover, and because a margin call does not wait for the distribution to mean revert.

Put it in the units a risk budget actually uses. Suppose a position is sized so that a one standard deviation session, where the standard deviation is estimated from the previous 21 sessions, costs exactly one unit of budget. That is the ordinary construction and it uses only information available before the session.

The worst single session consumed 7.0 budgets. That is bad and it is survivable at modest leverage. The worst ten session stretch consumed 10.0 times the ten session budget, which is the same failure compounded across a fortnight rather than delivered in one day. A position sized on a standard deviation, with an implicit assumption that sessions arrive independently, is sized correctly for the first row of that table and not for the third. Under leverage the third row is the one that decides whether the position still exists when the market turns.

This is also why the worst decline in a record is a poor guide to the worst decline available: the observed one is a single draw from a process whose extremes cluster, and a longer record has a deeper one almost by construction.

It is not one index's peculiarity

A single series can always be an accident. The same measurement across the other published index series, over exactly the same sessions, says whether this is a property of the market or of one index.

Every series holds more sessions beyond three standard deviations than the 9.1 a normal predicts, and every series holds more beyond four than the 0.21 a normal predicts. Excess kurtosis runs from 3.3 on the realty series to 17.6 on the broad fifty share index series.

Note which series sit at the thin end, because it runs against the expectation people bring to this table. The lowest excess kurtosis belongs to the realty series, whose daily standard deviation is 1.96 per cent, nearly double the broad index's 1.02 per cent. The most violent series has the thinnest relative tail, for a reason that is arithmetic rather than economic: kurtosis measures the extreme relative to the standard deviation, and when the standard deviation is already large, a bad session is a smaller multiple of it. A low kurtosis is not a claim of safety, and reading it as one is the same category error as reading a standard deviation as a worst case. The two rankings answer different questions, and a risk note needs both. The same distinction shows up in how correlation behaves across regimes, where a ratio and a level tell opposite stories.

What to use instead of a standard deviation

None of this argues for abandoning the standard deviation. It is the right instrument for the middle of a distribution, it is what position sizing is built on, and there is no better single number for ordinary conditions. The argument is against the second step, in which it is multiplied and called a worst case. Three replacements, in rising order of effort.

Count the exceedances. The simplest and most robust. For whatever thresholds matter to the book, count how many sessions in the record crossed them, and compare that against what your model predicts for a sample of the same size. It requires no distributional assumption, it is invariant to the choice of estimation window in a way kurtosis is not, as the window table above shows directly, and a discrepancy is immediately legible. If a model predicts 0.21 sessions beyond four standard deviations and the record contains 17, that model is not usable for tail work and the count said so in one line.

That table is the whole problem in one place. A normal derived threshold at 95 per cent was breached 131 times against 169 expected, so it was conservative. At 99.9 per cent it was breached 23 times against 3.4 expected, nearly 7 times too often. A model that passes routine validation at ninety five per cent and fails by a factor of 7 at the level actually relevant to survival is the most dangerous kind, because it accumulates a record of being checked and passing.

Measure the conditional average beyond the threshold. A threshold tells you how often you are wrong. It says nothing about how badly, and that is the question that determines whether a breach is an inconvenience or an ending. Average the moves that actually went past the threshold.

Under a normal, crossing a threshold barely overshoots it: beyond three standard deviations the conditional average is -3.28, an overshoot of 0.28. In this record the conditional average beyond the same threshold is -4.82, an overshoot of 1.82, about 6.4 times as far. In plain terms, the sessions that broke the threshold broke it by -4.7 per cent on average, not by a whisker. A normal not only under counts the breaches, it under sizes each one, and the two errors multiply.

Reprice the book at real moves. The least statistical and often the most useful. Take the largest moves this record actually contains, apply them to the position as it stands today, and read the result. Not a scaled standard deviation, not a fitted distribution, the actual moves. This requires no assumption about shape at all, it is reproducible by anyone with the same file, and it converts an abstract tail into a rupee figure against a real book. The weakness is that it can only show you what has already happened, which is the honest limit discussed next, and the correct response to that is to scale the observed moves upward deliberately rather than to assume they are the ceiling.

A practical note on horizon. Aggregating to longer holding periods does thin the tail, as it should, but it runs out of data quickly.

Excess kurtosis falls from 17.6 at one session to 7.3 over five session blocks, which is the expected thinning. Over ten and twenty one session blocks it reads 20.0 and 17.4, back near the daily figure. That is not a market result. There are 328 and 149 non overlapping blocks of those lengths in the entire record, one of each contains March 2020, and a fourth moment computed on so few observations is dominated by whichever one is largest: starting the ten session blocks on each of the ten possible sessions gives anything from 3.0 to 32.6 on the identical returns. The numbers are correctly computed and they are worth almost nothing, which is exactly the trap this page is about.

The honest limit: estimating the rare from the few

Every figure above is an estimate from a finite sample, and the specific difficulty with tail estimates is that the sample size that matters is not the number of sessions but the number of extreme sessions. This record holds 3,373 sessions and 17 of them lie beyond four standard deviations. The effective sample for a statement about the four standard deviation region is 17, not 3,373, and 17 observations support very little.

The instability is measurable rather than rhetorical. Excess kurtosis by calendar year in this record runs from 0.18 in 2023 to 12.48 in 2020, with a median across years of 1.59. The whole record reads 17.6, which is higher than every individual year in it. That is not a contradiction: pooling across years mixes volatility regimes, and the mixing itself generates kurtosis, which is the same effect the window table isolated. But it does mean the pooled figure is not an average of the annual figures and should not be described as the market's kurtosis, as though the market had one.

The jackknife interval stated earlier, roughly -1 to 36, is the formal version of the same point. So is the expanding record table, where seven years of data supported one answer and a single session replaced it. So is the drop one table, where one session out of 3,373 carries more than half the statistic. Anybody who tells you the Indian equity tail index is a particular number, to two decimal places, is reporting a point estimate from a handful of observations and omitting its uncertainty.

The correct response is not paralysis. It is to choose instruments that degrade gracefully under this uncertainty. An exceedance count is robust because it does not weight by a high power. A conditional average is less robust but is at least reporting something directly observed. A repricing at historical moves makes no distributional claim at all. A fitted tail parameter quoted to two decimals from 17 observations is the opposite, and the confidence it projects is the least earned quantity in the whole exercise.

It is also why the more useful question is usually not how fat the tail is but how much of a fat tail a position can absorb. That question has an answer you control, and it is answerable today with the sample you have. The related pieces sit next to it: how much data would settle a statistical question at all, what a result should be compared against, and how a pooled figure hides the market states it averaged over. All three are versions of one discipline: state what the number is an estimate of, state how much data stands behind it, and never let a summary statistic answer a question it was not measuring.