Nearly Every NPS Confidence Interval Is Too Narrow

Net Promoter Score is a difference of two percentages:

$NPS = % Promoters - % Detractors$

Two percentages, one difference. So the standard error is the usual root-sum-of-squares, isn't it?

$SE_{naive} = \frac{P ( 1 - P )}{n} + \frac{D ( 1 - D )}{n}$

No. That formula assumes the two proportions are independent, and they emphatically are not: they are computed from the same respondents, on the same question, and they are mutually exclusive. Every respondent who turns out to be a promoter is a respondent who could not have been a detractor. The two estimates are negatively correlated by construction, and a negative covariance entering a difference makes the variance larger, not smaller.

The naive formula therefore understates the standard error — in the scenarios below, by between 16% and 26%.

Part 1 — The correct variance, in one line

The derivation is short enough to do in your head once you see the trick. Don't think of NPS as a difference of two statistics. Think of it as the mean of a single per-respondent score.

For respondent $i$ , define:

$s_{i} = ⎩ ⎨ ⎧ + 1 0 - 1 promoter (9-10) passive (7-8) detractor (0-6)$

Then $\overline{s} = % P - % D = NPS$ exactly. NPS is a sample mean, and the variance of a sample mean is a first-year formula.

Since $s_{i}^{2} = 1$ for promoters and detractors and $0$ for passives, $E [s^{2}] = P + D$ , so:

$Var (s_{i}) = E [s^{2}] - (E [s])^{2} = P + D - (P - D)^{2}$

and therefore:

$Var (NPS) = \frac{P + D - ( P - D ) ^{2}}{n}$

That is the whole thing. No covariance term to look up, because framing NPS as a mean absorbs it.

How far wrong is the naive version?

Expand both. The naive variance is $\frac{1}{n} [P - P^{2} + D - D^{2}]$ ; the correct one is $\frac{1}{n} [P + D - P^{2} + 2 P D - D^{2}]$ . The difference is exactly:

$Var_{correct} - Var_{naive} = \frac{2 P D}{n}$

which is the $- 2 Cov (P, D)$ term, since for a multinomial $Cov (P, D) = - P D / n$ .

The missing term grows with the product $P D$ . It is largest when promoters and detractors are both substantial — a polarised customer base — and smallest when one group is nearly empty. Polarised brands are exactly the ones where people most want to track NPS movements.

Part 2 — What it costs, simulated

Multinomial samples of $n = 500$ , 40,000 replications per scenario. "Realised SD" is the actual standard deviation of NPS across replications — the thing a standard error is trying to estimate.

Scenario (D / Passive / P)	Realised SD	Exact SE	Naive SE	NPS-as-proportion
Typical B2B (20 / 30 / 50)	3.505	3.489	2.861	4.263
Strong brand (10 / 25 / 65)	2.992	2.986	2.516	3.731
Weak brand (40 / 35 / 25)	3.577	3.538	2.921	4.419
Polarised (35 / 10 / 55)	4.172	4.143	3.079	4.378

(All figures in NPS points.)

The exact formula tracks the realised standard deviation to within about 1%. The naive formula understates it by 18.4%, 15.9%, 18.3% and 26.2% respectively — worst, as predicted, on the polarised distribution.

The third column is a different error I see often: treating NPS as though it were itself a proportion, rescaling to $[0, 1]$ and applying $p (1 - p) / n$ . That runs 20–25% too wide in most scenarios. It is conservative rather than dangerous, but it is still wrong, and on the polarised case it happens to land close to correct for entirely the wrong reason.

Confidence interval coverage, nominal 95%:

Scenario	Exact	Naive	NPS-as-proportion
Typical B2B	94.9%	89.1%	98.3%
Strong brand	94.9%	89.8%	98.5%
Weak brand	94.7%	89.2%	98.3%
Polarised	94.8%	84.9%	96.0%

The part that actually bites: comparing two groups

A single slightly-narrow interval is a modest sin. The damage compounds when you compare two NPS figures — this segment against that one, this quarter against last.

Two independent samples of $n = 500$ , identical underlying distributions, so every rejection is a false positive. 40,000 replications, testing at the 5% level:

Method	Type I error rate
Exact variance	5.41%
Naive variance	11.23%

The naive approach rejects more than twice as often as it should. A tracker comparing NPS across a dozen segments every quarter, using the naive formula, is manufacturing "significant" movements at better than one in nine.

Part 3 — Weights, and the general case

Real NPS data is weighted, which rules out the closed form above but not the approach. Because NPS is a mean of $s_{i} \in {- 1, 0, 1}$ , it is a Hájek ratio estimator — exactly the object handled in the sampling weights article:

$NPS = \frac{\sum _{i} w _{i} s _{i}}{\sum _{i} w _{i}}$

So the influence value is the one derived there, with $s_{i}$ in place of $y_{i}$ :

$u_{i} = \frac{s _{i} - NPS}{N ^}, \hat{V} (NPS) = \frac{n}{n - 1} \cdot \frac{\sum _{i} w _{i}^{2} ( s _{i} - NPS ) ^{2}}{( \sum _{i} w _{i} ) ^{2}}$

This is the general answer, and it handles stratification, clustering and calibration for free. Recoding to $s_{i}$ and treating NPS as a mean is the single most useful thing you can do here — every piece of standard survey machinery then applies without modification.

The same trick covers the wider family of derived metrics:

Top-2-box minus bottom-2-box — identical structure, $s_{i} \in {- 1, 0, 1}$ .
Net agreement / net sentiment — same.
Top-2-box alone — an ordinary proportion; no covariance issue, the standard formulas are fine.
Mean satisfaction score — an ordinary mean.

Any metric of the form "percentage in group A minus percentage in group B, where A and B are mutually exclusive categories of one variable" has this problem, and recoding to a per-respondent score solves it.

Part 4 — Implementation

import numpy as np
from scipy import stats

def nps_score(ratings):
    """Map 0-10 recommendation ratings to per-respondent NPS scores."""
    r = np.asarray(ratings)
    return np.where(r >= 9, 1.0, np.where(r <= 6, -1.0, 0.0))

def nps_with_ci(ratings, weights=None, conf=0.95):
    """NPS with a correct standard error, weighted or not."""
    s = nps_score(ratings)
    n = len(s)
    z = stats.norm.ppf(0.5 + conf / 2)

    if weights is None:
        nps = s.mean()
        var = (s.var(ddof=1)) / n            # equals [P + D - NPS^2]/n
    else:
        w = np.asarray(weights, float)
        nps = np.sum(w * s) / np.sum(w)
        # Hajek / linearised variance - see the sampling weights article
        var = (n / (n - 1)) * np.sum(w**2 * (s - nps) ** 2) / np.sum(w) ** 2

    se = np.sqrt(var)
    return {"nps_pp": nps * 100, "se_pp": se * 100,
            "ci_pp": ((nps - z * se) * 100, (nps + z * se) * 100)}

def nps_difference(ratings_a, ratings_b, w_a=None, w_b=None):
    """Test two independent groups. Variances add; each must be correct."""
    a, b = nps_with_ci(ratings_a, w_a), nps_with_ci(ratings_b, w_b)
    diff = a["nps_pp"] - b["nps_pp"]
    se = np.hypot(a["se_pp"], b["se_pp"])
    z = diff / se
    return {"diff_pp": diff, "se_pp": se, "p": 2 * stats.norm.sf(abs(z))}

R, with a complex design — note that once you recode, nothing is NPS-specific:

library(survey)

dat$nps_score <- ifelse(dat$rec >= 9, 1, ifelse(dat$rec <= 6, -1, 0))

des <- svydesign(ids = ~psu, strata = ~stratum, weights = ~w, data = dat)
svymean(~nps_score, des)                       # NPS/100, with correct SE
confint(svymean(~nps_score, des))

svyby(~nps_score, ~segment, des, svymean)      # by segment
svycontrast(svyby(~nps_score, ~segment, des, svymean),
            quote(A - B))                      # segment comparison

For a wave-on-wave NPS movement in a tracker with a shared panel, the covariance between waves matters too — see the tracker article. The two corrections are independent and both apply.

Part 5 — Checklist

Recode to $- 1/0/ + 1$ and treat NPS as a mean. This is the whole fix, and it makes every other survey tool work correctly.
Never compute the NPS standard error as $Var (P) + Var (D)$ . It omits $2 P D / n$ .
Never treat NPS as a proportion. Rescaling to $[0, 1]$ and using $p (1 - p) / n$ is wrong in the other direction.
Expect the error to be worst on polarised bases — high promoters and high detractors is where the missing term peaks.
Check group comparisons especially. A naive SE roughly doubles the false-positive rate on segment and quarter-on-quarter comparisons.
Weighted data: use the linearised Hájek variance on the recoded score. Do not apply the unweighted closed form to weighted percentages.
Report the interval. On a base of 500 the correct 95% interval is roughly ±7 NPS points. Most NPS movements presented as meaningful are inside that.

Reproducing the simulation

import numpy as np
from scipy import stats

rng = np.random.default_rng(31415)
n, reps = 500, 40000
d, passive, p = 0.20, 0.30, 0.50          # detractors / passives / promoters

draws = rng.multinomial(n, [d, passive, p], size=reps)
d_hat, p_hat = draws[:, 0] / n, draws[:, 2] / n
nps = p_hat - d_hat

se_exact = np.sqrt((p_hat + d_hat - nps**2) / n)
se_naive = np.sqrt(p_hat*(1-p_hat)/n + d_hat*(1-d_hat)/n)

print(nps.std(ddof=1) * 100)      # realised SD:  3.505
print(se_exact.mean() * 100)      # exact SE:     3.489
print(se_naive.mean() * 100)      # naive SE:     2.861

Sources and further reading

Reichheld, F. (2003) "The one number you need to grow", Harvard Business Review — the origin of NPS. Note it contains no standard error at all, which is arguably where the trouble starts.
Grisaffe, D. (2007) "Questions about the ultimate question", Journal of Consumer Satisfaction, Dissatisfaction and Complaining Behavior 20 — a measurement-quality critique of NPS, including its statistical properties.
Agresti, A. (2013) Categorical Data Analysis, 3rd edn, Section 1.4 — multinomial covariance structure, from which $Cov (P, D) = - P D / n$ falls out directly.