Split Test Sample Size Calculator
The formula
What the split-test sample size tells you
The pre-flight calculation for an A/B test: visitors per arm needed to detect a stated relative lift at chosen confidence and power.
Calculation: n per arm = (z_conf + z_power)² × (p₁(1−p₁) + p₂(1−p₂)) ÷ (p₂ − p₁)². Enter your inputs and every substitution is shown below the result.
Reading the result
The number is dominated by baseline rate and lift size — halving the target lift quadruples the traffic; set the minimum economically interesting effect, not the smallest.
Rules of thumb: A 3% CVR baseline with a 20% lift needs roughly 13–14k visitors per arm
Limitations and best practice
Sizing on conversions you expect rather than visitors available produces the perpetual-inconclusive test; invert the formula and ask what lift your traffic can honestly prove.
This calculator gives an estimate based on standard formulas; it is not professional financial, medical, legal or engineering advice. Verify anything you plan to act on with a qualified professional and your own records.
Frequently asked questions
Why relative lift not absolute?
Does segmenting the test change the math?
Source & methodology
Two-proportion power formula (standard).
Split Test Sample Size: what this tool answers
You are here because you need a specific number, and you want to see how it was produced before you rely on it. There are plenty of pages about Split Test Sample Size; almost none of them show the working. This one does: the formula, every input, a fully computed example, and a sensitivity analysis calculated live from this engine’s own defaults.
Business runs on numbers that are cheap to compute and expensive to ignore. Split Test Sample Size in Business & Entrepreneurship is one of those: n per arm = (z_conf + z_power)² × (p₁(1−p₁) + p₂(1−p₂)) ÷ (p₂ − p₁)², and the gap between founders who check and founders who hope shows up exactly here.
In Business & Entrepreneurship the same quantity is often quoted under two or three different conventions. The formula printed above fixes one convention and applies it consistently, which is what makes results from this page comparable with each other.
What makes a calculation page trustworthy is that its copy is derived from its own arithmetic: here visitors per arm = 8,107.7, recomputed every time a field changes, and the sensitivity section ranks the inputs that move the result — Confidence level first, at ≈55.7 % of the headline for a ten-percent error either way in that input — which is the thread the rest of this guide pulls on.
Inside the Split Test Sample Size calculation: formula and every input
One formula, published in full: n per arm = (z_conf + z_power)² × (p₁(1−p₁) + p₂(1−p₂)) ÷ (p₂ − p₁)². Anything you can read, you can reproduce in a spreadsheet, and that reproducibility is the whole point of printing it rather than hiding it behind a button.
Baseline conversion rate (%) (example value 4) is a measured number in the unit named by the label; the formula does not convert for you. Rates are the most mis-transcribed input on any page like this; copy it from the document rather than retyping from memory.
Lift to detect (% relative) (example value 20) is a quantity rather than a category — precision here is cheap and worth having. Enter it exactly as your document states it — quoted annual, monthly and effective rates are three different numbers.
Statistical power (%) (example value 78) is a quantity rather than a category — precision here is cheap and worth having. Check the convention first: if the source says “6.5 % per year”, type 6.5 and let the formula do the rest.
Confidence level (%) (example value 97) is a measured number in the unit named by the label; the formula does not convert for you. Enter it exactly as your document states it — quoted annual, monthly and effective rates are three different numbers.
A fully worked Split Test Sample Size example
Abstract formulas are hard to trust, so here is the complete substitution using the default inputs, written the way an examiner would want it. Starting figures: Baseline conversion rate (%), Lift to detect (% relative), Statistical power (%), Confidence level (%). The computed visitors per arm comes out at 8,107.7, which the interpretation band labels “Total 16,215 visitors — a normal sprint-sized test”.
- p₁ = 0.0400, p₂ = 0.0480 (+20% relative)
- n = (1.96 + 0.52)² × joint-variance ÷ delta² = 8,108 per arm
Total across both arms — 16,215.4. Read next to the headline, this is the figure that shows the shape of the trade-off.
Absolute lift detected — 0.80 %. Read next to the headline, this is the figure that shows the shape of the trade-off.
Sample if 90% power used instead — 13,810.9. If your inputs change, this is usually the first companion figure to move.
Keep this worked example open while you enter your own figures: matching the shape of the substitution is the fastest way to spot a unit or convention mistake.
The Split Test Sample Size corridor: your result across a realistic band
The honest version of any estimate is a corridor, and this one is cheap to produce: Split Test Sample Size was recomputed at 5 points along the dominant input, with every other field left untouched.
| Confidence level setting | Visitors per arm |
|---|---|
| 80 % of the example value | 6,181.8 |
| 90 % of the example value | 6,181.8 |
| example inputs | 8,107.7 |
| 110 % of the example value | 12,627.5 |
| 120 % of the example value | 12,627.5 |
The corridor runs from 6,181.8 to 12,627.5 across that band — which is the range worth planning inside rather than the single 8,107.7 point estimate. Reproduce it in the form above by editing Confidence level alone; the substitution steps will follow your figures exactly.
Where Split Test Sample Size is fragile — and where it is not
Nobody should have to guess which input matters. The engine was re-run once per field below — ten percent up, ten percent down, everything else frozen — and the results ranked by how far they moved the visitors per arm. Read it as a map of where Split Test Sample Size is fragile — drawn for the example inputs, which is the map you want when deciding what to measure carefully.
- Confidence level: ten percent up gives 12,627.5, ten percent down gives 6,181.8 — ≈55.7 % of the visitors per arm either way
- Nudge Statistical power by ±10 % and Split Test Sample Size lands between 8,107.7 and 10,316.5 (sensitivity ≈27.2 %)
- Lift to detect: ten percent up gives 6,758.2, ten percent down gives 9,923.4 — ≈22.4 % of the visitors per arm either way
- Nudge Baseline conversion rate by ±10 % and Split Test Sample Size lands between 7,336.4 and 9,050.4 (sensitivity ≈11.6 %)
Confidence level is the dominant lever here: ten percent either way moves the result by about 55.7 %, from 6,181.8 to 12,627.5. That is where a careless figure does the most damage, so check it against a source, and treat Statistical power as the second priority (≈27.2 % per ten percent). The rest of the inputs earn only a rounding-error of attention, and that asymmetry is the practical payoff of measuring rather than asserting.
Reading the result: bands, zones and thresholds
Nobody acts on a bare figure — they act on a figure with a comparison attached. The default case lands in the “Total 16,215 visitors — a normal sprint-sized test” band, and bands are deliberately wide because real situations overlap. Before treating the band as fact, note its width: ten percent either way on Confidence level already spans 6,181.8 to 12,627.5. The practical reading rules: read the band as a starting hypothesis, then test it: change the two dominant inputs by ten percent and see whether the verdict survives. Verdicts that survive stress are the ones worth acting on.
Where Split Test Sample Size sits next to similar measures: this tool answers “how much is the Split Test Sample Size”, while the related calculators answer “what do I do about it”. Reading the result alone tends to over-weight a single snapshot; pairing it with the companion tools above turns the number into a decision.
Common mistakes with Split Test Sample Size (and the fix)
Sizing on conversions you expect rather than visitors available produces the perpetual-inconclusive test; invert the formula and ask what lift your traffic can honestly prove. This calculator gives an estimate based on standard formulas; it is not professional financial, medical, legal or engineering advice. Verify anything you plan to act on with a qualified professional and your own records.
Chasing precision in Lift to detect (% relative) while Baseline conversion rate (%) is still a guess. The sensitivity ranking above shows exactly where the accuracy budget should go.
One mistake is specific to this engine: assuming every input matters equally. The computation above shows Confidence level dominating — a careful figure there beats false precision everywhere else.
A final trap is presentation: reporting Split Test Sample Size to more decimal places than the inputs support. The tool prints what the arithmetic gives; you should quote only what your least precise field justifies.
Six things to confirm before acting on Split Test Sample Size
- Re-run the calculation with a ±10 % stress on the two most sensitive inputs.
- Compare the result with one independent method or published benchmark.
- Re-verify Confidence level against a source document — ten percent of error there moves the headline by ≈55.7 %.
- Record the date and the inputs alongside the number so future-you knows what was assumed.
- State, in one sentence, which convention each rate, unit or period in the form is using.
- Confirm the answer against one independent method or published benchmark for Split Test Sample Size.
Step by step: getting a trustworthy Split Test Sample Size out of this tool
Using a calculator well sounds trivial, but the difference between a decision-grade figure and a throwaway one lives entirely in the routine. The routine for Split Test Sample Size takes a minute to learn and applies to every input above.
Step 1 — collect, then type. Everything this tool asks for is listed above (baseline conversion rate (%), lift to detect (% relative), statistical power (%), confidence level (%)). Pull each figure from something you can point at — paperwork, a dashboard, a label — before touching a field. Guesswork at the input stage cannot be repaired at the interpretation stage. The fastest sanity test is to guess the answer first and see how far the calculation disagrees.
Step 2 — start from the case you can defend. Enter pessimistic-but-honest figures first; the optimistic run is a ceiling, not a plan. A conclusion that holds on conservative inputs holds everywhere. Rounding early is the quietest way to introduce a permanent error.
Step 3 — measure what deserves measuring. The ranked levers above tell you where a careful figure pays off. Everything else can stay approximate, which is a licence to stop hunting for data you do not need. If the answer changes your behaviour, it deserves a second run with different assumptions.
Step 4 — stress before you commit. A decision that survives a ten-percent squeeze on its two biggest inputs needs no luck to be sound. One that does not survive is a decision about hope, not about arithmetic. A result worth acting on survives having its two shakiest inputs halved.
Step 5 — schedule the next run. In Business & Entrepreneurship the inputs drift quietly. Put a reminder a quarter out, re-enter the same fields, and read the change rather than the level. When two sources disagree, it is almost never the arithmetic that differs.
Who actually uses Split Test Sample Size — and when
Business & Entrepreneurship arguments end at arithmetic or they do not end. Split Test Sample Size is the arithmetic: 8,107.7 on the example inputs, reproducible by anyone who types the same numbers. Comparisons are cheap: run the identical calculation for the alternative and look at the gap.
Used properly this is a five-minute filter that saves weeks. Compute Split Test Sample Size, reject what fails, and only then bring in the expensive analysis for the survivors. Estimates improve most when the worst case is written down in ink.
Nothing here is stored or transmitted, so the habit of checking is free: open the page, re-enter current figures, compare with the number you saved last time. A result worth acting on survives having its two shakiest inputs halved.
The questions that come up most about Split Test Sample Size
What affects Split Test Sample Size the most?
The short answer: Confidence level. The longer answer is the sensitivity table on this page, where every numeric input was probed by ten percent and the visitors per arm recalculated — Confidence level moved it furthest, by roughly 55.7 % (6,181.8 to 12,627.5).
Which matters more: statistical power or confidence level?
By the numbers, Confidence level: a ten-percent change in it moves the result ≈55.7 %, versus ≈27.2 % for Statistical power. Spend your measuring effort in that order — precision on the weaker lever buys almost nothing.
What is a typical Split Test Sample Size?
With the example inputs on this page the result is 8,107.7, labelled “Total 16,215 visitors — a normal sprint-sized test”. “Typical” varies with situation, which is why every result here ships with its verdict band and its sensitivity list: they turn one figure into a range you can locate yourself inside.
How do I calculate Split Test Sample Size by hand?
Three steps: write the formula (n per arm = (z_conf + z_power)² × (p₁(1−p₁) + p₂(1−p₂)) ÷ (p₂ − p₁)²), insert your values, and simplify. The worked example section shows exactly that chain for the default inputs; replicating it once on paper teaches you more about Split Test Sample Size than any number of calculator runs.
What happens to Split Test Sample Size if confidence level changes by 10 %?
The engine was re-run with that field moved by ten percent while everything else stayed fixed: the visitors per arm goes across 6,181.8 to 12,627.5, a swing of roughly 55.7 % around the 8,107.7 baseline. That ratio is the practical definition of how much this input matters.
Is the Split Test Sample Size result of 8,107.7 good or bad?
Neither, on its own — the label “Total 16,215 visitors — a normal sprint-sized test” is the band this figure falls in for the example inputs, and your own inputs will place you elsewhere on the same ladder. Judge the number against your history and your alternatives, and remember that a ten-percent error in Confidence level alone moves it by ≈55.7 %.
Can Split Test Sample Size be negative or empty?
Yes, and that is informative rather than broken: a negative Split Test Sample Size usually means the inputs encode an outflow or a shortfall, and the verdict band above labels exactly that situation instead of hiding it.
What affects Split Test Sample Size the most?
Sensitivity depends on where you start, but the inputs Baseline conversion rate (%) and Lift to detect (% relative) typically move Split Test Sample Size further than the rest because they enter the formula multiplicatively. If you can only measure two things well, measure those and hold the others at conservative defaults.
Why relative lift not absolute?
Because that is how effect sizes scale with baselines — a 1-point lift at 3% and at 12% have very different evidentiary weight; state both if finance asks in points.
Does segmenting the test change the math?
Yes — analysing by device or cohort means sizing per segment; a test powered overall can be laughably under-powered for the segment you conclude on.
Do I need an account to use this Split Test Sample Size calculator?
No, and no data leaves the tab. The engine runs in the page itself, which is why the worked example, the bands and the sensitivity ranking all respond instantly to your keystrokes.
What the labels on this page mean
- Total across both arms
- 16,215.4 on the example inputs — a companion figure printed beside the headline of Split Test Sample Size so that no result is read in isolation.
- Absolute lift detected
- 0.80 % on the example inputs — a companion figure printed beside the headline of Split Test Sample Size so that no result is read in isolation.
- Sample if 90% power used instead
- 13,810.9 in the default Split Test Sample Size calculation — one of the secondary figures an experienced reader checks before trusting the primary number.
- Visitors per arm
- The primary output of this tool: 8,107.7 for the default inputs, computed exactly from n per arm = (z_conf + z_power)² × (p₁(1−p₁) + p₂(1−p₂)) ÷ (p₂ − p₁)².
- Worked example
- The step-by-step substitution printed above, so the arithmetic can be re-derived by hand or in a spreadsheet.
- Verdict band
- The plain-language label — here “Total 16,215 visitors — a normal sprint-sized test” — mapping the computed figure onto experience-based ranges for Business & Entrepreneurship.
- Sensitivity (±10 %)
- How far the Split Test Sample Size moves when a single input is nudged ten percent either way; the ranked list on this page is computed live from this engine.
The provenance of this Split Test Sample Size calculation
There is no black box on this page — the engine evaluates n per arm = (z_conf + z_power)² × (p₁(1−p₁) + p₂(1−p₂)) ÷ (p₂ − p₁)². Source of record: Two-proportion power formula (standard). Specialist software layers extra coefficients on top of the same algebra; none are hidden here, and that transparency is what makes a disagreement traceable to a specific input instead of a black box.
Treat the output as a precise answer to the question you typed: exact arithmetic, transparent formula, published source, no hidden rounding. The sensitivity table above is part of that promise rather than a bonus: it lets you see how much of the result depends on how carefully you measured.
Turning this Split Test Sample Size into a decision
Save the number together with its inputs and today’s date; a Split Test Sample Size figure without context is unusable the moment an input changes.
The audience here is people who need one trustworthy number before their next decision. Everything you need to defend the result — formula, substitution, sensitivity, source — is printed above, so the only remaining variable is the honesty of your inputs.