BIZOMAT MAKEOS · Quality
Two-Sample Comparison (t-test)
Is the difference between these two averages statistically significant?
Welch’s two-sample t-test comparing the means of two groups (machines, suppliers, before and after) from their means, standard deviations and sample sizes.
Open the calculatorRuns in your browser. No account needed.
How this tool works
Inputs, units and where to find them
| Input | Unit | Typical source |
|---|---|---|
| Group 1 | ||
| Mean | Average of group 1. | |
| Standard deviation | Sample standard deviation of group 1. | |
| Sample size | items | Number of independent results in group 1. |
| Group 2 | ||
| Mean | Average of group 2. | |
| Standard deviation | Sample standard deviation of group 2. | |
| Sample size | items | Number of independent results in group 2. |
| Test | ||
| Significance level (two-sided) | choice | Probability of wrongly declaring a difference that is not real (0.05 is common). |
What each input means
Open an input to see what it is, its units, the worked example and the common mistakes, with a small diagram.
Group 1
What this means: Mean
- What it is
- The average of the results in group 1, such as one machine, supplier or the before condition.
- Units and format
- Enter a plain number with no unit. Negative values are accepted. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 52.3 as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- A larger gap between the two means makes a significant difference more likely.
- Get it right
- Use the same units and measurement method for both groups.
What this means: Standard deviation
- What it is
- The sample standard deviation of the results in group 1, showing how much they vary around their mean.
- Units and format
- Enter a plain number with no unit. The form accepts 0.000001 or more. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 2.1 as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- More spread makes it harder to show a significant difference.
- Get it right
- Use the sample standard deviation, not the standard error of the mean.
What this means: Sample size
- What it is
- How many independent results are in group 1.
- Units and format
- Enter a number in items: a count of items. The form accepts 2 items to 10000000 items. Use a whole number. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 15 items as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- More results make a real difference easier to detect.
- Get it right
- Repeat readings on the same part are not independent results.
Group 2
What this means: Mean
- What it is
- The average of the results in group 2, the group being compared with group 1.
- Units and format
- Enter a plain number with no unit. Negative values are accepted. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 50.1 as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- A larger gap between the two means makes a significant difference more likely.
- Get it right
- Use the same units and measurement method for both groups.
What this means: Standard deviation
- What it is
- The sample standard deviation of the results in group 2, showing how much they vary around their mean.
- Units and format
- Enter a plain number with no unit. The form accepts 0.000001 or more. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 2.4 as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- More spread makes it harder to show a significant difference.
- Get it right
- Use the sample standard deviation, not the standard error of the mean.
What this means: Sample size
- What it is
- How many independent results are in group 2.
- Units and format
- Enter a number in items: a count of items. The form accepts 2 items to 10000000 items. Use a whole number. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 12 items as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- More results make a real difference easier to detect.
- Get it right
- Repeat readings on the same part are not independent results.
Test
What this means: Significance level (two-sided)
- What it is
- The chance you accept of declaring a difference when there really is none. The difference is called significant when the p-value falls below it.
- Units and format
- Choose one of 3 options from the list: 0.10; 0.05; 0.01.
- Example
- The form opens with “0.05” selected as a worked example. Choose the option that matches your case.
- Get it right
- Decide it before looking at the data; a smaller level demands stronger evidence.
Method and formulas
- SE = √(s₁²/n₁ + s₂²/n₂); t = (m₁ − m₂) ÷ SE.
- Welch–Satterthwaite degrees of freedom: ν = SE⁴ ÷ ((s₁²/n₁)²/(n₁ − 1) + (s₂²/n₂)²/(n₂ − 1)).
- p = 2 × P(T_ν > |t|), from the regularised incomplete beta function.
- Significant when p < α.
- Example: the default groups differ by 2.2 with t = 2.5006, ν = 22.09 and p = 0.020303.
Limitations
- Independent random samples from roughly normal populations; strongly skewed data or small samples need care.
- Statistical significance is not practical importance: judge the size of the difference too.
- Example values are illustrations. Replace them with values for your actual task.
Typical uses
- Check whether a process change really improved the average
- Compare two suppliers or machines
- Support a trial report with statistics
Who it is for
- Quality engineers
- Process engineers
- Test engineers
What you get
- Difference of means and its standard error
- t statistic and degrees of freedom
- Two-sided p-value
- Confidence interval of the difference
Interpreting the result
- A significant result with a tiny difference may not matter; a large but non-significant difference may need more data.
References
- Machinery’s Handbook, 29th edition (Industrial Press, 2012), printed pp. 130–133. Topic reference; no source tables or recommended settings are reproduced.
Learn the Concepts Behind This Tool
These BIZOMAT Library readings explain the method this tool uses, with worked examples.
Method explained here · BIZOMAT original book
Electrical conductivity and what it reveals about a metal
In Eddy-Current Inspection by the Numbers
What it covers: The %IACS scale, why alloying and heat treatment change conductivity, correcting readings to 20 °C, calibrating a conductivity meter, and catching a mislabelled aluminium bar.
Method explained here · BIZOMAT original book
From microstructure to decision
In Microstructures by the Numbers
What it covers: The uncertainty of a microstructure result from sample to report, comparing laboratories and batches fairly, avoiding pseudo-replication, decision rules near limits, and writing specifications.
Method explained here · BIZOMAT original book
Monitoring, soiling and degradation
In Rooftop Solar for Factories by the Numbers
What it covers: How to monitor an array's performance ratio, check the irradiance sensor's calibration, measure soiling with a cleaning trial and a t-test, and separate soiling from module degradation.
Method explained here · BIZOMAT original book
Verify, validate and decide
In Simulating Noisy Systems by the Numbers
What it covers: Checking that a stochastic model is coded right and that it matches the plant, testing how much its answers depend on uncertain inputs, comparing design options fairly and reporting a decision.
BIZOMAT tools are screening and decision-support utilities. Verify important financial, engineering, safety, legal or operational decisions using appropriate professional standards and your organisation’s approved data.