BIZOMAT MAKEOS · Quality
Abbe Offset Error
How much error does an offset between the scale and the measured feature cause?
Measurement error caused by an offset between the measuring axis and the scale axis when the slide tilts (Abbe error), compared with the tolerance being checked.
Open the calculatorRuns in your browser. No account needed.
How this tool works
Inputs, units and where to find them
| Input | Unit | Typical source |
|---|---|---|
| Offset between scale and measured line | mm | Perpendicular distance between the instrument scale and the line along which the feature is measured (e.g. caliper jaw depth). |
| Angular error of the slide | arc-seconds | Angular play or straightness error of the slide, from the instrument specification or a test. |
| Tolerance band being checked (total) | mm | Total tolerance band of the feature, upper minus lower limit. |
What each input means
Open an input to see what it is, its units, the worked example and the common mistakes, with a small diagram.
What this means: Offset between scale and measured line
- What it is
- The distance at right angles between the line along which the instrument’s scale reads and the line along which the part is actually measured, for example how far a caliper’s jaws reach out from its beam.
- Units and format
- Enter a number in mm: millimetres. The form accepts 0 mm to 100000 mm. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 30 mm as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- The Abbe error rises in proportion to the offset.
- Get it right
- Measure square to the measuring direction; the error is zero only when the scale and the measured line coincide.
What this means: Angular error of the slide
- What it is
- The small angle by which the slide or moving jaw tips as it travels, from play or straightness error, taken from the instrument specification or a test.
- Units and format
- Enter a number in arc-seconds: arc-seconds. The form accepts 0 arc-seconds to 36000 arc-seconds. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 20 arc-seconds as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Effect on the result
- For these small angles the error rises almost in proportion to the tilt.
- Get it right
- Enter arc-seconds, not degrees or minutes: 1 arc-minute is 60 arc-seconds.
What this means: Tolerance band being checked (total)
- What it is
- The full tolerance band of the feature being checked: upper limit minus lower limit. The tool shows how much of it the Abbe error alone uses up.
- Units and format
- Enter a number in mm: millimetres. The form accepts 0.000001 mm to 10000 mm. Required: the tool does not calculate until every input is valid.
- Example
- The form opens with 0.05 mm as a worked example. It is an illustration, not a recommended value: replace it with your own figure.
- Get it right
- Enter the total band; for ±0.025 that is 0.05, not 0.025.
Method and formulas
- Abbe error = offset × tan(tilt angle).
- Share of tolerance = error ÷ tolerance band × 100%.
- Example: a 30 mm offset with 20″ of tilt gives 2.9089 µm, 5.8178% of a 0.05 mm band.
Limitations
- Only the first-order Abbe error is calculated; other error sources (scale, temperature, force) add to it.
- Example values are illustrations. Replace them with values for your actual task.
Typical uses
- Judge whether calipers or a height gauge are suitable for a tolerance
- Explain differences between instruments
- Plan a measurement set-up
Who it is for
- Quality inspectors
- Toolmakers
- Metrology technicians
What you get
- Abbe error
- Error as a share of the tolerance
Interpreting the result
- Compare the share with the measurement-capability rule your organisation uses (often a fraction of the tolerance).
References
- Machinery’s Handbook, 29th edition (Industrial Press, 2012), printed pp. 709–735. Topic reference; no source tables or recommended settings are reproduced.
BIZOMAT tools are screening and decision-support utilities. Verify important financial, engineering, safety, legal or operational decisions using appropriate professional standards and your organisation’s approved data.