Professional Detailing2 min read

Inspector Agreement in Coating QC: Look beyond the Percentage Agreeing

Evaluate inspector consistency with class-specific results and an appropriate agreement analysis, not only an overall percentage.

Updated · NANOLAB Knowledge

Editorial illustration: A Practical Quality-Control Process for Ceramic Coating
Editorial illustration

Two coating inspectors can agree often while still missing the defects that matter. If almost every sample is acceptable, repeatedly calling everything acceptable may produce a high overall agreement percentage.

Build a meaningful sample set

Include representative acceptable, unacceptable and borderline conditions. Use a trustworthy reference classification where available, and distinguish agreement with that reference from agreement between inspectors.

Blind and repeat the assessments

Randomise sample order and conceal previous decisions. Repeated assessments can reveal within-inspector consistency, while comparisons across people reveal between-inspector differences. Keep lighting and viewing instructions controlled.

Read several outputs

Output Question
Overall agreement How often do decisions match?
Class-specific results Which defects or acceptable cases cause difficulty?
Agreement with reference Are matching decisions also correct?
Appropriate kappa analysis How does agreement compare with its chance model?

Improve the decision system

Kappa depends on the design and class distribution, so avoid treating one generic threshold as universal proof of competence. Review disagreement images and clarify acceptance boundaries. For example, repeated disagreement on faint haze may call for a better reference panel or lighting method. Reassess with fresh or appropriately blinded samples after training rather than using memorised examples as the sole evidence of improvement.

Technical references

The sources below support the stated concepts or describe their own methods. The worked scenarios are editorial examples, not NANOLAB test results.

Ask NANOLAB about your surface and intended use. For application and care, use the current instructions for the exact product.