Evaluation of Annotation Ambiguity in Common Supervised Machine Learning Classification Approaches for Cultural Heritage

The paper investigates the impact of annotation ambiguity on supervised machine learning classification processes applied to cultural heritage data. Focusing on semantic segmentation of 3D point clouds, the study highlights how manual annotations—used as training data—are inherently subjective and influenced by the annotator’s background, expertise, and interpretation of architectural elements. Through an experimental framework involving multiple annotators, the research quantifies variability in labeling using metrics such as Intersection over Union (IoU), revealing significant discrepancies in the definition of semantic classes (e.g., columns, arches, moldings). The work proposes integrating uncertainty-aware approaches, such as semantic maps, to encode annotation ambiguity and improve the robustness, interpretability, and interoperability of AI-based classification systems for heritage documentation.