The paper investigates the impact of annotation ambiguity on supervised machine learning classification processes applied to cultural heritage data. Focusing on semantic segmentation of 3D point clouds, the study highlights how manual annotations—used as training data—are inherently subjective and influenced by the annotator’s background, expertise, and interpretation of architectural elements. Through an experimental framework involving multiple annotators, the research quantifies variability in labeling using metrics such as Intersection over Union (IoU), revealing significant discrepancies in the definition of semantic classes (e.g., columns, arches, moldings). The work proposes integrating uncertainty-aware approaches, such as semantic maps, to encode annotation ambiguity and improve the robustness, interpretability, and interoperability of AI-based classification systems for heritage documentation.
Hybrid AI-Based Annotations of the Urban Walls of Pisa for Stratigraphic Analyses
This paper proposes a hybrid methodology for the semantic annotation and stratigraphic analysis of architectural heritage, combining supervised machine learning techniques with photogrammetric 3D modeling. The research aims to support the interpretation of complex masonry structures by integrating 2D image-based classification with 3D spatial representations.
The workflow involves the acquisition of photogrammetric datasets, followed by supervised classification of images using a Random Forest algorithm trained on manually annotated samples. As illustrated in the workflow diagram (Fig. 2, p. 783), semantic information related to lithotypes and construction layers is first propagated across the entire image set and subsequently transferred to the 3D point cloud through 2D–3D reprojection. This process enables the creation of semantically enriched 3D models, preserving the relationship between geometric data and interpretative annotations.
Results, shown in the classification outputs (Figs. 4–6, pp. 786–788), demonstrate that the method effectively identifies different construction phases and material layers, particularly in cases with clear colorimetric variation. However, limitations arise in areas with homogeneous textures, where classification accuracy decreases. The study highlights the potential of hybrid AI-based annotation systems to enhance stratigraphic analysis and heritage documentation, while emphasizing the need to integrate additional geometric features to improve robustness.
Semantic Mapping of Architectural Heritage via Artificial Intelligence and H-BIM
Starting from the virtual photogrammetric 3D reconstruction, this work proposes a classification method, based on Artificial Intelligence, allowing to semi-automatically characterize the digital models of existing architectural heritage in terms of material mapping and/or decay condition. The obtained data, once classified, is used and transferred in BIM environment, so to favor the construction of in-formative models rich in analytical content. The proposed approach is described with reference to the significant case study of the Chiesa del Carmine in Pisa, for the study and restitution of the liturgical and decorative apparatus, as part of a large-scale research project, still underway, on the reconstruc-tion of the tramezzo screens for the churches of the Mendicant orders.
NEURAL RADIANCE FIELDS (NERF) FOR MULTI-SCALE 3D MODELING OF CULTURAL HERITAGE ARTIFACTS
This research aims to assess the adaptability of Neural Radiance Fields (NeRF) for the digital documentation of cultural heritage objects of varying size and complexity. We discuss the influence of object size, desired scale of representation, and level of detail on the choice to use NeRF for cultural heritage documentation, providing insights for practitioners in the field. Case studies range from historic pavements to architectural elements or buildings, representing diverse and multi-scale scenarios encountered in heritage documentation procedures. The findings suggest that NeRFs perform well in scenarios with homogeneous textures, variable lighting conditions, reflective surfaces, and fine details. However, they exhibit higher noise and lower texture quality compared to other consolidated image-based techniques as photogrammetry, especially in case of small-scale artifacts.
Connecting geometry and semantics via artificial intelligence: from 3D classification of heritage data to H-BIM representations.
Cultural heritage information systems, such as H-BIM, are becoming more and more widespread today, thanks to their potential to bring together, around a 3D representation, the wealth of knowledge related to a given object of study. However, the reconstruction of such tools starting from 3D architectural surveying is still largely deemed as a lengthy and time-consuming process, with inherent complexities related to managing and interpreting unstructured and unorganized data derived, e.g., from laser scanning or photogrammetry. Tackling this issue and starting from reality-based surveying, the purpose of this paper is to semi-automatically reconstruct parametric representations for H-BIM-related uses, by means of the most recent 3D data classification techniques that exploit Artificial Intelligence (AI). The presented methodology consists of a first semantic segmentation phase, aiming at the automatic recognition through AI of architectural elements of historic buildings within points clouds; a Random Forest classifier is used for the classification task, evaluating each time the performance of the predictive model. At a second stage, visual programming techniques are applied to the reconstruction of a conceptual mock-up of each detected element and to the subsequent propagation of the 3D information to other objects with similar characteristics. The resulting parametric model can be used for heritage preservation and dissemination purposes, as common practices implemented in modern H-BIM documentation systems. The methodology is tailored to representative case studies related to the typology of the medieval cloister and scattered over the Tuscan territory.
Semantic segmentation through Artificial Intelligence from raw point clouds to H-BIM representation
This work describes a semi automatic workflow for the 3D reconstruction of Heritage Building Information Models from raw point clouds, based on Artificial Intelligence techniques. The BIM technology applied to historical architecture has made it possible to create a virtual repository of many heterogeneous pieces of information in order to make the process of storing and collecting data on the built heritage more effective. The modelling phase of an artefact is the most complex and problematic in terms of time, as the large architectural heritage of historical buildings does not allow the use of parametric models, so that manual modelling of components is required. Current scientific research focuses on automating this phase by means of segmentation and classification methods: these are based on associating different semantic information to the products of the three dimensional surveying as point clouds or polygonal meshes. To address these problems, the proposed approach relies on: (i) the application of machine learning algorithms with a multi level and multi resolution (MLMR) method to semantically classify 3D heritage data; (ii) the use of annotated data identified by relevant features to boost the scan to BIM process for 3D digital reconstruction. The procedure is tested and evaluated on the complex case of the Church of Santa Caterina d’Alessandria in Pisa, Italy. The classification results show the reliability and reproducibility of the developed method.
Machine Learning and Deep Learning for the Built Heritage Analysis: Laser Scanning and UAV-Based Surveying Applications on a Complex Spatial Grid Structure
The reconstruction of 3D geometries starting from reality-based data is challenging and timeconsuming due to the difficulties involved in modeling existing structures and the complex nature of built heritage. This paper presents a methodological approach for the automated segmentation and classification of surveying outputs to improve the interpretation and building information modeling from laser scanning and photogrammetric data. The research focused on the surveying of reticular, space grid structures of the late 19th–20th–21st centuries, as part of our architectural heritage, which might require monitoring maintenance activities, and relied on artificial intelligence (machine learning and deep learning) for: (i) the classification of 3D architectural components at multiple levels of detail and (ii) automated masking in standard photogrammetric processing. Focusing on the case study of the grid structure in steel named La Vela in Bologna, the work raises many critical issues in space grid structures in terms of data accuracy, geometric and spatial complexity, semantic classification, and component recognition.