Enhancing Construction Equipment Detection with Generative AI-Based Synthetic Image Data

SeongHyun Moon; Ahreum Lee; Yong-Ju Lee; Man-Woo Park

doi:10.13161/kibim.2025.15.3.077

All Issue

2025 Vol.15, Issue 3 Preview Page

Research Article

Enhancing Construction Equipment Detection with Generative AI-Based Synthetic Image Data 건설장비 객체인식 모델 성능 향상을 위한 생성형 AI 기반 합성 이미지 데이터 활용

30 September 2025. pp. 77-86

PDF XML

Abstract

Dynamic and complex construction environments require effective monitoring to enhance productivity and ensure safety. While recent advancements in computer vision and deep learning have enabled the application of object detection models for this purpose, their performance is often constrained by the scarcity and diversity of training data. The process of collecting and annotating sufficient real-world construction images is labor-intensive and frequently limited by specific site conditions. This study addresses this challenge by investigating whether synthetic images, generated via a Text-to-Image model, can effectively supplement real datasets. For our experiments, we extracted roller and dozer classes from two publicly available datasets, ACID and MOCS, and generated corresponding synthetic images. We trained YOLOv11n models under three distinct scenarios: using synthetic images only, real images only, and a combination of both. Models trained with real data augmented by synthetic images showed consistent performance gains in both mAP50 and mAP50-95. This performance improvement was most significant with the smaller MOCS dataset, suggesting that synthetic data has a stronger supplementary effect when real-world data is limited.

Keywords

Construction Site

Synthetic Image

Object Detection

Construction Equipment

Generative AI

키워드

건설 현장

합성 이미지

객체 탐지

건설장비

생성형 AI

References

Cheng, R., He, X., Zheng, Z., Wang, Z. (2021). Multi-scale Safety Helmet Detection based on SAS-YOLOv3-tiny, Applied Sciences, 11(8), 3652.

10.3390/app11083652

Deng, J., Lu, Y., Lee, V. C. S. (2021). Imaging-based Crack Detection on Concrete Surfaces using You Only Look Once Network, Structural Health Monitoring, 20(2), pp. 484-499.

10.1177/1475921720938486

Eum, I., Kim, J., Wang, S., Kim, J. (2025). Heavy Equipment Detection on Construction Sites Using You Only Look Once (yolo-version 10) with Transformer Architectures, Applied Sciences, 15(5), 2320.

10.3390/app15052320

Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M., An, W. (2018). Detecting non-hardhat-use by a Deep Learning Method from Far-field Surveillance Videos, Automation in Construction, 85, pp. 1-9.

10.1016/j.autcon.2017.09.018

Golparvar-Fard, M., Peña-Mora, F., Savarese, S. (2009). D4AR-a 4-dimensional Augmented Reality Model for Automating Construction Progress Monitoring Data Collection, Processing and Communication, Journal of Information Technology in Construction, 14(13), pp. 129-153.

Hacıefendioğlu, K., Başağa, H. B. (2022). Concrete Road Crack Detection using Deep Learning-based Faster R- CNN Method, Iranian Journal of Science and Technology, Transactions of Civil Engineering, 46(2), pp. 1621-1633.

10.1007/s40996-021-00671-2

Jocher, G., Qiu, J. (2024). Ultralytics YOLO11 (11.0.0), https://github.com/ultralytics/ultralytics (Aug. 12. 2025).

Kumar, S., Gupta, H., Yadav, D., Ansari, I. A., Verma, O. P. (2022). YOLOv4 Algorithm for the Real-time Detection of Fire and Personal Protective Equipments at Construction Sites, Multimedia Tools and Applications, 81(16), pp. 22163-22183.

10.1007/s11042-021-11280-6

Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C. L. (2014). Microsoft Coco: Common Objects in Context. In European Conference on Computer Vision, pp. 740-755.

10.1007/978-3-319-10602-1_48

Luo, H., Wang, M., Wong, P. K. Y., Cheng, J. C. (2020). Full Body Pose Estimation of Construction Equipment using Computer Vision and Deep Learning Techniques. Automation in Construction, 110, 103016.

10.1016/j.autcon.2019.103016

OpenAI (2023). ChatGPT, https://chat.openai.com (April. 30. 2025)

Padilla, R., Netto, S. L., Da Silva, E. A. (2020). A Survey on Performance Metrics for Object-detection Algorithms, Proceedings of the International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 237-242.

10.1109/IWSSIP48289.2020.9145130

Park, S. E., Eem, S. H., Jeon, H. (2020). Concrete Crack Detection and Quantification using Deep Learning and Structured Light, Construction and Building Materials, 252, 119096.

10.1016/j.conbuildmat.2020.119096

Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I. (2021). Zero-shot text-to- image Generation, In International Conference on Machine Learning. Pmlr, pp. 8821-8831.

Ribeiro, W. S., Zanetti, J., Totola, L. B., Junqueira, S. Â. C., Lauff, P. H. P. (2024). Crack Detection in Buildings using the YOLO v8 Network, Revista Alconpat, 14(3), pp. 288-298.

10.21041/ra.v14i3.765

Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, K., Lopes, R. G., Ayan, B. K., Salimans, T., Ho, J., Fleet, D. J., Norouzi, M. (2022). Photorealistic Text-to-image Diffusion Models with Deep Language Understanding, Advances in Neural Information Processing Systems, 35, pp. 36479-36494.

Seong, J., Kim, H. S., Jung, H. J. (2023). The Detection System for a Danger State of a Collision between Construction Equipment and Workers using Fixed CCTV on Construction Sites. Sensors, 23(20), 8371.

10.3390/s2320837137896464PMC10610944

Shorten, C., Khoshgoftaar, T. M. (2019). A Survey on Image Data Augmentation for Deep Learning, Journal of Big Data, 6(1), pp. 1-48.

10.1186/s40537-019-0197-0

Son, H., Kim, C. (2021). Integrated Worker Detection and Tracking for the Safe Operation of Construction Machinery, Automation in Construction, 126, 103670.

10.1016/j.autcon.2021.103670

Wang, H., Hu, Z., Guo, Y., Yang, Z., Zhou, F., Xu, P. (2020). A Real-time Safety Helmet Wearing Detection Approach based on CSYOLOv3. Applied Sciences, 10(19), 6732.

10.3390/app10196732

Wu, P., Liu, A., Fu, J., Ye, X., Zhao, Y. (2022). Autonomous Surface Crack Identification of Concrete Structures based on an Improved One-stage Object Detection Algorithm. Engineering Structures, 272, 114962.

10.1016/j.engstruct.2022.114962

Xiao, B., Kang, S. C. (2021). Development of an Image Data Set of Construction Machines for Deep Learning Object Detection, Journal of Computing in Civil Engineering, 35(2), 05020005.

10.1061/(ASCE)CP.1943-5487.0000945

Xiao, B., Wang, Y., Kang, S. C. (2022). Deep Learning Image Captioning in Construction Management: A Feasibility Study, Journal of Construction Engineering and Management, 148(7), 04022049.

10.1061/(ASCE)CO.1943-7862.0002297

Xuehui, A., Li, Z., Zuguang, L., Chengzhi, W., Pengfei, L., Zhiwei, L. (2021). Dataset and Benchmark for Detecting Moving Objects in Construction Sites, Automation in Construction, 122, 103482.

10.1016/j.autcon.2020.103482

Information

Publisher :Journal of KIBIM
Publisher(Ko) :한국BIM학회
Journal Title :Journal of KIBIM
Journal Title(Ko) :한국BIM학회논문집
Volume : 15
No :3
Pages :77-86
Received Date : 2025-08-30; 2025-09-05
Accepted Date : 2025-09-12
DOI :https://doi.org/10.13161/kibim.2025.15.3.077

Journal of KIBIM ISSN:2288-1697(Print) 2288-1697(Online) 한국BIM학회논문집

All Issue