Survey of Applications of Material Defect Detection Based on Machine Vision and Deep Learning
YANG Chuanli1, ZHANG Xiuqing2,*
1 School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, China 2 Key Laboratory of Pressure System Safety Science of Ministry of Education, East China University of Science and Technology, Shanghai 200237, China
Abstract: The safety problem caused by material defects has always been a hot issue, and how to realize the rapid accurate identification and location of material defects is the focus of current research on material defects. The traditional non-destructive testing method mainly uses ultrasonic, X-ray and other advanced technologies to realize the identification and positioning of material defects. Although this method solves the problem of low manual testing efficiency, it is still difficult to achieve the requirements of intelligentization, automation and high precision. Advances in the computer technology has stimulated the rapid development of machine vision in the detection of material defects. The advantages of machine vision inspection technology are mainly the combination of non-destructive inspection, automation and intelligentization, which has not only good safety and high efficiency, but also high detection accuracy. However, an image processing algorithm usually can be effective to the recognition of only one specific kind of defect, which causes low versatility of the equipment and elevated cost of production and maintenance. In recent years, the rise of deep learning has promoted the rapid development of the field of artificial intelligence, and solved the problem that traditional machine vision needs different image processing algorithms to classify different tasks. Deep learning has also become a popular research direction in material defect detection. Many scholars have applied deep learning technology to the field of material defect detection, and they have achieved good results whether they are related to the classification of material defects, or the location and the segmentation of material defects. The article summarizes the application of traditional methods and machine vision methods in material defects, introduces the principles of deep learning in material defect detection, and systematically describes the application of deep learning in the classification, positioning and segmentation of material defects at home and abroad, and the future development trend of the application of deep learning in the field of material defect detection is prospected.
杨传礼, 张修庆. 基于机器视觉和深度学习的材料缺陷检测应用综述[J]. 材料导报, 2022, 36(16): 20070136-9.
YANG Chuanli, ZHANG Xiuqing. Survey of Applications of Material Defect Detection Based on Machine Vision and Deep Learning. Materials Reports, 2022, 36(16): 20070136-9.
1 Eisenmann D J, Enyart D, Lo C, et al. AIP Conference Proceedings, 2014, 1581, 1505. 2 Chen Z, Wang J, Sun P. Internal Combustion Engine & Parts, 2020(4), 44(in Chinese). 陈朝, 王靖, 孙鹏. 内燃机与配件, 2020(4), 44. 3 Chiem T P, Wei Q Y. International Journal of Digital Crime and Forensics, 2014, 6(4), 50. 4 Chang W J, Pan J, Meng F H. Machine China, 2014(10), 108(in Chinese). 常文见, 潘杰, 孟凡辉. 中国机械, 2014(10), 108. 5 John H G. Quality, 2016, 55(11), 41. 6 Xie S J, Duan Z R, Li, J, et al. Sensors and Actuators A: Physical, 2020, 309, 112030. 7 Chesnokova A A, Kalayeva S Z, Ivanova V A. Journal of Physics: Conference Series, 2017, 881(1), 012022. 8 Cheng J, He C F, Zheng Y, et al. NDT&E International, 2020, 114, 102283. 9 Anton D P, Ina Y B, Igor Y B. Materials and Design, 2020, 187, 108385. 10 Du W Z, Shen H Y, Fu J Z, et al. Journal of Intelligent Manufacturing, 2021, 32, 141. 11 Tang B, Kong J Y, Wu S Q. Journal of Image and Graphics, 2017, 22(12), 1640(in Chinese). 汤勃, 孔建益, 伍世虔. 中国图象图形学报, 2017, 22(12), 1640. 12 Zhai D, Ding Y N, Li T. Electronic Science and Technology, 2014, 27(1), 160(in Chinese). 翟栋,丁亚男,李涛.电子科技, 2014,27(1), 160. 13 Wang H, Zhang Y, Shen H H, et al. Chinese Journal of Optics, 2017, 10(4), 438(in Chinese). 王浩, 张叶, 沈宏海, 等. 中国光学,2017,10(4), 438. 14 Chen Y H. Journal of Baoji University of Arts and Sciences(Natural Science Edition), 2013, 33(1), 16(in Chinese). 陈一虎. 宝鸡文理学院学报(自然科学版), 2013, 33(1), 16. 15 Herve A, Williams L J. Wiley Interdiplinary Reviews Computational Stats, 2010, 2(4), 433. 16 Chen Q L, Wang D X, Lin J G, et al. Forging & Stamping Technology, 2020(6), 168(in Chinese). 陈麒麟, 王东兴, 林建钢, 等. 锻压技术, 2020(6), 168. 17 Zhang Y Y, Li Y B, Wen Y T, et al. Acta Metrologica Sinica, 2020, 41(5), 544(in Chinese). 张玉燕, 李永保, 温银堂, 等. 计量学报, 2020, 41(5), 544. 18 Qi J Y, Li J Y. China Welding, 2019, 28(1), 56. 19 Yu H L, Hao L H, Liang H, et al. Journal of Forestry Research, 2019, 30(6), 2379. 20 Hu X Z, Sui Q M. Instrumentation Customer, 2017, 34(1), 21(in Chinese). 胡秀珍, 隋青美. 仪器仪表用户, 2017, 34(1), 21. 21 Ding W, Wang Q G, Zhu J F. IEEE Access, 2020, 8, 44661. 22 Chao C H, Eugene S, Po L, et al. Advances in Technology Innovation, 2020, 5(2), 76. 23 Zhou X, Wang Y N, Zhu Q, et al. Journal of Electronic Measurement and Instrument, 2016, 30(5), 702. 24 Wang Y, Wu Z H, Duan X Y, et al. IOP Conference Series-Earth and Environmental Science, 2018, 108, 022025. 25 Yang J, Yang G. Algorithms, 2018, 11(3), 28. 26 Jieun P, Dokkyun Y, Sangmin J. Symmetry, 2020,12(4),615. 27 Hinton G E, Osindero S, Teh Y W. Neural Computation, 2014, 18(7), 1527. 28 Hinton G, Salakhutdinov R. Science, 2006, 313(5786), 504. 29 Goodfellow I J, Pouget A J, Mirza M, et al. Advances in Neural Information Processing Systems, 2014, 3,2672. 30 Krizhevsky A, Sutskever I, Hinton G. In: Conference Record of 2012 Conference and Workshop on Neural Information Processing Systems. Lake Tahoe, 2012, pp. 1097. 31 Deng J, Dong W, Socher R, et al. In: Conference Record of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, Florida, USA,2009, pp. 248. 32 Simonyan K, Zisserman A. Computer Science, DOI: 10.48550/arXiv.1409.1556. 33 Szegedy C, Liu W, Jia Y, et al. In: Conference Record of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, 2015, pp.1. 34 Ioffe S, Szegedy C. In: Conference Record of the 32nd International Conference on Machine Learning. Lille, 2015, pp. 448. 35 Szegedy C, Vanhoucke V, Ioffe S, et al. In: Conference Record of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, 2016, pp. 2818. 36 He K, Zhang X, Ren S, et al. In: Conference Record of Computer Vision and Pattern Recognition (CVPR). Las Vegas, 2016, pp. 770. 37 He K, Zhang X, Ren S, et al. Computer Vision, 2016, 9908, 630. 38 Huang G, Liu Z, Van D M, et al. In: Conference Record of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, 2017, pp. 2261. 39 Howard A G, Zhu M, Chen B, et al.In: Conference Record of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA, 2017. 40 Park J K, Kwon B K, Park J H, et al. International Journal of Precision Engineering and Manufacturing-Green Technology, 2016, 3(3), 303. 41 Kiryong K, Heeyoung K. IEEE Transactions on Semiconductor Manufacturing, 2018, 31(3), 395. 42 Yao M H, Chen Z H. Computer Measurement & Control, 2018, 26(9), 29(in Chinese). 姚明海, 陈志浩. 计算机测量与控制, 2018, 26(9), 29. 43 Szegedy C, Ioffe S, Vanhoucke V. In: Conference Record of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-2017). San Francisco,2017. 44 Deitsch S, Christlein V, Berger S, et al. Solar Energy, 2019, 185, 455. 45 Yang Y X, Zhao M. Heavy Machinery, 2019(2), 25(in Chinese). 杨延西, 赵梦. 重型机械, 2019(2), 25. 46 Zhang Z, Wen G, Chen S. Journal of Manufacturing Processes, 2019, 45, 208. 47 Guan L L. Research on crystal defect classification and detection techno-logy based on deep learning. China Academy of Engineering Physics, China, 2019(in Chinese). 贯丽丽. 基于深度学习的晶体缺陷分类检测技术研究. 硕士学位论文, 中国工程物理研究院, 2019. 48 Fu L. IFAC-Papers on Line, 2018, 51(17), 45. 49 Wei R F, Bi Y B. Materials, 2019, 12(10), 1681. 50 Zhan X L, Zhao W T. Laser & Optoelectronics Progress, 2020, 57(10), 143(in Chinese). 詹湘琳, 赵婉婷. 激光与光电子学进展, 2020, 57(10), 143. 51 Bang H T, Park S, Jeon H. Composite Structures, 2020, 246, 112405. 52 Di H, Ke X, Peng Z, et al. Optics and Lasers in Engineering, 2019, 117, 40. 53 Yu He, Song K, Dong H, et al. Optics and Lasers in Engineering, 2019, 122, 294. 54 Pan S J, Yang Q. IEEE Transactions on Knowledge & Data Engineering, 2010, 22(10), 1345. 55 Sun J, Wang P, Luo Y K, et al. IEEE Transactions on Instrumentation & Measurement, 2019, 68(12), 4787. 56 O'Leary J, Sawlani K, Mesbah A. IEEE Transactions on Semiconductor Manufacturing, 2020, 33(1), 72. 57 Zhang N, Donahue J, Girshick R, et al. In: European Conference on Computer Vision. Springer International Publishing, Germany,2014. 58 He K, Zhang X, Ren S, et al. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 37(9), 1904. 59 Girshick R. In: Conference Record of the 2015 IEEE International Conference on Computer Vision. Santiago, Chile, 2015. 60 Ren S, He K, Girshick R, et al. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6), 1137. 61 Redmon J, Divvala S, Girshick R, et al. In: Conference Record of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA.2016, pp. 779. 62 Redmon J, Farhadi A. In: Conference Record of the 2017 IEEE Confe-rence on Computer Vision and Pattern Recognition. Honolulu, HI, USA, 2017, pp. 6517. 63 Redmon J, Farhadi A. In: Conference Record of the 2018 IEEE Confe-rence on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, 2018. 64 Bochkovskiy A, Wang C Y, Liao H Y M. In: Conference Record of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. USA, 2020. 65 Zhao X Y, Cai C P, Wang S, et al. Light Industry Machinery, 2019, 37(3), 60(in Chinese). 赵欣洋, 蔡超鹏, 王思, 等. 轻工机械, 2019, 37(3), 60. 66 Zhang Y Y, Li Y B, Wen Y T, et al. Acta Armamentarii, 2019, 40(11), 2329(in Chinese). 张玉燕, 李永保, 温银堂, 等. 兵工学报, 2019, 40(11), 2329. 67 Gao X, Jian M, Hu M, et al. Advances in Structural Engineering, 2019, 22(13), 2907. 68 Guo M H, Xu H W. Computer Systems & Applications, 2019, 28(11), 265. 69 Li Z J, Jiang X J, Zhu Z T, et al. Modular Machine Tool & Automatic Manufacturing Technique, 2019(9), 102(in Chinese). 利照坚, 江秀娟, 朱铮涛, 等. 组合机床与自动化加工技术, 2019(9), 102. 70 Peng Y, Xiao S H, Yuan J H, et al. Modular Machine Tool & Automatic Manufacturing Technique, 2020(3), 91(in Chinese). 彭煜, 肖书浩, 阮金华, 等. 组合机床与自动化加工技术, 2020(3), 91. 71 He Y, Song K, Meng Q, et al. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4), 1493. 72 Tao X, Zhang D P, Wang Z H, et al. IEEE Transactions on Systems, Man and Cybernetics: Systems, 2020, 50(4), 1486. 73 Yuan D D, Yan X Y, Lu D D, et al. Construction Machinery and Equipment, 2020, 51(1), 23(in Chinese). 苑丹丹, 阎相伊, 路丹丹, 等. 工程机械, 2020, 51(1), 23. 74 Dai J F, Li Y, He K M, et al. In: Conference Record of the 2016 Conference on Neural Information Processing Systems. Barcelona, 2016. 75 Wei X, Wei D, Suo D, et al. IEEE Access, 2020, 8, 61973. 76 Zhang Y, Cheng X S, Chui H H, et al. Optics & Optoelectronic Technology, 2020, 18(3), 75(in Chinese). 张逸, 程筱胜, 崔海华, 等. 光学与光电技术, 2020, 18(3), 75. 77 Athanasiadis T, Mylonas P, Avrithis Y, et al. IEEE Transactions on Circuits & Systems for Video Technology, 2007, 17, 298. 78 Long J, Shelhamer E, Darrell T. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4), 640. 79 Ronneberger O, Fischer P, Brox T. In: Conference Record of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention(MICCAI). Switzerland, 2015, pp. 234. 80 Badrinarayanan V, Kendall A. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 39(12), 2481. 81 Noh H, Hong S, Han B. In: Conference Record of the 2015 IEEE International Conference on Computer Vision( ICCV). Santiago, Chile, 2015, pp. 1520. 82 Liangchieh C, Papandreou G, Kokkinos I, et al. Computer Science, 2014(4), 357. 83 He K, Gkioxari G, Piotr D, et al. In: Conference Record of the 2017 IEEE International Conference on Computer Vision. Venice, Italy,2017, pp. 322. 84 Liu S, Qi L, Qin H, et al. In: Conference Record of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, 2018, pp. 8759. 85 Liu H, Peng C, Yu C, et al. In: Conference Record of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, 2019, pp. 6172. 86 Mohan R, Valada A. International Journal of Computer Vision, 2021,129,1551. 87 Han H, Gao C, Zhao Y, et al. Pattern Recognition Letters, 2018, 130, 234. 88 He T, Liu Y, Xu C, et al. IEEE Access, 2019, 7, 123453. 89 Dung C V, Anh L D. Automation in Construction, 2019, 99, 52. 90 Roberts G, Haile S Y, Sainju R, et al. Scientific Reports,2019,9(1),1. 91 Rahman M R, Chen H. IEEE Access, 2020, 8, 40547. 92 Zhang Q, Chang X, Bian S B, et al. IEEE Access, 2020, 8, 6997. 93 Han M, Wu Q X, Zeng X J. Computer Systems & Applications, 2020, 29(3), 100(in Chinese). 韩明, 吴庆祥, 曾雄军. 计算机系统应用, 2020, 29(3), 100.
渝公网安备50019002502923号 版权所有：《材料导报》编辑部
2022, Vol. 36 
