AI-based Smart Diagnosis System for CCTV Camera Inspection in Tokyo

1. Introduction:
Tokyo Metropolitan Government (TMG) has adopted the 'Mirror-lens Type TV Camera' one of CCTV camera inspections, which enable to capture clear, distortion-free images of the entire sewer wall only by moving straight ahead since 2010, in surveys of approximately 16,200 km deteriorating sewer. Specifically, the CCTV camera inspection companies output the images showing all directions of sewer inside taken by Mirror-lens Type TV Cameras. They are sliced from top of sewers and converted to zonal images by application of the 'System for Unfolding of Sewer Wall'. This system is jointly developed by Tokyo Metropolitan Sewerage Service Corporation and Nippon Koei Co., Ltd. Finally, coding and grading on deterioration (hereinafter referred to diagnosis) is performed by engineers visually confirming the damage with the help of the 'Sewer Inspection and Diagnosis Support System'. This system displays candidate of damage areas such as crack and corrosion in colors on the unfolded images. This paper presents the result of developing a smart diagnosis system in which AI performs the final diagnosis, without the need for engineers. It also reports on our verification of the feasibility of AI-based diagnosis on other cameras that can create unfolded images by targeting the general-purpose 'fisheye-lens type TV camera' for purpose of expanding the use of this system.

2. AI-based Sewer Degradation Diagnosis System:
(1) Existing Technologies and Introduction of Smart Diagnosis System: TMG conducts CCTV camera inspections and diagnosis in sewer pipes using three technologies: 'Mirror-lens Type TV Camera', 'System for Unfolding of Sewer Wall', and 'Sewer Inspection and Diagnosis Support System'. A FS survey was conducted to see if AI could replace the final diagnosis work (Fig. 1 STEP-4) that had been conducted by engineers. This was conducted with the financial support of the government in 2020.
(2) Supervised Data and Test Data: From the unfolded images of approximately 200,000 segments (approximately 12,000 km of pipeline) owned by TMG, approximately 40,000 damaged images were extracted as supervised data, and 25,000 damaged images were extracted as test data for five key codings of damage that lead to road cave-ins: crack, break, corrosion, joint lag, and water intrusion.(3) Evaluation Method: From the viewpoint of not overlooking serious damage that leads to road cave-ins, we evaluated the result of 'Image classification' by deep learning based on the supervised data with 'recall'. The recall is an index that indicates the percentage of images that the AI also predicted to be positive among in fact they are positive, and is a method of evaluating the 'little of missed cases' (Fig. 2).
(4) Evaluation Result: For the 5,963 image data that had been previously diagnosed by engineers as having damage grade A, which is the most damaged of the three grade ratings A, B, and C, AI also determined that 5,702 of them were damaged, for the recall of 95.6% (Table 1). Thus, the AI diagnosis was able to obtain the high recall.

3. Feasibility Study for Expansion of Application:
(1) Purpose of Feasibility Study: In 2020, the feasibility of introducing AI diagnosis was verified using unfolded images created from the image data of Mirror-lens Type TV Cameras. In order to verify several representative types of cameras, we conducted a feasibility study to see whether AI can automatically diagnose the images taken by fisheye-lens type TV camera A and B that create unfolded images by running a test pipe. Two types of mylar paper with different grid patterns printed on and a piece of paper with actual images of damages were glued to a 250mm diameter PVC pipe. The grid patterns are used to check the distortion of unfolded images, and the actual image of the damage is used to check whether automatic diagnosis by AI is possible. (2) Confirmation of Unfolded Images: Figs. 3 and 4 show the results of the unfolded images taken by fisheye-lens type TV camera A and B respectively. Although a slight distortion was observed in the grid pattern with both cameras, there were no major problems in reading the damage status, and thus the AI diagnosis was considered applicable. (3) Availability of Automatic diagnosis by AI: The actual images of damage in sewers bonded to the test pipe showed two types of damage codings: crack and break. The possibility of automatic diagnosis by AI was verified after going through the Sewer Inspection and Diagnosis Support System for the unfolded images of fisheye-lens type TV camera A and B. The result showed that both cameras were able to reliably determine the presence or absence of damage, although the damage codings were not accurate (Table 2). This confirmed that the automatic diagnosis by AI is effective even for the unfolded images of fisheye-lens type TV cameras. 4.: Conclusion In this study, AI diagnostics achieved the recall of more than 95% for damage grade A, indicating that the system has sufficient potential to be used in practice. In addition, the AI was able to determine the presence/absence of damage even for the unfolded images acquired with other general-purpose cameras, confirming the possibility for expanding use of this system to a wide range of cameras. Through wide use of this AI model, sewer maintenance management will be conducted even more efficiently.