Cost-effective flight strategy for aerial thermography inspection of photovoltaic power plants

Thermography from an unmanned aerial vehicle (UAV) offers efficient fault inspection in solar parks. This paper presents an optimized fault detection procedure, including a two-stage autonomous flight strategy. The first stage images the entire park quickly at low resolution, flying fast at high altitude to detect possible faults. In the second stage, the detections are imaged at higher resolution, by flying an optimized path at lower altitude, for fault classification. Based on actual data, including a park-scale survey, this strategy is shown to have potential for significant savings in operation time, on the order of 70% or more, at realistic fault densities.


INTRODUCTION
The ever-increasing scale of photovoltaic (PV) parks requires new and more efficient operation and maintenance. Thermography is a robust inspection tool because deviations in the module temperature distribution is a strong indication of a defect [1]. However, manual thermal inspection is too time consuming and expensive with today's utility scale parks, leading to a need for smarter, faster and more cost-effective methods. Recent advances in aerial technology, sensors, and control systems make it possible to utilize unmanned aerial vehicles (UAVs) for inspection and monitoring of PV energy plants. With an onboard thermal camera and a precise GPS, thermal module anomalies can be efficiently detected and localized. Anomalies can then be classified according to their appearance: Short-circuited bypass diodes result in heating of one substring, a whole short-circuited module is recognized as a patchwork of hot cells, and external shadowing or cell cracking can make a single cell clearly warmer than the others. Also heating of a part of a cell or a point can be observed [2].
The thermal image processing is quite well studied, but less attention has been paid to the UAV flight procedure and image acquisition. With today's large PV parks the cost effectiveness of a UAV monitoring routine depends on an efficient flight plan, since the main cost is related to the presence of trained personnel [3]. Here we discuss factors influencing the data quality and inspection efficiency, such as velocity, altitude, and camera characteristics. We propose a two-stage autonomous flight planning strategy involving an exploration stage to quickly detect possible defects in the entire park from a high altitude, and a verification and classification stage revisiting only the detected anomalies, at a lower altitude, for high-confidence verification and classification.

II. BACKGROUND
The UAV flight time depends on several factors, but dominating is the velocity and the altitude of the UAV. The upper limit for flight speed that still enables thermography with sufficient image quality is determined by the drone stability and the camera. The UAV induces camera movements due to vibration and automated maneuvering, which may lead to image motion blur. A camera gimbal can compensate for these corrections to some extent, and therefore permit flight in a wider range of wind conditions, as well as a higher UAV speed.
The camera characteristics are important for image quality and inspection efficiency. The image sharpness is often not well described by the pixel count, due to point spread function "smear" from lens imperfections. The sensitivity is limited by noise expressed as a Noise Equivalent Temperature Difference, NETD. Cost constraints will dictate use of uncooled microbolometer cameras whose speed is limited by a pixel time constant fixed in the design. The pixel area A, focal length, flight speed, and the fixed camera time constant τ determine the amount of motion blur. A technological tradeoff exists, described by a figure of merit for time constant and noise [4]: FOM = NETD • • . For observing the relatively large temperature contrasts of interest here, it is preferable to use cameras with a small time constant, tolerating some increase in NETD to minimize motion blur and enable faster flights.
The UAV flight altitude determines the resolution and area coverage. The required resolution depends on the goal of the image analysis, where two different paths can be taken: fault classification or fault detection. Fault classification requires high module resolution to differentiate between different defect types, preferably at least 5x5 pixels per cell [5]. Classification is necessary to rank the severity of a defect -a bypassed substring will for instance result in a module power reduction of 33%, while a defective cell can be less than 1%. On the other hand, a cell hotspot can reach temperatures above 100ºC, and pose a risk to the safety of e.g. rooftop PV systems.
It can be observed that if the objective is only to detect thermal anomalies, with no consideration of the underlying cause, requirements on resolution and image quality are relaxed, and the flight speed and/or altitude may be increased. An altitude that allows for imaging of several module rows simultaneously will allow faster coverage, but may only permit detection of faults, and not classification.

III. FLIGHT STRATEGY
Noting the efficiency potential in a flight pattern that only permits defect detection, we here explore a combined measurement strategy with two stages: In the first exploration stage, the entire park is imaged with a fast flight at high altitude, in a pre-defined "lawn-mower" pattern, in order to detect and geolocate thermal anomalies. In the second verification and classification stage, the UAV follows an optimized path to visit only the detected anomalies, at a lower altitude and slower speed to collect higher quality imagery for classification. Based on a real data set, we show that significant savings are possible by reducing the total duration of the operation.

IV. EXPERIMENTAL
A park-scale dataset was obtained by waypoint UAV flights over the PV park Megasol in Arvika, Sweden. Megasol consists of around 4000 modules with a tilt angle of 40°. The UAV was a DJI Matrice 100 with an Optris Pi 640 thermal camera with 640×480 pixels and a field of view of 33°. The thermal imagery was recorded by an Udoo onboard computer, which also streamed the images to a ground station for real time monitoring. For this initial evaluation of the proposed concept, we recorded high resolution imagery of the entire park from an altitude of 20 meters to get a resolution of 5x5 pixels per cell. The duration of the flight was 32 minutes. We use these data to predict performance and flight time for different scenarios, as discussed in the following. To illustrate the effects of altitude and motion blur we use imagery from flights at varying altitudes over Glava Energy Center in Sweden.
V. DATA PROCESSING For the first exploration stage, we implemented a fast automatic fault detection process. First, pixel values are clipped to a temperature range of 15 degrees [6], starting at five degrees below the temperature of healthy modules. By ignoring lower temperatures, the background is mostly removed. An object detection algorithm based on the "Binary Large Object Detection" function (blob detection) from OpenCV [7] is then run on the images. To reduce false detections, selection criteria are applied to the detected blobs: Only detections with a size ranging from 10% of a cell to two cells are accepted, to include both hot module points and bypassed substrings, and the temperature difference ΔT between the object and the background must exceed 10 degrees to be counted, in accordance with IEA criteria [6].
All detections are tracked through consecutive images to ensure that they are registered only once. The tracking algorithm is based on "Simple object tracking with OpenCV" [8], with some modifications to better fit the task: Instead of an isotropic tracking search, we use the UAV altitude, velocity and heading to calculate the expected position of each detection in the next frame. If a detection matches a prediction, it is associated to the previous detection, otherwise it is added to the list of detected anomalies. Finally, the global positions of all objects are estimated using the logged UAV position and orientation, together with the intrinsic and extrinsic camera parameters.
Before the verification and classification stage, we seek to find the shortest path that revisits all the detected anomalies. This is a "traveling salesman problem", and it is well known that an optimal solution is computationally unfeasible. In our case, however, even an approximate solution provides the benefits we seek. We therefore use the heuristic Ant Colony Optimization (ACO) algorithm [9] to plan the flight path for the second stage. This method, modelled on ant behavior, is computationally feasible even for on-board path planning on a small UAV computer.

VI. RESULTS AND DISCUSSION
A. Image quality Setting the UAV velocity and altitude is a tradeoff between image quality and operation time. In Fig. 1 a) and b) the same cell hotspot is imaged from an altitude of 30 and 80 meters, respectively. The object detection algorithm successfully detects the hotspot even at 80 meters, but we observe that the image quality is then insufficient for classification: The detected hotspot is one heated cell, which covers 1.6% of the module area (for a 60-cell module). The insets show a temperature contour drawn at the same overtemperature, relative to the module, in both cases. At 30 meters the area within the contour has the nominal area of 1.6%. At 80 meters we observe a smearing due to lens imperfections, resulting in an apparent hotspot area of 3.3%. Then it is not possible to classify the hotspot robustly as one hot cell. This illustrates that detection can be possible at a large area coverage rate, while classification requires closer distances.
The influence of velocity on image quality is demonstrated in Fig. 1 c) and d), where the same modules are imaged at 4 and 10 m/s, respectively. Motion blur is clearly distorting the image quality at high velocities, again making classification more difficult. A gimbal can compensate for random UAV motion, but cost constraints will preclude the use of more advanced systems for full elimination of motion blur.

B. UAV Inspection of PV Park
The fault detection algorithm was run on the Megasol image set to represent the exploration phase. The image processing steps are illustrated in Fig. 2, and the detections are marked in yellow on the map in Fig. 3 a). A total number of 93 detections were found. These were compared to a manual inspection of the data to determine the accuracy of the detection, which identified the 93 detections as 74% hotspots, 12% hotspots with ΔT just below 10 degrees, and 14% false detections. 69 of the 81 defects in the park were detected, which makes the detection rate 85%.

C. UAV Path Planning
The 32-minute flight to record high resolution data at Megasol can be taken as a baseline case representing a normal flight strategy. We first note that for this moderate-scale park, a fast flight at 80 meters with a velocity of 5 m/s for the exploration phase would require only 4 minutes to cover the park area. The automatic detections in Fig. 3 a) are taken to represent the output of this first stage. The ACO is then used to calculate a near optimal path between the defects, which can be used by the UAV flight controller during the verification and classification stage. The resulting path is shown as a white line in Fig. 3 a). The green marker is the starting and landing spot for the UAV.
If the verification and classification phase uses a velocity of 3 m/s at 20 m altitude, this phase requires about 7 minutes, plus some extra time for maneuvering more and sharper turns than in a lawnmower pattern. The two stages then give a time reduction of almost 70%. A further time reduction can be achieved with a more dynamic UAV velocity by a faster transit speed between detections. This will be especially desirable in PV parks with few defects and potentially long distances between detections.
At Megasol, 81 thermal anomalies were found, corresponding to approximately 2% of the modules. This is a somewhat higher defect density than usually found in utility scale PV parks, where around 0.5% to 1% has been reported [10]. We therefore also estimate the flight time for a lower density of defective modules. A random sample of 40 and 20 defects were drawn from the pool of detections, corresponding to a defect density of 1% and 0.5%. The new optimized paths are shown in Fig. 3 b) and c). This gave a total inspection time of approximately 9 minutes and 8 minutes, respectively. It is thus clear that a two-stage flight strategy can reduce the inspection time significantly.

VII. SUMMARY AND CONCLUSION
Today's large PV parks require efficient UAV monitoring routines to achieve cost-effective inspections. We look at different factors that influence the data quality and costeffectiveness, and demonstrate an automatic fault detection and tracking algorithm on a UAV waypoint mission over a PV park in Sweden. The waypoint mission is compared to a proposed two-stage autonomous flight planning strategy. In the first exploration stage the defects are quickly mapped out from a high altitude with a predefined flight path and an automatic fault detection. An ACO algorithm is then used to estimate an optimal route that revisits the detections during the verification and classification stage. A lower altitude and speed will in this stage allow for fault classification. This lets us benefit from the relaxed image quality requirements for fault detection, and only image modules of interest with a higher image quality. A comparison between the two flight strategies shows that the two-stage flight plan potentially leads to significant savings in operation time, by 70% or more, as well as a reduced data load. Fig. 2. Image processing: a) original image, b) temperature clipping, c) thresholding, d) automatic fault detection Fig. 3. Detected defects marked in yellow, with a near-optimal travel path from the ACO in white. The green marker shows UAV start and landing. a) All detected defects, b) random sample constituting 1% defective modules, c) 0.5% defective modules 978-1-6654-1922-2/21/$31.00 ©2021 IEEE