Work / Ultra-high-resolution YOLO detection
Ultra-high-resolution YOLO detection
The YOLO detection and matching I wrote place a big inspection project's components in 3D in about a day. Cluster QC, my own idea, cut missed detections by roughly 70 %.
Re-creation. Only the terrain and the faint orange wire returns are real, from a public LiDAR tile (ARSO, GKOT TM_426_66, CC BY 4.0). The towers, conductors, dampers, photos, boxes, the matcher and the second opinion are simulated. None of my employer's code, data or imagery is used.
What the real one does
My feature matching algorithm places a big inspection project's YOLO detections in 3D in about a day, where linking them by hand would take more than two years. The detector power grid elements on 150-megapixel imagery. An original procedure I designed and implemented cut wrong and missed detections by roughly 70 %.
- A damper takes up a tiny part of a 150 MP frame. Shrink the photo and the detector loses it, and a single photo can't tell how far away it is.
- That is why the detection runs on full-resolution tiles with SAHI, which I patched to process tiles in parallel for faster runs. Each component appears in many overlapping photos, and matching works out which detections belong to one object. It then places that object in 3D, to a few centimetres at about 50 m from the camera.
- Cluster QC, the second opinion. The matcher sometimes counts a damper twice or misses one, and I started Cluster QC on my own to catch that. Cluster QC asks every photo of a spot how many dampers it sees. Each photo is a vote, and the count most photos agree on wins. On a project of about 30,000 elements it cut missed detections, and the manual review they cause, by roughly 70 %.
- I wrote the training, the dataset tools for annotation in CVAT, batch inference and the matching. For the 3D placement I chose epipolar geometry over structure-from-motion, because it is fast and the photos never need opening again once detection has run. Cluster QC was my idea and my initiative, and I wrote its 880-line design document and the ground-truth benchmark that measures every change.
- Result. A couple of weeks of flight data take a couple of days of YOLO detection and about a day of 3D placement. People then inspect and check the results in the geotagger for one to two weeks, depending on the project. For a project of about 30,000 elements, I assessed whether the job could be done at all and worked out how.
How the demo works
The demo follows one spot on the line: the tiles, the boxes in six photos, rays and proposals in 3D, the matcher's elements, and the second opinion.
The photos come from a helicopter flying beside the line, each with a position and an orientation. Here each one is cut down to one spot: two or three dampers near a tower's clamp. The first step shows a whole photo and the tiles the detector runs over.
A box is only a direction. Each one becomes a ray from its camera, and rays from different photos that meet near one point are a proposal. The 3D view shows the rays. Hover a photo to pick out its own.
The matcher groups the rays into elements, and that is where most of the mistakes happen. Two dampers less than a metre apart can fall into one group. A small error in some photos' orientation can split one damper into two.
The second opinion counts instead of matching. Each photo votes how many elements it sees on the spot. Seen at a slant, two neighbours can share one box, so a few photos vote low, but most photos still agree. Where their count differs from the matcher's, the spot is fixed. Untick the second opinion to see what the matcher alone would have sent on.
Next spot cycles through three spots: one the matcher gets right, one where it joins two dampers, and one where it counts a damper twice.