Egej Vencelj Senior software engineer

Work / Point cloud to photo projection

Point cloud to photo projection

I wrote the camera geometry behind Panarch's inspection photos. It draws a line's 3D model into each photo and casts rays back out of it.

Re-creation. None of it is the production code. The terrain and the conductors come from a public LiDAR tile (ARSO, GKOT TM_426_66, CC BY 4.0), with the conductors fitted for the wire-vectorization demo. The camera, the towers and the "photo" are simulated.

Re-creation: a 3D view of a power-line span with two camera frustums beside the camera's view, with the model drawn over it.
Fig. 1 Re-creation: a 3D view of a power-line span with two camera frustums beside the camera's view, with the model drawn over it.

What the real one does

The camera geometry I wrote projects a power grid's 3D data into each inspection photo. If the camera pose is even a little off, the drawing misses the wires, and solving the pose makes it land.

  • It covers mount rotations, the camera's intrinsics, projecting LiDAR wires and towers into a photo, and the ray back out from a pixel.
  • Panarch uses it to check detections against the model and to place in 3D what the camera saw.

How the demo works

Spoil the camera's pose and the model misses the photo. Solve the pose and it lands again.

A helicopter photo comes with a position from GNSS and an orientation from the gimbal. The position is good. The orientation is a few degrees off, and at 100 m a degree is almost two metres. Draw the 3D model into the photo with that pose and the wires land beside the wires.

The left view is the world: terrain and conductors from public LiDAR, a white frustum for the true camera and an orange one for our estimate. The right view is the camera's own picture, with the model drawn through the orange camera.

Solving the pose means finding the yaw, pitch, roll and focal length that make the drawn model and the picture agree. Here that is Gauss-Newton on the reprojection error over points along the conductors, one step per frame so you can watch it. In practice the matches come from the image, not from a known answer, and that is where most of the work goes.