Why I trust LiDAR for low-light scenes — and where it falls down

I run the hardware end of the rigs we ship. That means I am the person at five a.m. in a parking lot in Pennsylvania checking that the sensor housing has not iced over on the drive out. Cameras, depth sensors, IMUs, the mount, the cable harness, the enclosure tolerance — that is my beat. So when I say "I trust LiDAR for low-light scenes," I mean it from the angle of a person who has built and broken a lot of them.

The case for LiDAR after dark is straightforward and worth saying plainly: an RGB camera in a dimly lit scene is making up most of what it sees. Long exposures smear motion. High-ISO short exposures bury the signal in sensor noise. A stereo-RGB pair degrades together; both halves of the pair lose contrast at the same rate, so disparity matching collapses exactly when you need depth most. A time-of-flight or scanning LiDAR does not care about ambient light. It emits its own. Range returns are independent of how dark the parking lot is.

That is the simple case. The interesting case is where it stops being true.

The geometry I actually deploy

For outdoor low-light capture work I default to a three-sensor stack: a scanning LiDAR for primary depth, a low-light RGB camera for color, and an IMU rigidly coupled to both for motion compensation. The stack is calibrated as one unit and stays bolted together across the engagement. If the customer needs thermal, that is a fourth sensor and a separate calibration. I do not bolt thermal on at the last minute.

The data flow looks like this:

   ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
   │  Scanning    │     │  Low-light   │     │  IMU         │
   │  LiDAR       │     │  RGB camera  │     │  (6-axis)    │
   └──────┬───────┘     └──────┬───────┘     └──────┬───────┘
          │                    │                    │
          │ range returns      │ frames + ts        │ ω, a + ts
          │                    │                    │
          ▼                    ▼                    ▼
   ┌──────────────────────────────────────────────────────┐
   │  Hardware time-sync trigger (PPS or shared clock)    │
   └──────────────────────────────────────────────────────┘
                              │
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │  Per-frame fusion: depth + color + ego-motion        │
   │  → calibration applied (extrinsics + intrinsics)     │
   │  → outlier filter (multi-sweep agreement)            │
   └──────────────────────────────────────────────────────┘
                              │
                              ▼
                   colored point cloud / mesh
        

Time-sync is not optional. Without a shared clock, the LiDAR thinks the car is twelve meters away and the RGB frame thinks it is fourteen, and the resulting coloured point cloud has a ghost where the car used to be. We ship every rig with a hardware trigger line — a cable, not a software handshake.

Where it falls down

Wet asphalt. I mentioned this in the takeaways and it deserves the long-form. A dry road is a diffuse reflector. A wet road is close to a mirror. A scanning LiDAR pulse hitting a wet road at a shallow angle bounces forward, away from the sensor. The road registers as a void. On rainy-night captures we either change capture geometry to steeper angles, or we fall back to a stereo-RGB configuration with extra illumination. Stereo-RGB at night is a worse pipeline, but it is a pipeline that works on water. LiDAR alone there is not.

Matte black surfaces. A flat black painted vehicle, at the kind of ranges that matter for vehicle perception, returns very little energy on a single sweep. We have learned to never trust a single-sweep "no return" as evidence of absence. Our standard practice is to require multi-sweep confirmation — three consecutive sweeps showing the same void — before the downstream pipeline treats it as real geometry. That latency is acceptable for static-scene reconstruction; for moving-target perception it is a known limitation we flag.

Glass and reflective signage. The pulse goes through. Or it bounces back as a clean return at the wrong range. A storefront window will give you a depth estimate that places the storefront ten meters behind the building. Fusion with RGB catches this — there is a colour-edge discontinuity where the LiDAR depth jumps — but only if you are running the fusion. LiDAR alone gets it wrong confidently.

The decision I actually make on a job

For a daytime outdoor capture, I will often skip LiDAR entirely. Multi-view photogrammetry from a calibrated RGB rig produces a denser, prettier mesh, and the customer can interpret the result without specialised tooling. LiDAR earns its place when the lighting is bad, the geometry needs to be metric without per-job photogrammetric scale recovery, or the surfaces are textureless (a large blank wall is the photogrammetrist's nightmare and the LiDAR's afternoon).

The right answer is almost always "fuse them, and let the pipeline weight by confidence." That is harder to set up. Once it is set up, it is the answer across most of our outdoor work — daytime included.

What I want the next buyer to take away

• Active depth is not photometric truth.

  LiDAR returns range, not reflectance. If your downstream pipeline cares about color, surface material, or texture, you still need an RGB stream calibrated against the LiDAR. The two together are the deliverable; the LiDAR alone is a wireframe.

• Wet asphalt eats class-1 lasers for breakfast.

  The mirror-like specular return on wet road surfaces sends most of the energy somewhere other than the sensor. You see the car on the wet road; you do not see the road. Plan the capture geometry around it, or fuse with stereo RGB on rainy nights.

• Matte black surfaces will lie to you.

  A black sedan at fifteen meters in low light absorbs enough of the laser pulse to register as "no object detected" on a single sweep. We aggregate over multiple sweeps and apply a confidence threshold before accepting a hole as real geometry.

• Mount it like you mean it.

  Vibration is the silent killer of multi-sensor rigs. Every LiDAR I have deployed has needed a damped mount, a torque spec on the bolts, and a re-calibration check at the start of every capture day. There is no software fix for a sensor that moved on the bracket.

• Decide on the fusion strategy before you order the parts.

  Time-of-flight and rolling-shutter RGB do not synchronize themselves. If your reconstruction depends on per-frame correspondence between the two streams, you need either hardware sync or post-hoc alignment with a known fiducial. Order the trigger cable now, not on the day of the shoot.

— AK-mee Engineering