Project Bowling Ball — CV meets ten pins, on a Jetson, over a different box

This one started over coffee. We had just finished a long round of Project Sand enclosure work — depth sensor, pico projector, Jetson, all bolted into a contained box with a sandbox sitting under it. Real-time terrain visualisation. Gestures. Contour lines on physical sand. Fun project, ships, works.

Then somebody said "what about pins?" and the rest of the afternoon was a whiteboard.

So this is a paper case study. Not a product, not a roadmap, not a pitch. Just the write-up of the whiteboard, because the answer turned out to be more interesting than we expected.

Why pins are a clean CV problem

Most of the computer-vision work AK-mee runs is in the wild. Outdoor capture rigs. Variable lighting. Wet asphalt at midnight. Pipelines that have to work when the weather, the geometry, and the customer's schedule are all hostile.

A bowling lane is the opposite. Ten pins. Bounded geometry — a known triangular arrangement, fixed lane width, fixed pin-deck distance. Two states per pin: down or standing. Bright, controlled, repeatable lighting. The pins do not move except when the ball hits them, and after that they settle within a second. If you were designing a textbook problem for showing off edge inference, you would design this one.

The constrained-ness is what makes the hardware reuse interesting. You do not need a new sensor stack. You do not need a new compute platform. The thing we already shipped — depth-sensor + projector + Jetson + a contained enclosure — drops in almost cleanly, just over a different box.

The block diagram

   ┌──────────────────┐     ┌──────────────────┐
   │  Depth sensor    │     │  RGB camera      │
   │  (above pin deck)│     │  (above pin deck)│
   └────────┬─────────┘     └────────┬─────────┘
            │                        │
            │ depth + RGB at ~30 fps │
            ▼                        ▼
   ┌──────────────────────────────────────────┐
   │  Jetson edge compute                     │
   │   • per-pin segmentation (10 ROIs)       │
   │   • state classifier: down / standing    │
   │   • settle-window: stable for N frames   │
   │   • frame-to-frame scoring state machine │
   └──────────────────────┬───────────────────┘
                          │ pin-state vector + score
                          ▼
              ┌────────────────────┐
              │  Pico projector    │
              │  +  Side display   │
              │  (lane scorecard)  │
              └────────────────────┘
        

Camera and depth above the pins, on a fixed gantry behind the pin spotter. Inference on the Jetson. Overlay or scorecard out the back. The same shape as Sand, just looking at a different surface.

What I would actually decide on day one

• Sensor placement: above-deck, looking down.

  Side-mounted depth sensors lose line-of-sight the moment a pin tips toward them. A fixed gantry behind the pin spotter, looking straight down, gives clean geometry on all ten pins for the price of a real installation. Sand makes the same call from the other direction — sensor above the box.

• Frame rate: 30 fps is plenty.

  Pin motion settles in well under a second. The inference pipeline only needs to sample post-settle. Running the Jetson at 30 fps continuously is a power decision, not a performance one.

• Latency budget: under three seconds, end-to-end.

  From "ball hits pins" to "score on display." Real inference is sub-100 ms per frame on the model classes we are imagining; the bottleneck is the settle-window wait, not compute. Generous, achievable, and means the projector overlay can update on the same frame as the side display.

• Lean is "down."

  A pin partially fallen but resting on a neighbour scores as down under standard rules. The classifier needs a tilt threshold, not a binary upright check. Small detail, the kind of thing that turns a working demo into a working product.

• Calibrate at lane-open, not on every frame.

  A 12-hour shift drifts. Same pattern Sand uses against the empty-deck reference: check the depth baseline at open, recalibrate if it has shifted more than a few millimetres. No software fix for a sensor that moved on the bracket.

What we would learn if we built it

• The ten-pin problem is constrained in the right ways.

  Ten objects, two states, bounded geometry, bright lighting. Compared to outdoor capture work in a parking lot at five a.m. in the rain, this is a vacation. Most of the engineering judgement goes into the rules-of-bowling edge cases, not the CV.

• Hardware reuse is a real story, not a brochure line.

  The same depth sensor, the same projector, the same Jetson, the same containerised vision pipeline pattern. The thing that changes is the box.

• The output channel matters as much as the inference.

  A pico projector might be bright enough to overlay on a polished lane surface, or it might not. A side-display per lane probably matters more than the lane overlay. We would learn that fast on the first install.

Where this stops

This is a paper case study. There is no project board, no parts order, no customer pitch — the design lives on the whiteboard and in this post.

The reason we are publishing it anyway is that the hardware-reuse story is the interesting story. We have built one of these rigs before. The next one over a different box should not be hard. If somebody — a bowling-alley operator, a partner, a curious engineer — wants to take this further, the door is open and the design is on the table.

And in the meantime, it is a good story about how the boring, contained, constrained problems are sometimes the most fun ones to think about.

— AK-mee Engineering