Face recognition on a Raspberry Pi 3 at Shuttl

Hey folks,

So I just tweeted this today and I am pretty pleased with myself, not gonna lie:

I was on TV demonstrating the face recognition tech we made. Isn’t that neat? Used #Python + #OpenCV3 + #AzureAPI

It IS neat! The project is called S.A.F.E., and behind it is a desk full of jumper wires, a Raspberry Pi 3 and at least one proper crunch. I joined Shuttl in May, and a couple of months in I was already deep in R&D, posting photos of hand-soldered boards. 😎

June 2017, a few weeks into Shuttl. A Pi booting up on my desk.

Here’s the story behind that tweet, including the questions I ended up asking strangers on the internet.

What are we building?

Shuttl runs buses. The idea is simple: a screen with a camera inside the bus, a rider looks at it, the system recognises them, greets them by name and authenticates their entry. Boom.

The full kit is face-recognition onboarding, ShuttlTV and an always-on alcohol detector with ignition cut-off, all running on a single Raspberry Pi 3. Yes, one Pi. :)

The crunch

The first thing I posted from this whole thing was a hand-soldered board on my desk, with the caption “Mood right now.”

July 2017. “Mood right now.”

Twelve days later I tweeted: “That’s what we did in last few days of crunch. More coverage to follow.” The link was NDTV Gadgets’ story, Shuttl app gets new passenger safety features.

Two days after that, the bench looked like this:

“Today.” A Pi 3, an Arduino Uno, a gas sensor and the Pi camera. The paperwork on my desk is blurred on purpose.

That’s the Pi 3 in its case, jumper-wired to the Arduino Uno, with the gas sensor wired in at the bottom left and the camera module up top on its ribbon cable. Beautiful mess, right?

The FPS problem

Face detection on the Pi 3 works. The FPS and the resolution are just pretty low. 😕

I’d been following the PyImageSearch face detection tutorials for the Pi, so in August I asked Adrian directly:

Hi Adrian, I just loved your face detection with RasPi tutorials. However, the FPS/Resolution is pretty low for RPi3…

…Sure it’s due to bottleneck from RAM/CPU on Pi, what alternative (cheapest) hardware would you suggest for reliable 18FPS/720P detection?

18 FPS at 720p was my target. His answer was polite and honest: that would be extremely challenging on a Pi. Instead of throwing hardware at it, he pointed me to threading the camera capture, with his 2016 post on wrapping picamera and cv2.VideoCapture in one class.

Here’s the deal. Reading a frame from the camera blocks. If your main loop reads a frame, then detects faces, then draws, then reads the next frame, you’re paying for the camera wait on EVERY single loop. Put the capture in its own thread that keeps grabbing the latest frame, and the main loop only ever does the processing work. Simple, but so smart.

I replied “Sure, thanks. I will check threading. :)”

On TV!

The TV appearance was WION’s Tech It Out, episode 44, “The future of Transportation”. My bit starts at 13:08. The episode went up on YouTube on 9 August, and yeah, I only got round to tweeting it about four weeks later. Better late than never, eh?

About an hour after the TV tweet I posted the onboarding screen in action, with three fire emoji and a misspelt hashtag (#FaceRecogniton, I know, I know).

A frame from the GIF I just posted. That’s me in the box, and the people behind me are blurred. The moving version is on the tweet.

The flow is: stand in front of the screen, get your face into the box, and as soon as your face is matched the system plays an audio greeting with your name. “Hello Jatin”, in my case. Never gets old. 🔥

If you want more of my Raspberry Pi tinkering, check out my Raspberry Pi 2 walkthrough, along with the NAS and LAMP server projects I built on it.

Hit me up on twitter if you have questions about any of this. Till then, keep hacking! 😎