Heads up: Our Ideas Factory has been refreshed, levelled up, and grown-up into Alphero Intelligence. Some of our old posts are pretty cool tho'. Check this one out.
- OCR (optical character recognition) is a type of machine learning that can convert image-based text into machine-encoded text.
- Credit card OCR scanners are common in apps, but we wanted to scan driver licences.
- We needed to add a whole bunch of acceptance criteria for different types of NZ licences.
Our developers started with an off-the-shelf machine learning model for recognising and classifying images, and retrained it to specifically identify and classify NZ driver licences.
To improve the user experience of the sign up process for NZTA’s Drive Go app, we identified an opportunity to remove the need for the user to manually enter their licence details themselves. Instead, you could just scan the front and back of your card using your phone camera.
But it’s not as easy as it sounds. Behind the scenes, there are a number of complex calculations being made as to how and what is being “read”. OCR (optical character recognition) can be a tricky beast. There were a number of things our clever developers had to consider, including different licence formats and different licence stages (i.e. learning, restricted, full).
Essentially, the process starts all the way from asking the user for camera permissions, to displaying the camera preview in-app, to running the frames through OCR and algorithmically processing the results.
The machine learning based implementation didn’t make it to the production app. Instead, we implemented a rule-based recognition approach that utilised the OCR capability to pull data from the drivers’ licence. The resulting experience is still the same, and so it was favoured over the machine learning recognition component.