Doodles to deliveries: GrabFood uses AI to deliver meals your kids drew up

To keep your children entertained at home, Grab recently partnered with Google to launch Grab Delivery Doodles, which uses Google's AI, TensorFlow, to turn any child's doodles of their favourite dishes into food orders delivered via GrabFood.

Whether it is cake, dim sum, tempura or pho, children just need to doodle their food cravings and scan their drawings using the new Delivery Doodles feature within the Grab app. Delivery Doodles will guess the dish and suggest nearby restaurants with that food to deliver it. GrabFood Delivery Doodles is available as a pilot in Malaysia today, and subsequently roll out in Vietnam on 25 May and Indonesia on 1 June.

Grab Delivery Doodles was built using a machine learning model that can recognise drawings of local dishes such as "martabak", "bánh mì" and "nasi lemak". Volunteers from Grab, Google and their families submitted over 10,000 doodles to help teach the AI to recognise kids’ drawings. Many of the same parents and children also became talents and DIY videographers in a demo video.

In a phone conversation with A+M, Grab's regional country marketing head Sulin Lau said this initiative was completed within a span of three and a half weeks. 

"It is getting harder to keep kids busy during the lockdown and parents are also running out of ideas for meals. This idea came about after a conversation with Google and we were initially hesitant to take it on since it sounded really complex. We required hundreds of training data per local dish in each country in order for the AI to recognise the dishes and learn the distinguishing traits of each dish," she explained.

To obtain sufficient training data, the team needed a minimum of 200 drawings per dish - simply because different children draw the same food differently. For example, if the team gets two children to draw "chicken", one might draw a drumstick and another might draw chicken wings or the whole chicken.

In the cases of a few dishes we collected a lot more than 200 drawings because there are even more variations - dishes such as dim sum, for example.

According to Lau, the team reached out to the parent groups among employees at Grab and Google via Zoom drawing classes and WhatsApp mum groups of friends. It went through as many avenues as possible to get as many children's doodles as possible. "One thing we all learnt - it is really hard to get any child to draw a salad," Lau said.

She explained that if the Delivery Doodles AI does not recognise the dish drawn, it is likely because the doodle is not approximating the distinguishing traits that the AI has picked up. For example, if the child is trying to draw bubble tea but only draws the cup without the boba, the AI might not recognise it and mistakenly interpret it as juice instead.

"It is unlikely that the dishes that the kids draw is not on the GrabFood platform. It would most likely be that the machine has not been trained to recognise all drawing interpretations of a particular dish," she added. On whether to expand Delivery Doodles into other markets, Lau said Grab will re-evaluate the need as lockdowns start to ease and children eventually return to schools and kindergartens. 

Having learnt from its previous virtual shoot for its Ramadan campaign, Lau said Grab went one step further and had the same parents who volunteered their children to submit the doodles, to direct and film them for the video. The Grab employees were briefed by creatives, shown a shoot board and given tips on how to use their phones to frame the shots. 

"In the end, 100% of this video was filmed by Grab employees who were parents, and will run on Google's platforms including YouTube," Lau said.

Do not dismiss great insights due to technical complexity

Initially, the Grab team had the impression that building a new user experience with AI would be much more problematic and take more time than it did. From that very first Zoom call, Lau said the team went from "This is a pretty cool idea, let's explore" to launching it in 35 days.

What it has taught us is the confidence to act on slightly crazier ideas - do not dismiss a great insight just because it is technically complex.

Lau explained that this was "a super valuable learning experience", and that being able to transform the doodles of a five-year-old into food orders is a new way of creating usage stimulus.

"New orders of GrabFood here come not from physical hunger but when parents creatively engage their children and teach them to appreciate local dishes. This also results in orders for our restaurant partners who need business more than ever," she added.

When marketers think about empathy during COVID-19, Lau said it has tended to result in somewhat serious ideas. Grab Delivery Doodles, however, is not only deeply rooted in COVID-lockdown insight, but also the idea and tech that came out of this was also delightful, magical and fun, she added.

Meanwhile, Aurélien Pichon, MD, Asia Pacific operations at Google, said underpinning Grab Delivery Doodles is a custom machine learning model built using tensorflow.js to run on device in near real-time. "We are hoping this will be helpful (and fun) for families as they are spending more time at home, while also supporting businesses across the region," Pichon added.