Pilingo
#UI, #UX, #userResearch, #prototyping, #needFinding, #machineLearning
When I’m in a different country, how do I take ownership of my own health?
It delivers a pain free search for the correct medicine.
I created UX and UI flows for the app, and the final InVision prototype.
Research and need-finding
Did you know that more than 60% of the people in the UK don’t have a sufficient functional health literacy to actively participate and engage at their doctor appointments?
CERN has some of the most advanced machine learning and image recognition technologies in the world.
And what has the most cold, hard, data in the field of medicine? Drugs!
Finding medicines in foreign countries has always been an issue for ALL of us! But is this an issue for anyone else?
Result was unanimous. Everyone has, at some point, had an issue with finding medicine when traveling or living abroad!
Two pain-points emerged
1. Identifying and selecting medicine people feel comfortable with
2. Finding this medicine
We saw this as a perfect intervention point and ended up with our final research question.
Brainstorming Solutions
Your phone already has all the capabilities we need to make a service like this work.
An app is the most accessible way to help the most people.
Prototyping
We took to the whiteboard to map out our user journey.
And turned that into more granular user flows based on what the user can see and do at any given moment in the app.
Hi-Fi Mocks
Pilingo is a simple way to translate, locate, and communicate unfamiliar drug packaging for people travelling abroad.
Pilingo identifies medicine brands through image recognition and translates the packaging into a local alternative from a home medicine that you’re familiar with.
After comparing different medicines for you to choose from, Pilingo will use your geolocation to guide you to the nearest pharmacy that stocks your selection. When you’re feeling unwell in a foreign country, Pilingo helps improve the communication between you and the pharmacist.
By letting you use images rather than words, Pilingo helps pharmacists understand your needs better, help you quicker, and empowers you to have greater autonomy over your own health!
Technological Viability
While most medicine information stays the same, the packaging can vary. To make sure that our database can be useful, it has to be up-to-date.
Using machine learning and image recognition, we can create a deep convolutional neural network model that can correctly identify the drug packaging.
But… we need a lot of images to train it!
-
Run a daily script scouring Google and Bing images to build out our baseline image database.
-
Each time the user takes a picture, we ask them to confirm whether we’ve correctly identified their medicine.
- Use that picture to train the model further. So the more users use Pilingo, the better it actually gets!
Presenting our work
We were invited to show Pilingo at the Work-In-Progress show at the RCA in January 2019
Next Steps
After the initial MVP, we plan to improve our ML model through crowdsourcing, by adding an "ANSWER" mode.
Users can swipe LEFT or RIGHT to confirm or deny whether our model has correctly identified the image.
To incentivize people to use it we’ll show them the number of people they’ve helped with their translation and partner with drug companies to offer discounts based on how many medicines you’ve helped to identify!