SF AppWorks is a full service digital agency focused on emerging technologies, but every once in awhile we get to have a little fun prototyping advanced concepts. At the 2016 TechCrunch Disrupt London Hackathon, our team pushed the limits of technology by creating a voice journal with AI powered real-time sentiment analysis. We built the product end-to-end in less than 24 hours, then took first place out of 700 developers. Here’s how we did it.

THE INSPIRATION


For years SF AppWorks has had a fascination with the connection between the human mind and computers, and has tried to explore how artificial intelligence can be used to address mental health issues. We build the Autism therapy App Auteachism pro-bono, and earlier in 2016 a team from the company prototyped an app to roleplay anxiety-inducing situations and improve the user’s response to his or her anxiety triggers. 

Journaling has proven psychological, stress-reducing, goal-orienting benefits when done regularly. But for many people, writing is harder than talking. It seemed a perfect use case to bring mental health awareness to the forefront with an AI-powered solution that everybody could benefit from. 

The team used IBM Watson to create a smart voice journal. You talk to it about your day and it analyzes your emotions and stores the logs. If you do it once a day you can see a visual representation of your feelings and experiences over time. Using artificial intelligence to improve people’s mental health is quite neat.


THE BACKSTORY


In Early 2016, SF AppWorks decided to refocus our future efforts on AI (we prefer Augmented Intelligence to Artificial Intelligence). To spur innovation and get the entire team up to speed, we held a series of brainstorms that culminated in an internal Hackathon. 25 people (almost half of the team!) participated, spending 24 grueling, fun, blurry hours brainstorming, planning, coding, ripping up plans, restarting, then struggling to finish a proof of concept by demo time. In the end, we built working prototypes of a smart mirror, a therapeutic chatbot, a digital travel agent, a predictive shopping platform, and an intelligent restaurant suggestion guide. Though we were incredibly proud of all of the hacks’, we could only pick one team to go to London to compete in the TechCrunch Disrupt London Hackathon.

 

SF AppWorks Hackathon Teaser from SF AppWorks on Vimeo.


THE HACKATHON


To win the Hackathon, we needed a powerful demo that could easily convey the concept. 

We had it narrowed down to three ideas, but ultimately decided to focus on an AI-powered voice journal because it was the most sophisticated use of AI, and because we felt the demo would resonate well with the audience. 

One advantage we realized was to focus on the demo from the very beginning. We had a few clever ideas that wouldn’t present well, so we ditched them early on. We also knew we only had one minute to present, so before we even wrote a line of code we were already going through the demo flow and practicing transitions. This helped us hone the product definition to features that would improve the demo. If not, we cut them, saving us time and energy.

Once we had the idea, we got to work, coding, refining, and practicing through the night. By 2AM we could have wrapped up, but we spent the remaining time tweaking colors and copy, improving the UI, and speeding up the AI-integration.

THE DEMO


The Emotion Journal is a voice journal. You talk to it about your day and it analyzes your emotions and stores the logs. If you do it once a day you can see a visual representation of your feelings and experiences over time.

A demo of the Emotion Journal is available at emotionjournal.online

We used IBM Watson to power real-time emotional analyses and speech-to-text/text-to-speech, and React/Node.JS/Express.JS to create a voice-powered interface for real-time feedback, journal analysis and storage. Accurate detection of emotions via text is a work in progress, but far more accessible to developers than it was even a few years ago. With the tools being developed to assist developers, there are going to be many opportunities to use AI to improve people’s mental health, confidence, and productivity. We’d like to play a part in that progress.