PEOPLE with anxiety, depression or stress are often asked to record their mood changes throughout the day, helping psychologists fine-tune their treatment. But they often forget, recording only sparse information at best. Now an emotion-sensing smartphone app that automatically generates someone's "mood diary" could give psychologists all the data they need.
It's the brainchild of Matt Dobson and Duncan Barclay, founders of speech recognition firm EI Technologies, based in Saffron Walden, UK. Instead of relying on people writing diaries, the app, called Xpression, listens for telltale changes in a person's voice that indicate whether they are in one of five emotional states: calm, happy, sad, angry or anxious/frightened. It then lists a person's moods against the times they change, and automatically emails the list to their psychologist at the end of the day.
To work, the app has to be always on, listening out for the user's voice once every second, whether they are talking to family, friends, colleagues or even pets. It also listens in on phone calls. If the user is silent, the app does nothing. Crucially for the users' privacy, it doesn't record their words, instead seeking out telltale acoustic features – like pitch – that are indicative of emotional state.
This kind of emotion recognition via voice pattern already works well and is a "hot area" of research, says Stephen Cox, head of the speech processing lab at the University of East Anglia in Norwich, UK, who is scientific adviser for the firm.
Initially, Xpression will send 200-millisecond-long acoustic snapshots to a remote server where a machine-learning system will work out a person's emotional state before sending it back to the app for storage. Factors like voice loudness, intensity, changes in pitch and speaking pace allow the system to accurately estimate somebody's emotional state. "We extract acoustic features and let the machine-learning system work it out," says Cox. This ability will be built into the app itself eventually, says Dobson.
There's a strong need for this kind of technology, says Adrian Skinner, a clinical psychologist with the UK's National Health Service in Harrogate, North Yorkshire. "With conditions like depression, people tend to stop doing things like filling in mood diaries. If this app gives us more complete diaries it could help us better find the day-to-day triggers that raise or lower a patient's mood," he says.
The firm is a finalist in a UK government competition to identify the nation's top mobile tech company, to be judged on 26 February. An insurance company has already expressed an interest in using the app to ensure the workplace stress therapy it pays for is effective. Clinical trials are due to take place later this year.
This article appeared in print under the headline "We know how you really feel"
If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.
All comments should respect the New Scientist House Rules. If you think a particular comment breaks these rules then please use the "Report" link in that comment to report it to us.
If you are having a technical problem posting a comment, please contact technical support.
