Google unveils cough-analysing AI for disease diagnosis

As the digital cough monitoring landscape gains momentum, a team led by Google scientists has unveiled an AI tool that could one day facilitate disease diagnosis by evaluating human sounds like coughing and breathing.

The AI system, dubbed Health Acoustic Representations (HeAR), was trained on a dataset of 313 million two-second audio clips of human sounds such as coughs, breathing, and throat clearing culled from YouTube videos.

Most AI tools being developed in this space are trained on audio recordings that are labelled  with the health status of the person who made the sounds. But unlike traditional supervised learning models that learn to link characteristics of the sounds with labelled data, HeAR takes a self-supervised approach allowing it to learn from a diverse dataset without requiring explicit human annotations.

Self-supervised learning

The Google research team extracted visual representations of sound, or spectrograms, from the audio clips and trained the model to predict missing portions, similar to techniques used to train large language models.

The team adapted HeAR to detect whether a person might be a smoker or has a respiratory infection like COVID-19 or tuberculosis. They reported their findings recently in a preprint uploaded to arXiv.

When evaluated on cough inference tasks, they said HeAR "performed better" than the baselines across 10 of 14 tasks, including COVID tasks; while on tuberculosis and chest X-ray tasks, its performance was "comparable to the best performing model."

Specifically, HeAR scored as high as 0.710 and 0.739 for COVID detection and tuberculosis, respectively, on a scale where 0.5 represents a model that performs no better than a random prediction and 1 represents an accurate prediction each time.

Whether HeAR eventually becomes available as a commercial product is unclear at this stage, but Sujay Kakarmath, a product manager at Google who worked on the project, said that the team’s goal “is to spur innovation in this nascent field."

Real-world challenges

While Google's HeAR could be a step forward in the field of audiomics, companies like Hyfe have been pioneering the use of cough monitoring for disease detection for several years. The company's CoughMonitor platform allows clinicians to get real-time data on their patients' cough patterns, frequency, and intensity over a period of time, enabling improved monitoring and management of conditions such as chronic obstructive pulmonary disease, asthma and lung cancer.

However, Mindaugas Galvosas, the company's digital health lead, told FirstWord, "We currently believe that 'spot-check' models, where a person coughs into a microphone and the AI interprets that single cough, are still far away from scalable real-world implementation." Hyfe has submitted its CoughMonitor to the FDA for de novo authorisation, and if greenlit, it could be the first cough monitoring solution approved by the US regulator.