Mass General Brigham AI predicts cancer patient survival from photos

FAHR-Face model trained on 40 million images

Mak and his colleagues trained an algorithm on 40 million facial images spanning diverse ages and ethnic backgrounds to form the base model, known as the Foundation Artificial Intelligence for Health Recognition-Face (FAHR-Face). The researchers then fine-tuned this base model to develop two task-specific algorithms.

The first, FAHR-FaceAge, estimates a person’s biological age from a single photo. The second, FAHR-FaceSurvival, predicts a cancer patient’s chances of survival by combining facial features with other clinical information.

When researchers tasked the algorithm with predicting cancer patients’ risk of dying, it was described as "shockingly accurate" for some people.

AI analyzes specific facial regions for health insights

Raymond Mak, a radiation oncologist at the Mass General Brigham Cancer Institute and faculty member in the Artificial Intelligence in Medicine program, explained the specific facial regions the models analyze. The FAHR-FaceAge model calculates a patient’s biological age by examining the temples, the forehead, the area above the eyes, and the nasolabial fold, which is the line connecting the nose to the corner of the mouth.

Conversely, the FAHR-FaceSurvival model predicts mortality risk by observing the bridge of the nose and the area under the eyes.

Clinical data shows high accuracy in mortality risk assessment

Clinical testing of the FAHR-FaceSurvival model showed a clear distinction in outcomes for high-risk patients. Among 122 patients identified as high risk, 52.5% died within 90 days, while only 2% of the 975 patients in the lowest-risk group died in that same timeframe.

The FAHR-FaceAge model also demonstrated predictive capabilities regarding mortality. Among 1,060 patients whose biological age was more than 20 years above their chronological age, 21% died within 90 days. In comparison, 6% of the 4,195 patients whose biological age was at least 10 years younger than their chronological age died within 90 days.

AI findings mirror ancient observations by Hippocrates

The AI models identified facial regions consistent with those described by the Greek physician Hippocrates around 400 B.C.. These specific areas were detailed in the Book of Prognostics as indicators of muscle wasting and mortality risk.

The algorithms were trained purely on facial photographs and data regarding age or mortality. They had no access to the ancient texts during this training process.

Raymond Mak described this correlation as "eerie" because the software functioned without any internet text. The system independently arrived at medical conclusions similar to those documented millennia ago.

Cosmetic procedures do not impact AI performance

Researchers found that the performance of the AI models was not affected by cosmetic changes. This includes alterations such as makeup, surgical procedures, and injectables like Botox.

Potential for informed cancer treatment decisions

Raymond Mak, a radiation oncologist at the Mass General Brigham Cancer Institute, led the development of these tools. He is also a faculty member in the Artificial Intelligence in Medicine program at Mass General Brigham.

The models were trained on more than 40 million facial images. These images spanned diverse ages and ethnic backgrounds to create a broad data foundation.

Facial analysis is suggested as an additional tool for doctors to evaluate a patient’s survival chances. It may assist clinicians in deciding which treatment approaches will work best for individual cases.

Researchers hope the technology will help doctors and patients make informed, objective decisions about cancer treatment. This includes providing more clarity regarding end-of-life care options.