Google researchers have developed an artificial intelligence tool that estimates body fat using only smartphone photos, offering a simpler alternative to traditional medical imaging. The technology, known as PhotoScan, leverages advancements in computer vision and machine learning to interpret visual data in ways that were previously limited to high-cost diagnostic equipment. By analyzing subtle variations in body shape, posture, and surface contours, the system attempts to replicate the precision of clinical methods without requiring physical contact or specialized sensors.
How PhotoScan AI works
The system, called PhotoScan, analyzes a series of 2D images of a person’s body to estimate body composition. It was trained on data from dual-energy X-ray absorptiometry (DXA) scans, which measure overall body fat levels and specific fat distribution ratios, including the android-to-gynoid (A/G) ratio and visceral-to-subcutaneous (V/S) fat ratio. The A/G ratio compares fat stored in the abdominal region—often associated with higher metabolic risks—to fat in the hips and thighs, which is considered less harmful. The V/S ratio differentiates between visceral fat, which surrounds internal organs and is linked to inflammation and disease, and subcutaneous fat, which lies just beneath the skin and is generally less metabolically active.
Researchers combined DXA data with MRI scans, which provide detailed cross-sectional images of soft tissues, to enhance the model’s ability to interpret fat distribution patterns. The training process involved aligning these high-resolution medical images with corresponding smartphone photos to teach the AI how visual cues correlate with underlying fat composition. Later refinements incorporated real-world smartphone images, accounting for variations in lighting, clothing, and camera angles to improve robustness. The result, they say, is a tool that estimates body fat more accurately than bioelectrical impedance analysis (BIA) sensors found in smartwatches and other wearables. Unlike BIA, which relies on electrical currents to estimate body fat and is affected by hydration levels, PhotoScan’s visual approach avoids these limitations. It also provides insights into fat distribution that BIA sensors cannot, such as distinguishing between visceral and subcutaneous fat, which have different health implications.
Beyond body fat: Predicting health risks
Google’s research didn’t stop at body composition. The team tested whether PhotoScan could help predict insulin resistance, a condition linked to diabetes and other metabolic disorders. Insulin resistance occurs when cells in the body become less responsive to insulin, leading to raised blood sugar levels and increasing the risk of type 2 diabetes, cardiovascular disease, and fatty liver disease. Traditional methods for assessing insulin resistance often require blood tests, such as fasting glucose or hemoglobin A1c measurements, which can be inconvenient for regular monitoring. By incorporating PhotoScan’s body composition estimates into predictive models, the researchers found that accuracy improved, nearly matching results from clinical DXA scans. In comparison, BIA data from wearables did not enhance predictions, likely because these sensors lack the granularity to detect fat distribution patterns that correlate with metabolic dysfunction.
Related: Apple macOS update hints at new products
This suggests the tool could eventually support non-invasive screening for cardiometabolic health risks. For now, it remains a research prototype, but Google envisions integrating it with wearable data, glucose readings, and other health metrics to create a more full view of metabolic health. For example, combining PhotoScan’s fat distribution estimates with continuous glucose monitoring data could help identify individuals at higher risk for insulin resistance before symptoms appear. Similarly, pairing it with activity tracking from wearables might reveal how lifestyle changes affect fat storage patterns over time. The goal is to provide a full, real-time assessment of metabolic health without the need for frequent clinic visits or invasive testing.
The idea of tracking body composition without specialized equipment isn’t new, but the accessibility of smartphones makes this approach different. Previous attempts to estimate body fat from images relied on controlled environments, such as 3D scanners or professional photography setups, which are impractical for everyday use. Most people already carry a device capable of capturing the necessary images, removing a major barrier to regular monitoring. That convenience could matter more than precision in some cases—especially for those who wouldn’t otherwise seek out clinical testing. For individuals in remote areas or with limited access to healthcare, a smartphone-based tool could offer an early warning system for metabolic risks. Even in well-served regions, the ease of use might encourage more frequent tracking, helping users make informed decisions about diet, exercise, and medical follow-ups.
Accuracy vs. accessibility
Clinical DXA scans remain the gold standard for body composition analysis, but they require expensive equipment and trained technicians. A single DXA machine can cost over $100,000, and scans typically take 10-20 minutes to perform, making them impractical for widespread screening. BIA sensors, while convenient, provide only basic estimates. These devices, commonly found in smart scales and fitness trackers, measure how electrical signals travel through the body, with fat resisting the current more than muscle or water. However, their accuracy varies based on factors like hydration, recent exercise, and even the time of day. PhotoScan aims to bridge that gap, offering near-DXA accuracy without the need for specialized hardware. The system’s reliance on visual data means it isn’t affected by the same variables that skew BIA results, though it may still face challenges with factors like loose clothing or inconsistent posing.
Google has not announced plans to release PhotoScan as a consumer product. The research team emphasized that the tool is still in development, and further validation is needed before it could be used in medical or wellness settings. Potential hurdles include ensuring consistent performance across different smartphone models, body types, and environmental conditions. Privacy concerns also arise when dealing with personal health data derived from images, requiring robust safeguards to protect user information. If it does become widely available, it could change how people track changes in body composition over time—without stepping on a scale. Instead of relying on weight or BMI, which fail to distinguish between muscle and fat, users could monitor shifts in fat distribution that are more closely tied to health outcomes. This could be particularly valuable for athletes, older adults, or individuals recovering from illness, where changes in body composition may not be reflected in weight alone.
Leave a Reply