Limitations of Generative AI for Numerical Aggregation of Apple Health Step Count Data: A Validation Study Using XML Records, Manual Calculation, and Health Auto Export
Abstract
Zhenghua Li and Kenichi Yamamura
Background: Generative artificial intelligence (AI) tools are increasingly used to assist data handling and research workflows. However, their reliability for numerical aggregation of raw health data has not been fully validated. Objective: This study examined whether generative AI could accurately aggregate Apple Health step count data from original XML records and compared the results with manual calculation and Health Auto Export.
Methods: Step count records were extracted from Apple Health XML data. The original data consisted of multiple measurement records per day. Daily step counts were calculated using four approaches: two independent aggregations using Apple Intelligence plus ChatGPT, Health Auto Export, and manual calculation in Excel from the original XML- derived records. Daily totals and weekly summaries were compared across methods.
Results: The two AI-based aggregations produced different daily and total step counts and did not agree with either Health Auto Export or manual calculation. In contrast, Health Auto Export and manual Excel calculation showed complete agreement for all daily step counts examined. The initial AI-based analysis also incorrectly treated the data as steps per measurement rather than steps per day.
Conclusion: Generative AI was unreliable for primary numerical aggregation of Apple Health step count data, even for simple summation tasks. Health Auto Export, validated against manual calculation, provided reproducible daily step count data suitable for subsequent analysis. Generative AI may be useful for research planning, interpretation, and manuscript preparation, but raw data extraction, numerical aggregation, and statistical calculation should be performed using reproducible computational tools.