Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features

Abstract

Malnutrition, including both undernutrition and obesity, is a significant problemin low- and middle-income countries (LMICs). In order to study malnutrition and develop effective intervention strategies, it is crucial to evaluate nutritional status in LMICs at the individual, household, and community levels. In a multinational research project supported by the Bill & Melinda Gates Foundation, we have been using a wearable technology to conduct objective dietary assessment in sub-Saharan Africa. Our assessment includes multiple diet-related activities in urban and rural families, including food sources (e.g., shopping, harvesting, and gathering), preservation/storage, preparation, cooking, and consumption (e.g., portion size and nutrition analysis). Our wearable device ("eButton" worn on the chest) acquires real-life images automatically during wake hours at preset time intervals. The recorded images, in amounts of tens of thousands per day, are post-processed to obtain the information of interest. Although we expect future Artificial Intelligence (AI) technology to extract the information automatically, at present we utilize AI to separate the acquired images into two binary classes: images with (Class 1) and without (Class 0) edible items. As a result, researchers need only to study Class-1 images, reducing their workload significantly. In this paper, we present a composite machine learning method to perform this classification, meeting the specific challenges of high complexity and diversity in the real-world LMIC data. Our method consists of a deep neural network (DNN) and a shallow learning network (SLN) connected by a novel probabilistic network interface layer. After presenting the details of our method, an image dataset acquired from Ghana is utilized to train and evaluate the machine learning system. Our comparative experiment indicates that the new composite method performs better than the conventional deep learning method assessed by integrated measures of sensitivity, specificity, and burden index, as indicated by the Receiver Operating Characteristic (ROC) curve.

Description
Keywords
egocentric image, wearable device, technology-based dietary assessment, low- and middle-income country, artificial intelligence, DOUBLE BURDEN, MALNUTRITION, TECHNOLOGY, COMPUTER, SENSORS, DIET, Computer Science, Artificial Intelligence, Computer Science, Information Systems
Citation
Chen, G., Jia, W., Zhao, Y., Mao, Z.-H., Lo, B., Anderson, A. K., Frost, G., Jobarteh, M. L., McCrory, M. A., Sazonov, E., Steiner-Asiedu, M., Ansong, R. S., Baranowski, T., Burke, L., & Sun, M. (2021). Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features. In Frontiers in Artificial Intelligence (Vol. 4). Frontiers Media SA. https://doi.org/10.3389/frai.2021.644712