Accurate measurement and estimation of energy intake (EI) are important for the understanding of energy balance and body weight dynamics. Food is the primary source of energy in the body. Therefore, the objective monitoring of food intake patterns and eating behavior is necessary, as excessive EI leads to medical conditions such as obesity and overweight. Traditionally, self-report methods (e.g. food records, 24-hour recall) have been used for EI measurement. Most of these methods rely on a participant’s own declaration in one form or another and suffer from misreporting of EI. To lessen the misreporting problems, various methods have been proposed ranging from image-assisted estimation to wearable on-body sensors, each with its own strengths and limitations. While some of the methods show great promise under certain circumstances, objective, accurate and cost-efficient methods for estimating of EI are yet to be developed. Towards automatic food intake detection, this dissertation first explores the desired time resolution of sensor-based food intake detection to characterize meal microstructure. This dissertation then investigates a wearable sensor system for automatic food intake detection, microstructure parameter estimation, passive image capture, and EI estimation. The automatic food intake detection with this system is developed by monitoring chewing activity associated with ingestion. Along with an accelerometer sensor, a novel chewing sensor is introduced to capture chewing information and detect food intake events. A wearable camera, capturing passive images in 15sec intervals, is used to acquire food and non-food images. A visual review of the food intake images is performed to identify food items, estimate portion size and validate that food was consumed. A study was performed with participants wearing the wearable sensor system (Automatic Ingestion Monitor, AIM-2) for 24h in pseudo-free-living and 24h in a free-living environment. Classification models were developed for automatic food intake detection. The estimation of chew counts was obtained using the new chewing sensor which was not adhesively attached to the body. The dissertation further presents an EI estimation method using both sensor-extracted features and image estimated portion size. Results suggest the potential of the wearable system to estimate EI in a free-living environment.