Food Intake Cycle Dataset – v1

The Food Intake Cycle (FIC) dataset was created by the Multimedia Understanding Group towards the investigation of in-meal eating behavior. The FIC dataset contains the triaxial acceleration and orientation velocity signals (6 DoF) from 21 meal sessions provided by 12 unique subjects. All meals were recorded in the restaurant of Aristotle University of Thessaloniki using a commercial smartwatch, the Microsoft Band 2™ for ten out of the twenty-one meals and the Sony Smartwatch 2™ for the remaining meals. In addition, the start and end moments of each food intake cycle as well as of each micromovement are annotated throughout the FIC dataset.

Description

A total of 12 subjects were recorded while eating their launch at the university’s cafeteria. The total duration of the 21 meals sums up to 246 minutes, with a mean duration of 11.7 minutes.

Each participant was free to select the food of their preference, typically consisting of a starter soup, a salad, a main course and a desert. Prior to the recording, the participant was asked to wear the smartwatch to the hand that he typically uses in his everyday life to manipulate the fork and/or the spoon.

A GoPro™ Hero 5 camera was already set at the table of the participant using a small, 23 cm in height, tripod facing the participant, including both the food tray and upper body part in it’s field of view. The purpose of video recording was to obtain ground truth data by manually annotating the IMU sequences based on the video stream.

Participants were also asked to perform a clapping hand movement both at the start and end of the meal, for synchronization purposes (as this movement is distinctive in the accelerometer signal). No other instructions were given to the participants.

It should be noted that the FIC dataset does not contain instances related with liquid consumption or eating without the fork, knife and spoon (e.g. eating directly with hands).

Dataset history

  • FIC v0 (2017) can be found by following this link.

Annotation

Micromovements

For all recordings, the start and end points of all 6 micromovements of interest were manually labeled. The micro-movements of interest include:

  • pick food, wrist manipulates a fork to pick food from the plate
  • upwards, wrist moves upwards, towards the mouth area
  • downwards, wrist moves downwards, away from the mouth area
  • mouth, wrist inserts food in mouth
  • no movement, wrist exhibits no movement
  • other movement, every other wrist movement

The annotation process was performed in such a way that the start and end times of each micro-movement span the whole meal session, without overlapping each other.

Food intake cycles

For all recordings, we annotated the start and end points for each intake cycle (i.e. every bite). Each food intake cycle starts with a p, ends with a d and contains an m micromovement.

Data Statistics

Meals & food intake cycles

Type#Mean (s)Std (s)Median (s)Total (s)
Meal sessions21703.56186.18717.8814,774.80
Food intake cycles1,3324.523.223.556,023.07

Micromovements

Micromovement#Mean (s)Std (s)Median (s)Total (s)
p1,3761.651.401.162,275
u1,3690.930.510.811,274
d1,3430.630.450.53848
m1,3440.470.240.43632
n3286.035.754.131,978
o1,5175.657.303.278,576

Download

If you plan to use the FIC dataset or any of the resources found in this page, please cite our work:

  • @ARTICLE{kyritsis2019modeling, author={K. Kyritsis and C. Diou and A. Delopoulos}, journal={IEEE Journal of Biomedical and Health Informatics}, title={Modeling Wrist Micromovements to Measure In-Meal Eating Behavior from Inertial Sensor Data}, year={2019}, pages={1-1}, doi={10.1109/JBHI.2019.2892011}, ISSN={2168-2194}, month={},}
  • @INPROCEEDINGS{kyritsis2017automated, author={K. Kyritsis and C. L. Tatli and C. Diou and A. Delopoulos}, booktitle={2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)}, title={Automated analysis of in meal eating behavior using a commercial wristband IMU sensor}, year={2017}, pages={2843-2846}, doi={10.1109/EMBC.2017.8037449}, ISSN={1558-4615}, month={July},}
  • @InProceedings{kyritsis2017food, author=”Kyritsis, Konstantinos and Diou, Christos and Delopoulos, Anastasios”, title=”Food Intake Detection from Inertial Sensors Using LSTM Networks”, booktitle=”New Trends in Image Analysis and Processing — ICIAP 2017″, year=”2017″, publisher=”Springer International Publishing”, pages=”411–418″, isbn=”978-3-319-70742-6″ }
  • @INPROCEEDINGS{kyritsis2018end, author={K. Kyritsis and C. Diou and A. Delopoulos}, booktitle={2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)}, title={End-to-end Learning for Measuring in-meal Eating Behavior from a Smartwatch}, year={2018}, pages={5511-5514}, doi={10.1109/EMBC.2018.8513627}, ISSN={1558-4615}, month={July},}
  • @INPROCEEDINGS{papadopoulos2018personalised, author={A. Papadopoulos and K. Kyritsis and I. Sarafis and A. Delopoulos}, booktitle={2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)}, title={Personalised meal eating behaviour analysis via semi-supervised learning}, year={2018}, pages={4768-4771}, doi={10.1109/EMBC.2018.8513174}, ISSN={1558-4615}, month={July},}

We provide the FIC dataset  in three formats. Use the following links to download the format that best fits your needs:

Technical details

After loading the dataset with your preferred framework you will find 6 keys:

  • subject_id
  • session_id
  • raw_signals
  • proc_signals
  • mm_gt
  • bite_gt
  • metadata

Each of these 6 keys contains a list of length 21 (equal to the number of meals). Elements across lists are aligned, e.g. the 3rd element of the session_id list corresponds to the 3rd element of the bite_gt list.

subject_id list

Each element of this list corresponds to the unique integer identifier of the subject (1 to 12).

session_id list

Each element of this list corresponds to the unique integer identifier of the session (1 to 3). It should be noted that not all subjects have the same number of sessions.

raw_signals list

Each element of this list is a dictionary/struct that contains 2 keys:

  • accelerometer

An N*4 matrix. The first column is the time, the second is the x-axis, the third is the y-axis and the forth is the z-axis.

  • gyroscope

An M*4 matrix. The first column is the time, the second is the x-axis, the third is the y-axis and the forth is the z-axis.

Keep in mind that all signals in raw_signals list are unprocessed, i.e. as obtained by the smartwatches.

proc_signals list

Each element of this list is an N*7 matrix that contains the processed accelerometer and gyroscope measurements.  Data from the first column indicate the timestamp of the sample, the second/third/forth the x/y/z channels of the accelerometer and the fifth/sixth and seventh the x/y and z channels of the gyroscope.

mm_gt list

Each element of this list is a L*3 matrix. Each row represents a single micromovement interval. The first column contains the start moments, the second column the end moments and the third column the label of the micromovement (from the set {p,u,d,m,o,n}}.

bite_gt list

Each element of this list is a K*2 matrix. Each row represents a single food intake (i.e. bite). The first column contains the start moments while the second column contains the end moments of the food intake cycle interval.

metadata list

Finally, each element of this list is a dictionary/struct that contains metadata using free text. The dictionary contains the following keys:

  • sensor

The sensor used for this specific recording

  • acceleromter_raw_units

The measurement units of the raw accelerometer (e.g. m/s^2)

  • gyroscope_raw_units

The measurement units of the raw gyroscope (e.g. rad/s)

  • timestamps_raw_units

The measurement units of the timestamps in the raw_signals list (e.g. ms)

  • accelerometer_proc_units

The measurement units of the preprocessed accelerometer

  • gyroscope_proc_units

The measurement units of the preprocessed gyroscope

  • timestamps_proc_units

The measurement units of the timestamps in the proc_signals list

  • bites_gt_units

Units of the food intake cycles start and stop moments

  • mm_gt_units

Units of the micromovement start and stop moments

Contact

For any inquiries about the FIC dataset contact Konstantinos Kyritsis at kokirits@mug.ee.auth.gr