The Food Intake Cycle (FIC) Dataset

Important update: Since 6/Jan/2021 the FIC dataset has migrated to Zenodo and can be downloaded by following this link. The Zenodo link also provides access to a script that help visualize the recordings. Finally, from now on we will only support the pickle format.


The Food Intake Cycle (FIC) dataset was created by the Multimedia Understanding Group towards the investigation of in-meal eating behavior. The FIC dataset contains the triaxial acceleration and orientation velocity signals (6 DoF) from 21 meal sessions provided by 12 unique subjects. All meals were recorded in the restaurant of Aristotle University of Thessaloniki using a commercial smartwatch, the Microsoft Band 2 for ten out of the twenty-one meals and the Sony Smartwatch 2 for the remaining meals. In addition, the start and end moments of each food intake cycle as well as of each micromovement are annotated throughout the FIC dataset.

Description

A total of 12 subjects were recorded while eating their launch at the university’s cafeteria. The total duration of the 21 meals sums up to 246 minutes, with a mean duration of 11.7 minutes.

Each participant was free to select the food of their preference, typically consisting of a starter soup, a salad, a main course and a desert. Prior to the recording, the participant was asked to wear the smartwatch to the hand that he typically uses in his everyday life to manipulate the fork and/or the spoon.

A GoPro Hero 5 camera was already set at the table of the participant using a small, 23 cm in height, tripod facing the participant, including both the food tray and upper body part in it’s field of view. The purpose of video recording was to obtain ground truth data by manually annotating the IMU sequences based on the video stream.

Participants were also asked to perform a clapping hand movement both at the start and end of the meal, for synchronization purposes (as this movement is distinctive in the accelerometer signal). No other instructions were given to the participants.

It should be noted that the FIC dataset does not contain instances related with liquid consumption or eating without the fork, knife and spoon (e.g. eating directly with hands).

FIC is also tightly related to FreeFIC, a dataset we created in order to investigate the in-the-wild eating behavior. More information about FreeFIC can be found here.

Annotation

Micromovements

For all recordings, the start and end points of all 6 micromovements of interest were manually labeled. The micro-movements of interest include:

  • pick food, wrist manipulates a fork to pick food from the plate
  • upwards, wrist moves upwards, towards the mouth area
  • downwards, wrist moves downwards, away from the mouth area
  • mouth, wrist inserts food in mouth
  • no movement, wrist exhibits no movement
  • other movement, every other wrist movement

The annotation process was performed in such a way that the start and end times of each micro-movement span the whole meal session, without overlapping each other.

Food intake cycles

For all recordings, we annotated the start and end points for each intake cycle (i.e. every bite). Each food intake cycle starts with a p, ends with a d and contains an m micromovement.

Data Statistics

Meals & food intake cycles

Type # Mean (s) Std (s) Median (s) Total (s)
Meal sessions 21 703.56 186.18 717.88 14,774.80
Food intake cycles 1,332 4.52 3.22 3.55 6,023.07

Micromovements

Micromovement # Mean (s) Std (s) Median (s) Total (s)
p 1,376 1.65 1.40 1.16 2,275
u 1,369 0.93 0.51 0.81 1,274
d 1,343 0.63 0.45 0.53 848
m 1,344 0.47 0.24 0.43 632
n 328 6.03 5.75 4.13 1,978
o 1,517 5.65 7.30 3.27 8,576

Download

If you plan to use the FIC dataset or any of the resources found in this page, please cite our work:

  • @article{kyritsis2019modeling,
        title={Modeling Wrist Micromovements to Measure In-Meal Eating Behavior from Inertial Sensor Data},
        author={Kyritsis, Konstantinos and Diou, Christos and Delopoulos, Anastasios},
        journal={IEEE journal of biomedical and health informatics},
        year={2019},
        publisher={IEEE}}
    
  • @inproceedings{kyritsis2017automated,
        title={Automated analysis of in meal eating behavior using a commercial wristband IMU sensor},
        author={Kyritsis, Konstantinos and Tatli, Christina Lefkothea and Diou, Christos and Delopoulos, Anastasios},
        booktitle={2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},
        pages={2843--2846},
        year={2017},
        organization={IEEE}}

We provide the FIC dataset  in three formats. Use the following links to download the format that best fits your needs:

  • Pickle
  • Mat file
  • H5

Important update: Since 6/Jan/2021 the FIC dataset has migrated to Zenodo and can be downloaded by following this link. The Zenodo link also provides access to a script that help visualize the recordings. Finally, from now on we will only support the pickle format.

 

Technical details

After loading the dataset with your preferred framework you will find 5 keys:

  • subject_id
  • session_id
  • signals
  • mm_gt
  • bite_gt

Each of these 5 keys contains a list of length 21 (equal to the number of meals). Elements across lists are aligned, e.g. the 3rd element of the session_id list corresponds to the 3rd element of the bite_gt list.

subject_id list

Each element of this list corresponds to the unique integer identifier of the subject [1-12].

The subject identifier in FIC is in line with the subject identifier in the FreeFIC dataset; i.e. FreeFIC’s subject with subject_id = 2 is the same person as FIC’s subject with subject_id = 2.

session_id list

Each element of this list corresponds to the unique integer identifier of the session [1-3]. It should be noted that not all subjects have the same number of sessions.

signals list

Each element of this list is an M*7 data matrix that contains the timestamps, 3D accelerometer and 3D gyroscope measurements for each meal. Specifically, the first column contains the timestamps in seconds, the second, third and forth columns contain the x,y and z accelerometer values in g and the fifth, sixth and seventh columns contain the x,y and z gyroscope values in degrees/second.

All data are recorded at 100Hz. In addition to that, all sensor streams are transformed in such a way that reflects all participants wearing the smartwatch at the same hand with the same orientation, thusly achieving data uniformity. This transformation is in par with the signals in the FreeFIC dataset.

mm_gt list

Each element of this list is a K*3 matrix. Each row represents a single micromovement interval. The first column contains the timestamps of the start moments in seconds, the second column the timestamps of the end moments in seconds and the third column an integer representing the type of the micromovement, taking values in [1-6].

The integer identifier to micromovement mapping is provided below.

1 -> no movement
2 -> upwards
3 -> downwards
4 -> pick food
5 -> mouth
6 -> other movement

bite_gt list

Each element of this list is a N*2 matrix. Each row represents a single food intake event (i.e. a bite). The first column contains the start moments while the second column contains the end moments of each intake event. Both the start and end moments are provided in seconds.

Dataset history

  • FIC v0 (2017) can be found by following this link.
  • FIC v1 (2018) can be found by following this link.
  • FIC migrated to Zenodo (6/1/2021). The version of the FIC dataset that was previously available in this page can be downloaded by clicking on this link.

Contact

For any inquiries about the FIC dataset contact Konstantinos Kyritsis at kokirits@mug.ee.auth.gr