Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

ImageCLEF 2018 Lifelog - ADLT

Activities of Daily Living understanding


Completed
13
Submissions
45
Participants
2065
Views

Overview


Important note:

The ImageCLEF Lifelog - Activities of Daily Living understanding (ADLT) challenge has officially ended and we would like to thank everybody for their participation. You can find the official results at http://imageclef.org/2018/lifelog.

Post-challenge submissions and the leaderboard will remain enabled for a few weeks so you will still be able to submit result files and have them continuously evaluated during a limited period. Please consider that in order to see the version of the leaderboard with the post-challenge submissions integrated, you have to turn on the switch Show post-challenge submission right below the leaderboard.

At the same time we’d like to encourage you to submit a CLEF Working notes paper until the end of May.

Please also note that participants registering from now on will not be automatically registered with CLEF anymore.


Note: ImageCLEF Lifelog 2018 is divided into 2 subtasks (challenges). This challenge is about Activities of Daily Living understanding (ADLT). For information on the Lifelog moment retrieval (LMRT) challenge click here . Both challenges share the same dataset, so registering for one of these challenges will automatically give you access to the other one.

Note: Do not forget to read the Rules section on this page

Motivation

An increasingly wide range of personal devices, such as smartphones, video cameras as well as wearable devices that allow capturing pictures, videos, and audio clips in every moment of our life are becoming available. Considering the huge volume of data created, commonly referred to as lifelogs, there is a need for systems that can automatically analyse the data in order to categorize, summarize and also query to retrieve the information the user may need.

Despite the increasing number of successful related workshops and panels ( JCDL 2015 , iConf 2016 , ACM MM 2016 , ACM MM 2017 ) lifelogging has seldom been the subject of a rigorous comparative benchmarking exercise as, for example, the new lifelog evaluation task at NTCIR-13 or the last year edition of the ImageCLEFlifelog task. In this edition of this task we aim to bring the attention of lifelogging to an as wide as possible audience and to promote research into some of the key challenges of the coming years.

Challenge description

Given a period of time, e.g., “From 13 August to 16 August” or “Every Saturday”, the participants should analyse the lifelog data and provide a summarisation based on the selected concepts (provided by the task organizers) of Activities of Daily Living (ADL) and the environmental settings / contexts in which these activities take place.

Some examples of ADL concepts: “Commuting (to work or another common venue)”, “Travelling (to a destination other than work, home or another common social event)”, “Preparing meals (include making tea or coffee)”, “Eating/drinking”, and contexts: “In an office environment”, “In a home”, “In an open space”. The summarisation should be described as the frequency and spending time for ADL concepts and total time for contexts concepts. For example:

  • ADL: “Eating/drinking: 6 times, 90 minutes”, “Travelling: 1 time, 60 minutes”;

  • Context: “In an office environment: 500 minutes”, “In a church: 30 minutes”.

Data

The task will be split into two related subtasks using a completely new multimodal dataset which consists of 50 days of data from a lifelogger, namely: images (1,500-2,500 per day from wearable cameras), visual concepts (automatically extracted visual concepts with varying rates of accuracy), semantic content (semantic locations, semantic activities) based on sensor readings (via the Moves App) on mobile devices, biometrics information (heart rate, galvanic skin response, calorie burn, steps, etc.), music listening history. The dataset is built based on the data available for the NTCIR-13 - Lifelog 2 task .

Format of the metadata

The metadata is stored in an .xml file, which is a simple aggregation of all users data. It is structured as follows:

The root node of the data is the USERS tag. Each user element contains all the data of that user (u1 or u2). Each user has a tag USER that contains the user ID as an attribute, example: [user id=”u1”]. For this year, only user u1 is considered. Inside the USER element, is his/her data:

Following that there is a tag DAYS, this tag contains the lifelogging information of that user organised per day, each day is included in a tag DAY that has the data (a tag DATA), the relative path to the directory that contains the images captured in that particular day (the tag IMAGES-DIRECTORY), then the minutes of of that day under a root tag called MINUTES.

At the start of each day there is a set of daily metatdata for that user. This data is of three forms; BIOMETRICS, ACTIVITIES & PERSONAL LOGS. The biometrics contains WEIGHT, FAT MASS, HEART RATE, SYSTOLIC blood pressure & DIASTOLIC blood pressure, which were readings taken after waking up each day. The activities contains summary activities: STEPS taken that day, DISTANCE walked in metres that day & ELEVATION climbed in metres that day. The personal logs contain HEALTH LOGS, including the TIME of reading, GLU Glucose levels in the blood, BP Blood Pressure, HR Heart Rate, MOOD manually logged every morning and sometimes a COMMENT, as well as DRINK LOGS and FOOD LOGS which were manually logged throughout the dat.

Following that, the day’s data is organised into minutes. The MINUTES element, contains exactly 1440 child elements (called MINUTE), each child has an ID (example: [minute id=“0”], [minute id=“1”], [minute id=“2”]… etc), and it represent one minute in the day ordered from 0 = 12:00 AM, to 1439 = 23:59PM.

Each minute contains: 0 or 1 location information (LOCATION tag), 0 or one activity information (ACTIVITY tag), biometrics, 0 or more captured images (IMAGES tag with IMAGE child element (each element has has a relative path to the image and a unique image ID), and 0 or 1 MUSIC tag giving details of the music listened to at that point in time.

-The location information is captured by Moves app (https://www.moves-app.com/), and they represent to semantic locations (Home, Work, DCU Computing building, GYM, Name of a Store, etc…), or to landmark locations registered by Moves. This tag can contain information in several languages. For locations that are not (HOME) or (WORK), the GPS locations are provided.

Submission instructions


As soon as the submission is open, you will find a “Create Submission” button on this page (just next to the tabs)


A submitted run for the ADLT sub-task must be in the form of a CSV file in the following format:

[topic id, number of times, number of minutes]

Where: - topic id: Number of the queried topic, e.g., from 1 to 10 for the development set. - number of times, number of minutes: positive integer numbers.

Then comes the optional information in the following format, starts with a line of 6 asterisk characters ‘**’, and the optional information:

[topic id, time id, image id]

Where: - topic id: Number of the queried topic, e.g., from 1 to 10 for the development set. - image id: ID of a relevant image. - time id: the occasion id for each time the ADL occurs, counted from 1.

Sample:

1, 3, 300

2, 4, 25

3, 5, 200

10, 3, 400

******

1, 1, u1_2015-08-01_145314_1

1, 1, u1_2015-08-01_145345_2

1, 2, u1_2015-08-01_145531_1

… // should have about 300 lines for topic id 1, then 25 lines for topic id 2, and so on

Evaluation

Metrics

The final score is computed as the average of the times and minutes, as follows:

$ADL_{score} = \frac{1}{2} \left(max(0, 1 - \frac{abs(n - n_{gt})}{n_{gt}}) + max(0, 1 - \frac{abs(m - m_{gt})}{m_{gt}})\right)$

where $n, n_{gt}$ are the submitted and ground-truth values for how many times the events occurred, respectively, and $m, m_{gt}$ are the submitted and ground-truth values for how long (in minutes) the events happened, respectively.

Rules

Note: In order to participate in this challenge you have to sign an End User Agreement (EUA). You will find more information on the ‘Dataset’ tab.

ImageCLEF lab is part of the Conference and Labs of the Evaluation Forum: CLEF 2018. CLEF 2018 consists of independent peer-reviewed workshops on a broad range of challenges in the fields of multilingual and multimodal information access evaluation, and a set of benchmarking activities carried in various labs designed to test different aspects of mono and cross-language Information retrieval systems. More details about the conference can be found here .

Submitting a working note with the full description of the methods used in each run is mandatory. Any run that could not be reproduced thanks to its description in the working notes might be removed from the official publication of the results. Working notes are published within CEUR-WS proceedings, resulting in an assignment of an individual DOI (URN) and an indexing by many bibliography systems including DBLP. According to the CEUR-WS policies, a light review of the working notes will be conducted by ImageCLEF organizing committee to ensure quality. As an illustration, ImageCLEF 2017 working notes (task overviews and participant working notes) can be found within CLEF 2017 CEUR-WS proceedings.

Important

Participants of this challenge will automatically be registered at CLEF 2018. In order to be compliant with the CLEF registration requirements, please edit your profile by providing the following additional information:

  • First name

  • Last name

  • Affiliation

  • Address

  • City

  • Country

  • Regarding the username, please choose a name that represents your team.

This information will not be publicly visible and will be exclusively used to contact you and to send the registration data to CLEF, which is the main organizer of all CLEF labs

Participating as an individual (non affiliated) researcher

We welcome individual researchers, i.e. not affiliated to any institution, to participate. We kindly ask you to provide us with a motivation letter containing the following information:

  • the presentation of your most relevant research activities related to the task/tasks

  • your motivation for participating in the task/tasks and how you want to exploit the results

  • a list of the most relevant 5 publications (if applicable)

  • the link to your personal webpage

The motivation letter should be directly concatenated to the End User Agreement document or sent as a PDF file to bionescu at imag dot pub dot ro. The request will be analyzed by the ImageCLEF organizing committee. We reserve the right to refuse any applicants whose experience in the field is too narrow, and would therefore most likely prevent them from being able to finish the task/tasks.

Prizes

ImageCLEF 2018 is an evaluation campaign that is being organized as part of the CLEF initiative labs. The campaign offers several research tasks that welcome participation from teams around the world. The results of the campaign appear in the working notes proceedings, published by CEUR Workshop Proceedings (CEUR-WS.org). Selected contributions among the participants, will be invited for publication in the following year in the Springer Lecture Notes in Computer Science (LNCS) together with the annual lab overviews.

Resources

Recommanded Reading

[1] Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler, Giulia Boato, Liting Zhou, Cathal Gurrin, “Overview of ImageCLEFlifelog 2017: Lifelog Retrieval and Summarization” , CLEF2017 Working Notes, Dublin, Ireland, 2017, vol 1866.

[2] Cathal Gurrin, Xavier Giro-i-Nieto, Petia Radeva, Mariella Dimiccoli, Håvard Johansen, Hideo Joho, Vivek K Singh, “LTA 2016: The First Workshop on Lifelogging Tools and Applications” , ACM Multimedia, Amsterdam, The Netherlands, 2016.

[3] Cathal Gurrin, Xavier Giro-i-Nieto, Petia Radeva, Mariella Dimiccoli, Duc Tien Dang Nguyen, Hideo Joho, “LTA 2017: The Second Workshop on Lifelogging Tools and Applications” , ACM Multimedia, Mountain View, CA USA, 2017.

[4] Cathal Gurrin, Hideo Joho, Frank Hopfgartner, Liting Zhou, Rami Albatal, “Overview of NTCIR-12 Lifelog Task” , Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, Tokyo, Japan, 2016.

[5] Duc-Tien Dang-Nguyen, Luca Piras, Giorgio Giacinto, Giulia Boato, Francesco GB De Natale, “Multimodal Retrieval with Diversification and Relevance Feedback for Tourist Attraction Images” , ACM Transactions on Multimedia Computing, Communications, and Applications, vol 13, n° 4, 2017.

[6] Duc-Tien Dang-Nguyen, Luca Piras, Giorgio Giacinto, Giulia Boato, Francesco GB De Natale, “A hybrid approach for retrieving diverse social images of landmarks” , IEEE International Conference on Multimedia and Expo (ICME), Turin, Italy, 2015.

[7] Working notes of the 2015 MediaEval Retrieving Diverse Social Images task , CEUR-WS.org, Vol. 1436, ISSN: 1613-0073.

[8] B. Ionescu, A.L. Gînscă, B. Boteanu, M. Lupu, A. Popescu,H. Müller, “Div150Multi: A Social Image Retrieval Result Diversification Dataset with Multi-topic Queries” , ACM MMSys, Klagenfurt, Austria, 2016.

Helpful tools and resources

Contact us

We strongly encourage you to use the public channels mentioned above for communications between the participants and the organizers. In extreme cases, if there are any queries or comments that you would like to make using a private communication channel, then you can send us an email at :

  • Sharada Prasanna Mohanty: sharada.mohanty@epfl.ch
  • Duc-Tien Dang-Nguyen: duc-tien[DOT]dang-nguyen[AT]dcu[DOT]ie
  • Luca Piras: luca[DOT]piras[AT]diee[DOT]unica[DOT]it
  • Ivan Eggel: ivan[DOT]eggel[AT]hevs[DOT]ch

More information

You can find additional information on the challenge here: http://imageclef.org/2018/lifelog