This pages provides more information about the data format and other technical details of the TIMo dataset. Go to the overview page if you want learn about what the TIMo dataset is and how to get access to it.

Directory Structure, File Naming Scheme and Annotation Format:


Anomaly Dataset

Directory Structure:
TIMo_ANOMALY_DATASET
│
├──── TILTED_VIEW
│     ├──── annotation
│     ├──── calibration
│     ├──── test
│     │     └──── [sequence_name]
│     │           ├──── IR
│     │           │     └──── [frame_name]
│     │           └──── Z
│     │                 └──── [frame_name]
│     └──── train
│           └──── [sequence_name]
│                 ├──── IR
│                 │     └──── [frame_name]
│                 └──── Z
│                       └──── [frame_name]
│
└──── TOP_DOWN_VIEW
      ├──── SCENE_1
      │     ├──── calibration
      │     └──── train
      │           └──── ...
      └──── SCENE_2
            ├──── annotation
            ├──── calibration
            ├──── test
            │     └──── ...
            └──── train
                  └──── ...
File Name Scheme:
Annotations:

Anomalous frame ranges are provided as a CSV file. If no anomaly occured in the sequence, both start and end are set to -1. Below is an example of how the CSV looks like.


Person Detection Dataset:

Directory Structure:

TIMo_PERSON_DETECTION_DATASET
│
├──── calibration.yml
├──── timo_complex_testing
│     ├──── timo_complex_testing_annotations
│     │     └──── [calibration_id]
│     │           └──── [sequence_name]
│     │                 ├──── boxes_2d.csv
│     │                 └──── boxes_3d.csv
│     ├──── timo_complex_testing_calibration
│     │     └──── calibration
│     │           └──── [calibration_id]
│     │                 └──── [sequence_name]
│     │                       └──── calibration.yml
│     ├──── timo_complex_testing_depth
│     │     └──── calibration
│     │           └──── [calibration_id]
│     │                 └──── [sequence_name]
│     │                       └──── [frame_name]
│     ├──── timo_complex_testing_ir
│     │     └──── calibration
│     │           └──── [calibration_id]
│     │                 └──── [sequence_name]
│     │                       └──── [frame_name]
│     └──── timo_complex_testing_masks
│           └──── calibration
│                 └──── [calibration_id]
│                       └──── [sequence_name]
│                             ├──── [classes_frame_name]
│                             └──── [instances_frame_name]
├──── timo_complex_training
│     └──── ...
├──── timo_testing
│     └──── ...
└──── timo_training
      └──── ...

File Name Scheme:

Annotations:

The 2D and 3D boxes are provided as CSV files. Examples of them are shown below.

Instance and class masks are provided as PNG files where pixel values are set to the respective instance/class ID. See below for a colored visualization.


List of Normal Choreographies

IDDescription
COF-DRINKPerson going to coffee machine, drinking coffee quickly, leaving
COF-HANDSPerson going in kitchen, washing hands
COF-KNIFEPerson going to kitchen with knife, cutting cake, leaving
COF-REFILLPerson going to coffee machine, refilling coffee or water in the machine
COF-TABLEPerson going to coffee machine, drink coffee while sitting at table, leaving
COF-TALKLike COF-DRINK, second person coming and discussion briefly with coffee-drinker, leaving scene
COF-TOGOPerson going to coffee machine, preparing coffee leaving then the scene with coffee
COF-TRASHPerson going in kitchen, dropping an object in the trash bin, leaving
CROSSAt least two persons walk in opposite direction and cross themselves in the scene, at normal walking speed
CROSS-12OPerson handling object to another person while crossing
CROSS2Same as CROSS but at a faster walking speed
CROSS-HNDSKTwo persons meet in the scene, shake hands, leave scene
CROSS-TALKTwo persons meet in the scene talk briefly, leave scene
CRWDAt least 6 persons come in the scene and stop, another person crosses the crowd, other persons leave again
EATPerson eating/drinking while sitting on table
PASSOne person enters the scene and stops, other people enter from the same direction and pass the first person at normal speed. Once those two persons are out of the scene the first one follow them
PASS-12OFirst person handling object to second person while being passed
PASS-1OSame test as PASS, but first person with an object
PASS-21OSecond person handling object to first person while passing
PASS-2OSame test as PASS, but second person with an object
PCKOne person enters the scene, stops and picks something on the ground (full bending to pick up your keys on the ground) and leave the scene
ROW132 Persons walking behind each other with half-a-meter separation through the scene, smaller person in front
ROW312 Persons walking behind each other with half-a-meter separation through the scene, taller person in front
ROWY132 Persons walking side by side with 30cm distance, smaller person left of tall person
ROWY312 Persons walking side by side with 30cm distance, smaller person right of tall person
RTRA person or a group of persons enter the scene from one direction and do a U-turn in the scene (leaving scene from the same point than entering) at normal walking speed
RUNSame as WALK but persons run across the scene
SN-PMA number of N persons waiting (standing) and M persons passing at normal speed
STPAt least one person walks in the scene, stops for few seconds and leaves the scene. Tests to be repeated with several objects
STPLAt least one person walks in the scene, stops, laces shoe and leaves the scene
TABLE-TALKTwo persons meet in the scene sit down at table, talk briefly, leave scene
UTBBend upper torso slightly forth/back/right/left while standing
WALKAt least one person walks through the entire scene at normal walking speed
WALK2Same as WALK but at a faster walking speed
CROSS-1OSame as CROSS, first person carrying an object
CROSS-1OSame as CROSS, first person carrying an object

List of Anomalous Choreographies

IDDescription
COF-ARGLike COF-DRINK, second person coming and start fighting, both leaving scene
COF-LBOPerson hiding object in kitchen
COF-NONSPerson going to coffee machine, makes nonsense, taking coffee machine away
COLLAPSEPerson collapsing (1. person entering empty scene, 2. person collapsing in FOV, 3 persons lying on ground and staying there)
CRAWLPerson crawling on the floor
CROSS-ARGLike COF-DRINK, second person coming and start fighting, both leaving scene
CROSS-SOSecond person snatches object from first person while crossing
FLOORPeople sitting or lying on floor
PASS-1LBOLike PASS, first person leaving object while another person is passing, at the end object is left alone in the scene
PASS-2LBOLike PASS, second person leaving object while passing first person, at the end object is left alone in the scene
PASS-SOLike PASS, person snatches object from first person while passing
RUN-LBOLeft behind object: A person runs through the scene with an object and drops the object in the FOV of the camera
RUN-THOThrown object: A person runs through the scene throwing an object and running away
RUN-WEAPPerson running and holding weapon/knife upwards to attack somebody
SPORTPerson exercising (jumping jack, push-ups …)
STAGGERPeople staggering/stumbling
STP-LBOPerson walking in the scene, stopping, then leaving the object behind
UMBTwo persons hiding behind an open umbrella while crossing the scene
WALK-LBOLeft behind object: a person walks through the scene with an object and drops the object in the FOV of the camera
WALK-THOThrown object: a person walks through the scene with an object and throws the object in the FOV of the camera
WALK-WEAPPerson walking and holding weapon/knife upwards to attack somebody