This pages provides more information about the data format and other technical details of the TIMo dataset. Go to the overview page if you want learn about what the TIMo dataset is and how to get access to it.
Directory Structure, File Naming Scheme and Annotation Format:
Anomaly Dataset
Directory Structure:
TIMo_ANOMALY_DATASET
│
├──── TILTED_VIEW
│ ├──── annotation
│ ├──── calibration
│ ├──── test
│ │ └──── [sequence_name]
│ │ ├──── IR
│ │ │ └──── [frame_name]
│ │ └──── Z
│ │ └──── [frame_name]
│ └──── train
│ └──── [sequence_name]
│ ├──── IR
│ │ └──── [frame_name]
│ └──── Z
│ └──── [frame_name]
│
└──── TOP_DOWN_VIEW
├──── SCENE_1
│ ├──── calibration
│ └──── train
│ └──── ...
└──── SCENE_2
├──── annotation
├──── calibration
├──── test
│ └──── ...
└──── train
└──── ...
File Name Scheme:
Annotations:
Anomalous frame ranges are provided as a CSV file. If no anomaly occured in the sequence, both start and end are set to -1. Below is an example of how the CSV looks like.
Person Detection Dataset:
Directory Structure:
TIMo_PERSON_DETECTION_DATASET
│
├──── calibration.yml
├──── timo_complex_testing
│ ├──── timo_complex_testing_annotations
│ │ └──── [calibration_id]
│ │ └──── [sequence_name]
│ │ ├──── boxes_2d.csv
│ │ └──── boxes_3d.csv
│ ├──── timo_complex_testing_calibration
│ │ └──── calibration
│ │ └──── [calibration_id]
│ │ └──── [sequence_name]
│ │ └──── calibration.yml
│ ├──── timo_complex_testing_depth
│ │ └──── calibration
│ │ └──── [calibration_id]
│ │ └──── [sequence_name]
│ │ └──── [frame_name]
│ ├──── timo_complex_testing_ir
│ │ └──── calibration
│ │ └──── [calibration_id]
│ │ └──── [sequence_name]
│ │ └──── [frame_name]
│ └──── timo_complex_testing_masks
│ └──── calibration
│ └──── [calibration_id]
│ └──── [sequence_name]
│ ├──── [classes_frame_name]
│ └──── [instances_frame_name]
├──── timo_complex_training
│ └──── ...
├──── timo_testing
│ └──── ...
└──── timo_training
└──── ...
File Name Scheme:
Annotations:
The 2D and 3D boxes are provided as CSV files. Examples of them are shown below.
Instance and class masks are provided as PNG files where pixel values are set to the respective instance/class ID. See below for a colored visualization.
Image Format
Both depth and infrared images are stored as 16 bit Portable Network Graphics (PNG). Note that especially the infrared images often only use little of the dynamic range and can thus appear black when opened with a normal image viewer. The visualizations here and in the paper use a logarithmic mapping to convert the images, but the dataset contains the raw data from the sensor.
List of Normal Choreographies
| ID | Description |
|---|
| COF-DRINK | Person going to coffee machine, drinking coffee quickly, leaving |
| COF-HANDS | Person going in kitchen, washing hands |
| COF-KNIFE | Person going to kitchen with knife, cutting cake, leaving |
| COF-REFILL | Person going to coffee machine, refilling coffee or water in the machine |
| COF-TABLE | Person going to coffee machine, drink coffee while sitting at table, leaving |
| COF-TALK | Like COF-DRINK, second person coming and discussion briefly with coffee-drinker, leaving scene |
| COF-TOGO | Person going to coffee machine, preparing coffee leaving then the scene with coffee |
| COF-TRASH | Person going in kitchen, dropping an object in the trash bin, leaving |
| CROSS | At least two persons walk in opposite direction and cross themselves in the scene, at normal walking speed |
| CROSS-12O | Person handling object to another person while crossing |
| CROSS2 | Same as CROSS but at a faster walking speed |
| CROSS-HNDSK | Two persons meet in the scene, shake hands, leave scene |
| CROSS-TALK | Two persons meet in the scene talk briefly, leave scene |
| CRWD | At least 6 persons come in the scene and stop, another person crosses the crowd, other persons leave again |
| EAT | Person eating/drinking while sitting on table |
| PASS | One person enters the scene and stops, other people enter from the same direction and pass the first person at normal speed. Once those two persons are out of the scene the first one follow them |
| PASS-12O | First person handling object to second person while being passed |
| PASS-1O | Same test as PASS, but first person with an object |
| PASS-21O | Second person handling object to first person while passing |
| PASS-2O | Same test as PASS, but second person with an object |
| PCK | One person enters the scene, stops and picks something on the ground (full bending to pick up your keys on the ground) and leave the scene |
| ROW13 | 2 Persons walking behind each other with half-a-meter separation through the scene, smaller person in front |
| ROW31 | 2 Persons walking behind each other with half-a-meter separation through the scene, taller person in front |
| ROWY13 | 2 Persons walking side by side with 30cm distance, smaller person left of tall person |
| ROWY31 | 2 Persons walking side by side with 30cm distance, smaller person right of tall person |
| RTR | A person or a group of persons enter the scene from one direction and do a U-turn in the scene (leaving scene from the same point than entering) at normal walking speed |
| RUN | Same as WALK but persons run across the scene |
| SN-PM | A number of N persons waiting (standing) and M persons passing at normal speed |
| STP | At least one person walks in the scene, stops for few seconds and leaves the scene. Tests to be repeated with several objects |
| STPL | At least one person walks in the scene, stops, laces shoe and leaves the scene |
| TABLE-TALK | Two persons meet in the scene sit down at table, talk briefly, leave scene |
| UTB | Bend upper torso slightly forth/back/right/left while standing |
| WALK | At least one person walks through the entire scene at normal walking speed |
| WALK2 | Same as WALK but at a faster walking speed |
| CROSS-1O | Same as CROSS, first person carrying an object |
| CROSS-1O | Same as CROSS, first person carrying an object |
List of Anomalous Choreographies
| ID | Description |
|---|
| COF-ARG | Like COF-DRINK, second person coming and start fighting, both leaving scene |
| COF-LBO | Person hiding object in kitchen |
| COF-NONS | Person going to coffee machine, makes nonsense, taking coffee machine away |
| COLLAPSE | Person collapsing (1. person entering empty scene, 2. person collapsing in FOV, 3 persons lying on ground and staying there) |
| CRAWL | Person crawling on the floor |
| CROSS-ARG | Like COF-DRINK, second person coming and start fighting, both leaving scene |
| CROSS-SO | Second person snatches object from first person while crossing |
| FLOOR | People sitting or lying on floor |
| PASS-1LBO | Like PASS, first person leaving object while another person is passing, at the end object is left alone in the scene |
| PASS-2LBO | Like PASS, second person leaving object while passing first person, at the end object is left alone in the scene |
| PASS-SO | Like PASS, person snatches object from first person while passing |
| RUN-LBO | Left behind object: A person runs through the scene with an object and drops the object in the FOV of the camera |
| RUN-THO | Thrown object: A person runs through the scene throwing an object and running away |
| RUN-WEAP | Person running and holding weapon/knife upwards to attack somebody |
| SPORT | Person exercising (jumping jack, push-ups …) |
| STAGGER | People staggering/stumbling |
| STP-LBO | Person walking in the scene, stopping, then leaving the object behind |
| UMB | Two persons hiding behind an open umbrella while crossing the scene |
| WALK-LBO | Left behind object: a person walks through the scene with an object and drops the object in the FOV of the camera |
| WALK-THO | Thrown object: a person walks through the scene with an object and throws the object in the FOV of the camera |
| WALK-WEAP | Person walking and holding weapon/knife upwards to attack somebody |
