VIZTA Time of Flight Dataset and Benchmark

VIZTA aims at developing innovative technologies in the field of optical sensors and laser sources for short to long-range 3D-imaging and to demonstrate their value in several key applications including automotive, security, smart buildings, mobile robotics for smart cities, and industry 4.0. The VIZTA project also includes the development of demonstrators for these applications.

DFKI provides two time-of-flight datasets within the scope of this project to facilitate the development and evaluation of algorithms for two of the key use cases which are in-cabin monitoring and smart building management. For both use cases the dataset provides annotated data to train and evaluate deep learning algorithms for the detection and segmentation of persons and objects in 2D and 3D space.

Our in-car cabin dataset TICaM shall facilitate the realization of a robust object and person detection on the vehicle front seats. In addition, ground truth annotation for person activity recognition is provided. These algorithms shall be the basis for the realization of various in-cabin monitoring functions comprising active and passive safety functions as well as comfort functions and advanced human-vehicle interfaces for future autonomous and driver-less vehicles.

Our BUIDING dataset provides additionally to the frame-wise person annotation in 2D and 3D also annotations of anomalies in the scenes. In this way the dataset shall facilitate the realization of highly accurate people counting as well as security functions like access control to critical areas and anomalous behavior detection.

More information on the project can be found here.

BUILDING DatasetTICaM

Publications

If you would like to use our TICaM dataset, please cite our publication.

@misc{katrolia2021ticam,
      title={TICaM: A Time-of-flight In-car Cabin Monitoring Dataset}, 
      author={Jigyasa Singh Katrolia and Bruno Mirbach and Ahmed El-Sherif and Hartmut Feld and Jason Rambach and Didier Stricker},
      year={2021},
      eprint={2103.11719},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

License

This work, all the datasets and benchmarks are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

This means that:

  • You must give appropriate credit.
  • You may not use the material for commercial purposes.
  • If you remix, transform or build upon the material, you must distribute your contributions under the same license as the original.

Acknowledgement

This work was partially funded within the Electronic Components and Systems for European Leadership (ECSEL) Joint Undertaking in collaboration with the European Union’s H2020 Framework Program and Federal Ministry of Education and Research of the Federal Republic of Germany (BMBF), under grant agreement 16ESE0424 / GA826600 (VIZTA).

News

24.02.2021: VIZTA Time-of-flight website is launched.

12.04.2021: 4 RGB videos and 222 corresponding RGB frames are removed from the real test dataset.

26.04.2021: ‘Infant’ and ‘Child’ are now their own classes in synthetic dataset. Earlier they were labeled as ‘Person’.

15.06.2021: 2D keypoint annotations added for synthetic imageset.