Home
Welcome to the ECAT wiki!
- Introduction
- Getting Started
- Capture
- Annotation
- Tracking
- Publications
- Versions tracking
ECAT is an open-source interface tool for annotating events and their participants in videos, capable of extracting the 3D positions and orientations of objects in video captured by Microsoft's Kinect- hardware. ECAT was created to address the lack of an annotation tool that can handle multimodal motion captures and allow multiple layers of semantic annotation on top of captures. ECAT can also be used as a general-purpose toolkit for the development of multimodal resources for machine learning tasks.
ECAT provides a tracking mechanism based on OpenCV implementation of a detection algorithm for ARUCO markers. Users can put different ARUCO markers on different sides of a block so that ECAT can calculate its pose/orientation change over time. In a nutshell, the algorithm looks for parallelograms (demarcated by strong contrast pattern of a black and white border) by searching through a gray image. Then it looks for checker pattern after transforming the parallelograms to squares. Searching over the whole image is carried out every 10 frames, whereas for remaining frames, the algorithm looks for space in the neighborhood of the previous frame.
ECAT provides mapping back and forth between 2-D boundary-based representation of objects, and 3-D boundary based representation. Currently, it can infer corner coordinates of cubes in 3-D from detected ARUCO markers. It can also infer equation of the geometrical plane of a flat surface (such as the supporting table).
Users can mark the appearance of an object using the object toolbox. Marking the boundary of objects in 2-D can be used to infer object boundary in 3-dimensional by using depth-field data.
Annotators may also mark spatial relations between objects. For example, to mark a block on top of a table, users can create a link between a block object and the table object, using the Link_to button on the object tracking panel, and specify the relation between the objects as “on,” resulting in the creation of a predicate ON(Block1,Table).
ECAT allows annotators to mark subevent relations between events.
The tool is free to use according to MIT license. Please site the following paper if you use it for your research.
Tuan Do, Nikhil Krishnaswamy, and James Pustejovsky. "ECAT: Event Capture Annotation Tool." Interoperability for Semantic Annotation (ISA) 2016, Portoroz, Slovenia. May 28, 2016.
Please use the issue tracker on the project repository for reporting bug or feedback.