CalibNet is a self-supervised deep network capable of automatically estimating the 6-DoF rigid body transformation for extrinsic calibration between a 3D LiDAR and a 2D camera in real-time, by maximizing the photometric and geometric consistency between the input images and point clouds. [Project Page][Paper]
Geometric Consistency for Self-Supervised End-to-End Visual Odometry
We propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a self-supervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate
Stereo Pipeline: Deep Convolutional Network, Semi-Global Matching and Pointcloud Reconstruction
RGBD Facial Pose tracking for Advanced Driver Assistance Systems
Swaayatt Robots, India Point Cloud Processing package for tracking the face pose and central axis of gaze for RGBD based Advanced Driver
Assistance System. Alignment to standard model using 3D FPFH features for points and normals, Sampling Consensus and Iterative Closest Point.
Tracking and Segmentation of Vehicles for Annotation
Swaayatt Robots, India Annotation system for vehicle detection data. Propogating selected keypoints and feature points within vehicle boundaries using multi-scale template matching and particle filters. Results in a scale-changing contour of vehicles to be segmented.
Telepresence Robot with Stereoscopic Vision
Final Year Project, TCET, Mumbai University Small scale and inexpensive telepresence platform capable of streaming immersive 3D SBS live video feed (350x350 resolution, 40 fps). Base platform actuated using AtMega2560 and keyboard/console commands. Raspberry Pi with camera module receives axis-angle commands from smartphone to control a 2-DOF servo-gimbal.
Code: [Telepresence bot]
Grid Traversing Robots
eYantra Lab Setup Initiative, TCET, Mumbai University 1. Minesweeping Robot: Traverses a small grid to locate basic obstacles (mines) and display their co-ordinate locations after reaching the end point. Breadth First Search and Djikstras' Algorithms based traversal are demonstrated.
2. Warehouse Management Simulation: Implementation of a small scale automated supply chain using order picking algorithms. Objects are collected, sorted based on a requirement (eg. color) and then transported to the specified destination zone.