CalibNet: Self-Supervised Extrinsic Calibration using 3D Spatial Transformer Networks

Ganesh Iyer, Karnik Ram, Krishna Murthy, K. Madhava Krishna
Robotics Research Center, IIIT-H [Accepted to IROS 2018]

CalibNet is a self-supervised deep network capable of automatically estimating the 6-DoF rigid body transformation for extrinsic calibration between a 3D LiDAR and a 2D camera in real-time, by maximizing the photometric and geometric consistency between the input images and point clouds. [Project Page] [Paper]

Geometric Consistency for Self-Supervised End-to-End Visual Odometry

Ganesh Iyer*, Krishna Murthy*, Gunshi Gupta, K. Madhava Krishna,
Liam Paull
Robotics Research Center, IIIT-H and Montreal Institute for Learning Algorithms, Université de Montréal [Accepted to CVPR-Workshop 2018]

We propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a self-supervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate [Project Page] [Paper]


Stereo Pipeline: Deep Convolutional Network, Semi-Global Matching and Pointcloud Reconstruction

Swaayatt Robots, India
Implementation and Improvement of pipeline in Zbontar et. al. for fast Disparity Map Computation. Point Cloud Reconstruction of output Depth Map.
Open Source Code: [Deep Network] | [Census Transform and Semi-Global Matching]

RGBD Facial Pose tracking for Advanced Driver Assistance Systems

Swaayatt Robots, India
Point Cloud Processing package for tracking the face pose and central axis of gaze for RGBD based Advanced Driver Assistance System. Alignment to standard model using 3D FPFH features for points and normals, Sampling Consensus and Iterative Closest Point.

Tracking and Segmentation of Vehicles for Annotation

Swaayatt Robots, India
Annotation system for vehicle detection data. Propogating selected keypoints and feature points within vehicle boundaries using multi-scale template matching and particle filters. Results in a scale-changing contour of vehicles to be segmented.

Telepresence Robot with Stereoscopic Vision

Final Year Project, TCET, Mumbai University
Small scale and inexpensive telepresence platform capable of streaming immersive 3D SBS live video feed (350x350 resolution, 40 fps). Base platform actuated using AtMega2560 and keyboard/console commands. Raspberry Pi with camera module receives axis-angle commands from smartphone to control a 2-DOF servo-gimbal.
Code: [Telepresence bot]

Grid Traversing Robots

eYantra Lab Setup Initiative, TCET, Mumbai University
1. Minesweeping Robot: Traverses a small grid to locate basic obstacles (mines) and display their co-ordinate locations after reaching the end point. Breadth First Search and Djikstras' Algorithms based traversal are demonstrated.

2. Warehouse Management Simulation: Implementation of a small scale automated supply chain using order picking algorithms. Objects are collected, sorted based on a requirement (eg. color) and then transported to the specified destination zone.

Developed on the Firebird-V AtMega2560 Robotics Research Platform, Nex Robotics, IIT-Bombay. Code: [Mine Localization] | [Warehouse Management]