gradSLAM: Dense SLAM meets automatic differentiation

Krishna Murthy, Ganesh Iyer, Liam Paull
Robotics Institute, Carnegie Mellon University | Robotics and Embodied AI Lab, Université de Montréal, MILA [Accepted to ICRA 2020]

gradSLAM is a fully differentiable dense Simultaneous Localization and Mapping (SLAM) system. The central idea of gradslam is to construct computational graphs that can connect the various components of SLAM systems. By creating differentiable alternatives to traditionally non-differentiable components of SLAM such as optimization, odometry estimation, raycasting, and map fusion, we enable the backpropogation of error gradients from 3D map elements to sensor observations, such as 2D image pixels. Further, we implement differentiable variants of dense SLAM systems that operate on voxels, surfels, and pointclouds respectively. Therefore, we successfully demonstrate a method to denote the contribution of each pixel towards the eventual 3D reconstruction and hope that this can successfully encode spatial-understanding into future gradient-based learning architectures. [Project Page] [Paper]

CalibNet: Self-Supervised Extrinsic Calibration using 3D Spatial Transformer Networks

Ganesh Iyer, Karnik Ram, Krishna Murthy, K. Madhava Krishna
Robotics Research Center, IIIT-H [Accepted to IROS 2018]

CalibNet is a self-supervised deep network capable of automatically estimating the 6-DoF rigid body transformation for extrinsic calibration between a 3D LiDAR and a 2D camera in real-time, by maximizing the photometric and geometric consistency between the input images and point clouds. [Project Page] [Paper]

Geometric Consistency for Self-Supervised End-to-End Visual Odometry

Ganesh Iyer*, Krishna Murthy*, Gunshi Gupta, K. Madhava Krishna,
Liam Paull
Robotics Research Center, IIIT-H and Montreal Institute for Learning Algorithms, Université de Montréal [Accepted to CVPR-Workshop 2018]

We propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a self-supervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate [Project Page] [Paper]


Team R.A.M.S: Robust Aerial Manipulation System

Capstone Project | MBZIRC Challenge 2020, Carnegie Mellon University
Participated in the design and development of an aerial manipulation platform capable of recognizing objects and lifting targeted payloads upto 1.5kg using an onboard perception subsystem and visual servoing.
Check out our work! : [Project Page] | [Demos]

Independent Study: Researching Skill Models for Autonomous Food Preparation

Carnegie Mellon University, advised by Prof. Oliver Kroemer
We address the task of building a large-scale data collection pipeline for interacting with food objects of different properties (like vegetables, steak, and dough) in order to extend learning-based methods to food preparation skills. Tested on hardware (FRANKA EMIKA PANDA), and in simulation (NVIDIA FLeX simulation package), the framework enables the data-driven learning of transition models of various food objects to infer properties from a few interactions and to plan for long-horizon food preparation tasks.
Report: [Link] | Open Source Code: Coming soon

Learning Diverse Goal-Conditioned Policies for Frontier Selection in Navigation

Course Project: Special Topics - Deep Reinforcement Learning for Robotics, Carnegie Mellon University
In this work, we propose a hierarchical formulation to tackle the problem of learning based efficient exploration. We decompose the the task into two sub-problems: selecting the next best goal in the visible space, followed by efficiently navigating to this sub-goal in the partial map setting. We propose a global policy network that learns the selection of the next best goal(‘frontier’) in the observable space, followed by a local policy that learns low-level navigation conditioned on different goal embeddings in known environments
Report: [Link] | Open Source Code: Coming soon

DeepEvent-VO: Fusing Intensity Images and Event Streams for End-to-End Visual Odometry

Course Project: Robot Localization and Mapping, Carnegie Mellon University
Designed a recurrent convolutional network that fuses intensity and event based image feature streams to make continuous visual odometry predictions for high speed applications using event-based cameras.
Open Source Code: [Github Repository] | Report: [Link]

Stereo Pipeline: Deep Convolutional Network, Semi-Global Matching and Pointcloud Reconstruction

Swaayatt Robots, India
Implementation and Improvement of pipeline in Zbontar et. al. for fast Disparity Map Computation and point cloud reconstruction of output Depth Map.
Open Source Code: [Deep Network] | [Census Transform and Semi-Global Matching]

RGBD Facial Pose tracking for Advanced Driver Assistance Systems

Swaayatt Robots, India
Point Cloud Processing package for tracking the face pose and central axis of gaze for RGBD based Advanced Driver Assistance System. Alignment to standard model using 3D FPFH features for points and normals, Sampling Consensus and Iterative Closest Point.

Tracking and Segmentation of Vehicles for Annotation

Swaayatt Robots, India
Annotation system for vehicle detection data. Propogating selected keypoints and feature points within vehicle boundaries using multi-scale template matching and particle filters. Results in a scale-changing contour of vehicles to be segmented.

Telepresence Robot with Stereoscopic Vision

Final Year Project, TCET, Mumbai University
Small scale and inexpensive telepresence platform capable of streaming immersive 3D SBS live video feed (350x350 resolution, 40 fps). Base platform actuated using AtMega2560 and keyboard/console commands. Raspberry Pi with camera module receives axis-angle commands from smartphone to control a 2-DOF servo-gimbal.
Code: [Telepresence bot]

Grid Traversing Robots

eYantra Lab Setup Initiative, TCET, Mumbai University
1. Minesweeping Robot: Traverses a small grid to locate basic obstacles (mines) and display their co-ordinate locations after reaching the end point. Breadth First Search and Djikstras' Algorithms based traversal are demonstrated.

2. Warehouse Management Simulation: Implementation of a small scale automated supply chain using order picking algorithms. Objects are collected, sorted based on a requirement (eg. color) and then transported to the specified destination zone.

Developed on the Firebird-V AtMega2560 Robotics Research Platform, Nex Robotics, IIT-Bombay. Code: [Mine Localization] | [Warehouse Management]