gradSLAM: Dense SLAM meets automatic differentiation
Krishna Murthy, Ganesh Iyer, Liam Paull Robotics Institute, Carnegie Mellon University | Robotics and Embodied AI Lab, Université de Montréal, MILA [Accepted to ICRA 2020]
gradSLAM is a fully differentiable dense Simultaneous Localization and Mapping (SLAM) system. The central idea of gradslam is to construct computational graphs that can connect the various components of SLAM systems. By creating differentiable alternatives to traditionally non-differentiable components of SLAM such as optimization, odometry estimation, raycasting, and map fusion, we enable the backpropogation of error gradients from 3D map elements to sensor observations, such as 2D image pixels. Further, we implement differentiable variants of dense SLAM systems that operate on voxels, surfels, and pointclouds respectively. Therefore, we successfully demonstrate a method to denote the contribution of each pixel towards the eventual 3D reconstruction and hope that this can successfully encode spatial-understanding into future gradient-based learning architectures. [Project Page][Paper]
CalibNet: Self-Supervised Extrinsic Calibration using 3D Spatial Transformer Networks
CalibNet is a self-supervised deep network capable of automatically estimating the 6-DoF rigid body transformation for extrinsic calibration between a 3D LiDAR and a 2D camera in real-time, by maximizing the photometric and geometric consistency between the input images and point clouds. [Project Page][Paper]
Geometric Consistency for Self-Supervised End-to-End Visual Odometry
Ganesh Iyer*, Krishna Murthy*, Gunshi Gupta, K. Madhava Krishna, Liam Paull
Robotics Research Center, IIIT-H and Montreal Institute for Learning Algorithms, Université de Montréal [Accepted to CVPR-Workshop 2018]
We propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a self-supervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate
[Project Page][Paper]
Projects
Team R.A.M.S: Robust Aerial Manipulation System
Capstone Project | MBZIRC Challenge 2020, Carnegie Mellon University Participated in the design and development of an aerial manipulation platform capable of recognizing objects and lifting targeted payloads
upto 1.5kg using an onboard perception subsystem and visual servoing.
Check out our work! : [Project Page] | [Demos]
Independent Study: Researching Skill Models for Autonomous Food Preparation
Carnegie Mellon University, advised by Prof. Oliver Kroemer We address the task of building a large-scale data collection pipeline for interacting with food objects of different properties (like vegetables, steak, and dough) in order to extend learning-based methods to food preparation skills. Tested on hardware (FRANKA EMIKA PANDA), and in simulation (NVIDIA FLeX simulation package), the framework enables the data-driven learning of transition models of various food objects to infer properties from a few interactions and to plan for long-horizon food preparation tasks.
Report: [Link] | Open Source Code: Coming soon
Learning Diverse Goal-Conditioned Policies for Frontier Selection in Navigation
Course Project: Special Topics - Deep Reinforcement Learning for Robotics, Carnegie Mellon University In this work, we propose a hierarchical formulation to tackle the problem of learning based efficient exploration. We decompose the the task into two sub-problems: selecting the next best goal in the visible space, followed by efficiently navigating to this sub-goal in the partial map setting. We propose a global policy network that learns the selection of the next best goal(‘frontier’) in the observable space, followed by a local policy that learns low-level navigation conditioned on different goal embeddings in known environments
Report: [Link] | Open Source Code: Coming soon
DeepEvent-VO: Fusing Intensity Images and Event Streams for End-to-End Visual Odometry
Course Project: Robot Localization and Mapping, Carnegie Mellon University Designed a recurrent convolutional network that fuses intensity and event based image feature streams to make continuous visual odometry predictions for high speed applications using event-based cameras.
Open Source Code: [Github Repository] | Report: [Link]
Stereo Pipeline: Deep Convolutional Network, Semi-Global Matching and Pointcloud Reconstruction
RGBD Facial Pose tracking for Advanced Driver Assistance Systems
Swaayatt Robots, India Point Cloud Processing package for tracking the face pose and central axis of gaze for RGBD based Advanced Driver
Assistance System. Alignment to standard model using 3D FPFH features for points and normals, Sampling Consensus and Iterative Closest Point.
Tracking and Segmentation of Vehicles for Annotation
Swaayatt Robots, India Annotation system for vehicle detection data. Propogating selected keypoints and feature points within vehicle boundaries using multi-scale template matching and particle filters. Results in a scale-changing contour of vehicles to be segmented.
Telepresence Robot with Stereoscopic Vision
Final Year Project, TCET, Mumbai University Small scale and inexpensive telepresence platform capable of streaming immersive 3D SBS live video feed (350x350 resolution, 40 fps). Base platform actuated using AtMega2560 and keyboard/console commands. Raspberry Pi with camera module receives axis-angle commands from smartphone to control a 2-DOF servo-gimbal.
Code: [Telepresence bot]
Grid Traversing Robots
eYantra Lab Setup Initiative, TCET, Mumbai University 1. Minesweeping Robot: Traverses a small grid to locate basic obstacles (mines) and display their co-ordinate locations after reaching the end point. Breadth First Search and Djikstras' Algorithms based traversal are demonstrated.
2. Warehouse Management Simulation: Implementation of a small scale automated supply chain using order picking algorithms. Objects are collected, sorted based on a requirement (eg. color) and then transported to the specified destination zone.
