PDD Neural Slam

Mission statement

Modified 2017-11-23 by Adrien Ali Taiga

Build a map from a duckie driving around the road autonomously that will be used for planning

Motto

Modified 2017-11-23 by Adrien Ali Taiga

À l’idiot sans mémoire tout lui paraît nouveau et miraculeux

Project scope

Modified 2017-11-23 by Adrien Ali Taiga

What is in scope

Train agent to learn to explore an entire map efficiently and leep a representation of the current knowledge of the map.

What is out of scope

Need to run live on the duckie. We do not use real images but assume we have a tile detector

Stakeholders

Transfer learning team - tbd

Problem statement

Modified 2017-11-23 by Adrien Ali Taiga

We want the to keep a representation of the current map in memory that could used for downstream task such as planning

Assumptions

Modified 2017-11-23 by Adrien Ali Taiga

• We work on the tile level, and do not worry about low-level control.
• We assume there are only two types of tiles, road or not road

Approach

Modified 2017-11-23 by Adrien Ali Taiga

Stage 0: Create an environment where we can a simulated agent

Stage 1: Train an agent using a reinforcement learning method paired with an external memory

Stage 2: Deploy on the real robot and see how the method perform

Stage 3: Use a decoder that can recover the knowledge of the map

Functionality provided

Modified 2017-11-23 by Adrien Ali Taiga

• A exploration policy
• A map when the exploration is done

Resources required / dependencies / costs

Modified 2017-11-23 by Adrien Ali Taiga

• GPUs to train our policy
• Chip to run the neural network policy
• A “tile predictor” to infer the tile type from an image
• A simulator to train the agent

Performance measurement

Modified 2017-11-23 by Adrien Ali Taiga

• Check whether or not the agent actually explores the whole map
• Compare the decoded map with the ground truth map

Functionality-resources trade-offs

Modified 2017-11-23 by Adrien Ali Taiga

• Robust obstacle detection (many filters,…) vs. computational efficiency
• Maximizing speed (e.g. controlles might want to do that) vs. motion blur

Modules

Modified 2017-11-23 by Adrien Ali Taiga

• A grid map environment to train the agent.
• A policy network with an external memory
• A decoder to decode the external memory into a map
• A tile detector ??

Interfaces

Modified 2017-11-23 by Adrien Ali Taiga

Simulator

• takes the map size as input and generate a environment with agent output

Policy network

• take the current position of the robot w.r.t its original position, reads/write to the memory and give a control action.

Decoder

• takes the internal memory as input and give the actual map as output.

Preliminary plan of deliverables

Modified 2017-11-23 by Adrien Ali Taiga

Specifications

Modified 2017-11-23 by Adrien Ali Taiga

No need to revise duckietown specifications

Software modules

Modified 2017-11-23 by Adrien Ali Taiga

• Pytorch / Tensorflow
• Grid world simulator

Timeline

Modified 2017-11-23 by Adrien Ali Taiga

• Build the environment to train the agent
• Implement the agent to be trained with an external memory for the exploration task
• Train the agent
• Check if the map an actually be decoded

Data collection

Modified 2017-11-23 by Adrien Ali Taiga

• No need for data collection

Data annotation

Modified 2017-11-23 by Adrien Ali Taiga

• No need data annotation

Relevant Duckietown resources to investigate

Duckietown simulator

Other relevant resources to investigate

• Neural SLAM: Jingwei Zhang, Lei Tai, Joschka Boedecker, Wolfram Burgard, Ming Liu
• Learning to Navigate in Complex Environments: Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J. Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, Dharshan Kumaran, Raia Hadsell
• Hindsight experience replay: Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, Wojciech Zaremba
• Neural Turing Machines: Alex Graves, Greg Wayne, Ivo Danihelka Differentiable neural computer Alex Graves et al (DeepMind)

Risk analysis

Modified 2017-11-23 by Adrien Ali Taiga

• Might have issues to scale
• RL agent might have stability issues paired with an external memory the whole system could be hard to train