build details

Show: section status errors & todos local changes recent changes last change in-page changes feedback controls

Overview and Motivation

Modified 2019-04-13 by Liam Paull

What are the AI Driving Olympics? The AI Driving Olympics (AIDO) are a set of robotics challenges designed to exemplify the unique characteristics of data science in the context of autonomous driving.

To understand how to solve a robotics challenge, we will explore the various dimensions of performance and difficulties involved.


Modified 2019-04-13 by Liam Paull

Many competitions exist in the robotics field.

One example is the long-running annual Robocup, originally thought for robot soccer (wheeled, quadruped, and biped), and later extended to other tasks (search and rescue, home assistance, etc.). Other impactful competitions are the DARPA Challenges, such as the DARPA Grand Challenges in 2007-8 that rekindled the field of self-driving cars, and the recent DARPA Robotics Challenge for humanoid robots.

In the field of robotics, these competitions are generally considered to be extremely useful to push the state of the art, and to make the field more reproducible by imposing exact and uniform experimentation environments.

In ML in general, and at NIPS in particular, there exist no competitions that involve physical robots. Yet, the interactive, embodied setting is thought to be an essential scenario to study intelligence. The only kind of intelligence we know, animal intelligence, evolved exactly for organisms to move in an environment and to interact with it.

The application to robotics is often stated as a motivation in recent ML literature (e.g., [1][2] among many others). However, the vast majority of these works only report on results in simulation and on very simple (usually grid-like) environments.

The importance of standardized benchmarks in ML research is well understood. Therefore, it is only a matter of time before a robotic benchmark will be developed. This is our attempt at suggesting such a benchmark.

The following elements are necessary and unique (with respect to other ML “offline” benchmarks) attributes of a robotics benchmark:

  • closed-loop, realtime interactive tasks;
  • complex tasks and environments;
  • competitive solutions require complex architectures;
  • very concrete resource constraints (computation, bandwidth, etc.);
  • multidimensional metrics (safety, performance, compliance, etc.).

We believe that these unique elements within robotics make it a unique and timely application of ML that requires further study.

Overview of challenges

Modified 2019-04-17 by Liam Paull

The AI-DO challenges

The AI Driving Olympics competition is structured into the following separate challenges:

Participants may submit code to each challenge individually. Challenges proposed in the AI Driving Olympics are ordered first by type and secondly by increasing difficulty in a way which encourages modular reuse of solutions to previous challenges.

No questions found. You can ask a question on the website.