build details

# General Exercise Running Procedure

Modified 2021-10-31 by tanij

This page describes the exercises’ infrastructure. This infrastructure affords a seamless method to build on existing baselines, test them in simulation, test them on robot hardware either remotely or locally, and then evaluate and submit them as challenges with the AIDO challenges infrastructure.

A Duckiebot that is initialized

Laptop configured, according to Unit C-2 - Setup - Laptop.

That you are able to submit a challenge according to Unit C-3 - Make your first submission.

## Video Tutorial

Modified 2021-10-30 by Liam Paull

## Getting Started

Modified 2021-10-31 by Jacopo Tani

Fork the `mooc-exercices` repository and clone it onto your computer.

Set up an upstream remote. From inside the directory you just cloned:

``````laptop $git remote add upstream git@github.com:duckietown/mooc-exercises.git `````` Now to pull anything new from the original repository you can do: ``````laptop$ git pull upstream daffy
``````

Enter the `mooc-exercises` folder that you just cloned do:

``````laptop $cd mooc-exercises `````` In here you will see a number of folders. Each folder corresponds to an exercise. ## The Anatomy of an Exercise Modified 2021-04-03 by Liam Paull Exercises should contain all of the following: ### `config.yaml` Modified 2021-04-09 by tanij This contains information about the exercise. Example: ``````# Exercise configuration file agent_base: "duckietown_baseline" # the agent base image to use ws_dir: "exercise_ws" # directory that contains the code agent_run_cmd: "run_all.sh" # the script in "launchers" to run the agent with notebooks: - notebook: package_name: "encoder_pose" name: "odometry_activity" - notebook: package_name: "lane_controller" name: "control_activity" `````` The `agent_base` indicates which image to use as a baseline to build from. The mappings are listed here. Many of the existing exercises are build on the `duckietown_baseline` image which contains all of the code in the `dt-core` repository. In the case that you are using the `duckietown_baseline`, any package/node that you create in the `exercise_ws` directory will be run instead of the one in the `dt-core` repository if the package name and node name match. This is achieved through workspace overlaying. The `ws_dir` indicates the name of the subdirectory that contains the code that should be mounted into the image. The `agent_run_cmd` indicates the command that should be run when the container is run to start things. The `notebooks` contains the list of the notebooks that have to be converted to python scripts. For each notebook must be specified the name of the notebook `name`, and the name of the package where the generated script has to be copied `package_name`. Multiple notebooks can be listed. ### `exercise_ws` Modified 2021-04-09 by tanij As indicated above, the `exercise_ws` directory is where the code should go. For the case of ROS packages, they should go inside a `src` directory inside `exercise_ws`. ### `assets` Modified 2021-04-09 by tanij The `assets` folder contains two subfolders, `setup` and `calibrations`. • The `setup` subfolder contains all the configuration information (mostly in terms of environment variables) that are needed to run the various docker images that we are running (which depends on the configuration). • The `calibrations` folder contains robot calibrations with a similar directory structure as is on the Duckiebot. ### `launchers` Modified 2021-04-09 by tanij The `launchers` folder contains scripts that can be run by the agent. Specifically, the one that is indicated in the `config.yaml` file will be run by default when the agent container is run or when your exercise is submitted through the challenges infrastructure. You can specify different launchers to run depending on whether you are testing/developing with the exercises infrastructure or submitting through the challenges infrastructure. ### `notebooks` Modified 2021-04-09 by tanij The `notebooks` folder contains pedagogical notebooks that can be run. Some parts of the notebooks indeed are dedicated to tests, to check if the code is working properly before starting the simulation or testing on the Duckiebot. Before running the test cells make sure you run also the cells with the code. There are different ways to run a cell: 1. click on the `play` button on the top left of the cell. 2. pressing `crtl`+`enter`. The code in the notebooks can also be compiled and become accessible inside the code in the `exercise_ws` directory. In order to do so, from inside the exercise folder run: ``````laptop$ dts exercises build
``````

This command convert the notebook into a python script and place it inside the package in `exercise_ws` directory specified in the `config.yaml` file.

The same is when running the `run` command, with the difference that in this case the ROS workspace is not built:

``````laptop $dts exercises run options `````` ### `requirements.txt` Modified 2021-04-09 by tanij The `requirements.txt` file contains any specific python requirements that you need for your submission. Note that these are requirements need over and above the base image. ### `Dockerfile` Modified 2021-04-09 by tanij The `Dockerfile` contains the recipe for making your submission. In the normal case, this is relatively straightforward. We install the requirements, copy in the code and run a `launcher`. ## The Exercises API Modified 2021-04-09 by tanij In the following we will describe the current commands that are supported within `dts exercises` and how they are used. ### Building your code Modified 2021-04-09 by tanij You can start by building your code with: ``````laptop$ dts exercises build
``````

If you go inside the `exercises_ws` folder you will notice that there are more folders that weren’t there before. These are build artifacts that persist from the building procedure because of mounting.

every time you run a `dts exercises` command you have to be inside an exercise folder or you will see an error.

Modified 2021-04-03 by Liam Paull

With `dts exercises test` you can test your agent:

1. in the simulated environment,
2. on your robot but with the agent code running on your laptop,
3. with all of the code running on your robot.

#### Running in Simulation

You can run your current solution in the gym simulator with:

``````laptop $dts exercises test --sim `````` Then you can look at what’s happening by looking through the “novnc” browser at http://localhost:8087 . If you are running an exercise with a ROS-based baseline, you can use all of the existing ROS tools from this browser desktop. For example, open up the `rqt_image_view`, resize it, and choose `/agent/camera_node/image/compressed` in the dropdown. You should see the image from the robot in the simulator. You might want to launch a virtual joystick by opening a terminal and doing: ``````laptop$ dt-launcher-joystick
``````

If you are running the `duckietown_baseline`, by default the duckiebot is in joystick control mode, so you can freely drive it around. You can also set it to `LANE FOLLOWING` mode by pushing the `a` button when you have the virtual joystick active. If you do so you will see the robot move forward slowly and never turn.

You might also explore the other outputs that you can look at in `rqt_image_view`.

Also useful are some debugging outputs that are published and visualized in `RViz`. You can open `RViz` through the terminal in the `novnc` desktop by typing:

``````laptop $rviz `````` In the window that opens click “Add” the switch to the topic tab, then find the `segment_markers`, and you should see the projected segments appear. Do the same for the `pose_markers`. Another tool that may be useful is `rqt_plot` which also can be opened through the terminal in novnc. This opens a window where you can add “Topics” in the text box at the top left and then you will see the data get plotted live. All of this data can be viewed as data through the command line also. Take a look at all of the `rostopic` command line utilities. TODO: add pictures. previous task next (2 of 10) index task The following was marked as "todo". TODO: add pictures. Location not known more precisely. Created by function `n/a` in module `n/a`. #### Testing Your agent on the Robot If you are using a Linux laptop, you have two options, local (i.e., on your laptop) and remote (i.e., on the Duckiebot). If you are Mac user stick to the remote option. To run “locally” ``````laptop$ dts exercises test --duckiebot_name ROBOT_NAME --local
``````

To run on the Duckiebot:

``````laptop $dts exercises test --duckiebot_name ROBOT_NAME `````` In both cases you should still be able to look at things through novnc by pointing your browser to http://localhost:8087 . If you are running on Linux, you can load up the virtual joystick and start lane following as above. #### Interactive Mode You may find it annoying to completely shut down all of the running images and restart them to make a simple change to your code. To make things faster, you can use the `--interactive` flag with `dts exercises test`. In this case, when all of the containers other than the agent have started, you will be given a command line inside the agent container (overriding the comand specified in `config.yaml`. From here you can run your launcher from the command line manually. If you are running an exercise based on the `duckeitown_baseline` image, the first time you will have to start the “interface” part of the agent. To do this run ``````laptop container$ launchers/run_interface.sh.
``````

You will see some output of some ros nodes starting. At the end, if you push ENTER you will get your command line back. Then you can run the lane_following demo using your lane_controller by running

``````laptop container launchers/run_agent.sh
``````

You can do the normal thing of going to novnc and putting it into lane following mode or driving around with the joystick or whatever.

If you would like to change your code and re-run, just edit your code on your laptop, and then go to that terminal and do CTRL-C. You will see everything start to shut down. Then you can simply rerun the agent and it will have the new code that you just modified since it’s mounted into the agent container. So just do `launchers/run_agent.sh` again and it will start up again.

You will see an output from the anti-instagram node saying it’s waiting for the first image. Don’t worry, if you go to novnc and put the agent in lane following mode or drive with the joystick, it will start to receive images and that output will go away

There is a timeout on the simulator, so if you do CTRL-C and then spend a while editing your code, it’s likely that the simulator will have shut down. So either leave it running while you edit your code or just restart everything. You can get out of your terminal by typing

``````laptop container \$ exit
``````