activated
Home
build details

Modified 2018-06-24 by Andrea Censi


This section focuses on the infrastructure and background of the embodied individual robotic tasks as outlined in Section 1.3 - Overview of tasks.

For examples of Duckiebot driving see a set of demo videos of Duckiebots driving in Duckietown.

The actual embodied tasks will be described in more detail in Unit A-3 - Task: Lane following (LF), Unit A-4 - Task: Lane following + Dynamic vehicles (LFV), Unit A-5 - Task: Navigation + Dynamic vehicles (NAV). Note that the sequence tasks was chosen to gradually increase the difficulty of tasks by extending previous task solutions to more general situations.

## Performance metrics

Modified 2018-06-17 by Andrea Censi

Measuring performance in robotics is less clear cut and more multidimensional than traditionally encountered in machine learning settings. To nevertheless achieve reliable performance estimates, we define $N$ to be the number of experiments. Let $\objective$ denote our objective or cost function to optimize which we report for every experiment.

In the following we will summarize the objectives used to quantify how well an embodied task is completed. We will evaluate performance in three different categories: performance objective, traffic law objective and comfort objective. Please note that the three provided objectives are not merged into one number.

### Performance objective

Modified 2018-06-17 by Andrea Censi

As a performance indicator for the lane following task and the lane following task with other dynamic vehicles, we choose the average speed $v_t$ over time of the Duckiebot as measured across the moved distance (lap time). This both encourages faster driving as well as algorithms with lower latency. The factors $\kappa$ and $\rho$ are multiplied to be able to weigh the performance indicators relative to other metrics.

$$\objective_{PS} = \expectation_t[-\kappa v_t]$$

Similarly, for the navigation with dynamic vehicles task, we choose the average speed to go from point $A$ to point $B$ within Duckietown as performance indicator.

$$\objective_{SAB}(t) = \expectation_{A,B}[T_{A \to B}]$$

### Traffic law objective

Modified 2018-06-17 by Andrea Censi

The following are a list of rule objectives the Duckiebots are supposed to abide by within Duckietown. All individual rule violations will be summarized in one overall traffic law objective $\objective_{TL}$.

The Duckietown traffic laws say:

The vehicle should stay at all times in the right lane, and ideally near the center.

We quantify this as follows: Suppose that $d$ is the absolute perpendicular distance of the body from the midline of the lane, such that $d=0$ corresponds to the robot at the center. While $d$ stays within an acceptable range no cost is incurred. When the small safety margin $d_{safety}$ is violated cost starts accruing proportional to the squared distance $d$ up to an upper bound $d_{max}$. If even this bound is violated a lump penalty $\alpha$ is incurred.

So we define the “stay-in-lane” cost function as follows:

$$\Delta \objective_{LF} = \begin{cases} 0 & \text{if } d \lt{} d_{safety} \\ \beta d^2 & \text{if } d_{safety} \leq d \leq d_{max} \\ \alpha & \text{if } d \gt{} d_{max} \text{ or if d is not within field-of-view anymore} \end{cases}$$

#### Quantification of “Stopping at red intersection line”

The Duckietown traffic laws say:

Every time the vehicle arrives at an intersection with a red stop line, the vehicle should come to a complete stop, before continuing.

During each intersection traversal, the vehicle gets a penalty $\gamma$ if there was not a time $t$ such that the vehicle was in the stopping zone (defined as being between $0$ and $5$ cm of the stop line) and $v_t = 0$, where $v_t$ is the longitudinal velocity of the vehicle with respect to the direction of the lane. The condition that the position $p$ of the Duckiebot is in the stopping zone is denoted with $p_{bot} \in S_{zone}$.

$$\objective_{SI} = \begin{cases} 0 & \text{if } p_{bot} \notin S_{zone}\\ \gamma & \text{if } \nexists t \text{ s.t. } v_t=0 \text{ and } p_{bot} \in S_{zone}\\ 0 & \text{if } p_{bot} \in S_{zone} \text{ and } \exists v_t=0 \end{cases}$$

To measure this cost, the velocities $v_t$ are saved while the robot is in the stopping zone $S_{zone}$.

The Duckietown traffic laws say:

The vehicle should stay at an adequate distance from vehicles in front of it at all times.

We quantify this rule as follows: Let $d$ denote the distance between the Duckiebot and the closest Duckiebot in front of it which is also in the same lane. Furthermore let $d_{safe}$ denote a cut-off distance after which a Duckiebot is deemed “far away” and let $W \times H$ be the dimensions of the Duckiebot. Let $\delta$ denote a scalar positive weighting factor. Then $$\objective_{SD} = \delta \max(0, d_{safe}-d)^2.$$

The Duckietown traffic laws say:

At any time a vehicle shall not collide with another object or vehicle.

The vehicle gets a penalty $\nu$ if within a time interval in time of length $T$: $t \in [t, t+T]$, the distance $d$ between the vehicle and a nearby object or other vehicle is zero or near zero at any point in the interval (as measured by being greater than $\epsilon$). For each time interval $T$ the collision cost is:

$$\objective_{AC} = \begin{cases} 0 & \text{if } d \gt{} \epsilon, \\ \nu & \text{if } d \lt{} \epsilon. \end{cases}$$

Time intervals are chosen to allow for maneuvering after collisions without incurring further costs.

An illustration of a collision is displayed in Figure 4.4b - .

The law says:

Every time the vehicle arrives at an intersection with a lane joining on the right, it needs to check whether there are vehicles on the right-hand joining lane. If so these vehicles shall traverse the intersection first.

$$\objective_{YR} = \begin{cases} 0 & \text{driving while right hand side if free at intersection} \\ \mu & \text{driving while right hand side if occupied at intersection} \end{cases}$$

The yield situation at an intersection is depicted in Figure 2.14d - yield.

To account for the relative importance of rules, the factors $\alpha, \beta, \gamma, \delta, \nu, \mu$ of the introduced rules will be weighted relatively to each other.

We hereby propose the following order of rule importance:

$$\objective_{AC} \gt{} \objective_{SI} \gt{} \objective_{YR} \gt{} \objective_{SD} \gt{} \objective_{SL}$$

Put into words:

Collision avoidance $\gt{}$ stop sign line $\gt{}$ Yielding $\gt{}$ Respecting distances $\gt{}$ Staying in the lane.

This puts constraints on the factors $\alpha, \beta, \gamma, \delta, \nu, \mu$ whose exact values will be determined empirically to enforce this relative importance.

While the infractions of individual rules will be reported, as a performance indicator all rule violations are merged into one overall traffic law objective $\objective_{TL}$. Let $\task$ denote a particular task, then the rule violation objective is the sum of all individual rule violations $\objective_i$ which are an element of that particular task.

$$\objective_{TL} = \sum_i \mathbb{I}_{\objective_i \in \task} \objective_i,$$

where $\mathbb{I}_{\objective_i \in \task}$ is the indicator function that is $1$ if a rule belongs to the task and $0$ otherwise.

### Comfort metric

Modified 2018-06-17 by Andrea Censi

In the single robot setting, we would like to encourage “comfortable” solutions. In this case this refers to penalizing large accelerations to achieve smoother driving solutions. To this end, changes of the input commands are penalized.

As a comfort metric, we measure the average absolute changes in input commands $u_k$ over time.

$$\objective_{CM} = \expectation[ ||\Delta u_k ||_1] \approx \frac{1}{M} \sum_k^M |\Delta u_k|,$$

where $M$ denotes the number of time steps.

No questions found. You can ask a question on the website.