This score results in a value in [0,1]. Good estimations are expected to have high area scores.

This score results in a value in [0,1]. Good estimations are expected to have high area scores. It is important to note that such score is not symmetric: estimations with larger triangle areas than references are less penalized than those with smaller triangle areas than references.

### Final score

...

...

@@ -108,7 +111,7 @@ The final score is finally given by,

S = \frac{1}{2} S_a(S_s + S_t) .

```

Note that challenge 1 is evaluated on specific regions of the body mesh. The head and hands are ignored, rest of the body is considered. In challenge 2, the full mesh is considered for evaluation.

Note that challenge 1 is evaluated on specific regions of the body mesh. The head and hands are ignored, rest of the body is considered. In this case, masks are computed on the reference and included in the computation of the scores. In challenge 2, the full mesh is considered for evaluation.

...

...

@@ -125,11 +128,94 @@ Notation:

The quality of the estimations, $`Y'`$ and $`E'`$, are evaluated quantitatively with respect to the ground-truth, $`Y`$ and $`E`$, respectively, using a metric evaluating:

- the shape reconstruction of the corrupted scans `X` with respect to CAD models `Y`.

- the recovery of the sharp edges with respect to ground-truth edges `E`.

- Shape reconstruction: the shape reconstruction of the corrupted scans `X` with respect to CAD models `Y`.

- Edge recovery: the recovery of the sharp edges with respect to ground-truth edges `E`.

### Shape reconstruction

The shape reconstruction is evaluated on $`Y'`$ with respect to the corresponding CAD model $`Y`$ using a similar formulation to SHARP [challenges 1 and 2](evaluation.md#quantitative-evaluation-of-challenges-12).

The first subscore ['surface-to-surface distances'](evaluation.md#1-surface-to-surface-distances) only involves the shape scores of the meshes ( $`Y'`$, $`Y`$) by computing ($`d_{ER}^{shape}`$, $`d_{RE}^{shape}`$) with the same manner described [here](evaluation.md#1-surface-to-surface-distances). <imgalign="right"height="100"src="data/points3-example.png">

In order to better reflect the shape reconstrution around the sharp edges on the shape score, additional points are sampled on the triangles nearby the sharp edges. An example of these additional points is shown in this figure:

The hit-rates [hit-rates](evaluation.md#2-surface-hit-rates) and the [surface area scores](evaluation.md#3-surface-area-scores) are also computed following a similar manner as described for [challenges 1 and 2](evaluation.md#quantitative-evaluation-of-challenges-12). The only difference consists of making the surface area scores more symmetric than in challenges 1 and 2 where the focus was on shape completion. In particular, the shape area score is computed as follows:

where $`\alpha`$ and $`\beta`$ are two parameters allowing the control of the symmetry of the score function. The values of these parameters were set to $`\alpha=2.6`$ and $`\beta = -0.6`$.

Finally, the three subscores are combined as described [here](evaluation.md#final-score) to obtain a final shape reconstuction score $`S_s`$.

Note that the ground-truth CAD models $`Y`$ might contain some inner-parts which are missing in the provided input scans $`X`$. These parts are not visible and cannot be scanned. Consequently, these parts are masked and not included in computing the shape scores. An example of these parts and the masked version of the CAD model is shown in this figure:

### Edge recovery

In addition to the shape reconstruction, the estimated sharp edges $`E'`$ are evaluated with respect to the ground-truth sharp edges $`E`$. The evaluation of the edges is conducted using two criterions:

#### 1. Edge-to-edge distances

Consist of two directed distances:

1. $`d_{ER}^E`$ is computed from the estimation to the reference

2. $`d_{RE}^E`$ is computed from the reference to the estimation.

The directed distance $`d_{AB}^E`$ between edges $`A`$ and $`B`$ is

approximated in practice by sampling points on $`A`$ and computing their distances to the nearest lines in edges $`B`$.

The points are sampled according to the length of the edges. An example of these points is given in this figure (top: sampled points; bottom: original edge lines):

For each sampled point in $`A`$, the euclidean distance to the nearest line is computed using a point-to-line distance calculation over all the edges of $`B`$. The distances are then averaged for all the points. This calculation is conducted in two directions to compute $`d_{ER}^E(Y',Y)`$ and $`d_{RE}^E(Y,Y')`$.

#### 2. Edge length scores

More details about the metric will be communicated soon.

Consists of a score that quantifies the similarity between the edge length of the estimation and that of the reference. The length of the estimated edges and the reference edges denoted as $`L_{E}`$ and $`L_{R}`$, respectively, are computed by summing over the lengths of the edges of each edge-set.

The edge length score is obtained using a similar strategy to [surface area scores](evaluation.md#L146).

where $`\alpha`$ and $`\beta`$ are two parameters that were set to 2.5 and -0.6, respectively. This score results in a value in [0,1]. Good estimations are expected to have high length scores.

Note: As mentioned in [challenge 3 description](challenge_3.md#challenge-3-recovery-of-feature-edges-in-object-scans), the ground-truth sharp edges were filtered to match the visble parts of the scans. In order to not penalize submissions that were able to estimate sharp edges in the missing parts of the input scans, the edge recovery scores are computed on the filtered edges and the original ones and the best scores are considered.

### Final score

Consists of a combination of the measures evaluating the shape reconstruction and the edge recovery as explained above.

The shape reconstruction score is computed as follows,

where $`\Phi_{k_i}(d) = e^{-k_id^2}`$ maps a distance $`d`$ to a score in [0,1]. The parameters $`k_i`$ are chosen according to some conducted baselines.

where $`\Phi_{k_i}(d) = e^{-k_id^2}`$ maps a distance $`d`$ to a score in [0,1]. The parameters $`k_i`$ are chosen according to some conducted baselines.