The purpose of this toolbox is to advance the performance of global geometric registration algorithms in the context of scene reconstruction. The toolbox provides a platform for evaluating such algorithms. If you have a global geometric registration algorithm, please follow the instructions on this page and let us know if you are able to achieve high recall and precision. We are looking for efficient global registration algorithms with high recall. We are happy to collaborate. To date, the best global registration results we are aware of are obtained by our variant of Rusu et al., which achieves 59.2% recall and 19.6% precision on our benchmark dataset.
To evaluate your registration algorithm, run it on four sets of fragments in the following table.
| Scene | Fragments (PLY or PCD) | Evaluation File | Source | |
|---|---|---|---|---|
| Living Room 1 | PLY (390M) | PCD (394M) | ZIP (1M) | Dataset |
| Living Room 2 | PLY (332M) | PCD (336M) | ZIP (1M) | |
| Office 1 | PLY (282M) | PCD (286M) | ZIP (1M) | |
| Office 2 | PLY (330M) | PCD (334M) | ZIP (1M) | |
Each package contains a set of geometric fragments. They do not have an initial pose. For each set of fragments, your algorithm needs to extensively test each non-consecutive fragment pair $(\mathbf{P}_{i}, \mathbf{P}_{j})_{(i+1 \lt j)}$. If your algorithm decides they can be aligned, output an item to a log file. The item should consist of a three-number-metadata (i, j, total number of fragments) and a transformation matrix $\mathbf{T}_{ji}$ that aligns $\mathbf{P}_{j}$ to $\mathbf{P}_{i}$. An example output of a global registration algorithm is as follows:
4 6 57
0.931813 -0.151833 0.329652 1.2445
-0.0433474 0.855229 0.516434 -0.78035
-0.36034 -0.49551 0.790332 0.711526
0 -0 -0 1
7 10 57
-0.0657651 -0.0242835 0.99754 1.44585
-0.983248 0.171894 -0.0606383 3.72015
-0.169998 -0.984816 -0.0351812 3.10725
0 0 0 1
1 3 57
0.554312 0.236264 -0.798071 1.46312
-0.2289 0.965163 0.126745 0.205479
0.800214 0.112423 0.589083 -1.43515
-0 -0 -0 1
7 11 57
0.977749 -0.0522724 0.20316 -0.0895188
0.202217 -0.0228047 -0.979075 2.13087
0.0558116 0.998373 -0.011727 -1.28311
0 0 0 1
......Some facts that can be helpful:
Fragments do NOT have an initial pose. In other words, we are NOT looking for ICP variants that require an initial alignment. Instead, we seek global alignment methods that align fragments without any prior knowlege of the poses.
Fragments are constructed locally within a 3m × 3m × 3m volume at 6mm resolution.
Fragments are constructed from depth images that were synthesized with a noise model measured on a real PrimeSense camera.
Fragments that are barely overlapping are not considered "aligned" in the ground truth. To be more specific, you should not consider fragments that are aligned with less than 30% overlap as aligned.
Only non-consecutive fragments $(i+1 < j)$ should be tested. Fragment pairs not following this guideline will be discarded and will not affect the evaluation result.
To be maximally useful, the algorithm should not be too slow. The average running time for aligning two fragments using a single thread should ideally be less than 30 seconds.
Refer to the detailed description for all the details on fragment construction, parameters, and evaluation protocol.
For evaluation, download the toolbox from https://github.com/qianyizh/ElasticReconstruction/tree/master/Matlab_Toolbox. Then run the following Matlab script and get recall/precision numbers. Email us (Qianyi.Zhou@gmail.com) with your results.
gt = mrLoadLog( 'gt.log' );
gt_info = mrLoadInfo( 'gt.info' );
result = mrLoadLog( 'result.log' );
[ recall, precision ] = mrEvaluateRegistration( result, gt, gt_info );For a complete evaluation example including results for a few prior approaches, refer to: https://github.com/qianyizh/ElasticReconstruction/blob/master/Matlab_Toolbox/Example/demo_mrRegistrationEvaluation.m