Other dataset testing

[nanfangzhe]

Hello ，may I ask how to test and verify other data sets?

[jbehley]

The whole design of the dataset is that the unknowns are really unknown, such that it's not possible to "cheat" by looking at the unknown instances and simply tune the approach towards getting these instances right.

As we understand that some hyperparameter tuning or maybe ablation studies are needed, we decided to have to Codalab tracks for submissions: a validation track with max 100 submissions and a test track with max. 10 submissions. You can find the Codalab challenge at https://codalab.lisn.upsaclay.fr/competitions/2183

As with SemanticKITTI, validation is performed on Sequence 08 and the test is performed on Sequences 10-21.

For the submissions files, we use the SemanticKITTI format, where you have to provide instance ids for all identified instances. @nuneslu can maybe point you to the location in the implementation where these are generated as reference.

[nuneslu]

To convert the predictions into the competition format you can use this script