Skip to content
This repository has been archived by the owner on Jul 22, 2024. It is now read-only.

Is it possible to provide a uniform id between Leaderboard and D2A? #8

Closed
xshub opened this issue Feb 21, 2022 · 1 comment
Closed
Assignees

Comments

@xshub
Copy link

xshub commented Feb 21, 2022

Hi,
Thank you for your good work!
Is there a direct way to get the mapping between the id of the "trace" in Leaderboard Data and the id of the sample in the D2A dataset? e.g. "36270 " (an "id" of sample in trace task ) correspond to "openssl_78ad164cf891c8081d89d03a0c062090d0f10551_1" (an "id " in D2A dataset)

I noticed that the id in each task is unique. But, is it possible to provide a uniform id? especially for the "Trace" and "Trace+Code" tasks.

Thanks.

@saurabhpujar saurabhpujar self-assigned this Nov 3, 2022
@saurabhpujar
Copy link
Collaborator

saurabhpujar commented Nov 3, 2022

Thanks for using our dataset!

We deliberately have different ids so that there is no direct mapping between the main D2A dataset and the leaderboard tasks test set.
This is just a simple way to encourage participants to actually generate the labels for test set.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants