Can we upload our own dataset?

[sandro272]

Do you have scripts available/any easy way to convert raw data to your processed dataset files . So that i can test your on my own dataset .

[svjan5]

Hi @sandro272,
By dataset you mean training dataset (wikipedia corpus) or evaluation data?

[sandro272]

@svjan5 Uh... I mean that because I want to use our own dataset, so can you provide a script or method that converts raw data into your processed data (eg voc2id.txt, etc.). Thank you！

[svjan5]

Ok, got it. Actually, I cannot give a script for that because it requires getting dependency parse of the text which requires Stanford CoreNLP. So, you first need to get a dependency parse of the text, then I think everything is quite straight forward. voc2id.txt will contain the mapping of tokens to their unique ids and data.txt contains listing of tokens and dependency parse edges for each sentence in the corpus. Let me know if you face any difficulty in the whole process.

[sandro272]

@svjan5 OK,thank you!