Support localized documents

[andrew2net]

The allrecords.xml contains translated documents. For example NIST CSWP 6 fre, NIST CSWP 6 ukr etc. These documents don't contain translated titles or abstracts, but identifiers and URLs are different. Should we merge these metadata into one relaton bibliographic item?

[ronaldtse]

Yes (create a language-independent entry) and no (allow language-specific entries).

We should allow citation of the language-independent "NIST CSWP 6", and also the language-specific "NIST CSWP eng", "NIST CSWP fre"...

Is the best mechanism to generate an additional entry for the "language independent version" if we see there are multiple language versions of the same entry?

[andrew2net]

Is the best mechanism to generate an additional entry for the "language independent version" if we see there are multiple language versions of the same entry?

I think it's better to have language independent entries in our dataset and generate language specific version by removing all language irrelevant metadata.