As a content admin, I want scholarship records exported to github so that there is a publicly accessible, versioned copy of project data available for researchers.

[rlskoeser]

testing notes

test the following scenarios for exporting to CSV:

• test exporting a subset of sources (scholarship records) from django admin
• test exporting all sources from django admin
• test exporting a subset of footnotes from django admin
• test exporting all footnotes from django admin (likely slow because includes transcription content; please report if it finishes and if it's usable or not)

retesting notes

• confirm that footnotes with no transcription content don't display "None" in the content field
• confirm that opening the downloaded CSV in Excel automatically reads transcription content properly

dev notes

• re-implement existing tabular export for footnotes with new export class
• re-implement existing tabular export for sources with new export class
• check if revised source export resolves standing issue As a content editor, I want to view source URLs when I download the sources CSV in order to more easily find/update external sources. #278
• make public version available for including in data export manage command

[kseniaryzhova]

@rlskoeser so the footnote download did not have transcription content aside from the fact that transcriptions exist or not and the download was super quick (for all footnotes). Is the csv for footnotes supposed to have actual transcription content?

[rlskoeser]

@kseniaryzhova I should have probably given you links, just to make sure we were talking about the same things.

Were you testing the footnotes export from this page? https://test-geniza.cdh.princeton.edu/admin/footnotes/footnote/

It looks to me like it does include the text content of the transcription — happy to remove it if it's not useful / important to have here or to have in this format.

And the sources export should be tested from this page: https://test-geniza.cdh.princeton.edu/admin/footnotes/source/

[kseniaryzhova]

@rlskoeser I tried it again with the links you provided - sources look good, they have the URLs, etc. But I'm still not seeing the transcription content for footnotes.

[rlskoeser]

@kseniaryzhova so weird!

Would you try downloading the subset from this link and report back?
https://test-geniza.cdh.princeton.edu/admin/footnotes/footnote/?q=5454
select them and then use the action -> export selected to csv -> go

I'm getting None for content for the first one and transcription content for the second one. (We need to fix the None at least, but I'm still not sure if we should have the content here or not – if it continues to not work for you maybe we should drop it.)

I tried exporting all footnotes and I think it failed; it was definitely slow. So that slowness might be enough reason to drop the content from this export, since we have the content in the annotation backup repo already.

[rlskoeser]

discussed with @kseniaryzhova and finally figured out what's going on here:

• Excel limits rows to one line by default, so it's hard to see if there's any content (if anything you probably only see an english language label like recto or verso)
• Excel is not reading as UTF-8, so transcription content is garbage

changes needed for this to be acceptable:
— should not display None when there is no content
— should have byte order mark so that Excel will automatically read as unicode

[kseniaryzhova]

@rlskoeser works, closing!