Provide functionality for use case regarding exporting all data for import into a FindMyCourt lookup database (e.g., CSV export)

[bguayante]

No description provided.

[bguayante]

World Wide @JDziurlaj

This is the issue associated with what we were discussing earlier today. Contrary to the Issue title, we are scrapping the csv export but still need to output a json file from the RDF store. I tagged you in a comment to @jungshadow, but it might be better to create a new .py file for the export rather than adding it to the existing script.

In either case, the import script is located in scripts and is named db_importer.py. Let me know if you have any questions.

[bguayante]

Also indev is the most current branch. The scripts in main are mostly the same, but indev is where all the action is.

[bguayante]

@JDziurlaj I've put everything together but your db_importer.py script hangs forever when I run it. When I installed the updated Pipfile I got an error that said Ignoring importlib-metadata: markers 'python_version < "3.10"' don't match your environment. I'm pretty sure it's the library related to oxigraph. I'm on Python 3.10 or 3.11 and I remember that you were around 3.6 a few weeks ago. I tried to cheese the Pipfile by changing the < to <= and was able to install everything but the script still hung. I tried both the pre-merge feature/json-ld branch and the merged indev branch. Do you mind comparing what's on your local machine to the db_exporter script that's here on GH?

[JDziurlaj]

I pulled the latest indev from GitHub, and even installed Python 3.10.8, and encountered no issues, other than one with relative paths in db_exporter.py (pull latest). Keep in mind I am running in WSL/Debian.

[bguayante]

@jungshadow Do you mind pulling the most recent indev and running the exporter script? You might need to run the pipeline first. @JDziurlaj reports it's working with WSL/Debian but I can't get the same code to run on macOS. Do you mind verifying if it's an OS issues?

[bguayante]

We sorted this out. It was an issue with using duplicate data in the db. The export probably would work with duplicates, but it would take a very, very long time. I created additional test data to eliminate duplicates and all works as expected.