-
Notifications
You must be signed in to change notification settings - Fork 11
2016 08 07 Meeting with Kumar @ Ba Le
Feihong, Kumar
I focused on these two questions:
- Would it make more sense to do the export of the NML file using the data in DJDB?
- Would it make more sense to do artist disambiguation using the data in DJDB?
Unfortunately, the answer to both of these questions is no. The GAE datastore used for DJDB is not a traditional database, so it has some unusual weaknesses. In particular, queries that ask for a lot of rows at once will time out before completion. That means it is not practical to get the full set of rows for any table on demand. For now, the master database has to be the one sitting on the filesystem of the Linux server.
I also asked him this question:
Besides the previous list of items above, what else does the import process do?
We didn't have time to go into any detail, but Kumar feels that there is enough additional logic in the import process that we wouldn't want to scrap it.
We should change the code that deals with the artist white list to use the local database rather than Github.
We should try to enhance the existing web-based app rather than write a new one from scratch.
The command-line program should be refactored so that it can do one album at a time (instead of all albums at once), so that it can give more immediate feedback.
I'll spend a bit more time trying to see why I can't successfully run chirpradio-manager on my own machine.
I looked it up and Google does offer hosted MySql databases. MySql is a relational database that would allow us to use DJDB as the canonical database. However, migrating from the datastore to a relational database is a pretty big change, and doesn't seem like a realistic goal in the short-term.