Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Database dump needs fixing #64

Open
thelovebug opened this issue Dec 28, 2017 · 17 comments
Open

Database dump needs fixing #64

thelovebug opened this issue Dec 28, 2017 · 17 comments
Assignees
@thelovebug
Labels
Bug Investigation

Comments

@thelovebug
Copy link
Contributor

thelovebug commented Dec 28, 2017 •
edited

The database dump option in /about#database is broken.

In the dump cchits.2018-09-11_0657.sql are the following SQL statement starters:

  • CREATE TABLE `applications` (
  • CREATE TABLE `artists` (
  • INSERT INTO artists VALUES
  • CREATE TABLE `chart` (

... and then nothing more.

At the same time, we need to make sure that the export is correct to the database structure.
@thelovebug thelovebug self-assigned this Dec 28, 2017
@computamike
Copy link
Contributor

Is this fixed - I just clicked on it and got a download (haven't tried running it yet)

@thelovebug
Copy link
Contributor Author

I can't remember why I logged this one initially. It certainly wasn't a good quality bug report!

@computamike
Copy link
Contributor

I think I've found the issue - I just performed a database dump and this is what the site returned :

('1516', '[\"Jamfolder\"]', '', '[\"http:\\/\
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>

</head>
<body>

<h1>Error Code 500 - Internal Server Error</h1>

Sorry, an error has occured which prevents us from delivering the requested page. Our team is working on this, and the site should hopefully be available soon. Thanks for your patience.
</body>
</html>

I believe this is probably a formatting issue in the API - possibly related to data within the Artist Record.

@computamike
Copy link
Contributor

Just retried - got 266kb download which seems complete no Error code 500 in this extract.

@computamike
Copy link
Contributor

I'd also point out that the database structure code (in the project) doesn't have an application table - maybe we need to re-export the data structure scripts? Should I raise an issue for this?

@thelovebug
Copy link
Contributor Author

I'd also point out that the database structure code (in the project) doesn't have an application table - maybe we need to re-export the data structure scripts? Should I raise an issue for this?

Yes please!

@thelovebug
Copy link
Contributor Author

I'd also point out that the database structure code (in the project) doesn't have an application table - maybe we need to re-export the data structure scripts? Should I raise an issue for this?

Actually no... hold fire on this... I think this might be related to something that @JonTheNiceGuy or @ymauray were working on. Guys, can you wade in on this?

@ymauray
Copy link
Contributor

ymauray commented Sep 12, 2018

The application is something that will be used by the new API to monitor usage.
Personnaly, I would put the database dump to the trash in favor of the API.
Database dumps lead to outdated data and would not reflect the quality we are trying to achieve with the site.
Just my 0.02 (insert currecy symbol here) thought, though.

@JonTheNiceGuy
Copy link
Contributor

So, whether the database dump is useful? I was inspired by the ur1.ca project (a link shortener run by Evan Prodromo of Identi.ca fame) to provide the full dump of the database, however, whether that is in-and-of-itself useful, I'm not sure - particularly if there is an API we can request. I'd like to be able to allow someone to pull the entire db though, if that's do-able?

Regarding the 500 error in the middle of that, I suspect we hit a timeout in Dreamhost-land, hence the error which can't be further replicated.

Should there be an application table? Is one referenced somewhere?

@computamike
Copy link
Contributor

computamike commented Sep 12, 2018 •
edited

Hi @JonTheNiceGuy -

So, whether the database dump is useful? I was inspired by the ur1.ca project (a link shortener run by Evan Prodromo of Identi.ca fame) to provide the full dump of the database, however, whether that is in-and-of-itself useful, I'm not sure - particularly if there is an API we can request. I'd like to be able to allow someone to pull the entire db though, if that's do-able?

The Database dump would be useful for me (or something similar) - I am setting up a local cchits server to have a play with phinx as a Database migration tool - thought it might be nice to set up some migrations, and then additionally some random seeds. But to do that I'd need a database that was stable.

Regarding the 500 error in the middle of that, I suspect we hit a timeout in Dreamhost-land, hence the error which can't be further replicated.

Maybe the export should be generated overnight, meaning that the export is just a file download, without the server overhead - maybe it could be scheduled to operate overnight?

Should there be an application table? Is one referenced somewhere?
There is an application table in the Database Extract. This table isn't in the db.mysql script

There does seem to be some differences between the structure of the database in the extract and the database in the script file :

In the Extract the tables exported are :
applications
artists
chart

In the db script, the following tables are created :
artists
chart
config
redirectmedia
shows
showtracks
tracks
trends
users
votes

@JonTheNiceGuy
Copy link
Contributor

The Database dump would be useful for me (or something similar) - I am setting up a local cchits server to have a play with phinx as a Database migration tool - thought it might be nice to set up some migrations, and then additionally some random seeds. But to do that I'd need a database that was stable.

Is this a one-off requirement? If so, I can probably do you a mysqldump and mail it to you :)

Maybe the export should be generated overnight, meaning that the export is just a file download, without the server overhead - maybe it could be scheduled to operate overnight?

The only downside to this is that I think you're one of the very few people who have actually looked at that dbexport for quite some time. As such, we'd be building and storing that DB extract for very little reason :(

There does seem to be some differences between the structure of the database in the extract and the database in the script file :

D'oh, OK, we need a separate issue which says "The dbexport is wrong" :)

@thelovebug
Copy link
Contributor Author

D'oh, OK, we need a separate issue which says "The dbexport is wrong" :)

That's kinda what this issue is, but worded slightly differently. I'll add this to the description above.

@JonTheNiceGuy
Copy link
Contributor

Well, not exactly - we have two things here (in my view - may be wrong!):

  1. The dbexport has issues, primarily with being built consistently (OP is that tables aren't right, later we have a 500 error in the middle of the code
  2. When it is right, the export is incomplete, because certain tables are not included.

I think this issue relates to 1. but we need a separate issue for 2.

Thoughts?

@thelovebug thelovebug mentioned this issue Sep 12, 2018
Open
@ymauray
Copy link
Contributor

ymauray commented Sep 12, 2018

Except, the "Application" is not in use at the moment. It's for the upcoming new API. So it doesn't need to be in the dump at the moment :-)

@computamike
Copy link
Contributor

Is this a one-off requirement? If so, I can probably do you a mysqldump and mail it to you :)

@JonTheNiceGuy If you could arrange a SQL dump that'd be cool thanks - email is : computa_mike@hotmail.com

@JonTheNiceGuy
Copy link
Contributor

Spotted error in the log

[12-Sep-2018 23:32:46 America/Los_Angeles] PHP Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future:
use mysqli or PDO instead in cchits.net/CLASSES/class_HTML.php on line 1323
[12-Sep-2018 23:32:50 America/Los_Angeles] PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 144410176 bytes)
in cchits.net/CLASSES/class_HTML.php on line 1339

@ymauray
Copy link
Contributor

ymauray commented Sep 13, 2018

On it :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thelovebug thelovebug

Labels
Bug Investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@JonTheNiceGuy @computamike @ymauray @thelovebug