Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added handling for knex-migrator being executed in Ghost 2.0 #765

Merged
merged 3 commits into from
Jul 31, 2018

Conversation

kirrg001
Copy link
Contributor

@kirrg001 kirrg001 commented Jul 25, 2018

refs #759

This PR is not finished, please no code review. Feel free to add questions or concerns 👍

The current port polling logic to detect if a blog is up or down on start/restart is too simple and the error handling was not good, because we weren't able to receive the reason why a blog didn't start up. Using the http port to communicate with simple messages between the CLI and Ghost is impossible, because it's the http port (!). Browsers use the same transport layer.

Ghost 2.0.0 has more requirements. It executes knex-migrator internally and brings the blog into maintenance mode. That means the server starts already, but the blog is not ready or fully started.

We just change how we communicate. We don't connect to Ghost, Ghost has to connect to the CLI.

  • use net server for communication
  • do final manual testing on ubuntu
  • ensure we haven't break the local process manager
  • fix tests
  • optimise error message for the websockets

This change is backwards compatible, because the CLI will support v1 and v2 (latest and previous major).

@coveralls
Copy link

coveralls commented Jul 25, 2018

Coverage Status

Coverage decreased (-0.2%) to 99.826% when pulling 38f3489 on kirrg001:execute-km-on-v1 into c9cf6d3 on TryGhost:1.9.

// If you are already on ^2 or you update from ^1 to ^2, then skip the task.
if (semver.satisfies(instance.cliConfig.get('active-version'), '^2.0.0') ||
semver.satisfies(context.version, '^2.0.0')) {
tasks.splice(3, 1);

This comment was marked as abuse.

This comment was marked as abuse.

refs TryGhost#759

- We have moved the execution of knex-migrator into Ghost 2.0.0
- This commit will ensure we skip the db migration when you:
  - migrate from ^1 to ^2
  - you update from ^2 to ^2
@kirrg001
Copy link
Contributor Author

kirrg001 commented Jul 25, 2018

We currently support ghost setup migrate. Should we leave this functionality as is and make it possible to run the db migrations from outside if you really want to on Ghost 2.0? 🤔

@kirrg001
Copy link
Contributor Author

We currently support ghost setup migrate. Should we leave this functionality as is and make it possible to run the db migrations from outside if you really want to on Ghost 2.0? 🤔

It might be useful to re-execute the migrations via the CLI. I would keep the functionality as is.

@kirrg001 kirrg001 changed the base branch from master to 1.9 July 26, 2018 23:52
@kirrg001 kirrg001 force-pushed the execute-km-on-v1 branch 3 times, most recently from b927349 to 67a67e8 Compare July 27, 2018 12:40
…ling)

refs TryGhost#759

- 100% better error handling between Ghost and the CLI
- Ghost 2.0 executes knex-migrator
  - it will turn maintenance on if migrations need to be executed
  - the handling of receiving success or failure state requires a better communication
- Ghost will tell the CLI when it's ready by using an extension: write a socket url into the config and send the success/failure state
- this is much better than using the http socket to communicate, because
  - A) port polling connects to the http port, it's impossible to send simple messages over this transport layer
  - B) the code is much simpler, CLI opens a socket port and Ghost pushes a notification if the notification is available
  - C) we receive any error from Ghost - even if the http server wasn't started yet
- we don't communicate with Ghost, Ghost communicates with the CLI - this is so much simpler
- port polling for v1 blogs is untouched
refs TryGhost#759

- same as for `ghost update`
- do not run db migrations on `ghost install`
@vikaspotluri123
Copy link
Member

Working on testing it right now! I have 3 dev instances on a local server, but unfortunately, it's a Debian instance 😬 Everything else is above / beyond stack requirements, though 😄

@kirrg001
Copy link
Contributor Author

Great thanks . You need the 2.0 pre build to test the code with a 2.0 blog or upgrading from v1 to v2, will send to you.

@vikaspotluri123
Copy link
Member

  1. ✅ install Ghost 2.0 with no issues (baby steps!)
  2. ✅ upgrade Ghost 1.25.2 to Ghost 2.0 (zip) - first hit 502 while it started (which took 5-10 seconds), then said ghost is starting up, please wait a moment (which was fast), then ghost {something} (I don't remember exactly what!) and then it was live
  3. ❌ rollback from 2.0-zip to 1.52.2 - this might be because I had to update the CLI version to 1.9.0 for it to allow 2.0 to install 🤔 I'm not sure what I did but it works now 😮

Something that's happening all of a sudden is all of the test instances are said to be running in development, although there is no development config, and running cat .ghost-cli says "running": "production" 😕

Ghost 1.25.0 -> Ghost 2.0.0 update log
> ghost update --zip ../ghost2.zip  --verbose --force
 Running sudo command: systemctl is-active ghost_...
[15:44:26] Checking system Node.js version [started]
[15:44:26] Checking system Node.js version [completed]
[15:44:26] Ensuring user is not logged in as ghost user [started]
[15:44:26] Ensuring user is not logged in as ghost user [completed]
[15:44:26] Checking if logged in user is directory owner [started]
[15:44:26] Checking if logged in user is directory owner [completed]
[15:44:26] Checking current folder permissions [started]
[15:44:26] Checking current folder permissions [completed]
[15:44:26] Checking folder permissions [started]
[15:44:26] Checking folder permissions [completed]
[15:44:26] Checking file permissions [started]
[15:44:26] Checking file permissions [completed]
[15:44:26] Checking content folder ownership [started]
[15:44:26] Checking content folder ownership [completed]
[15:44:26] Checking memory availability [started]
[15:44:26] Checking memory availability [completed]
✔ Checking for available migrations
✔ Checking for latest Ghost version
[15:44:26] Downloading and updating Ghost [started]
[15:44:28] Downloading and updating Ghost to v2.0.0 [title changed]
[15:44:28] Extracting release from local zip [started]
[15:44:28] Extracting release from local zip [completed]
[15:44:28] Installing dependencies [started]
[15:44:29] → yarn install v1.7.0
[15:44:29] → [1/5] Validating package.json...
[15:44:29] → [2/5] Resolving packages...
[15:44:30] → [3/5] Fetching packages...
[15:44:31] → [4/5] Linking dependencies...
[15:44:35] → [5/5] Building fresh packages...
[15:44:38] → Done in 8.77s.
[15:44:38] Installing dependencies [completed]
[15:44:38] Downloading and updating Ghost to v2.0.0 [completed]
[15:44:38] Stopping Ghost [started]
Running sudo command: systemctl is-active ghost_...
Running sudo command: systemctl stop ghost_...
[15:44:38] Stopping Ghost [completed]
[15:44:38] Linking latest Ghost and recording versions [started]
[15:44:38] Linking latest Ghost and recording versions [completed]
[15:44:38] Restarting Ghost [started]
Running sudo command: systemctl is-active ghost_...
[15:44:38] Ensuring user is not logged in as ghost user [started]
[15:44:38] Ensuring user is not logged in as ghost user [completed]
[15:44:38] Checking if logged in user is directory owner [started]
[15:44:38] Checking if logged in user is directory owner [completed]
[15:44:38] Checking current folder permissions [started]
[15:44:38] Checking current folder permissions [completed]
[15:44:38] Validating config [started]
Running sudo command: systemctl is-active ghost_...
[15:44:38] Validating config [completed]
[15:44:38] Checking folder permissions [started]
[15:44:38] Checking folder permissions [completed]
[15:44:38] Checking file permissions [started]
[15:44:38] Checking file permissions [completed]
[15:44:38] Checking content folder ownership [started]
[15:44:38] Checking content folder ownership [completed]
Running sudo command: systemctl start ghost_...
[15:44:42] Restarting Ghost [completed]
[15:44:42] Removing old Ghost versions [started]
[15:44:42] Removing old Ghost versions [skipped]

@vikaspotluri123
Copy link
Member

I know this isn't related, but here's the SS

image

The first 3 are test instances that are idle,
second one is the one which I upgraded from 1.25.x to 2.0
third one is local, clean install of 2.0 from zip

@kirrg001
Copy link
Contributor Author

@vikaspotluri123 Thanks for testing 🙃

Something that's happening all of a sudden is all of the test instances are said to be running in development, although there is no development config, and running cat .ghost-cli says "running": "production" 😕

That is weird and should be unrelated to this branch.

@acburdine Did something change since we have released 1.8.1?

@kirrg001 kirrg001 changed the title [WIP] Added handling for knex-migrator being executed in Ghost 2.0 Added handling for knex-migrator being executed in Ghost 2.0 Jul 27, 2018
@kirrg001
Copy link
Contributor Author

FYI: I'll merge this PR on Monday into the 1.9 branch. I did much testing on production and locally. We have minimum 3 days (13th-17th August) to do another full QA.

@kirrg001 kirrg001 merged commit 06bc19c into TryGhost:1.9 Jul 31, 2018
kirrg001 added a commit that referenced this pull request Aug 2, 2018
refs #759

- We have moved the execution of knex-migrator into Ghost 2.0.0
- This commit will ensure we skip the db migration when you:
  - migrate from ^1 to ^2
  - you update from ^2 to ^2
  - when you install ^2
- Added net socket server for Ghost 2.0 (alternative to simple port polling)
- way better error handling between Ghost and the CLI
- Ghost 2.0 executes knex-migrator
  - it will turn maintenance on if migrations need to be executed
  - the handling of receiving success or failure state requires a better communication between the CLI and Ghost, because the blog stays in maintenance mode and runs the migrations in background
- Ghost will tell the CLI when it's ready by using an extension: write a socket url into the config and send the success/failure state
- this is much better than using the http socket to communicate, because
  - A) port polling connects to the http port, it's impossible to send simple messages over this transport layer
  - B) the code is much simpler, CLI opens a socket port and Ghost pushes a notification if the notification is available
  - C) we receive any error from Ghost - even if the http server wasn't started yet
- we don't communicate with Ghost, Ghost communicates with the CLI
- port polling for v1 blogs is untouched, still works as expected
- coverage has decreased a very little 0,2% - will try to add more tests when we merge the 1.9 branch into master
acburdine pushed a commit to acburdine/Ghost-CLI that referenced this pull request Aug 16, 2018
refs TryGhost#759

- We have moved the execution of knex-migrator into Ghost 2.0.0
- This commit will ensure we skip the db migration when you:
  - migrate from ^1 to ^2
  - you update from ^2 to ^2
  - when you install ^2
- Added net socket server for Ghost 2.0 (alternative to simple port polling)
- way better error handling between Ghost and the CLI
- Ghost 2.0 executes knex-migrator
  - it will turn maintenance on if migrations need to be executed
  - the handling of receiving success or failure state requires a better communication between the CLI and Ghost, because the blog stays in maintenance mode and runs the migrations in background
- Ghost will tell the CLI when it's ready by using an extension: write a socket url into the config and send the success/failure state
- this is much better than using the http socket to communicate, because
  - A) port polling connects to the http port, it's impossible to send simple messages over this transport layer
  - B) the code is much simpler, CLI opens a socket port and Ghost pushes a notification if the notification is available
  - C) we receive any error from Ghost - even if the http server wasn't started yet
- we don't communicate with Ghost, Ghost communicates with the CLI
- port polling for v1 blogs is untouched, still works as expected
- coverage has decreased a very little 0,2% - will try to add more tests when we merge the 1.9 branch into master
acburdine pushed a commit that referenced this pull request Aug 16, 2018
refs #759

- We have moved the execution of knex-migrator into Ghost 2.0.0
- This commit will ensure we skip the db migration when you:
  - migrate from ^1 to ^2
  - you update from ^2 to ^2
  - when you install ^2
- Added net socket server for Ghost 2.0 (alternative to simple port polling)
- way better error handling between Ghost and the CLI
- Ghost 2.0 executes knex-migrator
  - it will turn maintenance on if migrations need to be executed
  - the handling of receiving success or failure state requires a better communication between the CLI and Ghost, because the blog stays in maintenance mode and runs the migrations in background
- Ghost will tell the CLI when it's ready by using an extension: write a socket url into the config and send the success/failure state
- this is much better than using the http socket to communicate, because
  - A) port polling connects to the http port, it's impossible to send simple messages over this transport layer
  - B) the code is much simpler, CLI opens a socket port and Ghost pushes a notification if the notification is available
  - C) we receive any error from Ghost - even if the http server wasn't started yet
- we don't communicate with Ghost, Ghost communicates with the CLI
- port polling for v1 blogs is untouched, still works as expected
- coverage has decreased a very little 0,2% - will try to add more tests when we merge the 1.9 branch into master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants