Can't import my backup #8365

pascalandy · 2017-04-20T01:06:18Z

Issue Summary

Can’t import my backup.

Steps to Reproduce

Export
Delete all content
Import the file
“Import failed” error. I see also “
Attempting to import data linked to unknown user id 2”

Screenshot: https://cl.ly/0r0n3u2u3u0S

I did this in the past and my backups were imported successfully.

Technical details:

Ghost Version: 0.11.8
Node Version: 4.8.1
Browser/OS: Chrome Version 57.0.2987.133 (64-bit).
OSX 10.11.6
Database: mysql 5.7
I also have the same result in Ghost desktop 1.2.1

pascalandy · 2017-04-20T13:56:04Z

Katharina jumped on slack and we troubleshooted the issue. Here are our findings. Below are the highlight of our conversation:

pascalandy
I think there is a glitch with previous users than do not exist anymore.

On blog 1, there is only one user at the moment. But there were 4 previously.

So for now I decide to copy-paste everything manually on blog 2 and test how it will react if I do the same operation on blog 3
So when re-importing the bkp on blog 2, Ghost is trying to associate posts with users than are deleted.

kate
There is a post

  uuid: 'cccfa1d5-8361-43cb-9bec-47c4a3af9756',
  name: 'portfolio',
  slug: 'portfolio',
  description: 'An overview of the project I realized over the years.',
  image: 'https://raw.githubusercontent.com/firepress-org/theme-assets/master/professionnal-services/martine-nadall/images/light-sky-blurry.jpg',
  parent_id: null,
  visibility: 'public',
  meta_title: null,
  meta_description: null,
  created_at: '2017-04-02T22:36:50.000Z',
  created_by: 2,
  updated_at: '2017-04-19T17:15:39.000Z',
  updated_by: 1 }

Which has no author_id. Also has no status.
But if you fix this post in your json, then it should work.

Not sure why this happened. I have to investigate and see if i can reproduce this case - to ship a fix.

There are a few
created_by:2
published_by: 2

Please update them with:

created_by:1
published_by: 1

pascalandy
Kaboom !!
works here as well :)

kate
oh i am happy it works 😄
i will care about a fix.

kirrg001 · 2017-04-20T13:59:05Z

@pascalandy 👋 Thanks for the update!

What we have to ensure is that if a user get's deleted, we either update the related posts or we allow this when importing.

pascalandy · 2017-04-20T14:00:21Z

Make perfect sense! Thank you for everything :)

kirrg001 · 2017-05-21T20:33:07Z

This error won't happen with Ghost 1.0 anymore. (see #8473)
We allow the import of unknown user id's. e.g. you import a post with published_by:2, but this user was deleted. The admin UI can handle unknown user id's and published_by or author_id fields can be updated. @kevinansfield Do you have any concerns?

This is still an LTS issue, a fix is welcome 🙃

pascalandy · 2017-05-22T00:25:35Z

Sounds great!

kevinansfield · 2017-05-22T10:31:08Z

@kevinansfield Do you have any concerns?

None that I can think of, we may want to add some smoke tests in Ghost-Admin to ensure that the posts list and editor will load without error if the referenced user is missing.

kirrg001 · 2017-05-22T10:36:08Z

None that I can think of, we may want to add some smoke tests in Ghost-Admin to ensure that the posts list and editor will load without error if the referenced user is missing.

Thanks. I did a couple some smaller tests already, nothing crashed for me. Everything behaved okay.
I will assign you to #8473 later as well - so you could do some special testing regarding the UI 👍

refs TryGhost#5422 - this is the new integration of the importer - i have used a class inheritance mechanismn to achieve an easier readability and maintenace - most of the logic bits were copied from the old importer or modified/easified - here a list: - schema validation (happens on model layer) was ignored - allow to import unknown user id's (see TryGhost#8365) - most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database) - this PR has not concentrated on clear error messages or logs (this comes tomorrow) - i've added debug messages - the import is splitted into three steps: - beforeImport --> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS) - doImport --> does the actual import - afterImport --> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database) --> the background why we are doing it after the import is that we don't have to write complicated logic to detect outdated user references before we know which user get's which id --> plus: we don't care about the order of inserting the data --> keep in mind: we never ever import id's, that's why user references are usually outdated --> e.g. you import a post with `published_by:2`, but user 2 get's a new ObjectId 'X', we have to update the reference

refs #5422 - we can support null titles after this PR if we want - user model: fix getAuthorRole - user model: support adding roles by name - we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users) - base model: support for null created_at or updated_at values - post or tag slugs are always safe strings - enable an import of a null slug, no need to crash or to cover this on import layer - add new DataImporter logic - uses a class inheritance mechanism to achieve an easier readability and maintenance - schema validation (happens on model layer) was ignored - allow to import unknown user id's (see #8365) - most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database) - the import is splitted into three steps: - beforeImport --> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS) - doImport --> does the actual import - afterImport --> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database) - import images: markdown can be null - show error message when json handler can't parse file - do not request gravatar if email is null - return problems/warnings after successful import - optimise warnings in importer - do not return warnings for role duplications, no helpful information - error handler: return context information of error - we show the affected json entries as one line in the UI - show warning for: detected duplicated tag - schema validation: fix valueMustBeBoolean translation - remove context property from json parse error

kirrg001 · 2017-07-14T10:58:48Z

I am going to close this issue, because we've overhauled the importer in Ghost 1.0. If you experience this problem, please migrate to 1.0 👍 😬

kirrg001 self-assigned this Apr 20, 2017

cobbspur mentioned this issue Apr 20, 2017

Ghost Importer: Current State, Problems and Future Improvements #5422

Closed

18 tasks

kirrg001 added import / upgrade life-cycle stuff: Importing, exporting, upgrading and migrating LTS labels Apr 20, 2017

kirrg001 added this to the Next LTS Release milestone Apr 20, 2017

kirrg001 added the server / core Issues relating to the server or core of Ghost label May 12, 2017

kirrg001 modified the milestones: Upcoming LTS release, 0.11.9 May 15, 2017

kirrg001 removed their assignment May 15, 2017

kirrg001 mentioned this issue May 21, 2017

🎨 refactor the importer #8473

Merged

8 tasks

kirrg001 added the help wanted [triage] Ideal issues for contributors to help with label May 23, 2017

kirrg001 closed this as completed Jul 14, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't import my backup #8365

Can't import my backup #8365

pascalandy commented Apr 20, 2017

pascalandy commented Apr 20, 2017 •

edited

Loading

kirrg001 commented Apr 20, 2017

pascalandy commented Apr 20, 2017

kirrg001 commented May 21, 2017

pascalandy commented May 22, 2017

kevinansfield commented May 22, 2017

kirrg001 commented May 22, 2017

kirrg001 commented Jul 14, 2017

Can't import my backup #8365

Can't import my backup #8365

Comments

pascalandy commented Apr 20, 2017

Issue Summary

Steps to Reproduce

Technical details:

pascalandy commented Apr 20, 2017 • edited Loading

kirrg001 commented Apr 20, 2017

pascalandy commented Apr 20, 2017

kirrg001 commented May 21, 2017

pascalandy commented May 22, 2017

kevinansfield commented May 22, 2017

kirrg001 commented May 22, 2017

kirrg001 commented Jul 14, 2017

pascalandy commented Apr 20, 2017 •

edited

Loading