New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dradis-CE 3.1.0RC2 or 3.6 - Failed Recovery from Backup - Validation Failed - Taggable Can't be Blank #112

Closed
Kalaratri opened this Issue Apr 22, 2017 · 9 comments

Comments

Projects
None yet
3 participants
@Kalaratri

Kalaratri commented Apr 22, 2017

See http://discuss.dradisframework.org/t/dradis-ce-3-6-failed-project-import-invalid-project-template-format/451

So as part of an effort to upgrade to 3.6 I realised that as of 12th April my 3.1.0RC2 backups are no longer viable. I was trying to export my data from 3.1.0RC2 and import into 3.6. The instance 3.1.0RC2 environment is still workable, but now I am incredibly nervous committing further to using it for another 40 days through a project. But I'm also so far into the project it will be a real battle to switch to something else. Talk about a rock and a hard place.

Here's my extensive notes from the upgrade, backup import attempts and troubleshooting. As you can see there's many days of work spent trying to diagnose this issue:

dradis 3.6 upgrade notes.txt

All backups dated 11th April or earlier restore correctly into Dradis-CE 3.1.0 RC2 or 3.6. The key element to watch for is here:

[23:02:43] New tag detected: !9467bd_critical
[23:02:44] New tag detected: !d62728_high
[23:02:45] New tag detected: !ff7f0e_medium
[23:02:47] New tag detected: !6baed6_low
[23:02:47] New tag detected: !2ca02c_info
[23:02:49] Wrapping up...
[23:02:49] Setting issue_id for evidence

I then tried to manually re-enter data lost since these backups (102 pages copied from current working instance, saving a new backup file to try a node at a time). I then try making subsequent recovery attempts to bring the data into both 3.1.0RC2 or 3.6. All attempts fail with the error: Validation Failed - Taggable Can't be Blank at this point in the import process:

[23:41:26] New tag detected: !9467bd_critical
[23:41:26] Validation failed: Taggable can't be blank
[23:41:26] Worker process completed.

So i tried immediately exporting the project (in both 3.1.0RC2 & 3.6), reset the database and tried reimporting the same file. All recovery attempts fails with: Validation failed: Taggable can't be blank error.

Each time I've been trying a thor dradis:reset:database / thor dradis:reset:attachments and ALL recoveries fail. I've been trying a full bundle exec thor dradis:reset but also receive the same error mentioned in this thread:

#76

It appears not only have the backup files become corrupted for some unknown reason, but when trying to recovery from backups, whilst the initial recovery works, all subsequent rework, which is then backed up cannot be imported into Dradis again should another recovery be required. Is seems Dradis does not have a viable backup recovery system.

The 11th April file: The dradis-repository.xml is 3.3MB, the 12th is 1.7MB. All backups files dated 12th April onwards fail.

I could keep using the current working 3.1.0RC2 version with data current as of 21st April, however I now know that backups cannot be recovered.

I need to think long and hard where to go from here, so any guidance to resolve this, much appreciated.

Thanks

@Kalaratri Kalaratri changed the title from Dradis-CE 3.6 - Failed Project Import - Invalid project template format to Dradis-CE 3.0.0RC2 or 3.6 - Failed Recovery from Backup - Validation Failed - Taggable Can't be Blank Apr 23, 2017

@Kalaratri Kalaratri changed the title from Dradis-CE 3.0.0RC2 or 3.6 - Failed Recovery from Backup - Validation Failed - Taggable Can't be Blank to Dradis-CE 3.1.0RC2 or 3.6 - Failed Recovery from Backup - Validation Failed - Taggable Can't be Blank Apr 23, 2017

@rachkor

This comment has been minimized.

Show comment
Hide comment
@rachkor

rachkor Apr 25, 2017

Member

Hi @Kalaratri, I wanted to let you know that the technical team is looking into this. I don't have any more details for you at this point but we will be back as soon as they're available. Thanks for passing this on to us so that we can investigate and resolve this for you!

Member

rachkor commented Apr 25, 2017

Hi @Kalaratri, I wanted to let you know that the technical team is looking into this. I don't have any more details for you at this point but we will be back as soon as they're available. Thanks for passing this on to us so that we can investigate and resolve this for you!

@Kalaratri

This comment has been minimized.

Show comment
Hide comment
@Kalaratri

Kalaratri Apr 25, 2017

Thanks for your support! If you'd like a copy of any of the relevant files let me know how I can get them to you privately.

Kalaratri commented Apr 25, 2017

Thanks for your support! If you'd like a copy of any of the relevant files let me know how I can get them to you privately.

@dormi

This comment has been minimized.

Show comment
Hide comment
@dormi

dormi Apr 26, 2017

Contributor

@Kalaratri if you can send us those backup files (working and broken) that could help to find out what is going on.
Also, if it's possible, add for each backup file from what dradis version it was generated and how it was generated (thor?, "export results" ? ...)
You could send the files to email@securityroots.com or via https://dradis.slack.com

Contributor

dormi commented Apr 26, 2017

@Kalaratri if you can send us those backup files (working and broken) that could help to find out what is going on.
Also, if it's possible, add for each backup file from what dradis version it was generated and how it was generated (thor?, "export results" ? ...)
You could send the files to email@securityroots.com or via https://dradis.slack.com

@Kalaratri

This comment has been minimized.

Show comment
Hide comment
@Kalaratri

Kalaratri Apr 26, 2017

Kalaratri commented Apr 26, 2017

@Kalaratri

This comment has been minimized.

Show comment
Hide comment
@Kalaratri

Kalaratri Apr 26, 2017

I've signed up for my own slack account and send you an invite to email@securityroots.com

Kalaratri commented Apr 26, 2017

I've signed up for my own slack account and send you an invite to email@securityroots.com

@dormi

This comment has been minimized.

Show comment
Hide comment
@dormi

dormi Apr 26, 2017

Contributor

cannot see that invitation yet 😞
Please get your invitation to dradis.slack.com in https://evening-hamlet-4416.herokuapp.com/

Contributor

dormi commented Apr 26, 2017

cannot see that invitation yet 😞
Please get your invitation to dradis.slack.com in https://evening-hamlet-4416.herokuapp.com/

@Kalaratri

This comment has been minimized.

Show comment
Hide comment
@Kalaratri

Kalaratri Apr 26, 2017

Awesome, joined, which user do I share the backup files with?

Kalaratri commented Apr 26, 2017

Awesome, joined, which user do I share the backup files with?

@dormi

This comment has been minimized.

Show comment
Hide comment
@dormi

dormi Apr 27, 2017

Contributor

@Kalaratri I think this can be closed, a new issue with details on how to reproduce this is available #116

Contributor

dormi commented Apr 27, 2017

@Kalaratri I think this can be closed, a new issue with details on how to reproduce this is available #116

@Kalaratri

This comment has been minimized.

Show comment
Hide comment
@Kalaratri

Kalaratri Apr 27, 2017

Agreed, thanks for your first class support! The bad characters copied and pasted into evidence were indeed breaking the backups

Kalaratri commented Apr 27, 2017

Agreed, thanks for your first class support! The bad characters copied and pasted into evidence were indeed breaking the backups

@Kalaratri Kalaratri closed this Apr 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment