Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Storj DCS] Update and rename Tardigrade-Backend #4492

Merged
merged 30 commits into from Jul 20, 2021
Merged

[Storj DCS] Update and rename Tardigrade-Backend #4492

merged 30 commits into from Jul 20, 2021

Conversation

TopperDEL
Copy link
Contributor

Storj just recently renamed their Tardigrade Decentralised Cloud Storage to "Storj DCS". This PR updates the backend accordingly. In Detail, the following is changed:

  • Rename all "Tardigrade"-strings to "Storj DCS"
  • Update the list of available satellites (endpoints)
  • Add an additional passphrase verification field
  • Add some input-field-validation
  • Update the underlying uplink.NET to the latest version

A huge question from my side is to be discussed: I left all Tardigrade-C#-Object-names the same and also the protocol key. I assume especially the latter would break existing configurations, right? Can someone suggest how to proceed? Shall the backend itself stay "tardigrade"? Is there a way to support both protocols (the old "tardigrade" and the new one tbd)? Should I at least rename all objects and projects from tardigrade to storj?
Any suggestions are highly appreciated. :)

@warwickmm
Copy link
Member

warwickmm commented Apr 30, 2021

Thanks @TopperDEL. Are the old endpoint URLs still valid? We were planning on releasing our next beta version soon.

@TopperDEL
Copy link
Contributor Author

TopperDEL commented Apr 30, 2021

The old endpoints are still valid, yes. They get routed to the new one. For new backup-settings, though, it is good to have the new ones in place.
You may put this in the next beta or we may wait - this is up to you. From my point there is no need to wait, though. Everything should still work.

@warwickmm
Copy link
Member

Thanks. Since we've finished most of our testing for this upcoming release, we will wait before integrating these changes.

I was able to create a free Storj DCS account. I then created and ran a backup using the old Tardigrade code, then upgraded to this branch, and was able to continue running backups with no issues. 👍

At the very least, it would be nice if all user-facing objects were renamed. We have an overload of the CommandLineArgument constructor that marks an option as deprecated (see pull request #4443 for example). This would allow us to support both tardigrade-auth-method and storj-auth-method for some time until the former is removed.

public CommandLineArgument(string name, ArgumentType type, string shortDescription, string longDescription, string defaultValue, string[] aliases, string[] values, string deprecationMessage)
: this(name, type, shortDescription, longDescription, defaultValue, aliases, values)
{
m_deprecated = true;
m_deprecationMessage = deprecationMessage;
}

If it's also possible to rename all internal objects without affecting existing user configurations, that would be a nice bonus for consistency and avoid any confusion for future developers.

@warwickmm
Copy link
Member

warwickmm commented Apr 30, 2021

On second thought, I'd also be fine with keeping the internal names as tardigrade, if that simplifies things. We can save the internal refactoring for a future task.

We should still consider deprecating the old tardigrade-options in favor of new storj- options.

@TopperDEL
Copy link
Contributor Author

I've now renamed all objects and parameters but led the protocol-key unaffected. The old parameters are marked as "deprecated" now. My tests with old configurations worked as expected, but it might be nice to have a second test from you @warwickmm.

If everything is good from your point, too, than this change could be merged whenever you're ready. Thank you!

@TopperDEL
Copy link
Contributor Author

@warwickmm Thank you very much for your in-depth review! I fixed all issues, so you may have a look again. Please come back to me if there are any issues left.
Thank you!

@warwickmm
Copy link
Member

Sorry @TopperDEL, I'm now starting to question my decision to rename everything. When "upgrading" a Tardigrade backup to this branch, I encounter a bunch of warnings about deprecated options and don't see a trivial way to rename the options in the destination URL. Moreover, since the protocol is still Tardigrade, the command-line help still requires one to refer to it as Tardigrade instead of Storj, which can be confusing.

$ mono Duplicati/GUI/Duplicati.GUI.TrayIcon/bin/Debug/Duplicati.CommandLine.exe help Storj
Topic not found: Storj

@ts678, @drwtsn32x, @kenkendk, do you have any preferences here? While Tardigrade cloud storage has been renamed to Storj DCS, would it be confusing if Duplicati continued to refer to it as Tardigrade? Or, is there a simple way to update existing configurations to use the new option names, etc.?

@warwickmm
Copy link
Member

@TopperDEL, I also noticed that the backend files are not visible when I log in to my Storj account and go to "Objects" and click on the bucket name. Should I expect to see the Duplicati files there? The backups are working fine, so this is perhaps more just a usability question.

@warwickmm
Copy link
Member

I think the number of Tardigrade users is pretty small, so the impact will be limited. Given that, creating a new "Storj DCS" backend and deprecating the Tardigrade backend is probably the cleanest solution.

I'm not sure what the best way is to consolidate the code. One option might be to add a new "Storj DCS" backend, and modify the Taridgrade backend to contain an instance of the Storj backend and forward the member implementations as appropriate. The JavaScript and HTML might be more difficult, but those changes are less common so perhaps duplication there is less of a concern.

Revision 8dae320 contains an example of how we warned users about Amazon Cloud Drive being shut down.

@TopperDEL
Copy link
Contributor Author

So finally it should be done. I copied the Tardigrade-Backend over to the new Storj DCS and tried to reduce the duplicate code by internally use the new Storj-Backend from the old Tardigrade-Backend. I would be happy if someone would test this in order to see if I did any mistake - but from my point of view it looks good.

image

@warwickmm
Copy link
Member

Thanks @TopperDEL. There appear to be some build errors. Are you able to reproduce them?

@TopperDEL
Copy link
Contributor Author

Sorry @warwickmm , my bad. I did a quick change and let it go without another test. Let's see what the CI has to say, now. Builds locally here now.

@warwickmm
Copy link
Member

I ran several tests and everything is working nicely so far. Thanks @TopperDEL.

@TopperDEL
Copy link
Contributor Author

All fixed from my side. :) @warwickmm

@warwickmm
Copy link
Member

Thanks @TopperDEL.

@warwickmm warwickmm merged commit 184dfea into duplicati:master Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants