Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplistic Qt GUI for borgbackup #2960

Closed
vduseev opened this issue Aug 24, 2017 · 71 comments
Closed

Simplistic Qt GUI for borgbackup #2960

vduseev opened this issue Aug 24, 2017 · 71 comments

Comments

@vduseev
Copy link

vduseev commented Aug 24, 2017

Idea of GUI client

Create a simple desktop application that executes borg behind the scenes and runs the very basics of backups.

Goal

Aimed at users with no understanding of backup tools, directory structures, ssh, backup algorithms, etc. Allow them to run their favorite backup tool on any platform.

Proposed tech stack

  • PyQt 5 (Qt framework)
  • QML for UI
  • borg executed as subprocess underneath

Upd (8/24/2017):

  • require borg >= 1.1.0rc1 (and later >= 1.1.0)
  • use --json(-lines)

Basic requirements

  • Backup files to external hard drive
  • Run on every major platform: Windows, MacOS X, Linux, FreeBSD?

Available target platforms for Qt

Note: Qt is not targeting FreeBSD as of latest 5.9 version

Platform
Windows 7, 8.1, 10 (32/64-bit)
openSUSE 42.1 (64-bit)
Red Hat Enterprise Linux 6.6, 7.2 (64-bit)
Ubuntu 16.04 (64-bit)
(Linux 32/64-bit)
macOS 10.10, 10.11, 10.12
Embedded Linux
QNX 6.6.0, 7.0 (armv7le and x86)
INTEGRITY 11.4.x
Universal Windows Platform (UWP) (x86, x86_64, armv7)
Android (API Level: 16)

Development plan

I'd like to start on this project and create a separate repository for the Desktop GUI. I suppose "borgbackup" can then fork my repository.

Looking forward to any ideas and comments about requirements for such desktop client.

@ThomasWaldmann
Copy link
Member

Suggestions:

  • require borg >= 1.1.0rc1 (and later >= 1.1.0)
  • use --json(-lines)
  • do not initially support windows (we have no native windows support yet, just cygwin), but freebsd. also not only ubuntu, but linux in general.
  • make a very simple thing to start with (see borgweb), release it, improve on that by small steps.

@copart
Copy link

copart commented Aug 24, 2017

With Crashplan Home announcing that they are killing the program, there will be a demand for a this. The Crashplan Business version does not support PC to PC backup!! Crashplan was an easy way to backup Windows/Linux computers to other PCs in a deduplicated fashion. I use Borg to backup all my Linux PCs, but I used Crashplan to backup family computers that use Windows! I would love to use Borg on Windows, but the support is "beta"!

@ThomasWaldmann
Copy link
Member

@copart see windows branch in the repo. It's not finished, nor does it have enough developers / reviewers - that's the main obstacle for getting native windows support (and having that stuff merged into master).

@vduseev
Copy link
Author

vduseev commented Aug 24, 2017

Also Qt in open source implies following restrictions:

  • Available under GPL & LGPLv3 licenses
  • Using parts of Qt that are only available under GPL requires open sourcing of your application when distributing
  • Must provide a relinking mechanism for Qt libraries
  • Must provide a license copy & explicitly acknowledge Qt usage
  • Must make a Qt source code copy available for customers
  • Qt source code modifications aren’t proprietary
  • Must make “open” consumer devices
  • For Digital Rights Management see (L)GPL FAQ
  • Special consideration should be taken when attempting to enforce software patents

Are these restrictions harmful for borg?

@robaato
Copy link

robaato commented Aug 24, 2017

For borg - no - it's on 3 clause BSD. So the GUI can redistribute Borg (with copyright notice in docs or in-app).
It is only an issue for the GUI app itself.

@enkore
Copy link
Contributor

enkore commented Aug 24, 2017

Are these restrictions harmful for borg?

In the general case the licenses of two programs talking over pipes to each other are irrelevant. When distributing them together (like an installer), then the license needs to permit that (no problem here). (Also Borg is BSD so it's even permitted to make a closed source fork and sell it).

@ThomasWaldmann
Copy link
Member

Any progress on this?

@dragetd
Copy link
Contributor

dragetd commented Nov 1, 2017

I'd also love to contribute to that project. Did you already open a repository for this @vduseev ? We could brainstorm some basic architectural ideas on IRC (or mumble, discord etc.). I am 'draget' on freenode and online today. :)

@ThomasWaldmann
Copy link
Member

@enkore ^^

@dimejo
Copy link

dimejo commented Nov 3, 2017

Nice! Looking forward to a testable version.

@ghost
Copy link

ghost commented Nov 5, 2017

I'm interested in this as well, please keep us posted, i know i have a few servers which this would greatly help

@roynielsen17
Copy link

Since BorgBackup is Python, is there any interest in a PyQt or Pyside interface?

@ThomasWaldmann
Copy link
Member

@roynielsen17 sure.

I played a bit with PyQt recently, but guess I don't have enough time to do the GUI work. I could maybe help with the interfacing to borg, though. If code is Python, I tend to have a look / help, if not, then rather not.

Last time I looked, PySide was quite behind current Qt, so maybe PyQt makes more sense.

@dragetd
Copy link
Contributor

dragetd commented Dec 9, 2017

Developing Qt/QML is a lot more comfortable when using the native libraries and QT-Creator. Since borg has the json interface, my personal recommondation would be to is this and subprocesses. :)

@roynielsen17
Copy link

roynielsen17 commented Dec 10, 2017 via email

@bkloppenborg
Copy link

@marcpope Do you need any help with Borg Backup Server? I am about to move all of my machines over to borg and would likely replicate deployment scripts. Having a console like the screenshot above to keep track or send notifications would be of very high interest to me.

@henfri
Copy link
Contributor

henfri commented Dec 27, 2017

Hello,
I just stumbled upon this: https://demo.burp-ui.org/
Have a look. It's a nice example of a to-be situation.

Greetings,
Hendrik

@Nebucatnetzer
Copy link

Hi everyone,

I'm currently studying software engineering.
For my "thesis" I would like to develop a GUI for borg. I've had this idea since quite a while and decided I do it for my thesis because it would allow me to provide quite some time to it.
However since it will be my "thesis" I can't work on it as part of a project.
It will be open source from the beginning however and when I'm finished and it's useful for someone I would be more than glad to open it up to everyone.
At the moment I don't know if my school allows me to go forward with this project because I didn't submit it yet. That will happen sometime during this year before fall.

The idea is to create an application similar to "Back in Time":
https://github.com/bit-team/backintime
Which I think is quite a good programme. The major problem is that it lacks deduplication.
Technology wise I thought about using Python because the main platform will be Linux but I would like to maintain crossplatform compatibility. At least technology wise, I don't think I will be able to code it completely crossplatform from the beginning. For that I'm to inexperienced. For the GUI part I thought about using Qt for the same reasons as with Python. I will have to work this out in detail as part of my project so this is not set in stone yet.

I just wanted to let you know that I'm planning to do this and maybe it might be useful if I'm allowed to do it and I don't produce a huge pile of garbage :).

The start of the project would be in fall 2018 and I have to be finished in spring 2019.

Best Regards

Nebucatnetzer

@ThomasWaldmann
Copy link
Member

@Nebucatnetzer sounds good! :)

@ghost
Copy link

ghost commented Mar 12, 2018

I recently wrote Borgend. It's not a complete GUI, but after configuration, does the behind the scenes stuff: https://bitbucket.org/tuomov/borgend

@ThomasWaldmann
Copy link
Member

@vomout The first paragraph there does not even say it is a GUI. Also, if it works on Linux, maybe say so (not just not on macOS).

IIRC, I had visited that URL before, but guess you lost me after a few seconds, thinking it is just a scheduler/tray icon for Mac.

@ghost
Copy link

ghost commented Mar 12, 2018

@ThomasWaldmann It's a tray icon, so a GUI-of-sorts. But no GUI configuration, etc. Novices can use it to trigger backups once configured by an expert. The README says it works on other systems, but without the tray icon.

@henfri
Copy link
Contributor

henfri commented Jul 3, 2018

@marcpope what is the Status of your project?

@henfri
Copy link
Contributor

henfri commented Jul 21, 2018

Hello Marc,

good to hear.
it looks great!
Can you tell a bit more about it? Who is 'we'? Are you going to relase it as open source?

Greetings,
Hendrik

@dragetd
Copy link
Contributor

dragetd commented Jul 21, 2018

I really love the projects and would very much like to hear more about them.
But this issue is about a simple QT-GUI for borg and got very off topic.

It would be awesome if @marcpope could open a new issue and post updates and infos there. Then this issue could be cleaned up back to comments around the desktop/QT GUI for borg :-)

@Nebucatnetzer
Copy link

Nebucatnetzer commented Jul 21, 2018

@marcpope This sounds like something very useful and it would be really great if you could open source it.

About my "thesis" idea. I got told that it is a worthy project which means I can start development in December.
A mirror of the documentation repositoy can be found here:
https://github.com/Nebucatnetzer/thesis
edit: the thesis will be in German however the code etc. will be in English.

I'll inform you as soon as I've created the code repository.

@henfri
Copy link
Contributor

henfri commented Jul 21, 2018

@marcpope sounds huge!

Is it "open source"? I really don't know. I am open for suggestions. We've got hundreds of hours of programming in this project and honestly, it was never going to be more than an internal utility for our company, but I know the need is out there for something like this. So extra work is being done to make it a usable product for others.

I understand that.
The same (hundrets of hours) applies to borg aswell though. So it's a give and take, and maybe in the end you will even profit from hundrets of hours that others may invest to futher improve.

Greetings,
Hendrik

@ghost
Copy link

ghost commented Jul 24, 2018

hi Marc, can I please offer a little advice?!

First, kudos on a really cool looking GUI for a BORG backup system. It looks more than terrific.

Second, I too am going to offer something to open source - and that's my room escape CREPE (Cloudburst Room Escape Puzzle Engine). I may charge a tiny bit for back-end services.

Here's what I would recommend. I see that this project is getting stymied. This may seem obvious but I would recommend choosing to make a much, much smaller partial list of stuff to do and get done. That will allow for a release 1.0. The old Leen approach.

Why? I need this now, is my selfish reason. But seriously, things like fancy graphics can wait. It doesn't need to be working 100%. Most of the UI can be left for later. Lots and lots of the long list that you posted above and that I see, can certainly wait.

And monetization? I think that's a great idea. Releasing to perhaps Mac OS X store and charging maybe $20 / month subscription would send you lots of money for further development on the project. Please get it on the Mac. There's already a Windows BORG solution.

I probably don't need to say this but the first iteration, the focus of the short list - would be to just provide a basic solution for backup, and restore. Just get it basically working for starters...

Anyway, my two cents!!!

@Nebucatnetzer
Copy link

@henfri, I don't know about the others.
For my application I plan to support as many borg features as possible/sensible.

The settings are currently read from an .ini file but there's currently no GUI to write to it.
It's however something I would like to provide.

There are many more features I would like to add.
I've listed a few here:
https://github.com/Nebucatnetzer/thesis/blob/master/projektantrag/projektantrag.org#zielsetzung
Unfortunately it's in German but I'm going to add those to the project repo in English once that is up.
Cross-plattform support is a must have target and making backups over SSH has priority 5/5 (it is very easy to do and in a few tests it didn't really mater if the repo was local or on a server except for the speed difference).

@MTrage
Copy link

MTrage commented Sep 29, 2018

Since I myself wanted a GUI that corresponds to my wishes, I created a simple one. It meets my requirements but is currently being extended by me.

For Borg users who want to have a look or use it, here is the link:
https://github.com/MTrage/Borg-BackUP-GUI

@ghost
Copy link

ghost commented Sep 29, 2018 via email

@ghost
Copy link

ghost commented Sep 29, 2018 via email

@m3nu
Copy link
Contributor

m3nu commented Oct 27, 2018

Took a shot at a Borg GUI as well. Here what I have so far. Features are roughly as discussed here, except that it uses the older Qt Creator instead of Qt Quick to keep it nice and boring.

Here the repo. I'll upload a binary package tomorrow or so. Currently only tested on macOS.

@MTrage
Copy link

MTrage commented Oct 27, 2018

@m3nu this looks good.
Finally some competition in the area of BORG GUI :D

I will upload my newest version of BORG BackUP GUI in the next days (maybe tomorrow (sunday)).

Have fun coding.

@m3nu
Copy link
Contributor

m3nu commented Oct 28, 2018

Next update is out. Main changes:

  • now lives in the menubar
  • implement exclude-settings
  • sign and package properly

@enkore
Copy link
Contributor

enkore commented Oct 28, 2018

Using a src/ based project layout might save you a bunch of trouble in the future (e.g. when using unit tests or tox or ...).

@m3nu
Copy link
Contributor

m3nu commented Oct 29, 2018

Thanks for checking it out @enkore . I read up on src-based layouts and see the benefits. Will change it today.

@m3nu
Copy link
Contributor

m3nu commented Oct 30, 2018

Some progress today:

  • move to src-based setup
  • add first tests (qt-style)
  • add scheduler
  • add rule to choose allowed WIFIs (the only macOS-specific feature currently)
  • save repo passwords to keychain

Too late for a proper release today, but hoping to get it in front of beta users in a few days.

Any other features that could be worth including?

@mspeder
Copy link

mspeder commented Nov 2, 2018

Tried to install vorta different ways. Unfortunately all failed.

OSX prebuilt binary gives

  File "vorta/application.py", line 20, in __init__
  File "vorta/scheduler.py", line 14, in __init__
  File "vorta/scheduler.py", line 19, in reload
AttributeError: can't set attribute
[93565] Failed to execute script __main__
Sentry is attempting to send 1 pending error messages

OSX from git clone gives

    load_entry_point('vorta', 'gui_scripts', 'vorta')()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 484, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2714, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2332, in load
    return self.resolve()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2338, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'vorta'

Linux from pip install gives pretty much the same

    load_entry_point('vorta==0.2.3', 'gui_scripts', 'vorta')()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 484, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2725, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2343, in load
    return self.resolve()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2349, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'vorta'

@m3nu any idea ?

@m3nu
Copy link
Contributor

m3nu commented Nov 2, 2018

Hi Matthieu, thanks for testing.

It seems I caused some packaging issues after trying to put too much in setup.cfg. When I moved some of the options back to setup.py, it worked as expected.

Regarding the issue on macOS, you may have an old version. Unfortunately I didn't include a version number in my Sentry error reports before. I've changed that now. Since your errors first appeared 2 days ago, that may be the reason. Did you try the latest build from Github?

To summarize:

  • the Python packaging issue should be fixed. Latest version 0.2.6 is on pypi. Tested with tox and some conda-envs. Before I got the same error as you, when not using a develop-install.
  • the latest macOS version should also work. Your may have been 1-2 days old. If the error happens again, I will get full version details at least. Tested in a macOS guest account, but will test on a different machine or VM as well. I'm still in the process of setting this up.

@mspeder
Copy link

mspeder commented Nov 2, 2018

Hi Manuel

Thanks for your quick answer !
I downloaded latest OSX binary and I confirm the app now launches successfully.

OSX from pip package works too (got version 0.2.8).

Linux from pip package (got version 0.2.8) now fails with a message complaining about the absence of file /Library/Preferences/SystemConfiguration/com.apple.airport.preferences.plist :-)

Now that I have a version that works, I did some testing on OSX binary.
Overall, Vorta looks very promising to me. I especially like to have it directly in the menubar (a different icon while backup is running would be a nice addition).
Thanks for the very good work you did !

Here are some unexpected behaviors I spotted (maybe some of them are simply not-yet-implemented) :

  • when the backup is finished, the cancel button remains enabled
  • on snapshots view : delete and refresh button are always disabled (hence can't delete any snapshot manually)
  • on schedule view : prune checkbox has status "mixed values"/undefined each time I relaunch the app
  • is pruning implemented yet ? I never had any of my snapshots pruned (tested multiple shots with prune config = 1 hourly)
  • is there a way to clear the log ?

Thanks again !

@m3nu
Copy link
Contributor

m3nu commented Nov 3, 2018

Good to have it working (sort of) now. I'll look into the stuff you noticed and everything that came in on Sentry. Another user was testing on macOS as well. In short,

  • It should be cross-platform and the Wifi-list is the only thing Mac-specific for now. I'll make it fail gracefully and then find a way to get the Wifi-list on Linux if it exists somewhere.
  • Pruning, deleting snapshots and repo-checking isn't implemented yet. I left a note about it in the README and will add it soon.

For the smaller things you reported, I'll have an update out later today.

I now realize that a desktop-app is much harder than a web-app because debugging and updating is more tricky. Still finding the best ways to tackle this.

@m3nu
Copy link
Contributor

m3nu commented Nov 3, 2018

One more note since I saw some database-errors: I added database schema migrations a few versions ago. Settings DBs created before this change won't fully work when leaving out some versions.

You probably noticed this and deleted ~/Library/Application Support/Vorta (or the equivalent on Linux). This only affects those who installed an early version and then went to a very recent one. In all newer builds the database is versioned and will migrate the schema before starting.

@m3nu
Copy link
Contributor

m3nu commented Nov 3, 2018

Collecting GUI-related bugs here: https://github.com/borgbase/vorta/issues

@m3nu
Copy link
Contributor

m3nu commented Nov 17, 2018

Large update to support multiple profiles and local repos: https://github.com/borgbase/vorta/releases/tag/v0.4.0

Looking to focus on bug-fixes and getting Linux-support into better shape over the next days.

@MTrage
Copy link

MTrage commented Nov 19, 2018

After a long time of work I am now handing over a completely new version of BORG BackUP GUI.

Compared to the previous version the version 0.8 is extremely extended and automated to make it possible for a new version to do everything via the GUI.

At the first start you will be asked to create a new BackUP

new-archiv

Create a new snapshot

new-snapshot

A detailed overview (with videos) can be found in the Wiki section:
https://github.com/MTrage/Borg-BackUP-GUI/wiki/Short-instruction

@MTrage
Copy link

MTrage commented Nov 19, 2018

In order to make it easier for other developers to use BORG BackUP GUI as a basis for their own ideas, the license model was changed to GPL-3.0.

@MTrage
Copy link

MTrage commented Nov 21, 2018

Conversion to the free selection of Qt5 themes.

qt5-style-min

A detailed overview (with videos) can be found in the Wiki section: https://github.com/MTrage/Borg-BackUP-GUI/wiki/Short-instruction

@Nebucatnetzer
Copy link

Just to let everyone know that I've started my diploma project today.
However due to the fact that so many now have started to create a GUI I unsubscribed from this issue.
It's just too tempting to look at the code you guys create :).

However I will post here from time to time to keep you updated how it is going.
At minimum when I start the main repository with the code.

@MTrage
Copy link

MTrage commented Dec 10, 2018

@Nebucatnetzer, my intuition was to provide a basis on which to build or develop.

Nevertheless I am curious what you will develop in the future. I keep my fingers crossed for you and wish you a lot of fun in realizing your new goal.

Greetings Marc

@Nebucatnetzer
Copy link

@MTrage, I hope you understood your post correctly.
I'm still going to develop the GUI for borg as part of the project.
However because there are now so many similar projects making the rounds I would like to minimize the chance that I copy of you guys.

After I'm finished with the project I'm more than happy to work with you guys :)

@MTrage
Copy link

MTrage commented Dec 12, 2018

@Nebucatnetzer copy it, I won't tell anyone! °D
Borg Backup offers so many features that you can still use or embed, many of them I didn't use either.
Either way I am curious about your work.

@henfri
Copy link
Contributor

henfri commented Dec 26, 2018

@Nebucatnetzer there is nothing bad copying something. Even during a diploma thesis. There is no point reinventing the wheel.

Greetings,
Hendrik

@Nebucatnetzer
Copy link

@henfri sure to a certain point you're right.

Anyway, I've created now the main repository where the project will live.
You can find it here:
https://github.com/borg-qt/borg-qt

@Nebucatnetzer
Copy link

Nebucatnetzer commented Jan 24, 2019

Hey guys,

Another small update from me.
As promissed I translated all the targets for the application. You can find them here: https://github.com/borg-qt/borg-qt/blob/master/docs/todos.org

In addition here are some screenshots of the UI. At the moment the settings part is working. Next I'm going to start working on the part which controls BorgBackup.

1
2
3
4
5
6

Two strangest things so far have been that configparser doesn't provide the functionality to read/write lists and the QtListWidget doesn't have a method which returns all it's items in a list.
However for both of them I found workarounds.

@Nebucatnetzer
Copy link

Hey guys another little update from my side. I've uploaded now the first binary. For the moment I'm done with coding and have to focus on finalizing the thesis.

Currently it contains the following functionality: Creating, restoring, deleting and mounting archives. Graphical configuration for the provided options and for me most importantly background backups through systemd timers.

There is a lot of room for improvement and I'm looking forward to all constructive inputs so that I can improve the application further after the thesis is finished.

screenshot from 2019-02-25 10-58-46
borgqt_progress_v2
borgqt_settings_exclude_v2
borgqt_settings_schedule

@ThomasWaldmann
Copy link
Member

I am closing this as solved, there are multiple GUIs now for borg.

All of them should be linked from the community repo. If not, please do an update PR there.

https://github.com/borgbackup/community/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests