Redesign Web API #7610

glassez · 2017-10-18T07:52:11Z

Normalize Web API method names.
Invoke Web API methods via Qt Meta Object system.
Allow to use alternative Web UI.
Switch Web API version to standard form (i.e. "2.0").
Improve Web UI translation code.
Retranslate changed files.

glassez · 2017-10-18T08:29:53Z

Normalize Web API method names.

As this is a major update, proposals to change any other API method names are welcome.

zeule · 2017-10-18T09:37:11Z

Why did you change 'command' to 'api' prefix?

glassez · 2017-10-18T09:58:18Z

Why did you change 'command' to 'api' prefix?

Not only 'command'. Now all API methods are under api prefix. All other paths are considered as regular files.
Having subprefixes is meaningless, IMO. Besides, the web developers mixed/confused it in the current code.

WolfganP · 2017-10-18T16:53:34Z

Thx a lot @glassez... again. As per https://github.com/qbittorrent/qBittorrent/pull/7610/files#diff-177e7d5725be515944f83f47a0dc59bb I assume it can be enabled at run time via the preferences file, right?

What I couldn't detect from the commit is if the alt webUI is cached somehow at start, or it can be interactively changed while running and be reloaded on page refresh... could you please confirm?
Thx again!

@naikel do you mind to open a repository for your current alt webUI so we adapt it and avoid starting from scratch?

naikel · 2017-10-18T16:56:57Z

My WebUI has almost nothing implemented only the transfer list!! Not even stop/start buttons! Sadly I haven't got the time to implement more features, but it uses ReactJS and Bootstrap for look n' feel and menus and stuff (and jQuery as well).

glassez · 2017-10-18T17:10:17Z

@WolfganP, you can enable alternative UI either during runtime via Preferences (e.g. via Options dialog) or via configuration file editing (when application is not running).

glassez · 2017-10-18T17:19:53Z

What I couldn't detect from the commit is if the alt webUI is cached somehow at start, or it can be interactively changed while running and be reloaded on page refresh...

AFAIK, the application use file translation that caches already translated files. So if you switch to another UI source folder you can refresh, but if you update files in currently used source folder you need to restart the application. Unless I missed something... Need to be tested.

zeule · 2017-10-19T14:44:20Z

src/base/preferences.h

@@ -188,6 +188,10 @@ class Preferences: public QObject
    void setWebUiHttpsCertificate(const QByteArray &data);
    QByteArray getWebUiHttpsKey() const;
    void setWebUiHttpsKey(const QByteArray &data);
+    bool isAltWebUiEnabled() const;


These two new options should be merged. Probably with a "Set default" button in the UI.

Initially I have only one option. But someone might want to turn "Alternative UI" off, but save the path for further use. Besides, I accept the idea that other settings can appear under "Alternative UI".

The path can be saved elsewhere, what's the problem with that?

Can't follow your thoughts...
We need one setting to enable/disable Alt UI, and the second one to store path.
How to disable it but store save path using only one setting?

IMO, there is no need to save the path. Those who need it, can save it as they like. Or, even better, you may give them a list to choose from. Then there will be no need to save and re-type that path.

The program is incorrect if it produces the incorrect output for correct input and settings.

I'm talking not about correctness of the program, because it is correctly implemented to contain the designed security hole.

"Allow the user to do what he has a right" is a security hole?

If a user is authenticated and passes a correct CSRF token with the request, what's the issue with allowing them to set the webui directory? At that point they have full control over the webui anyway, including the option to specify ANY external program to execute upon torrent completion.

@Piccirello, shhh... If @evsh reads this, you'd be anathematized and burned at the stake!

Yes, any binary can be downloaded to the target system and executed via post-download command. Yet I strongly believe that presence of a mistake or design flaw in a program may not be used as an excuse for adding more problems.

glassez · 2017-10-19T14:53:36Z

PR updated:
Improve Web UI translation code (retranslate changed UI files).

glassez · 2017-10-19T14:54:34Z

if you update files in currently used source folder you need to restart the application.

@WolfganP, fixed.

zeule · 2017-10-19T14:57:32Z

I did not read the PR completely, but already want to know what could you, @glassez, comment on the following. I can't see how we separate a Web UI from qBt HTTP API.

glassez · 2017-10-19T15:06:13Z

I can't see how we separate a Web UI from qBt HTTP API.

What exactly do you want to know? How is it separated in the code? Or in the request path?
The paths starting with /api is handled as qBt HTTP API requests. All other paths handled as some kind of HTTP source files. Default path (/) has special meaning; if session isn't active yet we send login.html, otherwise index.html.

zeule · 2017-10-19T15:31:15Z

What exactly do you want to know?

Consider yet another qBt API, which differs from the HTTP one mainly by marshaling scheme. For example, DBus. Should there be an object qBtRemoveService, one would just add another marshaling and export scheme. But right now the HTTP service and UI are quite bound to each other.

glassez · 2017-10-19T15:49:06Z

But right now the HTTP service and UI are quite bound to each other.

Ok. I think I undestand you.
This is a long-existing behavior. This PR is not to change it.
Honestly, I don't imagine how to unify it. Currently Web API methods mainly doing a little service work: analyze the received parameters and (based on it) execute some command or sending some data.
When someone starts to implement a different API, it will be clear what code can be reused.

Chocobo1 · 2017-10-19T15:50:52Z

src/icons.qrc

@@ -354,7 +354,7 @@
        <file>icons/skin/paused.png</file>
        <file>icons/skin/qbittorrent-tray-dark.svg</file>
        <file>icons/skin/qbittorrent-tray-light.svg</file>
-        <file>icons/skin/qbittorrent16.png</file>
+        <file alias="www/favicon.ico">icons/skin/qbittorrent16.png</file>


It is possible to move www/favicon.ico to cpp code? it's quite hard to find here.

or move it to HTML?

It is possible to move www/favicon.ico to cpp code?

It was previous behavior.

or move it to HTML?

What do you mean?

What do you mean?

https://en.wikipedia.org/wiki/Favicon#How_to_use

@Chocobo1: how a creator of an alternative UI would get the true icon name?

@Chocobo1: how a creator of an alternative UI would get the true icon name?

Now I think we should properly introduce an ICO format favicon instead of creating an alias of the png.
We can do it later if you won't handle it now.

how a creator of an alternative UI would get the true icon name?

He just add the icon he want into his www folder...

@Chocobo1, <link rel="icon" type="image/png" href="images/skin/qbittorrent16.png"> is used now.

OK thank you!

Chocobo1 · 2017-10-19T15:52:31Z

src/webui/abstractwebapplication.cpp

-
-        data = file.readAll();
-        file.close();
+    data = file.readAll();


Can you put an upper limit for it?

Chocobo1 · 2017-10-19T15:53:34Z

src/webui/abstractwebapplication.cpp

    return true;
 }

+QString AbstractWebApplication::getContentType(const QString &path)
+{
+    QString ext = "";


the assignment is redundant.

and should be possible to use QStringRef if you like.

and should be possible to use QStringRef if you like.

Don't see any sense. It will be converted to QString later.

It will be converted to QString later.

OK and I'll continue the review later.

Chocobo1 · 2017-10-19T15:54:30Z

src/webui/abstractwebapplication.cpp

    return true;
 }

+QString AbstractWebApplication::getContentType(const QString &path)


hope you don't find marking it a const function

Can't follow you...
This is static method.

sorry, didn't saw it.

Chocobo1 · 2017-10-19T16:03:10Z

src/webui/webapplication.cpp

-
-    return actions;
-}
+constexpr Utils::Version<int, 3, 2> API_VERSION {2, 0, 0};


When converted to QString this will print 2.0.0 right?
If so, then it will break all existing usage/detection.

Well, the points of incompatibility creation should be it the development process, otherwise we will always have to live with the mistakes previously...
Really, there should not be a big problem since it a major update and old clients can't work with it anyway (because of changed API method names/paths).

Really, there should not be a big problem since it a major update and old clients can't work with it anyway (because of changed API method names/paths).

OK, then we should inform the public the changes beforehand, making an announcement on the qbt website news page when this PR is merged, and this should be as early as possible as we have a lot of users using webAPI.
@sledgehammer999

@glassez
I was thinking this should start following Semantic Versioning.
If so, this should go {<old API_VERSION> + 1, 0, 0};, what do you think?

I follow it but in another way. I try to follow qBittorrent main versioning rule when we change major version only on some very big changes. So I interpret old version as 1.x.y in new scheme and just perform major update on it.
But if most of you guys prefer "to be measured penises" (as Firefox and Chrome do), then I'll change it to the option you proposed.

Using Semantic Versioning is one thing, we can start practicing it if the majority agrees.
IMO Semantic Versioning is invented just for this situation.

The {2, 0, 0}; you proposed is another, problem is: with the current /version/api GET it returns an integer 15, but after this PR it drops back to 2.0.0 which is a confusion for our users.
So from a user perspective I would recommend using 16.0.0 (even we don't follow Semantic Versioning).
Also why not use Utils::Version<int, 3, 3>?
https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-Documentation#get-api-version

it returns an integer 15, but after this PR it drops back to 2.0.0 which is a confusion for our users.

It's not a problem, IMO. But I'll do whatever the team decides.

Also why not use Utils::Version<int, 3, 3>?

Personally I prefer the way when zeros after minor version are truncated. Waiting other reviewers.

@lgallard
what do you think about the suggestion in: https://github.com/qbittorrent/qBittorrent/pull/7610/files#r145924980

Chocobo1 · 2017-10-19T16:05:10Z

src/webui/webapplication.cpp

-}
+constexpr Utils::Version<int, 3, 2> API_VERSION {2, 0, 0};
+
+const QString PATH_PREFIX_API {"/api/"};


personally I prefer using = for string...

I would prefer leave it as it is now.

Chocobo1 · 2017-10-19T16:14:35Z

src/webui/webapplication.cpp

-
-    if (args_.contains(".") || args_.contains("..")) {
+    const QStringList pathItems {request().path.split('/', QString::SkipEmptyParts)};
+    if (pathItems.contains(".") || pathItems.contains("..")) {


Just asking, isn't pathItems.contains(".") redundant? It's basically the current/same path, why should it be prohibited?

I don't know. It's very legacy code. Should I change it?

Should I change it?

We both unsure, then just leave it.

Chocobo1 · 2017-10-19T16:27:36Z

src/webui/webapplication.cpp

+//        }
+    }
+    else {
+        printFile(m_rootFolder + request().path);


I think we should not allow a requester to get anything they want when they are not authenticated, i.e. they should only be allow to get the login page resources.

What about images/scripts/css used in login page?
Anyway html files are useless without API access.

What about images/scripts/css used in login page?

I was thinking of a tighter control scheme but it's not beneficial with user's webUI, so whatever.

Excellent security hole, BTW! Right from a textbook...

Look, stop being sarcastic! In your comments some emotions, there is no useful information. What is the problem here?

There are specific proposals? Or just wait for the requests from real users of this feature?

It's just still a primitive idea in my head, it might not make full sense.
I was thinking there should always be 2 folders under a user specified path (also applies for built-in UI), name them public & private.

The public folder contains resources that anyone can freely access, html files, js files, or even some API.
Then we can put every resources that is needed for login in this folder.

The private folder contains other resources that is only accessible after login.

The reason for this, is that I want to minimize the exposure to the public.
Why? Because the first step of web security penetration is reconnaissance, i.e. knowing what software & which version it is running, thus reducing information leakage is important IMO.

Also replying #7610 (comment) here.

It's meaningless at least for api version. Client should know it to understand how to authenticate (authentication can be changed in some future version).

Given the above idea, I think the api version is not a critical info at the authentication stage, webAPI users can simply try another login scheme when the previous failed, assuming we aren't inventing a login scheme every year.

Something like that used in the current (current master) implementation, but it is somehow flawed... so I decided to abandon it in favor of simplicity. But I can return to it if I can see a complete picture of what is what.

Then we can put every resources that is needed for login in this folder.

AFAIK, all common resources (js, css, images) are public in web apps (I mean images used in page formatting etc.).

AFAIK, all common resources (js, css, images) are public in web apps (I mean images used in page formatting etc.).

Ideally only login-related resources should be in public folder, everything else should go in private folder.
And login-related resources should be independent from other resources, some resource might get duplicated but it's a very small cost IMO.

And as a side effect (bonus?) of this model, you can turn qbt into a dumb public file server by placing any file (with max file size limitation though) in the public folder.

I don't like to have private/public in request paths. I have idea to make it transparent for the client side, i.e. when client send request with /page.html path we search for page.html in public subfolder if client isn't authorized, otherwise we search in both private/public subfolders (in this order). This approach allows to avoid resources duplicating.

I don't like to have private/public in request paths.

Me too, yet I haven't consider how to implement it.

I have idea to make it transparent for the client side

👍 seems probable.

glassez · 2018-02-02T03:06:55Z

@WolfganP, this PR is ready and it will be merged once v4.0.4 is released.

Very overdated "changes request", so GitHub can't even provide linked code.

glassez · 2018-02-16T07:10:34Z

Thanks to all the reviewers!

WolfganP · 2018-02-16T15:29:57Z

Excellent work, thx a lot to all devs and reviewers.
@glassez do you mind to briefly document how to run an alternate webUI? (ie files names and locations, program switches / config entries, etc etc)
My first test will be copy the existing core webUI files to a new location and run them as alt, and then start modifying them, but just want to make sure I run them as intended by this feature.
Thx again!

Piccirello · 2018-02-20T02:19:24Z

@glassez Can you document the updated API urls on the wiki? This will be really useful for other applications that make use of the qBittorrent api.

Also, is it a good idea to introduce such a major API overhaul with a minor version bump? 4.1.0 seems far more appropriate.

WolfganP · 2018-02-21T18:54:44Z

I fiddled a bit with the current API documentation in the wiki and made sure each article explicitly states which version it applies to, updating also the Home page in preparation for whatever documentation comes up for this new release.

I also created https://github.com/qbittorrent/qBittorrent/wiki/Alternate-WebUI-notes to start documenting the feature and its usage (ie files names and locations, program switches / config entries, etc etc), which I'll initially work out from the current code but it will need a review from @glassez for sure.

I agree with @Piccirello that this API overhaul should be tied to a major version rather than a v.4.0.5 minor one.

Chocobo1 · 2018-02-22T03:48:10Z

I fiddled a bit with the current API documentation in the wiki and made sure each article explicitly states which version it applies to

Please revert the document title (the current API one), as it is tied to the page URL and someone might already bookmark it.
You can still keep the displayed link in Home page.

I also created https://github.com/qbittorrent/qBittorrent/wiki/Alternate-WebUI-notes to start documenting the feature

Great!
You might want to pick a title that needn't be changed for some time, e.g. "How to Use Alternate WebUI" or just "Alternate WebUI".
And nitpick, it's WebUI instead of webUI.

WolfganP · 2018-02-22T12:15:47Z

@Chocobo1 Changes done!

glassez · 2018-02-22T13:06:36Z

I agree with @Piccirello that this API overhaul should be tied to a major version rather than a v.4.0.5 minor one.

Guys, what exactly is your concern?
Legacy API is still available (via compatibility layer) and it will available, at least, until v4.1.

Can you document the updated API urls on the wiki? This will be really useful for other applications that make use of the qBittorrent api.

Of course, I'll do it when I have some time for this job.

Piccirello · 2018-02-26T20:16:55Z

src/webui/webapplication.cpp

+    APIController *controller = m_apiControllers.value(scope);
+    if (!controller) {
+        if (request().path == QLatin1String("/version/api")) {
+            print(QString(COMPAT_API_VERSION), Http::CONTENT_TYPE_TXT);


This should be QString::number(COMPAT_API_VERSION)

Piccirello · 2018-02-26T20:17:12Z

src/webui/webapplication.cpp

-            print(QObject::tr("I/O Error: Could not create temporary file."), Http::CONTENT_TYPE_TXT);
-            continue;
+        if (request().path == QLatin1String("/version/api_min")) {
+            print(QString(COMPAT_API_VERSION_MIN), Http::CONTENT_TYPE_TXT);


Use QString::number

Piccirello · 2018-02-26T20:22:12Z

Legacy API is still available (via compatibility layer) and it will available, at least, until v4.1.

A majority of the legacy apis are no longer working. With some quick testing, I was able to identify these as broken (non-inclusive):

/command/* (m_apiLegacyPathPattern doesn't contain the path "command")
/version/api
/version/api_min
/version/qbittorrent (log contains File :/www/public/version/qbittorrent was not found! )
/sync/torrent_peers
/query/torrents
/query/propertiesGeneral
/query/propertiesTrackers
/query/propertiesFiles
/query/getPieceHashes
/query/getPieceStates

Please test all of the legacy apis to catch any additional ones that are broken.

glassez · 2018-02-27T02:44:16Z

Please test all of the legacy apis to catch any additional ones that are broken.

OK. I'll fix it. Thank you for testing.

glassez · 2018-02-27T17:22:14Z

/version/api
/version/api_min

Fixed via QString::number()

/command/* (m_apiLegacyPathPattern doesn't contain the path "command")

Fixed via fixing regexp (for some reason I put "control" instead of "command" there)

/query/torrents
/query/propertiesGeneral
/query/propertiesTrackers
/query/propertiesFiles
/query/getPieceHashes
/query/getPieceStates

Work for me.

/version/qbittorrent

Fixed.

glassez · 2018-02-27T17:33:09Z

@Piccirello, see #8515.

Piccirello · 2018-03-07T08:21:52Z

@glassez It looks like TransferController::infoAction() is no longer used, and its logic migrated to SyncController::maindataAction(). This would mean we can get rid of TransferController::infoAction(). Can you confirm?

glassez · 2018-03-07T08:40:23Z

If some API isn't used in official Web UI, it's not a reason to drop it out. Some other apps may want to use it, or may want to use it in the future.

Piccirello · 2018-03-08T00:03:24Z

Good point, it makes sense to leave it in.

glassez · 2018-03-11T13:31:23Z

@WolfganP, @Piccirello, @Chocobo1, the documentation for new API is here. I followed the style of legacy API documentation, even though I don't like it (I have no time to mess with it). Please correct any errors or inaccuracies you have noticed.

WolfganP · 2018-03-11T13:40:43Z

@glassez do you mind to get a quick review & editing to this article https://github.com/qbittorrent/qBittorrent/wiki/Alternate-WebUI-usage so there's a base in the docs for those who want to start fiddling with alt WebUIs?

ngodber · 2020-01-10T00:18:39Z

@glassez Is there a recommended/durable/invariant method to check appropriate webAPI to use? I'm a tad confused as it seems you need to know/guess the correct API to use to check the webAPI version, eg

vs

Or am I misunderstanding something?

glassez added the WebUI WebUI-related issues/changes label Oct 18, 2017

glassez added this to the 4.1.0 milestone Oct 18, 2017

glassez requested review from sledgehammer999, zeule and Chocobo1 October 18, 2017 07:52

glassez force-pushed the webapi2 branch from 8268c74 to b763858 Compare October 18, 2017 09:17

glassez mentioned this pull request Oct 18, 2017

WebUI off-program extensions #7217

Closed

zeule reviewed Oct 19, 2017

View reviewed changes

glassez force-pushed the webapi2 branch from b763858 to 6e843e9 Compare October 19, 2017 14:51

Chocobo1 reviewed Oct 19, 2017

View reviewed changes

WolfganP mentioned this pull request Feb 4, 2018

[REQUEST] New WebUI? #8352

Closed

glassez merged commit d07ece5 into qbittorrent:master Feb 16, 2018

glassez deleted the webapi2 branch February 16, 2018 07:10

Chocobo1 mentioned this pull request Feb 18, 2018

[WebUI] Cannot delete torrent job #8443

Closed

Piccirello reviewed Feb 26, 2018

View reviewed changes

Piccirello mentioned this pull request Mar 1, 2018

Fix Legacy Web API to be fully available #8515

Merged

WolfganP mentioned this pull request Jun 29, 2018

WebUI should be mobile friendly #9143

Closed

WolfganP mentioned this pull request Oct 17, 2018

Alt WebUI not working #9720

Closed

Asuanin mentioned this pull request Jan 6, 2019

Alternative WebUI is enabled iFelix18/Dark-qBittorrent-WebUI#1

Closed

WolfganP mentioned this pull request Apr 1, 2020

Alternate WebUI development discussion #12354

Closed

This pull request was closed.

Redesign Web API #7610

Redesign Web API #7610

Conversation

glassez commented Oct 18, 2017 • edited Loading

glassez commented Oct 18, 2017

zeule commented Oct 18, 2017

glassez commented Oct 18, 2017

WolfganP commented Oct 18, 2017

naikel commented Oct 18, 2017

glassez commented Oct 18, 2017

glassez commented Oct 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

glassez commented Oct 19, 2017

glassez commented Oct 19, 2017

zeule commented Oct 19, 2017

glassez commented Oct 19, 2017

zeule commented Oct 19, 2017

glassez commented Oct 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeule Oct 19, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chocobo1 Oct 19, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chocobo1 Oct 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

glassez commented Feb 2, 2018

glassez commented Feb 16, 2018

WolfganP commented Feb 16, 2018

Piccirello commented Feb 20, 2018 • edited Loading

WolfganP commented Feb 21, 2018

Chocobo1 commented Feb 22, 2018

WolfganP commented Feb 22, 2018

glassez commented Feb 22, 2018

Choose a reason for hiding this comment

glassez commented Oct 18, 2017 •

edited

Loading

zeule Oct 19, 2017 •

edited

Loading

Chocobo1 Oct 19, 2017 •

edited

Loading

Chocobo1 Oct 20, 2017 •

edited

Loading

Piccirello commented Feb 20, 2018 •

edited

Loading

Piccirello commented Feb 26, 2018 •

edited

Loading