Block External Content from Zim Web Pages #1116

ShaopengLin · 2024-05-23T07:27:53Z

Fix #904
Changes:

Added the toggle button to disable external content
Refresh all opened articles when the button is toggled.
Requests that do not start with "zim://" are blocked.

You can add this code to TabBar::openUrl to add an image, sourced online, to the top of each article.

connect(webView, &QWebEngineView::loadFinished, [=](){
        webView->page()->runJavaScript(
            "var img = document.createElement(\"img\");"
            "img.src = \"https://picsum.photos/200/300\";"
            "var body = document.body;"
            "body.insertBefore(img, body.firstChild);"
        );
    });

kelson42 · 2024-05-28T05:09:37Z

@veloman-yunkan A review please?

veloman-yunkan

Regarding the history of this PR, I think that blocking of external resources must be introduced unconditionally in the first commit (so that it is immediately obvious how ExternReqInterceptor is going to be utilized) and then made optional in subsequent commit.

veloman-yunkan · 2024-05-29T13:57:14Z

src/webview.h

+{
+    Q_OBJECT
+public:
+    ExternalReqInterceptor(bool isActive, QObject* parent=nullptr) : QWebEngineUrlRequestInterceptor(parent), m_isActive(isActive) {}


Make explicit

Fit into line length of 80 chars

src/webview.h

veloman-yunkan · 2024-05-29T14:01:57Z

src/webview.h

+            qDebug() << "Blocked external request to URL >"
+                    << reqUrl << "<";


What a strange way of quoting the URL (as >URL<)

src/tabbar.cpp

veloman-yunkan · 2024-05-29T14:34:32Z

src/portutils.h

+#if QT_VERSION < QT_VERSION_CHECK(5, 13, 0) // Earlier than Qt 5.13
+    inline void setWebViewRequestInterceptor(WebView* webView, QWebEngineUrlRequestInterceptor* interceptor) {
+      webView->page()->profile()->setRequestInterceptor(interceptor);
+    }
+#else // Qt 5.13 and later
+    inline void setWebViewRequestInterceptor(WebView* webView, QWebEngineUrlRequestInterceptor* interceptor) {
+      webView->page()->setUrlRequestInterceptor(interceptor);
+    }
+#endif


If a non-portable functionality is used in a single place (and is not likely to be used somewhere else in the future) it doesn't need to go into portutils

veloman-yunkan · 2024-05-29T14:56:45Z

src/webview.h

+    virtual void interceptRequest(QWebEngineUrlRequestInfo &info) override
+    {
+        if (!m_isActive)
+            return;
+
+        const QString reqUrl = info.requestUrl().toString();
+        if (!reqUrl.startsWith("zim://"))
+        {
+            qDebug() << "Blocked external request to URL >"
+                    << reqUrl << "<";
+            info.block(true);
+        }
+    }


Move the implementation into webview.cpp.

src/settingsmanager.cpp

veloman-yunkan · 2024-05-29T15:14:47Z

You can add this code to TabBar::openUrl to add an image, sourced online, to the top of each article.

connect(webView, &QWebEngineView::loadFinished, [=](){
        webView->page()->runJavaScript(
            "var img = document.createElement(\"img\");"
            "img.src = \"https://picsum.photos/200/300\";"
            "var body = document.body;"
            "body.insertBefore(img, body.firstChild);"
        );
    });

I'd rather test using a ZIM file with references to external resources.

ShaopengLin · 2024-05-29T16:24:45Z

You can add this code to TabBar::openUrl to add an image, sourced online, to the top of each article.

connect(webView, &QWebEngineView::loadFinished, [=](){
        webView->page()->runJavaScript(
            "var img = document.createElement(\"img\");"
            "img.src = \"https://picsum.photos/200/300\";"
            "var body = document.body;"
            "body.insertBefore(img, body.firstChild);"
        );
    });

I'd rather test using a ZIM file with references to external resources.

Could you point me to one? I am not aware of how to filter Zims to find ones with external resources.

veloman-yunkan · 2024-05-29T16:49:39Z

I'd rather test using a ZIM file with references to external resources.

Could you point me to one? I am not aware of how to filter Zims to find ones with external resources.

I don't know of any such ZIM file off the top of my head and I don't even remember if I have seen one. As long as Kiwix is concerned a non-fully-self-contained ZIM file could only be created due to a bug in the scraper. @kelson42 and/or @rgaudin are more likely to point to such a file. If no such test data is available it can be created using zim-tools:

Extract contents of a ZIM file using zimdump
Modify some of its HTMLs
Create a new ZIM file using zimwriterfs.

Or you can take https://github.com/openzim/zim-tools/blob/main/test/data/zimfiles/create_test_zimfiles as a starting point and modify its source data to meet your requirements.

rgaudin · 2024-05-29T18:18:46Z

We surely have some zimit files that includes this but I can't point to one in particular.

Best is probably as suggested to create a custom one. zimwriterfs might even be simpler for this.

kelson42 · 2024-05-29T18:57:52Z

Added the toggle button to disable external content
Refresh all opened articles when the button is toggled.

@ShaopengLin Sorry but I only see that properly now. I see no scenario where this option should allow external loading. Do you? I believe this should not be any option. Sorry I shoukd have seen this "detail" in the original feature request.

ShaopengLin · 2024-05-29T19:03:54Z

Added the toggle button to disable external content
Refresh all opened articles when the button is toggled.

@ShaopengLin Sorry but I only see that properly now. I see no scenario where this option should allow external loading. Do you? I believe this should not be any option. Sorry I shoukd have seen this "detail" in the original feature request.

I agree. we shouldn't really allow external loading in the first place. Zim files are supposed to be the 'offline' versions of websites after all and are not supposed to generate any network traffic.

ShaopengLin · 2024-05-30T21:21:36Z

I used singleton for the class since it is more space-efficient.

Here is the test file, where the main page should try to load an image externally.
test_external.zip

src/tabbar.cpp

veloman-yunkan · 2024-05-31T12:01:20Z

The commit description better be changed too (in this case it will be quite appropriate to tell what you have achieved instead of how you did it).

Webpages blocks all requests with Url not starting with "zim://", i.e. native to the Zim file.

kelson42 · 2024-06-01T12:52:28Z

CI broken because of #1121, but this has nothing to do with this PR. Therefore merging.

ShaopengLin force-pushed the Issue#904-block-non-zim-external-requests branch 2 times, most recently from 0558bd9 to f46e0df Compare May 23, 2024 08:04

ShaopengLin mentioned this pull request May 23, 2024

Fixed Whiteouts of Book Descriptions on Hover #1115

Merged

kelson42 requested a review from veloman-yunkan May 23, 2024 08:17

veloman-yunkan requested changes May 29, 2024

View reviewed changes

ShaopengLin force-pushed the Issue#904-block-non-zim-external-requests branch from f46e0df to ce8aef3 Compare May 30, 2024 21:01

ShaopengLin requested a review from veloman-yunkan May 30, 2024 21:35

veloman-yunkan reviewed May 31, 2024

View reviewed changes

src/tabbar.cpp Outdated Show resolved Hide resolved

veloman-yunkan changed the title ~~Added Setting to Block External Content from Zim Web Pages~~ Block External Content from Zim Web Pages May 31, 2024

ShaopengLin force-pushed the Issue#904-block-non-zim-external-requests branch from ce8aef3 to fcfa124 Compare June 1, 2024 06:01

ShaopengLin requested a review from veloman-yunkan June 1, 2024 06:01

ShaopengLin force-pushed the Issue#904-block-non-zim-external-requests branch from fcfa124 to 33582f3 Compare June 1, 2024 06:02

veloman-yunkan approved these changes Jun 1, 2024

View reviewed changes

Disables external resource fetching

f8d646d

Webpages blocks all requests with Url not starting with "zim://", i.e. native to the Zim file.

kelson42 force-pushed the Issue#904-block-non-zim-external-requests branch from 33582f3 to f8d646d Compare June 1, 2024 12:47

kelson42 merged commit 79efd58 into main Jun 1, 2024
1 of 4 checks passed

kelson42 deleted the Issue#904-block-non-zim-external-requests branch June 1, 2024 12:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Block External Content from Zim Web Pages #1116

Block External Content from Zim Web Pages #1116

ShaopengLin commented May 23, 2024 •

edited

Loading

kelson42 commented May 28, 2024

veloman-yunkan left a comment

veloman-yunkan May 29, 2024

veloman-yunkan May 29, 2024

veloman-yunkan May 29, 2024

veloman-yunkan May 29, 2024

veloman-yunkan commented May 29, 2024

ShaopengLin commented May 29, 2024

veloman-yunkan commented May 29, 2024

rgaudin commented May 29, 2024

kelson42 commented May 29, 2024 •

edited

Loading

ShaopengLin commented May 29, 2024

ShaopengLin commented May 30, 2024

veloman-yunkan commented May 31, 2024 •

edited

Loading

kelson42 commented Jun 1, 2024

		qDebug() << "Blocked external request to URL >"
		<< reqUrl << "<";

Block External Content from Zim Web Pages #1116

Block External Content from Zim Web Pages #1116

Conversation

ShaopengLin commented May 23, 2024 • edited Loading

kelson42 commented May 28, 2024

veloman-yunkan left a comment

Choose a reason for hiding this comment

veloman-yunkan May 29, 2024

Choose a reason for hiding this comment

veloman-yunkan May 29, 2024

Choose a reason for hiding this comment

veloman-yunkan May 29, 2024

Choose a reason for hiding this comment

veloman-yunkan May 29, 2024

Choose a reason for hiding this comment

veloman-yunkan commented May 29, 2024

ShaopengLin commented May 29, 2024

veloman-yunkan commented May 29, 2024

rgaudin commented May 29, 2024

kelson42 commented May 29, 2024 • edited Loading

ShaopengLin commented May 29, 2024

ShaopengLin commented May 30, 2024

veloman-yunkan commented May 31, 2024 • edited Loading

kelson42 commented Jun 1, 2024

ShaopengLin commented May 23, 2024 •

edited

Loading

kelson42 commented May 29, 2024 •

edited

Loading

veloman-yunkan commented May 31, 2024 •

edited

Loading