[fix] replace embedded HTML by data_src

[return42]

What does this PR do?

[mod] result_templates/videos.html replace embedded HTML by data_src
[mod] result_templates/default.html replace embedded HTML by data_src

Embedded HTML breaks SearXNG architecture. To modularize, HTML is generated in
the templates (oscar & simple) and result parameter 'embedded' is replaced by
'data_src', an URL for embedded content (<iframe>).

[mod] hostname_replace: replace hostnames in result's data_src param

How to test this PR locally?

The (iframe & audio) templates of the following engines has been touched (oscar & simple theme)

• searx/engines/dailymotion.py
• searx/engines/invidious.py
• searx/engines/peertube.py --> Peertube URL is offline #881
• searx/engines/sepiasearch.py
• searx/engines/vimeo.py
• searx/engines/youtube_api.py
• searx/engines/bandcamp.py
• searx/engines/deezer.py
• searx/engines/freesound.py
• searx/engines/mixcloud.py
• searx/engines/soundcloud.py

To test you need to redirect embedded videos (e.g.) from youtube to a invidios
instance. Search for videos using engine !youtube lebowski. The result URLs
and the embedded videos should link to the invidios instance.

Here is an example of such a hostname_replace configuration::

    hostname_replace:

      # youtube --> Invidious

      '(.*\.)?youtube-nocookie\.com': 'invidio.xamh.de'
      '(.*\.)?youtube\.com$': 'invidio.xamh.de'
      '(.*\.)?invidious\.snopyta\.org$': 'invidio.xamh.de'
      '(.*\.)?vid\.puffyan\.us': 'invidio.xamh.de'
      '(.*\.)?invidious\.kavin\.rocks$': 'invidio.xamh.de'
      '(.*\.)?inv\.riverside\.rocks$': 'invidio.xamh.de'

Related issues

Closes: #873

[dalf]

To modularize, HTML is generated in the templates (oscar & simple)

💯 👍

result parameter 'embedded' is replaced by 'data_src', an URL for embedded content (<iframe>).

IMO: embedded is clearer than data_src. Why choose data_src?

---

The vertical video are fine:

---

There are issues with the height of the audio media:

In the case of audio, there is a blank space at the bottom of the iframe:

In some other cases the iframe is too small:

Possible solution: the engine returns an additional field named embedded_height.

[dalf]

See my comments

[return42]

result parameter 'embedded' is replaced by 'data_src', an URL for embedded content (<iframe>).

IMO: embedded is clearer than data_src. Why choose data_src?

IMO embedded is not clear, at least when we look at the default.html template ...

{% if result.data_src -%}
<div id="result-media-{{ index }}" class="embedded-content invisible">
  <iframe data-src="{{result.data_src}}" frameborder="0" allowfullscreen></iframe>
</div>
{%- endif %}
{% if result.audio_src -%}
<div id="result-media-{{ index }}" class="audio-control">
  <audio controls><source src="{{result.audio_src}}"></audio>
</div>
{%- endif %}

... where we have an embedded <iframe> and an embedded audio control.

That's why I was looking for a name that better fits to the template. I don't know if data_src and audio_src are good names. If you have better suggestions, we can change the names, but whatever names we choose they should point out their usage in the templates. And we should consider that we might have other embedded content in the future, content we do not think about today. All in all I think embedded as a name is to general.

[dalf]

what about iframe_src and audio_src?
or embedded_iframe and embedded_audio?

data-src is already used to lazy load images:

searxng/searx/templates/simple/result_templates/images.html

Line 12 in a494221

<img src="" data-src="{{ image_proxify(result.img_src) }}" alt="{{ result.title|striptags }}">

[return42]

what about iframe_src and audio_src?

I renamed data_src to iframe_src in 9c46856

Possible solution: the engine returns an additional field named embedded_height.

I don't want the layout hard coded by the engines / therefore I added 1d72ada

---

@dalf: thanks for your review and helping comments .. I fixed all your requests or leaved a comment at least.

PR is now ready for the next review cycle ..

[dalf]

After few days rereading the code, I end up to think there is no need to for a custom.less file (for now).

The code inside this file can simplified, see the comments.

After that, I think the PR is ready.

[dalf]

The host names and other CSS class names are overlapping. While the chance there is an actual overlap is small, it would be better to avoid to take the risk.

It is possible to use CSS attribute selector: https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors

iframe[src^="https://w.soundcloud.com"] {
  height: 120px;
}

iframe[src^="https://www.deezer.com"] {
  // The real size is 92px, but 94px are needed to avoid an inner scrollbar of
  // the embedded HTML.
  height: 94px;
}

iframe[src^="https://www.mixcloud.com"] {
  // the embedded player from mixcloud has some quirks: initial there is an
  // issue with an image URL that is blocked since it is an a Cross-Origin
  // request. The alternative text (<img alt='Mixcloud Logo'> then cause an
  // scrollbar in the inner of the iframe we can't avoid.  Another quirk comes
  // when pressing the play button, somtimes the shown player has an height of
  // 200px, somtimes 250px.
  height: 250px;
}

So there is no need

• to add kwargs['urlparse'] = urllib.parse.urlparse
• to call urlparse from video.html and default.html (iframe and audio element).

( changes done in commit 1d72ada )

[return42]

Thanks, implemented in 8acf540

[dalf]

Currently it is okay.
However, some engines return YouTube videos (like qwant, google video, bing video ?)
Currently the iframe_src attribute is not set for these engines.
If in the future, this is attribute is set , this condition is going to prevent the hostname replacement in the iframe_src attribute.

[return42]

I rewrote this commit, see 795e8af

[return42]

The code inside this file can simplified, see the comments.
After that, I think the PR is ready.

@dalf I dropped several commits and rewrote others, all I changed is documented in the answers to your comments above.

[dalf]

Thank you for this contribution.
It fixes an long standing issue.

---

In addition to the test described, I have check this configuration:

hostname_replace:
  '(.*\.)?youtube-nocookie\.com': false

👍 it works