Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Duplicate images show in the content of theverge.com's feed with add_dynamic_image and fetch original content enabled #1979

Closed
kris-anderson opened this issue Jul 12, 2023 · 3 comments
Closed

Duplicate images show in the content of theverge.com's feed with add_dynamic_image and fetch original content enabled #1979

kris-anderson opened this issue Jul 12, 2023 · 3 comments
Labels
feed problems

Comments

@kris-anderson
Copy link

Here is a link to the feed in question:
https://www.theverge.com/rss/index.xml

Fetch original content is enabled.

For rewrite rules, I have the following:

remove(".duet--recirculation--related-list"), add_dynamic_image

Without add_dynamic_image, no images show in the body of the content. I have also tried the option use_noscript_figure_images, and that option does not get the images to show up either.

What does this end up looking like? Here's a couple examples:
image

image

Here's the HTML on one of those pages that contains the image:

<a class="relative block aspect-square h-min w-full sm:float-left sm:mb-20 sm:mr-20 sm:w-[300px] sm:min-w-[300px]"
  href="https://www.amazon.com/gp/product/B08MQZXN1X/?tag=theverge02-20&amp;ascsubtag=___vg__p_23537924__t_w__d_D"
  rel="sponsored nofollow noopener" target="_blank"><span style="
      box-sizing: border-box;
      display: block;
      overflow: hidden;
      width: initial;
      height: initial;
      background: none;
      opacity: 1;
      border: 0;
      margin: 0;
      padding: 0;
      position: absolute;
      top: 0;
      left: 0;
      bottom: 0;
      right: 0;
    "><img
      alt="A close-up of the Amazon Fire TV Stick 4K Max streaming dongle beside its remote control on a brown couch."
      src="https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/2400x2400/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg"
      decoding="async" data-nimg="fill" style="
        position: absolute;
        top: 0;
        left: 0;
        bottom: 0;
        right: 0;
        box-sizing: border-box;
        padding: 0;
        border: none;
        margin: auto;
        display: block;
        width: 0;
        height: 0;
        min-width: 100%;
        max-width: 100%;
        min-height: 100%;
        max-height: 100%;
        object-fit: cover;
      " sizes="(max-width: 768px) 100vw, 300px" srcset="
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/376x376/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    376w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/384x384/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    384w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/415x415/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    415w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/480x480/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    480w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/540x540/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    540w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/640x640/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    640w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/750x750/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    750w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/828x828/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg    828w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1080x1080/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg 1080w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1200x1200/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg 1200w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1440x1440/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg 1440w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1920x1920/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg 1920w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/2048x2048/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg 2048w,
        https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/2400x2400/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg 2400w
      " /><noscript>&lt;img alt="A close-up of the Amazon Fire TV Stick 4K Max streaming
      dongle beside its remote control on a brown couch." loading="lazy"
      decoding="async" data-nimg="fill"
      style="position:absolute;top:0;left:0;bottom:0;right:0;box-sizing:border-box;padding:0;border:none;margin:auto;display:block;width:0;height:0;min-width:100%;max-width:100%;min-height:100%;max-height:100%;object-fit:cover"
      sizes="(max-width: 768px) 100vw, 300px"
      srcSet="https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/376x376/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      376w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/384x384/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      384w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/415x415/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      415w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/480x480/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      480w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/540x540/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      540w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/640x640/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      640w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/750x750/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      750w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/828x828/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      828w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1080x1080/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      1080w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1200x1200/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      1200w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1440x1440/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      1440w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/1920x1920/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      1920w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/2048x2048/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      2048w,
      https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/2400x2400/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg
      2400w"
      src="https://duet-cdn.vox-cdn.com/thumbor/0x0:2040x1360/2400x2400/filters:focal(1020x680:1021x681):format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/22909969/DSCF5782.jpg"/&gt;</noscript></span></a>

Is this a an issue with the full text conversion process of how it's handling whatever html nonsense theverge.com is using? Or maybe there's a rewrite rule I can use that I'm missing?
@whaler-ragweed
Copy link

I have the same problem.

@dzaikos
Copy link
Contributor

dzaikos commented Nov 20, 2023 •
edited
Loading

I reviewed the Miniflux code as well as The Verge's HTML. Miniflux is working correctly, the problem is that The Verge web site (the full content loaded when "fetch original content" is checked) actually does have the image twice: a lower and higher resolution.

The higher resolution image is encapsulated in a div tag with a hidden CSS class. This is probably a full-screen image that is shown when a user clicks the lower-resolution version.

My suggestion is to use the remove() rewrite rule to get rid of the hidden image:

remove(".hidden")

I have not tested this but it should resolve the issue.

@kris-anderson
Copy link
Author

Thank you so much for looking into this @dzaikos. I was able to verify that your suggestion does in fact fix this issue!

For anyone that happens to find this in the future, here is what I'm using for the rewrite rule:

remove(".duet--recirculation--related-list, .hidden"), add_dynamic_image, add_youtube_video

hyyyjinx pushed a commit to hyyyjinx/v2 that referenced this issue Jun 9, 2024
Update theverge.com rewrite rule: fix duplicate image
cc7d03b 
See: miniflux#1979
@hyyyjinx hyyyjinx mentioned this issue Jun 9, 2024
Merged
5 tasks
fguillot pushed a commit that referenced this issue Jun 11, 2024
@fguillot
Update theverge.com rewrite rule: fix duplicate image
0bab8fa 
See: #1979
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feed problems
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kris-anderson @dzaikos @whaler-ragweed