Headless chrome rendering to check malicious links sent by user

So when a link is sent in the server, the link can be used with a command to run it with rendertron, a headless chrome rendering service written in javascript, with a little tinkering it works with Python too, (tried and tested), for example, if a grabify link is sent, the IP registered is that of the rendertron link(tried and tested again)

If I get an approval from maintainers I could start working on it :D

[onerandomusername]

We have GH-1961, unfurling redirects.

Is this what you were meaning we implement?

I forgot to mention what it does, I'm dumb, but coming to the topic, it sends a screenshot of what the link has, suppose I send you a link: https://google.com it runs a headless chrome browser and takes a scrennshot.

[HassanAbouelela]

If we were to implement this, I don’t think we should do it from the bot, but rather from a worker (doing it from the bot leaves us open to any vulnerabilities presented by the app). That means it’ll most likely be JavaScript anyways.

Why rendertron? I see it uses puppeteer, what’s the benefit of using it over puppeteer directly? If we do this in python, why not a python specific library?

Finally, what’s the motivation behind this command, aka, what use cases are you envisioning for this.

As for rendertron, calling the api is super easy and can be done in few lines of code, I tried doing the same in various other libraries but to no avail, it needs selenium webdriver

Use cases are plenty, for starters, when someone posts a link in the server to check the contents of the website we could use this command and then it sends us the screenshot, and verify if the link is safe, and as for grabify links and other malicious links of the sorts could be verified using this.

[HassanAbouelela]

That benefit you’re explaining comes at a pretty hefty cost for devs, since you’re trading install difficulty (it’ll still be difficult because we need to add npm), with storage (puppeteer on its own is 300mb on average?).

As for grabify links, they’re outright banned from being posted on the server, so it won’t serve that use case. If you’re looking for a general utility to do this, one of the existing websites that do this seem a better fit.

To expand further on the above point: we usually ban malicious links.

Rendertron doesn't need npm to be run, it's written in javascript agreed, it can be used using requests and url lib, but again since it fails the use case ig it will be of no use, grabify links can also be disguised as normal links for a few extra bucks

[HassanAbouelela]

I’m looking at their GitHub, and I’m not seeing any binaries, so I assume you need npm to install it

I shall be sending the code implementation for python in sometime.

def web_scr(link:str) -> None:
    BASE = 'https://render-tron.appspot.com/screenshot/'
    url = link
    path = 'ScreenShot.jpg'
    response = requests.get(BASE + url, stream=True)
    if response.status_code == 200:
        with open(path, 'wb') as file:
            for chunk in response:
                file.write(chunk)

this is an API implementation of it
oh and ofcourse, import requests

[HassanAbouelela]

Well, this isn’t running on python, the api is hosted by someone, and they’re running it via JavaScript. We wouldn’t use it in our bot, as they ask people not to. If we were to use it we’d host our own, which is what I was alluding to with CF workers.

Regardless, I don’t think this is something that will be worth pursuing with the limitations listed above. If a demand for it presents itself in the future, it’ll most likely be from the mod team, and we can reconsider then.

Thank you for your patience and time. :D