Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Page download failing for Youtube when using browser cookie #24

Open
Spicadox opened this issue Mar 14, 2023 · 2 comments
Open

Page download failing for Youtube when using browser cookie #24

Spicadox opened this issue Mar 14, 2023 · 2 comments

Comments

@Spicadox
Copy link

single-file-cli is failing when using it on a Youtube Community page with the below error and only failing strictly when the browser height is too large(>~50000 pixels) and browser cookie is used from an account that has membership of the channel.

Command: single-file --browser-cookies-file="I:\archive scripts\batch scripts\singlefile_script\cookie.txt" --browser-executable-path="C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --browser-script="I:\archive scripts\batch scripts\singlefile_script\replace_image_url_script.js" --browser-load-max-time=120000 --browser-wait-delay=20000 --browser-height=200000 --crawl-replace-urls=true "https://www.youtube.com/channel/UCqm3BQLlJfvkTsX_hvm0UmA/community"

Error:

Evaluation failed: RangeError: Invalid string length
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479 URL: https://www.youtube.com/channel/UCqm3BQLlJfvkTsX_hvm0UmA/community
Stack: Error: Evaluation failed: RangeError: Invalid string length
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at Array.forEach (<anonymous>)
    at <anonymous>:1:239464
    at cy (<anonymous>:1:239632)
    at <anonymous>:1:239479
    at ExecutionContext._ExecutionContext_evaluate (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:229:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async ExecutionContext.evaluate (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:107:16)
    at async getPageData (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\back-ends\puppeteer.js:150:10)
    at async Object.exports.getPageData (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\back-ends\puppeteer.js:56:10)
    at async capturePage (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\single-file-cli-api.js:254:20)
    at async runNextTask (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\single-file-cli-api.js:175:20)
    at async Promise.all (index 0)
    at async capture (C:\Users\test\AppData\Roaming\npm\node_modules\single-file-cli\single-file-cli-api.js:126:2)
@gildas-lormeau
Copy link
Owner

Is it possible for me to reproduce the issue?

@Spicadox
Copy link
Author

Spicadox commented Mar 16, 2023

I guess not, unless you're willing to use my cookie file, then I could dm you my cookie file.

After looking into it more, I found that my script file may also be contributing to the issue. I may be grasping at straws and since you haven't reproduced the issue you probably won't know, but my script basically ensures the image quality is at the highest plus with cookies there would be more images and content populating the page. As such, for a huge webpage with lots of images, I'm guessing this could cause an issue since removing either my cookies or script in the command(so either less data being saved or a very large image is perhaps causing issue) won't result in the error?

edit: Manually running each part of my script on chrome and then using the single file extension seems to work though so I'm at a lost

edit2: Sorry for being a pain but setting images to a smaller size of max height of 720 fixed the issue so I am guessing large image(s) is causing issue with base64 string being too large?

Script in question(not great with javascript, but it's been doing it's job fine on other smaller pages):

// ==UserScript==
// @name         Change images' resolution
// @version      1.0
// @description  [SingleFile] Replace img url with a higher resolution version
// ==/UserScript==



(() => {
    // Get the image element that has a src attribute and an empty alt(use to ignore profile images)
    const imgElementsSelector = "img[src]:not([src=''])";

    // Regex grouping[0] is the original string
    // Group[1] is the entire string before =s where everything after the domain name is between 30-85 characters
    // Group[2] will be used and set to 0 to obtain the max resolution of the image
    const pattern = "(https:\/\/yt3\.(ggpht|googleusercontent)\.com\/.{30,85})(=[a-z]{1})(.*)";

    const DELAY = 10000;

    function waitForElm(selector) {
        return new Promise(resolve => {
            if (document.querySelector(selector)) {
                return resolve(document.querySelector(selector));
            }

            const observer = new MutationObserver(mutations => {
                if (document.querySelector(selector)) {
                    resolve(document.querySelector(selector));
                    observer.disconnect();
                }
            });

            observer.observe(document.body, {
                childList: true,
                subtree: true
            });
        });
    }

    function unCollapse() {
        // uncollapse show perks info on membership pages(to show stamps)
        try {
            document.querySelector("ytd-sponsorships-expandable-perks-renderer.style-scope.ytd-section-list-renderer").removeAttribute("is-collapsed");
            console.log("Uncollapsed")
            changeImageQuality()
        } catch (error) {
            console.log("Unable to find or remove show perk info button");
            console.error(error);
        }
    }
    function changeImageQuality() {
        // Only want images in the content element
        let content = document.getElementById("contents").getElementsByClassName("style-scope ytd-section-list-renderer")
        for (let i = 0; i < content.length; i++) {
            content[i].querySelectorAll(imgElementsSelector).forEach(element => {
                try {
                    let img_src = element.getAttribute("src");
                    group_match = img_src.match(pattern);
                    new_img_src = group_match[1] + '=s0';
                    element.setAttribute("src", new_img_src);
                } catch (error) {
                    console.log("Encountered Error");
                    console.error(error);
                }
            })
        }
    }
    dispatchEvent(new CustomEvent("single-file-user-script-init"));
    addEventListener("single-file-on-before-capture-request",  async event => {
        //Wait for the element before uncollapse 
        try {
            waitForElm('ytd-sponsorships-expandable-perks-renderer.style-scope.ytd-section-list-renderer').then((elm) => {
                console.log('Element is ready');
                unCollapse()
            });
        } catch (error) {
            console.error(error)
        }
        //Remove multi-image box items
        try {
            //remove the image icon on the bottom right of the images
            document.querySelectorAll("yt-icon.style-scope.ytd-backstage-image-renderer").forEach(element => {
                element.remove();
            })
            //remove the right arrow container and all
            document.querySelectorAll("div.arrow-container.style-scope.ytd-post-multi-image-renderer").forEach(element => {
                element.remove();
            })
            //remove the box container
            document.querySelectorAll(".style-scope ytd-post-multi-image-renderer #items").forEach(element => {
                element.removeAttribute("class");
            })
        } catch (error) {
            console.log("Unable to remove multi-image box items");
            console.error(error);
        }

        //Wait for the membership images to load in
        try {
            event.preventDefault();
            await new Promise(resolve => setTimeout(resolve, DELAY));
        } catch (error) {
            console.error(error)
        }

        changeImageQuality()
        // Wait for the page to load after the previous changes before capturing
        event.preventDefault();
        try {
            await new Promise(resolve => setTimeout(resolve, DELAY));
        } finally {
            dispatchEvent(new CustomEvent("single-file-on-before-capture-response"));
        }
        console.log("Finished script");
        return;
    });
})();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants