Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhacement] Use Flexible ascii progress bar. #4

Closed
LeMoussel opened this issue Apr 10, 2022 · 2 comments
Closed

[enhacement] Use Flexible ascii progress bar. #4

LeMoussel opened this issue Apr 10, 2022 · 2 comments

Comments

@LeMoussel
Copy link

For a better understanding of the progress of the proxies checks, here is an example of code using an flexible ascii progress bar: node-progress.

Rem: Pool is not accessible via require('free-proxy-checker')

// https://github.com/antoinevastel/free_proxy_checker
const { ProxyChecker, downloadAllProxies, } = require('free-proxy-checker');
// https://github.com/visionmedia/node-progress
const ProgressBar = require('progress');

const { Pool } = require('./pool.js');

class ProgressProxyChecker extends ProxyChecker {
    async checkProxies() {
        this.lastCheck = Date.now();
        const pool = new Pool(this.options.concurrency);
        const pbar = new ProgressBar('Checking proxies [:bar] :percent :etas', {
            complete: '=',
            incomplete: ' ',
            width: 20,
            total: this.proxies.length
        });

        this.proxies.forEach((proxy) => {
            pool.addTask(async() => {
                try {
                    await proxy._testConnection(this.options.timeout);
                } catch (_) { }
                finally {
                    if (this.options.verbose) {
                        pbar.tick();
                    }
                }
            })
        })

        await pool.run();
    }
}

(async () => {
    // Clear console
    process.stdout.write('\033c');

    // Download all proxies from all proxy providers at once
    const allProxies = await downloadAllProxies();
    console.log(`There are ${allProxies.length} proxies to test`);

    // Check the availability of the proxies we downloaded
    const proxyChecker = new ProgressProxyChecker(allProxies, {
        concurrency: 50,
        timeout: 7500,
        verbose: true
    })

    await proxyChecker.checkProxies();
    const proxiesUp = proxyChecker.getProxiesUp();
    console.log(`There are ${proxiesUp.length} proxies UP:`);
})();

Result:
image

@antoinevastel
Copy link
Owner

Hi, thanks for your idea. Since node-progress is a single file + seems really stable I think it's reasonable to add it as a dependency without going through package.json.
I keep the issue open. I will probably do a PR in the next days/weeks :)

@antoinevastel
Copy link
Owner

I close the issue, I added the feature in this PR (#5) + publish new version on NPM. Once again thanks for idea + the code example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants