Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only first accomplishment returned #102

Closed
lhl2617 opened this issue Jun 4, 2020 · 2 comments
Closed

Only first accomplishment returned #102

lhl2617 opened this issue Jun 4, 2020 · 2 comments

Comments

@lhl2617
Copy link

lhl2617 commented Jun 4, 2020

Hi,

When scraping for accomplishment sections for detailed accomplishments (added in #89, by @EmaSuriano), only the first item is returned. Was wondering if you are seeing the same bug.

Thanks

@lhl2617
Copy link
Author

lhl2617 commented Jun 4, 2020

I just need this for an ad-hoc projects scrape, so this is code that works for me

const scrapProjectsPanel = async (page) => {
    const openingButton = await page.$(
        `.pv-accomplishments-block.projects button`,
    );

    if (openingButton) {
        await openingButton.click();
        await new Promise((resolve) => {
            setTimeout(() => {
                resolve();
            }, 500);
        });

        let showMoreButton;

        while (showMoreButton = await page.$(`.pv-profile-section__see-more-inline`)) {
            await showMoreButton.click();
            await new Promise((resolve) => {
                setTimeout(() => {
                    resolve();
                }, 250);
            });
        }

        const selectors = await page.$$(`.pv-accomplishment-entity`)

        const got = await Promise.all(selectors.map((selector) =>
            Object.keys(template.projects.fields)
                .reduce(scrapSelectorFields(selector, template.projects), Promise.resolve({}))
        ))

        
        return got;
    }
}

I will PR when I have time - but if others have time-critical uses for this, please go ahead and PR for me

@cmtr
Copy link

cmtr commented Jun 28, 2020

Proposal for this issue in PR #118

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants