Skip to content

feat: stealth deprecation#1314

Merged
B4nan merged 7 commits intomasterfrom
feature/stealth-deprecation
Mar 28, 2022
Merged

feat: stealth deprecation#1314
B4nan merged 7 commits intomasterfrom
feature/stealth-deprecation

Conversation

@petrpatek
Copy link
Copy Markdown
Contributor

No description provided.

@petrpatek petrpatek changed the title Feature stealth deprecation feat: stealth deprecation Mar 14, 2022

Changing browser fingerprints can be a tedious job. Luckily, Apify SDK provides this feature out of the box with zero configuration necessary. Let's take a look at how it is done.

Changing browser fingerprints is available in `PuppeteerCrawler` and `PlaywrightCrawler`. You have to pass the `useFingerprints` option to the `browserPoolOptions`.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formatting and indenting is messed up in this file, also lets use the same code style in examples as in the rest of the docs and codebase:

  • spaces inside object literals { ... }
  • space after : (e.g. browserPoolOptions: {)
  • trailing commas
  • single quotes

Comment thread docs/guides/avoid_blocking.md Outdated
TODO: this is only for browser-based scraping. We should reflect in the name somehow... IMHO, browser fingerprinting does not remember that it reduces blocking.


Your scraper might get blocked for numerous reasons. Let's narrow it down to two main reasons. The first one is a bad or blocked IP address. This topic is covered in the [proxy management guide](proxy_management.MD). The second reason we will explore more is browser fingerprints or signatures.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to use we/our then you/your, or even just A scraper might or Scrapers might.

also the link is wrong, .MD vs .md

const { hideWebDriver, ...newStealthOptions } = this.stealthOptions;

log.deprecated(
'Puppeteer "stealth" and "stealthOptions" is deprecated'
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are deprecated, also it looks like you are missing a dot and a space in here, this would result in ... is deprecatedYou should ...

Comment thread website/i18n/en.json Outdated
Comment on lines +20 to +42
"api/browser-crawler": {
"title": "BrowserCrawler"
},
"api/browser-plugin": {
"title": "BrowserPlugin"
},
"api/cache-container": {
"title": "CacheContainer"
},
"api/cheerio-crawler": {
"title": "CheerioCrawler"
},
"api/configuration": {
"title": "Configuration"
},
"api/cookie-parse-error": {
"title": "CookieParseError"
},
"api/dataset": {
"title": "Dataset"
},
"api/exports": {
"title": "exports"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what this is, but it feels wrong :)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think it matters much, this is just some translation file

(but its indeed weird)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't even know where it came from. I remember running only generate and start commands.

@petrpatek
Copy link
Copy Markdown
Contributor Author

@B4nan Do you know how to edit the examples? I want to add the deprecation notice also to the stealth example.

@B4nan
Copy link
Copy Markdown
Member

B4nan commented Mar 15, 2022

Check the generate script, its in another repo

@B4nan
Copy link
Copy Markdown
Member

B4nan commented Mar 21, 2022

Is this still draft on purpose?

One small thing I am missing are @deprecated jsdoc comments on those options, so we get also IDE warnings in place.

@petrpatek petrpatek marked this pull request as ready for review March 22, 2022 06:36
Comment thread docs/guides/avoid_blocking.md Outdated
@B4nan B4nan merged commit 7d20ac2 into master Mar 28, 2022
@B4nan B4nan deleted the feature/stealth-deprecation branch March 28, 2022 11:38
@B4nan B4nan mentioned this pull request Mar 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants