[crawler] Add types for crawler

[pzmarzly]

This PR adds types for crawler (npm, GitHub). Most of its development happened in the years 2014-2016. Later, @raynor85 and @wakhh published their TS augmentation, which I took and updated to get it to work on modern TS compilers.

The types are not 100% accurate:

• there are some options that library accepts but are not typed here,
• user can choose not to use cheerio, in which case $ is of different type.

---

By the way, it seems like packages that types for request are broken, which causes its dependents' tests to fail. For an example, run npm test artillery in the DT repo.

$ npm test artillery

> definitely-typed@0.0.3 test /home/pzmarzly/Projects/DefinitelyTyped
> dtslint types "artillery"

Error: Errors in typescript@4.1 for external dependencies:
../request/index.d.ts(24,27): error TS7016: Could not find a declaration file for module 'form-data'. '/home/pzmarzly/Projects/DefinitelyTyped/node_modules/form-data/lib/form_data.js' implicitly has an 'any' type.

---

Select one of these and delete the others:

If adding a new definition:

• The package does not already provide its own types, or cannot have its .d.ts files generated via --declaration
• If this is for an NPM package, match the name. If not, do not conflict with the name of an NPM package.
• Create it with dts-gen --dt, not by basing it on an existing project.
• Represents shape of module/library correctly
• tslint.json should be present and it shouldn't have any additional or disabling of rules. Just content as { "extends": "dtslint/dt.json" }.
• tsconfig.json should have noImplicitAny, noImplicitThis, strictNullChecks, and strictFunctionTypes set to true.

[typescript-bot]

@pzmarzly Thank you for submitting this PR!

This is a live comment which I will keep updated.

1 package in this PR

• crawler (new!) on npm, on unpkg

Code Reviews

This PR adds a new definition, so it needs to be reviewed by a DT maintainer before it can be merged.

Status

• ✅ No merge conflicts
• ✅ Continuous integration tests have passed
• ❌ Only a DT maintainer can approve changes when there are new packages added

Once every item on this list is checked, I'll ask you for permission to merge and publish the changes.

---

Diagnostic Information: What the bot saw about this PR

{
  "type": "info",
  "now": "-",
  "pr_number": 48692,
  "author": "pzmarzly",
  "owners": [],
  "dangerLevel": "ScopedAndConfiguration",
  "headCommitAbbrOid": "a9051ae",
  "headCommitOid": "a9051ae4cfc02c4cf4f9e6f67f28d61ac830a1f5",
  "mergeIsRequested": false,
  "stalenessInDays": 1,
  "lastPushDate": "2020-10-11T15:34:21.000Z",
  "lastCommentDate": "2020-10-11T15:34:21.000Z",
  "maintainerBlessed": false,
  "reviewLink": "https://github.com/DefinitelyTyped/DefinitelyTyped/pull/48692/files",
  "hasMergeConflict": false,
  "authorIsOwner": false,
  "isFirstContribution": false,
  "popularityLevel": "Well-liked by everyone",
  "newPackages": [
    "crawler"
  ],
  "packages": [
    "crawler"
  ],
  "files": [
    {
      "path": "types/crawler/crawler-tests.ts",
      "kind": "test",
      "package": "crawler"
    },
    {
      "path": "types/crawler/index.d.ts",
      "kind": "definition",
      "package": "crawler"
    },
    {
      "path": "types/crawler/tsconfig.json",
      "kind": "package-meta",
      "package": "crawler",
      "suspect": "created"
    },
    {
      "path": "types/crawler/tslint.json",
      "kind": "package-meta-ok",
      "package": "crawler"
    }
  ],
  "hasDismissedReview": false,
  "ciResult": "pass",
  "reviewersWithStaleReviews": [],
  "approvalFlags": 0,
  "isChangesRequested": false
}

[typescript-bot]

🔔 @pzmarzly — you're the only owner, but it would still be good if you find someone to review this PR in the next few days, otherwise a maintainer will look at it. (And if you do find someone, maybe even recruit them to be a second owner to make future changes easier...)

[typescript-bot]

👋 Hi there! I’ve run some quick measurements against master and your PR. These metrics should help the humans reviewing this PR gauge whether it might negatively affect compile times or editor responsiveness for users who install these typings.

Let’s review the numbers, shall we?

These typings are for a package that doesn’t yet exist on master, so I don’t have anything to compare against yet! In the future, I’ll be able to compare PRs to crawler with its source on master.

Comparison details 📊

Batch compilation
Type count	9360
Assignability cache size	1676
Language service measurements
Samples taken	176
Identifiers in tests	176
getCompletionsAtPosition
Mean duration (ms)	376.5
Mean CV	9.7%
Worst duration (ms)	454.8
Worst identifier	console
getQuickInfoAtPosition
Mean duration (ms)	379.2
Mean CV	10.3%
Worst duration (ms)	453.2
Worst identifier	log
System information
Node version	v12.18.4
CPU count	2
CPU speed	2.294 GHz
CPU model	Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
CPU Architecture	x64
Memory	6.8 GiB
Platform	linux
Release	4.15.0-1096-azure

[orta]

Nice work @pzmarzly - external types like request can have a less strict set of compiler flags which is what cause that, though I'm not sure what you did to work around it. Everything seems green now. The tests look good and the d.ts looks good, thanks

[typescript-bot]

I just published @types/crawler@1.2.0 to npm.