Skip to content

Commit 9d3745b

Browse files
committed
Remove autoload
1 parent 76d5329 commit 9d3745b

File tree

109 files changed

+1109
-481
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

109 files changed

+1109
-481
lines changed

README.md

Lines changed: 6 additions & 118 deletions
Original file line numberDiff line numberDiff line change
@@ -131,116 +131,12 @@ Rules work as fallback between them:
131131

132132
**metascraper** do that until finish all the rule or find the first rule that resolves the value.
133133

134-
## Loading rules
135-
136-
When you call **metascraper** in your code, a set of [core rules](#core-rules) are loaded by default.
137-
138-
Although these rules are sufficient for most cases, **metascraper** was designed to be easy to adapt and load more or custom rules set.
139-
140-
We provide two approach for do that.
141-
142-
### Configuration file
143-
144-
This consists in declaring a configuration file that contains the names of the rule sets corresponding to npm packages that **metascraper** will be load automagically.
145-
146-
The configuration file could be declared via:
147-
148-
- A `.metascraperrc` file, written in YAML or JSON, with optional extensions: **.yaml**, **.yml**, **.json** and **.js**.
149-
- A `metascraper.config.js` file that exports an object.
150-
- A **metascraper** key in your `package.json` file.
151-
152-
The configuration file will be resolved starting from the location of the file being formatted, and searching up the file tree until a config file is (or isn't) found.
153-
154-
The order of rules are loaded are important: Just the first rule that resolve the value will be applied.
155-
156-
#### Basic Configuration
157-
158-
Declared an **array** of **rules**, specifying each rule as **string** name of the module to load.
159-
160-
##### JSON
161-
162-
```json
163-
// .metascraperrc
164-
{
165-
"rules": [
166-
"metascraper-author",
167-
"metascraper-date",
168-
"metascraper-description",
169-
"metascraper-image",
170-
"metascraper-lang",
171-
"metascraper-logo",
172-
"metascraper-publisher",
173-
"metascraper-title",
174-
"metascraper-url"
175-
]
176-
}
177-
```
178-
179-
##### YAML
180-
181-
```yaml
182-
# .metascraperrc
183-
rules:
184-
- metascraper-author
185-
- metascraper-date
186-
- metascraper-description
187-
- metascraper-image
188-
- metascraper-lang
189-
- metascraper-logo
190-
- metascraper-publisher
191-
- metascraper-title
192-
- metascraper-url
193-
```
194-
195-
#### Advanced Configuration
196-
197-
Additionally, you can pass specific configuration per module using a **object** declaration:
198-
199-
##### JSON
200-
201-
```json
202-
// .metascraperrc
203-
{
204-
"rules": [
205-
"metascraper-author",
206-
"metascraper-date",
207-
"metascraper-description",
208-
"metascraper-image",
209-
"metascraper-lang",
210-
"metascraper-logo",
211-
{"metascraper-clearbit-logo": {
212-
"format": "jpg"
213-
}},
214-
"metascraper-publisher",
215-
"metascraper-title",
216-
"metascraper-url"
217-
]
218-
}
219-
```
220-
221-
##### YAML
222-
223-
```yaml
224-
# .metascraperrc
225-
rules:
226-
- metascraper-author
227-
- metascraper-date
228-
- metascraper-description
229-
- metascraper-image
230-
- metascraper-lang
231-
- metascraper-clearbit-logo:
232-
format: jpg
233-
- metascraper-publisher
234-
- metascraper-title
235-
- metascraper-url
236-
```
237-
238-
### Constructor
134+
## Usage
239135

240-
If you need more control, you can load the rules set calling directly the metascraper constructor [`.load`](#metascraperloadrules):
136+
**metascraper** exports a constructors that need to be initalized providing a collection of rules set to load:
241137

242138
```js
243-
const metascraper = require('metascraper').load([
139+
const metascraper = require('metascraper')([
244140
require('metascraper-author')(),
245141
require('metascraper-date')(),
246142
require('metascraper-description')(),
@@ -255,25 +151,17 @@ const metascraper = require('metascraper').load([
255151

256152
Again, the order of rules are loaded are important: Just the first rule that resolve the value will be applied.
257153

258-
Use the first parameter to pass custom options if you need it:
154+
Use the first parameter to pass custom options specific per each rules set:
259155

260156
```js
261-
const metascraper = require('metascraper').load([
157+
const metascraper = require('metascraper')([
262158
require('metascraper-clearbit-logo')({
263159
size: 256,
264160
format: 'jpg'
265161
})
266162
])
267163
```
268164

269-
Using this way you are not limited to load just npm modules as rules set. For example, you can load a custom file of rules:
270-
271-
```js
272-
const metascraper = require('metascraper').load([
273-
require('./my-custom-rules-file')()
274-
])
275-
```
276-
277165
## Rules
278166

279167
?> Can't find a rules set that you want? Let's [open an issue](https://github.com/microlinkhq/metascraper/issues/new) to create it.
@@ -368,7 +256,7 @@ Type: `Array`
368256

369257
You can pass additional rules on execution time. These rules will be merged with your loaded rules.
370258

371-
### metascraper.load(rules)
259+
### metascraper([rules])
372260

373261
Create a new **metascraper** instance declaring the rules set to be used explicitly.
374262

packages/metascraper-amazon/package.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,9 @@
4141
"test": "NODE_PATH=.. TZ=UTC NODE_ENV=test nyc mocha test"
4242
},
4343
"license": "MIT",
44+
"peerDependencies": {
45+
"metascraper": "^3"
46+
},
4447
"standard": {
4548
"env": [
4649
"mocha"

packages/metascraper-amazon/test/index.js

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ const fs = require('fs')
99

1010
const readFile = promisify(fs.readFile)
1111

12-
const metascraper = require('metascraper').load([
12+
const metascraper = require('metascraper')([
1313
require('metascraper-amazon')(),
1414
require('metascraper-author')(),
1515
require('metascraper-date')(),
@@ -25,16 +25,21 @@ const metascraper = require('metascraper').load([
2525
describe('metascraper-amazon', () => {
2626
describe('amazon.co.uk', () => {
2727
it('product url', async () => {
28-
const html = await readFile(resolve(__dirname, 'fixtures/amazon-co-uk/product-url.html'))
29-
const url = 'https://www.amazon.co.uk/Vegetable-Perfection-tasty-recipes-shoots/dp/1849757097/ref=asap_bc?ie=UTF8'
28+
const html = await readFile(
29+
resolve(__dirname, 'fixtures/amazon-co-uk/product-url.html')
30+
)
31+
const url =
32+
'https://www.amazon.co.uk/Vegetable-Perfection-tasty-recipes-shoots/dp/1849757097/ref=asap_bc?ie=UTF8'
3033
const meta = omit(await metascraper({ html, url }), ['date'])
3134
snapshot(meta)
3235
})
3336
})
3437

3538
describe('amazon.com', () => {
3639
it('ansi url', async () => {
37-
const html = await readFile(resolve(__dirname, 'fixtures/amazon-com/ansi-url.html'))
40+
const html = await readFile(
41+
resolve(__dirname, 'fixtures/amazon-com/ansi-url.html')
42+
)
3843
const url = 'https://www.amazon.com/gp/product/B0057OC5O8/'
3944
const metadata = await metascraper({ html, url })
4045

@@ -44,8 +49,11 @@ describe('metascraper-amazon', () => {
4449
})
4550

4651
it('product url', async () => {
47-
const html = await readFile(resolve(__dirname, 'fixtures/amazon-com/product-url.html'))
48-
const url = 'https://www.amazon.com/The-Whole-Truth-Shaw-Book-ebook/dp/B0011UCPM4/ref=pd_zg_rss_ts_b_17_6?ie=UTF8&tag=recomshop-22'
52+
const html = await readFile(
53+
resolve(__dirname, 'fixtures/amazon-com/product-url.html')
54+
)
55+
const url =
56+
'https://www.amazon.com/The-Whole-Truth-Shaw-Book-ebook/dp/B0011UCPM4/ref=pd_zg_rss_ts_b_17_6?ie=UTF8&tag=recomshop-22'
4957
const metadata = await metascraper({ html, url })
5058

5159
// omit date because it is non deterministic
@@ -56,8 +64,11 @@ describe('metascraper-amazon', () => {
5664

5765
describe('amazon.es', () => {
5866
it('product url', async () => {
59-
const html = await readFile(resolve(__dirname, 'fixtures/amazon-es/product-url.html'))
60-
const url = 'https://www.amazon.es/aspirador-Excellence-Programable-limpieza-Silencioso/dp/B01MUGXRT9'
67+
const html = await readFile(
68+
resolve(__dirname, 'fixtures/amazon-es/product-url.html')
69+
)
70+
const url =
71+
'https://www.amazon.es/aspirador-Excellence-Programable-limpieza-Silencioso/dp/B01MUGXRT9'
6172
const metadata = await metascraper({ html, url })
6273

6374
// omit date because it is non deterministic

packages/metascraper-author/package.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,8 @@
3131
"scripts": {
3232
"test": "exit 0"
3333
},
34-
"license": "MIT"
34+
"license": "MIT",
35+
"peerDependencies": {
36+
"metascraper": "^3"
37+
}
3538
}

packages/metascraper-clearbit-logo/package.json

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,9 @@
2020
"clearbit",
2121
"metascraper"
2222
],
23+
"dependencies": {
24+
"got": "~9.0.0"
25+
},
2326
"devDependencies": {
2427
"mocha": "latest",
2528
"nyc": "latest",
@@ -36,12 +39,12 @@
3639
"test": "NODE_PATH=.. TZ=UTC NODE_ENV=test nyc mocha test"
3740
},
3841
"license": "MIT",
42+
"peerDependencies": {
43+
"metascraper": "^3"
44+
},
3945
"standard": {
4046
"env": [
4147
"mocha"
4248
]
43-
},
44-
"dependencies": {
45-
"got": "~9.0.0"
4649
}
4750
}

packages/metascraper-clearbit-logo/test/index.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
const should = require('should')
44

5-
const metascraper = require('metascraper').load([
5+
const metascraper = require('metascraper')([
66
require('metascraper-author')(),
77
require('metascraper-date')(),
88
require('metascraper-description')(),

packages/metascraper-date/package.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,8 @@
3131
"scripts": {
3232
"test": "exit 0"
3333
},
34-
"license": "MIT"
34+
"license": "MIT",
35+
"peerDependencies": {
36+
"metascraper": "^3"
37+
}
3538
}

packages/metascraper-description/package.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,8 @@
3131
"scripts": {
3232
"test": "exit 0"
3333
},
34-
"license": "MIT"
34+
"license": "MIT",
35+
"peerDependencies": {
36+
"metascraper": "^3"
37+
}
3538
}

packages/metascraper-image/package.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,5 +30,8 @@
3030
"scripts": {
3131
"test": "exit 0"
3232
},
33-
"license": "MIT"
33+
"license": "MIT",
34+
"peerDependencies": {
35+
"metascraper": "^3"
36+
}
3437
}

packages/metascraper-lang/package.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@
1919
"lodash": "~4.17.10"
2020
},
2121
"devDependencies": {
22-
"metascraper": "latest",
2322
"mocha": "latest",
2423
"nyc": "latest",
2524
"should": "latest",
@@ -36,6 +35,9 @@
3635
"test": "NODE_PATH=.. TZ=UTC NODE_ENV=test nyc mocha test"
3736
},
3837
"license": "MIT",
38+
"peerDependencies": {
39+
"metascraper": "^3"
40+
},
3941
"standard": {
4042
"env": [
4143
"mocha"

0 commit comments

Comments
 (0)