Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100755 108 lines (55 sloc) 2.832 kB
daec582 @chriso Added word_count to builtin
authored
1 # Built-in modules
cbb1912 @chriso Added initial built-in modules
authored
2
3 Node.io comes with several built-in modules which can be accessed through the command line or web interface.
4
5 To run a built-in module, run
6
7 $ node.io [MODULE] [ARGS]
08c57bc @chriso Removed the duplicates builtin
authored
8
cbb1912 @chriso Added initial built-in modules
authored
9 To see usage details, run
08c57bc @chriso Removed the duplicates builtin
authored
10
cbb1912 @chriso Added initial built-in modules
authored
11 $ node.io [MODULE] help
12
0cb932c @chriso Updated built-in module usage details
authored
13 ### digest
cbb1912 @chriso Added initial built-in modules
authored
14
15 This module calculates the hash/checksum of each element of input. Available hashes are [md5, crc32, sha1, sha256, sha512, ...]
16
17 Example 1 - find the MD5 hash of a string
08c57bc @chriso Removed the duplicates builtin
authored
18
cbb1912 @chriso Added initial built-in modules
authored
19 $ echo "this is a string" | node.io digest md5
20 => b37e16c620c055cf8207b999e3270e9b
21
0cb932c @chriso Updated built-in module usage details
authored
22 ### pagerank
cbb1912 @chriso Added initial built-in modules
authored
23
24 This module checks a URL's Google pagerank (rate limits apply)
25
26 Example 1 - find the pagerank of mastercard.com
27
08c57bc @chriso Removed the duplicates builtin
authored
28 $ echo "mastercard.com" | node.io pagerank
cbb1912 @chriso Added initial built-in modules
authored
29 => mastercard.com,7
08c57bc @chriso Removed the duplicates builtin
authored
30
0cb932c @chriso Updated built-in module usage details
authored
31 ### resolve
cbb1912 @chriso Added initial built-in modules
authored
32
33 This module provides DNS resolution utilities
34
35 Example 1 - resolve domains and output "domain,ip"
08c57bc @chriso Removed the duplicates builtin
authored
36
cbb1912 @chriso Added initial built-in modules
authored
37 $ node.io resolve < domains.txt
38
39 Example 2 - return domains that do not resolve (potentially available)
40
41 $ node.io resolve available < domains.txt
42
43 Example 3 - return domains that do resolve
44
45 $ node.io resolve found < domains.txt
46
47 Example 4 - return unique IPs
08c57bc @chriso Removed the duplicates builtin
authored
48
cbb1912 @chriso Added initial built-in modules
authored
49 $ node.io resolve ips < domains.txt
08c57bc @chriso Removed the duplicates builtin
authored
50
c29e15b @chriso Added a statuscode builtin
authored
51 ### statuscode
52
53 Makes a HEAD request to each URL of input and returns the status code
54
55 Example 1 - return the status code (url,status)
56
57 $ cat urls.txt | node.io -s statuscode
58
59 Example 2 - find URLs that 404
60
61 $ cat urls.txt | node.io -s statuscode 404
62
63 Example 3 - find URLs that redirect
64
65 $ cat urls.txt | node.io -s statuscode 3
66
ea47864 @chriso Added a new builtin - query
authored
67 ### query
68
69 The query module can be used to quickly select data from a URL. Usage: `$ node.io query url [selector] [attribute]`
70
71 Example 1 - pull front page stories from reddit
72
73 $ node.io query "http://www.reddit.com" a.title
74
75 Example 2 - pull the href attribute from these links
76
77 $ node.io query "http://www.reddit.com" a.title href
78
0cb932c @chriso Updated built-in module usage details
authored
79 ### validate
cbb1912 @chriso Added initial built-in modules
authored
80
81 This module is a simple wrapper for [node-validator](https://github.com/chriso/node-validator). Available filters are: [int, url, ip, alpha, alphanumeric, email]
82
83 Example 1 - remove lines that **do not** match a filter
84
85 $ node.io validate [FILTER] < list.txt
86
87 Example 2 - output lines that do not match a filter (remove valid lines)
88
89 $ node.io validate not [FILTER] < list.txt
08c57bc @chriso Removed the duplicates builtin
authored
90
0cb932c @chriso Updated built-in module usage details
authored
91 ### eval
cbb1912 @chriso Added initial built-in modules
authored
92
93 This module evaluates an expression on each line of input and emits the result (unless the result is null)
94
95 Example 1 - remove empty lines
08c57bc @chriso Removed the duplicates builtin
authored
96
cbb1912 @chriso Added initial built-in modules
authored
97 $ node.io -s eval "input.trim() != '' ? input : null" < input.txt > modified.txt
08c57bc @chriso Removed the duplicates builtin
authored
98
cbb1912 @chriso Added initial built-in modules
authored
99 Example 2 - convert a TSV (tab separated file) to CSV
08c57bc @chriso Removed the duplicates builtin
authored
100
cbb1912 @chriso Added initial built-in modules
authored
101 $ node.io -s eval "input.split('\t').join(',')" < data.tsv > data.csv
102
0cb932c @chriso Updated built-in module usage details
authored
103 ### word_count
daec582 @chriso Added word_count to builtin
authored
104
105 This module uses map/reduce to count word occurrences in a file
106
107 $ node.io word_count < input.txt
Something went wrong with that request. Please try again.