Skip to content

Commit 161368f

Browse files
committed
Add template for morph.io scraper
0 parents  commit 161368f

File tree

5 files changed

+84
-0
lines changed

5 files changed

+84
-0
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Ignore output of scraper
2+
data.sqlite

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
This is a scraper that runs on [Morph](https://morph.io). To get started [see the documentation](https://morph.io/documentation)

composer.json

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"repositories": [
3+
{
4+
"url": "https://github.com/openaustralia/scraperwiki-php.git",
5+
"type": "git"
6+
}
7+
],
8+
"require": {
9+
"openaustralia/scraperwiki": "dev-morph_defaults",
10+
"ext-sqlite3": "*",
11+
"ext-pdo_sqlite": "*",
12+
"ext-gd": "*",
13+
"ext-mbstring": "*"
14+
}
15+
}

composer.lock

Lines changed: 39 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

scraper.php

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
<?
2+
// This is a template for a PHP scraper on morph.io (https://morph.io)
3+
// including some code snippets below that you should find helpful
4+
5+
// require 'scraperwiki.php';
6+
// require 'scraperwiki/simple_html_dom.php';
7+
//
8+
// // Read in a page
9+
// $html = scraperwiki::scrape("http://foo.com");
10+
//
11+
// // Find something on the page using css selectors
12+
// $dom = new simple_html_dom();
13+
// $dom->load($html);
14+
// print_r($dom->find("table.list"));
15+
//
16+
// // Write out to the sqlite database using scraperwiki library
17+
// scraperwiki::save_sqlite(array('name'), array('name' => 'susan', 'occupation' => 'software developer'));
18+
//
19+
// // An arbitrary query against the database
20+
// scraperwiki::select("* from data where 'name'='peter'")
21+
22+
// You don't have to do things with the ScraperWiki library.
23+
// You can use whatever libraries you want: https://morph.io/documentation/php
24+
// All that matters is that your final data is written to an SQLite database
25+
// called "data.sqlite" in the current working directory which has at least a table
26+
// called "data".
27+
?>

0 commit comments

Comments
 (0)