-
Notifications
You must be signed in to change notification settings - Fork 2
/
add-static-pages.sql
241 lines (130 loc) · 19.7 KB
/
add-static-pages.sql
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
INSERT INTO "public"."content" ("id", "type", "content") VALUES ('staticPages/about.md', 'text/plain', '---
title: About
---
Data.gov.au is the central source of Australian open government data.
Anyone can access the anonymised public data published by federal, state and local government agencies.
You are encouraged to search and reuse the data on data.gov.au. This data is a national resource that holds considerable value for growing the economy, improving service delivery and transforming policy outcomes.
Find out more [about open data](https://blog.data.gov.au/about-open-data).
In addition to government data, you can also find publicly-funded research data and datasets from private institutions that are in the public interest.
The site has over 30,000 publically available datasets and continues to grow. The federal government’s <a href=\"https://www.pmc.gov.au/sites/default/files/publications/aust_govt_public_data_policy_statement_1.pdf\" target=\"_blank\">public data policy statement<span class=\"visuallyhidden\"> Opens in new window</span></a> requires all government agencies to make non-sensitive data open by default.
In addition to free, open datasets, data.gov.au now includes information about unpublished data and data available for purchase.
### Visualising data
Many of the data sets can be visualised using the in-built mapping tools.
Datasets that contain a geospatial field (such as latitude and longitude)can be mapped and viewed in <a href=\"http://nationalmap.gov.au/\" target=\"_blank\">NationalMap<span class=\"visuallyhidden\"> Opens in new window</span></a>.
### Contact us
Contact the Digital Transformation Agency (DTA) at [data@digital.gov.au](mailto:data@digital.gov.au) for support related to data.gov.au and NationalMap.
Let us know how you are using government data or if you want to contribute to our [blog](http://blog.data.gov.au/) about public data and data.gov.au.
Please do not include any unnecessary personal, financial or sensitive information.
Information will only be used for purposes for which your provide it. Please see our Privacy policy for further information.
### How do I register or upload data
Anyone can [register an account](https://data.gov.au/user/register) on data.gov.au which can be used for commenting or liking datasets.
But to publish government data you must have an account with an Australian Government email address (ending with “.gov.au”).
Before you consider registering an account for a government (federal, state, local) or government-funded research agency, you should do a quick search of existing [organisations](https://search.data.gov.au/organisations) to see if your own agency is already listed. The listed contact can help you to log in and publish your data sets. If not, you can register for the first time.
[How to set up an agency account to upload public data](https://toolkit.data.gov.au/index.php?title=Starting_on_datagovau#Starting_on_datagovau)
### Development
Data.gov.au was transferred from the Department of Premier and Cabinet to the Digital Transformation Agency (DTA). The redeveloped platform MAGDA was developed in partnership with CSIRO <a href=\"http://data61.csiro.au\" target=\"_blank\">Data61<span class=\"visuallyhidden\"> Opens in new window</span></a>.
MAGDA (Making Australian Government Data Available) is a fully open source project.
Read more about the project at <a href=\"https://magda.io/\" target=\"_blank\">magda.io<span class=\"visuallyhidden\"> Opens in new window</span></a>
## Terms of Use
### Copyright
With the exception of the Commonwealth Coat of Arms and where otherwise noted, material presented on data.gov.au is provided under a <a href=\"http://creativecommons.org/licenses/by/3.0/au/\" target=\"_blank\">Creative Commons Attribution 3.0 Australia licence<span class=\"visuallyhidden\"> Opens in new window</span></a>.
### How to attribute datasets
When using content from this website that is licensed under a Creative Commons Licence, you are required to attribute the work in the manner specified in the licence (but not in any way that suggests that the publishing organisation endorses you or your use of the work).
This website requires that you use the following form of attribution:
Attribution to: Organisation name, jurisdiction, title of dataset, date the content was sourced, dataset URL
Example: The Department of the Primary Industries and Regions, South Australia, Field Crop Estimates, Sourced on 22 July 2013, <a href=\"http://www.data.sa.gov.au/dataset/field-crop-production-estimates\" target=\"_blank\">http://www.data.sa.gov.au/dataset/field-crop-production-estimates<span class=\"visuallyhidden\"> Opens in new window</span></a>
### Lawful purposes
You agree to use data.gov.au only for lawful purposes, and in a manner that does not infringe the rights of or restrict or inhibit the use and enjoyment of data.gov.au by any third party. This includes conduct which is unlawful or which may harass or cause distress or inconvenience to any person, the transmission of obscene or offensive content or disruption to data.gov.au.
You must not post or transmit via data.gov.au any unlawful, defamatory, obscene, offensive or scandalous material, or any material that constitutes or encourages conduct that would contravene any law.
### Disclaimer
Your use of data.gov.au does not affect or reduce any obligation any Agency owes You, affect or reduce any obligation You owe to any Agency.
The data.gov.au site is not a substitute for independent professional advice and you should obtain any appropriate professional advice relevant to your particular circumstances.
We take reasonable care in linking to other websites but we have no direct control over the content presented in those websites or the availability of those websites.
We make no representation that the material on any linked websites does not infringe the intellectual property rights or any other rights of any person. We do not authorise the reproduction of such material.
We do not endorse or recommend any links to external websites or third party content, including products and services offered by, from or through those websites or their content.
### Content and comments
The datasets provided through data.gov.au have been created by many different government agencies. Because of this, we cannot guarantee the quality or timeliness of the data. While we welcome your feedback, we cannot guarantee that we will be able to correct any problems identified with the datasets.
You are responsible for the comments you make on data.gov.au. If you wish to comment on data.gov.au, you must do so in accordance with our Moderation Policy.
### Moderation Policy
Data.gov.au uses a pre-moderation process that aligns with the APS Code of Conduct.
Each comment is subject to a manual check to determine its suitability for publication. The aim of this process is to make sure that comments:
- are not inflammatory, unreasonable or obscene;
- are civil and respectful of other opinions;
- do not include any material or encourage any conduct that would break the law; and,
- are considerate of your privacy and the privacy of others.
The moderation process is not intended to stifle discussion, but rather to detect spam or content that breaches the terms of use. We endeavour to publish your comments as quickly as possible.
### Security
We strive to protect comments and information you provide on data.gov.au. We will use all reasonable endeavours to ensure that your comments and information are not compromised. However, we cannot guarantee that no harmful code will enter data.gov.au. You should be aware of the risks associated with using websites.
We recommend that you use appropriate and up-to-date firewall and anti-virus software to protect your computer systems.
If you experience a problem or loss that is caused by the information you send to or via data.gov.au, your computer being compromised in some way or by something beyond our control, we cannot take responsibility for causing the problem. We will, however, do our best to help you if we can.
Where connection to a system or website outside our control compromises the objectives of data.gov.au, we may sever links to that website or system.
### General
These Terms of Use are governed by the laws of the Australian Capital Territory, Australia and you agree to submit to the exclusive jurisdiction of the courts of the Australian Capital Territory.
If any of these Terms of Use are held to be invalid, unenforceable or illegal for any reason, the remaining Terms of Use will continue in full force.
These Terms of Use do not negate any Terms or Conditions that apply to your use of any other government website. We may revise these Terms and Conditions from time to time.
'),
('staticPages/dataset-quality.md', 'text/plain', '---
title: Dataset Quality
---
## How we calculate open data quality
Our open data quality ratings are an automatic rating based on the licensing, format and availability of data, derived from (but different to) Tim Berners-Lee’s [5-star linked data system](https://www.w3.org/2011/gld/wiki/5_Star_Linked_Data).
A dataset is given the rating of its best file or API – for example, if a dataset has 3 files rated at 1 star, 2 stars and 3 stars, the whole dataset will have a 3-star rating. The criteria for each star rating is described below.
We will continue to update the data quality rating as research continues. If you have any feedback – particularly if you feel that a certain dataset has been misclassified – please contact us at [data@digital.gov.au](mailto:data@digital.gov.au).
### ⭐ Is the data available on the web, and does it have an open license?
This has two components – whether we were able to successfully check that the URL for the data is working, and whether its declared license matched our list of open licenses.
When a dataset is indexed by [data.gov.au](https://data.gov.au/), we automatically check every file or API link to make sure it’s not broken. The link will be marked as broken if it doesn’t respond (and times out), returns an error or not found HTTP code, requires a login to access, doesn’t support an HTTP HEAD request, or rate-limits our crawler without returning the correct HTTP code (429) so that we know to slow down our requests.
If a dataset’s links are all broken it will be marked as zero stars.
The list of data licenses that we consider to be open is based on [the opendefinition.org list of Conformant Licenses](https://opendefinition.org/licenses/). You can find the list we use [here](https://github.com/TerriaJS/magda/blob/master/magda-minion-linked-data-rating/src/openLicenses.ts).
Essentially, if a dataset declares a license field with text containing one of those licenses (for example, “CC0” contains “CC”), then we assume that it’s open.
### ⭐⭐ Is the data available as structured, machine-readable data?
To get 2 stars, the data must qualify for 1-star and also be structured and machine-readable. So data contained in images, plain text, Microsoft Word documents and Portable Document Files (PDF) do not qualify, but machine readable formats such as Microsoft Excel files do.
We determine this by comparing the file’s format against a list of formats that we understand to be 2-star formats – you can see the list [here](https://github.com/TerriaJS/magda/blob/master/magda-minion-linked-data-rating/src/openFormats.ts#L2).
### ⭐⭐⭐ Is it available in a non-proprietary format?
To get 3 stars, the data must qualify for 2-star (machine-readable) but also be available in a [non-proprietary](https://en.oxforddictionaries.com/definition/non-proprietary) format. For example, plain comma-separated values (CSV), JSON and XML documents are all formats that are in the public domain or open standards, while a Microsoft Access MDB file is not.
We determine this by comparing the file’s format against a list of formats that we understand to be 3-star formats – you can see the list [here](https://github.com/TerriaJS/magda/blob/master/magda-minion-linked-data-rating/src/openFormats.ts#L3).
### ⭐⭐⭐⭐ Does the data declare a schema?
To get 4 stars, data must not only be available in a non-proprietary, machine readable format, but also specify some kind of schema – meaning the data isn’t just readable by machines, but also includes metadata that allows it to be understood by machines. This usually means that it’s specified in an open, Linked Data format that includes the schema inside the data such as [RDF](https://en.wikipedia.org/wiki/Resource_Description_Framework) or [JSON-LD](https://en.wikipedia.org/wiki/JSON-LD), or uses a non-proprietary format that enforces a schema, such as [CSV GEO AU](https://github.com/TerriaJS/nationalmap/wiki/csv-geo-au).
Currently we determine this by comparing the file’s format against a list of formats that we understand to be 4-star formats – you can see the list [here](https://github.com/TerriaJS/magda/blob/master/magda-minion-linked-data-rating/src/openFormats.ts#L21).
### ⭐⭐⭐⭐⭐ Does the data link to other data?
5-star datasets are those that link to other data – for example, a dataset that included “Canberra” as a value for something wouldn’t simply include the word “Canberra”, but a link to data about Canberra, [such as its record in Geonames](http://sws.geonames.org/2172517/about.rdf).
5-star datasets are very rare, and our automated process (currently) is not yet able to determine which datasets qualify for 5 stars.
'),
('staticPages/privacy-policy.md', 'text/plain', '---
title: Privacy Policy
---
data.gov.au is an online service managed by the Digital Transformation Agency. We respect and protect the privacy of our users. This privacy policy tells you how we collect and use information.
The term \"Personal Information\" in this privacy policy means any information from which your identity is apparent or can be reasonably ascertained. We do not collect Personal Information about you when you visit this online service.
You can use this service without telling us who you are or revealing other Personal Information. If you send us a contact or feedback form you do not need to identify yourself or use your real name.
### Location of Content
Content on the data.gov.au main site, search index, administrator account information and completed form information is hosted on Amazon Web Services and Google Cloud Platform servers located within Australia.
### Personal information
If you contact us we will collect the email address you nominate and any other identifying information you provide, such as a name or phone number.
Please do not give us other personal or sensitive information.
Other than circumstances such as unlawful activity or serious threats to health and safety, we do not share Personal Information with other government agencies. If you ask us about an issue that needs to be dealt with by another agency, we will provide you with the necessary details to make contact yourself. You may opt out of further contact from us at any time.
### How we deal with complaints and requests
You may request access to Personal Information about you that we hold and you may ask us to correct your Personal Information if you find that it is not accurate, up-to-date or complete. You may also make a complaint about our handling of your Personal Information. These services are free of charge.
To protect your privacy and the privacy of others, we will need evidence of your identity before we can grant you access to information about you or change it.
You can contact us by email, or send your request or complaint to the postal address below. We undertake to respond within 30 days. If the request or complaint will take longer to resolve, we will provide you with a date by which we expect to respond.
Our postal address:
```
Digital Transformation Agency
PO Box 457
Canberra City ACT 2601
```
### How we protect your Personal Information
This online service is hosted in Australia in secure, government accredited facilities. To help protect the privacy of data and personal information we collect and hold, we maintain physical, technical and administrative safeguards. We update and test our security technology on an ongoing basis.
We train our employees about the importance of confidentiality and maintaining the privacy and security of your information. Access to your Personal Information is restricted to employees who need it to provide benefits or services to you.
### Website analytics
To improve your experience on our site, we use \"cookies\". Cookies are an industry standard and most major websites use them. A cookie is a small text file that our site may place on your computer as a tool to remember your preferences. You may refuse the use of cookies by selecting the appropriate settings on your browser, however please note that if you do this you may not be able to use the full functionality of this website.
Our website may contain links to other websites. Please be aware that we are not responsible for the privacy practices of such other sites. When you go to other websites from here, we advise you to be aware and read their privacy policy.
This website uses the analytics tools Google Analytics and Visual Web Optimiser. These services transmit website traffic data to Google and IBM SoftLayer servers, that may be located outside of Australia. These services do not identify individual users or associate your IP address with any other data held by Google or Wingify (the parent company of Visual Web Optimiser). We use reports provided by these analytics tools to help us understand website traffic and usage.
By using this website, you consent to the processing of data about you by analytics tool providers in the manner described in [Google''s Privacy Policy](http://www.google.com/policies/privacy/), [Visual Web Optimiser''s Privacy Policy](https://vwo.com/privacy-policy/) and for the purposes set out above.
You can opt out of Google Analytics if you disable or refuse the cookie, disable JavaScript, or [use the opt-out service provided by Google](https://tools.google.com/dlpage/gaoptout). You can opt out of Visual Web Optimiser if you disable or refuse the cookie, disable JavaScript, or [use the opt-out service provided by Wingify.](https://data.gov.au/?vwo_opt_out=1)
### Uses
We only use your personal information for the purposes for which you gave it to us.
We do not share information about you with other government agencies or other organisations without your permission unless it is necessary to provide you with a service that you have requested; is required or authorised by law; or will prevent or lessen a serious and imminent threat to somebody’s health.
We analyse non-identifiable website traffic data to improve our services. We only use session-based cookies for anonymous website usage data to help us improve the structure and functionality of data.gov.au and for agency users who log in to the site to upload and manage their datasets. You can change your web browser settings to reject cookies or to prompt you each time a website wishes to add a cookie to your browser. In spite of our efforts to reduce the impact, some functionality on the website may be affected by this.
### Our obligations
This online service is bound by the provisions of the [Privacy Act 1988 (Cth)](http://www.comlaw.gov.au/Details/C2004A03712), including the [Australian Privacy Principles](http://www.oaic.gov.au/privacy/privacy-act/australian-privacy-principles).
We also follow the [Guidelines for Federal and ACT government websites](http://www.oaic.gov.au/privacy/privacy-resources/privacy-guides/guidelines-for-federal-and-act-government-websites%20%E2%80%9CGuidelines%20for%20Federal%20and%20ACT%20government%20websites%E2%80%9D) issued by the [Office of the Australian Information Commissioner](http://www.oaic.gov.au/%20%E2%80%9COffice%20of%20the%20Australian%20Information%20Commissioner%E2%80%9D).
');