forked from signxer/HistoricalWeatherTW
/
README.rst
152 lines (102 loc) · 5.29 KB
/
README.rst
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
.. sectnum::
.. raw:: html
<p align="left">
<a href="https://pypi.org/project/carson-tool.HistoricalWeatherTW/">
<img src="https://img.shields.io/static/v1?&style=plastic&logo=pypi&label=App&message=HistoricalWeather&color=00FFFF"/></a>
<a href="https://pypi.org/project/carson-tool.HistoricalWeatherTW/">
<img src="https://img.shields.io/pypi/v/carson-tool.HistoricalWeatherTW.svg?&style=plastic&logo=pypi&color=00FFFF"/></a>
<a href="https://pypi.org/project/carson-tool.HistoricalWeatherTW/">
<img src="https://img.shields.io/pypi/pyversions/carson-tool.HistoricalWeatherTW.svg?&style=plastic&logo=pypi&color=00FFFF"/></a>
<a href="https://github.com/CarsonSlovoka/HistoricalWeatherTW/blob/master/LICENSE">
<img src="https://img.shields.io/pypi/l/carson-tool.HistoricalWeatherTW.svg?&style=plastic&logo=pypi&color=00FFFF"/></a>
<br>
<a href="https://github.com/CarsonSlovoka/HistoricalWeatherTW">
<img src="https://img.shields.io/github/last-commit/CarsonSlovoka/HistoricalWeatherTW?&style=plastic&logo=github&color=00FF00"/></a>
<img src="https://img.shields.io/github/commit-activity/y/CarsonSlovoka/HistoricalWeatherTW?&style=plastic&logo=github&color=0000FF"/></a>
<a href="https://github.com/CarsonSlovoka/HistoricalWeatherTW">
<img src="https://img.shields.io/github/contributors/CarsonSlovoka/HistoricalWeatherTW?&style=plastic&logo=github&color=111111"/></a>
<a href="https://github.com/CarsonSlovoka/HistoricalWeatherTW">
<img src="https://img.shields.io/github/repo-size/CarsonSlovoka/HistoricalWeatherTW?&style=plastic&logo=github"/></a>
<br>
<a href="https://pepy.tech/project/carson-tool-historicalweathertw">
<img src="https://pepy.tech/badge/carson-tool-historicalweathertw"/></a>
<a href="https://pepy.tech/project/carson-tool-historicalweathertw/month">
<img src="https://pepy.tech/badge/carson-tool-historicalweathertw/month"/></a>
<a href="https://pepy.tech/project/carson-tool-historicalweathertw/week">
<img src="https://pepy.tech/badge/carson-tool-historicalweathertw/week"/></a>
<!--
<img src="https://img.shields.io/github/commits-since/m/CarsonSlovoka/HistoricalWeatherTW/Dev?label=commits%20to%20be%20deployed"/></a>
-->
</p>
HistoricalWeatherTW 台灣歷史天氣爬蟲
==============================================
This script is to crawl the information of `觀測資料查詢 <https://e-service.cwb.gov.tw/HistoryDataQuery/index.jsp>`_ website
Data from `觀測資料查詢系統 <http://e-service.cwb.gov.tw/HistoryDataQuery/>`_
Usage
============
#. Prepare `station.csv`_.
.. note:: station.csv (you can find it from the https://e-service.cwb.gov.tw/wdps/obs/state.htm)
#. run the function of ``collect_weather_tw`` you will see the result!
.. code-block:: python
def collect_weather_tw(station_csv_path: Path, output_path,
end_date: datetime.date, begin_date: datetime.date,
query_format,
convert2num):
**You can refer to** `__init__.py`_ **for more help**
1. Prepare `config.yaml`_ and use this path as input parameter
#. run `__init__.py`_
QUICKLY START
---------------
.. code-block:: python
from Carson.Tool.HistoricalWeatherTW import collect_weather_tw, QueryFormat
from pathlib import Path
import datetime
import os
if __name__ == '__main__':
STATION_CSV = '../config/CSV/station_test.csv'
OUTPUT_PATH = Path(f'../temp/year_result.csv')
BEGIN_DATE = datetime.date(2019, 10, 1)
END_DATE = datetime.date(2019, 10, 2)
QUERY_FORMAT = QueryFormat.DAY
CONVERT2NUM = True
collect_weather_tw(Path(STATION_CSV), OUTPUT_PATH,
BEGIN_DATE, END_DATE,
QUERY_FORMAT,
CONVERT2NUM)
os.startfile(OUTPUT_PATH)
Data
================
The output depends on ``QueryFormat``!
.. csv-table:: Day
:file: test/output/day.csv
:header-rows: 1
.. csv-table:: MONTH
:file: test/output/month.csv
:header-rows: 1
.. csv-table:: YEAR
:file: test/output/year.csv
:header-rows: 1
.. note:: There have some non-numeric forms of the original data.
Such as indefinite wind direction V, rain track T, etc. the variable of ``CONVERT2NUM`` will replace them with numbers.
Release note
======================
v4.0
---------
Encapsulated as API
v3.0
---------
Features:
- All outputs into a single file. (making it easier to use for SQL)
- The output header field will automatically grab the content on the web page (not use hard coding)
- You can choose the type of query (year, month, day) according to your needs.
Other:
- Make the code easier to read.
V2.0
-------
加入全台觀測站
V1.0
---------
第一版
.. _`__init__.py`: https://github.com/CarsonSlovoka/HistoricalWeatherTW/blob/master/Carson/Tool/HistoricalWeatherTW/__init__.py#L15-L33
.. _`config.yaml`: https://github.com/CarsonSlovoka/HistoricalWeatherTW/blob/master/Carson/Tool/HistoricalWeatherTW/config/config.yaml
.. _`station.csv`: https://github.com/CarsonSlovoka/HistoricalWeatherTW/blob/master/Carson/Tool/HistoricalWeatherTW/config/CSV/station.csv