WAFHowto

The WOUDC WAF

• Overview
• Data Policy
• Using wget
• Troubleshooting
• Using HTTrack

Overview

The WOUDC archive is made available via Web Accessible Folder (WAF). WAF provides a web-based access mechanism to download raw files using your web browser or tools.

The entire archive of contributed data files in Extended CSV format is made available at https://woudc.org/archive for users to browse, download or integrate into their workflows.

Data Policy

All data and metadata are subject to the WOUDC data policy.

Using wget

Below are examples of working with the WAF using wget:

# download entire archive
wget --timestamping --recursive --no-parent --reject "index.html*" https://woudc.org/archive/Archive-NewFormat/
# download only Lidar data
wget --timestamping --recursive --no-parent --reject "index.html*" https://woudc.org/archive/Archive-NewFormat/Lidar_1.0_1/
# download all OzoneSonde data from a specific station
wget --timestamping --recursive --no-parent --reject "index.html*" https://woudc.org/archive/Archive-NewFormat/OzoneSonde_1.0_1/shp440/
# mirroring mode
wget --timestamping --recursive --no-parent --reject "index.html*" --mirror https://woudc.org/archive/Archive-NewFormat
# mirroring mode, strip root directory (prevent woudc.org directory creation)
wget --timestamping --recursive --no-parent --reject "index.html*" --mirror -nH https://woudc.org/archive/Archive-NewFormat

Troubleshooting

• Connection refused errors: Check with your networking / server administrator as you may be behind a proxy. Check for http_proxy environment variables and set/unset/update

Using HTTrack

Below are examples of working with the WAF using HTTrack on Windows:

(tested with WinHTTrack 3.48-21)

Initial Mirroring

• run Start / All Programs / WinHTTrack / WinHTTrack Website Copier
• Enter a project name under New project name (e.g. woudc-waf-mirror)
• enter/update where the WAF will be copied to on your local computer (Base path, default is C:\My Web Sites)
• click Next
• For initial mirroring, under Action, choose Download web site(s)
• under Web Addresses: (URL), enter https://woudc.org/archive/Archive-NewFormat
• click Next
• click Finish
• WinHTTrack will begin downloading the WAF until completion
• when the mirror is complete, click Browse Mirrored Website to navigate to your mirror
• click Finish or Exit

Subsequent Mirroring

• run Start / All Programs / WinHTTrack / WinHTTrack Website Copier
• Choose the previously entered project name under Existing project name (e.g. woudc-waf-mirror)
• click Next
• by default the Action select list will be set to * Update existing download
• click Next
• click Finished