v0.4-RC3 (#25)

* fix proxy error, closes #24

* update ReadMe and metadata

CKAN-Browser/ckan_browser_dialog_dataproviders.py

@@ -98,7 +98,8 @@ def window_loaded(self):
                 # , headers=self.ua_chrome
                 , verify=False
                 , stream=True
-                , proxies=self.settings.get_proxies()[1]
+                # not needed anymore, as we use QgsNetworkAccessManager.instance() now
+                #, proxies=self.settings.get_proxies()[1]
                 , timeout=self.settings.request_timeout
             )
 
@@ -178,7 +179,7 @@ def delete_custom_server(self):
         self.row_server_selected_in_list = -1
 
     def lbl_clicked_enter_data_provider_url(self):
-        self.IDC_leManualUrl.setText('https://ckan0.cf.opendata.inter.sandbox-toronto.ca/api/3/')
+        self.IDC_leManualUrl.setText('https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/')
 
     def searchTermChanged(self, text):
         results = []

CKAN-Browser/ckanconnector.py

@@ -222,7 +222,8 @@ def download_resource(self, url, resource_format, dest_file, delete):
                 , headers=self.ua_chrome
                 , verify=False
                 , stream=True
-                , proxies=self.settings.get_proxies()[1]
+                # not needed anymore, as we use QgsNetworkAccessManager.instance() now
+                #, proxies=self.settings.get_proxies()[1]
                 , timeout=self.settings.request_timeout
             )
 
@@ -332,7 +333,8 @@ def __get_data(self, api, action):
                 url
                 , headers=self.ua_chrome
                 , verify=False
-                , proxies=self.settings.get_proxies()[1]
+                # not needed anymore, as we use QgsNetworkAccessManager.instance() now
+                #, proxies=self.settings.get_proxies()[1]
                 , timeout=self.settings.request_timeout
             )
 

CKAN-Browser/i18n/CKANBrowser_de.ts

@@ -159,7 +159,7 @@
     <message>
         <location filename="../ckan_browser_dialog_dataproviders.ui" line="50"/>
         <source>dlg_dataproviders_lbl_enter_dataprovider_url</source>
-        <translation>CKAN API Endpunkt, z.B. https://ckan0.cf.opendata.inter.sandbox-toronto.ca/api/3/</translation>
+        <translation>CKAN API Endpunkt, z.B. https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/</translation>
     </message>
     <message>
         <location filename="../ckan_browser_dialog_dataproviders.ui" line="63"/>

CKAN-Browser/i18n/CKANBrowser_en.ts

@@ -159,7 +159,7 @@
     <message>
         <location filename="../ckan_browser_dialog_dataproviders.ui" line="50"/>
         <source>dlg_dataproviders_lbl_enter_dataprovider_url</source>
-        <translation>CKAN API Endpoint, eg: https://ckan0.cf.opendata.inter.sandbox-toronto.ca/api/3/</translation>
+        <translation>CKAN API Endpoint, eg: https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/</translation>
     </message>
     <message>
         <location filename="../ckan_browser_dialog_dataproviders.ui" line="63"/>

CKAN-Browser/metadata.txt

@@ -1,17 +1,9 @@
-# This file contains metadata for your plugin. Since 
-# version 2.0 of QGIS this is the proper way to supply 
-# information about a plugin. The old method of 
-# embedding metadata in __init__.py will 
-# is no longer supported since version 2.0.
-
-# This file should be included when you package your plugin.# Mandatory items:
-
 [general]
 name=CKAN-Browser
 qgisMinimumVersion=3.0
 about=Download and display Open Data from CKAN Metadata Servers
 description=Download and display Open Data from CKAN Metadata Servers
-version=0.3.94
+version=0.3.95
 author=BergWerkGIS
 email=wb@BergWerk-GIS.at
 
@@ -20,9 +12,45 @@ email=wb@BergWerk-GIS.at
 # Optional items:
 
 # Uncomment the following line and add your changelog:
-changelog=v0.3.3 (2017-03-11)
-* Fix https://open.nrw/api/3 WMS https://open.nrw/api/3WMS https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/13
+changelog=v0.3.90 - v0.3.96 (aka v0.4-RC1 - RC6) (Mai/June/July/August 2019)
+    * experimental upgrade to QGIS 3.x https://github.com/BergWerkGIS/QGIS-CKAN-Browser/pull/20
+    * add new custom CKAN server instance directly from the main dialog https://github.com/BergWerkGIS/QGIS-CKAN-Browser/pull/22
+    * trimmed down settings dialog
+    * CKAN server list gets prepopulated with entries from https://ckan.org/about/instances/
+    * fix proxy error when using any proxy setting except HTTP-Proxy https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/24
+    * QGIS Authentication Manager Integration https://github.com/BergWerkGIS/QGIS-CKAN-Browser/pull/14 https://github.com/BergWerkGIS/QGIS-CKAN-Browser/pull/15
+
+  v0.3.3 (2017-03-11)
+    * Fix https://open.nrw/api/3 WMS https://open.nrw/api/3WMS https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/13
+
+  v0.3.3 (2017-03-11)
+    * Fix https://open.nrw/api/3 WMS https://open.nrw/api/3WMS https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/13
+
+  v0.3.2 (2017-03-02)
+    * Merge #12 to add server of the State of Alagoas , Brazil.
+    * Remove deprecated server: "Open Data Kärnten" "http://ckan.data.ktn.gv.at/api/3/"
+    * Remove deprecated server: "Open Government Data Graz": http://ckan.data.graz.gv.at/api/3/
+
+  v0.3.1 (2016-05-25)
+    * Finnish translation by Kari Salovaara
+
+  v0.3.0 (2016-05-21)
+    * data.gv.at WFS fix
+    * remove new lines in resource urls
+    * Error loading resource with ? in its location https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/10
+    * ProxySchemeUnknown error blocks service connection https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/11
+
+  v0.2.9 (2016-02-11)
+    * Japanese translation by https://github.com/yoichigmf https://github.com/BergWerkGIS/QGIS-CKAN-Browser/pull/8
+    * Support for proxies with login/password https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/5
+    * Extracting of geodata containing non ascii-characters https://github.com/BergWerkGIS/QGIS-CKAN-Browser/issues/3
+
+  v0.2.8 (2016-01-27)
+    * First public release
+    * minor fixes for WM(T)S, WFS
 
+  v0.0.1 -> v0.2.7 (2015 - January 2016)
+    * internal test versions
 
 
 # Tags are comma separated with spaces allowed

CKAN-Browser/settings.py

@@ -4,6 +4,8 @@
 import os
 import configparser
 
+from qgis.core import QgsMessageLog, Qgis
+
 
 class Settings:
 
@@ -31,7 +33,7 @@ def load(self):
         self.cache_dir = qgis_settings.value(self.KEY_CACHE_DIR, '')
         if self.cache_dir is None:
             self.cache_dir = ''
-        self.ckan_url = qgis_settings.value(self.KEY_CKAN_API, 'https://ckan0.cf.opendata.inter.sandbox-toronto.ca/api/3/')
+++        self.ckan_url = qgis_settings.value(self.KEY_CKAN_API, 'https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/')
         self.selected_ckan_servers = qgis_settings.value(self.KEY_SELECTED_CKAN_SERVERS, '')
         self.custom_servers = qgis_settings.value(self.KEY_CUSTOM_SERVERS, {})
         self.debug = qgis_settings.value(self.KEY_SHOW_DEBUG_INFO, False, bool)
@@ -48,31 +50,6 @@ def save(self):
         qgis_settings.setValue(self.KEY_CUSTOM_SERVERS, self.custom_servers)
         qgis_settings.setValue(self.KEY_SHOW_DEBUG_INFO, self.debug)
 
-    def get_proxies(self):
-        s = QSettings()
-        # if user has never clicked on proxy settings in GUI,
-        # this settings does not even exist -> default to 'false'
-        proxy_enabled = s.value('proxy/proxyEnabled', 'false')
-        proxy_type = s.value('proxy/proxyType', '')
-        proxy_host = s.value('proxy/proxyHost', '')
-        proxy_port = s.value('proxy/proxyPort', '')
-        proxy_user = s.value('proxy/proxyUser', None)
-        proxy_password = s.value('proxy/proxyPassword', None)
-        if proxy_enabled == 'false' or not proxy_enabled:
-            return False, None
-        if proxy_type == 'HttpProxy':
-            proxy_string = ''
-            if proxy_user is not None and proxy_password is not None\
-                    and proxy_user != '' and proxy_password != '':
-                proxy_string += proxy_user + ':' + proxy_password + '@'
-            proxy_string += proxy_host + ':' + proxy_port
-            #print proxy_string
-            return True, {
-                'http': 'http://' + proxy_string,
-                'https': 'https://' + proxy_string
-            }
-
-
     def _determine_version(self):
         """http://gis.stackexchange.com/a/169266/8673"""
         # error handling?

README.md

@@ -1,37 +1,46 @@
 
-# Update for QGIS 3.x is in the works!
-
 QGIS CKAN-Browser
 =============
 
 QGIS PlugIn to load and display Open Data from CKAN enabled data portals.
 
+Sponsors:
+* v0.2.8 -> v0.3.0: [Amt der Kärntner Landesregierung - KAGIS](http://www.kagis.ktn.gv.at)
+* v0.4: [City of Toronto](https://open.toronto.ca/)
 
-Sponsored by [Amt der Kärntner Landesregierung - KAGIS](http://www.kagis.ktn.gv.at)
-
-[Change log](ChangeLog.md)
+[Change log](CKAN-Browser/metadata.txt)
 
 __DISCLAMER__
 
-_This plugin has been tested with various CKAN servers (**API v3 supported only**), but we soon came to realize that each data portal implements its very own strategy for structuring metadata._
+_This plugin has been tested with various CKAN servers (**API v3 supported only**), but each and everyone has its own quirks which might result in failed downloads._
+
+The plugin queries the provided CKAN server, parse its response and try to downloads resource according to the links provided in the response.
+If the provided metadata contains invalid links the plugin of course fails to download the selected resources.
 
-**_It is very likely that data from some servers cannot be browsed or downloaded!_**
+**Before opening an issue about a failed download please try to download the failing resource with a browser.**
+If the download works in the browser then it might be a bug in the plugin.
 
-References for the time being are:
-* http://ckan.data.ktn.gv.at/api/3/
-* https://www.data.gv.at/katalog/api/3/)
+1. Click on the resource
+2. Click to copy resource link
+3. Paste into browser and check if download works
 
-**_Pull requests that make such servers accessible (and don't break others) are welcome!_**
+![](img/copy-resource-url.png)
+
+Bonus points for including log messages in the issue:
+`CKAN Browser Settings -> Miscellaneous -> Check "Show debuf information in the 'Log Messages' panel"`
 
 ## Install
 
 #### Within QGIS
 
-`Plugins -> Manage and Install Plugins -> Search: CKAN-Browser -> Install plugin`
+`Plugins -> Manage and Install Plugins -> All -> Search: CKAN-Browser -> Install plugin`
 
 #### From Zip
 
-Download [CKANBrowser.zip](https://github.com/BergWerkGIS/QGIS-CKAN-Browser/blob/master/CKANBrowser.zip) and extract to `~/.qgis2/python/plugins`
+* Download [CKANBrowser.zip](https://github.com/BergWerkGIS/QGIS-CKAN-Browser/blob/master/CKANBrowser.zip)
+* `QGIS -> Manage and Install Plugins -> Install from ZIP`
+* Select downloaded file `...`
+* `Install Plugin`
 
 #### From Source
 
@@ -52,29 +61,44 @@ After successful install there is a new toolbar available (click on `Settings`):
 #### Settings
 
 1. Select a local directory where downloaded data will be cached
-2. Choose a pre-configured CKAN Server
-3. `Apply` to copy selected CKAN Server URL to `4.`
-4. Leave URL as is or enter your own
-5. Check if connection is possible
-6. `Save` settings and close
+2. Edit authentication settings if necessary
+3. Check debug messages - only necessary when reporting bugs
+4. `Save` settings and close
 
 
 ![CKAN Browser Settings](img/settings.png?raw=true)
 
-#### Open Data (CKAN) Browser
+#### Open CKAN Browser
 
-1. Enter your search term and press `<ENTER>` or click on `2`
-2. Search: query server with search term
-3. List of data sets found, click to get more details in `4`, `5` and `7`
-4. Description of selected data set (if supported by server)
-5. Resources associated with selected data set (if supported by server)
+1. Select the CKAN Server to use (see below)
+2. Enter your search term and press `<ENTER>` or click on `3`
+3. Search: query server with search term
+4. List of data sets found, click to get more details in `5`, `6` and `7`
+5. Description of selected data set (if provided by server)
+6. Resources associated with selected data set (**if supported by server**)
   1. Click on resource to see resource URL `7` and copy to clipboard in case automatic download does not work
-  2. Check resource (or several, if supported by server) to select for download
-6. `Load Data` to download the selected resources and automatically open them
-7. Download URL of selected resource. Handy to copy and try the link in a browser if the download does not work, e.g.: some resources link to HTML or 404 pages
-8. List of categories/groups available at the selected CKAN server (if supported by server)
+  2. Check the box of the resource (or several, **if supported by server**) to select for download
+7. Resource url and copy url button if the resource cannot be downloaded (eg link to webpage or an unsupported file format)
+8. `Load Data` to download the selected resources and automatically open them
+9. List all data available sets
+10. List of categories/groups available on the selected CKAN server (**if supported by server**)
   1. Check to limit your search to one or several group(s).
   2. Double click on a group to get all data sets associated with that group
-9. List all data available sets
 
 ![Open Data (CKAN) Browser](img/open-data-ckan-browser.png?raw=true)
+
+
+#### Add Custom CKAN Server Instance
+
+The list of available CKAN server intances gets prepopulated on every start from https://ckan.org/about/instances/ but is limited to those which provide an API URL.
+
+Custom CKAN server instances can be added as well.
+
+1. Enter the CKAN endpoint, including trailing `v3/`
+2. Test the connection
+3. If the connection succeeded, name the instance and add it to the list
+4. To select an instance for use check the box
+5. Custom instances are marked by a blue background.
+5.1. Right click on a custom instance brings up the `Delete` context menu to remove the instance from the list. **Only custom instances can be deleted.**
+
+![Custom CKAN Server instances](img/ckan-instances.png)

plugins.xml

@@ -2,7 +2,7 @@
 <plugins>
     <pyqgis_plugin name="CKAN-Browser" version="0.3.94">
         <description><![CDATA[Download and display CKAN enabled Open Data Portals]]></description>
-        <version>0.3.94</version>
+        <version>0.3.95</version>
         <qgis_minimum_version>3.0.0</qgis_minimum_version>
         <qgis_maximum_version>3.99.0</qgis_maximum_version>
         <homepage>https://github.com/BergWerkGIS/QGIS-CKAN-Browser</homepage>