Import / Identification - Erreur encodage Python sur QGIS 2.18.10 #106

mdouchin · 2017-07-05T09:02:52Z

Constaté avec une récente version de QGIS 2.18.10 (et peut-être antérieure ?).

Suite au commit suivant qgis/QGIS@14ab5eb

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 431: ordinal not in range(128) 
Traceback (most recent call last):
  File "/home/mdouchin/.qgis2/python/plugins/cadastre/cadastre_menu.py", line 361, in getParcelleInfo
    self.cadastre_search_dialog
  File "/home/mdouchin/.qgis2/python/plugins/cadastre/cadastre_dialogs.py", line 2386, in __init__
    self.setProprietairesContent()
  File "/home/mdouchin/.qgis2/python/plugins/cadastre/cadastre_dialogs.py", line 2493, in setProprietairesContent
    [header, data, rowCount, ok] = cadastre_common.fetchDataFromSqlQuery(self.connector, sql)
  File "/home/mdouchin/.qgis2/python/plugins/cadastre/cadastre_dialogs.py", line 461, in fetchDataFromSqlQuery
    c = connector._execute(None,unicode(sql))
  File "/usr/share/qgis/python/plugins/db_manager/db_plugins/connector.py", line 81, in _execute
    cursor.execute(str(sql))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 431: ordinal not in range(128)

The text was updated successfully, but these errors were encountered:

mdouchin · 2017-07-05T09:37:29Z

It seems I can find a workaround by replacing in my code L 461 of cadastre_dialogs.py
c = connector._execute(None,unicode(sql))
by
c = connector._execute(None,unicode(sql).encode('utf-8'))

@jusabatier Can you confirm this workaround allows you to import data ( AND have good accentuated chars visibles in tables and in the plugin dialogs ) ?

jusabatier · 2017-07-05T09:46:48Z

Nope, I replaced this line with your code in ~/.qgis2/python/plugins/cadastre/cadastre_dialogs.py and restarted my QGis, but still have the same error when importing in a PostGIS database.

jusabatier · 2017-07-05T09:49:01Z

Moreover, it seems like your stacktrace is different, the one I have is :

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 23239: ordinal not in range(128) 
Traceback (most recent call last):
  File "/home/jusabatier/.qgis2/python/plugins/cadastre/cadastre_dialogs.py", line 1083, in processImport
    qi.installCadastreStructure()
  File "/home/jusabatier/.qgis2/python/plugins/cadastre/cadastre_import.py", line 251, in installCadastreStructure
    self.executeSqlScript(s, item.has_key('constraints'))
  File "/home/jusabatier/.qgis2/python/plugins/cadastre/cadastre_import.py", line 1064, in executeSqlScript
    self.executeSqlQuery(sql, ignoreError)
  File "/home/jusabatier/.qgis2/python/plugins/cadastre/cadastre_import.py", line 1115, in executeSqlQuery
    c = self.connector._execute_and_commit(sql)
  File "/usr/share/qgis/python/plugins/db_manager/db_plugins/connector.py", line 95, in _execute_and_commit
    self._execute(None, sql)
  File "/usr/share/qgis/python/plugins/db_manager/db_plugins/connector.py", line 81, in _execute
    cursor.execute(str(sql))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 23239: ordinal not in range(128)

The problem seems to be in cadastre_import.py

mdouchin · 2017-07-05T09:51:24Z

yep, I will try to adapt the line "/home/jusabatier/.qgis2/python/plugins/cadastre/cadastre_import.py", line 1115
There are probably more locations (but hopefully not, and if so, I would need to refactor the code...)

I would find a way to try catch and use one or other method to avoid bugs in versions prior and after the QGIS commit

jusabatier · 2017-07-05T09:59:42Z

I confirm that replace cadastre_import.py line 1115 :
c = self.connector._execute_and_commit(sql)
with
c = self.connector._execute_and_commit(unicode(sql).encode('utf-8'))

solved the problem and allow to perform a successful import in PostGIS database

mdouchin · 2017-07-05T12:29:03Z

Ok. I will try to provide updated code as a workaround

guillaumelaurant · 2017-07-05T13:04:37Z

Bonjour,

J'ai essayé de modifié cadastre_dialogs.py comme demandé mais ça ne fonctionne toujours pas, sous QGIS 2.18 comme 2.14...

jusabatier · 2017-07-05T13:08:25Z

Il y a probablement 2 fichiers à modifier :

cadastre_dialogs.py ligne 461 :
c = connector._execute(None,unicode(sql).encode('utf-8'))
cadastre_import.py ligne 1115 :
c = self.connector._execute_and_commit(unicode(sql).encode('utf-8'))

Et bien penser à relancer QGis après, pour moi ça a résolu le problème et l'import dans PostGIS s'est fait correctement en 2.18.10

guillaumelaurant · 2017-07-05T13:28:53Z

Ça ne fonctionne pas, voici le message d'erreur :

Traceback (most recent call last):
File "C:/Users/g.laurant/.qgis2/python/plugins\cadastre\cadastre_menu.py", line 361, in getParcelleInfo
self.cadastre_search_dialog
File "C:/Users/g.laurant/.qgis2/python/plugins\cadastre\cadastre_dialogs.py", line 2371, in init
self.setProprietairesContent()
File "C:/Users/g.laurant/.qgis2/python/plugins\cadastre\cadastre_dialogs.py", line 2477, in setProprietairesContent
[header, data, rowCount, ok] = cadastre_common.fetchDataFromSqlQuery(self.connector, sql)
File "C:/Users/g.laurant/.qgis2/python/plugins\cadastre\cadastre_dialogs.py", line 460, in fetchDataFromSqlQuery
c = connector._execute(None,unicode(sql).encode('utf-8'))
File "C:/PROGRA~~1/QGIS2~~1.18/apps/qgis/./python/plugins\db_manager\db_plugins\connector.py", line 89, in _execute
raise DbError(e, sql)
File "C:/PROGRA~~1/QGIS2~~1.18/apps/qgis/./python/plugins\db_manager\db_plugins\plugin.py", line 66, in init
self.query = unicode(query) if query is not None else None
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 424: ordinal not in range(128)

Faut-il que je recharge la base cadastre du plugin ?

Merci par avance,

jusabatier · 2017-07-05T14:27:00Z

Après avoir fait les modification de mon dernier post, j'ai relancé QGis puis réimporté les données dans le schema PostGIS (eventuellement le supprimer puis recréer).

Par la suite celle-cis se chargent sans soucis.

mdouchin · 2017-07-06T09:06:20Z

Le commit 58beda1 doit régler le souci, et permettre au plugin Cadastre de fonctionner avec les version de QGIS avant et après le commit qgis/QGIS@14ab5eb

Merci de télécharger la dernière version "master" du plugin depuis ce site et de vérifier si c'est bon

mdouchin · 2017-07-13T13:23:29Z

Bug QGIS référencé par d'autres ici:
https://issues.qgis.org/issues/16833

MaelREBOUX · 2019-10-08T08:39:41Z

Vu que cela concernait la version 2.18 de QGIS : doit-on fermer ce ticket @mdouchin ?

mdouchin · 2019-10-08T15:29:08Z

on peut fermer. Le ticket n'est pas récent, un fix a été proposé (try catch si je me souviens...). On peut fermer, merci

boillodmanuel · 2021-03-23T01:00:56Z

Bonjour,
J'ai rencontré un problème similaire : UnicodeDecodeError: 'ascii' codec can't decode
Mon env :

QGIS : 3.18
Plugin Cadastre :1.10.2
OS : macos bigsur 11.2.3

J'ai fini par trouver comment corriger mon problème (la résolution de celui-ci est peut-être similaire)

Dans le fichier cadastre_import.py, fonction replaceParametersInScript, il manque l'encoding sur cette ligne :

QgisCadastrePlugin/cadastre/cadastre_import.py

Lines 1015 to 1016 in 2b2c0e3

    
           with open(scriptPath, 'w') as fout: 
        
               fout.write(data)

Voici le fix:

with open(scriptPath, 'w', encoding='utf-8') as fout:
    fout.write(data)

La locale système ne doit pas être configuré comme il faut dans QGIS.
Pour le vérifier, j'ai ouvert la console python (Plugins > Console Python) et exécuté le code suivant :

# Explicitly open with ascii encoding:
# ❌ Error:
#   UnicodeEncodeError: 'ascii' codec can't encode character '\u2122' in position 0: ordinal not in range(128)
with open('/tmp/test', 'w', encoding='ascii') as fout:
    fout.write(u"\u2122")

# Open without encoding specified:
# ❌ Error or ✅ Works depending of system locale :
#   UnicodeEncodeError: 'ascii' codec can't encode character '\u2122' in position 0: ordinal not in range(128)
with open('/tmp/test', 'w') as fout:
    fout.write(u"\u2122")
    
# Explicitly open with utf-8 encoding:
# ✅ Works
with open('/tmp/test', 'w', encoding='utf-8') as fout:
    fout.write(u"\u2122")

Pour forcer la locale dans QGIS, j'ai ajouté la variable d'environnement LC_ALL dans les préférences (interface en anglais) :.

Open QGIS > Preferences > System
Check "Use custom variable"
Add LC_ALL = en_US.UTF-8 or whatever you want.

Je ne suis pas sûr qu'il faille garder ce paramétrage mais ca aide pour débugger.

mdouchin added the bug label Jul 5, 2017

mdouchin self-assigned this Jul 5, 2017

mdouchin referenced this issue in qgis/QGIS Jul 5, 2017

More unicode/utf8 fixes

14ab5eb

mdouchin changed the title ~~Outil d'identification de parcelle - Erreur Python sur QGIS 2.18~~ Import / Identification - Erreur encodage Python sur QGIS 2.18.10 Jul 5, 2017

mdouchin mentioned this issue Jul 6, 2017

Problème d'import des données cadastrales #108

Closed

qgib mentioned this issue May 25, 2019

Encoding error in DB Manager qgis/QGIS#24655

Closed

MaelREBOUX added the Python remonte une erreur pur Python label Aug 19, 2019

MaelREBOUX added the à fermer ? label Oct 8, 2019

mdouchin closed this as completed Oct 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import / Identification - Erreur encodage Python sur QGIS 2.18.10 #106

Import / Identification - Erreur encodage Python sur QGIS 2.18.10 #106

mdouchin commented Jul 5, 2017 •

edited

Loading

mdouchin commented Jul 5, 2017

jusabatier commented Jul 5, 2017

jusabatier commented Jul 5, 2017 •

edited

Loading

mdouchin commented Jul 5, 2017

jusabatier commented Jul 5, 2017

mdouchin commented Jul 5, 2017

guillaumelaurant commented Jul 5, 2017

jusabatier commented Jul 5, 2017

guillaumelaurant commented Jul 5, 2017

jusabatier commented Jul 5, 2017

mdouchin commented Jul 6, 2017 •

edited

Loading

mdouchin commented Jul 13, 2017

MaelREBOUX commented Oct 8, 2019

mdouchin commented Oct 8, 2019

boillodmanuel commented Mar 23, 2021

Import / Identification - Erreur encodage Python sur QGIS 2.18.10 #106

Import / Identification - Erreur encodage Python sur QGIS 2.18.10 #106

Comments

mdouchin commented Jul 5, 2017 • edited Loading

mdouchin commented Jul 5, 2017

jusabatier commented Jul 5, 2017

jusabatier commented Jul 5, 2017 • edited Loading

mdouchin commented Jul 5, 2017

jusabatier commented Jul 5, 2017

mdouchin commented Jul 5, 2017

guillaumelaurant commented Jul 5, 2017

jusabatier commented Jul 5, 2017

guillaumelaurant commented Jul 5, 2017

jusabatier commented Jul 5, 2017

mdouchin commented Jul 6, 2017 • edited Loading

mdouchin commented Jul 13, 2017

MaelREBOUX commented Oct 8, 2019

mdouchin commented Oct 8, 2019

boillodmanuel commented Mar 23, 2021

mdouchin commented Jul 5, 2017 •

edited

Loading

jusabatier commented Jul 5, 2017 •

edited

Loading

mdouchin commented Jul 6, 2017 •

edited

Loading