Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

Decodes input and encodes output

Currently cinderclient doesn't handle properly incoming and outgoing
encode / decode process. As a solution for this, this patch implements a
decoding process for all data incoming from the user side and encodes
everything going out of the client, i.e: http requests, prints, etc.

This patch introduces a new module (strutils.py) taken from
oslo-incubator in order to use 2 of the functions present in it:

About safe_(decode|encode):

    Both functions try to encode / decode the incoming text using the
    stdin encoding, fallback to python's default encoding if that
    returns None or to UTF-8 as the last option.

    In both functions only basestring objects are accepted and they both
    raise TypeError if an object of another type is passed.

About the general cinderclient changes:

    In order to better support non-ASCII characters, it is a good
    practice to use unicode interanlly and encode everything that has to
    go out. This patch aims to do that and introduces this behaviour in
    the client.

Testing:

    A good test (besides using tox) is to use cinder client with and
    without setting any locale (export LANG=).

Fixes bug: 1130572

Change-Id: Idb7d06954c29e003f68a0c4aa0b80ecc7017cbc9
  • Loading branch information...
commit 03a4806d972ac150e717451566e9c4c35a141d8f 1 parent 7369310
Flavio Percoco Premoli FlaPer87 authored
133 cinderclient/openstack/common/strutils.py
... ... @@ -0,0 +1,133 @@
  1 +# vim: tabstop=4 shiftwidth=4 softtabstop=4
  2 +
  3 +# Copyright 2011 OpenStack LLC.
  4 +# All Rights Reserved.
  5 +#
  6 +# Licensed under the Apache License, Version 2.0 (the "License"); you may
  7 +# not use this file except in compliance with the License. You may obtain
  8 +# a copy of the License at
  9 +#
  10 +# http://www.apache.org/licenses/LICENSE-2.0
  11 +#
  12 +# Unless required by applicable law or agreed to in writing, software
  13 +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  14 +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
  15 +# License for the specific language governing permissions and limitations
  16 +# under the License.
  17 +
  18 +"""
  19 +System-level utilities and helper functions.
  20 +"""
  21 +
  22 +import logging
  23 +import sys
  24 +
  25 +LOG = logging.getLogger(__name__)
  26 +
  27 +
  28 +def int_from_bool_as_string(subject):
  29 + """
  30 + Interpret a string as a boolean and return either 1 or 0.
  31 +
  32 + Any string value in:
  33 +
  34 + ('True', 'true', 'On', 'on', '1')
  35 +
  36 + is interpreted as a boolean True.
  37 +
  38 + Useful for JSON-decoded stuff and config file parsing
  39 + """
  40 + return bool_from_string(subject) and 1 or 0
  41 +
  42 +
  43 +def bool_from_string(subject):
  44 + """
  45 + Interpret a string as a boolean.
  46 +
  47 + Any string value in:
  48 +
  49 + ('True', 'true', 'On', 'on', 'Yes', 'yes', '1')
  50 +
  51 + is interpreted as a boolean True.
  52 +
  53 + Useful for JSON-decoded stuff and config file parsing
  54 + """
  55 + if isinstance(subject, bool):
  56 + return subject
  57 + if isinstance(subject, basestring):
  58 + if subject.strip().lower() in ('true', 'on', 'yes', '1'):
  59 + return True
  60 + return False
  61 +
  62 +
  63 +def safe_decode(text, incoming=None, errors='strict'):
  64 + """
  65 + Decodes incoming str using `incoming` if they're
  66 + not already unicode.
  67 +
  68 + :param incoming: Text's current encoding
  69 + :param errors: Errors handling policy. See here for valid
  70 + values http://docs.python.org/2/library/codecs.html
  71 + :returns: text or a unicode `incoming` encoded
  72 + representation of it.
  73 + :raises TypeError: If text is not an isntance of basestring
  74 + """
  75 + if not isinstance(text, basestring):
  76 + raise TypeError("%s can't be decoded" % type(text))
  77 +
  78 + if isinstance(text, unicode):
  79 + return text
  80 +
  81 + if not incoming:
  82 + incoming = (sys.stdin.encoding or
  83 + sys.getdefaultencoding())
  84 +
  85 + try:
  86 + return text.decode(incoming, errors)
  87 + except UnicodeDecodeError:
  88 + # Note(flaper87) If we get here, it means that
  89 + # sys.stdin.encoding / sys.getdefaultencoding
  90 + # didn't return a suitable encoding to decode
  91 + # text. This happens mostly when global LANG
  92 + # var is not set correctly and there's no
  93 + # default encoding. In this case, most likely
  94 + # python will use ASCII or ANSI encoders as
  95 + # default encodings but they won't be capable
  96 + # of decoding non-ASCII characters.
  97 + #
  98 + # Also, UTF-8 is being used since it's an ASCII
  99 + # extension.
  100 + return text.decode('utf-8', errors)
  101 +
  102 +
  103 +def safe_encode(text, incoming=None,
  104 + encoding='utf-8', errors='strict'):
  105 + """
  106 + Encodes incoming str/unicode using `encoding`. If
  107 + incoming is not specified, text is expected to
  108 + be encoded with current python's default encoding.
  109 + (`sys.getdefaultencoding`)
  110 +
  111 + :param incoming: Text's current encoding
  112 + :param encoding: Expected encoding for text (Default UTF-8)
  113 + :param errors: Errors handling policy. See here for valid
  114 + values http://docs.python.org/2/library/codecs.html
  115 + :returns: text or a bytestring `encoding` encoded
  116 + representation of it.
  117 + :raises TypeError: If text is not an isntance of basestring
  118 + """
  119 + if not isinstance(text, basestring):
  120 + raise TypeError("%s can't be encoded" % type(text))
  121 +
  122 + if not incoming:
  123 + incoming = (sys.stdin.encoding or
  124 + sys.getdefaultencoding())
  125 +
  126 + if isinstance(text, unicode):
  127 + return text.encode(encoding, errors)
  128 + elif text and encoding != incoming:
  129 + # Decode text before encoding it with `encoding`
  130 + text = safe_decode(text, incoming, errors)
  131 + return text.encode(encoding, errors)
  132 +
  133 + return text
8 cinderclient/shell.py
@@ -30,6 +30,7 @@
30 30 from cinderclient import client
31 31 from cinderclient import exceptions as exc
32 32 import cinderclient.extension
  33 +from cinderclient.openstack.common import strutils
33 34 from cinderclient import utils
34 35 from cinderclient.v1 import shell as shell_v1
35 36 from cinderclient.v2 import shell as shell_v2
@@ -486,13 +487,16 @@ def start_section(self, heading):
486 487
487 488 def main():
488 489 try:
489   - OpenStackCinderShell().main(sys.argv[1:])
  490 + OpenStackCinderShell().main(map(strutils.safe_decode, sys.argv[1:]))
490 491 except KeyboardInterrupt:
491 492 print >> sys.stderr, "... terminating cinder client"
492 493 sys.exit(130)
493 494 except Exception, e:
494 495 logger.debug(e, exc_info=1)
495   - print >> sys.stderr, "ERROR: %s" % e.message
  496 + message = e.message
  497 + if not isinstance(message, basestring):
  498 + message = str(message)
  499 + print >> sys.stderr, "ERROR: %s" % strutils.safe_encode(message)
496 500 sys.exit(1)
497 501
498 502
14 cinderclient/utils.py
... ... @@ -1,4 +1,3 @@
1   -import locale
2 1 import os
3 2 import re
4 3 import sys
@@ -7,6 +6,7 @@
7 6 import prettytable
8 7
9 8 from cinderclient import exceptions
  9 +from cinderclient.openstack.common import strutils
10 10
11 11
12 12 def arg(*args, **kwargs):
@@ -143,14 +143,14 @@ def print_list(objs, fields, formatters={}):
143 143 row.append(data)
144 144 pt.add_row(row)
145 145
146   - print pt.get_string(sortby=fields[0])
  146 + print strutils.safe_encode(pt.get_string(sortby=fields[0]))
147 147
148 148
149 149 def print_dict(d, property="Property"):
150 150 pt = prettytable.PrettyTable([property, 'Value'], caching=False)
151 151 pt.aligns = ['l', 'l']
152 152 [pt.add_row(list(r)) for r in d.iteritems()]
153   - print pt.get_string(sortby=property)
  153 + print strutils.safe_encode(pt.get_string(sortby=property))
154 154
155 155
156 156 def find_resource(manager, name_or_id):
@@ -164,7 +164,7 @@ def find_resource(manager, name_or_id):
164 164
165 165 # now try to get entity as uuid
166 166 try:
167   - uuid.UUID(str(name_or_id))
  167 + uuid.UUID(strutils.safe_decode(name_or_id))
168 168 return manager.get(name_or_id)
169 169 except (ValueError, exceptions.NotFound):
170 170 pass
@@ -180,11 +180,7 @@ def find_resource(manager, name_or_id):
180 180 return manager.find(name=name_or_id)
181 181 except exceptions.NotFound:
182 182 try:
183   - # For command-line arguments that are in Unicode
184   - encoding = (locale.getpreferredencoding() or
185   - sys.stdin.encoding or
186   - 'UTF-8')
187   - return manager.find(display_name=(name_or_id.decode(encoding)))
  183 + return manager.find(display_name=name_or_id)
188 184 except (UnicodeDecodeError, exceptions.NotFound):
189 185 try:
190 186 # Volumes does not have name, but display_name
2  openstack-common.conf
... ... @@ -1,7 +1,7 @@
1 1 [DEFAULT]
2 2
3 3 # The list of modules to copy from openstack-common
4   -modules=setup,version
  4 +modules=setup,version,strutils
5 5
6 6 # The base module to hold the copy of openstack.common
7 7 base=cinderclient

Git Notes

review

Verified+2: Jenkins
Approved+1: John Griffith <john.griffith@solidfire.com>
Code-Review+2: John Griffith <john.griffith@solidfire.com>
Submitted-by: Jenkins
Submitted-at: Fri, 29 Mar 2013 17:23:46 +0000
Reviewed-on: https://review.openstack.org/23089
Project: openstack/python-cinderclient
Branch: refs/heads/master

0 comments on commit 03a4806

Please sign in to comment.
Something went wrong with that request. Please try again.