Skip to content

Commit

Permalink
Performance improvement 12: Optimize NocaseDict._real_key()
Browse files Browse the repository at this point in the history 
Details:

- This change optimizes the implementaiton of NocaseDict._real_key()
  by replacing a check using isinstance() with a check for None,
  turning around the order of checks. Also, replaced if/elif/else
  with a sequence of if's.

  This eliminates a call to isinstance().

Improvement:

  Output files before: perf/tupleparse/perf_11-nocasedict_*
  Output files after:  perf/tupleparse/perf_12-nocasedict_*

Signed-off-by: Andreas Maier <maiera@de.ibm.com>
  • Loading branch information
@andy-maier
andy-maier committed Mar 2, 2019
1 parent 80c816d commit 19450c4
Show file tree
Hide file tree
Showing 4 changed files with 161 additions and 6 deletions.
41 changes: 41 additions & 0 deletions perf/tupleparse/perf_12-nocasedict_cprofile_20190302-165534_10000_500.out
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
9800503 function calls (9560498 primitive calls) in 6.034 seconds

Ordered by: internal time, call count
List reduced from 158 to 25 due to restriction <25>

ncalls tottime percall cumtime percall filename:lineno(function)
80000 0.901 0.000 1.196 0.000 collections.py:50(__init__)
140005 0.426 0.000 0.683 0.000 tupleparse.py:165(check_node)
140005 0.365 0.000 0.444 0.000 tupletree.py:96(startElement)
92 0.275 0.003 1.272 0.014 {built-in method Parse}
1740019 0.265 0.000 0.265 0.000 {isinstance}
250005 0.184 0.000 0.263 0.000 tupleparse.py:117(kids)
80000 0.175 0.000 0.189 0.000 _abcoll.py:548(update)
120004/1 0.174 0.000 4.752 4.752 tupleparse.py:2057(parse_any)
40000 0.170 0.000 1.363 0.000 cim_obj.py:4324(__init__)
100000/20000 0.160 0.000 2.843 0.000 tupleparse.py:323(list_of_matching)
30000 0.155 0.000 1.944 0.000 tupleparse.py:1318(parse_property)
80000 0.139 0.000 1.335 0.000 _nocasedict.py:90(__init__)
300007 0.120 0.000 0.174 0.000 tupletree.py:115(characters)
140005 0.109 0.000 0.572 0.000 expatreader.py:335(start_element)
260000 0.078 0.000 0.132 0.000 _utils.py:41(_ensure_unicode)
40000 0.073 0.000 0.673 0.000 tupleparse.py:2147(unpack_value)
300007 0.070 0.000 0.244 0.000 pyexpat.c:441(CharacterData)
570010 0.069 0.000 0.069 0.000 {method 'append' of 'list' objects}
10000 0.068 0.000 3.661 0.000 tupleparse.py:1128(parse_instance)
40000 0.066 0.000 0.229 0.000 cim_obj.py:2635(__setitem__)
20000 0.065 0.000 0.462 0.000 cim_obj.py:1252(keybindings)
80000 0.062 0.000 0.099 0.000 abc.py:148(__subclasscheck__)
10000 0.058 0.000 0.656 0.000 tupleparse.py:1375(parse_property_array)
550019 0.058 0.000 0.058 0.000 tupleparse.py:103(name)
60000 0.057 0.000 0.057 0.000 collections.py:71(__setitem__)


Results: profile=cprofile, response_counts=[10000],
response-sizes=[500], runid=12-nocasedict 20190302-165534
+------------+------------+------------+---------+--------------+-------------+
| Exp inst | Act inst | Response | XML | Parse time | Instances |
| size | size | Count | size | sec. | per sec |
+============+============+============+=========+==============+=============+
| 500 | 589 | 10000 | 5896919 | 6.12 | 1634 |
+------------+------------+------------+---------+--------------+-------------+
Binary file added perf/tupleparse/perf_12-nocasedict_cprofile_20190302-165534_10000_500.profile
View file
Binary file not shown.
113 changes: 113 additions & 0 deletions perf/tupleparse/perf_12-nocasedict_pyinst_20190302-165607_10000_500.out
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@

_ ._ __/__ _ _ _ _ _/_ Recorded: 16:56:33 Samples: 2641
/_//_/// /_\ / //_// / //_'/ // Duration: 5.284
/ _/ v3.0.1

Program: tests/manualtest/run_response_performance.py -i -c 1000 10000 -s 500 -n 25 -l -p pyinst -r 12-nocasedict

5.284 execute_test_code run_response_performance.py:193
├─ 4.136 parse_cim pywbem/tupleparse.py:384
│ └─ 4.136 one_child pywbem/tupleparse.py:250
│ └─ 4.136 parse_any pywbem/tupleparse.py:2057
│ └─ 4.136 parse_message pywbem/tupleparse.py:1631
│ └─ 4.136 one_child pywbem/tupleparse.py:250
│ └─ 4.136 parse_any pywbem/tupleparse.py:2057
│ └─ 4.136 parse_simplersp pywbem/tupleparse.py:1892
│ └─ 4.136 one_child pywbem/tupleparse.py:250
│ └─ 4.136 parse_any pywbem/tupleparse.py:2057
│ └─ 4.136 parse_imethodresponse pywbem/tupleparse.py:1947
│ └─ 4.136 list_of_various pywbem/tupleparse.py:302
│ └─ 4.136 parse_any pywbem/tupleparse.py:2057
│ └─ 4.136 parse_ireturnvalue pywbem/tupleparse.py:2014
│ └─ 4.128 list_of_same pywbem/tupleparse.py:339
│ └─ 4.107 parse_any pywbem/tupleparse.py:2057
│ └─ 4.073 parse_value_namedinstance pywbem/tupleparse.py:538
│ ├─ 3.310 parse_instance pywbem/tupleparse.py:1128
│ │ ├─ 2.842 list_of_matching pywbem/tupleparse.py:323
│ │ │ └─ 2.743 parse_any pywbem/tupleparse.py:2057
│ │ │ ├─ 2.123 parse_property pywbem/tupleparse.py:1318
│ │ │ │ ├─ 1.442 __init__ pywbem/cim_obj.py:4324
│ │ │ │ │ ├─ 1.153 qualifiers pywbem/cim_obj.py:4789
│ │ │ │ │ │ └─ 1.132 __init__ pywbem/_nocasedict.py:90
│ │ │ │ │ │ ├─ 0.635 __init__ collections.py:50
│ │ │ │ │ │ │ [8 frames hidden] collections, _abcoll, abc, _weakrefset
│ │ │ │ │ │ └─ 0.497 [self] 
│ │ │ │ │ └─ 0.068 [self] 
│ │ │ │ ├─ 0.361 unpack_value pywbem/tupleparse.py:2147
│ │ │ │ │ ├─ 0.198 list_of_matching pywbem/tupleparse.py:323
│ │ │ │ │ │ └─ 0.165 parse_any pywbem/tupleparse.py:2057
│ │ │ │ │ │ └─ 0.115 parse_value pywbem/tupleparse.py:450
│ │ │ │ │ │ └─ 0.063 check_node pywbem/tupleparse.py:165
│ │ │ │ │ └─ 0.126 unpack_single_value pywbem/tupleparse.py:2178
│ │ │ │ │ └─ 0.073 unpack_numeric pywbem/tupleparse.py:2291
│ │ │ │ ├─ 0.133 check_node pywbem/tupleparse.py:165
│ │ │ │ │ └─ 0.107 [self] 
│ │ │ │ └─ 0.109 [self] 
│ │ │ ├─ 0.541 parse_property_array pywbem/tupleparse.py:1375
│ │ │ │ ├─ 0.211 __init__ pywbem/cim_obj.py:4324
│ │ │ │ │ └─ 0.107 qualifiers pywbem/cim_obj.py:4789
│ │ │ │ │ └─ 0.102 __init__ pywbem/_nocasedict.py:90
│ │ │ │ │ └─ 0.077 __init__ collections.py:50
│ │ │ │ │ [8 frames hidden] collections, _abcoll, abc, _weakrefset
│ │ │ │ ├─ 0.202 unpack_value pywbem/tupleparse.py:2147
│ │ │ │ │ └─ 0.170 list_of_matching pywbem/tupleparse.py:323
│ │ │ │ │ └─ 0.151 parse_any pywbem/tupleparse.py:2057
│ │ │ │ │ └─ 0.143 parse_value_array pywbem/tupleparse.py:469
│ │ │ │ │ └─ 0.105 list_of_various pywbem/tupleparse.py:302
│ │ │ │ │ └─ 0.071 parse_any pywbem/tupleparse.py:2057
│ │ │ │ └─ 0.061 [self] 
│ │ │ └─ 0.070 [self] 
│ │ ├─ 0.183 __init__ pywbem/cim_obj.py:2253
│ │ │ ├─ 0.083 qualifiers pywbem/cim_obj.py:2460
│ │ │ │ └─ 0.076 __init__ pywbem/_nocasedict.py:90
│ │ │ │ └─ 0.064 __init__ collections.py:50
│ │ │ │ [8 frames hidden] collections, _abcoll, abc, _weakrefset
│ │ │ └─ 0.077 properties pywbem/cim_obj.py:2405
│ │ │ └─ 0.072 __init__ pywbem/_nocasedict.py:90
│ │ │ └─ 0.066 __init__ collections.py:50
│ │ │ [8 frames hidden] collections, _abcoll, abc, _weakrefset
│ │ ├─ 0.153 __setitem__ pywbem/cim_obj.py:2635
│ │ │ └─ 0.073 __setitem__ pywbem/_nocasedict.py:232
│ │ └─ 0.095 check_node pywbem/tupleparse.py:165
│ │ └─ 0.069 [self] 
│ ├─ 0.405 parse_instancename pywbem/tupleparse.py:942
│ │ ├─ 0.172 list_of_various pywbem/tupleparse.py:302
│ │ │ └─ 0.153 parse_any pywbem/tupleparse.py:2057
│ │ │ └─ 0.132 parse_keybinding pywbem/tupleparse.py:1013
│ │ │ └─ 0.098 one_child pywbem/tupleparse.py:250
│ │ │ └─ 0.086 parse_any pywbem/tupleparse.py:2057
│ │ │ └─ 0.069 parse_keyvalue pywbem/tupleparse.py:1034
│ │ └─ 0.154 __init__ pywbem/cim_obj.py:1102
│ │ └─ 0.125 keybindings pywbem/cim_obj.py:1252
│ │ └─ 0.069 __init__ pywbem/_nocasedict.py:90
│ └─ 0.268 path pywbem/cim_obj.py:2499
│ └─ 0.264 copy pywbem/cim_obj.py:1438
│ └─ 0.244 __init__ pywbem/cim_obj.py:1102
│ └─ 0.199 keybindings pywbem/cim_obj.py:1252
│ └─ 0.080 __init__ pywbem/_nocasedict.py:90
│ └─ 0.061 __init__ collections.py:50
│ [8 frames hidden] collections, _abcoll, abc, _weakrefset
└─ 1.097 xml_to_tupletree_sax pywbem/tupletree.py:125
└─ 1.089 parseString xml/sax/__init__.py:35
[6 frames hidden] xml
1.087 feed xml/sax/expatreader.py:209
├─ 0.549 StartElement ../../../../../private/tmp/python@2-20190212-94033-m4zuyi/Python-2.7.15/Modules/pyexpat.c:566
│ └─ 0.531 start_element xml/sax/expatreader.py:335
│ [4 frames hidden] xml
│ 0.446 startElement pywbem/tupletree.py:96
│ └─ 0.409 [self] 
├─ 0.191 CharacterData ../../../../../private/tmp/python@2-20190212-94033-m4zuyi/Python-2.7.15/Modules/pyexpat.c:441
│ └─ 0.151 characters pywbem/tupletree.py:115
└─ 0.116 EndElement ../../../../../private/tmp/python@2-20190212-94033-m4zuyi/Python-2.7.15/Modules/pyexpat.c:618
└─ 0.091 end_element xml/sax/expatreader.py:338
[2 frames hidden] xml


Results: profile=pyinst, response_counts=[10000],
response-sizes=[500], runid=12-nocasedict 20190302-165607
+------------+------------+------------+---------+--------------+-------------+
| Exp inst | Act inst | Response | XML | Parse time | Instances |
| size | size | Count | size | sec. | per sec |
+============+============+============+=========+==============+=============+
| 500 | 589 | 10000 | 5896919 | 5.33 | 1877 |
+------------+------------+------------+---------+--------------+-------------+
13 changes: 7 additions & 6 deletions pywbem/_nocasedict.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -204,14 +204,15 @@ def _real_key(self, key):
Return the normalized key to be used for the internal dictionary,
from the input key.
"""
if isinstance(key, six.string_types):
if key is not None:
return key.lower()
elif self.allow_unnamed_keys and key is None:

if self.allow_unnamed_keys:
return None
else:
raise TypeError(
_format("NocaseDict key {0!A} must be a string, but is {1}",
key, type(key)))

raise TypeError(
_format("NocaseDict key {0!A} must be a string, but is {1}",
key, type(key)))

def __getitem__(self, key):
"""
Expand Down

0 comments on commit 19450c4

Please sign in to comment.