gen_perf_cases fails to run

[whitelynx]

As of commit e417fd0, running gen_perf_cases with Python 2.7.1 gives a traceback:

# python2 gen_perf_cases.py 
generate tmp-aspn-casesTraceback (most recent call last):
  File "gen_perf_cases.py", line 276, in <module>
    gen_aspn_cases(limit)
  File "gen_perf_cases.py", line 24, in gen_aspn_cases
    recipe_dicts = eval(open(recipes_path).read())
IOError: [Errno 2] No such file or directory: '/home/whitelynx/as/code.as.com/db/aspn/recipes.pprint'

I haven't checked against earlier commits yet.

[livibetter]

I have no idea what perf is for, but I managed to produce this patch:

diff --git a/perf/gen_perf_cases.py b/perf/gen_perf_cases.py
index ecd4c54..5a5a841 100755
--- a/perf/gen_perf_cases.py
+++ b/perf/gen_perf_cases.py
@@ -20,7 +20,12 @@ def gen_aspn_cases(limit=0):
         return 
     os.makedirs(base_dir)
     sys.stdout.write("generate %s" % base_dir); sys.stdout.flush()
-    recipes_path = expanduser("~/as/code.as.com/db/aspn/recipes.pprint")
+    for recipes_path in (expanduser("~/as/code.as.com/db/aspn/recipes.pprint"), 'recipes.pprint'):
+      if exists(recipes_path):
+        break
+    else:
+      print >>sys.stderr, 'Can not find recipes.pprint'
+      sys.exit(1)
     recipe_dicts = eval(open(recipes_path).read())
     for i, r in enumerate(recipe_dicts):
         sys.stdout.write('.'); sys.stdout.flush()
@@ -28,8 +33,7 @@ def gen_aspn_cases(limit=0):
         f.write(r["desc"])
         f.close()

-        for j, c in enumerate(sorted(r["comments"],
-                        key=operator.itemgetter("pub_date"))):
+        for j, c in enumerate(sorted(r["comments"])):
             text = _markdown_from_aspn_html(c["comment"])
             headline = c["title"].strip()
             if headline:

I think this perf thing is pretty outdate or somehow messed up because comments doesn't have pub_date as you can see I remove the key for sorting.

Although it seems to generate the files, but perf.py runs into trouble (not going to look into this):

Time conversion of tmp-test-cases/*.text (plat=linux2):
  Markdown.pl: best of 3: 0.358s
Traceback (most recent call last):
  File "./perf.py", line 153, in <module>
    sys.exit( main(sys.argv) )
  File "./perf.py", line 150, in main
    timer(cases_dir, repeat=opts.repeat)
  File "./perf.py", line 82, in time_all
    time_markdown_py(cases_dir, repeat=repeat)
  File "./perf.py", line 42, in time_markdown_py
    markdowner.convert(content)
  File "../test/markdown.py", line 1614, in convert
    doc = self._transform()
  File "../test/markdown.py", line 1187, in _transform
    self.lines = prep.run(self.lines)
  File "../test/markdown.py", line 527, in run
    left_tag = self._get_left_tag(block)
  File "../test/markdown.py", line 484, in _get_left_tag
    return block[1:].replace(">", " ", 1).split()[0].lower()
IndexError: list index out of range

[whitelynx]

I thought I responded to this last week, but apparently not...

I have no idea what perf is for, but I managed to produce this patch:

gen_perf_cases.py is for generating the performance test cases that are referenced on the wiki: https://github.com/trentm/python-markdown2/wiki/Performance-Notes

Thanks for the patch! I'll try it out soon, and see if I can get perf.py running too.

[whitelynx]

Well, with your patch, the second of the two listed test cases (tmp-aspn-cases) works; however, the first (tmp-test-cases) gives me the same traceback you got:

# python2 perf.py -r 10 tmp-test-cases
Time conversion of tmp-test-cases/*.text (plat=linux2):
  Markdown.pl: best of 10: 0.337s
We've got a problem header!
Traceback (most recent call last):
  File "perf.py", line 153, in <module>
    sys.exit( main(sys.argv) )
  File "perf.py", line 150, in main
    timer(cases_dir, repeat=opts.repeat)
  File "perf.py", line 82, in time_all
    time_markdown_py(cases_dir, repeat=repeat)
  File "perf.py", line 42, in time_markdown_py
    markdowner.convert(content)
  File "../test/markdown.py", line 1614, in convert
    doc = self._transform()
  File "../test/markdown.py", line 1187, in _transform
    self.lines = prep.run(self.lines)
  File "../test/markdown.py", line 527, in run
    left_tag = self._get_left_tag(block)
  File "../test/markdown.py", line 484, in _get_left_tag
    return block[1:].replace(">", " ", 1).split()[0].lower()
IndexError: list index out of range

[livibetter]

I checkout last modification to perf.py @eb1cca9b867d4687fc46bc3928195c54e5f0121b, cleaned up tmp-test-cases, regenerated the tmp-test-cases.

Tada! No errors.

I went back and regenerated, then added exception catching. This test case tmp-test-cases/issue21_gt_escaping.text crashes markdown.py.

http://code.google.com/p/python-markdown2/issues/detail?id=21

>

<>

<b>

Can't believe this did that.

(edit: I am going to clean up the code a bit and open a pull request.)