Skip to content
This repository

Psych-based yaml in Ruby 1.9.3 too slow #84

Open
mattneub opened this Issue September 15, 2012 · 51 comments

12 participants

Matt Neuburg Aaron Patterson Kevin Rood Kevin Menard Bert Goethals Joe Rafaniello Gernot Kogler Jason Frey Jay Feldblum Jack Royal-Gordon Aaron Stone Marten Veldthuis
Matt Neuburg

Here is a script to illustrate the difference in timing between load_file under Psych and under Syck. It is simple-minded but is very illustrative of my actual use case. The output on my machine reads:

Psych
17.156296968460083
Syck
5.016614198684692
Resulting loaded hash the same? true

So you can see that Psych is taking over three times as long. (Timings on Ruby 1.8.7 are comparable to the Syck time shown here. It was the switch to Ruby 1.9.3 that alerted me to the issue.) Here's the script:

require 'yaml'

h1 = h2 = nil
f = Pathname.new(__FILE__).dirname + "autoglossary.yaml"

t = Time.new.to_f
300.times {h1 = YAML.load_file(f)}
puts "Psych", Time.new.to_f - t

YAML::ENGINE.yamler = 'syck'

t = Time.new.to_f
300.times {h2 = YAML.load_file(f)}
puts "Syck", Time.new.to_f - t

puts "Resulting loaded hash the same? #{h1 == h2}"


=begin here's what autoglossary.yaml looks like
---
index: &2161004220
  :path: !ruby/object:Pathname
    path: index.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/index.txt
  :linetext: Script Debugger Help
script debugger help: *2161004220
develop:
  :path: !ruby/object:Pathname
    path: indexfolder/develop.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/develop.txt
  :linetext: Develop
debug:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debug.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debug.txt
  :linetext: Debug
breakpoi: &2160978360
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/breakpoi.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/breakpoi.txt
  :linetext: Breakpoints
breakpoints: *2160978360
breakpointsinsp: &2153040020
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/breakpoifolder/breakpointsinsp.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/breakpoifolder/breakpointsinsp.txt
  :linetext: Breakpoints Inspector
breakpoints inspector: *2153040020
executet: &2153027260
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/breakpoifolder/executet.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/breakpoifolder/executet.txt
  :linetext: Execute to Here
execute to here: *2153027260
tempbreak: &2152998760
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/breakpoifolder/tempbreak.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/breakpoifolder/tempbreak.txt
  :linetext: Temporary Breakpoints
temporary breakpoints: *2152998760
callchain: &2152944680
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/callchain.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/callchain.txt
  :linetext: Call Stack
call stack: *2152944680
codecove: &2152936000
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/codecove.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/codecove.txt
  :linetext: Code Coverage
code coverage: *2152936000
exceptio: &2152928080
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/exceptio.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/exceptio.txt
  :linetext: Exceptions
exceptions: *2152928080
executew: &2152907460
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/executew.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/executew.txt
  :linetext: Execute When Debugging
execute when debugging: *2152907460
expressi: &2152888420
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/expressi.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/expressi.txt
  :linetext: Expressions
expressions: *2152888420
external: &2152867220
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/external.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/external.txt
  :linetext: External Debugging
external debugging: *2152867220
mini: &2152779820
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/mini.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/mini.txt
  :linetext: The Mini Debugger
the mini debugger: *2152779820
pause:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/pause.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/pause.txt
  :linetext: Pause
step:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/step.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/step.txt
  :linetext: Step
trace:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/trace.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/trace.txt
  :linetext: Trace
variablesdebug: &2152445340
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/debugfolder/variablesdebug.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/debugfolder/variablesdebug.txt
  :linetext: Variables (Debug Mode)
variables (debug mode): *2152445340
edit:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/edit.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/edit.txt
  :linetext: Edit
editinga: &2152290020
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editinga.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editinga.txt
  :linetext: Editing and Navigation
editing and navigation: *2152290020
balance: &2152181400
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/balance.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/balance.txt
  :linetext: Delimiters
delimiters: *2152181400
bbedit: &2152117320
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/bbedit.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/bbedit.txt
  :linetext: External Editor
external editor: *2152117320
block: &2151906120
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/block.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/block.txt
  :linetext: Block Structure
block structure: *2151906120
clipping: &2151882160
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/clipping.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/clipping.txt
  :linetext: Clippings
clippings: *2151882160
clippingdetails: &2151876760
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/clippingfolder/clippingdetails.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/clippingfolder/clippingdetails.txt
  :linetext: How Clippings Work
how clippings work: *2151876760
comment:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/comment.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/comment.txt
  :linetext: Comment
completion: &2164232840
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/completion.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/completion.txt
  :linetext: Text Completion
text completion: *2164232840
dragdrop: &2164229640
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/dragdrop.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/dragdrop.txt
  :linetext: Inserting Content
inserting content: *2164229640
find:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/find.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/find.txt
  :linetext: Find
goto: &2164223040
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/goto.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/goto.txt
  :linetext: Go To Line
go to line: *2164223040
misctype: &2164219520
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/misctype.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/misctype.txt
  :linetext: Miscellaneous Typing and Selection
miscellaneous typing and selection: *2164219520
navigate:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/navigate.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/navigate.txt
  :linetext: Navigate
shift:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/shift.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/shift.txt
  :linetext: Shift
split: &2164209620
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/split.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/split.txt
  :linetext: Splitting the Editor
splitting the editor: *2164209620
substitution: &2164206420
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/substitution.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/substitution.txt
  :linetext: Text Substitution
text substitution: *2164206420
tab:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/tab.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/tab.txt
  :linetext: Tab
tell: &2164199720
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/editingafolder/tell.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/editingafolder/tell.txt
  :linetext: Tell Blocks and Terms Blocks
tell blocks and terms blocks: *2164199720
scriptwi: &2164196400
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwi.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwi.txt
  :linetext: Script Window
script window: *2164196400
default: &2164192800
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/default.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/default.txt
  :linetext: Default Script Size and State
default script size and state: *2164192800
view: &2164189460
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/view.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/view.txt
  :linetext: View Options in a Script Window
view options in a script window: *2164189460
chevron: &2164185940
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/chevron.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/chevron.txt
  :linetext: Raw (Chevron) Syntax
raw (chevron) syntax: *2164185940
invisibl: &2164182460
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/invisibl.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/invisibl.txt
  :linetext: Invisibles
invisibles: *2164182460
linenumb: &2164178920
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/linenumb.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/linenumb.txt
  :linetext: Line Numbers
line numbers: *2164178920
spaces:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/spaces.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/spaces.txt
  :linetext: Spaces
tabstops: &2164172500
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/tabstops.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/tabstops.txt
  :linetext: Tab Stops
tab stops: *2164172500
wrap: &2164169180
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/wrap.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/editfolder/scriptwifolder/viewfolder/wrap.txt
  :linetext: Line Wrapping
line wrapping: *2164169180
run:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/run.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/run.txt
  :linetext: Run
compile:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/compile.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/compile.txt
  :linetext: Compile
errors:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/errors.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/errors.txt
  :linetext: Errors
leaks:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/errorsfolder/leaks.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/errorsfolder/leaks.txt
  :linetext: Leaks
execute:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/execute.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/execute.txt
  :linetext: Execute
handlers: &2164148440
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/executefolder/handlers.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/executefolder/handlers.txt
  :linetext: Testing Handlers
testing handlers: *2164148440
log: &2164144740
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/log.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/log.txt
  :linetext: Event Log
event log: *2164144740
logwindow: &2164141360
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/logfolder/logwindow.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/logfolder/logwindow.txt
  :linetext: Event Log Window
event log window: *2164141360
parent: &2164137740
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/parent.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/parent.txt
  :linetext: Parent Script
parent script: *2164137740
record:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/record.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/record.txt
  :linetext: Record
result:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/result.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/result.txt
  :linetext: Result
target: &2164125920
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/target.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/target.txt
  :linetext: Default Target
default target: *2164125920
times:
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/times.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/times.txt
  :linetext: Times
variable: &2164118680
  :path: !ruby/object:Pathname
    path: indexfolder/developfolder/runfolder/variable.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/developfolder/runfolder/variable.txt
  :linetext: Variables
variables: *2164118680
explore:
  :path: !ruby/object:Pathname
    path: indexfolder/explore.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explore.txt
  :linetext: Explore
dictiona: &2164111140
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictiona.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictiona.txt
  :linetext: Dictionary
dictionary: *2164111140
appexplorer: &2164107160
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/appexplorer.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/appexplorer.txt
  :linetext: Application Explorer
application explorer: *2164107160
tellcontextinspector: &2164103460
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/appexplorerfolder/tellcontextinspector.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/appexplorerfolder/tellcontextinspector.txt
  :linetext: Tell Context Inspector
tell context inspector: *2164103460
dictwindow: &2164099860
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindow.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindow.txt
  :linetext: Dictionary Window
dictionary window: *2164099860
backandf: &2164095820
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/backandf.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/backandf.txt
  :linetext: Back and Forward
back and forward: *2164095820
browser: &2164092300
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/browser.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/browser.txt
  :linetext: Terminology List
terminology list: *2164092300
browseritemtypes: &2164088500
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/browserfolder/browseritemtypes.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/browserfolder/browseritemtypes.txt
  :linetext: Types of Entities Shown in the Terminology List
types of entities shown in the terminology list: *2164088500
hierarch: &2164085300
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/hierarch.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/hierarch.txt
  :linetext: Hierarchies and Diagrams
hierarchies and diagrams: *2164085300
info: &2164082020
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/info.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/info.txt
  :linetext: Dictionary Info Pane
dictionary info pane: *2164082020
codes: &2164078400
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/codes.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/codes.txt
  :linetext: Apple Event Codes
apple event codes: *2164078400
extra: &2164075220
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/extra.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/extra.txt
  :linetext: Extra Documentation
extra documentation: *2164075220
inherita: &2164071860
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/inherita.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/inherita.txt
  :linetext: Inheritance
inheritance: *2164071860
size:
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/size.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/infofolder/size.txt
  :linetext: Size
lookupde: &2164064680
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/lookupde.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/lookupde.txt
  :linetext: Look Up Definition
look up definition: *2164064680
otherthi: &2164061100
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/otherthi.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/otherthi.txt
  :linetext: Miscellaneous Dictionary Actions
miscellaneous dictionary actions: *2164061100
searchin: &2164057540
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/dictwindowfolder/searchin.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/dictwindowfolder/searchin.txt
  :linetext: Search in Dictionary
search in dictionary: *2164057540
opendict: &2164053980
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendict.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendict.txt
  :linetext: Open Dictionary Window
open dictionary window: *2164053980
currentappdict: &2164050040
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendictfolder/currentappdict.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendictfolder/currentappdict.txt
  :linetext: Running Applications
running applications: *2164050040
currentcontextdict: &2164046240
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendictfolder/currentcontextdict.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendictfolder/currentcontextdict.txt
  :linetext: Current Context
current context: *2164046240
dictionariesinsp: &2164042300
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendictfolder/dictionariesinsp.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendictfolder/dictionariesinsp.txt
  :linetext: Dictionaries Inspector
dictionaries inspector: *2164042300
openanydict: &2164038420
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendictfolder/openanydict.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendictfolder/openanydict.txt
  :linetext: Open Any Dictionary Window
open any dictionary window: *2164038420
recentappdict: &2164034140
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendictfolder/recentappdict.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendictfolder/recentappdict.txt
  :linetext: Recent and Favorite Applications
recent and favorite applications: *2164034140
scriptingadditionsdict: &2164029860
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/dictionafolder/opendictfolder/scriptingadditionsdict.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/dictionafolder/opendictfolder/scriptingadditionsdict.txt
  :linetext: Scripting Additions
scripting additions: *2164029860
explorer:
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorer.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorer.txt
  :linetext: Explorer
aeprint:
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/aeprint.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/aeprint.txt
  :linetext: AEPrint
best:
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/best.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/best.txt
  :linetext: Best
outliner:
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/bestfolder/outliner.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/bestfolder/outliner.txt
  :linetext: Outliner
outlinermore: &2164010660
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/bestfolder/outlinermore.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/bestfolder/outlinermore.txt
  :linetext: Outliner Editing
outliner editing: *2164010660
explorerviewoptions: &2164006160
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/explorerviewoptions.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/explorerviewoptions.txt
  :linetext: Explorer View Options and Actions
explorer view options and actions: *2164006160
explorerwindows: &2164000500
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/explorerwindows.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/explorerwindows.txt
  :linetext: Explorer Windows
explorer windows: *2164000500
source:
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/source.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/source.txt
  :linetext: Source
whereexplorers: &2163993560
  :path: !ruby/object:Pathname
    path: indexfolder/explorefolder/explorerfolder/whereexplorers.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/explorefolder/explorerfolder/whereexplorers.txt
  :linetext: Where Explorers Appear
where explorers appear: *2163993560
openinga: &2163989600
  :path: !ruby/object:Pathname
    path: indexfolder/openinga.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openinga.txt
  :linetext: Opening and Saving Scripts
opening and saving scripts: *2163989600
bundle:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/bundle.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/bundle.txt
  :linetext: Bundle
descript: &2163980940
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/descript.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/descript.txt
  :linetext: Description
description: *2163980940
library:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/library.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/library.txt
  :linetext: Library
librarytech: &2163972700
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/libraryfolder/librarytech.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/libraryfolder/librarytech.txt
  :linetext: Technical Details About Libraries
technical details about libraries: *2163972700
manifest:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/manifest.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/manifest.txt
  :linetext: Manifest
open:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/open.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/open.txt
  :linetext: Open
chooser: &2163961560
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/openfolder/chooser.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/openfolder/chooser.txt
  :linetext: The Template Chooser
the template chooser: *2163961560
compatib: &2163957700
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/openfolder/compatib.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/openfolder/compatib.txt
  :linetext: Compatibility
compatibility: *2163957700
importre: &2163953640
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/openfolder/importre.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/openfolder/importre.txt
  :linetext: Opening a Compiled Script as Text
opening a compiled script as text: *2163953640
tabs:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/openfolder/tabs.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/openfolder/tabs.txt
  :linetext: Tabs
save:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/save.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/save.txt
  :linetext: Save
fileowne: &2163941980
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/fileowne.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/fileowne.txt
  :linetext: File Owner
file owner: *2163941980
formats:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/formats.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/formats.txt
  :linetext: Formats
applicat: &2163934740
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/formatsfolder/applicat.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/formatsfolder/applicat.txt
  :linetext: Application
application: *2163934740
compiled: &2163930160
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/formatsfolder/compiled.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/formatsfolder/compiled.txt
  :linetext: Compiled Script
compiled script: *2163930160
text:
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/formatsfolder/text.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/formatsfolder/text.txt
  :linetext: Text
runonly: &2163922280
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/runonly.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/runonly.txt
  :linetext: Run-Only Script
run-only script: *2163922280
spotlight: &2163918000
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/spotlight.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/spotlight.txt
  :linetext: Spotlight and Quick Look
spotlight and quick look: *2163918000
whatissaved: &2163914400
  :path: !ruby/object:Pathname
    path: indexfolder/openingafolder/savefolder/whatissaved.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/openingafolder/savefolder/whatissaved.txt
  :linetext: What Is Saved
what is saved: *2163914400
referenc: &2163910340
  :path: !ruby/object:Pathname
    path: indexfolder/referenc.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referenc.txt
  :linetext: Reference
reference: *2163910340
faq: &2163906600
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faq.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faq.txt
  :linetext: Frequently Asked Questions
frequently asked questions: *2163906600
appsopen: &2163903160
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faqfolder/appsopen.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faqfolder/appsopen.txt
  :linetext: Why Do Applications Open Spontaneously?
why do applications open spontaneously?: *2163903160
lineendings: &2163899580
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faqfolder/lineendings.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faqfolder/lineendings.txt
  :linetext: What's The Big Deal With Line Endings?
what's the big deal with line endings?: *2163899580
reformatting: &2163895900
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faqfolder/reformatting.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faqfolder/reformatting.txt
  :linetext: Hey, Script Debugger Changed My Formatting!
hey, script debugger changed my formatting!: *2163895900
scriptab: &2163892440
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faqfolder/scriptab.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faqfolder/scriptab.txt
  :linetext: How Do I Script Script Debugger?
how do i script script debugger?: *2163892440
sediffs: &2163888100
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faqfolder/sediffs.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faqfolder/sediffs.txt
  :linetext: Is Script Debugger's AppleScript the Same as AppleScript Editor's?
is script debugger's applescript the same as applescript editor's?: *2163888100
whatsinstalledwhere: &2163884400
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/faqfolder/whatsinstalledwhere.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/faqfolder/whatsinstalledwhere.txt
  :linetext: What's Installed Where?
what's installed where?: *2163884400
helpglossary: &2163880260
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossary.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossary.txt
  :linetext: Glossary
glossary: *2163880260
bundledef: &2163876280
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/bundledef.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/bundledef.txt
  :linetext: ! 'Glossary: Bundle'
! 'glossary: bundle': *2163876280
bytecode: &2163872140
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/bytecode.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/bytecode.txt
  :linetext: ! 'Glossary: Bytecode'
! 'glossary: bytecode': *2163872140
compiledscriptfile: &2163867820
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/compiledscriptfile.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/compiledscriptfile.txt
  :linetext: ! 'Glossary: Compiled Script File'
! 'glossary: compiled script file': *2163867820
dictionarydef: &2163862880
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/dictionarydef.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/dictionarydef.txt
  :linetext: ! 'Glossary: Dictionary'
! 'glossary: dictionary': *2163862880
fork: &2163858380
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/fork.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/fork.txt
  :linetext: ! 'Glossary: Fork'
! 'glossary: fork': *2163858380
objectmodel: &2163853980
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/objectmodel.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/objectmodel.txt
  :linetext: ! 'Glossary: Object Model'
! 'glossary: object model': *2163853980
scriptingaddition: &2163848520
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/scriptingaddition.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/scriptingaddition.txt
  :linetext: ! 'Glossary: Scripting Addition'
! 'glossary: scripting addition': *2163848520
sdefglossary: &2163843280
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/sdefglossary.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/sdefglossary.txt
  :linetext: ! 'Glossary: Sdef'
! 'glossary: sdef': *2163843280
tellcontext: &2163838780
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/helpglossaryfolder/tellcontext.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/helpglossaryfolder/tellcontext.txt
  :linetext: ! 'Glossary: Tell Context'
! 'glossary: tell context': *2163838780
menus:
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menus.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menus.txt
  :linetext: Menus
applicationmenu: &2163829940
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/applicationmenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/applicationmenu.txt
  :linetext: Application Menu
application menu: *2163829940
clippingsmenu: &2163826100
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/clippingsmenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/clippingsmenu.txt
  :linetext: Clippings Menu
clippings menu: *2163826100
dictionarymenu: &2163822480
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/dictionarymenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/dictionarymenu.txt
  :linetext: Dictionary Menu
dictionary menu: *2163822480
editmenu: &2163818800
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/editmenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/editmenu.txt
  :linetext: Edit Menu
edit menu: *2163818800
filemenu: &2163813960
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/filemenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/filemenu.txt
  :linetext: File Menu
file menu: *2163813960
scriptme: &2163809300
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/scriptme.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/scriptme.txt
  :linetext: Script Menu
script menu: *2163809300
scriptsmenu: &2163805360
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/scriptsmenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/scriptsmenu.txt
  :linetext: Scripts Menu
scripts menu: *2163805360
searchme: &2163801360
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/searchme.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/searchme.txt
  :linetext: Search Menu
search menu: *2163801360
viewmenu: &2163797960
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/viewmenu.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/viewmenu.txt
  :linetext: View Menu
view menu: *2163797960
windowme: &2163794060
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/menusfolder/windowme.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/menusfolder/windowme.txt
  :linetext: Window Menu
window menu: *2163794060
preferenc: &2163790520
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferenc.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferenc.txt
  :linetext: Preferences
preferences: *2163790520
applescript: &2163786700
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/applescript.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/applescript.txt
  :linetext: ! 'Preferences: Fonts & Colors'
! 'preferences: fonts & colors': *2163786700
debugger: &2163782860
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/debugger.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/debugger.txt
  :linetext: ! 'Preferences: Execution'
! 'preferences: execution': *2163782860
dictionaryprefs: &2163779020
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/dictionaryprefs.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/dictionaryprefs.txt
  :linetext: ! 'Preferences: Dictionary'
! 'preferences: dictionary': *2163779020
editor: &2163775020
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/editor.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/editor.txt
  :linetext: ! 'Preferences: Editor'
! 'preferences: editor': *2163775020
general: &2163771280
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/general.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/general.txt
  :linetext: ! 'Preferences: General'
! 'preferences: general': *2163771280
keybindings: &2163767520
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/keybindings.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/keybindings.txt
  :linetext: ! 'Preferences: Key Bindings'
! 'preferences: key bindings': *2163767520
softwareupdate: &2163763760
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/softwareupdate.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/softwareupdate.txt
  :linetext: ! 'Preferences: Software Update'
! 'preferences: software update': *2163763760
textsubstitutions: &2163759980
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/preferencfolder/textsubstitutions.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/preferencfolder/textsubstitutions.txt
  :linetext: ! 'Preferences: Text Substitution'
! 'preferences: text substitution': *2163759980
windows:
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/windows.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/windows.txt
  :linetext: Windows
toolbar:
  :path: !ruby/object:Pathname
    path: indexfolder/referencfolder/windowsfolder/toolbar.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/indexfolder/referencfolder/windowsfolder/toolbar.txt
  :linetext: Toolbar
toc: &2162794100
  :path: !ruby/object:Pathname
    path: toc.html
  :adr: !ruby/object:Pathname
    path: /Users/mattleopard/anger/Word Process/jobs/sd5/sd5docs/toc.txt
  :linetext: Table of Contents
table of contents: *2162794100

=end
Aaron Patterson
Owner

Can you try with the latest gem, or with edge 1.9.3?

Matt Neuburg

I'd be delighted to. Let me give it a shot.

Matt Neuburg

Installed the gem, started the script this way:

gem 'psych', '1.3.4'
require 'yaml'
p Psych::VERSION #=> "1.3.4"

So if I understand correctly we are using the gem. No faster. More to follow...

Matt Neuburg

Tried with ruby 2.0.0dev (2012-09-15 trunk 36975) [x86_64-darwin10.8.0]. Psych is a little faster: about 13.5 seconds, as opposed to the previous 17 or 18 seconds. But we are still not operating at anything like the speed of Syck (5 seconds). I can't test with Syck under ruby 2 as it isn't there (we neatly fall back to Psych).

Kevin Rood

@tenderlove We are also dealing with really slow performance of psych vs sych with an app upgraded to 1.9.3. Any status update on this? Any plans to implement this as a c-based lib again?

Aaron Patterson
Owner

Well, it uses libyaml, so it is already a c based library. The question is where the perf bottleneck is. I suspect it's libyaml's parser, but I'm not certain.

Kevin Rood

Please forgive my ignorance, and thank you for the details. I came across references that refer to psych as "pure ruby".

Aaron Patterson
Owner

@kevinrood don't worry about it! I need to research the libyaml parser more closely.

Matt Neuburg

Here's ruby-prof output from my example. First under Psych (first 25 lines only):

 %self     total     self     wait    child    calls   name
  9.29     10.30    10.30     0.00     0.00  2120100   Regexp#=== 
  8.64     28.41     9.58     0.00    18.83   450600   Psych::ScalarScanner#tokenize 
  4.68      5.19     5.19     0.00     0.00   450600   Psych::Nodes::Scalar#initialize 
  4.04     16.67     4.48     0.00    12.19   764700  *Class#new 
  3.99     33.28     4.42     0.00    28.86   450600   Psych::Visitors::ToRuby#deserialize 
  3.81     27.71     4.22     0.00    23.49      300   Psych::Parser#parse 
  3.62      5.13     4.02     0.00     1.12    46200   Kernel#Integer 
  3.51     13.72     3.89     0.00     9.83   450600   Psych::TreeBuilder#scalar 
  2.90      3.21     3.21     0.00     0.00   856800   <Class::BasicObject>#allocate 
  2.86     38.94     3.17     0.00    35.78   450600   Psych::Visitors::ToRuby#visit_Psych_Nodes_Scalar 
  2.26      2.50     2.50     0.00     0.00   450600   Psych::Visitors::ToRuby#register 
  1.85      2.05     2.05     0.00     0.00   474304   Hash#[]= 
  1.73      1.92     1.92     0.00     0.00   692400   String#=== 
  1.54      1.71     1.71     0.00     0.00   624604   Kernel#hash 
  1.54      1.71     1.71     0.00     0.00   624600   Kernel#class 
  1.54      1.71     1.71     0.00     0.00   624600   Hash#empty? 
  1.44      2.21     1.59     0.00     0.62   138900   Psych::Nodes::Mapping#initialize 
  1.41      7.09     1.57     0.00     5.52    92400   Psych::Visitors::ToRuby#init_with 
  1.34      1.48     1.48     0.00     0.00   543000   String#empty? 
  1.32      6.20     1.47     0.00     4.73   138900   Psych::TreeBuilder#start_mapping 
  1.31      1.45     1.45     0.00     0.00    92400   Psych::Coder#initialize 
  1.26      1.40     1.40     0.00     0.00   450600   Hash#key? 
  1.20     33.74     1.33     0.00    32.42    92400   Psych::Visitors::ToRuby#revive 
  1.18      2.06     1.31     0.00     0.75   139200   Psych::TreeBuilder#pop 
  1.02      1.73     1.13     0.00     0.59    92400   Psych::Visitors::ToRuby#resolve_class 

Now under Syck (first 11 lines only):

 %self     total     self     wait    child    calls   name
 23.50      8.75     3.61     0.00     5.14   231000   Syck::Resolver#transfer 
 22.24     15.31     3.42     0.00    11.90      300   Syck::Parser#load 
 19.88     11.81     3.05     0.00     8.75   589500   <Object::Syck::Resolver>#node_import 
 10.98      2.52     1.69     0.00     0.83   138600   <Class::Symbol>#yaml_new 
  7.42      1.14     1.14     0.00     0.00   415800   Kernel#respond_to_missing? 
  2.88      0.44     0.44     0.00     0.00   138600   String#intern 
  2.74      0.42     0.42     0.00     0.00    92400   <Class::Object>#yaml_tag_subclasses? 
  2.66      0.41     0.41     0.00     0.00    92400   Hash#each 
  2.54      0.39     0.39     0.00     0.00   138600   Module#=== 
  2.42      0.37     0.37     0.00     0.00    92400   Module#yaml_tag_read_class 
  1.81      0.28     0.28     0.00     0.00    92400   <Class::BasicObject>#allocate 
Kevin Menard

I'm seeing very similar performance issues and I think I narrowed it down. As noted in the profile trace above, there's a very high number of RexExp matches being performed as part of the tokenization process. Basically, if you have a String that doesn't start with [A-Za-z~], you're going to perform 10 failed RegExp matches and then potentially a failed cast to an Integer (depends on whether the String contains less than two '.' characters). Incidentally, every single one of those path values above has a single '.' and as such, an Integer is attempted to be created from it and fails.

The easiest solution is to short-circuit all that. Since the parse order matters in order to guarantee a proper match, the short-circuit code is a bit ugly. As far as I can tell, it boils down to one of two approaches: look for a character you know would fail any of the matches or look for just a different starting character than the other matches look for. The former would yield the fewest number of comparisons in the case of the YAML above, but is indeed ugly.

By changing the first when clause from when /^[A-Za-z~]/ to when /^[A-Za-z~]/, /^[^:-][\D\s\(\)!\?\{\}\/\\]+/, I managed to reduce loading of the above file roughly 3x. Running benchmark, loading the file 300 times, I saw:

                 user     system      total        real
load YAML   20.160000   0.020000  20.180000 ( 20.236022)

                 user     system      total        real
load YAML    6.500000   0.040000   6.540000 (  6.560076)

Basically the second regular expression looks for anything that doesn't start the characters ":" or "-" and then consists of a string of characters that have a space, parenthesis, curly braces, non-digits, and some punctuation. Basically characters that would make it look distinctly different from a date, a float, an int, special floats like Infinity, and so on.

I guess the downside is for all other non-String types, you'd have one extra RegExp failed match. So, a YAML loaded with dates would be marginally slower. But I think the gain here is quite substantial for the common cases of loading up gemspecs and VCR cassettes.

Kevin Menard

It probably goes without saying, but implementing the above change locally removed Psych completely as a hotspot from some of my more involved VCR tests.

Kevin Menard

And finally, as grotesque as my proposed solution is, it's not unprecedented. The early String handling code does a sub-case statement where it's first check is that the string isn't a Boolean or nil value (and it took me way too long to figure out that that's what it's doing :-P)

Other suggestions are appreciated.

Aaron Patterson
Owner

@nirvdrum That first where clause was put there to short circuit the rest of the tests. I know your regexp is ugly, but I'm fine with adding ugly code in that spot to prevent so many tests.

Kevin Menard

Great. How solid are the tests for all the other cases? I obviously wouldn't want to inadvertently break parsing in general.

Kevin Menard

I can pull together a pull request. As of now, I do have two test failures. I was sorta just mucking around to see what I could do with the example YAML above. If fixing those two is good enough though, I could have that wrapped up tonight. If more testing is needed, I guess we'll need to develop the test cases.

Aaron Patterson
Owner

@nirvdrum I am very confident in the tests. I'll test against Rails or something, but all the nasty branches in the scanner were tdd'd, so I'm not worried that you'll break anything.

Matt Neuburg

I'm liking where you're going with this! Thx.

Kevin Menard

@mattneub If you wouldn't mind checking out psych master and verifying it speeds up for you, that'd be great. Unfortunately, the performance here is more probabilistic based upon common usage patterns than something that could be improved across the board.

Matt Neuburg

@nirvdrum It's better in my actual use case (11 seconds instead of 14 seconds) but not as good as Ruby 1.8.7/sych (9 seconds). I'm currently working around this by using yaml less often...

Kevin Menard

@mattneub Using the above YAML file? Or do you have a more dastardly one?

Matt Neuburg

The timings I just gave are for an actual real-world process in my RubyFrontier web framework. It was the slowdown in this particular operation when switching to Ruby 1.9.3 that alerted me to the issue in the first place. You asked for "common usage patterns" so I tested with my actual usage. YAML is not the only thing that happens in that process, of course; I'm just trying to give a sense of what difference the RegExp change makes. It definitely makes a difference!

Kevin Menard

Okay. No worries. I'm just trying to see if I could tune any more. I drew from my own app and the YAML file you have above but that's not a huge sample. If you could do another profile trace, that'd be quite helpful.

Matt Neuburg

Well, you already know that running my original test script with caveman timings, Psych with your modifications is still much slower than Syck. I rewrote your VERSION string as "1.3.4b" so as to be certain that we are using your changes:

gem 'psych'
require 'yaml'

h1 = h2 = nil
f = Pathname.new(__FILE__).dirname + "autoglossary.yaml"

YAML::ENGINE.yamler = 'psych'

t = Time.new.to_f
300.times {h1 = YAML.load_file(f)}
puts "Psych #{YAML::VERSION}", Time.new.to_f - t

YAML::ENGINE.yamler = 'syck'

t = Time.new.to_f
300.times {h2 = YAML.load_file(f)}
puts "Syck #{YAML::VERSION}", Time.new.to_f - t

puts "Resulting loaded hash the same? #{h1 == h2}"

Output:

Psych 1.3.4b
12.906046390533447
Syck 0.60
5.742939710617065
Resulting loaded hash the same? true

So we are still a long way from the speed of Syck. A new profile starts like this:

Thread ID: 2151954760
Total: 102.067805
Sort by: self_time

 %self     total     self     wait    child    calls   name
  8.43     20.01     8.60     0.00    11.41   450600   Psych::ScalarScanner#tokenize 
  8.06      8.23     8.23     0.00     0.00  1704300   Regexp#=== 
  5.26      5.37     5.37     0.00     0.00   450600   Psych::Nodes::Scalar#initialize 
  4.38     24.93     4.47     0.00    20.47   450600   Psych::Visitors::ToRuby#deserialize 
  4.18     15.97     4.27     0.00    11.71   718500  *Class#new 
  3.88     13.62     3.96     0.00     9.66   450600   Psych::TreeBuilder#scalar 
  3.61     27.27     3.69     0.00    23.59      300   Psych::Parser#parse 
  3.11     30.62     3.18     0.00    27.45   450600   Psych::Visitors::ToRuby#visit_Psych_Nodes_Scalar 
  2.63      2.68     2.68     0.00     0.00   810600   <Class::BasicObject>#allocate 
  2.46      2.52     2.52     0.00     0.00   450600   Psych::Visitors::ToRuby#register 
  1.89      1.93     1.93     0.00     0.00   692400   String#=== 
  1.76      1.80     1.80     0.00     0.00   474304   Hash#[]= 
  1.70      1.74     1.74     0.00     0.00   624600   Kernel#class 
  1.70      1.73     1.73     0.00     0.00   624600   Hash#empty? 
  1.70      1.73     1.73     0.00     0.00   624604   Kernel#hash 
  1.60      2.29     1.63     0.00     0.66   138900   Psych::Nodes::Mapping#initialize 
  1.56      7.13     1.59     0.00     5.54    92400   Psych::Visitors::ToRuby#init_with 
  1.47      1.50     1.50     0.00     0.00   543000   String#empty? 

So we've moved Regexp#=== down the list a little, and this has the effect of shaving some time off of other calls too, but we are still spending a lot of time in the scanner/parser. It might be more revealing to use ruby-prof's graphing printer which displays call nesting.

Kevin Menard

Thanks. I was using perftools.rb, but seeing timings nowhere near yours. Raw time is relative to the hardware being run on. So the profile timings help more with cross-referencing. In my case, I was seeing psych handle that file in about 6 seconds. I'll dig a little deeper.

Matt Neuburg

I can't explain the discrepancy. It's possible I've done something wrong with the installation, but I did download from master and rake gem and gem install, and the gem is in place and the version number is reported correctly, so I do believe I'm using your code (and I can see the changed "when" test). Maybe contact me offline and we can coordinate our tests?

Kevin Menard

Sounds good. I tried to find your contact info but was unable to do so. Please feel free to e-mail me at nirvdrum@gmail.com.

Bert Goethals

Hi,

I wanted to bring some (anecdotal) evidence for the true slowness we encountered. I had read this post before switching from Syck to Psych; but I assumed the impact would be minimal. Oh boy where we wrong.

Graph

This is a Rails 3.2.8 app. We serialize quite a lot of stuff in the DB; so at least each request there are several YAML conversions executed.

  • ruby 1.9.3p194 (Debian)
  • Psych::VERSION => "1.3.2" stdlib
  • Psych.libyaml_version => 0.1.4

I don't know if this is helpful, or if you'd like me to collect some more info?

Joe Rafaniello

@Bertg Yes, psych is slower than syck but you're running into a fixed performance bug.

ruby 1.9.3p194 had a very slow psych library (version 1.3.2) due to a bug. It was fixed here: 0845b07 and released in psych 1.3.3.

An example benchmark: https://gist.github.com/2936844

Install Ruby 1.9.3-p327 or p286 which included this fix or upgrade to the psych 1.3.3+ gem and retry your performance comparison.

Matt Neuburg

@jrafanie That doesn't make any difference. My test, quoted at the start of this Issue, running under Ruby 1.9.3-p327 and psych 1.3.4, yields:

Psych 1.3.4
18.122817993164062
Syck 0.60
5.312050104141235
Resulting loaded hash the same? true

So Psych is still taking over 3 times as long to perform the same task. Nothing has changed here.

Kevin Menard

That's not strictly true. Psych has different performance characteristics depending on the structure of the underlying file. We've been trying to normalize that. But without seeing @Bertg's file it's not possible to say one way or another. For some file structures it is faster, for others there's no difference.

Matt Neuburg

@nirvdrum Kevin: that's true, good correction. But that's just as true of @jrafanie with the claim that what @Bertg is seeing is a "fixed performance bug". Exactly you say, it depends on the structure of the underlying file. As for me, I've provided my underlying file, so my benchmark is determinate.

Bert Goethals

Next week ill be testing with the latest Gem version. If the problem persist; ill send some example YAML structures that are being parsed, if desired. (Note that here I'm not reading files, but parsing serialised DB records).

If it does not persists, it's indeed the problem @jrafanie was mentioning.

Joe Rafaniello

@mattneub, @Bertg reported a performance issue that can be lessened significantly by upgrading from 1.9.3p194 to a newer psych.

@mattneub's issue still exists.

There is a known issue in version 1.3.2 of psych included in 1.9.3p194 that causes tables with serialized columns to perform very badly with 1.9.3-p194. The more serialized columns, the worse the results.

Anyone seeing performance issues with multiple yaml serialized columns on some Rails models on p194 would see some improvement going to p327 or the latest gem.

From my simple benchmark, https://gist.github.com/2936844, p194 has Psych.load 8x slower than syck, whereas p125 and p327 has Psych.load 4x slower.

So, yes, Psych is still orders of magnitude slower than Syck.

Gernot Kogler

We rely heavily on YAML serialization in a large rails application. We gave psych a try, but the performance penalty compared to Syck is unacceptable, unfortunately (even when using 1.9.3-p327 or the latest gem). Is there any hope to get performance comparable to Syck?

Kevin Rood

@gernotkogler if you still have performance issues, you might consider using json serialization. We migrated important columns over to json serialization and are in the process of progressively migrating other yaml serialized columns over to json as well. It's not an ideal solution, but something you might consider.

Kevin Menard

Please try the latest master. It should be substantially faster. Profiling your app would help immensely as well.

But, psych is almost certainly never going to be as fast as syck. There's more overhead by virtue of actually adhering to the YAML specs. Hopefully we can get it in the "good enough" category for most though.

Gernot Kogler

@kevinrood
Thanks for the suggestion. JSON is not an option since it is typeless. When I serialize an object I expect the serializer to restore that object, not just a hash of strings.

@nirvdrum
I really respect your work but in our case "good enough" means at least as fast as Syck. We serialize complex object graphs to database columns using ActiveRecords "serialize" option. I guess we'll have to rethink our serialization strategy.

Kevin Menard

@gernotkogler It's not an absolute. But it's impossible to see where things can be sped up if you're unable to share your document and/or a profiling session.

Kevin Rood

@gernotkogler it's true it takes some work to use JSON for column serialization. ActiveRecord supports creating your own column serializer, which is what we chose to do for some columns that gave us horrible yaml serialization performance.

Jason Frey

I've implemented the ScalarScanner as a Ragel state machine in my branch here https://github.com/Fryguy/psych/tree/ragel . If people could test it out for performance with your own workloads, that would be great.

There are three commits on there, each passing the tests, and each implementing it incrementally differently.

  • Fryguy/psych@93b6f3c - Ruby based state machine inline in scalar_scanner.rb without string nor symbol caching
  • Fryguy/psych@9e43eb4 - Same as above, but with string and symbol caching.
  • Fryguy/psych@2ec9620 - C based state machine moved from the Ruby code (with the caching). This is the tip of the branch.

There are still more improvements that could be made in there. For example, some of the Time parsing logic that is in Ruby, could be moved into the state machine and built up as the state machine walks the characters, so that we don't have to run extra Regexes afterwards. Also, I did not apply any of the Ragel optimizations, opting for the default. When I changed the optimizations, I didn't notice any difference in my benchmarking, but that could just be specific to those files.

I've benchmarked the yml from @mattneub (albeit slightly differently...I only read it once into a string, and called Psych.load 300 times). My times are:

Before             4.859s
Ruby based Ragel   8.603s
  with caching     6.548s
C-based Ragel      4.224s

I also noticed the entire rake suite seemed a little faster:

Before             0.298681s, 1687.4190 tests/s, 4650.4465 assertions/s
Ruby based Ragel   0.353835s, 1424.3927 tests/s, 3925.5585 assertions/s.
  with caching     0.382208s, 1318.6537 tests/s, 3634.1469 assertions/s.
C-based Ragel      0.279431s, 1803.6653 tests/s, 4970.8157 assertions/s.

Also, if you're interested, the generated graph of the state machine is here (I recommend zooming out)

Aaron Patterson
Owner

@Fryguy great work.

First, can you run your benchmarks with benchmark ips so that we can see the standard deviation for the numbers. This looks like possibly a 13% gain, but we can't really know without seeing the stddev.

Second, did you calculate the percentage of time spent in libyaml vs the scalar scanner?

Jason Frey

autoglossary is @mattneub's yml. vcr is my own personal yml.

Before

Calculating -------------------------------------
        autoglossary         6 i/100ms
                 vcr         4 i/100ms
-------------------------------------------------
        autoglossary       62.7 (±3.2%) i/s -        318 in   5.075001s
                 vcr       47.3 (±2.1%) i/s -        240 in   5.077008s

After

Calculating -------------------------------------
        autoglossary         7 i/100ms
                 vcr         5 i/100ms
-------------------------------------------------
        autoglossary       71.8 (±1.4%) i/s -        364 in   5.068371s
                 vcr       50.9 (±2.0%) i/s -        255 in   5.007362s

And, no, I did not calculate where the time was spent. I noticed that the scalar scanner looked like a good candidate to be a ragel state machine, and just tried it, especially since tokenize was a bottleneck in the earlier ruby-prof runs. How would I go about calculating the time spent...another ruby-prof run?

Jason Frey

Anything new on this one? Has anyone else tried the ragel branch? Should I make a pull request?

Jay Feldblum

It looks to me like plenty of time in Psych.load is taken up by the ToRuby visitor. There may be places to optimize that.

For example, rather than allocating so many strings so often when checking a node's tag, why not stick those strings into some constants?

Kevin Menard

I have pull request I need to pull together that will drastically reduce the number of objects created. This is both faster in psych and on the ruby garbage collector. I was seeing around 20% improvement using @mattneub's test case. In contrast, I was only seeing about an 11% improvement with the ragel work.

If you can get that faster, great. But (and my vote really doesn't count), I'd be -1 on merging that in as is just because it introduces a lot of complexity for little gain. E.g., that class would have to be completely rewritten for JRuby.

Jay Feldblum

Object allocations are not free in JRuby either. Also, recent work in JRuby has made constant-lookup much faster (caveat: with invokedynamic).

Kevin Menard

Right. @tenderlove and I were working on something on IRC about a week ago that cut object allocations down 64% across all rubies. There's certainly more work that can be done, but I think that's a much better base to start from. I'll try to get that together this weekend.

Jason Frey

@nirvdrum Sounds great. Can't wait to see your pull request.

Jack Royal-Gordon

Maybe kind of a dumb question, but can I use the newer version of Psych with Ruby 1.9.2-p290? If I do, will I see the performance improvements you all are working on? In my app, when I load ~2800 records with a serialized field, the difference between using Psych and Syck is about 6:1 (60 seconds vs 10 seconds), but my tests are not yet in good enough shape for me to trust a Ruby version change. Using Rails 3, am I correct that just referencing a newer version of Psych in the gem file will do it?

Aaron Stone

Ping @nirvdrum have you made progress on the reduced object allocation work? This thread has been an awesome read, looking forward to the next round of improvements to Psych!

Marten Veldthuis

We're having performance issues that block a bunch of upgrades too. And if anything, Psych 2.0 on Ruby 1.9.3 is immensely worse than 1.3.4. And in Ruby 2.0 Syck is gone, so at some point we'll be pretty much forced to upgrade.

This bites us for ActiveRecord column serialization.

                    user         system   total      real
1.9.3 Syck  v0.60   113.040000   2.560000 115.600000 (118.310391)  <--- what we run now
1.9.3 Psych v1.3.4  291.480000   6.390000 297.870000 (307.099166)
1.9.3 Psych v2.0.0  559.920000   5.980000 565.900000 (570.774377)
2.0.0 Psych v2.0.0  223.930000   6.000000 229.930000 (233.354199)

Which come from this benchmark, which admittedly hits everything from ActiveRecord through to MySQL, but I wanted to start off with a real world test.

require 'benchmark'

Benchmark.bm do |x|
  x.report "#{RUBY_VERSION} #{YAML.inspect} v#{YAML::VERSION}" do
    1000.times do
      Questionnaire.find_each do |questionnaire|
        questionnaire.properties # where properties is a serialized attribute
      end
    end
  end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.