New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#line annotation costly to generate because of too much I/O #693

nddrylliog opened this Issue Sep 7, 2013 · 2 comments


None yet
1 participant

nddrylliog commented Sep 7, 2013


amos at loki in ~/Dev/rock (97x)
$ time rock rock.use --onlygen
[ OK ]
rock rock.use --onlygen  11.14s user 0.24s system 99% cpu 11.396 total


amos at loki in ~/Dev/rock (97x)
$ time rock rock.use --onlygen --nolines
[ OK ]
rock rock.use --onlygen --nolines  5.44s user 0.11s system 99% cpu 5.558 total

A whopping half the time taken by line numbers! It's because we read the file every time. We gotta have at least some sort of caching system for .ooc files being read.. or better yet, Tokens storing line numbers directly when being read.

@ghost ghost assigned nddrylliog Sep 7, 2013


This comment has been minimized.


nddrylliog commented Oct 8, 2013

This has been observed on Windows and it seems to have a huge impact. We have indeed a few million I/O reads (O_O) because we seek so much in files and the like.

As a temporary stopgap measure, instead of implementing a byte offset <=> line number map, we could just read whole files in memory and work from there. It'd still be stupid but a whole lot more efficient.

nddrylliog added a commit that referenced this issue Nov 27, 2013

Take advantage of nagaqueen's line numbers instead of recomputing the…
…m all the time. Large codegen speed-up, esp. on Windows. Closes #693

This comment has been minimized.


nddrylliog commented Nov 27, 2013

The solution was to make nagaqueen expose its own line info, which it already has. No need to recompute it all the time ourselves!

@nddrylliog nddrylliog closed this Nov 27, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment