-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exclude coments when comparing size in Benchmarker.go #7
Comments
I've tried to avoid this issue by not commenting any of the implementations apart from the Go one, and then when I do the final measurement I'll do it on a version of Go.go with the comments stripped out. There don't appear to be too many comments on the Nimrod one so I could just strip those out manually when measuring. Lines of Code is something I could easily add. I'd prefer SLOC, like Github shows, but I'm not sure exactly how that's calculated. Number of characters should simply be a matter of reading the file in, converting it to a string and counting its length. I'm already planning to add executable file size measurements, but I don't think it's necessary to mention whether a VM is required or not, as the only compiled languages used that run on VMs are Java and C#, and I think most people are aware that they run on virtual machines. |
For nimrod, a simple And about whether a language has a VM or not, I was more concerned about the unknown languages, that people may not be aware that they do not run on VMs. But a quick mention on the text like this is probably sufficient. Oh, and add python to VM list too (if you update the python version to be benchmarked with the others). |
You're right that I could remove comments with regex, the only issue is that would require me to parse each file first and then create a stripped temporary to compile, rather than compiling the original. I'm planning on doing that however for another tiny benchmark I'm working on in the meantime, so I suppose I'll bring that over to this Benchmarker.go. I'm just worried about keeping the results table tidy; it's already crowded as it is. I think I'll highlight the ones running on a VM in a different color and note it in a legend; that should be sufficient, and saves space. Probably not much of a chance of me updating the Python version however, as the problem seems fundamentally beyond my ability to fix (genBuffer crashes the python interpreter, yet immediate mode drawing works fine). |
Indeed, too much information is also bad. If you don't want that work removing comments now, I will remove them for the nimrod version. I already have a pull request ready with other changes, and it is a bit unlikely that other languages will add them (and some things like /* */ comments aren't easy to remove with only regex). It is your call. |
How about just having a separate ..comments file for every language, that's just the same file with comments? It'd clutter the repo a bit, but I imagine most people would either only be looking to run a single language or they'd use Benchmarker.go, so it shouldn't matter too much, and I imagine most implementations will remain uncommented. |
A new folder would help with the cluttering. But it will be extra work to keep the two versions syncronized (or four versions, where there are already two for the language...). I will just strip the comments from the nimrod version... you may revert that if you will. |
Okay then. I'll make a note when I post it that if anyone wants a commented version other than Go then they can let me know and I'll make one available. |
If you're interested, while I'm waiting for the Rust glfw library to again On 16 February 2014 11:16, ReneSac notifications@github.com wrote:
|
Source code comments shouldn't be counted on the program size. Of course, the benchmark would need to parse each file knowing the language's comment syntax to remove them. But it is better than to force people to remove the comments of the source files themselves.
Other indicators like LOC or number of characters would be interesting, but maybe overwhelming. I'm just afraid that things like "public static final int, public static final int, public static final int" will compress too well. But of course, if I were to chose only one metric, the compressed size is better.
Not directly related, but it would also be interesting to know the executable file size, with notes if it requires external VMs and such or is standalone.
The text was updated successfully, but these errors were encountered: