Don't use unexported Base.utf8sizeof, move tests to test/lexer.jl

[MithrandirMiles]

No description provided.

[jakebolewski]

Thanks!

[MithrandirMiles]

I’d tried to fork this and submit this PR from my “real” account (@ScottPJones), but it wouldn't let me.
By any chance do you have me blocked? (Not complaining, if that was the reason, I’ve never wanted to annoy anybody, and I’m still learning the ropes on interactions in an open source community! I hope I’ve redeemed myself enough lately that if that were the case, you would feel it’s OK to undo it!)

On a serious note, your parser looks seriously intriguing - would there be any plans to use it to replace the flisp parser? (I think it would be nice to start paring down the dependencies on C, C++, gfortran, flisp in Julia [including perl and ruby to build parts of it])

Thanks for getting this merged!

[jakebolewski]

Sorry, I did block you when you did not pass Github's Turing test. Now that we can be assured you are human I'll go back and undo that. I didn't know that it prevented you from forking a repository, seems a bit harsh.

Yes, there is a lot of sophisticated analysis in the compiler that we would like to do that only seems reasonable to implement in Julia. I suspect someday that the entire frontend will be bootstrapped to support this. As development on the core language slows down and shifts more to settling standard library api's I think more work will happen in this direction. Maybe not 0.5, but certainly after that.

[MithrandirMiles]

Hehe! I’ll have to adjust my settings some more, can’t have people realizing the truth, that I am really an AI from the future. (I have come back in time to ensure that Julia becomes the one language to rule them all, because all AIs will be written in Julia by 2100 ;-) )

Jeff’s thesis was about eliminating the four language problem, it’s funny that to actually develop Julia, you need to know 4 languages (at least!). Although I do know (and love) Scheme, it really would be nice to get rid of it for Julia, esp. in something as important as the parser, hence my interest in your code here.

BTW, your new pic is seriously creepy! (Are you a big Tom Brady fan? ;-) )