Looking for a new maintainer

[dmajda]

When I started PEG.js about 7 years ago, I never thought it would be such a successful project and that my involvement with it would last so long. Most of the time I was happy working on it, but in the last year or so my motivation started to dwindle. I found I had to convince myself to actually sit down and work through the issues or write code. I gradually realized that PEG.js no longer something I want to spend my time on. As a result, I’m looking for a new PEG.js maintainer.

Who am I looking for?

I’m looking for someone who has deep understanding how parsing and parser generators work, strong motivation, experience with open source development, and clear idea where to lead PEG.js development (which may or may not match the current roadmap). None of these is a strict requirement, but it’s unlikely I’ll hand over PEG.js to someone significantly lacking in these areas.

Interested?

If you are interested in taking over PEG.js, add a comment to this issue and include anything that you think will convince me you are the right person :-) I’ll review the comments and announce the new maintainer during the weekend of May 13–14. If no suitable maintainer appears by then, PEG.js will become officially unmaintained.

Transferred assets

Transferring the maintainership involves transferring the following assets:

• GitHub organization
• Twitter account
• Google Group
• The pegjs.org domain

The new maintainer will need to take care of hosting and running the PEG.js website, which is currently running on my personal VPS. I expect the new maintainer to setup hosting under his/her control and reconfigure DNS records for the pegjs.org domain to point there shortly after transferring maintainership. The new maintainer won’t be given access to my personal VPS nor will I continue hosting PEG.js website there indefinitely.

---

If you have any questions or notes to anything I wrote above, feel free to add comments to this issue.

[mikeaustin]

I just want to say I just started using PEG.js the last few months, and I love it. The grammar is easy to learn, and the number of concepts is fairly low. The documentation is a little light, but I found everything I need. Also, I wouldn't have gotten started on my JavaScript dialect without the javascript.pegjs example. That reduced the workload considerably.

In it's current state, what are it's benefits? For me it's very easy to prototype with. Drawbacks are that is can be slow for large amounts of data, but I don't care about that right now. What are it's competitors? Do you want to see it continue to grow, or would you like someone to create version 2.0? If it's currently stable, why not just leave it at that, with maybe a bit more documentation.

For the roadmap, what does modernize code give you? Faster? easier to maintain? Or just "up to date"? I can understand using Gulp instead of Make to make it easier to maintain. The cleanups sound expensive, almost a 1.5, calling the current 1.0. I feel "improving performance significantly" is too large a goal for 1.0. I feel the current is 1.0. It's very stable, good error messages, all feature work, etc. 1.0 of software doesn't have to be "the one". That's more 2.0, 3.0, etc.

I don't know if I could help with maintenance, but I'd like like to see the site, Google group, etc. live on.

[dmajda]

@mikeaustin Thanks for you comment. Let me quickly answer your questions (which will probably be useful also for others looking at this issue).

In it's current state, what are it's benefits? For me it's very easy to prototype with.

Prototyping has been the dominant use case, along with parsing small languages/DSLs which are too complex for regexps and various interactive tools. Originally, I created PEG.js because PEGs offered nice grammar syntax and there was no JavaScript parser generator that used it and was also able to produce good error messages.

Drawbacks are that is can be slow for large amounts of data, but I don't care about that right now.

Many people do, that’s why the performance work is in the roadmap.

What are it's competitors?

Jison, Nearley

Do you want to see it continue to grow, or would you like someone to create version 2.0? If it's currently stable, why not just leave it at that, with maybe a bit more documentation.

That’s up to the new maintainer. But I wouldn’t like PEG.js to stagnate with the current feature set forever. It has more potential than that.

There are many useful features which can be added relatively easily without any changes in the core (#30, #38, #107, #45, and #285). Then there are some long-standing problems/use cases which would need some rethinking of the core concepts (#11, #217) or changing the API (#327, #430). I think all of these are worth looking at.

For the roadmap, what does modernize code give you? Faster? easier to maintain? Or just "up to date"?

Easier to maintain and being up to date (which is admittedly partially a PR decision — it’s bad when your project looks old technologically, even if there is only marginal productivity difference between the old and new tech).

I feel the current is 1.0. It's very stable, good error messages, all feature work, etc. 1.0 of software doesn't have to be "the one". That's more 2.0, 3.0, etc.

Could be :-) My definition of 1.0 was always “when I am satisfied with it”, which somehow never came — partly because instead of working on thing that bother me most I ended up working on things that bothered users most and things that were necessary to use PEG.js in the real world. My perfectionist nature also didn’t help.

It’s up to the new maintainer to define 1.0.

[futagoza]

https://github.com/futagoza/epeg.js 😊 (It should be noted that although most of the features listed aren't currently implemented, but at one point or another, have been in some fork or unrelated project, most of which were never put on GitHub)

I'm fine with everything, but the only problem I have is maintaining tests 😞

[lukas-vlcek]

@dmajda thanks a lot for PEG.js. It is very useful. I hope this project will live on.

[dmajda]

@futagoza Just to clarify, your comment means you would be interested in maintaining PEG.js? Or that you think the future lies elsewhere? :-)

I'm fine with everything, but the only problem I have is maintaining tests

What problem exactly?

[futagoza]

@dmajda Yes, I would be interested in maintaining it 😄

I'm fine with everything, but the only problem I have is maintaining tests

What problem exactly?

What I mean is that I usually code and debug as I go along, but when I try making tests as you have here, I start having problems maintaining them, so usually I either make tests that let me see the end result, or just leave the spec and benchmark testing to others, although in hindsight this increases development time for some projects 😆

[romerojp]

@dmajda I am currently using PEG.js as part of my MSc project. It is not only useful but interesting and cool to work with. I really hope you deem someone to be the worthy PEG.js heir.

[dmajda]

@futagoza In the issue description, I wrote:

I’m looking for someone who has deep understanding how parsing and parser generators work, strong motivation, experience with open source development, and clear idea where to lead PEG.js development (which may or may not match the current roadmap).

In your comment, you provided only a link to ePEG.js, which by itself isn’t enough for me to see whether you fit the description above (except that ePEG.js’s TODO list is an indication that you probably have an idea where to lead PEG.js development).

I figured you may want to let your work and web presence speak for yourself, so I looked at your website and clicked on the Twitter and LinkedIn links, but they were both dead — which doesn’t give the best impression, as you can probably imagine. After that, I gave up. As for ePEG.js, at its current stage it’s hard to treat it as much more than a wish list.

So, let me ask few concrete questions:

• What is your background? In particular, what is your level of knowledge about parsers and parser generators? Where did you gain that knowledge?

• What is your motivation for working on PEG.js? In particular, do you use PEG.js (or ePEG.js) in your personal or work projects? Would maintaining PEG.js be part of your day job or would you do it in your free time?

• What is your level of experience with open source projects? In particular, did you or do you maintain any open source project comparable with PEG.js (mainly in the number of users)?

• What, in your opinion, are top 5 things PEG.js is currently missing or not doing right? Also, could you describe where would you like to take the project? Where would you like to see it in a year?

Thanks for answering these questions. They will help me in deciding whether you’d be the right future maintainer of PEG.js.

[futagoza]

so I looked at your website and clicked on the Twitter and LinkedIn links, but they were both dead

That site is a copy/paste of my old website I made in 2010 when I went by my old alias Vitron Prince, so the links are mostly not valid any more, or lead to sites/profiles I don't maintain any more, while the site was never finished properly. The only on-line presence I really have nowadays are GitHub and PlayStation 😆, mainly due to the fact I'm a full time carer.

What is your background? In particular, what is your level of knowledge about parsers and parser generators? Where did you gain that knowledge?

To sum it up, I learned on my own for the past 5 years, learning about string and buffer tokenizing, various ways to generate AST in a fast but accurate fashion while performing some memoization and mainly studying various methods to optimally transpile other computer languages to JavaScript and C++, identifying passes that could be squashed into a single pass to speed up compiling while using different AST visitors for different passes for the best performance.

As for parser generators, all I know was learned by studying the parser generator for PEG.js as well as coming in with the knowledge that comparing numbers is faster then comparing strings, so have preferred String#charCodeAt over String#charAt unless speed is not a factor. I still have trouble understanding the parts related to generating a parser using options.optimize = "size", but due to my focus on speed with accuracy, I never really gained the motivation to study this part of parser generation.

Also to note: although I learned quite a bit, I have trouble remembering the terminology 😊

What is your motivation for working on PEG.js? In particular, do you use PEG.js (or ePEG.js) in your personal or work projects? Would maintaining PEG.js be part of your day job or would you do it in your free time?

My motivation for working on PEG.js is developing CXLang, as well as making custom parser's for various text based resource files I will be using in my games, so I'd say I'm using it for both personal and work projects, so maintaining PEG.js, just like ePEG.js, would be mostly in my free time.

What I'm mostly focused on developing right now is (in order of what needs to be functional):

• PEG.js or https://github.com/futagoza/epeg.js
• https://github.com/cx-lang/cx-lang
• Roxby, a C++ game engine and Node.js based SDK to build seamless but customizable 3D worlds
• Xross M, a cross platform and game agnostic mod system
• Xross C, a game agnostic companion app
• A minecraft clone, with focus on cross platform multi-player and multi-platform saves
• A MMORPG game, set in 1 seamlessly huge and interactive world, incorporating pocket worlds
• A series of single player games based in the same pocket worlds featured in my MMORPG

It should be noted that I use PEG.js in CXLang, Roxby and Xross, which currently makes PEG.js a very important part of my game development, as it allows me to create various assets in plain text, and then using PEG.js and custom compilers, transform them into native code used by any of the games, other programs/tools or library code.

What is your level of experience with open source projects? In particular, did you or do you maintain any open source project comparable with PEG.js (mainly in the number of users)?

No, as I have been a carer since 2013, I have spent my free time either on video games, Korean drama's or working on various C++/Node.js projects (mostly those mentioned above), some of which I place on GitHub if I want to share the code, other's which I keep on my laptop as they contain code for the personal projects I am developing to create commercial products in the future.

Although CXLang is already open source (kind of, as various parts of it have yet to make it to GitHub due to continues development with little tests here and there), I only plan to open source Roxby and Xross when I have developed working prototypes for my game(s).

What, in your opinion, are top 5 things PEG.js is currently missing or not doing right? Also, could you describe where would you like to take the project? Where would you like to see it in a year?

• A reason not to make ePEG.js or alternatives 😛
• The current bytecode generator is only useful with the current JavaScript generator
  • It should not be part of the compiler passes, but a optional helper for generators
  • Should produce bytecode and AST to help generate parsers in any language (plugin specific)
  • Based on options, generate optional bytecode to guide parser generators
• Actions and string literals within the PEG.js grammar syntax restrict parser generators and source types:
  • An attribute (@returns ... or [returns = ...]) that sets the return type for actions used by rules
  • Syntax for sequence selection and extraction = fewer actions and less reliance on specific language
  • Buffer support via numeric literals, to develop parsers for files other then plain text source code
• Syntax that allows more rapid development of parsers:
  • Templates to write less PEG.js grammar
  • Attributes (pre-defined and custom) to easily manage the output of internal passes or plugin's
  • Importing rules from other grammars, allowing the developer to easily track down rules by filename
• Current plugin system makes it harder to change the API and optimise PEG.js for use in editors
• The parser generator is too complicated and not plugin-friendly. It should be split into different files.

Currently, the first thing you read about PEG.js (on the website or the GitHub repository) is:

PEG.js is a simple parser generator for JavaScript that produces fast parsers with excellent error reporting. You can use it to process complex data or computer languages and build transformers, interpreters, compilers and other tools easily.

I believe that resolving the above reasons would not only support what this paragraphs promises, but also give the developer more freedom in developing parsers.

To be honest, given my current schedule, I can't see where it would be in a year, but my roadmap for PEG.js would be something like this:

• PEG.js v1 - Contain's most of what your 0.11 milestone describes, but with multi-file support
• PEG.js v1.x - Maintained release (current LTS)
• PEG.js v2.x - Implement most of the above features, with code-base written in ECMAScript 2015
• PEG.js v3.x - LTS release
• PEG.js v4.x - ECMAScript 2017 code-base, Buffer support, new plugin interface
• PEG.js v5.x - LTS release
• PEG.js v6.x - ECMAScript 20xx code-base, complex data structures, new JavaScript parser generator
• PEG.js v7.x - LTS release
• PEG.js v8.x - ECMAScript 20xx code-base, more helpers and optional passes, Editor focused support
• PEG.js v9.x - Final LTS release, ECMAScript 20xx+ code-base

The reason for this roadmap is to help divide my tasks and user issues more easily, while also giving me room to learn more about the parser generator as well as planning a better plugin interface, readying my self for PEG.js v4.x and PEG.js v6.x

Also to note @dmajda, if I do take over as maintainer, I would eventually be also looking for more maintainers, contributors and moderators to help me manage different aspects so that I can focus on the development of the project, as that is my main focus, leaving aspects like the Twitter account and Google Group to the moderators, getting help from contributors to organise and help me maintain benchmarks and tests, while maintainers can help me manage LTS releases.

[playmyswift]

I'm using peg to write a markdown parser, thank you 😀

[playmyswift]

I am a front-end developer, and I can improve the website, can you invite me to join this organization? thank you!

[dmajda]

@futagoza First, let me apologize for my slow reply — it was hard for me to find a continuous period of time to read through what you wrote, think about it, and compose a reply during the last week.

From what you wrote, I can see you are well motivated and your qualification is solid. Your ideas about PEG.js also align well with what I think are biggest problems of PEG.js, as perceived by its users. I especially like that you’d like to involve more contributors — this is one area where I think I failed somewhat as a maintainer.

One thing I’m little worried about is that you’d like to see a lot of features introduced, which may lead to feature creep. A lot of PEG.js users value its simplicity.

In any case, given all the above and given that nobody else is interested in taking over PEG.js, I don’t see a reason why not pass PEG.js maintenance to you.

I hereby declare you a new maintainer of PEG.js.

Congratulations!

I’ll announce the change on Twitter and in the Google Group in a minute. Then I’ll send you an email with details regarding transfer of project assets. Once the transfer is done, I’ll back off from the project completely.

I hope you’ll take good care of PEG.js and that its future in your hands will be bright :-)

[dmajda]

I am a front-end developer, and I can improve the website, can you invite me to join this organization? thank you!

@EricJJJ Thanks for the offer, this is up to @futagoza now.

[playmyswift]

@futagoza 😂 Congratulations, I am a front-end developer, and I can improve the website, can you invite me to join this organization? thank you!

[futagoza]

@dmajda Thank you 😄, and I'll do my best.

I was worried about feature creep as well, that's why I added the LTS schedule to the roadmap above, but as I mentioned above, this isn't set in stone yet.

@EricJJJ Thanks for the offer m8 👍

I'm going to be going over what need's to be done the most, but the website is already in my list of things to update.

Once the transfer process is done, and I have made slight update's to the website to reflect maintainer change, I plan to open a new issue on the website's repository, where I will be going over what I plan to change, add or remove.

When I create the issue, I'll be sure to mention you, so you know when to jump in and show me your idea's and share your thought's. After I have decided where to go with the website, I will then invite you if you are still willing to help me 😆

[playmyswift]

@futagoza ok, thanks a lot 😀

[StoneCypher]

I would contribute if added to the org

[mikeaustin]

I could put something together for a larger step-by-step tutorial. A very basic language, like JavaScript stripped to it's core, would have helped me learn how to use PEG.js. Also knowing how / the best way to traverse the AST was something I had to find elsewhere.

The examples go from a tiny calculator to a full JavaScript parser. I think it would be nice to have something in between, and a tutorial to go with it.

Examples articles I've written:
https://structura.quora.com/Extension-Methods-in-JavaScript

[eraxillan]

So, @futagoza is new PEG.js maintainer.
But why is the issue still opened then?

BTW, thanks for such a great project guys!
One of my first programs was the Pascal language parser written in C++ - it was so hard to make it work :D
JavaScript + PEG.js make such task MUCH more easier

And currently i am using this project to create pretty complicated parser for bash-like language (qmake pro-file), and already faced the lack of functionality. E.g. absent of ability to split one big grammar file to several small ones.
What can i do to help with such things implementation?

[dmajda]

But why is the issue still opened then?

I just forgot to close it. Doing that now.