Future of Beautification: Beautifying All the Things

[Glavin001]

A place for us to discuss ideas 😃 .

/cc @prettydiff @bitwiseman

[prettydiff]

A big thing that has been on my mind is separating my parsers from my beautifiers. Right now I basically have three general language files that I support for: markup, css, and C-style languages. For more than a year I have come to the belief that I am likely much better at writing parsers than beautifiers.

For example there is a new popular beautifier, prettier, that might be a more flexible beautifier than Pretty Diff, but they struggle to add capabilities I have offered for a long time. They do not have their own parser, and so they have to work harder to provide capabilities that come a bit easier to the Pretty Diff code system. By separating the parser from the beautifiers I could allow different beautifiers on top of my parsers, which seem to do a good job of offering multi-language support.

---

A very real problem I have noticed historically with Atom Beautify is distribution and configuration. The degree of this problem seems to vary by geography, in the case of NPM/APM failures, and by language in the case of dependencies that include independent installation and configuration.

A major thorn is PHP support which requires that PHP language be locally installed and the PHP beautifier be independently configured. This is demanding a lot of work on the user to get Atom Beautify to work, and so the probability of questions and failure sharply increases.

Regarding the NPM/APM concerns I have noticed a large number of reported issues from first time users about missing dependencies. The answer is always to reinstall and hope for the best. This is a horrible answer. NPM finally appears to offer some level of integrity checks with NPM5 that is released this week. It will be important to monitor and prove that NPM5 directly fixes these concerns.

If NPM5 still does not solve the integrity problem I propose two things to solve the dependency and configuration concerns:

• Reduce the number of dependencies and configuration steps by creating a build step that does most of this before publishing an update. A good example is homebrew which simply downloads a single application package, runs the installation with all configuration steps, and verifies the result.
• Use IPFS for the distribution space. IPFS packages are like a file system fragment managed with git and pushed into the cloud as a universal package known by a hash identifier that is transmitted similar to a bittorrent item.

IPFS would solve the distribution and integrity parts. We would just need to pre-configure and package the application appropriately to publish a single update to IPFS, so that when a user requests the application it is already built and simply downloads to their OS ready to execute. I have been looking at IPFS recently and they already accomplish most of this by virtue of their technology. Since their technology is essentially a packaged section of a filesystem they provide limited dependency integration provided the dependencies are included and organized prior to publishing an update.

What they don't include is any sort of application management assistance for things like fulfilling configuration steps remotely or validating application installations in a post-download step. I don't think extending IPFS to become something like Homebrew has been considered yet by the IPFS project. These things can still be accomplished, but it would probably be something we would have to write. If NPM5 proves to still have problems this is the area I will likely spend the bulk of my time.

[Glavin001]

For example there is a new popular beautifier, prettier, that might be a more flexible beautifier than Pretty Diff, but they struggle to add capabilities I have offered for a long time. They do not have their own parser, and so they have to work harder to provide capabilities that come a bit easier to the Pretty Diff code system. By separating the parser from the beautifiers I could allow different beautifiers on top of my parsers, which seem to do a good job of offering multi-language support.

Is this the prettier package you mentioned?

On a related note, I would love to learn more about how Pretty Diff and JS-Beautify works so I can also contribute there. I also want the beautifier to provide more debugging tools which may help beautifier maintainers.

A major thorn is PHP support which requires that PHP language be locally installed and the PHP beautifier be independently configured. This is demanding a lot of work on the user to get Atom Beautify to work, and so the probability of questions and failure sharply increases.

I am investigating using Docker to install third-party beautifiers which are not written for Node. See Glavin001/atom-beautify#1687 . I will be testing this on Atom-Beautify soon and if it is received well by the users then it will be included in Unibeautify support, too.

Reduce the number of dependencies and configuration steps by creating a build step that does most of this before publishing an update. A good example is homebrew which simply downloads a single application package, runs the installation with all configuration steps, and verifies the result.

Unibeautify is going to split the beautifiers into individual packages. For example, see https://github.com/Unibeautify/beautifier-prettydiff . In this way, the users will install only what they want to use. And Unibeautify core itself should have a much more lightweight installation with improved performance.

[prettydiff]

Is this the prettier package you mentioned?

yes

I would also like to workout more debugging and analysis tools. These are great and fun to work on, but are typically low priority and always seem to fall by the way side.

[bitwiseman]

It looks to me like this issue is going to become a scratch pad of ideas and concerns from which we'll create/link other issues and continue down those paths.

So far we have ideas regarding:

• distribution
• dependency management
• runtime
• Can we separate parsers separate from beautifiers?

But to me it sounds like we haven't even agreed on terminology yet: What is a "beautifier"? What is a "parser"?

I've started #9 to talk about parsing, and #10 to talk about the scope/intent of unibeautify.

[Glavin001]

This is how I pictured breaking down Unibeautify and related systems: https://docs.google.com/drawings/d/1elu3OU4o37_lDiDNgovolXdY7D1VQ-_9nsDs5y1HlQY/edit

[bitwiseman]

@Glavin001 Interesting. I think I see what you're going for, but I'm not sure.

I appreciate that you want to discuss all these topics in one place, but made separate threads because each of those topics can be addressed independent of the others and if we try to address them all in one thread it is likely we'll loose details and shift topics. Could you edit your comments in the other thread to include the image itself instead of a reference? And then add some verbal comment appropriate to each topic explaining how the image applies in that case?

[Glavin001]

if we try to address them all in one thread it is likely we'll loose details and shift topics.

Agreed. Sorry about that, it was almost 3am and I wanted to share this before I went to sleep 😛 .

I'll go through and update the comments after work.

[bitwiseman]

@prettydiff - could you move this comment to #9 which is about the parsing framework?

[prettydiff]

Moved as requested.

[prettydiff]

On Reddit @vjuex pointed me to all the unit tests for Prettier, https://github.com/prettier/prettier/tree/master/tests

He also suggested we look at https://github.com/lydell/eslump as it help them find problems associated with edge cases.

[stevenzeck]

Can the https://github.com/Unibeautify/unibeautify/blob/master/ROADMAP.md be updated with any decisions that have been made? And/or create a GitHub project under this org? Just want to find where I can contribute more here without causing trouble.

Is the idea behind the vscode and unibeautify-sublime repos to code the extensions for those editors?

[prettydiff]

Is the idea behind the vscode and unibeautify-sublime repos to code the extensions for those editors?

Yes. The idea is that to achieve separation of concerns so that:

1. There is a single IDE library that works with the API for the given editor to read the code and provide the various options.
2. There is a universal parse framework to interpret from various languages but output a uniform format.
3. That format will then feed into the various beautifiers, which have different styles and capabilities in beautifying code.
4. The beautifiers output a string of beautified code back to the editor library so that it can be output to the user.

Help will be needed at every level. Here is an idea of the work ahead:

• The editor libraries will be new. The hard part here will be keeping these guys up to date as the editors change and as this project evolves to support new options/features.
• The parse framework is written, but currently it isn't supporting vary many languages. I am working on a Github style markdown right now (which is much harder than I thought going in). There is always going to be a need for language support, bug fixing, new language libraries.
• The beautifiers will demand a lot of work with regard to using the parse framework and accounting for their niche features that likely aren't already accounted for. It looks like Liam might have already started working on JS Beautify. I haven't even started migrating Pretty Diff yet, and Prettier will need somebody to work with it. I suspect Prettier might be the easier of the beautifiers to adapt because it does not have an internal parsing scheme of its own and has the largest number of test units.

I will work on starting some contributing documentation to explain these various facets more clearly and provide a diving board for people to contribute.

[stevenzeck]

@Glavin001 Is the intention to rewrite the Atom package in typescript as well?

[Glavin001]

Is the intention to rewrite the Atom package in typescript as well?

Yep. 🤞

[stevenzeck]

I created a repo for it, just boilerplate code right now as I learn Atom package development using Typescript.

[Glavin001]

I was going to keep "Atom-Beautify" as the "Unibeautify for Atom", in order to keep the existing user base and prevent broken links and confusion. 😝

[Glavin001]

I am also working on VSCode right now. What I do need help with is Sublime Text plugin, which I think should use the unibeautify-cli for running beautification and write the rest of the plugin in Python -- unless Sublime supports Node.js based plugins now.

[stevenzeck]

Ah ok, can that be moved under unibeautify or does that cause problems?

I'll delete that repo and start a new branch in atom beautify.

I've never used Sublime, so don't know how much I can help in the short term

[Glavin001]

Ah ok, can that be moved under unibeautify or does that cause problems?

For the foreseeable future it will stay where it is. I would like to eventually move it, however no rush to do so. Especially since the code is not ready yet 😉.

I'll delete that repo and start a new branch in atom beautify.

There is an existing branch: https://github.com/Glavin001/atom-beautify/tree/unibeautify

I will verify tonight if I have pushed all of my changes. I think I may have some Work-In-Progress changes.

I've never used Sublime, so don't know how much I can help in the short term

No worries. I am not a Sublime guy either 🤷‍♂️ . Will need someone to help us out with that.

[Glavin001]

Closing this issue so discussions can continue on more focused issues.