Proposal: Replace Abbot with Node based build tools #639

Closed
publickeating opened this Issue Dec 11, 2011 · 11 comments

Comments

Projects
None yet
7 participants
Owner

publickeating commented Dec 11, 2011

Description:

It has been a hot topic the past week, but it has also been a sore point for a long time and that is that getting a proper Ruby and gems installation isn't nearly easy enough. Anecdotally, I've seen dozens of people struggle with their Ruby installs and in particular the Windows users have an extremely hard time of it.

I propose we create an official sub-project, sproutcore/build_tools, based on Node.js that will become a 1-to-1 replacement of Abbot. Here's how I see it, but please throw in your comments below and educate me:

Pros:

  • We don't have to start from scratch. Martin Ottenwaelter & Maurits Lamers have been working on Garçon for over a year to do exactly this and I'm sure they wouldn't mind us bringing it in.
  • Node has their own installers, which eliminates the need for us to try and maintain the current installers made by Peter Wagenet. The Windows installer of Node is actually maintained by Microsoft! So yay, no more second-class support for Windows devs!
  • There are a lot of great Node modules that we can use to shortcut re-creating Abbot. See build modules and graphics modules. Personally, I've had wonderful success with the gm (GraphicsMagick) module on OS X and Linux and I bet we could crank out the Chance component without serious roadblocks.
  • Node build tools will be invariably faster than the Ruby ones (which have improved significantly as it were).
  • It's JavaScript, the language we all know and ❤
  • I think it's fair to say that Abbot is a difficult codebase to work on. This is a prime opportunity for us to rethink the architecture, properly do the command line help options, add cool new features (linting?), etc.
  • Others?

Cons:

  • Having an official sub-project that isn't complete is a gamble. Essentially, it has to have a certain level of support and maintain a high enough velocity of growth or else it will have to be removed again.
  • The amount of effort hasn't been spec'd out yet, but it's not likely trivial. The serious risks need to still be identified.
  • Others?

Like I said, please weigh in below.

1+ Node based build tools

I don't have anything against Abbot, it's got it's share of bugs and I think it has suited us well for a while. My primary reason for voting for Node based build tools is I have very little experience with Ruby. Abbot is very tough for me to develop and debug because of my lack of knowledge of Ruby. Even more so I find it a pain to setup and maintain a Ruby environment. Ultimately this all falls back to my lack of Ruby knowledge and I realize that.

In my opinion it makes SproutCore less attractive for new users. When the company I work for first evaluated the use of SproutCore as a web framework some of the developers, myself included, viewed the Ruby build tools as a disadvantage when compared to other frameworks because of the pain involved with getting an environment setup. In fairness things have gotten better since then, we even have installers (pkg and exe) to make it easier for new users.

Member

geojeff commented Dec 12, 2011

+1 Node based build tools

I used garçon fine in earlier versions of SproutCore, and have helped Maurits Lamers in recent explorations to move it forward to the current master. I'll let Maurits speak about progress he has made there, which sounds solid.

I am using ThothSC/Thoth for my backend, behind which I use python. Thoth is also written in node.js by Maurits Lamers. I have helped him with testing all along, even in its earlier incarnations, and have learned much in the process. Recently, I needed to add image uploading for my app, so naturally I looked at my options either in node.js by modifying Thoth or in Python, which I've done before quite a bit. I decided to try it in node.js, having seen node-imagemagick, node-canvas, and other graphics libraries. So far, node-imagemagick is only used for a simple call to im.identify to get image size data, and node-canvas, backed by the Cairo graphics library, is used for image resizing and thumbnails. It seems to work well, and I'd like to add some bells and whistles for backend image enhancement and manipulation using node-canvas.

This experience got me to thinking about the next generation of development tools for SproutCore. If we were to determine that one or several of these node.js graphics libraries are suitable for use, particularly if the dependencies are manageable for an installation package, we can dream up new ways of building tools. One way would be to stick to enhancing garçon itself, along the same lines that Chance worked. It is also possible to build a separate set of command line tools for the graphics handling sides of things. From my experience doing this sort of thing in python, it is best to build smaller tools that work standalone, then as you need to, either as part of an encompassing command line tool or as services used by a graphical user interface, you have maximum flexibility, at the cost of having to manage the different smaller bits. We have good repository tools that make management easier nowadays. And, about that graphical user interface possibility: I've also been thinking, and mentioning in irc, that we could build a simple SproutCore frontend to such a toolkit. For example, maybe one page of the app would be "button builder" where you select an image from your hard drive and fiddle with sliders, but all you would be doing is setting parameters for a call to one of the aforementioned command line graphics tools in node.js. A person with a more unixy preference could stay at the command line and work with scripting of the command line node.js tools, whereas a person, especially a new user, might like the convenience of the GUI, not even realizing perhaps that they are driving the node.js tools working behind the scenes.

Also, another bigger picture observation about languages: Yes, the javascript family connection and obvious benefit is a primary reason for new node.js tools to be front-and-center. But we should entertain the possibility of keeping Abbot/Chance in the mix -- we should weigh the burden of maintaining them, if fewer Ruby enthusiasts remain in our crowd, and also of having their API and design, especially if frozen, slow us down. If the SproutCore community were to gain popularity over the next few years for example, having a ruby build tools option could find enough maintainers, and this would work to bring in people from the large ruby/rails community. Likewise, I have contemplated building the equivalent of Abbot/Chance in python, something I could have a chance of doing myself, and am attracted by the idea, because, in part, there is a very capable and so far untapped wealth of python programmers who could not only help maintain such dev tools, but could help, obviously, on the datasource and backend side of SproutCore. Plus, some of these ruby and python programmers have top-notch skills in javascript as well, and if we could draw them, the core javascript system would benefit.

But of course, such dreams are dependent on the size of the community, even if technically sound and viable within a broad ecosystem.

Given that background about languages and synergies, I think a solid approach would go like this: Move to node.js development tools as the mainstay, because of the language family connection and because our small community can pull this off (I think we can, given the success of garçon already), then later, if the community grows, open the door for, or even provide encouragement for groups of developers to build similar systems in other languages. Even if such a broader ecosystem were to develop, keeping the officially supported build/dev tools in node.js would make sense because of the javascript connection.

Member

geojeff commented Dec 12, 2011

Also, I forgot to mention, that there could be a healthy draw of involvement from the community of node.js developers with the use of node.js build tools, and especially if we can use npm successfully.

Owner

mauritslamers commented Dec 12, 2011

+1 for node.js buildtools (of course :) )

I think that it is important to make a distinction between the version by Martin and by me, as my version (which lives on https://github.com/mauritslamers/garcon/tree/sc btw, not on master) is a SCified version with a number of changes to the workings (bundle support) and API (different approach to handlers).

I think it is important to make a choice of one of these versions. My version has a dependency on the SC runtime (and a few foundation things) from 1.4.5.

Moreover there is the issue of different sproutcore versions and how to support it. In the past different versions of SC have been accompanied by different versions of abbot. If this is also the way chosen for garcon, and my SCified version of garcon is chosen, it might be easier to integrate garcon within sproutcore itself. Installing sproutcore then also installs the right build tools in one go.

Contributor

mitchless commented Dec 12, 2011

Do we have a canonical set of source and expected result files that we can use as an acceptance suite? I recently had a conversation with Alex Iskander about regressions within chance that occurred when they moved from the SCCompress.jar to a Ruby based system.

I also like Maurits' idea of integrating the build tools into Sproutcore.

Contributor

lukemelia commented Jan 21, 2012

+1 I expect the challenges to fully replacing abbot master to be JS implementations of Sass/SCSS and Chance

Member

geojeff commented Jan 22, 2012

Spurred by a desire to finally grok build tools, to act on some of the ideas described above, and to learn coffeescript, I have started rewriting garçon in coffeescript as a new program called Busser: https://github.com/geojeff/busser [EDIT: changed from Waiter, because of npm conflict]. It doesn't work yet, but I decided to push it anyway. It comes up to a blank screen with errors. This seems like a big victory, however.

As for how it might fit in, who knows? I've received a couple of comments already, along the lines that people would prefer a node.js-based build system, but I am coming to think that the use of coffeescript to produce a javascript binary as a node program is fine and dandy -- coffeescript isn't needed to use busser -- a developer will be able to npm install it and use it, like other node.js programs. I have enjoyed learning coffeescript and am excited about using it more (Coming from 5 plus years of Python to Javascript and SproutCore has left me wanting for pythonic expressiveness). Not that I am uncomfortable writing javascript, and not that the hard-won understanding gained about javascript hasn't been critical for writing an async-style garçon derivative in coffeescript! I am still learning. Please see the README.md of busser for discussion of todo items, and description of a goal of angling toward the python world as a personal goal.

There are lots of things to look at, such as the use of nconf for command line and file configuration, the redesigned handler system, modification to the build and save sections, and the use of coffeescript classes and other coffeescript programming aspects (you can look at martoche garçon to see the original javascript).

Please see docs/busser.html for a docco presentation of comments in the code.

Many thanks to Maurits Lamers for continual help in explaining how garçon and async programming work!

As we all evaluate node-based build tools, I hope busser added to the mix will offer benefits to the community. I will continue its development as a personal exercise whether or not it becomes a serious contender. Other new node.js build tools are described in the Roadmap/Todos section of the busser README.md.

Owner

dcporter commented Oct 8, 2012

I just wanted to bump this issue and add that alpha-level node build tools have been developed and deployed by @mauritslamers and @geojeff, and are available at https://github.com/sproutcore/build-tools

Owner

mauritslamers commented Oct 9, 2012

In order to help this issue forward, it is extremely important that we make a few decisions on whether or how we are going to support sass and slicing under Windows.

For speed, I have been using node-sass, a wrapper around libsass (the C lib parsing sass) and node-canvas.
Especially the latter has quite a few typical *nix based dependencies (cairo, pixman, libpng).
I have been kicking the node-gyp build files around in order to get a binary package that could be distributed as part of the installation package of sproutcore, but so far without success.

While manual compilation under Windows is an option, we have to take into account that most Windows web developers don't have a compiler or compilation environment installed.

The best would perhaps be to have pure-JS alternatives, but these are either very, very slow and not entirely matching sass (stylus), or not implemented at all (slicing).

I would therefore really like the input of the coreteam first, and then the community on how to proceed on this.

Owner

dcporter commented Oct 10, 2012

I'm hearing that although folks are not willing to give up these features, they're generally willing to do an additional install (with compilation) in order to get them. This supports your idea of having a no-compile, single-command install that supports all systems, with simple instructions available for getting back up to feature parity re: sass and slices.

Owner

dcporter commented Feb 15, 2014

Closing this issue as the votes are in and the discussion has moved elsewhere.

@dcporter dcporter closed this Feb 15, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment