Moving Fractal forwards #197

Open
allmarkedup opened this Issue Jan 3, 2017 · 10 comments

Projects

None yet

7 participants

@allmarkedup
Member
allmarkedup commented Jan 3, 2017 edited

Over the last year and a half Fractal has evolved from an internal prototype at Clearleft into an open source project that we (and now many others) are using to create and manage component libraries. It’s been exciting to get validation of the core ideas behind the project and great to see all the different ways that others have been pushing at the boundaries of what is currently possible with it.

When we were developing Fractal our goal was to create a component library tool that was as agnostic as possible to things like build tool and template language choices, and also offer opportunities to build upon the core feature set to allow better integration and custom workflows for development teams. To some extent I think that we were successful in this; but it is also clear from the feedback over the last year or so that there is plenty of room for improvement!

After seeing how people are trying to fit Fractal into their development process, and thanks to some great input from the community, I’ve identified the following key areas to work on improving in 2017. Please chime in on the comments below if you think I’ve missed anything!


New features and usability improvements

The following are fairly loose ideas on potential areas of improvement with regards to the core functionality and general usability of Fractal. More detailed RFC-style issues may be posted separately for proposed improvements that warrant further technical discussion.

Components

A number of great suggestions for potential improvements to the way that components are created and handled by Fractal have come up in the past year. The current system has proven to be able to serve a lot of fairly disparate use cases, but not always without compromising the clarity of the component configuration or the structure_organisation of view templates and their partials_includes.

At Clearleft we have also been talking a lot about how we might be able to use component libraries earlier on in the design/build process, perhaps by creating ‘placeholder’ components that do not have any code yet, but do have a README or a static mockup of a component. In this way you could quickly scaffold out a component library structure, and iterate on the organisation and design even before adding a line of code. Components could then be built out with the templates and other assets as the project progresses.

Proposed improvements

A detailed list of proposed improvements to the component system (with more technical detail) will be published as a separate issue for people to comment on. However the main suggestions will include:

  • Moving towards a more explicit set of naming conventions for components and their associated files to simplify the parsing process and to be able to more easily move and rename components.
  • Supporting multiple, concurrent view template languages - i.e. having the ability to define (for example) Handlebars and Nunjucks versions of a component side by side in your component library.
  • Supporting the creation of placeholder components with just a README or graphical mockup of a component which can be viewed in the web UI.
  • Supporting multiple component source directories.
  • Simplify template engine adapters, look into support for client-side rendering where possible?
  • Providing better ways to supply templates with the default context data for their included partials when nesting component templates, without compromising the ability of the view template to be used in a non-Fractal (i.e. production) environment.
  • Improving the performance of the filesystem parsing through intelligent caching of unchanged collections/components.

Related issues

Pages/docs

The current documentation system is based on the same filesystem parsing, configuration logic and template engine handling as the component system is. The aim was to allow people to use a mix of markdown and templating language features to build docs and ‘styleguides’ that would complement the component library.

Whilst people have indeed been creating some nice examples of documentation/styleguides, the process is currently hampered somewhat by the lack of a standard ‘UI library’ of style guide components (such as colour swatches) and the inability to easily add custom styles to the documentation in the web UI. Additionally, the documentation system lacks ‘knowledge’ of the component library itself, which means that things like generating links to components, rendering components within the documentation or displaying ‘live’ information about components is currently not straightforward and requires significant technical knowledge.

Creating a UI library of ‘partials’ for style guide components and even helpers for linking to/displaying components is partly a significant challenge because of the flexibility Fractal offers when it comes to template engines and organisation of the component library.

Proposed improvements

While it’s clearly a key feature that Fractal supports multiple template languages for creating components, the use case for supporting the same functionality for the documentation generation is less clear. I propose switching Fractal to enforcing use of a single predetermined template engine (most likely Nunjucks) for generating documentation. This would make it considerably easier to create and document a canonical set of partials for creating styleguides/docs, and to tie in the components in the component library much more closely into the docs to allow for easier linking and embedding etc. This simplification would benefit less technical users of Fractal.

Related issues

Web UI/theming

The current model of theming was intended to give a lot of power to people that wanted to develop completely custom themes (right down to the URLs), completely distinct from the default ‘Mandelbrot’ theme that ships with Fractal. However, this was somewhat at the expense of making it really easy to customise the ‘look and feel’ of the theme itself without a fair bit of technical knowledge.

Since releasing Fractal, there has been considerable interest in customising the Mandelbrot theme, and very little interest in developing completely custom themes. This is understandable; Mandelbrot does a lot of the heavy lifting with regards to displaying previews, code etc and it would be a lot of work to re-implement this in a completely bespoke UI. (The custom theme route is also currently very under-documented which doesn’t help)

Proposed improvements

It seems to make sense to ‘pave the cow paths’ - double down on the 95% use case (customising Mandelbrot) and optimise that over the 5% use case (developing bespoke UIs).

To this end, future releases should concentrate on moving the current web UI server_builder_theme model in this direction. This would likely involve moving the responsibility for routing and view handling out of the theme and into the web UI server, which would simplify the theme templates significantly and have them act as more traditional simple static ‘view’ templates. This would decrease both the complexity of the the theme setup/configuration and mean that the templates require much less logic within them to handle rendering of components and displaying documentation pages.

The server instance itself should also be made available so that custom Express middleware can be added to extend the server functionality if required.

Related issues

Extensibility

Fractal currently lacks ways to customise its core parsing behaviour, or to extend the component configuration handling process. The result is that feature requests for additional functionality either have to be added to core, or not implemented at all. In the spirit of trying to keep the main codebase as focussed as possible this would be a key area to work on so that we can try and kick-start a plugin-based ecosystem for Fractal.

Proposed improvements: Add hooks to allow customisation of the core filesystem parsing behaviour, as well as a mechanism for extending the component, docs, sources and collection entities that are exposed via the API.

Related issues:

Tooling

Fractal’s in-built CLI currently has a very limited set of functionality, restricted to starting/stopping the web server and not much else. Given the availability of the API and the potential to build many more useful tasks, this is a shame.

Proposed improvements

Extending the list of built-in tasks to component scaffolding, renaming of components, ‘doctor’ tasks for identifying potential problems and much more would significantly increase the speed at which component libraries could be assembled and changes to existing components made.

Related issues


Codebase improvements and refactoring

To meaningfully be able to support many of the above changes going forward, the core codebase needs a significant overhaul. The current Fractal implementation suffers from the somewhat haphazard and ‘organic’ process from which it moved from prototype to open source project. This is also currently a pretty major blocker to getting more developers onboard with contributing to the project.

The following areas need to be addressed to make outside contribution easier and to have more confidence in releasing updates (and releasing with more regularity):

  • Modularity: The current codebase is a monolith, which makes contributions and testing much harder. There is plenty of opportunity to extract out of this codebase smaller, more focussed modules and manage these as dependencies instead.
  • Testing: The current lack of any real test coverage is a major barrier to being able to release updates with any confidence. New versions of Fractal will need to double down on testing in order to move the project forwards.
  • Clarity: The current codebase is an evolved prototype rather than a ‘designed’ piece of software. Now that the aims and feature set are much better understood, work needs to be done to improve the clarity and focus of the code to make it easier for for outside contributors to dive in and help out with the project.

Other

‘First run’ experience

Fractal’s un-opinionated nature when it comes to build tools and template languages can also increase the complexity when it comes to less technical users setting up new projects. Some quick-start, pre-built ‘templates’ or generators for a Gulp-based project, a Grunt-based project, etc. would be a great help for getting people who are not interested in customising these aspects themselves up and running quickly.

Community communication and contributions

Currently the Slack channel and Github issues are the only way of communicating with the community regarding upcoming releases, important bug fixes and trying to solicit feedback on proposed features. How can this be improved?

Some thoughts would be:

  • Move to Gitter (like Slack but publicly browsable). But is this too niche?
  • Twitter account for announcements
  • A blog for longer form tutorials/release notes?

We also need to provide better contribution guidelines and development information to help people get involved.

Better documentation and examples

This is an ongoing process, but apart from filling in the blanks in the Guide we need to look at how the docs are built and hosted. Currently it’s a pretty manual process but it would be good to open this up so that PRs can be quickly and easily pushed live.

Autogenerating API docs from Docblock comments or similar needs to be investigated as a way to keep these docs in sync with development.

An ‘official’ demo site is still sorely needed, as well as some sort of ‘tips and snippets’ repo/site where useful configuration examples and how-tos can quickly and easily be posted up.

In closing

There is a lot of potential work described above, and much of it will require some significant changes to Fractal’s underlying codebase. It will also likely involve some major breaking changes to some parts of the existing Fractal APIs, although keeping compatibility with existing configuration file formats and filesystem/folder structures will be treated as a priority.

However none of the above is set in stone, and much of it may not even make its way into the next major release. Feedback is most welcome, including ideas on what should be prioritised and if any of the above causes people any concerns with regards to the way that they are currently using Fractal.

I’ll shortly be publishing a roadmap of how I see some of the above features fitting into the next few cycles of development, but until them please throw any questions or comments my way :-)

@shawnbot
shawnbot commented Jan 3, 2017

Re: "Supporting multiple, concurrent view template languages", I would be happy to look into whether/how Fractal could use meta-template to convert templates from Nunjucks to other template systems automatically. (For instance, imagine if you could generate a directory of drop-in ERB or Django templates from a Fractal style guide with a single command!) This would really help me think through how collections of templates could be converted en masse while preserving partial references, etc.

@shawnbot
shawnbot commented Jan 3, 2017

+💯 to "Supporting multiple component source directories"! This would make customizing or composing style guides easier, which is something we would love to support when we (eventually!) port the US Web Design Standards over to Fractal.

@allmarkedup allmarkedup changed the title from [Request for Comments] Moving Fractal forwards to Moving Fractal forwards Jan 4, 2017
@tlenex
tlenex commented Jan 4, 2017

About new features and contribution: Some time ago I was thinking about creating components in angular2 way - using angular-cli would speedup development process alot.
But I'll have to make a research on how to create new component engine for fractal - api documentation/tests would be much welcome :)

@allmarkedup
Member

@shawnbot regarding meta-template integration - I'd love this! For me this seems like something that would be best suited to a bespoke CLI command however, rather than something integrated into core. I'm sure it would actually be possible with the current version of Fractal (although obviously Fractal wouldn't be able to display the converted templates).

And the more I've been thinking about the multiple source directories the more excited I get about the possibilities. Things like being able to distribute a core set of components as an NPM package that other libraries can then build off/override would be very cool indeed.

@allmarkedup
Member

@tlenex I'm not actually familiar with Angular 2 components, but I'll take a look. I'd definitely like to get some CLI tasks added to Fractal for creating new components or updating/renaming/moving existing ones however, it would definitely cut down on a lot of repetition.

And yes... test coverage and API docs is definitely needed and will certainly be made a priority for the next version of the component parser (and all the rest!).

@paulrobertlloyd
Contributor

I'm in agreement with many – if not all – of your thoughts here Mark; thanks for taking the time to reflect on progress so far, and your plans for the future. One other piece of information that might be useful is what Fractal is not (or will not be), i.e. a prototyping tool, task runner, CMS etc. The focused nature of this project is a strength, and I believe it’s important to avoid any feature creep/bloat. I’m sure making the code more modular and creating a more pluggable architecture will help in this respect.

@ixley
ixley commented Jan 4, 2017

@allmarkedup This looks great. I'm excited to see such a strong outline for moving Fractal forward. I've weighed in on a couple of the items you outlined already, but I'll chime in with my (unordered) priority wishlist.

  • Better support for modular components, including more straightforward methods for including default content and then selectively overriding some or all content for the new context (ease of nesting).
  • Allow components to be referenced in docs.
  • Breaking up Mandelbrot to make basic changes to the UI simpler and separate from the core functionality.

In terms of documentation and getting started, I really appreciate well documented or intuitive examples. Referencing the demo project helped me get oriented to a lot of the basics. I think a good sample project is just as valuable as well written docs or tutorials.

For communication, I personally prefer using Slack over Gitter, but don't have strong feelings. I think it's less important that chat rooms are publicly searchable though, and more important to have the relevant bits that maybe come out of discussions to be put into appropriate documentation. I think Framer offers a good model with their documentation, blog, and gallery. Obviously it is a more mature project, but the structure and quality seems worth referencing.

@allmarkedup
Member

@paulrobertlloyd great to hear Paul :-) And I think your comment about focus and communicating that focus is actually really important. I'll try and draft a concise description of what Fractal is and isn't going to be and add it add it to the post. Thanks.

@ixley many thanks, that is all good info. I think that all your wishlist items are covered by the proposed additions above. The default content one is certainly high on my list but it's also not an easy problem to solve if also trying to make sure that templates can be picked up and used outside of Fractal without any hoop-jumping required. But i've got a few ideas that I still need to hammer out to get the ramifications clear in my head.

I haven't looked at the Framer site in a very long time but you are right, they look like they are doing a great job with the mix of 'getting started' content and the technical deep-dive stuff. I'd love to eventually have something like that! Good inspiration for sure.

@samuells
samuells commented Jan 9, 2017

@allmarkedup Really nice summary and I can only agree with it. I played a lot with theme and GUI to change. At start it was funny, but it's not so complicated. What was missing - is documentation.

Maybe two little thing to think about

  • missing "front matter" in .md files of components - it opens a lot of possibilities
  • also assets linking - picture, svg, fonts, etc link on multiple places (Documentaion/Components/.md/.css files)
@msecret
msecret commented Jan 10, 2017

Thanks for asking for comments @allmarkedup. I think I agree with the direction things are heading in. In terms of custom theming, it would be nice to see a simpler method of customization rather then a completely new theme or even writing a CSS file with new rules. I't be helpful to override a set of variables, mostly for color and typography. When I themed our Fractal guide, that was mostly what I needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment