Add ability to build plugins #203

Closed
pascalduez opened this Issue Sep 8, 2014 · 24 comments

Comments

Projects
None yet
4 participants
@pascalduez
Member

pascalduez commented Sep 8, 2014

Observations

Every Sass project or library is unique, leading to varied needs and wants for their documentation. Ergonomy/usability problems arise as soon as the project crosses a certain size.
However we want to keep SassDoc core lean and agile with what we think are the required base features.
But being able to augment the range of possibilities would be definitely a plus and help target more projects. This is where allowing external plugins to extend core comes in mind. (adding new annotations or behaviors)

In the wild: “I want to be able to split my docs into several pages instead of one monolytic index.html, or build a backbone/angular/younameit app with it”.

Case study

I was reading through the @tutorial annotation, and though I would enjoy adding such thing to my own libs. This involve some logic and complexity that would perfectly fit into a plugin. So let's use it as an example.

Example: @tutorial <tutorial-01.md>

  • Add a new annotation to sassdoc.
  • Maybe read some new config keys (base path) from .sassdocrc
  • Do some path resolving (based on a config item ?) to fetch .md files.
  • Post process data to grab the .md files and convert them to .html file.
  • Copy .html files to destination folder.

Suggestions

SassDoc core should expose a registerAnnotation method.

Possible ways to include a plugin:

Calling SassDoc API:

var sassdoc = require('sassdoc');

sassdoc.use('sassdoc-annotation-tutorial');

sassdoc.documentize(source, dest, config);

Passing plugins to .sassdocrc:

{
  ...

  "plugins": [
    "annotation-tutorial",
    ...
  ]

}

Or enforce the data postprocessing to happen at the theme level index.js:
Note ideal to do the job (and require the module) at two different places

var tutorial = require('sassdoc-annotation-tutorial');

module.exports = function (dest, ctx) {

  ctx.tutorials = tutorial(ctx.data);

  return theme.apply(this, arguments);
};

EDIT: @tutorial is just an example, it might even make more sense to think of a more wide @page thing.

Discussion

  • Do we want plugins or not ?
  • What's the best way to implement them ?
  • We should not force plugins to be npm published modules, but also allow local ones ?
  • Should we have a sassdoc-extra repo/module with some more "exotic" stuff in ?
@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 8, 2014

Member

Do we want plugins or not ?

I've been thinking about something like this for a while now. I'm all in favor.

What's the best way to implement them ?

I'll leave that to you guys.

We should not force plugins to be npm published modules, but also allow local ones ?

Yep.

Should we have a sassdoc-extra repo/module with some more "exotic" stuff in ?

We already have sassdoc-filter and sassdoc-index. I don't think we need sassdoc-extra, else we need to gather everything inside.

I will add a question: do you think we can do this without breaking API?

Member

HugoGiraudel commented Sep 8, 2014

Do we want plugins or not ?

I've been thinking about something like this for a while now. I'm all in favor.

What's the best way to implement them ?

I'll leave that to you guys.

We should not force plugins to be npm published modules, but also allow local ones ?

Yep.

Should we have a sassdoc-extra repo/module with some more "exotic" stuff in ?

We already have sassdoc-filter and sassdoc-index. I don't think we need sassdoc-extra, else we need to gather everything inside.

I will add a question: do you think we can do this without breaking API?

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
@valeriangalliat

valeriangalliat Sep 8, 2014

Member

Do we want plugins or not?

+1 for plugins.

What's the best way to implement them?

I like both of your suggestions; the sassdoc.use functions is great when calling SassDoc programmatically, but the plugins key in .sassdocrc is suitable for every possible usage (we already have this .sassdocrc everywhere, even as literal object when calling from the API). Therefore I don't think we need a sassdoc.use function.

But while speaking of the sassdoc.use function, note that in your exemple, it has to modify the sassdoc context, so using a plugin would mean to use it globally. I prevented this in Themeleon by forcing to create a new Themeleon instance (hence require('themeleon')()). Other projects like Swig have a global instance by default (require('swig')) but let you create a new instance without side effects in the default one (require('swig').Swig). That's maybe a little more user friendly.

We should not force plugins to be npm published modules, but also allow local ones?

Yes. Taking (again) Themeleon as an example, the use function can take a string (passed to require, so it can be an absolute path to a local directory, or a regular npm module - Themeleon only misses a way to specify a relative path to CWD, any relative path would be relative to the Themeleon module directory because it's the default behavior of require) or even a function.

So this could be quite similar even in the .sassdocrc, a plugin item could be a regular npm module, absolute path to local module, or a path relative to .sassdocrc.

Should we have a sassdoc-extra repo/module with some more "exotic" stuff in?

sassdoc-filter and sassdoc-index are helper for the theme, so we can't gather everything inside them. I'm in favor of creating a sassdoc-extra (or sassdoc-extras?) module with official plugins, like they do with swig-extras if we have new annotations that are maybe too specific to be in the core.

I will add a question: do you think we can do this without breaking API?

We need some more investigation, but I don't think it will break the API. If we keep the global SassDoc instance and add a sassdoc.use function, and plugins key in .sassdocrc, this won't break anything.

Member

valeriangalliat commented Sep 8, 2014

Do we want plugins or not?

+1 for plugins.

What's the best way to implement them?

I like both of your suggestions; the sassdoc.use functions is great when calling SassDoc programmatically, but the plugins key in .sassdocrc is suitable for every possible usage (we already have this .sassdocrc everywhere, even as literal object when calling from the API). Therefore I don't think we need a sassdoc.use function.

But while speaking of the sassdoc.use function, note that in your exemple, it has to modify the sassdoc context, so using a plugin would mean to use it globally. I prevented this in Themeleon by forcing to create a new Themeleon instance (hence require('themeleon')()). Other projects like Swig have a global instance by default (require('swig')) but let you create a new instance without side effects in the default one (require('swig').Swig). That's maybe a little more user friendly.

We should not force plugins to be npm published modules, but also allow local ones?

Yes. Taking (again) Themeleon as an example, the use function can take a string (passed to require, so it can be an absolute path to a local directory, or a regular npm module - Themeleon only misses a way to specify a relative path to CWD, any relative path would be relative to the Themeleon module directory because it's the default behavior of require) or even a function.

So this could be quite similar even in the .sassdocrc, a plugin item could be a regular npm module, absolute path to local module, or a path relative to .sassdocrc.

Should we have a sassdoc-extra repo/module with some more "exotic" stuff in?

sassdoc-filter and sassdoc-index are helper for the theme, so we can't gather everything inside them. I'm in favor of creating a sassdoc-extra (or sassdoc-extras?) module with official plugins, like they do with swig-extras if we have new annotations that are maybe too specific to be in the core.

I will add a question: do you think we can do this without breaking API?

We need some more investigation, but I don't think it will break the API. If we keep the global SassDoc instance and add a sassdoc.use function, and plugins key in .sassdocrc, this won't break anything.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 8, 2014

Member

If we can pass it, let's schedule it for 1.8.

Member

HugoGiraudel commented Sep 8, 2014

If we can pass it, let's schedule it for 1.8.

@pascalduez

This comment has been minimized.

Show comment
Hide comment
@pascalduez

pascalduez Sep 8, 2014

Member

I will add a question: do you think we can do this without breaking API?

My first (very quick) scanning, resulted that it won't involve much changes to the actual code base actually. Just some hooks to inject new annotations, and being able to postprocess the context.
Also obviously plugin registering/loading.

but the plugins key in .sassdocrc is suitable for every possible usage

That's the nicest solution, agree.

So this could be quite similar even in the .sassdocrc, a plugin item could be a regular npm module, absolute path to local module, or a path relative to .sassdocrc.

Awsome.

sassdoc-extra (or sassdoc-extras ?)

With a s :)

If we can pass it, let's schedule it for 1.8.

I'm looking forward to give this @page thing a try.

Let's ping @FWeinb here, hopefully he can make it...

Member

pascalduez commented Sep 8, 2014

I will add a question: do you think we can do this without breaking API?

My first (very quick) scanning, resulted that it won't involve much changes to the actual code base actually. Just some hooks to inject new annotations, and being able to postprocess the context.
Also obviously plugin registering/loading.

but the plugins key in .sassdocrc is suitable for every possible usage

That's the nicest solution, agree.

So this could be quite similar even in the .sassdocrc, a plugin item could be a regular npm module, absolute path to local module, or a path relative to .sassdocrc.

Awsome.

sassdoc-extra (or sassdoc-extras ?)

With a s :)

If we can pass it, let's schedule it for 1.8.

I'm looking forward to give this @page thing a try.

Let's ping @FWeinb here, hopefully he can make it...

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 8, 2014

Member

I'm looking forward to give this @page thing a try.

Let's release 1.6 first.

PS: Still 10 days before Fabrice's return.

Member

HugoGiraudel commented Sep 8, 2014

I'm looking forward to give this @page thing a try.

Let's release 1.6 first.

PS: Still 10 days before Fabrice's return.

@HugoGiraudel HugoGiraudel modified the milestone: 1.8 Sep 8, 2014

@HugoGiraudel HugoGiraudel added the Feature label Sep 8, 2014

@HugoGiraudel HugoGiraudel changed the title from SassDoc plugins to Add ability to build plugins Sep 8, 2014

@FWeinb

This comment has been minimized.

Show comment
Hide comment
@FWeinb

FWeinb Sep 9, 2014

Member

I think that only a theme should be able to add plugins, I can't really think of a context where it would make sense to add a plugin without a custom theme. So to sum up I think we need this:

  • Plugin handling in SassDoc;
  • Expose a plugin API to the theme.

What do you think?

Member

FWeinb commented Sep 9, 2014

I think that only a theme should be able to add plugins, I can't really think of a context where it would make sense to add a plugin without a custom theme. So to sum up I think we need this:

  • Plugin handling in SassDoc;
  • Expose a plugin API to the theme.

What do you think?

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 9, 2014

Member

Makes sense. Sounds good to me.

Member

HugoGiraudel commented Sep 9, 2014

Makes sense. Sounds good to me.

@FWeinb

This comment has been minimized.

Show comment
Hide comment
@FWeinb

FWeinb Sep 9, 2014

Member

I need @valeriangalliat on this. He is the author of our theming framework.

Member

FWeinb commented Sep 9, 2014

I need @valeriangalliat on this. He is the author of our theming framework.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 9, 2014

Member

As long as you don't spoil him. :P

Member

HugoGiraudel commented Sep 9, 2014

As long as you don't spoil him. :P

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
@valeriangalliat

valeriangalliat Sep 9, 2014

Member

As you said there's two parts in the plugin system:

  1. Handle new annotations to overload the data interface.
  2. Handle overloaded data interface in the theme.

It's hard to support both plugins and themes; a theme would support the base data interface, but every theme would need to be modified to support a SassDoc plugin. We can't make it all dynamic without enforcing a common structure for SassDoc themes (like using hooks in some places where a plugin could add some HTML without being aware of the theme, but this can mess with the theme's CSS). Then we'll need to let a theme overload a module templates and styles like in Prestashop, and it soon becomes a mess. :/

I don't know what the best:

  1. Enforce a common structure (maybe a hooks system) in themes to allow any plugin to inject HTML at some strategic places in a theme (but to make theme creators happy, we need to allow them to override a module view (that could be in any template engine, or even without a template engine at all)).
  2. Say that a SassDoc plugin can add stuff in the raw data, but to display it you need to write a custom theme and do what you want with the modified data.
Member

valeriangalliat commented Sep 9, 2014

As you said there's two parts in the plugin system:

  1. Handle new annotations to overload the data interface.
  2. Handle overloaded data interface in the theme.

It's hard to support both plugins and themes; a theme would support the base data interface, but every theme would need to be modified to support a SassDoc plugin. We can't make it all dynamic without enforcing a common structure for SassDoc themes (like using hooks in some places where a plugin could add some HTML without being aware of the theme, but this can mess with the theme's CSS). Then we'll need to let a theme overload a module templates and styles like in Prestashop, and it soon becomes a mess. :/

I don't know what the best:

  1. Enforce a common structure (maybe a hooks system) in themes to allow any plugin to inject HTML at some strategic places in a theme (but to make theme creators happy, we need to allow them to override a module view (that could be in any template engine, or even without a template engine at all)).
  2. Say that a SassDoc plugin can add stuff in the raw data, but to display it you need to write a custom theme and do what you want with the modified data.
@FWeinb

This comment has been minimized.

Show comment
Hide comment
@FWeinb

FWeinb Sep 9, 2014

Member

I know what you are saying. But we a plugin without a custom theme is useless. And injecting HTML sounds really bad. We need to have a Theme system that augments everything from context creating to rendering.
I will think of something.

Member

FWeinb commented Sep 9, 2014

I know what you are saying. But we a plugin without a custom theme is useless. And injecting HTML sounds really bad. We need to have a Theme system that augments everything from context creating to rendering.
I will think of something.

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
@valeriangalliat

valeriangalliat Sep 9, 2014

Member

Yep, so we agree that a plugin will be strongly tied to a custom theme?

But I don't like packaging the parser plugin in the theme. That's two totally different things. While we can package it where we cant, I'd like to make it possible to make parser-only plugins.

For example, there could be a generic tutorial annotation plugin, that adds some stuff in SassDoc data object, then any compliant theme could use it.

On the other hand, the tutorial generic plugin could be a dependency of the theme, which would just plug tutorial in the parser instance. Now I'm saying it, I realize I like more this second solution.

But, including a parser plugin in the theme somehow violates the abstract theme interface. We're not only exposing a function taking a directory, a context, and returning a promise.

Maybe the cleanest approach would be to extend the theme interface with a special SassDoc addition, something like module.exports.plugins, which would be a function, an object, an array or whatever is the most suitable. Before rendering the theme, SassDoc could search for this addition in the theme function, and plug it to the parser before rendering the theme.

And in the case of a generic plugin, assuming the plugins export is an array, module.exports.plugins = [require('sassdoc-plugin-tutorial')] would suffice.

Member

valeriangalliat commented Sep 9, 2014

Yep, so we agree that a plugin will be strongly tied to a custom theme?

But I don't like packaging the parser plugin in the theme. That's two totally different things. While we can package it where we cant, I'd like to make it possible to make parser-only plugins.

For example, there could be a generic tutorial annotation plugin, that adds some stuff in SassDoc data object, then any compliant theme could use it.

On the other hand, the tutorial generic plugin could be a dependency of the theme, which would just plug tutorial in the parser instance. Now I'm saying it, I realize I like more this second solution.

But, including a parser plugin in the theme somehow violates the abstract theme interface. We're not only exposing a function taking a directory, a context, and returning a promise.

Maybe the cleanest approach would be to extend the theme interface with a special SassDoc addition, something like module.exports.plugins, which would be a function, an object, an array or whatever is the most suitable. Before rendering the theme, SassDoc could search for this addition in the theme function, and plug it to the parser before rendering the theme.

And in the case of a generic plugin, assuming the plugins export is an array, module.exports.plugins = [require('sassdoc-plugin-tutorial')] would suffice.

@FWeinb

This comment has been minimized.

Show comment
Hide comment
@FWeinb

FWeinb Sep 20, 2014

Member

Plugins should be theme agnostic but can only be added by a theme. So we can have generic annotation plugins but a theme must implement the rendering for it.

I just imagen a theme can do require('sassdoc-plugin-tutorial') and expose it via a unified way to SassDoc. SassDoc than will add this plugin to the annotations object.

Member

FWeinb commented Sep 20, 2014

Plugins should be theme agnostic but can only be added by a theme. So we can have generic annotation plugins but a theme must implement the rendering for it.

I just imagen a theme can do require('sassdoc-plugin-tutorial') and expose it via a unified way to SassDoc. SassDoc than will add this plugin to the annotations object.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 20, 2014

Member

1.7.0 is out. Let's get this started. @valeriangalliat, you're in charge. ;)

Member

HugoGiraudel commented Sep 20, 2014

1.7.0 is out. Let's get this started. @valeriangalliat, you're in charge. ;)

@pascalduez

This comment has been minimized.

Show comment
Hide comment
@pascalduez

pascalduez Sep 20, 2014

Member

Side note: the generator will make it easy to add plugins to a theme, with prompts.
See how everything fit nicely together ? :)

Member

pascalduez commented Sep 20, 2014

Side note: the generator will make it easy to add plugins to a theme, with prompts.
See how everything fit nicely together ? :)

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
@valeriangalliat

valeriangalliat Sep 21, 2014

Member

We need to expose "some data" from the theme to allow SassDoc adding the theme annotations to the parser (so we agree than by "build plugins" we only mean "plug annotations in the parser").

This is fairly easy: we can just add module.exports.{{name}} = {{data}} in the theme's index.js.

The hardest parts are:

  1. What {{name}} do we choose for the above? plugins, extensions, annotations, plural or not? (Damn, they were right about "naming things".)
  2. What {{data}} structure is the most suitable? It depends on how we plug annotations in the parser.

Then, it's just a step to add in documentize: check if {{name}} in config.theme, then iterate through the data structure (or whatever is needed) to plug the annotations in the parser.

Member

valeriangalliat commented Sep 21, 2014

We need to expose "some data" from the theme to allow SassDoc adding the theme annotations to the parser (so we agree than by "build plugins" we only mean "plug annotations in the parser").

This is fairly easy: we can just add module.exports.{{name}} = {{data}} in the theme's index.js.

The hardest parts are:

  1. What {{name}} do we choose for the above? plugins, extensions, annotations, plural or not? (Damn, they were right about "naming things".)
  2. What {{data}} structure is the most suitable? It depends on how we plug annotations in the parser.

Then, it's just a step to add in documentize: check if {{name}} in config.theme, then iterate through the data structure (or whatever is needed) to plug the annotations in the parser.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 21, 2014

Member

Plural question aside, plugin is great yet it's kind of loosy; extension is very loosy so I'm not in favor of this name; annotation makes perfect sense since it's basically extra annotations.

What do you think?

Member

HugoGiraudel commented Sep 21, 2014

Plural question aside, plugin is great yet it's kind of loosy; extension is very loosy so I'm not in favor of this name; annotation makes perfect sense since it's basically extra annotations.

What do you think?

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
@valeriangalliat

valeriangalliat Sep 21, 2014

Member

I agree. But if the exposed data structure is an array, i'm more in favor of annotations.

And I guess the best data structure would be an array of objects like src/annotation/annotations/*.

Member

valeriangalliat commented Sep 21, 2014

I agree. But if the exposed data structure is an array, i'm more in favor of annotations.

And I guess the best data structure would be an array of objects like src/annotation/annotations/*.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 21, 2014

Member

Makes sense. Agreed.

Member

HugoGiraudel commented Sep 21, 2014

Makes sense. Agreed.

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
Member

valeriangalliat commented Sep 21, 2014

Added a wiki page.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 25, 2014

Member

What if someone comes up with a custom annotation that needs Markdown? Can he extend sassdoc-filter as well?

Member

HugoGiraudel commented Sep 25, 2014

What if someone comes up with a custom annotation that needs Markdown? Can he extend sassdoc-filter as well?

@valeriangalliat

This comment has been minimized.

Show comment
Hide comment
@valeriangalliat

valeriangalliat Sep 25, 2014

Member

Since there's already a custom theme, it's easier to just parse the custom annotation as Markdown in the theme function.

Member

valeriangalliat commented Sep 25, 2014

Since there's already a custom theme, it's easier to just parse the custom annotation as Markdown in the theme function.

@FWeinb

This comment has been minimized.

Show comment
Hide comment
@FWeinb

FWeinb Sep 25, 2014

Member

Exactly. Annotations are theme specific so there should be no problem.

Member

FWeinb commented Sep 25, 2014

Exactly. Annotations are theme specific so there should be no problem.

@HugoGiraudel

This comment has been minimized.

Show comment
Hide comment
@HugoGiraudel

HugoGiraudel Sep 25, 2014

Member

Okay.

Member

HugoGiraudel commented Sep 25, 2014

Okay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment