New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic Dependencies #739

Open
chriseppstein opened this Issue May 8, 2013 · 123 comments

Comments

Projects
None yet
@chriseppstein
Copy link
Member

chriseppstein commented May 8, 2013

There's a way to compute dependencies efficiently and reliably on a first pass compile that would allow for the second pass to be optimized without needing to statically analyze the sass files for their @import dependencies. This would also allow for generic dependencies on things like images when used with compass sprites, or image-dimension helpers.

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 8, 2013

Doing this will allow us to remove the limitations on having no scripting in @import directives and allow @import to be used in mixins and conditionals.

See #408

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 10, 2013

...what is this way of which you speak?

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 10, 2013

I'll update with details next week :)

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 20, 2013

The only reason Sass needs to track dependencies is to answer the question "Should this sass file be re-compiled?". We use the list of imports to decide whether the file is out of date by checking its timestamp as well as the timestamp of the all transitive dependencies.

However, a change to a sass file is not the only reason why a stylesheet should need to be re-compiled. For instance, the compass image-width and image-height helper function return the dimensions of an image. If that image changes, the stylesheet is now potentially out of date and should be recompiled.

So my idea is to build a dependency graph during compilation of all the files that a sass file depends on. On first compile, we incur a compilation penalty if this dependency graph is not cached. But in most of these cases, it will be true that the css file itself is not present[^1]. When a new dependency is added, this is necessarily caused by a change to tracked dependency[^2] or the file itself, so we can still answer the question "is this file out of date".

So what I would do is introduce a Ruby API for declaring a dependency. This would be called when a file is imported, but other APIs like compass's image helpers could piggy back on it as well. Obviously, this should not be filesystem centric so that it will continue to work with Importer and Cache implementations.

I believe that taking this approach will make sass more responsive (because a dependency check still requires a parse) and it will make it more flexible because the dependency can be declared at runtime. Lastly, we will be able to do dependency invalidation as a single O(n) scan of all known dependencies and then work backwards to the invalidated sass files that need to be recompiled for an update -- currently our dependency code is O(n) in the number of imports instead of the number of dependencies.

[^1]: The exception to this being when css files are checked into source control. As such, we should store the dependency graph as it's own file that can also be checked into source control for people who do this.
[^2]: It would still be possible to create dependencies that change without a change to a source file if you do something crazy like using the current date to generate an import. In such a case, a custom importer would need to be defined that knows how to mark such imports as invalid.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 31, 2013

This is pretty complicated, and I'm not sure dynamic dependencies give us enough benefit to be worth it, but I do buy that it might be possible.

I'm very curious what the Ruby APIs you're talking about would look like.

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 31, 2013

Dynamic dependencies would support a number of very interesting and useful approaches to code organization. Consider theming. If you have theme specific css files, currently everything must be imported and you can disable some of the output that belongs to other themes. A much simpler approach is to only import the theme files specified by a configuration variable.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 31, 2013

I'm not convinced it's a bad thing to import everything you might need and then dynamically decide between them. That's how most programming languages work.

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 31, 2013

Programming languages don't do things if you don't call specific subroutines. selectors automatically output if you import them. This is the big difference between a programming language and a templating system -- sass is much closer to the latter than the former.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 31, 2013

Your library files shouldn't be defining top-level CSS. They should be defining mixins or placeholders that importers can selectively use.

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 31, 2013

I wasn't talking about library files.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 31, 2013

Then what are you talking about? Someone's local file filled with rules? If they want to conditionally use those rules, they should put them in a mixin.

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 31, 2013

@nex3 Mixins are good for small blocks of code meant to be re-used. A theme is a large block of code potentially spanning several files. Dynamic imports are an ideal way of handling this. Especially if you want to provide a code separation between the authoring of the core design and the theme.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 31, 2013

Imports are fundamentally clunky. They're tightly coupled to the physical filesystem in most cases, they take strings in a DSL that Sass has no means of manipulating (and I'm categorically opposed to adding path/URI manipulation functions). This is fine when they're authored by hand and refer to well-known physical files (or occasionally well-known generated files), but once dynamic code starts touching them I expect it to become very painful and confusing to work with.

Even if the theme is written to emit selectors directly, why not do %dark-theme { @import "dark-theme" }?

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 31, 2013

I have no desire to expose the current working directory to sass files. I think the load path is a sufficient abstraction.

Please explain the use cases you have concerns about.

Using a placeholder like you've suggested adds unnecessary specificity and bloat when it's extended.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented May 31, 2013

Please explain the use cases you have concerns about.

I don't have specific examples, just a general feeling that adding a feature that motivates people to want to manipulate paths in Sass is a bad idea.

Using a placeholder like you've suggested adds unnecessary specificity and bloat when it's extended.

I'd rather add support for top-level @extend than dynamic dependencies, given the choice.

@chriseppstein

This comment has been minimized.

Copy link
Member Author

chriseppstein commented May 31, 2013

People intuitively get dynamic imports. They ask for this feature all the time and the reason we have said no is dependency calculation. The dependency calculation argument is solvable given the algorithm above. I have no idea what a "top level extend" is and I don't think we should have to educate people on a new concept when a simple one will suffice. I don't buy the slippery-slope argument. As I said, load paths are sufficiently abstract, we don't need path manipulation and that is a line that I'm willing to hold.

@lifeiscontent

This comment has been minimized.

Copy link

lifeiscontent commented Feb 12, 2014

@chriseppstein @nex3 any updates here?

@jslegers

This comment has been minimized.

Copy link

jslegers commented Mar 28, 2014

@nex3 : In #779 you mentioned that you were planning on deprecating the @import feature, which is used in pretty much every Sass implementation of moderate to high complexity.

Why not just extend the possibilities of the @import feature by allowing it to be used within control directives and/or mixins? This is how PHP's include ( http://www.php.net/manual/en/function.include.php ) is implemented. It allows dynamic loading, greater modularity, easy polyfilling and many more features without having to change or deprecate the @import syntax.

Implementing the possibility of dynamic dependencies by allowing @import statements within control directives or mixins has one main advantage: Sass users would not notice any difference between "old school" dependency management and "new school" dependency management.

While the underlying mechanism may have to be altered significantly or even rewritten completely, the syntax for the "end user" would remain completely the same. No new syntax would have to be learnt and old @import statements would not have to be changed to be compatible with new versions of Sass.

Examples :

@if not function-exists(str-replace) {
  @import '_str-replace';
}

@if $sassversion > 3.2 {
  @import '_new-sass';
} @else {
  @import '_old-sass';
}

@mixin do_magic($option1 : false, $option2 : false) {
  @if $option1 {
    @import '_option1';
  }
  @if $option2 {
    @import '_option2';
  }
  @if $option1 and $option2 {
    @import '_bothoptions';
  }
}

See also #1194, where I posted this request as a separate issue.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented Apr 4, 2014

We're definitely doing a major overhaul of @import. The current system is busted for any number of reasons. Given that, there's no real benefit in maintaining compatibility with it.

@jslegers

This comment has been minimized.

Copy link

jslegers commented Apr 4, 2014

@nex3 :

There's always a reason to maintain compatibility : stopping old code from breaking when you upgrade to a new version of Sass.

This kind of crazy decisions makes me want to reconsider adopting Sass and just stick with plain CSS... or write my own Sass equivalent.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented Apr 4, 2014

We're going to have a deprecation process; we're not going to blithely break every existing Sass file. The old @import syntax will continue working for quite a while, likely a full major version cycle. But we need to have the flexibility to design for five years in the future, not just one year in the future, and sometimes that means removing functionality that once worked.

@Firanolfind

This comment has been minimized.

Copy link

Firanolfind commented Aug 2, 2017

@jamiebarrow I am not familiar with Ruby, but challenge accepted.

@jlove73071

This comment has been minimized.

Copy link

jlove73071 commented Aug 2, 2017

For performance's sake it seems like it'd be nice to do a "single import" -- so that importing 1 file would promise not to import other files... That'd be a HUGE help for me anyway.

@dhershman1

This comment has been minimized.

Copy link

dhershman1 commented Aug 24, 2017

My Node module Sass-Pack Supports Alias for import paths, it's an early feature support so the more people hammering it and feeding me bugs to improve/perfect it the better (Besides my own changes to it). If anyone is interested.

https://github.com/dhershman1/sass-pack
https://www.npmjs.com/package/sass-pack

@danyalaytekin

This comment has been minimized.

Copy link

danyalaytekin commented Nov 3, 2017

Bitcoin was $130 when this issue was opened. I wish I'd known.

@sass sass deleted a comment from landsman Nov 3, 2017

@sass sass deleted a comment from dominique-mueller Nov 3, 2017

@hrsetyono

This comment has been minimized.

Copy link

hrsetyono commented Jan 12, 2018

Any news regarding this? I've been using 2 year old version of node-sass (3.4.2) which luckily "bugged" to allow dynamic dependency. But they patched it on 3.5.1

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented Jan 12, 2018

Once there's news, it'll be visible here. For now, there's a lot on our plates, and this hasn't been a priority.

@jslegers

This comment has been minimized.

Copy link

jslegers commented Jan 13, 2018

For now, there's a lot on our plates, and this hasn't been a priority.

This should have been top priority years ago! I've literally been waiting for this feature for 4 years now. If we, at my employer's had so many of our customers wait for that long for such an important feature, I'm pretty sure we'd have gone out of business a long time ago.

And yes, this is an important feature, when you consider how much more flexible this makes the language and how many people have been asking for this feature since 2013, both here and elsewhere.

Less has had dynamic imports for years now. Then again, they never even bothered to implement something as fundamental as an IF-ELSE statement or a FOR loop, which is the main reason I haven't switched to Less yet, along with their cringe-worthy syntax.

You preprocessor devs have a really, really, réally odd sense of priorities!

@SaltyDH

This comment has been minimized.

Copy link

SaltyDH commented Jan 13, 2018

Maybe instead of waiting for four years, you could have spent that time learning the necessary skills to submit a pull request for the feature. Your priorities are not the only priorities. I'd love this feature too as it simplifies our style sheets and removes a lot of boiler plate code but I don't think this type of response is going to get you the faster feature release you're looking for...

@jslegers

This comment has been minimized.

Copy link

jslegers commented Jan 13, 2018

Maybe instead of waiting for four years, you could have spent that time learning the necessary skills to submit a pull request for the feature.

During the past 10 years, I've been working mostly in frontend (HTML/JS/CSS) and PHP backend environments, as well as ABAP-based (SAP) and a Java based ERP-environments. And I'm currently learning C++ and OpenGL, since a part of the GIS visualization framework I do R&D for is written in C++ (compiled to JavaScript with Emscripten).

Ruby isn't a language I ever needed professionally, nor is it a language I care much for privately. I've only ever used it to run Sass and some obscure legacy build scripts. Are you seriously suggesting I should have learned Ruby just to be able to add a feature to Sass that should have been added by the core devs years ago?

I'd love this feature too as it simplifies our style sheets and removes a lot of boiler plate code but I don't think this type of response is going to get you the faster feature release you're looking for...

I kinda gave up on this. I'm tired of waiting for people who clearly don't care about what their users want and need.

Professionally, I don't really need SCSS at the moment anyway. And privately, I've put any development on Cascade Framework 2 in the fridge indefinitely and moved on to other projects that don't depend on SCSS.

@SaltyDH

This comment has been minimized.

Copy link

SaltyDH commented Jan 13, 2018

I'm suggesting that instead of leaving demotivating comments on an open source project, you could choose to contribute to a solution rather than make demands.

@jslegers

This comment has been minimized.

Copy link

jslegers commented Jan 13, 2018

I'm suggesting that instead of leaving demotivating comments on an open source project, you could choose to contribute to a solution rather than make demands.

I already spent many hours trying to explain why this feature is so important 4 years ago, and again 3 years later. I tried to ask nicely (I realize that my social skills may be somewhat lacking in that area, though). And I tried to come up with as many use cases as possible.

After 4 years of waiting and many others requesting the same feature, it's become clear to me that asking nicely doesn't seem to motivate the core devs at all. Nor does coming up with a whole bunch of valid use cases. So maybe rants fueled by frustration are more productive? I don't know what to do when people decide to just ignore you.

Either way, I don't have the time to learn Ruby and create my own pull request for this. And if I would, I'd rather invest that time in creating my own preprocessor language. I've actually seriously considered that a few years ago, when I had far more spare time than I have right now...

If you can think of more productive ways to contribute here, go ahead. I tried and failed a long time ago!

@ArmorDarks

This comment has been minimized.

Copy link

ArmorDarks commented Jan 13, 2018

Either way, I don't have the time to learn Ruby and create my own pull request for this. And if I would, I'd rather invest that time in creating my own preprocessor language. I've actually seriously considered that a few years ago, when I had far more spare time than I have right now...

Reminds me of https://www.youtube.com/watch?v=e35AQK014tI

I've already stated my own position regarding this issue above. I'm not sure that issue valuable at all, and it doesn't solve your problem too.

In many languages imports are static, and there are serious performance reasons for it. And when you use non-static version, like import().then() in ES6 modules or require() in CommonJS, it hits performance hard.

But despite being static, somehow such imports still works magnificently in other languages, but not in Sass. The difference is the ability to declare abstraction and be explicit about what is exported and what is not. Everything else is built upon it. If you're familiar with JS, Java and C++, it shouldn't come to you as a surprise.

So, what we really need is a better ways for handling dependencies, ways to implement modules loading and, better ways to express abstractions and export them. But I'm not even sure that it should be solved by Sass itself. Think of JS or other languages, where that whole process is handled by other tools, like CommonJS, NPM and so on.

Though, I agree that Sass should provide a better foundation for such implementations, which isn't what happens right now (for instance, see #353). But it isn't related to this issue at all.

@jslegers

This comment has been minimized.

Copy link

jslegers commented Jan 13, 2018

I'm not sure that issue valuable at all, and it doesn't solve your problem too.

The Less documentation mentions this example to demonstrate the use of dynamic imports in Less :

// Variables
@themes: "../../src/themes";

// Usage
@import "@{themes}/tidal-wave.less";

This is probably how >90% would use dynamic imports in SCSS as well.

Now, combine this with allowing @import statements within control directives and mixins, and you have all that is needed to make SCSS far more dynamic & flexible than it currently is.

It would allow for easy polyfilling...

@if not function-exists(str-replace) {
  @import '_str-replace';
}

... for easy progressive enhancement :

@if $sassversion > 3.2 {
  @import '_new-sass';
} @else {
  @import '_old-sass';
}

... for easy config-based addition or removal of components :

@mixin do_magic($option1 : false, $option2 : false) {
  @if $option1 {
    @import '_option1';
  }
  @if $option2 {
    @import '_option2';
  }
  @if $option1 and $option2 {
    @import '_bothoptions';
  }
}

... and a lot more things that are currently not possible in a remotely efficient way!

It probably won't solve all of my headaches, but it sure would solve many - if not most - of them.

In many languages imports are static, and there are serious performance reasons for it.

This is one of the reasons PHP was my language of preference when PSR-2 was the prevailing coding standard.

Not only does PHP's include statement allow dynamic imports, but back in those days you could just drag-and-drop any PHP library that was PSR-0 compatible (which every major library was) in a path on your OS corresponding with that library's namespace and a pretty basic autoloader would automatically make every class in that folder available - thanks to PHP's [autoloading capabilities](http://php.net/manual/en/language.oop5.autoload.php - with a negligible performance drawback. No additional config whatsoever was needed. Nor did you need a package manager (like NPM or composer).

♪ And then they came and spoil it all ♪... by doing something stupid like replacing PSR-2 with the backwards incompatible PSR-4... which made configuration-free autoloading impossible and Composer a necessary evil for every modern PHP project.

This is one of several reasons I moved away from PHP.

The difference is the ability to declare abstraction and be explicit about what is exported and what is not. Everything else is built upon it. If you're familiar with JS, Java and C++, it shouldn't come to you as a surprise.

One thing I recently did in a legacy JavaScript project, is replacing Dojo's AMD loader and its plugins with a RequireJS based equivalent in a way that changing a single property of the AMD config allows customers to painlessly move back to Dojo if they want to.

This same project also uses a custom AMD plugin (compatible with both Dojo & RequireJS) that allows for a different AMD module to be loaded depending on a parameter in the URL query.

This is the kind of flexibility I'm looking for in my SCSS projects, and the lack of imports is the main stumbling block for this.

@hrsetyono

This comment has been minimized.

Copy link

hrsetyono commented Jan 15, 2018

@jslegers I recommend using old version of node-sass like I mentioned slightly above. It has done me well for 3 years.

It's a much faster compiler compared to ruby too.

@ArmorDarks

This comment has been minimized.

Copy link

ArmorDarks commented Jan 16, 2018

@jslegers Doing things most obvious way isn't always the right way.

... and a lot more things that are currently not possible in a remotely efficient way!

For all your cases conditional importing isn't a solution. I've already described solution above — encapsulate each your file into mixin and import all of them. Then, conditionally invoke only needed mixins.

Performance wise that would be faster working than conditional importing (if conditional importing would be implemented) because non-static imports will prevent lib from building dependency tree and thus caching imports.

Just to clarify, this isn't a "workaround". It is how it should be done. At first, in imports, you're declaring all your app dependencies, and then using only needed ones. This way a more reliable architecture provided where you can be sure, that everything that Sass styles need indeed exists.

@landsman

This comment has been minimized.

Copy link

landsman commented Jan 16, 2018

Hi there, is here some solution please?
It's annoying! So small problem and how the long topic....

@jslegers

This comment has been minimized.

Copy link

jslegers commented Jan 16, 2018

@hrsetyono :

I recommend using old version of node-sass like I mentioned slightly above. It has done me well for 3 years.

Not a bad idea if you're building a website.

But if you're creating a library or a framework, this isn't an option IMO, as you can't rely people to use a specific version Sass just to be able to exploit a bug.


@ArmorDarks :

Doing things most obvious way isn't always the right way.

No shit Sherlock!

Just to clarify, this isn't a "workaround". It is how it should be done. At first, in imports, you're declaring all your app dependencies, and then using only needed ones. This way a more reliable architecture provided where you can be sure, that everything that Sass styles need indeed exists.

Let's just agree to disagree here.


@landsman :

Hi there, is here some solution please?
It's annoying! So small problem and how the long topic....

There is no solution.
Nor is it likely there will be a solution in 2018.
I've been waiting for 4 years now, and they still haven't demonstrated any intention to implement this feature anytime soon.
Sass's core devs clearly don't care.

@ArmorDarks

This comment has been minimized.

Copy link

ArmorDarks commented Jan 17, 2018

@landsman :

Hi there, is here some solution please?
It's annoying! So small problem and how the long topic....

Depends on what you're looking for.

If you want to make conditional imports within @if directives or mixins, there is another approach. See my comment above #739 (comment)

If you want to use variables inside import paths, it is better to go with node-sass and it's importer option, which allows customizing @import directive handling. There, by doing some very basic string substitution or using templating, like, say, lodash.template it is possible to achieve the desired result.

And then you could pass in global variables and later use them in imports like so:

@import '<%= someVar %>/styles.scss';

But I'd recommend thinking twice before using variables in imports. In many cases what you really want is to pass few paths to includePaths option and allow Sass handle resolving.

@hrsetyono

This comment has been minimized.

Copy link

hrsetyono commented Jan 18, 2018

@jslegers

Not a bad idea if you're building a website. But if you're creating a library or a framework, this isn't an option IMO, as you can't rely people to use a specific version Sass just to be able to exploit a bug.

True, I do use it on framework but only for internal use so I can ask my team to use the old version.

@ArmorDarks

For all your cases conditional importing isn't a solution. I've already described solution above — encapsulate each your file into mixin and import all of them. Then, conditionally invoke only needed mixins.

I understand what you mean, but I'm using it for internal framework. Each file has lots of mixins, some even has the same name

For example I have v1/_grid.scss and v2/_grid.scss. Those are loaded conditionally depending on the $version variable. This is to make sure the framework styling doesn't change when we modify old project.

@ArmorDarks

This comment has been minimized.

Copy link

ArmorDarks commented Jan 18, 2018

For example I have v1/_grid.scss and v2/_grid.scss. Those are loaded conditionally depending on the $version variable. This is to make sure the framework styling doesn't change when we modify old project.

I see what you mean. We can look into the experience of more complete languages to see how that case handled there — JavaScript, Ruby etc.

There is a package semantic versioning to distribute changes in a predictable way. Thus, you will never encounter v1/... and v2/... dirs. Want older version? Just load older framework version. Mixin concept of package (framework) version with internal versions will hit badly predictability of library work. Besides, since your library already relies on global $version variable, it is no different from simply distributing needed modules versions with specific framework versions and not using $version and conditional loading at all.

In other words, what was v1/_grid.scss will become _grid.scss, a part of the older framework, distributed as package 1.0.0, while v2/_grid.scss will become same _grid.scss, but a part of package 2.0.0.

You can refer to this example.

@hrsetyono

This comment has been minimized.

Copy link

hrsetyono commented Jan 19, 2018

@ArmorDarks

I prefer having global npm since I only use it for compiling Sass and Yeoman generator. I mostly work with WordPress or ReactJS using WP API.

I know this is not a recommended way of working with npm, but it feels simpler not to have node_modules/ directory inside each project.

The $version variable is defined inside each project. So we have 1 global framework for all projects. Currently we're working on v3 that uses the new CSS3 Grid.

@nex3

This comment has been minimized.

Copy link
Contributor

nex3 commented Jan 19, 2018

I'm locking this issue for now because there's a lot of noise without a lot of value being added. To summarize, this is the plan:

  • The new @use directive will provide the ability to import a file as a mixin, so you can dynamically decide whether and where to include it. This will bring Sass more in line with other languages that work well without dynamic imports, since it means importing no longer has unavoidable side-effects.

  • We will add a load() function as described above that will allow stylesheets to load files at runtime based on variable values. This will support the more complex use-cases where stylesheets need to be loaded based on user input, while preserving the ability to statically trace the import graph and the mixins and functions it defines.

@sass sass locked as too heated and limited conversation to collaborators Jan 19, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.