Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ship a built-in package importer #2739

Open
nex3 opened this issue Aug 30, 2019 · 12 comments

Comments

@nex3
Copy link
Contributor

commented Aug 30, 2019

There's currently no standard way for Sass users to import packages from dependencies. There are a number of domain-specific solutions, such as Webpack's tilde-based imports, npm-sass's load-path-like functionality, or Dart Sass's support for Dart's package: URLs.

Most of these have the disadvantage of needing to be explicitly opted-into, and all of them make it difficult to share stylesheets across different contexts. For example, if package A depends on package B, how can A import B in a way that will work in both Webpack and in Dart?

This is the kind of situation where centralization is a boon. If we can build into the Sass language a notion of a "package import" that's flexible enough to work across contexts, we can make it usable by all stylesheets with confidence that even if they're used by different build runners or even ported to a different package manager their dependencies will continue to work.

What does it look like?

The Sass specification talks about imports in terms of URLs. The current JS API deals with them as a combination of raw strings and filesystem paths, but I'd like to move away from that as part of #2509.

The current most popular solution for package imports is probably sass-loader's, which passes any import that begins with ~ through Webpack's built-in resolution infrastructure. We could re-use this syntax, but it doesn't work well with URL semantics. A string beginning with a tilde is syntactically a relative URL, which means we'd need to check for the relative path ~package first before passing the URL to the package importer.

We'd have a similar problem if we automatically added node_modules to the load path. Every instance of @use "bootstrap" would need to check the relative path first, as well as potentially every load path, before checking the package importer. It also makes package stylesheets less visually distinctive, which can be confusing for readers.

As such, I propose that we use the URL scheme pkg: to indicate Sass package imports, so users would write @use "pkg:bootstrap". This doesn't conflict with any existing syntax and so producing backwards-compatibility headaches, and it nicely mirrors the syntax of Sass's core libraries (@use "sass:color").

What does it do?

The purpose of a standard package importer is in fact to avoid specifying the exact behavior of the importer, so that it can do something sensible for each context. However, since Node.js is by far the most popular context in which Sass is used, we should figure out what it does in that particular case.

When resolving a URL, I think it should check node_modules folders as described in Node's documentation, beginning from the parent directory of the module in which Sass is invoked (if it's invoked as a library from JS) or the parent directory of the entrypoint file (if Sass is invoked via the command line).

There's a convention of npm packages declaring their Sass entrypoints using "style" or "sass" keys in their package.json. While it's definitely useful to be able to write @import "bootstrap" rather than @import "bootstrap/scss/bootstrap", I actually think we shouldn't support this. Instead, I think we should encourage packages to define an _index.scss file that acts as the entrypoint. This will ensure that even if the package ends up on the load path, or installed through some way other than node_modules, it can still be imported correctly without needing to parse package.json.

@nex3

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2019

@evilebottnawi I'm particularly interested in your thoughts on this, as the sass-loader maintainer.

@MartijnCuppens

This comment has been minimized.

Copy link

commented Aug 30, 2019

I personally like the tilde approach Webpack offers, but reading the issues it introduces, @use "pkg:bootstrap" looks like a fine solution.

I think we should encourage packages to define an _index.scss file that acts as the entrypoint.

I'm personally ok with that.

As a Bootstrap maintainer I can confirm we get some questions from people asking if this can be simplified.

@evilebottnawi

This comment has been minimized.

Copy link

commented Aug 30, 2019

In webpack we will keep ~ resolution for compatibility.

I agree with @MartijnCuppens

Good idea

@lucpotage

This comment has been minimized.

Copy link

commented Aug 30, 2019

I like the idea of an entrypoint but I prefer index.scss over _index.scss. Is the underscore something usual in Dart?

For package imports, I agree the approach @import "bootstrap" is too generic. You way want to use this name for global JS lib with components that rely on stylesheets you may want to deliver as a different package. I see two approaches:

  • @import "sass-lemon" in case you have a global framework and release the sass part as a separate package. This naming convention is common in some ecosystems like vue-*. This way, you clearly identify from npm or code samples you are going to use a sass lib.
  • @import "lemon/sass" in case you have a package with multiple outputs. In this case this would resolve lemon/sass/index.scss, like with JS files. Even though less and stylus are minor, the author could have a lemon/framework.styl and lemon/framework.less as well.

As for the style and sass keys in package.json, why not using the main key that is also common (maybe in JS only)?

As for the tilde, I really find it convenient if you are authoring the lib with something like @import "~/mixins/media" wherever in your lib. It's used in Nuxt.js and I find it very useful.

@Vlasterx

This comment has been minimized.

Copy link

commented Aug 30, 2019

We would need to have similar functionality as we have for standard JS packages. I want to say right form the start that this new importer shouldn't rely on Webpack on any other similar library. It should have the possibility to work as standalone only for sass files, because there will be projects where we don't want to use anything related to webpack or javascript.

This is my proposal through examples:

File structure of our package

Lets say that This can be the structure of our package:

node_modules/

style/
  mixins/
  functions/
  styles/
  _index.scss

src/
  index.js

.gitignore
LICENCE
package.json
README.md

package.json

When creating SCSS package, it would be wise to define entry point for scss imports. For example, if this is the standard way of defining JS entry point:

{
  "name": "example-scss",
  "main": "src/index.js",
  ...
}

Then it would be wise to use similar approach for SCSS in the same file

{
  "name": "example-scss",
  "main": "src/index.js",
  "style": "style/_index.scss",
  ...
}

File style/_index.scss should contain all other imports required for this package to work. With this setup we should be able to use this functionality for any file format (scss, sass, css etc).

When using the NPM package

I like the idea that we would be able to import packages with @use "pkg:example-scss" in our own style sheets.

Lets say that this is the file structure of our project:

node_modules/
  example-scss/
  ...

styles/
  mixins/
    _myMixins.scss
  components/
    example/
      _example.scss
  style.scss

package.json

In the file _example.scss we would be able to use

@use "pkg:example-scss"

// Rest of our styles...

Importer for @use "pkg: should be able to calculate the relative depth from the file where we are using it towards the root of the project, so that the import from node_modules can work correctly. Problem with node projects is that it is difficult to find the project root, so we start calculating that relative directory depth in some other way. There is one very useful package that I use for similar tasks, it is called app-root-path. It might be good to take a look at it to see how it works.

Basically node-sass package should be able to parse these @use imports and calculate the relative directory depth on its own for node_modules -> random directory depth -> file where we import the package.

@nex3

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2019

@evilebottnawi

In webpack we will keep ~ resolution for compatibility.

Backwards-compatibility is definitely important 😃. Would you consider deprecating this syntax some time in the medium-term future, once both Dart Sass and Node Sass have supported pkg: imports for a while?


@lucpotage

I like the idea of an entrypoint but I prefer index.scss over _index.scss. Is the underscore something usual in Dart?

The underscore is the same partial indicator Sass has always had. index.scss will also work, but it makes it look like a file that's supposed to be compiled as an entrypoint rather than one that's meant to be loaded as a library.

For package imports, I agree the approach @import "bootstrap" is too generic. You way want to use this name for global JS lib with components that rely on stylesheets you may want to deliver as a different package. I see two approaches:

  • @import "sass-lemon" in case you have a global framework and release the sass part as a separate package. This naming convention is common in some ecosystems like vue-*. This way, you clearly identify from npm or code samples you are going to use a sass lib.
  • @import "lemon/sass" in case you have a package with multiple outputs. In this case this would resolve lemon/sass/index.scss, like with JS files. Even though less and stylus are minor, the author could have a lemon/framework.styl and lemon/framework.less as well.

Both of these use-cases will work; Sass already supports index files for any directory.

As for the style and sass keys in package.json, why not using the main key that is also common (maybe in JS only)?

For the same reason that I'd like to avoid the style and sass keys, as well as the fact that it's already being used for the JS entrypoint.

As for the tilde, I really find it convenient if you are authoring the lib with something like @import "~/mixins/media" wherever in your lib. It's used in Nuxt.js and I find it very useful.

Unless I'm mistaken, this isn't valid ~ syntax... according to the sass-loader docs, "It's important to only prepend [the URL] with ~, because ~/ resolves to the home directory." This is another good example of how ~ syntax may be more confusing to new users than a custom URL scheme.


@Vlasterx

We would need to have similar functionality as we have for standard JS packages. I want to say right form the start that this new importer shouldn't rely on Webpack on any other similar library. It should have the possibility to work as standalone only for sass files, because there will be projects where we don't want to use anything related to webpack or javascript.

I agree for the most part, although I'm not sure what it would mean for this syntax to work with Sass files that have absolutely no context—how would we even start knowing how to resolve imports?

@S0AndS0

This comment has been minimized.

Copy link

commented Aug 31, 2019

While I think Node and npm are groovy for some use cases, it doesn't seem necessary for Sass. For most of my projects utilizing Sass and Liquid I use Git Submodules for dependency management. Here's an abbreviated quick start for what setting-up some-project usually looks like...


Adding a git submodule to some-project

  1. Make a common modules directory for Sass dependencies

  2. Add the https URL as a Git Submodule

cd some-project
mkdir -vp "_sass/modules"


git submodule add\
 -b master\
 --name "vendor-prefixes"\
 "https://github.com/scss-utilities/vendor-prefixes.git"\
 "_sass/modules/vendor-prefixes"
  • -b master tracks the master branch

  • --name "vendor-prefixes" defines the name used within .gitsubmodules file, and .git/modules path. In the above example that'd be .git/modules/vendor-prefixes

  • https://github.com/scss-utilities/vendor-prefixes.git is the https URL that's tracked publicly within .gitsubmodules file

  • _sass/modules/vendor-prefixes is where files are available to some-project and most file-browsers; git diff will show that it's a special file that contains the submodule's tracked hash.

Example .gitmodules file, generated by above git command...

[submodule "vendor-prefixes"]
	path = _scss/modules/vendor-prefixes
	url = https://github.com/scss-utilities/vendor-prefixes.git
	branch = master

Initial clones then look something like...

git clone --recurse-submodules <url-for-your-project>

Pulls and fetches involving updated submodules are similar to...

git pull
git submodule update --init --merge --recursive

Developers of some-project can update the version/hash of tracked submodules via...

git submodule update --init --merge --recursive --remote


git commit -F- <<'EOF'
:arrow_up: Updates tracked submodule dependencies

# ... anything else worthy of note
EOF


git push origin master

Pros

  • Developers of some-project have more control over where dependencies are available to some-project

  • Private server setup is simpler; usually only requires that Git be setup with SSH authentication, and for more advanced setups a Domain Name Server so that private remotes point to consistent locations

  • Developer setup for some-project is relatively easy; generally only requires adding a few more Git options to one's vocabulary

  • Everything is still tracked by Git! Meaning one can checkout, fetch, and all the Git things foreach module...

git submodule status

git submodule foreach git branch -vv
  • Modules can be moved in the future with git mv command; great if developers of some-project have a mind to change such things

  • Modules are still treated as Git repositories, thus commit, push, and other Git commands work as expected with just a change of current working directory...

cd _sass/modules/vendor-prefixes
git status
  • Submodules are compatible with GitHub Pages, and there are tools such as Dependabot that'll submit Pull Requests automatically for some-project if any submodules it depends upon are updated

Con

Module developers, such vendor-prefixes have zero control of where a module may be imported from, meaning shared dependencies between modules one and two can become messy...

_sass/modules
    one/lib/some-dependency
    two/lib/some-dependency

... however, I think this could be resolved via some form of @requires <name>:<relative-path> or @module <name>:<relative-path> Sass keyword that recursively scours .gitsubmodules files within some-project for <name> if it isn't already imported.

@nex3

This comment has been minimized.

Copy link
Contributor Author

commented Aug 31, 2019

Maybe a better way than explicitly setting up a bunch of mappings would be to have a --package-dir option for the Sass CLI that specifies a dead-simple package resolution mechanism: pkg:path just looks for path in the given directory. If packages are consistently using index files so that @import "path/to/package" works, that should work if you create a directory full of submodules.

@S0AndS0

This comment has been minimized.

Copy link

commented Aug 31, 2019

One of the things I really like about Sass is the low-barrier to entry, not being required to preform any extra configuration and having a write, compile, and results cycle is wonderful. Where --package-dir to accept a list of sub-directories that might be cool, though I'm not sure it would fix anything, currently nothing is broken by using submodules the ways that I do.

If I remember correctly @import _sass/modules/vendor-prefixes.scss worked when that file was loaded with it's own @import statements, while that did make things look a bit more terse within any main.scss files, I switched back to having an assets/css/main.scss file that imports all modules then any usages after. Because of various reasons... mostly it's to easier to lose a thought-thread bouncing from file-to-file without seeing real code.

Mainly what I'm getting at is that Git already provides dependency management, and that the [submodule "vendor-prefixes"] line from the example .gitmodules could denote the pkg name, and path = _scss/modules/vendor-prefixes line a pkg-path. This would allow @import _sass/modules/vendor-prefixes/calc.scss, to transmute into something like @module "vendor-prefixes:calc.scss".

The biggest issue I see with using submodules this way, are with instances of shared dependencies, but like I already stated, I also think this could be worked around. Perhaps through parsing configurations or variables within @import paths, and clear documentation on what to do in such cases.

@Vlasterx

This comment has been minimized.

Copy link

commented Aug 31, 2019

@nex3

@Vlasterx
We would need to have similar functionality as we have for standard JS packages. I want to say right form the start that this new importer shouldn't rely on Webpack on any other similar library. It should have the possibility to work as standalone only for sass files, because there will be projects where we don't want to use anything related to webpack or javascript.

I agree for the most part, although I'm not sure what it would mean for this syntax to work with Sass files that have absolutely no context—how would we even start knowing how to resolve imports?

IMHO, you don't need to worry about anything else except providing a way to import style files from node_modules. Keep it simple.

It comes down to developers to create good code on either end. With command @use "pkg:example-scss", you are only telling the node-sass module that during compiling it should import example-scss package from node_modules/example-scss/ directory and with the entry file defined under a style key from package.json.

Another example

package.json

This is package.json defined in our module example-scss

{
  "name": "example-scss",
  "main": "src/index.js",
  "style": "style/_index.scss",
  ...
}

We don't care what's in the style/_index.scss. That is for the developer to decide. But for this example lets say that it can be something as simple as this

style/_index.scss

a { color: red }

Our project style file

Lets say that this is the directory structure of our project:

node_modules/
  example-scss/
  node-sass/
  ...

src/
  styles/
    myStyle.scss

Our file for styles is located in folder /src/styles/myStyle.scss.

That file contains the following code:

@use "pkg:example-scss"

a { font-size: 1rem; }

During compiling, our main plugin node-sass will find that we are using external module named example-scss. Its task should be to calculate the relative path from our project file myStyle.scss to the project root and then load the entry style file from package example-scss. At that step, that @use "pkg:example-scss" would practically be translated as @import '../../../node_modules/example-scss/style/_index.scss'

Resulting css on the far end of compilation would be:

a {
  color: red;
  font-size: 1rem;
}
@voltaek

This comment has been minimized.

Copy link

commented Sep 16, 2019

Using a specific namespace for functionality that differs from the typical module behavior seems like it could be easily missed while reading through a file, or for newer users to immediately understand. It also would make specific syntax-highlighting for package-related verbiage require specific cases for pkg: being in a string after @use, rather than just looking for a different @ rule or function name.

My suggestion would be using something like @package or @use-pkg instead of @use "pkg:abc", to increase clarity that this is meant to be handled by a package manager. Even something like @use package("abc") would at least make the package-dependency more obvious by moving it outside of the string.

@nex3

This comment has been minimized.

Copy link
Contributor Author

commented Sep 23, 2019

I think defining a totally separate syntax for package imports that would have identical semantics to @use except for which importer it used would increase confusion. Also, it's important that the package importer works just like any other Sass importer—it should be customizable in the same way as all other importers, so that users who want to define their own scheme for loading packages are able to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.