Add `--target=lib` flag by mwistrand · Pull Request #42 · dojo/cli-build-widget

mwistrand · 2018-09-12T20:07:49Z

Resolves #35

Allow widget libraries to be built with --target=lib (or -t lib for short). With this flag, all TypeScript and CSS files beneath src/ are processed and output to output/{mode}, along with any font and/or image assets. Although this does not utilize webpack, the --legacy flag is still used to determine the CSS and TypeScript targets.

At the moment this is excluded from eject since it is entirely independent of the webpack build used to bundle custom elements, although at some point it would be worthwhile to determine the best way to handle such cases.

agubler

A few observations. Also how does this behave with being ejected?

agubler · 2018-09-13T09:17:44Z

+		postcssCustomProperties({ preserve: 'computed' })
+	]);
+
+	return globby(cssFiles).then(files => {


no await? Might be a bit clearer.

That would improve readability here.

agubler · 2018-09-13T09:18:23Z

+	}
+
+	const spinner = ora().start('Cleaning previous output');
+	return createTask((callback: any) => rimraf(outDir, callback))


All these could be awaited also perhaps?

I'll see what I can come up with, but personally I prefer the then series to a try/catch block in this particularly case.

agubler · 2018-09-13T09:20:04Z

+		postcssImport(),
+		postcssPresetEnv(postcssPresetConfig),
+		postcssModules({
+			generateScopedName: mode === 'dist' ? '[hash:base64:8]' : '[name]__[local]__[hash:base64:5]',


We have standardised on just '[name]__[local]__[hash:base64:5]' now for all modes for the other build commands

Thanks, I should have looked at what we were already using in the repo instead of what @dojo/widgets was using from @dojo/grunt-dojo2.

agubler · 2018-09-13T09:30:46Z

+	const outDir = `output/${mode}`;
+	const cssFiles = path.join(basePath, outDir, '**/*.css');
+	const packageJsonPath = path.join(basePath, 'package.json');
+	const packageJson = fs.existsSync(packageJsonPath) ? require(packageJsonPath) : {};


Should we support building the a widget lib without a package json?

agubler · 2018-09-13T11:14:39Z

Also does this mean that building to the library target doesn't support watching (and maybe other options)? How is this going to work with cli-test-intern? As that would be the command we'd want to recommend for testing a dojo widget library?

mwistrand · 2018-09-13T13:13:38Z

@agubler I will look into adding a watch option as well as other flags available for the webpack build. As mentioned in the description, this does not yet work with eject, as it is independent of the webpack build, so we'd need provide a separate way to build libraries once ejected. I have this on my to-do list, but wanted to get this up for at least an initial review.

agubler · 2018-09-13T13:21:17Z

@mwistrand I should have read the description re eject really!

mwistrand · 2018-09-13T21:02:01Z

There are still a couple minor annoyances I need to resolve around logging (as well as an additional fix), but this is now compatible with --serve and --watch. As for compatibility with @dojo/cli-test-intern, that command expects fully-built output/test/tests/unit.js and output/test/tests/functional.js files to run, at least with dojo test -c local. The easiest solution is to require an index.ts file that can serve as the entry point for libraries, so that the webpack build could be used. Then again, that would result in testing something other than what is normally built with --target=lib.

agubler · 2018-09-14T18:30:16Z

@mwistrand coming together 👍 Would be great if the output was formatted like the output from the custom element target... Would that be possible?

mwistrand · 2018-09-16T19:20:49Z

@agubler After playing around with this a bit, I came up with what you see below. Note that we still need to determine how to best use this with cli-test-intern, and that the current expectation is that libraries will use separate src/ and tests/ directories (so this is not a drop-in replacement for the custom @dojo/widgets build, for example).

codecov · 2019-02-03T20:37:09Z

Codecov Report

Merging #42 into master will decrease coverage by 6.17%.
The diff coverage is 44.94%.

@@            Coverage Diff            @@
##           master     #42      +/-   ##
=========================================
- Coverage   70.67%   64.5%   -6.18%     
=========================================
  Files           8       8              
  Lines         358     400      +42     
  Branches       59      83      +24     
=========================================
+ Hits          253     258       +5     
- Misses        105     142      +37

Impacted Files	Coverage Δ
src/ejected.config.ts	`100% <100%> (ø)`	⬆️
src/util.ts	`100% <100%> (ø)`	⬆️
src/logger.ts	`100% <100%> (ø)`	⬆️
src/base.config.ts	`24% <2.7%> (-9.83%)`	⬇️
src/dist.config.ts	`37.5% <27.27%> (-6.95%)`	⬇️
src/dev.config.ts	`33.33% <60%> (+2.56%)`	⬆️
src/main.ts	`95.31% <82.35%> (+0.03%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc78b9e...2917832. Read the comment docs.

agubler · 2019-03-07T09:56:54Z

@mwistrand At the moment we can only build either .js (legacy) or .mjs (evergreen), we'll need a way to build both for publishing. What would the recommended way be to do that? For example, we cannot just do two builds because it cleans the output directory?

agubler · 2019-03-07T09:59:17Z

Also I think we'll want to minify the output css (we use cssnano currently in cli-build-app and dojo/widgets)

agubler · 2019-03-07T18:21:46Z

Could we configure the lib mode in the same way as we currently configure widgets to build as custom elements? It would be good if there was a consistent way to do this across the two modes. Perhaps we could rename the element dojorc configuration to something more generic like widgets and in ce mode it works as it does today, but in lib mode we generate the entry point for the user.

mwistrand · 2019-03-14T18:05:25Z

@agubler I've pushed changes that will 1) generate an entry point from the .dojorc and 2) output both legacy and evergreen side-by-side. Once tests are passing, I'll push my cssnano changes to my webpack-contrib PR.

agubler · 2019-03-29T09:11:32Z

Thanks @mwistrand! Let me know when it's ready for review again :)

agubler · 2019-03-29T09:12:58Z

One thing can we rename the elements option to widgets to cover both targets?

willstott101 · 2019-05-07T10:50:41Z

+	const basePath = process.cwd();
+	libEntryPath = path.join(basePath, 'src', 'main.ts');
+	if (!fs.existsSync(libEntryPath)) {
+		const tmpDir = fs.mkdtempSync(os.tmpdir());


Thanks for your work on this. It's going to be very useful!

Whilst trying to get this setup I've found this line doesn't work on linux as os.tmpdir does not include the path separator. This means the build tries to make a folder in the root of the filesystem.

util.ts in this repo already works around it with:

fs.mkdtempSync(`${os.tmpdir()}${path.sep}`)

I'm curious about on which OS this works for you?

Thanks for catching this. It's been working for me on macOS and Windows 10/11, but I'll update this as well.

willstott101 · 2019-05-07T15:05:54Z

Are there any patches required in a cli-build-app powered app to use a library built with this?

In my dev build main.js I've ended up with a button.m.css which is imported into button.m.css.js which is used to resolve class names. However button.m.css.js never uses the map in button.m.css, so the resulting class names on the generated elements do not match the final concatenated css, only matching the class names in the output from this tool.

EDIT: I'm sorry if that's really unclear I'm a pretty lost in all this complex build stuff.

This tool makes some css and JS files.
In an app I install my widget library built with this tool.
The app build at some point additionally mangles the css class names.
This second mangling step is not correctly honoured in the JS.

agubler

@mwistrand This is super close, just a few suggestions and a heads up that we are probably going to land #56 (changes the CE builds to using a single configuration rather than a configuration for each element), which will mean this will need a bit of a rebase (shouldn't be too much!).

Also have been thinking about the issue with creating a legacy and modern build together, for now to reduce the complexity I am going to reverse a bit and say that we should only support producing one at a time. It will be down to the consumer to run the command twice and package (copy, move etc) the output as needed for their final release bundled.

agubler · 2019-05-09T11:56:01Z

-			return entry;
-		}, {}),
+		mode: args.target === 'lib' ? 'none' : 'development',
+		entry:


Instead of creating a temporary file or using an existing main.ts as the entry point, we could iterate through the widgets and use the widget.path as the entry point?

For filtering the webpack assets we could prefix the js and css output name with something like throw-away, which would leave all the other assets.

Something like

elements.reduce((entry: any, element: any) => { entry[element.name] = [element.path]; return entry; }, {});

agubler · 2019-05-09T11:58:27Z

 	const outputPath = output!.path as string;
+	const modeOutputPath = path.join(outputPath, location);
+
+	if (module && args.target === 'lib') {


Can we do this in the base and change based on mode and target? It would save having to have a duplication to edit the ts-loader configuration.

In another change @rorticus is turning transpileOnly off for building custom elements, so that wouldn't need to be updated for the lib target.

@rorticus's PR is #56

agubler · 2019-05-09T11:59:31Z

+	config.plugins = removeEmpty([
+		...plugins!,
+		(args.target !== 'lib' || args.clean) && new CleanWebpackPlugin([location], { root: outputPath, verbose: false }),
+		args.target === 'lib' &&


Can we include the EmitAllPlugin in the base config as well and set inlineSourceMaps to false for both dev and dist outputs.

agubler · 2019-05-09T12:00:11Z

 			return child.assets.map((asset: any) => {
 				const size = (asset.size / 1000).toFixed(2);
-				return `${entry}/${asset.name} ${chalk.yellow(`(${size}kb)`)}`;
+				const assetName = isLibrary ? asset.name.replace(/^\//, '') : `${entry}/${asset.name}`;


How come we need to strip the leading / here?

agubler · 2019-05-09T12:00:52Z

+
+	const basePath = process.cwd();
+	libEntryPath = path.join(basePath, 'src', 'main.ts');
+	if (!fs.existsSync(libEntryPath)) {


As per the comment above, if we use the element.path as the module to load we shouldn't need to create any temporary files.

agubler · 2019-05-09T12:22:27Z

@mwistrand @dojo/webpack-contrib@6.0.0-alpha.1 has been released

mwistrand · 2019-05-10T22:14:01Z

@willstott101 If you're CSS is getting mangled then something is wrong; you shouldn't have to change anything to your app to use a library built with @dojo/cli-build-widget. For all intents and purposes, a library built with @dojo/cli-build-widget should look like any other library you might import into your application. Do you have a demo application (e.g., in codesandbox) you can share that displays this problem? In the meantime, I'll see if I can reproduce this issue on my end.

mwistrand · 2019-05-12T16:01:35Z

@willstott101 I haven't yet explored much more deeply than this, but the correct CSS classes are preserved when building @dojo/widgets with dojo build widget -t lib and using the resulting package with a default app created with dojo create app: https://github.com/mwistrand/dojo-widgets-target-lib

The deps/ directory includes both the widgets tarball, as well as a package/ directory that shows its contents, which were generated with dojo build widget -t lib (and then copying over the README and package.json).

Since @dojo/cli-build-app will only generate unique CSS classes for CSS files found within the src/, any CSS files found outside of src/ (for example, in node_modules/) should remain untouched. Please let me know if that is not your experience.

willstott101 · 2019-05-12T16:07:31Z

Thankyou very much for looking into this. It's quite possible I've got something wrong. I can't get at my work repos till Monday, but I can say that I'm trying to theme the imported widgets.

I'll try and use the widgets from my lib without themeing and see if the class names still change.

Should the names stay the same even if they're being themed by another (cli-build-theme) package?

agubler · 2019-05-17T11:33:19Z

    "chalk": "2.4.1",
    "clean-webpack-plugin": "1.0.0",
    "cli-columns": "3.1.2",
+    "copy-webpack-plugin": "^4.6.0",


@mwistrand Can we pin all the dependencies please?

matt-gadd · 2019-05-17T11:47:40Z

-								getLocalIdent
+								modules: true,
+								sourceMap: true,
+								url: args.target !== 'lib'


what does this do?

It prevents css-loader from replacing url() values for library builds.

don't we want to resolve url's for library builds? why should the css be different in this mode?

Since I imagine we want libraries to be able to use the url(~from/node_modules) syntax, then we do need to resolve the URLs. At the time, I experienced some difficulty getting a clean match between the replaced url() path and the actual location of the asset. For example, if the CSS file is theme/icon.m.css, then the font it imports would be output to output/{mode}/name.woff, but output/{mode}/theme/icon.m.css would import it as url(name.woff) instead of ../name.woff. Fortunately, a newer version of MiniCssExtractPlugin allows the publicPath option to be a function, which effectively mitigates this issue (so the font would be imported as url(../name.woff)).

matt-gadd · 2019-06-09T20:15:23Z

+			describe: 'the type of project',
+			alias: 't',
+			default: 'widget',
+			choices: ['widget', 'lib']


should we perhaps change widget to custom element now?

matt-gadd · 2019-06-09T20:17:04Z

-	config.plugins = [...plugins!, new CleanWebpackPlugin([location], { root: outputPath, verbose: false })];
+	config.plugins = removeEmpty([
+		...plugins!,
+		(args.target !== 'lib' || args.clean) && new CleanWebpackPlugin([location], { root: outputPath, verbose: false })


I can't see where this args.clean is set, now that we are building the dev and dist separately do we actually need this different behaviour anymore?

I believe it can be removed (from both here and the dist config), but I'll double-check before blindly doing so.

matt-gadd · 2019-06-09T20:20:38Z

+	]);
+
+	if (args.target === 'lib') {
+		config.devtool = 'source-map';


isn't this the default for production anyway? if so we can just set it for both and drop the if as it won't harm.

matt-gadd · 2019-06-09T20:47:30Z

+			} as any),
+			args.target === 'lib' &&
+				new CopyWebpackPlugin(
+					['src/**/*.css.d.ts', assetGlob].map(from => ({


won't any assets in the webpack build get copied over anyway?

agubler · 2019-06-26T10:46:59Z

@mwistrand Re the hashed assets like fonts, doesn’t everything reference the assets non hashed and by the original paths? How do the hashed asset names work?

agubler · 2019-06-26T12:46:21Z

@mwistrand Sorry I've given you conflicts again :( I took the serve changes also.

mwistrand · 2019-06-28T11:55:47Z

@agubler The file-loader reports the new name to the CSS loader(s) so the widget library files reference the correct filename. That said, since @dojo/cli-build-app would hash any assets anyway, I'll update to preserve the original filenames in their original directory locations, with the caveat that assets will be referenced from the project root (for example, src/theme/icon.m.css would import src/theme/fonts/dojo2.ttf with url(../theme/fonts/dojo2.ttf) instead of url(./fonts/dojo2.ttf)).

Add a `--target=lib` flag that, when set, compiles and emits the specified widgets as individual files within a library. Add Cypress tests Tests behave as follows: - Build the lib - Copy the output back into the test-app - Build the test app - Execute tests against that application

- Upgrade MiniCssExtractPlugin for asset fix - Simplify config based on review feedback

- Fix: the custom-element.js template should be copied to `config/build-widget` on eject. - Add `--env.target` to the ejected config - Generate single configs for all elements instead of mapping each element to a separate webpack config

agubler · 2019-06-28T13:28:34Z

@mwistrand Okay, I think that's okay - I don't think I have any comments left! Have we tried it with @dojo/widgets?

mwistrand · 2019-06-28T16:51:40Z

@agubler I’ve been using @dojo/widgets as my guinea pig for these changes, which I’ve then imported into a test application. For a sanity check I’ll test with the widget showcase one last time after I fix the failing build.

agubler · 2019-06-28T16:56:54Z

@mwistrand perfect!!

agubler · 2019-06-30T21:06:01Z

@mwistrand Let me know when you've done the sanity check 😄

As managing d.ts files for library builds will be handled by the EmitAllPlugin, the copy-webpack-plugin and corresponding types packages are no longer necessary as dependencies.

mwistrand requested review from agubler and matt-gadd September 12, 2018 20:07

agubler requested changes Sep 13, 2018

View reviewed changes

mwistrand force-pushed the 35-library-target branch from b35abc9 to fec5b12 Compare September 14, 2018 22:10

mwistrand force-pushed the 35-library-target branch from 12c3ed7 to c12c2d6 Compare February 3, 2019 20:37

mwistrand force-pushed the 35-library-target branch from c12c2d6 to cb1683a Compare March 14, 2019 18:04

mwistrand force-pushed the 35-library-target branch from cb1683a to 224864a Compare March 15, 2019 12:06

mwistrand force-pushed the 35-library-target branch from f112f91 to 936b7f4 Compare April 10, 2019 21:18

willstott101 reviewed May 7, 2019

View reviewed changes

agubler requested changes May 9, 2019

View reviewed changes

mwistrand force-pushed the 35-library-target branch from 62c7a87 to 2f04e46 Compare May 15, 2019 16:52

agubler reviewed May 17, 2019

View reviewed changes

matt-gadd reviewed May 17, 2019

View reviewed changes

mwistrand force-pushed the 35-library-target branch from 3f0cae7 to ed0626a Compare June 8, 2019 18:50

matt-gadd reviewed Jun 9, 2019

View reviewed changes

mwistrand force-pushed the 35-library-target branch 3 times, most recently from 09a9603 to 548c956 Compare June 22, 2019 19:02

mwistrand added 7 commits June 28, 2019 07:20

Fix tests

21e20ae

Updates

d0ba739

- Upgrade MiniCssExtractPlugin for asset fix - Simplify config based on review feedback

Replace http-server with serve

8798093

Fix/update the ejected config

6fa1615

- Fix: the custom-element.js template should be copied to `config/build-widget` on eject. - Add `--env.target` to the ejected config - Generate single configs for all elements instead of mapping each element to a separate webpack config

Rename elements option to widgets; update README

a83b3fe

Do not hash library assets

206ffd2

mwistrand force-pushed the 35-library-target branch from a7a8260 to 206ffd2 Compare June 28, 2019 12:46

update test lib app to 6 alpha

cb612a6

mwistrand added 2 commits July 1, 2019 14:35

Remove copy-webpack-plugin

54d7c44

As managing d.ts files for library builds will be handled by the EmitAllPlugin, the copy-webpack-plugin and corresponding types packages are no longer necessary as dependencies.

Use transformer provided by EmitAllPlugin

2917832

agubler approved these changes Jul 9, 2019

View reviewed changes

mwistrand merged commit 29f4cdf into dojo:master Jul 9, 2019

mwistrand deleted the 35-library-target branch August 7, 2019 13:29

Uh oh!

Conversation

mwistrand commented Sep 12, 2018

Uh oh!

agubler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agubler commented Sep 13, 2018

Uh oh!

mwistrand commented Sep 13, 2018

Uh oh!

agubler commented Sep 13, 2018

Uh oh!

mwistrand commented Sep 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agubler commented Sep 14, 2018

Uh oh!

mwistrand commented Sep 16, 2018

Uh oh!

codecov Bot commented Feb 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

agubler commented Mar 7, 2019

Uh oh!

agubler commented Mar 7, 2019

Uh oh!

agubler commented Mar 7, 2019

Uh oh!

mwistrand commented Mar 14, 2019

Uh oh!

agubler commented Mar 29, 2019

Uh oh!

agubler commented Mar 29, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willstott101 commented May 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agubler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agubler commented May 9, 2019

Uh oh!

mwistrand commented May 10, 2019

Uh oh!

mwistrand commented Sep 13, 2018 •

edited

Loading

codecov Bot commented Feb 3, 2019 •

edited

Loading

willstott101 commented May 7, 2019 •

edited

Loading

willstott101 commented May 12, 2019 •

edited

Loading

matt-gadd Jun 9, 2019 •

edited

Loading