BTI (Build Time Instructions) + LAZY LOAD proposal #312
Comments
|
i believe for the advanced mode above we should be able to cascade shared config like plugins,etc... to all bundles instead of just redefine them in each bundle |
|
@devmondo all options are copied! But you can override them! |
|
@nchanged perfecto!!!! that is the behavior i was trying to suggest |
|
This is really freaking nice. Suggestions are as follows:
// Suggestion 1
fsbx.init({
homeDir: './src',
outDir: 'dist',
outFile: '[name]-[hash]' // Could just be [name] for dev or just [hash] for prod or whatever
}).bundle('>[index.ts]')
// Suggestion 2
fsbx.init({
homeDir: './src',
// Having an output object might be a little more future-proof
// as/when more configuration options are added?
out: {
dir: 'dist',
name: '[name]-[hash]'
}
}).bundle('>[index.ts]')
{
"index.ts": "index-sdg3sj.js",
"components/Bar.ts": "Bar-as728d.js",
"components/Foo.ts": "Foo-jff82p1.js",
"some/image.jpg": "image-r5k3sj.jpg",
}Appreciate that FuseBox takes care of mapping modules to the hashed file names inside the primary bundle, but it's really useful to be able to extract this information for things like server-side rendering. If the primary bundle were to have a hashed name so that it could be cached, this manifest file would be required by the server to reference the latest primary bundle when creating the script tag. |
|
Fuse Code Splitting proposal.pdf @wagerfield I fully agree with. I share again my proposal on this way. I see an issue to ask someone to have the API FuseBox.load, this will lead to a big confusion about when I use this or not and what is the best practices. |
|
On the pdf, i didnt mention some ideas to avoid to polluate the intended goal. |
|
@nchanged I like the idea of implicit bundles and the I'm not sure I fully understand the rest of the proposition. It also seems that you threw in some content hashing that I believe is not directly related to code splitting. I'll describe my use case: code splitting + server side rendering with react, greatly inspired by what next.js is doing. The code executed on the server is a bundle ( The client ( There is a special directory Routing is handled internally with Ideal API: const fb = fuse.init({
// common config
homeDir: `./src`
})
fb.addBundles('pages/[name].js', '[pages/*.tsx]')
// notice the plural
fb.addBundle('renderer.js', '[lib/renderer.ts]', {serverOnly: true})
fb.addBundle('client.js', '> [lib/client.ts]', {clientOnly: true})
// currently I need to wrap code in `!FuseBox.isServer`)
fb.addBundle('common.js', '+react +react-dom')
// ideally fb.addCommonBundle('common.js')
fb.watch() |
|
A number of different things being discussed in this thread, so let me chime in on them separately. Environment Flags@geowarin what do the Instead, I propose that FuseBox adopt something like webpack's const fsbx = new FuseBox({
target: 'browser' // default, so not necessary here
})
fsbx.bundle('>[client.ts]') // will use 'browser' target
fsbx.bundle('[server.ts]', {
// this object simply overrides any values set in fsbx.init()
target: 'node'
})Code SplittingThinking about it a little more, I agree with @leon-andria and @geowarin —code splitting should be offloaded to FuseBox's config and kept out of the source code. There are simply too many complex cases that cannot be resolved when code splitting in the way you proposed above @nchanged. Consider webpack's Common Chunks Plugin—this is completely dynamic way of code splitting your app based on the number of times a module is imported/required. This could not be done within the source—it would have to be done in the config. It would seem that the main incentive for handling both code splitting and lazy loading with the same mechanism using a proprietary FuseBox API is to make dynamic configuration of code splitting "easy" and to allow HMR to work. These are both compelling reasons, but when you consider how infrequently you configure code splitting in the real world (such as when adding new third-party libs to a vendor/common.js bundle or branching off heavy/infrequently viewed pages in your app), I don't think it should be a driving factor. Instead, I propose that when running FuseBox in dev mode with the Lazy LoadingLazy loading should adopt proposals from the official ES spec rather than use a proprietary FuseBox API. Again, this would mean that source code is kept decoupled from the build tool. @nchanged as we have discussed on Gitter, there is is an ES6 proposal for an This would mean that you can both statically import modules using Using the The mapping between dynamically loaded modules and their hashed bundle names is where FuseBox could really shine. Since FuseBox knows about all of your bundles as well as their hashed names, you could dynamically // fuse.js
const fb = FuseBox.init({
homeDir: 'src', // this could be 'inputDir' to compliment the output object I/O ftw :)
target: 'browser', // not needed here since 'browser' would be default—just for demonstration purposes
output: {
dir: 'dist',
name: '[name]-[hash]', // bundle file name format with placeholders like [name] and [hash]
path: 'https://cdn.foo.bar/some/path/' // prefixed to all dynamic imports
},
manifest: 'manifest.json' // spit out a file that maps all friendly bundle names to their hashed file paths
})
fb.add({
'client': '>[client.ts]', // dist/client-sdf561.js
'pages/home': '[pages/home.ts]', // dist/pages/home-ksdg02.js
'pages/about': '[pages/about.ts]', // dist/pages/about-iw7dns.js
'pages/feed': '[pages/feed.ts]', // dist/pages/feed-yrn655q.js
'pages/*': '[pages/*.ts]', // perhaps the above 3 lines could be written with a wildcard as @geowarin suggested
'vendor': `
+ react
+ react-router
+ redux
+ redux-react
`
})
fb.add({
'server': '>[server.ts]'
}, {
target: 'node' // config overrides could be passed as an optional second arg
})
fb.watch()
----------------------------------------
// routes.js
function loadPageAsync(pageName) {
import(pageName).then(module => {
// do something with 'module' to require/import the loaded module
})
}
// some URL matching logic...
loadPageAsync('pages/home') // notice that there is no file ext here
loadPageAsync('pages/about') // these are just the friendly bundle names
loadPageAsync('pages/feed') // replaced by `${output.path}${output.name}.${ext}`
// import('pages/feed').then(...) would become:
// import('https://cdn.foo.bar/some/path/pages/feed-sdf561.js').then(...)Sorry for the essay! |
|
On first look, my mind says the code-based approach to code splitting looks really cool. But my intuition has 4 or 5 alarm bells going off. Basically, when I sat down to figure out why, I came to one simple conclusion: magic is really dangerous in code. If there is no obvious link to the bundling, or how code is split up, it becomes difficult to maintain, especially for a newcomer to the code. Imagine Joe writes a huge project with implicit code splitting, defined by a few Fusebox.load() calls in code. Joe moves to Google, and Jane takes over. Jane is unfamiliar with Fusebox, sees the bundle configuration, and thinks ok cool, I've got this. She is digging in the source, and sees the Fusebox.load() call and makes a minor change, replacing it with import because she likes standards. Suddenly, the production code load time goes through the roof, nobody knows why, users start fleeing the app, and the company goes under. Don't you love melodrama? :) So, maintainability is sacrificed with this magic. Now, another thing: if you force users to use fusebox in their code, it ties them to fusebox and also removes flexibility to change the fusebox API. I have the same problem with require.ensure in webpack. Put another way, if your app changes down the road, and you reshuffle which file is bundled where, You'd want to be able to handle that in the bundle configuration, and not care in the source. Testing code that has dynamic imports is tricky. Providing a synchronous version of the loader for testing purposes would be awesome. The most obvious way to do this is to use standard require syntax for loading, and rewrite this when bundling. To summarize: Code splitting is a bundling concern, and it would be great to keep the code separated from those concerns. Do code split definitions using an explicit section of the bundle configuration. This allows doing fancy stuff like splitting off most-used code, or simply splitting by page content, or splitting by file system, or thing-we-have-not-yet-thought-of. Then allow the Javascript to focus on simply stating unconditional dependencies (top-level import) and conditional dependencies (other imports/require) and have fusebox rewrite them after splitting is set up. The bundler is already doing magic by allowing separate files that will be bundled to pretend they are separate, this would just augment that. For me, this kind of magic is acceptable, because there is no ambiguity about what is happening. The code says it wants a dependency and then the dependency is there. Even bugs in fusebox related to this are easy to isolate to a single line as the source of the bug. Magic that can be debugged easily by an outsider is OK, in my book. Hope this perspective is useful. BTW for a case study in magic out of control, check out Meteor. They designed it so that you can't easily pull in outside sources. For example, wallabyjs is impossible to configure with meteor, unless you literally use about a thousand lines of mocking code. You can only test with their built in testing framework, which is very slow. The problem comes down to the way they decided to implement import. If you import "meteor/anything" it will instead magically rewrite that to including a meteor package (which is different from an nom package). This is why a huge stub has to be written to support wallabyjs. In the end, the time saved by having that magic is offset by the time wasted trying to work around its unintended consequences. I'd love to see fusebox avoid that kind of design flaw. |
|
@cellog I like melodrama :)
Fully agree! |
|
@wagerfield regarding webpack's |
|
regarding import("hello").then(module => {
})So our proposal is to convert |
|
I love our community! Quite compelling arguments you got here guys.
|
|
Great, so we're all agreed on lazy loading using the proposed Issue now becomes—what's the best way for dynamically importing modules/bundles that have both a hashed file name as well as some arbitrary path to them (such as a CDN). import('foo.js').then(module => {...})
// Needs to become
import('https://cdn.foo.com/path/foo-sdgi75.js').then(module => {...})I proposed a solution above with my |
|
My alarm bells are going off about import().then(). Remember when everyone was using @decorators? And the spec shifted and now all that code is obsolete. I'd rather not plan for future until it's sure. That's how one shoots oneself in the foot. I do have another idea, but before I do, some background would be useful. I've been using react and redux a fair amount recently, and for asynchronous stuff, redux-saga. Redux-saga solves the asynchronous nesting hell of both callbacks and promises using generator syntax. For those unfamiliar, it's similar to async/await syntax. If you don't know that, basically it allows using asynchronous code as if it were synchronous. This principle can be used for conditional includes. function thing() {
const it = require 'something'
it.shouldWork()
} If the code is loaded asynchronously, the above code fails, of course. But using generator syntax, we can pause execution and resume when ready, just like a promise or callback, but without the nesting: function *thing() {
const it = yield require 'something'
it.shouldWork()
} The fusebox code would need to call the generator, and on anything yielded, check to see if a promise is returned. If a promise isn't returned, then it immediately resumes execution with the result (synchronous require). Otherwise it attaches a then, and returns control to the main code. Upon resolution of the promise, it resumes execution with the result. There are a couple major issues with this idea. First, if we change a function definition into a generator, it will break all code that uses that function. So fusebox would have to instead create a wrapper function and a call to the generator Handler. This should work OK for 99% of circumstances, but would increase complexity of the run time. The bundler would have to detect all require statements that could be conditional and wrap them in the generator. Of course, this is mitigated by the fact that it would only be necessary to do this if the user explicitly requests code splitting. In any case, with this idea, the code would not need to know about code splitting. Fusebox could simply wrap any function that contains an import/require, and pop a "yield" in front of the require, and it would create instant asynchronous loading support without needing to change the original source. Let me know if I need to clarify this idea! |
|
@wagerfield this is solved by aliasing because, in theory, you never put the CDN path but the key. Fusebox will know exactly what to do based on the manifest it has. So you can have What is really important is that fusebox should be flexible and focuses on what it does best. Extensibilities are keys because each one has is own use cases simple or complex. |
|
@cellog I love the idea! But you can't use We can't use generators. Some people target their code to run in Wrapping all require statements will add an unnecessary overhead if we want to make it compatible with everything. However, fusebox could detect that a Sure, generators are nice but less straightforward. Some people would want just a promise. And yes, it goes sideways with the official es6 proposal. But it's nice. Wonder what other people think. |
|
I started out writing this response by agreeing with you, but by the time I got to the end, I thought of a condition that shatters the entire idea, and I have a new proposal at the bottom. If you are willing to follow the journey of how I got there, it will make more sense, I think, so here's the story. If we do use promises, it would transform something like: function thing() {
const it = require('something')
it.shouldWork()
}into: function thing() {
require('something').then(result => {
const it = result
it.shouldWork()
})
}but we run into a snag if the function expects to return a value from the conditionally loaded file: function thing() {
const it = require('something')
return it.shouldWork()
}
const x = thing()we need to be able to suspend all execution until the required thing is loaded. Most people implement this in their router, of course. Perhaps another way to look at this problem is how to keep things easily separated so coupling is minimal?
In other words, I'm contradicting my last idea :). Here's what I'm proposing:
What this means is that code should literally not try to do any asynchronous loading of other code. Instead, expect the user to implement that in their router (or equivalent state management system), and provide easy 1-liners to load/add code to fusebox. This way, the code is consistent, and isolated from how it gets loaded, making both debugging and testing simpler. Loading can be tested as part of the router tests, which is simpler, and easier to maintain. No need for injection of promises or generators (as clever as all that feels) into the source. The source simply replaces require/import as it does now. This will be far more maintainable in the long run, and make it easier to re-tool both fusebox and the end-user code if things change in the future. |
|
guys, awesome suggestions, and proposals. but let us think for a moment here about spec compliance. we all know that the specs are driven by people we don't have much control over (politics and power!). and history proved that they would change or push things down our throat even if we don't agree with. That said, we think the best approach would be to align with specs if they offer a stable, set in stone features. otherwise, we are just wasting time and resources on proposals that might not even make it past the editor they were written in. Another point to be highlighted is the fact that specs will never be as fast as us, we can't wait, and if we did so, none of the advanced and awesome FuseBox features would have been here. for example, how long would it take for them to give us something like FuseBox we introduced FuseBox because we were tired of the typical approach, and just because most of us out there use specific tools or trends, it does not necessarily mean it is the best approach, and this is how we conceived FusBox because we wanted to think outside the BOX :) To summarize it up, if there is a feature you guys (the community) see it needs to be implemented, we will listen to you first and not the specs! unless we all agree that the specs and your requirements align. We want to empower you and we want you to enlighten us, it is true that there is establishment behind FuseBox but it was decided with the first line of code we wrote that it is a community product and it shall always be this way. so please please please keep giving us your awesome ideas here, we want them and we can't live without them, you all Rock 💯 |
|
@cellog @devmondo totally agree about being cautious with non-stable specs while not wanting to wait around for them to solidify. ES6/7 wouldn't be where it is today if it were not for languages like CoffeeScript and transpilers like Babel experimenting and driving things forwards. @cellog I think I understand your latest proposal and it sounds sensible—though I would like some clarification on how you see FuseBox 'detecting' conditional
Transpiling should be opt-in. When using FuseBox with vanilla JS (not TS) with no transpiling plugins like Babel, bundled source code should look and behave the same par the Re. generators and promises, I now agree with @nchanged—adding polyfills for generators and the like adds unnecessary bloat to the FuseBox runtime that might not be used by the majority. So if we don't want to adopt non-stable specs like
In places where a user wants to dynamically load files or bundles they would do so with helpers from the // This is really important
import FuseBox from 'fusebox'
function loadPage(pageName) {
// This is what you already have I think
FuseBox.load(pageName, (module) => { // No Promise here, no polyfill needed
})
}Taking this approach would mean that your project source code now has a dependency on This would allow FuseBox to develop it's own API and it's own ways of doing things—experiment and provide cool new features. As a developer you can then chose to use it in your source code, or not use it and do your own thing with Promises and HTTP requests or whatever. The advantage of making The big question for me is:
FuseBox.load('some-bundle', module => {
const SomeModule = require('some-bundle')
// or
const SomeModule = FuseBox.import('some-bundle')
})
// We need to map 'some-bundle' to 'http://cdn.foo.com/some-bundle-yw7ja1.js'
FuseBox.load('http://cdn.foo.com/some-bundle-yw7ja1.js', module => {...})FuseBox bundler would still need a way of spitting out this const manifest = require('manifest.json')
fetch(manifest['some-bundle']).then(response => {
// Do something with the response and 'require' the loaded bundle
}) |
|
thanks @wagerfield all good points. How about: However |
|
Everyone, these are some really valid concerns and great suggestion. I may sound little bit critical here, but I feel we are maybe deviating from the main goal and the required end result. We are talking a lot about WHAT but not much on HOW. We should continue to elaborate more on how FuseBox internally will handle and resolve modules or packages. The more we provide insights to the Core Team the better and more transparent the approach would be. this would immensely allow them and us to have a better ideas of how to instrument a solid and straightforward API that fulfill the vast requirements for many of us out there. The key point to remember here is that not everyone uses webpack nor everyone have the same pipeline or set of standards to build their projects, to be more precise, we need to take into consideration that some would start immediately with FuseBox and would not to think at all about other techniques or methodologies out there, thus it is imperative to keep the personality and philosophy of FuseBox standout, at the same time i Do agree that we need to follow the Specs whenever it is possible but again, not on the expense of having the ability to be flexible and pragmatic. Personally I really want to talk about FuseBox events, metadata, or manifest and what FuseBox bake in as an abstraction to consume its API. coming from .NET and JAVA and working on a day to day basis with fortune 500 companies i can assure you that the image is not always easy on the eyes. and what JavaScript has achieved yet is not fully compatible with those environments, but with FuseBox i was able to start breaking this wall now! and my team is looking forward to make the transition, so i will try my best to chime in on this from that perspective as much as possible. thank you all 👍 |
|
Quick answer (hopefully): Detecting conditional require. One could of course parse and tokenize the source, or make a babel plugin, but the way I would detect it is with a simple convention: top-level import and require are like so: import blah from 'blah'
var blah = require('blah') Whereas conditional are indented. So look for whitespace before the declaration, and there you go. |
|
@cellog we already do analysis on each file. Whitespace often can lead to errors and confusion. I don't think that's the best approach here. |
|
Closing this for now. Please feel free to check the latest bleeding edge version |
BTI
BTI stands for Build Time Instructions and will allow users to split the code base painlessly A developer will be allowed to define the rules in the code, which will be analysed by FuseBox. Config can be involved for the sake of verbosity / for advanced users.
But the primary goal here is to make as simple as possible without functionality sacrifices.
Implicit case
Imagine a structure
In this case, FuseBox will understand that these 2 files should be bundled beforehand and ignored by the primary. Therefore it will create instructions as follows:
Lazy modules will be isolated by default as it's important to make them lightweight. A configuration will be copied from the primary config, adding
standalone : false(not implemented yet) to it.A filename will be generated (hash) and put next to the primary bundle. In our case, we will have 3 files.
FuseBox will be able to map
components/Bar.tsto3e83n3.jsfile since the information will be baked intobundle.js(primary)FuseBox will be able to check which files are already in the primary bundle, however, one should move to the advanced mode (below) when having a non-trivial case.
Explicit case
To allow customisation, one would define keys that start with
@, which will be mapped to the primary configuration.An advanced configuration would look like:
Or like this:
Simpler:
Building
In both cases, FuseBox will launch a separate process per instruction, AFTER the primary build is complete to ensure files uniqueness (We should avoid duplicates).
HMRwill support generic functionality to flush and reload affected scope.For Production
In the case above FuseBox will create bundle
@Bar.You could continue using
FuseBox.import("./filename)orFuseBox.lazy("@bar")Your ideas are invaluable, please express your concerns, suggest ideas!
The text was updated successfully, but these errors were encountered: