Delay fragment loading until something else loaded

[tomhillgreyridge]

Bug description

What I am trying to do is something along the following lines

• Make a request for a fragment using Unpoly
• Catch that request using an up:fragment:loaded event and inspect the response
• If it contains certain selectors load an additional script containing additional Unpoly compilers
• Delay the fragment insert until that new script has fully loaded

The idea is to allow the Unpoly compilers to be loaded dynamically as needed rather than all up front (large legacy codebase so there might be several hundred compilers which I don't want to load up front).

Problem is that I can't seem to make up:fragment:loaded wait for my script to load before inserting the fragment.

Reproduction project

I've made a basic reproduction here

https://glitch.com/edit/#!/nonstop-adhesive-way

which is based to some degree on

https://makandracards.com/makandra/52361-unpoly-loading-large-libraries-on-demand

Effectively, all I'm trying to do is "automate" the method above by inspecting the response html in a fragment and loading appropriate scripts based on the response content.

Steps to reproduce the behavior:

1. Go to reproduction project linked above
2. Switch on console logging
3. Click the "load fragment" link
4. Look at the messages in the console log

There are good explanations of expected behaviour and what I would hope to see here

https://glitch.com/edit/#!/nonstop-adhesive-way?path=scripts.js%3A38%3A91

Expected behavior

I was hoping to make the up:fragment:inserted wait until my additional script had loaded so that, when it is inserted, the compiler which is part of the new script will be run. Unfortunately, it doesn't which mean the fragment is inserted before my additional script has fully loaded.

[tomhillgreyridge]

Also, I'd just like to say thanks for the work put into this project. Implementing rich interfaces using Unpoly is a million times easier than messing around with webpack build files for a standard business application!

[adam12]

I'm not sure how this would work in practice, but if I was to approach this, I might look at doing something like this:

1. Have a file loaded on every page that knows about every compiler, but as a shim.
2. Each shim will load the proper compiler using loadScript, then up.hello(el) on the element.

You wouldn't want the shim compiler to run again, so perhaps you'd set an attribute on the element that knew the shim already ran (since up.hello might trigger it again).

ie.

up.compiler('[compiler-one]', function(el) {
  if (el.loaded) return

  el.loaded = true
  await loadScript("/scripts/compiler-one.js")
  up.hello(el)
})

up.compiler('[compiler-two]', function(el) {
  if (el.loaded) return

  el.loaded = true
  await loadScript("/scripts/compiler-two.js")
  up.hello(el)
})

// I duplicated to illustrate the example but I'd move this into a function to reduce the duplication

Re-calling up.hello on the element might have unintended side-effects, so maybe Henning or someone else knows how to handle this with your current approach, but my approach would be something a bit different.

[tomhillgreyridge]

I don't think that would work for me since what I'm trying to do is get away from a single point that knows about all compilers (we might well have hundreds). However your idea of double rendering did spark an idea.  I can probably *cancel* the original render, load the scripts and then manually call up.render with the render options and content for the original response. That way I am in control of when the render happens so can wait until any assets I need are loaded. Thanks for taking the time to reply

[triskweline]

I don't think that would work for me since what I'm trying to do is get away from a single point that knows about all compilers

If there is a up:fragment:loaded handler that knows which HTML needs to load which additional compilers, isn't this already a single point that knows about all compilers?

we might well have hundreds

Can you share why you expect to have that number of compilers?

Re-calling up.hello on the element might have unintended side-effects, so maybe Henning or someone else knows how to handle this with your current approach, but my approach would be something a bit different.

I would modify @adam12's approach so we don't need to re-call up.hello():

// lazy-compilers.js, always loaded
up.compiler('[one]', function(el) {
  await loadScript("/scripts/one.js")
  one(el)
})

up.compiler('[two]', function(el) {
  await loadScript("/scripts/two.js")
  two(el)
})

// one.js, loaded on demand
function one(element) {
  ...
}

// two.js, loaded on demand
function two(element) {
  ...
}

[tomhillgreyridge]

If there is a up:fragment:loaded handler that knows which HTML needs to load which additional compilers, isn't this already a single point that knows about all compilers?

Yes. This was a simplified example. I'm actually looking to add the behaviour TurboLinks / Hotwire has where it will correctly load and merge scripts in the head if they have not been loaded already, but wanted a simple example.

Can you share why you expect to have that number of compilers?

I work on large enterprise applications. Looking at ways of modernising legacy javascript on applications which can easily have 300-400 separate forms which have "behaviour" associated with them - enabling fields when other fields change etc - I believe Unpoly requires all Javascript to be registered via compilers so each form would require one.

Rather than having a single central massive file with a large number of compilers, I'd prefer Unpoly to simply load the new scripts required as it needed them. I know this isn't built into Unpoly so was looking to add it using the existing hooks.

I figured I could do this as follows

• Register an "up:fragment:loaded" event handler
• When the event happens, cancel it so the default unpoly merge does not happen
• Check the content of the new fragment head against the current page html
• If anything has changed, load the new scripts
• Once all new scripts have loaded, call up.render, passing the content of the new fragment and the original render options which I believe I can get from the loaded event.

Do you see anything technically wrong with that?

As far as I can see, all we are doing is "pausing" the render whilst we load additional compilers and then "resuming" by manually calling render with the original options. As long as we track which scripts have already been loaded, it should be the same as having loaded them all up front, just with a slight delay the first time a compiler needs to be dynamically loaded.

[tomhillgreyridge]

For reference, I have now written a quick test to try the above logic out. It works perfectly (delays the render until the script has loaded and thus causes the newly loaded compiler to execute on the fragment). However, since I am then doing a local render it does not update the URL in the same way Unpoly does.

• If I leave options.url set to the original URL it simply loads the page again
• If I clear options.url it doesn't update the browser url bar (obviously, since it doesn't know the URL)

Is there a method I can call with the URL which would do the same URL related logic as normally happens after a render? I tried to walk through the logic to find where this occurred, but couldn't find it.

[triskweline]

I work on large enterprise applications. Looking at ways of modernising legacy javascript on applications which can easily have 300-400 separate forms which have "behaviour" associated with them - enabling fields when other fields change etc - I believe Unpoly requires all Javascript to be registered via compilers so each form would require one.

I see. Have you considered a pattern like this:

/scripts
  /forms
    login.js
    new_user.js
    merge_records.js
    ... 400 more files

// /scripts/forms/login.js
export default function(form) {
  // initialize JavaScript behavior on the given form
}

<form behavior="/scripts/forms/login.js">
   ...
</form>

up.compiler('[behavior]', async function(element) {
  let scriptURL = element.getAttribute('behavior')
  let behavior = await import(scriptURL)
  behavior.default(element)
})

Is there a method I can call with the URL which would do the same URL related logic as normally happens after a render?

To change the browser's location bar while rendering content from a string, pass the URL as a { location } option.

[triskweline]

A different approach to handle this was proposed in RFC: Reconciliation of <head> elements

[tomhillgreyridge]

I've read through the RFC and it sounds absolutely perfect - it is vastly more comprehensive than I was expecting, but definitely covers every use case I can think of

[triskweline]

The use cases described in this issue should now be possible. See docs:

• Loading new assets
• Registering compilers after booting