Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use HTML meta to store asset map information #40

Merged
merged 2 commits into from Oct 29, 2018

Conversation

krallin
Copy link
Contributor

@krallin krallin commented Jun 7, 2018

There are a couple downsides to rewriting vendor.js in-place, which
are documented in #36:

  • It breaks SRI
  • It doesn't update the fingerprint for vendor.js, even though the
    contents have changed.

This patch fixes the issue by loading the asset map information (either
a JSON object or a URL for the asset map) into a HTML meta tag that is
stored in index.html instead, then read at runtime by
getAssetMapData (the meta tag is formatted as JSON and URL-encoded).

Conceptually, this is fairly similar to the storeConfigInMeta option
for the Ember build that stores the app's environment in a meta tag
(https://ember-cli.com/user-guide/#application-configuration).

@RuslanZavacky
Copy link
Contributor

O, @krallin thank you very much :) We had a big discussion in this one - #37. I've also got to a conclusion, that putting it into vendor.js wasn't right. When I've tried to fix it properly, I've quite got into a spaghetti of what comes before what and it kinda delayed my implementation.

I'll check yours, but can you please check tests?

Thank you :)

There are a couple downsides to rewriting vendor.js in-place, which
are documented in adopted-ember-addons#36:

- It breaks SRI
- It doesn't update the fingerprint for vendor.js, even though the
  contents have changed.

This patch fixes the issue by loading the asset map information (either
a JSON object or a URL for the asset map) into a HTML meta tag that is
stored in `index.html` instead, then read at runtime by
`getAssetMapData` (the meta tag is formatted as JSON and URL-encoded).

Conceptually, this is fairly similar to the `storeConfigInMeta` option
for the Ember build that stores the app's environment in a meta tag
(https://ember-cli.com/user-guide/#application-configuration).
@krallin
Copy link
Contributor Author

krallin commented Jun 7, 2018

@RuslanZavacky yep, just fixed tests (the root cause there was that we weren't interpolating tests/index.html).

@jbailey4
Copy link

Hey, my app recently ran into an issue in prod dealing with what our team expects is due to the vendor.js file's hash not changing and referencing an old assetMap hash. This PR looks like a step in the right direction, and IMO better tradeoff than rewriting vendor.js at build time. Also, falls in line with ember-cli storing config in meta tags.

Is there anything I can help out with in getting this merged in?

export default function getAssetMapData() {
// This placeholder is replaced in the build step
return '__asset_map_placeholder__';
const assetMapString = $("meta[name='ember-cli-ifa:assetMap']").attr('content');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the interest of supporting this RFC, it might make more sense to use
document.querySelector("meta[name='ember-cli-ifa:assetMap']").content and not use jQuery in this file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call; thanks. Updated my PR.

@krallin
Copy link
Contributor Author

krallin commented Jun 25, 2018

@jbailey4 I think this is in @RuslanZavacky's hands at this time: @RuslanZavacky, let me know if you have any thoughts on this PR 😄

export default function getAssetMapData() {
const assetMapString = $("meta[name='ember-cli-ifa:assetMap']").attr('content');
const assetMapString = document.querySelector("meta[name='ember-cli-ifa:assetMap']").content;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to run ember s with this change, it now complains on ember fastboot level, that document is not defined. For fastboot I think we have to come up with something different...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's super disappointing...if we have to stick with jQuery to support fastboot, then so be it, I guess. :\

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or just guard against it, as suggested here.

@Techn1x
Copy link

Techn1x commented Aug 28, 2018

So I've just spent a few hours today trying to figure out why SRI was failing on my build, and it turns out it was this addon (for the above mentioned reasons - vendor.js being modified but new hash not generated). I'm not sure why it's only just started to fail on me (I did just recently update to ember 3.2 and updated a bunch of packages).

I'm running 0.7.0 ember-cli-ifa, and 2.1.1 ember-cli-sri

I tried setting inline: true but it didn't seem to do anything (nothing was added to index.html)

Directly referencing this PR though, fixes my SRI issues (in testing/staging env). However I encounter an error when building for production. The error looks like it is because it can't find the file tests/index.html in the srihash_assets-output_path. If I comment out this line in the PR, it builds correctly.
replacePlaceholder(path.join(build.directory, 'tests/index.html'), assetMap);

Here's the error log

ERROR Summary:

  - broccoliBuilderErrorStack: [undefined]
  - codeFrame: [undefined]
  - errorMessage: ENOENT: no such file or directory, open '/home/ec2-user/environment/fsm-webapp/tmp/srihash_assets-output_path-nFLLxtjg.tmp/tests/index.html'
  - errorType: [undefined]
  - location:
    - column: [undefined]
    - file: [undefined]
    - line: [undefined]
  - message: ENOENT: no such file or directory, open '/home/ec2-user/environment/fsm-webapp/tmp/srihash_assets-output_path-nFLLxtjg.tmp/tests/index.html'
  - name: Error
  - nodeAnnotation: [undefined]
  - nodeName: [undefined]
  - originalErrorMessage: [undefined]
  - stack: Error: ENOENT: no such file or directory, open '/home/ec2-user/environment/fsm-webapp/tmp/srihash_assets-output_path-nFLLxtjg.tmp/tests/index.html'
    at Object.openSync (fs.js:434:3)
    at Object.readFileSync (fs.js:339:35)
    at replacePlaceholder (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli-ifa/index.js:10:23)
    at Class.postBuild (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli-ifa/index.js:80:5)
    at Class.superWrapper [as postBuild] (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/node_modules/core-object/lib/assign-properties.js:34:20)
    at addonPromises.project.addons.reduce (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/lib/models/builder.js:133:28)
    at Array.reduce (<anonymous>)
    at Builder.processAddonBuildSteps (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/lib/models/builder.js:130:43)
    at tryCatcher (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:323:19)
    at invokeCallback (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:495:31)
    at publish (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:481:7)
    at flush (/home/ec2-user/environment/fsm-webapp/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:2402:5)
    at process._tickCallback (internal/process/next_tick.js:61:11)

@Techn1x
Copy link

Techn1x commented Aug 28, 2018

A more permanent fix, works for both environments:

let filePath = path.join(build.directory, 'index.html');
replacePlaceholder(filePath, assetMap);
filePath = path.join(build.directory, 'tests/index.html');  // May not exist, eg prod build
if(fs.existsSync(filePath)) {
  replacePlaceholder(filePath, assetMap);
}

@Techn1x
Copy link

Techn1x commented Aug 28, 2018

I'd really like if we can get this PR across the line and merged/published

@Techn1x
Copy link

Techn1x commented Sep 19, 2018

Is there anything else we need for this? @krallin @RuslanZavacky I'm not sure how to add my above fix to this PR, but I'm happy to help test once its there.

@chrism
Copy link

chrism commented Oct 29, 2018

Hi @krallin have you tried your PR with Fastboot?

If I run FASTBOOT_DISABLED=true ember s it works fine, but if I run ember s I get this error

App is being served by FastBoot
DEBUG: -------------------------------
DEBUG: Ember      : 3.4.5
DEBUG: Ember Data : 3.4.4
DEBUG: -------------------------------
ReferenceError: document is not defined
    at getAssetMapData (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/addon-tree-output/ember-cli-ifa/utils/get-asset-map-data.js:9:1)
    at Object.initialize (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/addon-tree-output/ember-cli-ifa/initializers/asset-map.js:9:1)
    at /Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/@ember/engine/index.js:91:1
    at Vertices.each (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/dag-map.js:188:1)
    at Vertices.walk (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/dag-map.js:117:1)
    at DAG.each (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/dag-map.js:62:1)
    at DAG.topsort (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/dag-map.js:68:1)
    at Class._runInitializer (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/@ember/engine/index.js:112:1)
    at Class.runInitializers (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/@ember/engine/index.js:88:1)
    at Class._bootSync (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/@ember/application/lib/application.js:431:1)
    at Class.boot (/Users/chrismasters/git/chrismasters/tmp/broccoli_persistent_filterfastboot_manifest_rewrite__broccoli_asset_revfastboot_manifest_rewrite-output_path-2CC8jC1u.tmp/assets/@ember/application/lib/application.js:410:1)
    at EmberApp.buildAppInstance (/Users/chrismasters/git/chrismasters/node_modules/fastboot/src/ember-app.js:225:21)
    at EmberApp.visitRoute (/Users/chrismasters/git/chrismasters/node_modules/fastboot/src/ember-app.js:252:17)
    at EmberApp.visit (/Users/chrismasters/git/chrismasters/node_modules/fastboot/src/ember-app.js:321:17)
    at FastBoot.visit (/Users/chrismasters/git/chrismasters/node_modules/fastboot/src/index.js:79:22)
    at /Users/chrismasters/git/chrismasters/node_modules/fastboot-express-middleware/src/index.js:31:14
    at app.use (/Users/chrismasters/git/chrismasters/node_modules/ember-cli-fastboot/index.js:328:11)
    at Layer.handle [as handle_request] (/Users/chrismasters/git/chrismasters/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/Users/chrismasters/git/chrismasters/node_modules/express/lib/router/index.js:317:13)
    at /Users/chrismasters/git/chrismasters/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/Users/chrismasters/git/chrismasters/node_modules/express/lib/router/index.js:335:12)
    at next (/Users/chrismasters/git/chrismasters/node_modules/express/lib/router/index.js:275:10)
    at err (/Users/chrismasters/git/chrismasters/node_modules/ember-cli/lib/tasks/server/middleware/broccoli-watcher/index.js:54:11)
    at watcher.then (/Users/chrismasters/git/chrismasters/node_modules/broccoli-middleware/lib/watcher-middleware.js:35:7)
    at tryCatcher (/Users/chrismasters/git/chrismasters/node_modules/ember-cli/node_modules/rsvp/dist/lib/rsvp/-internal.js:39:19)
    at invokeCallback (/Users/chrismasters/git/chrismasters/node_modules/ember-cli/node_modules/rsvp/dist/lib/rsvp/-internal.js:211:31)
    at /Users/chrismasters/git/chrismasters/node_modules/ember-cli/node_modules/rsvp/dist/lib/rsvp/then.js:26:14
    at flush (/Users/chrismasters/git/chrismasters/node_modules/ember-cli/node_modules/rsvp/dist/lib/rsvp/asap.js:80:5)
    at process._tickCallback (internal/process/next_tick.js:61:11)

Thanks — it would be great to get this working because the current master branch doesn't seem to inline the asset map data into index.html now.

@krallin
Copy link
Contributor Author

krallin commented Oct 29, 2018

@chrism I don't remember - considering this PR has been stale for a few months at this point, I'm not really investing time in this anymore, so I'm afraid I can't really help you :/

@chrism
Copy link

chrism commented Oct 29, 2018

@krallin completely understand - thanks for getting back to me.

Just out of interest (hope you don't mind me asking) but have you found an alternative approach to solve this problem of fingerprinting?

@krallin
Copy link
Contributor Author

krallin commented Oct 29, 2018

@chrism I'm just using this patch from my branch. We don't use fastboot, though, which is why I can't really help you with that bit; I'm sorry!

@chrism
Copy link

chrism commented Oct 29, 2018

@krallin you've been a big help—i'm just trying to gauge whether to spend time trying to get it to work with Fastboot or not. Thanks again!

@RuslanZavacky
Copy link
Contributor

Sorry guys, I am not moving forward with it, as I wasn't still convinced that thats the best way to go.. as far as I see, the best way is to revert as it was pre-change to move code to vendor.js and upgrade to latest ember. Thats still seems that the only way, that worked well for fastboot and non-fastboot modes.

@RuslanZavacky
Copy link
Contributor

Actually, disregard my comment. Wrongly assumed something different. This PR looks good, but need to check more into fastboot land.

@RuslanZavacky RuslanZavacky merged commit e0c38a0 into adopted-ember-addons:master Oct 29, 2018
@RuslanZavacky
Copy link
Contributor

Thank you @krallin. I will test it out in master and we'll see if something needs some improvements.

@chrism
Copy link

chrism commented Oct 29, 2018

@RuslanZavacky @krallin

Thanks for the info. I'm using (almost) latest Ember 3.4.5 — but as I was unable to get this addon working inline by just installing with ember install ember-cli-ifa and following the instructions I started to look at the issues and PRs.

This PR works for me (with Fastboot disabled), but I'd like the continue using Fastboot

Personally I like the approach of using a meta tag, but I don't really have any strong opinions I'd just like to be able to access the fingerprinted asset URL from within a component.

If you have any advice for a currently stable version working with Fastboot and Ember 2.4.5 I'd go with that.

Thanks for the hard work on this.

@RuslanZavacky
Copy link
Contributor

I've merged this PR but remembered, why I've hesitated. For Fastboot to work with this PR, inline mode should work via global variable. It doesn't seem that there are any other viable options. I think I'll make a inline mode to be the same JSON object as it was before, as I am out of options how to make it work with meta for inline with fastboot. If anyone has better ideas, I'd be more than welcome :)

@chrism
Copy link

chrism commented Oct 29, 2018

@RuslanZavacky thanks very much for looking into this it looks like your PR #44 might address the error message I was getting with Fastboot too.

I hesitate to offer any advice about how it could work with Fastboot as I'd need to really dig into the code, but could using the shoebox be a good option?

It already exists and seems to be a place dedicated to storing data in this way? https://github.com/ember-fastboot/ember-cli-fastboot#the-shoebox.

Just out of interest—do you plan on working on a version which also supports Fastboot as it was before, or awaiting a PR from someone with an implementation?

@chrism
Copy link

chrism commented Oct 30, 2018

Hi again @RuslanZavacky

I've been looking into alternatives to a global variable for the Fastboot code path and think I have a possible solution.

Before I make a PR for an implementation I want to run it by you so I am not wasting my time on this.

There is a working version here: https://github.com/chrism/test-fastboot-config-from-addon to illustrate the concept.

Basically, instead of a global variable I'm taking advantage of the fastbootConfigTree hook for adding Fastboot config from an addon, like this. In my example app I've made a basic in-repo addon like this:

fastbootConfigTree() {
  return {
    [this.app.name]: {
      "asset-map": {
        "foo": "bar",
        "something": "different"
      }
    }
  }
}

And using the meta tag in the head for the non-Fastboot code path, like this:

<meta name="asset-map" content="%7B%22asset-map%22%3A%7B%22foo%22%3A%22bar%22%2C%22something%22%3A%22different%22%7D%7D">

With these two options it means that instead of an issue with document being unable to be used from node we can instead get the hash from config when using Fastboot.

In the test app there is a component with this code:

import Component from '@ember/component';
import { inject as service } from '@ember/service';

export default Component.extend({
  fastboot: service(),

  init() {
    this._super(...arguments);

    let configHash;

    if (this.get('fastboot.isFastBoot')) {
      configHash = FastBoot.config().default;
    } else {
      let rawConfig = document.querySelector("meta[name='asset-map']").content;
      configHash = JSON.parse(decodeURIComponent(rawConfig));
    }

    this.config = configHash['asset-map'].foo;
  }
});

Which seems to work well, for both Fastboot and javascript environments.

There is still some work to investigate whether this will be easily applied to the ifa build process, but i'm confident that it should be OK.

My question to you is—should I spend time creating a PR with this approach for you, would you merge it?

Or should I consider creating a new addon completely.

My personal preference would be to contribute to this project but I don't know your time constraints/commitment to it and completely understand if it isn't a priority for you.

Thanks.

@jbailey4
Copy link

Glad this got in! @RuslanZavacky Any chance this will be released in the next version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants