publishing a package fails if a resource handler modifies files #2488

Nemo64 · Aug 31, 2014

I build an i18n package which uses a resource-handler to make json out of yml files. This works great, however I included some yml files for tests. When I now try to publish a new version, the bundler tries to publish the json file, not the yml file.

Here is the console output:

/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/dev_bundle/lib/node_modules/fibers/future.js:173
                        throw(ex);
                              ^
Error: ENOENT, no such file or directory '/Users/marco/meteor/nemo64:translator/test/namespace.de_DE.json'
    at Object.fs.statSync (fs.js:689:18)
    at Object.files.copyFile (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/files.js:430:18)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/package-client.js:369:11
    at Array.forEach (native)
    at Function._.each._.forEach (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/dev_bundle/lib/node_modules/underscore/underscore.js:79:11)
    at bundleSource (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/package-client.js:368:5)
    at Object.exports.publishPackage (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/package-client.js:651:28)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/commands-packages.js:271:26
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:186:11
    at _.extend.withValue (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/fiber-helpers.js:112:14)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:183:27
    at _.extend.withValue (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/fiber-helpers.js:112:14)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:181:16
    at _.extend.withValue (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/fiber-helpers.js:112:14)
    at Object.capture (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:172:21)
    at main.registerCommand.name [as func] (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/commands-packages.js:268:29)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/main.js:1212:23

If my package is a too big as an example, I could create a minimal test package later.

However: my guess is that the bundler can't handle if the resource-handler changes the name of the file later, can that be?

glasser · Sep 2, 2014

Yes, please do show a minimal test package, or at the very least a step by step guide to reproducing with your maximal package (clone, check out this commit, run these commands, etc).

https://github.com/meteor/meteor/wiki/Contributing-to-Meteor#reporting-a-bug-in-meteor

Thanks!

mizzao · Sep 3, 2014

I experienced this as well, glad someone else is reporting it.

You can reproduce this by adding (locally installing) https://github.com/mizzao/meteor-jqueryui to an app, and verifying that it works. Then, try and publish it, and you will get the same error. I currently use the package as a submodule, which skips the publication process: https://github.com/mizzao/CrisisMapping.

This package uses mizzao:build-fetcher to turn .build.json files into bundled source by downloading libraries at build time. As you can see, the publisher tries to look for built files in the raw package contents. Perhaps the reason for this is that the publish code currently treats assets differently from compiled Javascript source, and anything that is not JS or CSS is an asset.

mizzao:jquery-ui$ meteor publish --create
Refreshing package metadata. This may take a moment.
Reading package...
Downloading http://code.jquery.com/ui/1.11.0/jquery-ui.min.js to jquery-ui.min.js
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/jquery-ui.min.css to jquery-ui.css
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png to images/ui-bg_flat_0_aaaaaa_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png to images/ui-bg_flat_75_ffffff_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png to images/ui-bg_glass_55_fbf9ee_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png to images/ui-bg_glass_65_ffffff_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png to images/ui-bg_glass_75_dadada_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png to images/ui-bg_glass_75_e6e6e6_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png to images/ui-bg_glass_95_fef1ec_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png to images/ui-bg_highlight-soft_75_cccccc_1x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_222222_256x240.png to images/ui-icons_222222_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_2e83ff_256x240.png to images/ui-icons_2e83ff_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_454545_256x240.png to images/ui-icons_454545_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_888888_256x240.png to images/ui-icons_888888_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_cd0a0a_256x240.png to images/ui-icons_cd0a0a_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/jquery-ui.min.js to jquery-ui.min.js
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/jquery-ui.min.css to jquery-ui.css
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png to images/ui-bg_flat_0_aaaaaa_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png to images/ui-bg_flat_75_ffffff_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png to images/ui-bg_glass_55_fbf9ee_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png to images/ui-bg_glass_65_ffffff_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png to images/ui-bg_glass_75_dadada_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png to images/ui-bg_glass_75_e6e6e6_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png to images/ui-bg_glass_95_fef1ec_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png to images/ui-bg_highlight-soft_75_cccccc_1x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_222222_256x240.png to images/ui-icons_222222_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_2e83ff_256x240.png to images/ui-icons_2e83ff_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_454545_256x240.png to images/ui-icons_454545_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_888888_256x240.png to images/ui-icons_888888_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_cd0a0a_256x240.png to images/ui-icons_cd0a0a_256x240.png
Bundling source...

/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/lib/node_modules/fibers/future.js:173
                        throw(ex);
                              ^
Error: ENOENT, no such file or directory '/home/mao/projects/CrisisMapping/mizzao:jquery-ui/images/ui-bg_flat_0_aaaaaa_40x100.png'
    at Object.fs.statSync (fs.js:689:18)
    at Object.files.copyFile (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/files.js:430:18)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/package-client.js:369:11
    at Array.forEach (native)
    at Function._.each._.forEach (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/lib/node_modules/underscore/underscore.js:79:11)
    at bundleSource (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/package-client.js:368:5)
    at Object.exports.publishPackage (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/package-client.js:651:28)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/commands-packages.js:271:26
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:186:11
    at _.extend.withValue (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/fiber-helpers.js:112:14)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:183:27
    at _.extend.withValue (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/fiber-helpers.js:112:14)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:181:16
    at _.extend.withValue (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/fiber-helpers.js:112:14)
    at Object.capture (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:172:21)
    at main.registerCommand.name [as func] (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/commands-packages.js:268:29)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/main.js:1212:23

You will need to jump through the hoops as indicated in #2392 in order to do the publish, though.

As I mentioned in #2506, the build system seems to be getting confused by raw files and built files. In cases where files are generated as part of the build, it isn't looking in the right place for them.

glasser · Sep 3, 2014

Thanks, fixed. Should get this into 0.9.2 if not 0.9.1.

glasser · Sep 4, 2014

OK, 0.9.1 is pretty tied down, so it'll go in 0.9.2. Before then, feel free to run your publish command from git or something (or if you really want I can publish a release that fixes just this). This bug only affects publish.

mizzao · Sep 4, 2014

Yep, that's what I will do. Thanks for the quick fix.

mizzao · Sep 4, 2014

@glasser I'd like to re-publish, but looks like the commit with that fix just hangs when trying to publish:

mizzao:jquery-ui$ ~/projects/meteor/meteor publish --create
Your dependency kit is out of date. I will download the new one.
######################################################################## 100.0%
Installed dependency kit v0.3.50 in dev_bundle.

ddp: updating npm dependencies -- faye-websocket, sockjs...
Refreshing package metadata. This may take a moment.

Can't Ctrl-C out of this either, suggesting it may be a bug.

EDIT: It published successfully after forcibly killing the shell and re-running. So this bug is fixed, although there might be another one which is probably unimportant.

glasser · Sep 4, 2014

Not surprised if you saw a slow sync tonight; I moved all the package URLs to CloudFront CDN which basically meant that the whole metadata file needed to be resynced (our current sync protocol is pretty simplistic). Surprised that ctrl-c didn't work though.

mizzao referenced this issue Sep 3, 2014
Closed
Packages need a principled way to access their files (and their build contents) #2506

This was referenced Sep 3, 2014

Closed

Incompatible with 0.9 mizzao/meteor-sharejs#24

Closed

Meteor 0.9 can't publish build-fetcher packages mizzao/meteor-build-fetcher#1

Closed

Meteor 0.9.0 mizzao/meteor-jqueryui#5

glasser closed this in 5b8504e Sep 3, 2014

mizzao added a commit to mizzao/meteor-jqueryui that referenced this issue Sep 4, 2014

Publish an updated package since meteor/meteor#2488 is now resolved.
Fixes #5.

f31ce0a

mizzao added a commit to mizzao/meteor-openlayers that referenced this issue Sep 4, 2014

Update package code now that meteor/meteor#2488 is resolved
Still awaiting a proper version for publication pending the discussion in meteor/meteor#2443.

453fcc8

Nemo64 referenced this issue Sep 5, 2014
Closed
Publishing the latest version #7

gerwinbrunner referenced this issue Sep 11, 2014
Closed
ENOENT, no such file or directory #5

meteor/meteor

publishing a package fails if a resource handler modifies files #2488

publishing a package fails if a resource handler modifies files #2488

Nemo64 commented Aug 31, 2014

This comment has been minimized.

glasser commented Sep 2, 2014

mizzao referenced this issue Sep 3, 2014

Packages need a principled way to access their files (and their build contents) #2506

This comment has been minimized.

mizzao commented Sep 3, 2014

This was referenced Sep 3, 2014

Incompatible with 0.9 mizzao/meteor-sharejs#24

Meteor 0.9 can't publish build-fetcher packages mizzao/meteor-build-fetcher#1

Meteor 0.9.0 mizzao/meteor-jqueryui#5

glasser closed this in `5b8504e` Sep 3, 2014

This comment has been minimized.

glasser commented Sep 3, 2014

glasser added a commit that referenced this issue Sep 3, 2014

This comment has been minimized.

glasser commented Sep 4, 2014

This comment has been minimized.

mizzao commented Sep 4, 2014

This comment has been minimized.

mizzao commented Sep 4, 2014

mizzao added a commit to mizzao/meteor-jqueryui that referenced this issue Sep 4, 2014

mizzao added a commit to mizzao/meteor-openlayers that referenced this issue Sep 4, 2014

This comment has been minimized.

glasser commented Sep 4, 2014

Nemo64 referenced this issue Sep 5, 2014

Publishing the latest version #7

gerwinbrunner referenced this issue Sep 11, 2014

ENOENT, no such file or directory #5

meteor/meteor

Join GitHub today

publishing a package fails if a resource handler modifies files #2488

Comments

Nemo64 commented Aug 31, 2014

This comment has been minimized.

glasser commented Sep 2, 2014

mizzao referenced this issue Sep 3, 2014

Packages need a principled way to access their files (and their build contents) #2506

This comment has been minimized.

mizzao commented Sep 3, 2014

This was referenced Sep 3, 2014

Incompatible with 0.9 mizzao/meteor-sharejs#24

Meteor 0.9 can't publish build-fetcher packages mizzao/meteor-build-fetcher#1

Meteor 0.9.0 mizzao/meteor-jqueryui#5

glasser closed this in 5b8504e Sep 3, 2014

This comment has been minimized.

glasser commented Sep 3, 2014

glasser added a commit that referenced this issue Sep 3, 2014

This comment has been minimized.

glasser commented Sep 4, 2014

This comment has been minimized.

mizzao commented Sep 4, 2014

This comment has been minimized.

mizzao commented Sep 4, 2014

mizzao added a commit to mizzao/meteor-jqueryui that referenced this issue Sep 4, 2014

mizzao added a commit to mizzao/meteor-openlayers that referenced this issue Sep 4, 2014

This comment has been minimized.

glasser commented Sep 4, 2014

Nemo64 referenced this issue Sep 5, 2014

Publishing the latest version #7

gerwinbrunner referenced this issue Sep 11, 2014

ENOENT, no such file or directory #5

glasser closed this in `5b8504e` Sep 3, 2014