New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

publishing a package fails if a resource handler modifies files #2488

Closed
Nemo64 opened this Issue Aug 31, 2014 · 7 comments

Comments

Projects
None yet
3 participants
@Nemo64

Nemo64 commented Aug 31, 2014

I build an i18n package which uses a resource-handler to make json out of yml files. This works great, however I included some yml files for tests. When I now try to publish a new version, the bundler tries to publish the json file, not the yml file.

Here is the console output:

/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/dev_bundle/lib/node_modules/fibers/future.js:173
                        throw(ex);
                              ^
Error: ENOENT, no such file or directory '/Users/marco/meteor/nemo64:translator/test/namespace.de_DE.json'
    at Object.fs.statSync (fs.js:689:18)
    at Object.files.copyFile (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/files.js:430:18)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/package-client.js:369:11
    at Array.forEach (native)
    at Function._.each._.forEach (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/dev_bundle/lib/node_modules/underscore/underscore.js:79:11)
    at bundleSource (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/package-client.js:368:5)
    at Object.exports.publishPackage (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/package-client.js:651:28)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/commands-packages.js:271:26
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:186:11
    at _.extend.withValue (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/fiber-helpers.js:112:14)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:183:27
    at _.extend.withValue (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/fiber-helpers.js:112:14)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:181:16
    at _.extend.withValue (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/fiber-helpers.js:112:14)
    at Object.capture (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/buildmessage.js:172:21)
    at main.registerCommand.name [as func] (/Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/commands-packages.js:268:29)
    at /Users/marco/.meteor/packages/meteor-tool/.1.0.26.p03e91++os.osx.x86_64+web.browser+web.cordova/meteor-tool-os.osx.x86_64/tools/main.js:1212:23

If my package is a too big as an example, I could create a minimal test package later.

However: my guess is that the bundler can't handle if the resource-handler changes the name of the file later, can that be?

@glasser

This comment has been minimized.

Member

glasser commented Sep 2, 2014

Yes, please do show a minimal test package, or at the very least a step by step guide to reproducing with your maximal package (clone, check out this commit, run these commands, etc).

https://github.com/meteor/meteor/wiki/Contributing-to-Meteor#reporting-a-bug-in-meteor

Thanks!

@mizzao

This comment has been minimized.

Contributor

mizzao commented Sep 3, 2014

I experienced this as well, glad someone else is reporting it.

You can reproduce this by adding (locally installing) https://github.com/mizzao/meteor-jqueryui to an app, and verifying that it works. Then, try and publish it, and you will get the same error. I currently use the package as a submodule, which skips the publication process: https://github.com/mizzao/CrisisMapping.

This package uses mizzao:build-fetcher to turn .build.json files into bundled source by downloading libraries at build time. As you can see, the publisher tries to look for built files in the raw package contents. Perhaps the reason for this is that the publish code currently treats assets differently from compiled Javascript source, and anything that is not JS or CSS is an asset.

mizzao:jquery-ui$ meteor publish --create
Refreshing package metadata. This may take a moment.
Reading package...
Downloading http://code.jquery.com/ui/1.11.0/jquery-ui.min.js to jquery-ui.min.js
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/jquery-ui.min.css to jquery-ui.css
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png to images/ui-bg_flat_0_aaaaaa_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png to images/ui-bg_flat_75_ffffff_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png to images/ui-bg_glass_55_fbf9ee_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png to images/ui-bg_glass_65_ffffff_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png to images/ui-bg_glass_75_dadada_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png to images/ui-bg_glass_75_e6e6e6_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png to images/ui-bg_glass_95_fef1ec_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png to images/ui-bg_highlight-soft_75_cccccc_1x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_222222_256x240.png to images/ui-icons_222222_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_2e83ff_256x240.png to images/ui-icons_2e83ff_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_454545_256x240.png to images/ui-icons_454545_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_888888_256x240.png to images/ui-icons_888888_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_cd0a0a_256x240.png to images/ui-icons_cd0a0a_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/jquery-ui.min.js to jquery-ui.min.js
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/jquery-ui.min.css to jquery-ui.css
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png to images/ui-bg_flat_0_aaaaaa_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png to images/ui-bg_flat_75_ffffff_40x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png to images/ui-bg_glass_55_fbf9ee_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png to images/ui-bg_glass_65_ffffff_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png to images/ui-bg_glass_75_dadada_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png to images/ui-bg_glass_75_e6e6e6_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png to images/ui-bg_glass_95_fef1ec_1x400.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png to images/ui-bg_highlight-soft_75_cccccc_1x100.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_222222_256x240.png to images/ui-icons_222222_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_2e83ff_256x240.png to images/ui-icons_2e83ff_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_454545_256x240.png to images/ui-icons_454545_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_888888_256x240.png to images/ui-icons_888888_256x240.png
Downloading http://code.jquery.com/ui/1.11.0/themes/smoothness/images/ui-icons_cd0a0a_256x240.png to images/ui-icons_cd0a0a_256x240.png
Bundling source...

/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/lib/node_modules/fibers/future.js:173
                        throw(ex);
                              ^
Error: ENOENT, no such file or directory '/home/mao/projects/CrisisMapping/mizzao:jquery-ui/images/ui-bg_flat_0_aaaaaa_40x100.png'
    at Object.fs.statSync (fs.js:689:18)
    at Object.files.copyFile (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/files.js:430:18)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/package-client.js:369:11
    at Array.forEach (native)
    at Function._.each._.forEach (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/lib/node_modules/underscore/underscore.js:79:11)
    at bundleSource (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/package-client.js:368:5)
    at Object.exports.publishPackage (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/package-client.js:651:28)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/commands-packages.js:271:26
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:186:11
    at _.extend.withValue (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/fiber-helpers.js:112:14)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:183:27
    at _.extend.withValue (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/fiber-helpers.js:112:14)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:181:16
    at _.extend.withValue (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/fiber-helpers.js:112:14)
    at Object.capture (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/buildmessage.js:172:21)
    at main.registerCommand.name [as func] (/home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/commands-packages.js:268:29)
    at /home/mao/.meteor/packages/meteor-tool/.1.0.26.19y7v10++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/tools/main.js:1212:23

You will need to jump through the hoops as indicated in #2392 in order to do the publish, though.

As I mentioned in #2506, the build system seems to be getting confused by raw files and built files. In cases where files are generated as part of the build, it isn't looking in the right place for them.

@glasser

This comment has been minimized.

Member

glasser commented Sep 3, 2014

Thanks, fixed. Should get this into 0.9.2 if not 0.9.1.

glasser added a commit that referenced this issue Sep 3, 2014

Assets added via a plugin are not source files
Fixes #2488.

Assets added via "no handler" already have their source files added to
`sources` via the `sources.push(relPath)` in the sourceItems loop, and
assets added via a handler should not appear as source files themselves
--- the file that triggered the handler should!

This fixes packages like `mizzao:build-fetcher` which add static
assets via a plugin.
@glasser

This comment has been minimized.

Member

glasser commented Sep 4, 2014

OK, 0.9.1 is pretty tied down, so it'll go in 0.9.2. Before then, feel free to run your publish command from git or something (or if you really want I can publish a release that fixes just this). This bug only affects publish.

@mizzao

This comment has been minimized.

Contributor

mizzao commented Sep 4, 2014

Yep, that's what I will do. Thanks for the quick fix.

@mizzao

This comment has been minimized.

Contributor

mizzao commented Sep 4, 2014

@glasser I'd like to re-publish, but looks like the commit with that fix just hangs when trying to publish:

mizzao:jquery-ui$ ~/projects/meteor/meteor publish --create
Your dependency kit is out of date. I will download the new one.
######################################################################## 100.0%
Installed dependency kit v0.3.50 in dev_bundle.

ddp: updating npm dependencies -- faye-websocket, sockjs...
Refreshing package metadata. This may take a moment.

Can't Ctrl-C out of this either, suggesting it may be a bug.

EDIT: It published successfully after forcibly killing the shell and re-running. So this bug is fixed, although there might be another one which is probably unimportant.

mizzao added a commit to mizzao/meteor-jqueryui that referenced this issue Sep 4, 2014

mizzao added a commit to mizzao/meteor-openlayers that referenced this issue Sep 4, 2014

Update package code now that meteor/meteor#2488 is resolved
Still awaiting a proper version for publication pending the discussion in meteor/meteor#2443.
@glasser

This comment has been minimized.

Member

glasser commented Sep 4, 2014

Not surprised if you saw a slow sync tonight; I moved all the package URLs to CloudFront CDN which basically meant that the whole metadata file needed to be resynced (our current sync protocol is pretty simplistic). Surprised that ctrl-c didn't work though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment