publish-for-arch broken in 1.0.4 (for packages whose names contain colons)

[mizzao]

On all archs:

build:~ meteor$ meteor publish-for-arch mizzao:sharejs@0.7.2
Malformed source tarball

Originally posted here: https://forums.meteor.com/t/trying-to-publish-for-arch-in-meteor-1-0-4-malformed-source-tarball/1369

[mizzao]

I found a workaround to requiring binary code and took out the npm package that has binary references. But I still get this message:

Published mizzao:sharejs@0.7.3.               

This package contains binary code and must be built on multiple architectures.

You can access Meteor provided build machines, pre-configured to support older versions of MacOS and Linux, by running:
  meteor admin get-machine os.osx.x86_64      
  meteor admin get-machine os.linux.x86_64    
  meteor admin get-machine os.linux.x86_32    

On each machine, run:                         

  meteor publish-for-arch mizzao:sharejs@0.7.3

For more information on binary ABIs and consistent builds, see:
  https://github.com/meteor/meteor/wiki/Build-Machines

Is it possible that Meteor "remembers" a package as being binary from older versions, even if newer ones aren't?

[glasser]

Hmm, this might just be a bug in some of the Windows compatibility code:

  // XXX Factor out with packageClient.bundleSource so that we don't
  // have knowledge of the tarball structure in two places.
  var packageDir = files.pathJoin(sourcePath, colonConverter.convert(name));
  if (! files.exists(packageDir)) {
    Console.error('Malformed source tarball');
    return 1;
  }

[glasser]

Aha, this was already fixed on devel: 89f86b1

@avital FYI, this fix that you did on devel a few days ago was to something that was broken in 1.0.4. Were there any other similar fixes you made since release-1.0.4 forked that should have gotten released?

[glasser]

Fixed in 1.0.4.1. Thanks for the report @mizzao !

[mizzao]

@glasser: So what do I need to use this fix? I'm still getting the same error.

Do I need to re-publish a new version of the package using 1.0.4.1?

build:~ meteor$ meteor --version              
Meteor 1.0.4.1
build:~ meteor$ meteor publish-for-arch mizzao:sharejs@0.7.3
Malformed source tarball 

[glasser]

Yes, publish-for-arch makes sure that it uses the same release that the package was published with.

In weird cases like this you can also carefully check out the same commit that you published the package with and run meteor publish --existing-version. But it is then your responsibility to make sure it's the same source!

[mizzao]

So what happens if you publish --existing-version with different source? Just curious.

[mizzao]

Even using that same commit does not seem to work. On my machine:

$ meteor publish --existing-version
Published mizzao:sharejs@0.7.3.               

WARNING: Your package contains binary code.   

On the build machine:

$ meteor publish-for-arch mizzao:sharejs@0.7.3
Malformed source tarball                      

I can tell it's not using 1.0.4.1 because it said Downloading meteor-tool@1.0.42 or whatever, which is the meteor-tool corresponding to 1.0.4 - 1.0.43 is 1.0.4.1.

[glasser]

Sorry, I mean run that command instead of publish-for-arch.

[mizzao]

Not sure what command you mean. I've run that command on my local machine and it works. But trying publish-for-arch on the build machines afterward produces the same error.

[glasser]

meteor publish --existing-version on the build machines.

In a (not quite literal) sense, publish-for-arch is syntactic sugar for "download the source tarball, make sure you're running the same release, and run publish --existing-version".

[mizzao]

Thanks, that worked.

If that's the case, why does meteor publish on my local machine still require a publish-for-arch of the same architecture on the build machines? It didn't used to, but now we have to do all three?

[glasser]

To encourage people to use build machines which have a carefully chosen architecture (glibc, etc) which we've chosen to maximize the span of machines that the binaries will work on.