Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Official Meteor integration directly from 3rd party libraries #14

Open
dandv opened this issue Nov 24, 2014 · 310 comments

Comments

Projects
None yet
@dandv
Copy link
Member

commented Nov 24, 2014

If you're looking for a popular 3rd party library on Atmosphere, chances are you'll find several wrappers, in various states of maintenance. To avoid this problem, several Meteor community members have started to provide authoritative, high-quality, Meteor integrations of 3rd party libraries, complete with tests and grunt scripts. These are forks of the library rather than Git submodules, because forks allow modifying the source to support Meteor (if necessary) and one can still pull upstream changes without having to re-apply the integration changes.

These forks live in the MeteorPackaging GitHub organization. This org was created so that multiple developers can collaborate on maintaining an integration in a centralized location. Now that Meteor's Package API is stable, we can submit PRs to popular JS library maintainers to add official Meteor integration. Some maintainers will accept the PR - great, we have integration straight from the source, with same-day updates on Atmosphere. Some won't accept the PR, in which case we continue with the MeteorPackaging fork.

I started this effort in November 2014, spoke about it at DevShop SF, and here's where we are now.

Going forward

As of September 2015, see http://autopublish.meteor.com. It's a system authored by @splendido that automatically publishes new versions of a wrapper package automatically on Atmosphere (once you've registered your repo with it), as soon as the upstream package has a new release. See the How it works page. For an example repo, see https://github.com/MeteorPackaging/numeraljs-core-wrapper.

The instructions from here on may be somewhat outdated.

Contribute (minimal effort)

  1. If you're working on integrating a 3rd party library with Meteor, please post here. I have also reserved Meteor Developer (Atmosphere) organization namespaces for many libraries; ask if you need one.

  2. If you have created an MD organization for a library, please let us know. Namespace is precious. Currently we're looking for:

    Who has these?

Popular libraries with official Meteor support

PR acceptance pending

  • 🕐 Mousetrap - keyboard events library [pending] [grunt, export file]
  • 🕙 Font Awesome - [acknowledged]. Great example of Meteor resolving CSS URL paths correctly.
  • ☑️ Moment [merged in develop, published]
  • 🕐 Isotope [pending] [grunt, export file]
  • 🕤 jsPlumb [pending, maintainer was in a pissed off mood today]
  • 🕗 Hammer - multi-touch library, ~10k GitHub stars [pending, author has changed jobs] [grunt, export file]. Use it as a template.

Refused to accept the PR

How to do this

Overview

  1. Post here asking @dandv and @splendido to add you to the MeteorPackaging organization and mention the library/libraries you want to package.
  2. We'll fork the library's repo into MeteorPackacing and grant you access
  3. Add the Meteor integration files,
    • package.js
    • tests
    • gruntfile (or makefile or equivalent) entries to run meteor publish
  4. publish the package to Atmosphere yourself (this is very important)
  5. submit a PR with the changes to the original library author (optional), or just notify them to get some exposure (they may offer to tweet about the integration or otherwise announce it)

Below are the detailed steps.

Get the integration working before even contacting the maintainer

  1. Fork <repo_url_of_the_library>

  2. git clone <your_fork_url>

  3. cd <repo_directory>

  4. git remote add upstream <repo_url_of_the_library>

  5. git fetch upstream

  6. Read through CONTRIBUTING.md or README.md and figure out which branch the original author wants pull requests against. Some can be very finicky.

  7. git checkout -b meteor-integration upstream/that_branch

  8. Create a meteor directory in the root of the project and copy there the Hammer.js Meteor package files, or the Font Awesome Meteor package files, depending on whether you integrate a library or CSS/image/font resources.

  9. Go through every file and make changes to reflect the actual library you're working on.

    • in particular, give a GOOD summary in package.js (for SEO on Atmosphere), instead of the library's tagline. Example:

      Mousetrap is a simple library for handling keyboard shortcuts in Javascript.

      For Atmosphere, a repository of JavaScript libraries, this is suboptimal: it repeats the library name, and it says "library" and "JavaScript" (well duh). You only have 100 characters. Don't waste them. Better:

      "Mousetrap (official): bind, trigger and handle keyboard events on keys, combinations and sequences"

  10. Export the library global(s)

    1. If the package is a jQuery package and adds itself to $, there's nothing to export.

    2. Otherwise, add a meteor/export.js file like this:

      /*global Hammer:true*/  // Meteor creates a file-scope global for exporting. This comment prevents a potential JSHint warning.
      Hammer = window.Hammer;
      delete window.Hammer;
  11. Add a package test file, at least checking if the library instantiates correctly.

  12. You want to make it as painless as possible for the maintainer to accept the PR. If there's a build process they use to publish, it would be great if you could add to it. Here's a gruntfile patch for Meteor and a Makefile patch (used for d3). If their README mentions Component/Bower/Npm, add a section mentioning Meteor (example).

  13. Run the tests using the build process, e.g. grunt meteor-test. If you want to run tests manually, execute this:

    cp meteor/package.js .
    spacejam --mongo-url mongodb:// test-packages ./
    

Pick a namespace and publish the package under it

Please be very careful with this, as renaming packages is impossible.

Log into your Meteor Developer account and create an organization that represents the package. Most often it will be the name of library, without any special characters (no -, .). No need to include the JS part, since we're always dealing with JavaScript libraries.

If the organization name is not available, contact me, @dandv. I've registered a bunch of them, so potentially evil individuals don't :)

  1. meteor publish the package to Atmosphere for the first time. This prevents the maintainer from running into an error that the package doesn't exist when they publish (meteor would complain about a --create flag).

  2. After you've ensured publishing works (namespace is available, the name in package.js is valid etc.), push to your git fork:

    git commit
    git push origin meteor-integration
    
  3. Congratulations! Reply to this issue and add the link to the package's Atmosphere page.

Now you can go to the GitHub page of the library and issue a pull request!

Contact the library maintainers

Here is the recommended PR message.

See d3 and Font Awesome above for addressing objections.

Contact the maintainers of other Meteor wrapper libraries

  1. Ask them to deprecate the library by creating a PR overwriting the README.md of their package with the text

    This package is now deprecated. Please use the official Meteor integration package:
    
    http://atmospherejs.com/<org>/<package>
    

    The commit message should be "Deprecating package in favor of official integration".

    The name of the PR issue (after you click "Create pull request) should be "Kind request to deprecate package in favor of official integration".

    The text of the PR should be similar to this:

    Hi X,
    
    Thanks for publishing this package. In the meantime an
    [official Meteor integration](https://atmospherejs.com/<namespace>/<package>)
    has become available.
    
    Would you like to support the official package?
    
    If so, you can hide [your package](<atmosphere_link_to_their_package>)
    from Atmosphere searches, while still keeping it installable by apps and dependent
    packages, by running the following command:
    
        meteor admin set-unmigrated <their_full:package_name>
    
    Thank you,
    The [Meteor official integations team](https://github.com/raix/Meteor-community-discussions/issues/14)
    
  2. Go to the Atmosphere page of their package and flag it anyway. Ideally, get others to flag the wrapper package. This is important because meteor doesn't warn the user when running meteor add on a deprecated package, so we hope users will check on Atmosphere. However, each package needs several flags from different users, in order to show as flagged globally.

TODO

  • figure out namespacing exactly. Ideally we'd have an MD organization with the library name, and ask the library maintainers to create MD accounts reflecting their GitHub usernames. Problem:
    • sometimes the names are taken, e.g. moment (pinging @estark37)
    • sometimes the maintainer uses their name as the organization, and you can't have an org and individual with the same name on MD - e.g. RubaXa/Sortable
    • rarely, several libraries have the same name, e.g. RubaXa/Sortable and voidberg/html5sortable (contrived example, need a better one)
@gadicc

This comment has been minimized.

Copy link

commented Nov 24, 2014

@dandv, re 2, yeah, if there's no module framework found it exports to window. Which is bad for a Meteor package. It's easy to get around, but unfortunately requires extra files. e.g. chuangbo/meteor-marked@e55d9c2. Maybe part of this issue should be to either support that type of wrap policy natively or allow a { wrap: 'exports' } or something in addFiles.

@dandv

This comment has been minimized.

Copy link
Member Author

commented Nov 24, 2014

@gadicc

This comment has been minimized.

Copy link

commented Nov 24, 2014

@dandv oh yeah sure, if you have access to the main package itself... forgot about that :> i mostly wrap git submodules, so don't like to touch the actual code. what you describe would work perfectly of course. have a vague memory though that Package name was used by something else too, not sure... it would solve our problems though :>

@PEM--

This comment has been minimized.

Copy link

commented Nov 24, 2014

👏

@dandv

This comment has been minimized.

Copy link
Member Author

commented Nov 25, 2014

Today I'm happy to present the official d3 integration.

Please log into Atmosphere with your MD account and,

Thanks!

@raix

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2014

@dandv cool - is it the d3js or meteor that maintains the package?

@dandv

This comment has been minimized.

Copy link
Member Author

commented Nov 25, 2014

@raix: the best permissions combo I've found so far was to create an organization for the library, add myself to it so I can publish the package (helps to remove barriers for the maintainer and make it as easy as possible for them), then ask them to sign up for an MD at meteor.com and tell me their username. Really a 30-second process. Then I'll add their username to the org so they can meteor publish as well.

@raix

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2014

@dandv that makes sense - it would be nice if possible to add the package.js in their repo (the underscore already got 3 package configs... would be nice with just one...)

@dandv

This comment has been minimized.

Copy link
Member Author

commented Nov 25, 2014

Do you mean https://github.com/jashkenas/underscore already has bower.json, component.json and package.json ? It would indeed be nice to have just one, but I'm not sure how that would work with Bower and Component.

A Meteor integration for component would look very much like the d3 integration. I wanted to keep all meteor-related files in the same directory, and package.js is not in the root directory of the project because it sometimes conflicts with Dojo's package.js.

@gadicc

This comment has been minimized.

Copy link

commented Nov 27, 2014

@dandv just wanted to commend you on your hard work here, and particularly, your vision... to be honest the thought of getting 3rd party libraries to directly publish Meteor had never occurred to me before. you're doing a stellar job, and I can see how it's been infectious.

this is good for so many reasons... i was always against wrapped packages being in atmosphere when they offered nothing new on their own (you and I have discussed this before). but this approach is a real pleasure. latest official release guaranteed, and it means that even if I do an integration package, that say, offers templates etc to use the lib, I don't need to release for every new version and an app developer can still use my package and the latest version of the lib. it's awesome :)

also i feel you've done a great job of instilling meteor's place in the JS community... I'm starting to feel more like a 1st class citizen now :>

@raix

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2014

@dandv should we hit famous?
EDIT: it would be nice if possible to have famous split up in alot of small packages - eg. have a famous:core that included famous:core.Surface package etc. Getting a more finegrained approach.

@splendido

This comment has been minimized.

Copy link
Member

commented Nov 27, 2014

@gadicc: are you chasing me?! ;-)

Since I was already in contact with Semantic-UI maintainers I decided to get their opinion before starting chasing meteor wrapper maintainers asking to deprecate packages.
I felt the request might be even more robust and acceptable by MD having the official maintainers behind it...

In the discussion @dandv already linked we're also trying to figure out which is the more appropriate style language to be used as the main official package: CSS, LESS, SASS, etc.
So its not only the way/script to be used to publish the package (@dandv is doing a great job about this!) but also how to select the proper meteor way.
I think it would be also great to have your opinion about this.
Do you think we should open another issue on this repo?

Tnx,
Luca

@gadicc

This comment has been minimized.

Copy link

commented Nov 27, 2014

@splendido, i'm following everyone doing good work in the community :> i'll wait and see how the discussion plays out, I'm less knowledgeable about styling formats. but back to the JS discussion, which I guess is related...

@raix, personally i find it annoying to both use and maintain split-up packages like that. also, it's quite likely they might want e.g. famous.views.GridLayout and not everything else in famous.views.*.

It's very clear where this is going :) Packages should include everything, and then you are going to rework famono to work directly on installed packages too :) It's good that you already got familiar with publishing meteor releases :> This is no small feat, and I think also, a revolutionary gamechanger, but your track record is impeccable, so expecting the proof-of-concept tomorrow sometime :>

@raix

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2014

@gadicc Haha, right on

@mitar

This comment has been minimized.

Copy link

commented Oct 25, 2015

Hm, velocity package is not updated on Atmosphere? I thought it should be updating automatically?

@trusktr

This comment has been minimized.

Copy link

commented Oct 25, 2015

Hey all, good news: ES6 modules are work in progress, so soon enough it'll be easy to import many of these libraries without needing to make wrapper packages. meteor/meteor#5475

@ccorcos

This comment has been minimized.

Copy link

commented Oct 28, 2015

I'd highly recommend just using webpack:webpack

@raohuaming

This comment has been minimized.

Copy link

commented Nov 19, 2015

@dandv @splendido, I am considering to wrap mathquill, a library to make typing math on web as easily as writing, so could you please add me to MeteorPackaging organization ?

Please refer to https://github.com/raohuaming/mathquill/tree/meteor-integration, and see if that looks good. thx a lot

@zurbchris

This comment has been minimized.

Copy link

commented Nov 20, 2015

Hey guys, anyone interested in helping us get Foundation for Sites 6 ready to rock on Meteor? We've never set up a Meteor package before and we want to make sure we do it properly, with auto updates, etc.

@juliancwirko

This comment has been minimized.

Copy link

commented Nov 20, 2015

@zurbchris Hi, I think that now the 'zurb:foundation-sites' package should work well. You need to remember about package.js file - versions, the scss file list in 'api.addFiles' need to be up to date. I think there is no need for this kind of auto packaging. You have your own 'Zurb' organisation on atmosphere so everything is ready to go. You just need to change the version in the package.js file and 'meteor publish' when the Foundation version changes. (of course if there will be no other scss file structure changes) There is even separate docs for it which is cool. It looks like real maintained package and not some automated build. I could keep an eye on it if you want. I maintain Foundation 5 package so there is no problem for me.

and congratulations, Foundation 6 is awesome :)

@zurbchris

This comment has been minimized.

Copy link

commented Nov 20, 2015

Thanks @juliancwirko we hope everyone likes it. I just want to make sure we are doing what we need to do to give the Meteor community access to the framework. Though we may ask for assistance in doing so from time to time 😄
Let me know if there's anything we can improve on the package.
Cheers,

  • Chris
@splendido

This comment has been minimized.

Copy link
Member

commented Nov 22, 2015

@zurbchris just in case you don't want to manually run meteor publish at every new release, or just simply don't want to run it on your servers, there's autopublish.meteor.com that can do it for you!
(Disclaimer: I've written it...)

It's working just fine for a number of packages, just let me know if you're interested to get more details.
I'll be happy to help you setting things up (nothing special to be done though...).

Cheers,
Luca

@dmihal

This comment has been minimized.

Copy link

commented Nov 27, 2015

@dandv and @splendido would it possible to be added to the MeteorPackaging organization? I'm currently working on packaging gitgraph.js, and I'm sure I'll be able to contribute more packages going forward! Thanks.

@mizzao

This comment has been minimized.

Copy link

commented Nov 30, 2015

I recently happened upon the nice demo at https://deck-of-cards.js.org/ ; since I wanted to use cards in a Meteor app I started a PR at deck-of-cards/deck-of-cards#26 and it would be nice to be able to transfer my fork into MeteorPackaging.

This would be very nice for creating card games and other fun stuff.

@ayhid

This comment has been minimized.

Copy link

commented Dec 5, 2015

@dandv , @splendido I am working on a package for seiyria/bootstrap-slider, could you please add me to the MeteorPackaging organization. It will be my first contribution and hope not the last.

@maxenceC

This comment has been minimized.

Copy link

commented Dec 10, 2015

@dandv , @splendido , Can you please add me to the MeteorPackaging organization ? I want to package those librairies :

Thank you !

@mitar

This comment has been minimized.

Copy link

commented Dec 10, 2015

Oh yes, I was just yesterday looking into packaging simple-peer, but then I discovered that it does not support MediaStream on the server, so I gave up.

@maxenceC

This comment has been minimized.

Copy link

commented Dec 10, 2015

@mitar :

but then I discovered that it does not support MediaStream on the server, so I gave up.

Sorry, but I don't really get it : Isn't simple-peer only client-sided ? Why should a meteor wrapper for simple-peer contains server code ? ( I might not understand at all what you meant).

In order to not pollute this thread with messages about a simple-peer wrapper for meteor, how can I send you messages ? If you have some time to spare, I'll might have some questions ? (As you've already tried to create a package for simple-peer).

@mitar

This comment has been minimized.

Copy link

commented Dec 10, 2015

No no! The beauty of simple-peer is that it works in node.js as well! And this allows a complete new programming paradigm, namely that you can simply use WebRTC for all data transmissions, so server is then only one peer, and you can have client-server, server-server, client-client communications, all with same technology. Even more, connections can be based on SCTP, have multi-path routing, you can have data packets which are reliable or unreliable, you can even have backpressure support (imagine that you could push back on Meteor publish because client cannot handle observes as quickly as server is pushing). So yea, that is the future. No more Websockets but simply one transport for everything. Where you can publish directly between clients as well. Crazy stuff.

Anyway, I think if you really want to package this you should make sure it works both on client and server side. This is how Meteor packages are normally packaged. With both client and server support. And especially for this package which has support for both out of the box. That could be a great replacement for meteor-streams. Sadly, it does not have support for MediaStreams which what I needed.

I think the best would be to open a new ticket here in this repository and we can have discussion there. I do not have time to help you much though. No time.

@mitar

This comment has been minimized.

Copy link

commented Dec 10, 2015

Opened #77. So yea, I think that WebRTC in Meteor can be quite a deal and a big paradigm shift.

@maxenceC

This comment has been minimized.

Copy link

commented Dec 10, 2015

@mitar

Thank you for your answer ! Helped me a lot understanding the exact point of simple-peer

@mperkh

This comment has been minimized.

Copy link

commented Dec 15, 2015

@dandv , @splendido , Can you please add me to the MeteorPackaging organization ? I want to package this library:

Here is my work so far:

Thank you !

@PolGuixe

This comment has been minimized.

Copy link

commented Dec 29, 2015

@dandv , @splendido I have packaged Clipper.js.
Work done so far:
*https://atmospherejs.com/polguixe/meteor-clipperjs
*https://github.com/PolGuixe/meteor-clipperjs/

I want to do it the right way, however the library is hosted in Sourceforge and not in Github. Will it be an issue?

@splendido

This comment has been minimized.

Copy link
Member

commented Dec 30, 2015

Hey guys,
Meteor 1.3 with support for NPM packages will be released soon.
I suggest to use wrapper packages as local ones for now and see what happens in the next weeks.

It might be that we won't need wrapper packages and autopublish.meteor.com anymore!

@zimme

This comment has been minimized.

Copy link
Member

commented Dec 30, 2015

autopublish is pretty nice for pure meteor packages too, but it's not the end of the world if I don't have it =)

Some "wrapper" packages are still gonna be needed, for example https://github.com/rzymek/meteor-fullcalendar.

However these kinds of package could just have weak dependencies too, but if there are packages that wrap npm modules with meteor specific code, I don't think you can have weak dependencies on npm packages?

@zimme

This comment has been minimized.

Copy link
Member

commented Dec 30, 2015

Also client-side only wrappers?

i.e. non-npm

@splendido

This comment has been minimized.

Copy link
Member

commented Dec 30, 2015

Honestly I still had no time to look well into 1.3... :-(

@mitar

This comment has been minimized.

Copy link

commented Aug 5, 2016

Does autopublish still work? Dogfalo/materialize#3057

@zimme

This comment has been minimized.

Copy link
Member

commented Aug 5, 2016

I believe autopublish has been decommissioned

@dandv

This comment has been minimized.

Copy link
Member Author

commented Aug 5, 2016

Autopublish no longer works because meteor.com hosting was retired.

The guidance is to use npm packages directly with 1.3+, but the problem is this:

percolatestudio/atmosphere#403

@splendido

This comment has been minimized.

Copy link
Member

commented Aug 5, 2016

yes guys, no more autopublish.meteor.com.
When they made the switch to *.meteorapp.com I got no offer for a free container to move autopublish onto Galaxy...
I mentioned this a couple times, but got no replies. This was a little sad...

@mitar

This comment has been minimized.

Copy link

commented Aug 5, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.