New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Cesiumjs Webpack and Browserify Friendly and publish on NPM #2981

Closed
mmacaula opened this Issue Aug 27, 2015 · 19 comments

Comments

Projects
None yet
7 participants
@mmacaula
Contributor

mmacaula commented Aug 27, 2015

See: forum post

The source code changes are minimal, just remove package.json files in Source/* directories and could be done in a quick PR.

To become more friendly, it would be nice to publish the compiled workers and source code on npm so that people could npm install it without having to do a special build step. Ideally they would just:

npm install cesiumjs
var viewer = require('cesiumjs/Source/Widgets/Viewer/Viewer');

and so on. The default js file could be the viewer since that is likely how most applications start out, but there's nothing to prevent people from requiring in just what they need.

Ideally, as mentioned on #1438 , @adius said he'd be willing to give up the cesiumjs name on npm for the official cesium once we're ready.

Thoughts?

@mramato

This comment has been minimized.

Show comment
Hide comment
@mramato

mramato Sep 2, 2015

Member

We're definitely interested in supporting Webpack and Browserify out of the box. I was just talking to @kring and @shunter about this last week. Hopefully one of them can chime in with some additional thoughts since I personally haven't used either yet. (If only there were more hours in the day).

As for publishing things on NPM. It's definitely in our long-term roadmap but I think there's definitely more work that we need to do under the hood. We'll also need to make changes to the release process and automate as much as possible in order to make it easy to maintain.

Member

mramato commented Sep 2, 2015

We're definitely interested in supporting Webpack and Browserify out of the box. I was just talking to @kring and @shunter about this last week. Hopefully one of them can chime in with some additional thoughts since I personally haven't used either yet. (If only there were more hours in the day).

As for publishing things on NPM. It's definitely in our long-term roadmap but I think there's definitely more work that we need to do under the hood. We'll also need to make changes to the release process and automate as much as possible in order to make it easy to maintain.

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Sep 2, 2015

Contributor

@mramato I'd be happy to help with the npm publishing, or at least dive into it to make sure I understand what needs to be done. Am I correct in assuming that most of the current process is in the build.xml file?

Contributor

mmacaula commented Sep 2, 2015

@mramato I'd be happy to help with the npm publishing, or at least dive into it to make sure I understand what needs to be done. Am I correct in assuming that most of the current process is in the build.xml file?

@mramato

This comment has been minimized.

Show comment
Hide comment
@mramato

mramato Sep 2, 2015

Member

Yes, and related js files they reference. In truth we would love to move Cesium to an all node/gulp based build process and remove the need for ant/java completely, but no one has had the time to do it (and there's quite a build involved given both the complex nature of Cesium and things like shaders and web workers).

We also want to make it easy to use Cesium as an NPM module on the server. One way we plan on accomplishing that is to switch to using ES6 module syntax in Source and then produce UMD modules as output (which is what we would ship).

Anyway, sorry for the brain dump. Like I said, hopefully Scott or Kevin can chime in with some thoughts.

Member

mramato commented Sep 2, 2015

Yes, and related js files they reference. In truth we would love to move Cesium to an all node/gulp based build process and remove the need for ant/java completely, but no one has had the time to do it (and there's quite a build involved given both the complex nature of Cesium and things like shaders and web workers).

We also want to make it easy to use Cesium as an NPM module on the server. One way we plan on accomplishing that is to switch to using ES6 module syntax in Source and then produce UMD modules as output (which is what we would ship).

Anyway, sorry for the brain dump. Like I said, hopefully Scott or Kevin can chime in with some thoughts.

@rutsky

This comment has been minimized.

Show comment
Hide comment
@rutsky

rutsky Sep 9, 2015

Contributor

@mmacaula, you mentioned in the mailing list that you managed to build pure Cesium with minimal changes (removing package.json) with webpack.
How did you handle building .glsl into .js files?
Did you handle debug macroses, like following?

        //>>includeStart('debug', pragmas.debug);
        throw new DeveloperError('Unable to infer material type: ' + value);
        //>>includeEnd('debug');
Contributor

rutsky commented Sep 9, 2015

@mmacaula, you mentioned in the mailing list that you managed to build pure Cesium with minimal changes (removing package.json) with webpack.
How did you handle building .glsl into .js files?
Did you handle debug macroses, like following?

        //>>includeStart('debug', pragmas.debug);
        throw new DeveloperError('Unable to infer material type: ' + value);
        //>>includeEnd('debug');
@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Sep 10, 2015

Contributor

@rutsky. I did not handle the debug macros. I believe the .glsl files are all part of web workers (if I'm not mistaken) and because of that, as long as we use the built versions of those web workers then we do not need to worry about it unless you want webpack to also build those webworkers. I of course could be mistaken on this and just not exercised the code enough.

I'm not an expert at webpack but those two items could also be done in a webpack friendly way too. I can imagine a glsl loader and a plugin that could strip out the macros based on your webpack config file.

Contributor

mmacaula commented Sep 10, 2015

@rutsky. I did not handle the debug macros. I believe the .glsl files are all part of web workers (if I'm not mistaken) and because of that, as long as we use the built versions of those web workers then we do not need to worry about it unless you want webpack to also build those webworkers. I of course could be mistaken on this and just not exercised the code enough.

I'm not an expert at webpack but those two items could also be done in a webpack friendly way too. I can imagine a glsl loader and a plugin that could strip out the macros based on your webpack config file.

@mramato

This comment has been minimized.

Show comment
Hide comment
@mramato

mramato Sep 10, 2015

Member

It has nothing to do with the workers. The glsl files are never used directly, they are transformed into JavaScript files during the ant build process. If you use a released version of Cesium modules or the combined Cesium.js, then the glsl files are never touched.

While it certainly doesn't have to be in the initial solution, handling debug macros should definitely be a long term goal, since it affects performance.

Member

mramato commented Sep 10, 2015

It has nothing to do with the workers. The glsl files are never used directly, they are transformed into JavaScript files during the ant build process. If you use a released version of Cesium modules or the combined Cesium.js, then the glsl files are never touched.

While it certainly doesn't have to be in the initial solution, handling debug macros should definitely be a long term goal, since it affects performance.

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Sep 11, 2015

Contributor

I see, thanks @mramato, I must have been mistaken on it only needing the workers built. I think I misspoke in the forum and used the downloaded version of cesium, which includes all the js files from the .glsl files.

@mramato, stepping back a bit, can you explain the reasons behind the package.json files in the source directory? If they are not used, then getting rid of them would allow us to publish steps to working with webpack with pure cesium. These steps I believe would be something like this:

  1. Clone Cesium repo in your package.json or bower.json or download it
  2. If cloned, run ant build
  3. Copy the Workers and Assets directory from Build and make sure they are hosted on your app
  4. set Base url: buildModuleUrl.setBaseUrl('path/to/Assets/parent/');

We could put these steps in the Getting Started Docs, or somewhere else. Then we could address the build and publishing on npm separately, while only minimally affecting Cesium as is. Thoughts?

Contributor

mmacaula commented Sep 11, 2015

I see, thanks @mramato, I must have been mistaken on it only needing the workers built. I think I misspoke in the forum and used the downloaded version of cesium, which includes all the js files from the .glsl files.

@mramato, stepping back a bit, can you explain the reasons behind the package.json files in the source directory? If they are not used, then getting rid of them would allow us to publish steps to working with webpack with pure cesium. These steps I believe would be something like this:

  1. Clone Cesium repo in your package.json or bower.json or download it
  2. If cloned, run ant build
  3. Copy the Workers and Assets directory from Build and make sure they are hosted on your app
  4. set Base url: buildModuleUrl.setBaseUrl('path/to/Assets/parent/');

We could put these steps in the Getting Started Docs, or somewhere else. Then we could address the build and publishing on npm separately, while only minimally affecting Cesium as is. Thoughts?

@bartvde

This comment has been minimized.

Show comment
Hide comment
@bartvde

bartvde Sep 16, 2015

Running into this issue as well with browserify, would be great if this could be handled properly out of the box.

bartvde commented Sep 16, 2015

Running into this issue as well with browserify, would be great if this could be handled properly out of the box.

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Sep 30, 2015

Contributor

@mramato, as far as strategy getting this done, would you approve with an approach of a PR that would remove the package.json files from the source directories first, followed by work on updating the build? I'm still not 100% what purpose those package.json files serve...

Contributor

mmacaula commented Sep 30, 2015

@mramato, as far as strategy getting this done, would you approve with an approach of a PR that would remove the package.json files from the source directories first, followed by work on updating the build? I'm still not 100% what purpose those package.json files serve...

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Oct 6, 2015

Contributor

So I was thinking a tutorial might be a good next step for those wanting to use cesium in webpack or browserify right away. Thoughts? I don't see a defined process for writing one. How is it done exactly?

Contributor

mmacaula commented Oct 6, 2015

So I was thinking a tutorial might be a good next step for those wanting to use cesium in webpack or browserify right away. Thoughts? I don't see a defined process for writing one. How is it done exactly?

@pjcozzi

This comment has been minimized.

Show comment
Hide comment
@pjcozzi

pjcozzi Oct 6, 2015

Member

A tutorial would be awesome. Email me, pcozzi@agi.com, and I will connect you with our editor.

Member

pjcozzi commented Oct 6, 2015

A tutorial would be awesome. Email me, pcozzi@agi.com, and I will connect you with our editor.

@bartvde

This comment has been minimized.

Show comment
Hide comment
@bartvde

bartvde Nov 30, 2015

is there any progress on the tutorial? TIA.

bartvde commented Nov 30, 2015

is there any progress on the tutorial? TIA.

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Nov 30, 2015

Contributor

hey @bartvde, it is still a work in progress (still being reviewed) but you can see it at https://github.com/aviture/cesium-webpack

Contributor

mmacaula commented Nov 30, 2015

hey @bartvde, it is still a work in progress (still being reviewed) but you can see it at https://github.com/aviture/cesium-webpack

@bartvde

This comment has been minimized.

Show comment
Hide comment
@bartvde

bartvde Dec 1, 2015

thanks for pointing me to this @mmacaula

bartvde commented Dec 1, 2015

thanks for pointing me to this @mmacaula

@andthenwhat

This comment has been minimized.

Show comment
Hide comment
@andthenwhat

andthenwhat Mar 10, 2016

has anyone had success with browserify? My attempts to used browserify.external and browserify.ignore have just produced different versions of failure. I can post the problems I'm having, but wanted to see if anyone had succeeded before I do.

andthenwhat commented Mar 10, 2016

has anyone had success with browserify? My attempts to used browserify.external and browserify.ignore have just produced different versions of failure. I can post the problems I'm having, but wanted to see if anyone had succeeded before I do.

@mramato

This comment has been minimized.

Show comment
Hide comment
@mramato

mramato Mar 25, 2016

Member

I believe our official cesium node module and @mmacaula's excellent blog post/example repository resolves this issue, correct? @mmacaula is there anything else on the cesium end you feel needs to be done? Thanks.

Member

mramato commented Mar 25, 2016

I believe our official cesium node module and @mmacaula's excellent blog post/example repository resolves this issue, correct? @mmacaula is there anything else on the cesium end you feel needs to be done? Thanks.

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Mar 25, 2016

Contributor

Yes, let's close. Thanks!

Contributor

mmacaula commented Mar 25, 2016

Yes, let's close. Thanks!

@bensleveritt

This comment has been minimized.

Show comment
Hide comment
@bensleveritt

bensleveritt Jul 13, 2016

It's worth noting that mmacaula's repo is more up to date (with post 1.16 instructions).

bensleveritt commented Jul 13, 2016

It's worth noting that mmacaula's repo is more up to date (with post 1.16 instructions).

@mmacaula

This comment has been minimized.

Show comment
Hide comment
@mmacaula

mmacaula Jul 13, 2016

Contributor

Nope all done, we can close
On Jul 13, 2016 10:17 AM, "Benjamin S. Leveritt" notifications@github.com
wrote:

It's worth noting that mmacaula's repo
https://github.com/mmacaula/cesium-webpack is more up to date (with
post 1.6 instructions).


You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#2981 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ADCeSnhy0cWgVwuSQROt-pKQa0hlML8iks5qVQGDgaJpZM4Fzkkm
.

Contributor

mmacaula commented Jul 13, 2016

Nope all done, we can close
On Jul 13, 2016 10:17 AM, "Benjamin S. Leveritt" notifications@github.com
wrote:

It's worth noting that mmacaula's repo
https://github.com/mmacaula/cesium-webpack is more up to date (with
post 1.6 instructions).


You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#2981 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ADCeSnhy0cWgVwuSQROt-pKQa0hlML8iks5qVQGDgaJpZM4Fzkkm
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment