Source maps: --join support? #2779

Closed
ashtuchkin opened this Issue Mar 5, 2013 · 31 comments

Comments

Projects
None yet
@ashtuchkin

Currently the map is generated but mentions compiled .js file as "sources". AFAIK it should go for a set of original filenames.

This comment has been minimized.

Show comment Hide comment
@ghost

ghost Mar 5, 2013

The main problem here is that --join mashes together the CoffeeScript input before running a compile on the whole thing. To fix this you'd have to perform the compilation separately, which breaks things, or record the line offsets for where each file begins, and special-case this in the source map generation.

ghost commented Mar 5, 2013

The main problem here is that --join mashes together the CoffeeScript input before running a compile on the whole thing. To fix this you'd have to perform the compilation separately, which breaks things, or record the line offsets for where each file begins, and special-case this in the source map generation.

@vendethiel

This comment has been minimized.

Show comment Hide comment
@vendethiel

vendethiel Mar 5, 2013

Collaborator
Collaborator

vendethiel commented Mar 5, 2013

@mariusk

This comment has been minimized.

Show comment Hide comment
@mariusk

mariusk Mar 5, 2013

+1 (yes, I'm affected by this one as well)

mariusk commented Mar 5, 2013

+1 (yes, I'm affected by this one as well)

@jashkenas

This comment has been minimized.

Show comment Hide comment
@jashkenas

jashkenas Mar 9, 2013

Owner

I believe @jwalton has plans to address this sometime in the medium-near future. I'll defer to him on the wisest implementation.

Owner

jashkenas commented Mar 9, 2013

I believe @jwalton has plans to address this sometime in the medium-near future. I'll defer to him on the wisest implementation.

@jwalton

This comment has been minimized.

Show comment Hide comment
@jwalton

jwalton Mar 9, 2013

Contributor

Well, it's certainly something I've been thinking about, but I'm not sure I know how to solve it.

What should happen, from a source-map standards perspective, is that "sources" should be a list of .coffee files, and then some of these mappings should be of the five-field variety as we go from file to file.

I like @mintplant 's suggestion of recoding line offsets where each file begins. What I'm thinking is we pass in a list of (lineOffset, filename) pairs to coffee-script.coffee#compile(). I think we'd also change SourceMap.addMapping() to accept an extra "filename" parameter. So, as we build the source map, we figure out which file we're in based on the current line number from the joined code, subtract the lineOffset for that file, and use the resulting line number as the source line when we call addMapping().

Contributor

jwalton commented Mar 9, 2013

Well, it's certainly something I've been thinking about, but I'm not sure I know how to solve it.

What should happen, from a source-map standards perspective, is that "sources" should be a list of .coffee files, and then some of these mappings should be of the five-field variety as we go from file to file.

I like @mintplant 's suggestion of recoding line offsets where each file begins. What I'm thinking is we pass in a list of (lineOffset, filename) pairs to coffee-script.coffee#compile(). I think we'd also change SourceMap.addMapping() to accept an extra "filename" parameter. So, as we build the source map, we figure out which file we're in based on the current line number from the joined code, subtract the lineOffset for that file, and use the resulting line number as the source line when we call addMapping().

@mzedeler

This comment has been minimized.

Show comment Hide comment
@mzedeler

mzedeler Mar 15, 2013

+1.

@jwalton: it sounds like the way that --concat has been written forces you to do the whole thing backwards. Maybe it would be easier to fix or hook into the concat code?

+1.

@jwalton: it sounds like the way that --concat has been written forces you to do the whole thing backwards. Maybe it would be easier to fix or hook into the concat code?

@mgutz

This comment has been minimized.

Show comment Hide comment
@mgutz

mgutz Mar 23, 2013

I have an example of a single CommonJS module from many CoffeeScripts files debuggable in the browser and WebStorm using rebased map files. I can't reliably make Chrome stop at breakpoints.

https://github.com/projmate/coffee-backbone-sourcemap

The logic is here

https://github.com/projmate/projmate-filters/blob/develop/src/lib/commonJsify.coffee

mgutz commented Mar 23, 2013

I have an example of a single CommonJS module from many CoffeeScripts files debuggable in the browser and WebStorm using rebased map files. I can't reliably make Chrome stop at breakpoints.

https://github.com/projmate/coffee-backbone-sourcemap

The logic is here

https://github.com/projmate/projmate-filters/blob/develop/src/lib/commonJsify.coffee

@adamflorin

This comment has been minimized.

Show comment Hide comment
@adamflorin

adamflorin Apr 17, 2013

+1

+1

@jwalton

This comment has been minimized.

Show comment Hide comment
@jwalton

jwalton Apr 22, 2013

Contributor

This is something I will hopefully have a chance to look at soonish. Someone on my team just pulled in snockets and started using it to join together coffee files, so I suspect we need it now, too. :)

Contributor

jwalton commented Apr 22, 2013

This is something I will hopefully have a chance to look at soonish. Someone on my team just pulled in snockets and started using it to join together coffee files, so I suspect we need it now, too. :)

@vendethiel

This comment has been minimized.

Show comment Hide comment
@vendethiel

vendethiel Apr 22, 2013

Collaborator
@pthrasher

This comment has been minimized.

Show comment Hide comment
@pthrasher

pthrasher May 1, 2013

@jwalton I've taken over dev of snockets pthrasher/snockets and I've added srcmap support. I rewired a lot of how the internals work, but it will compile coffee, gen src maps, uglify all the other js and coffee srcs, and then concatenate the source maps and the js.

I'm still working out some kinks, and it's not yet up on NPM (just on github) but it's a solid start. The goal is to get this working 100%. Though, the project desperately needs more (read: thorough and complete) tests, and some optimizations.

Definitely take a look and let me know what you think. I have a working example of using it with grunt for one of my own projects if you'd like me to post that grunt task.

@jwalton I've taken over dev of snockets pthrasher/snockets and I've added srcmap support. I rewired a lot of how the internals work, but it will compile coffee, gen src maps, uglify all the other js and coffee srcs, and then concatenate the source maps and the js.

I'm still working out some kinks, and it's not yet up on NPM (just on github) but it's a solid start. The goal is to get this working 100%. Though, the project desperately needs more (read: thorough and complete) tests, and some optimizations.

Definitely take a look and let me know what you think. I have a working example of using it with grunt for one of my own projects if you'd like me to post that grunt task.

@jwalton

This comment has been minimized.

Show comment Hide comment
@jwalton

jwalton May 1, 2013

Contributor

@pthrasher Definitely sounds interesting! @undashes (one of my co-workers) is probably interested in this, too.

Contributor

jwalton commented May 1, 2013

@pthrasher Definitely sounds interesting! @undashes (one of my co-workers) is probably interested in this, too.

@pthrasher

This comment has been minimized.

Show comment Hide comment
@pthrasher

pthrasher May 1, 2013

@jwalton Cool, I've got a target date for snockets 2.0 of the end of the month. Keep an eye out, and if you would like any functionality added, let me know.

@jwalton Cool, I've got a target date for snockets 2.0 of the end of the month. Keep an eye out, and if you would like any functionality added, let me know.

@tav

This comment has been minimized.

Show comment Hide comment
@tav

tav May 7, 2013

Just adding my 👍 to this feature request — it would be super helpful to be able to --join multiple coffeescript files into a single sourcemap with appropriate references to the individual source files.

tav commented May 7, 2013

Just adding my 👍 to this feature request — it would be super helpful to be able to --join multiple coffeescript files into a single sourcemap with appropriate references to the individual source files.

@antonkulaga antonkulaga referenced this issue in fairfieldt/coffeescript-concat May 13, 2013

Closed

coffeescript map support #16

@Kerrick

This comment has been minimized.

Show comment Hide comment
@Kerrick

Kerrick Jun 20, 2013

Adding a 👍 to the "Affects me" stack.

Kerrick commented Jun 20, 2013

Adding a 👍 to the "Affects me" stack.

@lucaspiller

This comment has been minimized.

Show comment Hide comment
@lucaspiller

lucaspiller Jul 7, 2013

👍 also affected

👍 also affected

@ajumell

This comment has been minimized.

Show comment Hide comment
@ajumell

ajumell Jul 31, 2013

+1

ajumell commented Jul 31, 2013

+1

@eddiemonge

This comment has been minimized.

Show comment Hide comment
@eddiemonge

eddiemonge Aug 21, 2013

+1 and @jwalton any new news?

+1 and @jwalton any new news?

@karlgoldstein

This comment has been minimized.

Show comment Hide comment
@karlgoldstein

karlgoldstein Oct 6, 2013

+1

+1

@lydell

This comment has been minimized.

Show comment Hide comment
@lydell

lydell Oct 8, 2013

Collaborator

When I added source map support to autoprefixer, which can also concatenate CSS, I solved the source map problem of joining files this way:

  1. Parse each file independently. The parser puts an object containing original source file path and original source location on each AST node.
  2. Join all the AST:s. That's really simple in CSS; I don't know about CoffeeScript/JavaScript.
  3. Compile the single joined AST.

(Actually, I developed the Climap module to help with the above.)

The key was to join the AST:s, not the source strings.

I hope any of this helps.

Collaborator

lydell commented Oct 8, 2013

When I added source map support to autoprefixer, which can also concatenate CSS, I solved the source map problem of joining files this way:

  1. Parse each file independently. The parser puts an object containing original source file path and original source location on each AST node.
  2. Join all the AST:s. That's really simple in CSS; I don't know about CoffeeScript/JavaScript.
  3. Compile the single joined AST.

(Actually, I developed the Climap module to help with the above.)

The key was to join the AST:s, not the source strings.

I hope any of this helps.

@treeno

This comment has been minimized.

Show comment Hide comment
@treeno

treeno Oct 10, 2013

also affected

treeno commented Oct 10, 2013

also affected

@backspaces

This comment has been minimized.

Show comment Hide comment
@backspaces

backspaces Oct 21, 2013

I've had to deal with this too.

My solution was to use Cake to watch the 5 files being joined, concatenate the files into a single .coffee file when any one changed, then compile the concatenated files via the coffee command, with the map flag. It works quite well.

I've had to deal with this too.

My solution was to use Cake to watch the 5 files being joined, concatenate the files into a single .coffee file when any one changed, then compile the concatenated files via the coffee command, with the map flag. It works quite well.

@lindskogen

This comment has been minimized.

Show comment Hide comment
@lindskogen

lindskogen Dec 9, 2013

I've just looked into how other libraries accomplish this, sass for example adds an offset when you @include a file.

I've just looked into how other libraries accomplish this, sass for example adds an offset when you @include a file.

@backspaces

This comment has been minimized.

Show comment Hide comment
@backspaces

backspaces Dec 10, 2013

This would require the coffee --join to create a coffee file of the
appropriate name. I.e. if your output file is foo.js, then a foo.coffee
would help.

Thus far, after several requests, they've felt it not necessary. Use
cake/grunt to cobble together your own, alas. Not that bad, however.

-- Owen

On Mon, Dec 9, 2013 at 3:48 PM, Johan Lindskogen
notifications@github.comwrote:

I've just looked into how other libraries accomplish this, sasshttps://github.com/harthur/sass/blob/master/lib/sass/source/map.rb#L40for example adds an offset when you
@include a file.


Reply to this email directly or view it on GitHubhttps://github.com/jashkenas/coffee-script/issues/2779#issuecomment-30182173
.

This would require the coffee --join to create a coffee file of the
appropriate name. I.e. if your output file is foo.js, then a foo.coffee
would help.

Thus far, after several requests, they've felt it not necessary. Use
cake/grunt to cobble together your own, alas. Not that bad, however.

-- Owen

On Mon, Dec 9, 2013 at 3:48 PM, Johan Lindskogen
notifications@github.comwrote:

I've just looked into how other libraries accomplish this, sasshttps://github.com/harthur/sass/blob/master/lib/sass/source/map.rb#L40for example adds an offset when you
@include a file.


Reply to this email directly or view it on GitHubhttps://github.com/jashkenas/coffee-script/issues/2779#issuecomment-30182173
.

@lydell

This comment has been minimized.

Show comment Hide comment
@lydell

lydell Dec 10, 2013

Collaborator

Why does --join exist at all?

Collaborator

lydell commented Dec 10, 2013

Why does --join exist at all?

@treeno

This comment has been minimized.

Show comment Hide comment
@treeno

treeno Dec 10, 2013

I "solved" that by using require.js. I'm not the biggest fan of it, because I'm used to something more straight forward from java, I guess, but it's the best way to solve something like requirements-management in JavaScript/CoffeeScript that I could find. I fear, that this will always be a pain in the JavaScript-World until they put something idiomatic into the language itself...

In the end I don't have a combined file, but several single files that load each other by means of require.js. So I don't have tons of script-tags (only a single one) in the html-code and the SourceMaps work too.

Greetings
treeno

treeno commented Dec 10, 2013

I "solved" that by using require.js. I'm not the biggest fan of it, because I'm used to something more straight forward from java, I guess, but it's the best way to solve something like requirements-management in JavaScript/CoffeeScript that I could find. I fear, that this will always be a pain in the JavaScript-World until they put something idiomatic into the language itself...

In the end I don't have a combined file, but several single files that load each other by means of require.js. So I don't have tons of script-tags (only a single one) in the html-code and the SourceMaps work too.

Greetings
treeno

@pwnall

This comment has been minimized.

Show comment Hide comment
@pwnall

pwnall Dec 10, 2013

@lydell --join places the compilation result of all the source files under the same function wrapper, so you don't have to export internal names. This can be achieved by compiling every source file with --bare, concatenating the results, and adding the function() wrapper yourself, but --join is more convenient.

pwnall commented Dec 10, 2013

@lydell --join places the compilation result of all the source files under the same function wrapper, so you don't have to export internal names. This can be achieved by compiling every source file with --bare, concatenating the results, and adding the function() wrapper yourself, but --join is more convenient.

@vendethiel

This comment has been minimized.

Show comment Hide comment
@vendethiel

vendethiel Dec 10, 2013

Collaborator

He's not asking what this does, he's asking why coffee is doing it. And indeed, this should be in your build tool, not in the compiler.

Collaborator

vendethiel commented Dec 10, 2013

He's not asking what this does, he's asking why coffee is doing it. And indeed, this should be in your build tool, not in the compiler.

@smaspe

This comment has been minimized.

Show comment Hide comment
@smaspe

smaspe Feb 18, 2014

The thing is, if I don't use join, my classes can't communicate, and if I don't use map, debugging is really not convenient, as the js file gets very long.

smaspe commented Feb 18, 2014

The thing is, if I don't use join, my classes can't communicate, and if I don't use map, debugging is really not convenient, as the js file gets very long.

@michaelficarra

This comment has been minimized.

Show comment Hide comment
@michaelficarra

michaelficarra Feb 18, 2014

Collaborator

if I don't use join, my classes can't communicate

Don't naively concatenate your files together to share scope. That is so 1996. Today, we use ES6 modules or CommonJS requires and compile them to a single file if that's what we need. I recommend commonjs-everywhere.

Collaborator

michaelficarra commented Feb 18, 2014

if I don't use join, my classes can't communicate

Don't naively concatenate your files together to share scope. That is so 1996. Today, we use ES6 modules or CommonJS requires and compile them to a single file if that's what we need. I recommend commonjs-everywhere.

@xixixao

This comment has been minimized.

Show comment Hide comment
@xixixao

xixixao May 16, 2014

Contributor

All these issues can now be closed, yay!!

Contributor

xixixao commented May 16, 2014

All these issues can now be closed, yay!!

@vendethiel vendethiel closed this May 16, 2014

NStal added a commit to NStal/leafJs that referenced this issue Sep 28, 2014

@szhu szhu referenced this issue in mtscout6/gulp-cjsx Jan 12, 2016

Closed

Options #8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment