New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for the sourcemap "standard" #1076

Open
Nava2 opened this Issue Aug 16, 2017 · 5 comments

Comments

3 participants
@Nava2
Member

Nava2 commented Aug 16, 2017

Description

In the world of JavaScript (where I find most my work these days), there is a "standard" amongst the many transpilers using sourcemaps to allow developers to see their source actual sources when debugging rather than the generated ones. The sourcemap "standard" is not standardized but it is widely supported.

By adding an optional generation of these maps, it should allow easier integration into existing tools that support them. I believe all of the meta information required is already present in the tracing code.

Relevant documentation

@vahdat-ab

This comment has been minimized.

Show comment
Hide comment
@vahdat-ab

vahdat-ab Aug 16, 2017

Member

Great one.
I just read the links you pointed out. Have some questions:
I wonder if we can handle the source map at the execution time?
Does it have a Java library to be used in our compiler?
Do we need to make the JSON file (or we can make it at the memory)?
Does it support one-to-many and many-to-one relationships between source and the compiled code? I understood that a one-to-many relationship between source and compiler is valid, but no idea if vice versa is doable or not.
how is mapping done by source map? How we make those relationships.

Member

vahdat-ab commented Aug 16, 2017

Great one.
I just read the links you pointed out. Have some questions:
I wonder if we can handle the source map at the execution time?
Does it have a Java library to be used in our compiler?
Do we need to make the JSON file (or we can make it at the memory)?
Does it support one-to-many and many-to-one relationships between source and the compiled code? I understood that a one-to-many relationship between source and compiler is valid, but no idea if vice versa is doable or not.
how is mapping done by source map? How we make those relationships.

@Nava2

This comment has been minimized.

Show comment
Hide comment
@Nava2

Nava2 Aug 16, 2017

Member

I wonder if we can handle the source map at the execution time?

Which execution time? Compiler execution time or when an application is running? For the latter, that'd be up to the tools trying to use the maps.

Does it have a Java library to be used in our compiler?

I doubt it. Though, the syntax is simple enough I don't believe it'd be difficult to do while utilizing a JSON library like Gson or Jackson. The VLQ implementation seems straightforward enough, I can't seem to find any libraries that are thin enough to include.

Do we need to make the JSON file (or we can make it at the memory)?

Most tools expect for any file, say Foo.umple that produces: Foo.java, IFoo.java, there would be two map files: Foo.java.map and IFoo.java.map which include the files used to produce the source. There is only ever one output file, but an array of source files.

Does it support one-to-many and many-to-one relationships between source and the compiled code? I understood that a one-to-many relationship between source and compiler is valid, but no idea if vice versa is doable or not.

I think I unintentionally answered this above.

how is mapping done by source map? How we make those relationships.

When a tool uses the source map, it is executing the actual generated code. However, it can use the sourcemap to look up the mapped location where the source actually exists allowing the developer to "debug" their written code rather than the generated code. This is done via IDEs or debuggers (e.g. WebStorm or Chrome's debugger).

Member

Nava2 commented Aug 16, 2017

I wonder if we can handle the source map at the execution time?

Which execution time? Compiler execution time or when an application is running? For the latter, that'd be up to the tools trying to use the maps.

Does it have a Java library to be used in our compiler?

I doubt it. Though, the syntax is simple enough I don't believe it'd be difficult to do while utilizing a JSON library like Gson or Jackson. The VLQ implementation seems straightforward enough, I can't seem to find any libraries that are thin enough to include.

Do we need to make the JSON file (or we can make it at the memory)?

Most tools expect for any file, say Foo.umple that produces: Foo.java, IFoo.java, there would be two map files: Foo.java.map and IFoo.java.map which include the files used to produce the source. There is only ever one output file, but an array of source files.

Does it support one-to-many and many-to-one relationships between source and the compiled code? I understood that a one-to-many relationship between source and compiler is valid, but no idea if vice versa is doable or not.

I think I unintentionally answered this above.

how is mapping done by source map? How we make those relationships.

When a tool uses the source map, it is executing the actual generated code. However, it can use the sourcemap to look up the mapped location where the source actually exists allowing the developer to "debug" their written code rather than the generated code. This is done via IDEs or debuggers (e.g. WebStorm or Chrome's debugger).

@vahdat-ab

This comment has been minimized.

Show comment
Hide comment
@vahdat-ab

vahdat-ab Aug 16, 2017

Member

Thanks for the comments.

Which execution time? Compiler execution time or when an application is running? For the latter, that'd be up to the tools trying to use the maps.

Both are important, however, the second one (when an application is running) is critical for users and developers. @TimLethbridge were working on this part and maybe he can comment as well here.

I am more interested in this scenario: I don't need to see the generated code and the Umple compiler gives me a binary version and I run it. If there is a failure I don't see the index related to the Java code but I see the index related to the Umple code. I believe this will be a major shift for us if we can deal with this.

Most tools expect for any file, say Foo.umple that produces: Foo.java, IFoo.java, there would be two map files: Foo.java.map and IFoo.java.map which include the files used to produce the source. There is only ever one output file, but an array of source files.

I think I unintentionally answered this above.

so it means if we have an attribute at the Umple level and we generate the code for it (the definition of the attribute and set and get methods), we can track them? For example, if there is a failure in set method, it should refer to the definition of the attribute in the Umple model!

Member

vahdat-ab commented Aug 16, 2017

Thanks for the comments.

Which execution time? Compiler execution time or when an application is running? For the latter, that'd be up to the tools trying to use the maps.

Both are important, however, the second one (when an application is running) is critical for users and developers. @TimLethbridge were working on this part and maybe he can comment as well here.

I am more interested in this scenario: I don't need to see the generated code and the Umple compiler gives me a binary version and I run it. If there is a failure I don't see the index related to the Java code but I see the index related to the Umple code. I believe this will be a major shift for us if we can deal with this.

Most tools expect for any file, say Foo.umple that produces: Foo.java, IFoo.java, there would be two map files: Foo.java.map and IFoo.java.map which include the files used to produce the source. There is only ever one output file, but an array of source files.

I think I unintentionally answered this above.

so it means if we have an attribute at the Umple level and we generate the code for it (the definition of the attribute and set and get methods), we can track them? For example, if there is a failure in set method, it should refer to the definition of the attribute in the Umple model!

@Nava2

This comment has been minimized.

Show comment
Hide comment
@Nava2

Nava2 Aug 16, 2017

Member

Both are important, however, the second one (when an application is running) is critical for users and developers. @TimLethbridge were working on this part and maybe he can comment as well here.

I am more interested in this scenario: I don't need to see the generated code and the Umple compiler gives me a binary version and I run it. If there is a failure I don't see the index related to the Java code but I see the index related to the Umple code. I believe this will be a major shift for us if we can deal with this.

That's exactly what these maps are meant todo. The intention of using the standard versions is that it is more likely that tools will read them and just "get" them. That being said, there is still some definite probabilities here that I'm unsure how many tools "naturally" know to use these. I'm sure if there's integration added for more editors/IDEs, any that have source map support for JS should hook into these same source maps.

so it means if we have an attribute at the Umple level and we generate the code for it (the definition of the attribute and set and get methods), we can track them? For example, if there is a failure in set method, it should refer to the definition of the attribute in the Umple model!

That sounds right. It would point to the attribute rather than to the setter. Though, most exception traces would still have the generated code -- but it's not hidden, just mapped to keep it simpler for debugging. Java itself can't use these maps, only IDEs/editors.

Member

Nava2 commented Aug 16, 2017

Both are important, however, the second one (when an application is running) is critical for users and developers. @TimLethbridge were working on this part and maybe he can comment as well here.

I am more interested in this scenario: I don't need to see the generated code and the Umple compiler gives me a binary version and I run it. If there is a failure I don't see the index related to the Java code but I see the index related to the Umple code. I believe this will be a major shift for us if we can deal with this.

That's exactly what these maps are meant todo. The intention of using the standard versions is that it is more likely that tools will read them and just "get" them. That being said, there is still some definite probabilities here that I'm unsure how many tools "naturally" know to use these. I'm sure if there's integration added for more editors/IDEs, any that have source map support for JS should hook into these same source maps.

so it means if we have an attribute at the Umple level and we generate the code for it (the definition of the attribute and set and get methods), we can track them? For example, if there is a failure in set method, it should refer to the definition of the attribute in the Umple model!

That sounds right. It would point to the attribute rather than to the setter. Though, most exception traces would still have the generated code -- but it's not hidden, just mapped to keep it simpler for debugging. Java itself can't use these maps, only IDEs/editors.

@Nava2

This comment has been minimized.

Show comment
Hide comment
@Nava2

Nava2 Aug 16, 2017

Member

To add the development effort estimate:

  1. Implement a VQL encoder
  2. When generating sources, the positions in the files need to be recorded with the original positions
  3. Convert all these positions to VQL and encode into the sourcemap format

Depending how much information we have available, this will be simple or will be hard. I'm learning towards easier as the source/dest positions should be known through the umple tokens for source, and the string buffer's current position for dest. Recording token positions seems like the best plan of attack.

Member

Nava2 commented Aug 16, 2017

To add the development effort estimate:

  1. Implement a VQL encoder
  2. When generating sources, the positions in the files need to be recorded with the original positions
  3. Convert all these positions to VQL and encode into the sourcemap format

Depending how much information we have available, this will be simple or will be hard. I'm learning towards easier as the source/dest positions should be known through the umple tokens for source, and the string buffer's current position for dest. Recording token positions seems like the best plan of attack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment