Replies: 5 comments 8 replies
-
|
A number of comments here. First, while all languages have different structures and different capabilities we should try to keep the runtimes as similar as possible so that we maintainers can more easily help across targets. Certainly we should not shuffle classes around because people might be relying on the package structure. While perhaps not the best, I see no reason to risk changing these structures and organization. Next, I have a preference for actual TypeScript source code for the library itself (versus types in a separate file) because that will be what typescript developers expect or at least optimally. I suppose it's not the end of the world to not have static types in the library code. On the other hand, it's very attractive having a single js "root" source and then a ts "derivative" that was essentially the same thing without having two nearly identical copies. Having to maintain two separate js and ts targets could be problematic. I'm already burning out managing the languages and operating systems we have already. I made the shocking statement this weekend that coding it's not that fun anymore, and it's directly related to maintaining ANTLR and the myriad of languages, build systems, and requirements. |
Beta Was this translation helpful? Give feedback.
-
|
This is a pretty interesting situation and you, Ter, have to take the decision after all, I guess. What we have (or are about to get) is:
IMO the "right" approach is to remove the current JS target completely and instead create a full TS target, from which a JS target can be derived easily. This should also replace the With this approach we would also have just one "source of truth", instead of duplicating target code. |
Beta Was this translation helpful? Give feedback.
-
|
Mike, Ter, thanks for your comments.
There are a few non-official runtimes out there: ts, kotlin, and I’m not sure it’s for Ter or this group to decide anything about them. That rules out option 1.
The reason I`m investing in option 2 is it provides the best ROI: low investment (a few days - 5 so far, 2 or 3 more required), low maintenance (same runtime), high return (single npm package for both js and ts users).
Not sure I agree with your comment that it 'doesn't really use what TS is for’: the generated classes are pure TS, which is what ts users care about primarily. I very much doubt that there are more than a handful of ts developers out there who want to step into the runtime code itself. They just want to use it.
Also it has the massive benefit that it makes it easy for js users migrating to ts: it’s the same api and runtime, so for them it’s a no-brainer: if their app works in js, it ‘should’ work the same in ts.
(FYI, the js classes have the same name and api as the java ones, so cross-target maintenance is not affected by the tighter folder structure. And other runtimes also use their own folder structure: Go and Dart - I don’t think it’s worth a conversation)
I agree that ts users would rather have a full ts runtime rather than a js one, but I believe they are pretty used to a mixed ts/js environment since many npm packages follow that approach for similar historical reasons, including major ones such as react. I am in no way excluding a migration to ts in the future. But that will be much smoother once we have a running ts target, and can be done gradually, on a per class basis (that said, I’d rather invest next in wasm to improve performance than in migrating working code to ts).
Both option 2 and a pure ts target achieve the single source of truth, although imho option 2 does that better since the code that runs is ‘written' rather than ‘generated' from the written code.
Also not sure what you mean by ‘lack of interpreter support’ ? I think the only missing piece is the TokenStreamRewriter, a gap that’s easy to fill (and also exists for Dart). Am I missing something ?
… Le 6 sept. 2022 à 09:37, Mike Lischke ***@***.***> a écrit :
This is a pretty interesting situation and you, Ter, have to take the decision after all, I guess.
What we have (or are about to get) is:
A TS target, which works pretty well once you got used to a few quirks, but which is still in alpha stage (and will probably stay forever there), which means it should not be used in production environments. The target got a number of optimizations, which make it probably the fastest TS/JS runtime. However, these optimizations (and other limits) prevent further development of this target, which will sooner or later make it obsolete.
There's a variant which is not really TS, but just adds typings to the existing JS target. This has the advantage of no code duplication and the smallest possible effort for TS support. However, this approach doesn't really use what TS is for. There are still no type checks within the target code, not to mention that the target is not complete (e.g. no interpreter support). The folder structure is different compared to the Java (and other language target), but this could be changed. Otherwise this variant is close to be generally available.
There's a variant which truly is a TS cousin of the main Java target, with the same structure, same naming and includes all files. This approach follows the Java variant very closely. There are even Java SDK shims <https://github.com/mike-lischke/a4tstool/tree/master/lib/java> to avoid having to rewrite the generated code manually, if a change occurs there (new features, bug fixes etc.). The used approach allows to auto generate individual classes, if the need arises, with minimal manual effort after the machine translation, to get an updated TS class. However, this TS variant will need several months yet, before it can be used.
IMO the "right" approach is to remove the current JS target completely and instead create a full TS target, from which a JS target can be derived easily. This should also replace the antlr4 node module. The generated JS target can be made part of the repo (like the current JS target is), to avoid forcing people not interested in TS to have to install a typescript compiler.
With this approach we would also have just one "source of truth", instead of duplicating target code.
—
Reply to this email directly, view it on GitHub <#3867 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJALKU73R5XZGM7JR2TV43YC7ANCNFSM6AAAAAAQFFDQCY>.
You are receiving this because you were mentioned.
|
Beta Was this translation helpful? Give feedback.
-
Maybe my formulation was a bit unclear. The decision I mentioned is about which TS implementation to include in the ANTLR4 repo (maybe even multiple?). The 3 options are the current and future solutions that I know of. They describe the current situation regarding TS support in ANTLR4.
Yes, that's what I mentioned as benefit of this solution. However, the NPM package is always JS, so it's also always for both JS and TS users.
The generated classes: yes, the runtime: no. I'm currently focused on the runtime as that is what produces most of the work. The few generated files really are peanuts compared to the runtime.
Yes, that's a pity.
It's not for historical reasons! NPM packages are written in JS. There are no TS packages. Packages are transpiled to JS from the TS source (if any). Look at the antlr4ts node package. It's JS with typings, even though the real source is TS.
That's not an advantage! The handwritten code uses own implementations and may therefore miss important aspects of the original implementation.
How do you even know if the list of classes is complete? You cannot compare the current structure with that of the Java runtime, because it's mostly different. I cannot find LexerInterpreter and ParserInterpreter, InterpreterDataReader or any of the XPath stuff. |
Beta Was this translation helpful? Give feedback.
-
|
Re NPM, with all due respect, sorry to disagree, but many npm packages are written in ts, and obviously shipped as ts/js. An example is react-bootstrap. So there are 3 strategies for publishing packages: js only, js with types, and ts compiled to js. |
Beta Was this translation helpful? Give feedback.
-
I didn't say TS would not be used to write node modules. Instead I wrote that npm packages use JS exclusively (see NPM registry or your local node_modules folder). I even brought up an example myself for a node module which is written in TS and published in NPM as JS with typings (antlr4ts) ;-)
Nonetheless they are part of the Java runtime and hence should be in every supported language. Either I do a job fully or not at all. deleted text contrary to House rules. yes, there are also economical aspects to consider, which we don't need to discuss here as they are very off-topic. All I wanted to point out is: either we continue using the JS target with all its flaws and save some development effort or we use the opportunity to start fresh and do it better this time. But please don't get me wrong! I'm not saying you did a bad job! The JS runtime is a child of its time and at some point we should go a step further. |
Beta Was this translation helpful? Give feedback.
-
|
I'm voting for the variant with the single TS runtime that can be easily converted to JS if required. My arguments are the following:
It looks like conversion TypeScript -> JavaScript should work fine since these languages are very close. But not sure about conversion Java -> TypeScript -> JavaScript. But if it generates working and optimal code it can be considered as well. |
Beta Was this translation helpful? Give feedback.
-
|
The conversion Java -> Typescript is more an initial machine translation, like we did with some of the other targets (e.g. C++). After that quite some manual work is necessary to make the generated code really work. And an additional design goal for me is to make the result as close as possible to the Java target, without sacrificing JS/TS specificity, which would make it easier in the future to update it when the Java code was changed (bug fixes, new features). The conversion from TS to JS is known as transpilation and done in myriads of projects out there on a regular basis. It's a well known process that produces results on which we can rely. |
Beta Was this translation helpful? Give feedback.
-
|
Folks, I'm still thinking about this. It's a difficult decision. |
Beta Was this translation helpful? Give feedback.
-
|
What do you think about waiting for completeness of implementations by @ericvergnaud and @mike-lischke and choosing the best suitable? In my opinion, ANTLR is mature tool and we haven't to force new changes, especially new targets. |
Beta Was this translation helpful? Give feedback.
-
|
There's no rush... I think it's more question of approach, which we can think about before hand. |
Beta Was this translation helpful? Give feedback.
-
|
ok, finally getting back to this. I have spent the past couple of months thinking about this and I don't think there's a perfect solution and so that means I'm going to disappoint everybody. haha. Story of my life ;). Anyway, I decided to make the minimal change which is to add type files for TS on top of the existing JS target. If I understand correctly, we are going to generate parsers that have type information baked in but when they use the runtime library, it's necessary to have the type files. @ericvergnaud what kind of shape is this PR in? #3870 when you think it's ready, let me know and I will poke around some more. |
Beta Was this translation helpful? Give feedback.
-
|
It's ready enough, I've been using it for the past 2-3 months without any issue. I'll do a rebase and get back. |
Beta Was this translation helpful? Give feedback.
-
@mike-lischke and @ericvergnaud are both working on typescript targets for ANTLR. A discussion started on the mailing list but I am moving it here.
Mike said:
Eric:
Mike:
Eric:
Alessio:
Eric:
Beta Was this translation helpful? Give feedback.
All reactions