Integrate smoya/multi-parser-js into asyncapi/parser-js #963

smoya · 2024-03-14T10:49:52Z

Context

@smoya/multi-parser is a library that allows users to parse an AsyncAPI document and get an output model based on a desired Parser-API version. Additionally, it allows to convert models from one Parser-API version to another.

This tool is required by some AsyncAPI tools that use the Parser-JS and have to send the parsed document model to another library, which could be compatible with a different Parser-API version than the one used in that particular version of the Parser-JS.
You have several examples of usage here:

CLI, to send the right parsed model to Modelina when Modelina didn't have support for Parser-JS v3: https://github.com/asyncapi/cli/blob/master/src/commands/generate/models.ts#L186C29-L186C77
Modelina, to ensure any AsyncAPI model input uses the right Parser-API version, so it can accept any Parser-API v1, v2, v3... model as input: https://github.com/asyncapi/modelina/blob/master/src/processors/AsyncAPIInputProcessor.ts#L68-L69
Modelina, to instantiate a Parser compatible with a particular Parser-API version: https://github.com/asyncapi/modelina/blob/master/src/processors/AsyncAPIInputProcessor.ts#L72

Current issues

1. 👤 Ownership

@smoya/multi-parser is a library hosted in a GH repository under my particular ownership (@smoya). Not a big issue per se, but I don't wanna be the only maintainer of such a tool. I thought on donating this to AsyncAPI org (TSC would need to vote first) but the reality is that won't be needed, see next point:

2. 🔃 Release is not in sync with Parser-JS

Every new Parser-JS release is not triggering an update on the @smoya/multi-parser. I agree I could configure a bot like Renovate (Dependabot doesn't allow NPM package aliases, and this package makes use of them) to try update those.
But still we have two issue with that situation:

Whenever a new major Parser-API version gets released, the need of small additions into the @smoya/multi-parser code will be needed. See https://github.com/smoya/multi-parser-js/blob/main/src/parse.ts#L45-L53. Could we automate it somehow by changing the code? Perhaps yes, but not ideal and probably not trustable in the long term. At this moment, this includes manual work and be always careful when a new version goes out.
For a certain period of time after a new release of Parser-JS, @smoya/multi-parser will be outdated and users of the library who also use Parser-JS as dependency could have runtime errors due to not using the same version of the models (different Parser-JS versions!). Of course, solved when Renovate updates in case it's not a major Parser-API version change.

The solution I propose

To integrate @smoya/multi-parser code into the Parser-JS repository, and to release two packages on each Parser-JS release:

@asyncapi/parser, the current one that is already happening.
@asyncapi/multi-parser, this a new one.

In that way, everything gets in sync, and also, by adding tests, we could ensure new versions of the Parser-JS are working with the @asyncapi/multi-parser package. The same when we adopt a new version of the Parser-API in the Parser-JS, support in @asyncapi/multi-parser will be added in the same PR.

That means, we should have a monorepo model in this repository.
Due to the fact @asyncapi/multi-parser has as dependency the Parser-JS but not only once, but once for each Parser-API version (atm 3 aliased Parser-JS deps pointing to different version of the Parser-JS), plus the fact the library will be used only by few particular libraries and most users won't need it ever, @asyncapi/multi-parser should be released completely as a separate package.

Examples of monorepo within AsyncAPI can be found here:

Some discussions in Slack about that topic:

BTW, since @smoya/multi-parser is under Apache 2.0 license, copying code right directly into the Parser-JS is completely legal, and since Parser-JS uses the same license, no need to specify any atribution of change on the license. So no need to even first do the donation to AsyncAPI GH org :) Additionally, I'm giving my written approval to do whatever is needed with that code so it can be integrated into the Parser-JS. Once that's done, and all dependants update to @asyncapi/multi-parser, I will completely archive @smoya/multi-parser in favor of @asyncapi/multi-parser.

cc @jonaslagoni @magicmatatjahu

The text was updated successfully, but these errors were encountered:

jonaslagoni · 2024-03-14T11:30:02Z

Makes sense to me 👍

aeworxet · 2024-03-18T16:04:09Z

Bounty Issue's service comment

Text labels: bounty/2024-Q2, bounty/advanced, bounty/coding
First assignment to third-party contributors: 2024-03-22 00:00:00 UTC+12:00
End Of Life: 2024-08-31 23:59:59 UTC-12:00

@asyncapi/bounty_team

Vishal2002 · 2024-03-20T06:10:26Z

@smoya I think you should ask the Asyncapi org to add this issue under GSOC 2024 idealist. It would be a good contribution to have in the parser-js.

smoya · 2024-03-20T06:20:39Z

@smoya I think you should ask the Asyncapi org to add this issue under GSOC 2024 idealist. It would be a good contribution to have in the parser-js.

This is part now of the AsyncAPI Bounty Program 2024 Q2. Additionally, the GSOC 2024 ideas submission period finished last month, and we got accepted.

derberg · 2024-03-20T08:06:22Z

@smoya if that works, I guess we can start thinking about pulling other projects inside parser-js for better maintainance and cooperation - like avro, protobuf, openapi and raml data parsers?

and then we can also work on something that @jonaslagoni brought up in past, that DX of using parser with schema parsers is hard, so we could enable publishing of parser-js on steroids, with schema parsers included by default (while still publishing parser alone of course, lightweight for web)

smoya · 2024-03-20T08:43:30Z

@smoya if that works, I guess we can start thinking about pulling other projects inside parser-js for better maintainance and cooperation - like avro, protobuf, openapi and raml data parsers?

It makes sense to me, yeah.

and then we can also work on something that @jonaslagoni brought up in past, that DX of using parser with schema parsers is hard, so we could enable publishing of parser-js on steroids, with schema parsers included by default (while still publishing parser alone of course, lightweight for web)

+1. I wasn't aware of this suggestion but i really like it. I've found myself involved in the issue of having to manually load the schema parsers all the time so I believe it will be so helpful for lot of users who just want everything to work at once.

derberg · 2024-03-20T09:19:02Z

+1. I wasn't aware of this suggestion but i really like it. I've found myself involved in the issue of having to manually load the schema parsers all the time so I believe it will be so helpful for lot of users who just want everything to work at once.

yeah, I see the same with semantic-versioning that at some point of time, decided to include in main package a number of default, more used plugins - so managing dependencies is now much much better

Vishal2002 · 2024-03-20T11:23:03Z

I want to give a try to this issue and want to work on it.

smoya · 2024-03-20T12:31:18Z

Hi @Vishal2002, thanks for showing interest. Let's wait at least a day since more contributors could jump in as well.
As per the Bounty rules:

Assignment of the Bounty Issue on GitHub to users that fall under the second and third categories is performed not earlier than three calendar days after the addition of the GitHub label bounty according to GitHub's timestamp.
Source: https://github.com/asyncapi/community/pull/897/files#diff-25ecb20a61754c225d6511ca08d7e7c9a14b9ca5a93e89bd42331e51c9ebf26dR39

In your case, you fall under the third category.

Toyin5 · 2024-03-20T12:38:49Z

@smoya I want to work on this issue

smoya · 2024-03-20T12:44:37Z

@smoya I want to work on this issue
Cool! 🚀

Same answer as the one I provided to @Vishal2002 in #963 (comment)

derberg · 2024-03-21T16:34:21Z

@smoya in case you don't get experienced candidates for this one, maybe you can wait for @ayushnau (maybe he will be interested) as he is going to work now on basically the same (with small differences) asyncapi/generator#1044 - so monorepo in generator. Once he finish it, he can then do the same here maybe. Just an idea, I did not even check with @aeworxet if it is possible but I guess yes 🤔 you do not have to decide today or tomorrow, you can decide in a month

smoya · 2024-03-21T16:58:38Z

@smoya in case you don't get experienced candidates for this one, maybe you can wait for @ayushnau (maybe he will be interested) as he is going to work now on basically the same (with small differences) asyncapi/generator#1044 - so monorepo in generator. Once he finish it, he can then do the same here maybe. Just an idea, I did not even check with @aeworxet if it is possible but I guess yes 🤔 you do not have to decide today or tomorrow, you can decide in a month

Yeah, good advise. I'm still evaluating each candidate individually in the meantime.

smoya · 2024-03-21T17:34:37Z

Whats your opinion on this @ayushnau ? If you are willing to work on this, do you believe you could become available at any time soon? When approx?

Thanks! 🙏

ayushnau · 2024-03-21T18:21:01Z

@smoya I'm excited about the opportunity to collaborate on this with you, I would love to work on this. Looking forward to it!

ayushnau · 2024-03-21T18:28:23Z

I will be available starting from next Monday. to pay full attention to this.

smoya · 2024-04-03T20:31:19Z

@ayushnau are you gonna work on this then? If so, when are you finally planning to do it? Thanks!

ayushnau · 2024-04-04T12:15:42Z

i am on it once it got assigned to me.

smoya · 2024-04-04T13:30:29Z

i am on it once it got assigned to me.

As spoken in Slack, you are working in asyncapi/generator#1044 in parallel. You mentioned you need a couple of weeks to finish such a task. Let's wait 2 weeks until I assign this issue to you.
Please @ayushnau ping me max on the 18 of April so I can assign this task to you.

ayushnau · 2024-04-04T16:02:15Z

sure will do that.

ayushnau · 2024-04-18T08:59:35Z

@smoya I would love to work on this please assign this issue to me.

smoya · 2024-04-18T09:00:04Z

@asyncapi/bounty_team issue got assigned today to @ayushnau

aeworxet · 2024-04-18T09:10:40Z

Bounty Issue's Timeline

Complexity Level	Assignment date (by GitHub)	Start date (by BP rules)	End date (by BP rules)	Draft PR submission	Final PR submission	Final PR merge
Advanced	2024-04-18	2024-04-22	2024-06-14	2024-05-10	2024-05-31	2024-06-14

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

smoya · 2024-06-05T10:05:01Z

Due to the fact Ci has been broken since long ago (we were not aware), and considering it took me some time (days) figuring out the reason, I believe @ayushnau should request an extension of 1 week for this Bounty Program delivery.

cc @aeworxet

aeworxet · 2024-06-06T05:32:22Z

Upon request of the AsyncAPI Maintainer, who is responsible for the resolution of the Bounty Issue from the AsyncAPI's side (@smoya), all remaining target dates of the Bounty Issue's Timeline are extended by one calendar week.

Bounty Issue's Timeline Extended

Complexity Level	Assignment date (by GitHub)	Start date (by BP rules)	End date (by BP rules)	Draft PR submission	Final PR submission	Final PR merge
Advanced	2024-04-18	2024-04-22	2024-06-21	2024-05-10	2024-05-31	2024-06-21

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

ayushnau · 2024-06-21T07:27:01Z

@aeworxet @smoya

I am writing to inform you of a recent change in our release process for the GitHub action. The release process, which was previously managed using semantic release, has now been transitioned to changeset. This unexpected shift has caused some delays, as I was not fully prepared for the new process. Consequently, I kindly request an extension of two weeks to accommodate this transition and ensure a smooth and accurate release.

Thank you for your understanding and consideration.

smoya · 2024-06-21T07:52:47Z

@aeworxet @smoya

I am writing to inform you of a recent change in our release process for the GitHub action. The release process, which was previously managed using semantic release, has now been transitioned to changeset. This unexpected shift has caused some delays, as I was not fully prepared for the new process. Consequently, I kindly request an extension of two weeks to accommodate this transition and ensure a smooth and accurate release.

Thank you for your understanding and consideration.

I agree the uncertainty nature of this issue, the knowns VS unknowns, and the several areas this issue affects (code, CI, CD) makes a difficult and, somehow, unpredictable one.
Even though we had a model to follow (Studio case), we (including me as maintainer) did assumptions wrongly; later found we were missing some changes such as the replacement of semantic-release by changesets (a change that it is not so obvious) and how that change affects our release pipeline.

Being honest, I had some doubts few days ago if this task should be given an extension or not, but atm I truly believe it should get it.

@aeworxet has the last words.

aeworxet · 2024-06-21T08:30:20Z

Upon request of the Bounty Program Participant (@ayushnau), all remaining target dates of the Bounty Issue's Timeline are extended by two calendar weeks.

Bounty Issue's Timeline Extended

Complexity Level	Assignment date (by GitHub)	Start date (by BP rules)	End date (by BP rules)	Draft PR submission	Final PR submission	Final PR merge
Advanced	2024-04-18	2024-04-22	2024-07-05	2024-05-10	2024-06-21	2024-07-05

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

smoya · 2024-07-23T14:09:27Z

@ayushnau After merging #1036 and related PRs, the release of both packages happened. I tested the @asyncapi/parser (basic stuff) and seems to work, however, when running a simple test for the @asyncapi/multi-parser, the following errors appears:

(node:55793) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/smoya/dev/asyncapi-parser-example/node_modules/@asyncapi/parser/esm/index.js:1
import { DiagnosticSeverity } from '@stoplight/types';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1001:16)
    at Module._compile (internal/modules/cjs/loader.js:1049:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:93:18)
    at Object.<anonymous> (/Users/smoya/dev/asyncapi-parser-example/node_modules/@asyncapi/multi-parser/cjs/parse.js:6:17)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)

ayushnau · 2024-07-25T13:35:52Z

@smoya working on it. will update you once i fix it.

smoya · 2024-08-02T09:41:10Z

This issue is now technically solved. Thanks @ayushnau for your work!

Cc @aeworxet

aeworxet · 2024-08-02T09:46:47Z

After a set of delays that were beyond the control of the Bounty Program Participant
#1036 (comment)
#1036 (comment)
#1036 (comment)
#1036 (comment)
#1036 (comment)
#1036 (review)
#1036 (comment)
#963 (comment)

Bounty Issue Completed 🎉

@ayushnau, please go to the AsyncAPI page on Open Collective and submit an invoice for USD 400.00 with the expense title Bounty parser-js#963, tag bounty, and full URL of this Bounty Issue in the description.

smoya added the help wanted Extra attention is needed label Mar 14, 2024

smoya mentioned this issue Mar 14, 2024

Deprecate this repo smoya/multi-parser-js#11

Open

asyncapi-bot added the bounty AsyncAPI Bounty label Mar 18, 2024

derberg mentioned this issue Mar 20, 2024

How about a monorepo 😄 asyncapi/generator#1044

Open

smoya assigned ayushnau Apr 18, 2024

smoya removed the help wanted Extra attention is needed label Apr 18, 2024

ayushnau mentioned this issue Apr 21, 2024

feat: introduce the turborepo to the parser.js #992

Merged

This was referenced Apr 24, 2024

[BUG] fails with @asyncapi/parser-js 3.0.14 smoya/asyncapi-adoption-metrics#30

Closed

Donate this project to AsyncAPI org smoya/asyncapi-adoption-metrics#16

Open

ayushnau mentioned this issue May 27, 2024

Added the multiparser repo to the turborepo. #998

Closed

ayushnau mentioned this issue Jun 9, 2024

feat: introduce the turborepo to the parser.js #1008

Merged

ayushnau mentioned this issue Jul 4, 2024

feat: add multiparser to the parserjs monorepo repository. #1036

Merged

ayushnau mentioned this issue Jul 23, 2024

feat: added the changeset for the parser. #1040

Merged

asyncapi deleted a comment from asyncapi-bot Jul 23, 2024

ayushnau mentioned this issue Aug 1, 2024

fix: updated the method of importing the parser #1045

Merged

smoya closed this as completed Oct 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate smoya/multi-parser-js into asyncapi/parser-js #963

Integrate smoya/multi-parser-js into asyncapi/parser-js #963

smoya commented Mar 14, 2024 •

edited

Loading

jonaslagoni commented Mar 14, 2024

aeworxet commented Mar 18, 2024

Vishal2002 commented Mar 20, 2024

smoya commented Mar 20, 2024

derberg commented Mar 20, 2024

smoya commented Mar 20, 2024

derberg commented Mar 20, 2024

Vishal2002 commented Mar 20, 2024

smoya commented Mar 20, 2024

Toyin5 commented Mar 20, 2024

smoya commented Mar 20, 2024

derberg commented Mar 21, 2024

smoya commented Mar 21, 2024

smoya commented Mar 21, 2024

ayushnau commented Mar 21, 2024

ayushnau commented Mar 21, 2024

smoya commented Apr 3, 2024 •

edited

Loading

ayushnau commented Apr 4, 2024

smoya commented Apr 4, 2024 •

edited

Loading

ayushnau commented Apr 4, 2024

ayushnau commented Apr 18, 2024

smoya commented Apr 18, 2024 •

edited

Loading

aeworxet commented Apr 18, 2024

smoya commented Jun 5, 2024 •

edited

Loading

aeworxet commented Jun 6, 2024

ayushnau commented Jun 21, 2024

smoya commented Jun 21, 2024

aeworxet commented Jun 21, 2024

smoya commented Jul 23, 2024 •

edited

Loading

ayushnau commented Jul 25, 2024

smoya commented Aug 2, 2024

aeworxet commented Aug 2, 2024

Integrate smoya/multi-parser-js into asyncapi/parser-js #963

Integrate smoya/multi-parser-js into asyncapi/parser-js #963

Comments

smoya commented Mar 14, 2024 • edited Loading

Context

Current issues

1. 👤 Ownership

2. 🔃 Release is not in sync with Parser-JS

The solution I propose

jonaslagoni commented Mar 14, 2024

aeworxet commented Mar 18, 2024

Bounty Issue's service comment

Vishal2002 commented Mar 20, 2024

smoya commented Mar 20, 2024

derberg commented Mar 20, 2024

smoya commented Mar 20, 2024

derberg commented Mar 20, 2024

Vishal2002 commented Mar 20, 2024

smoya commented Mar 20, 2024

Toyin5 commented Mar 20, 2024

smoya commented Mar 20, 2024

derberg commented Mar 21, 2024

smoya commented Mar 21, 2024

smoya commented Mar 21, 2024

ayushnau commented Mar 21, 2024

ayushnau commented Mar 21, 2024

smoya commented Apr 3, 2024 • edited Loading

ayushnau commented Apr 4, 2024

smoya commented Apr 4, 2024 • edited Loading

ayushnau commented Apr 4, 2024

ayushnau commented Apr 18, 2024

smoya commented Apr 18, 2024 • edited Loading

aeworxet commented Apr 18, 2024

Bounty Issue's Timeline

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

smoya commented Jun 5, 2024 • edited Loading

aeworxet commented Jun 6, 2024

Bounty Issue's Timeline Extended

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

ayushnau commented Jun 21, 2024

smoya commented Jun 21, 2024

aeworxet commented Jun 21, 2024

Bounty Issue's Timeline Extended

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

smoya commented Jul 23, 2024 • edited Loading

ayushnau commented Jul 25, 2024

smoya commented Aug 2, 2024

aeworxet commented Aug 2, 2024

Bounty Issue Completed 🎉

smoya commented Mar 14, 2024 •

edited

Loading

smoya commented Apr 3, 2024 •

edited

Loading

smoya commented Apr 4, 2024 •

edited

Loading

smoya commented Apr 18, 2024 •

edited

Loading

smoya commented Jun 5, 2024 •

edited

Loading

smoya commented Jul 23, 2024 •

edited

Loading