Typescript, Rollup, Jest, Terser, Rimraf, announcer, de-binarying, CI platforming

[StoneCypher]

Do not merge

Please hold, this removes the header process accidentally

This will resolve:

• Version floor conflict #100 Version Floor Conflict
• Browserify doesn't build under Windows #96 Browserify doesn't build under Windows
• Make scripts work on Windows #94 Make scripts work under Windows
• Move to rollup #83 Move to Rollup
• Post-monorepo commentary: CI builds should be matrix builds #75 CI builds should be matrix builds
• Add code coverage metrics #90 Add coverage metrics
• #77 Please consider not-Mocha

Behavioral changes

This now runs all steps as part of a build - previously, testing and static analysis was a separate call. If you want just the build, now use npm run make.

This is a strictly test-additive change, and should cause no breakage

Want to try it?

npm run build

Fixes #100, fixes #96, fixes #94, fixes #90, fixes #83, fixes #77, fixes #75

[StoneCypher]

oh right, coverage

[StoneCypher]

I'll do a nice HTML coverage report later, I'm sleepy

[hildjj]

I like the "lcov" reporter from nyc. It will generate both HTML and lcov. The tests we currently run against the compiled version are by hand in browsers; I was planning on using playwright to automate that (at least for the most modern browsers).

[StoneCypher]

I think that's a fantastic choice

Fairly warned: merging lcov is a nightmare. I'm of the (admittedly exotic) opinion that distinct coverage is better.

Does the general direction of this PR seem suitable so far?

[hildjj]

I've never checked in coverage output before. Can you talk a little bit about why that's interesting?

[hildjj]

we either need to drop the output in browser/peggy.min.js or change the package.json appropriately. A copy gets made to docs/vendor/peggy which is used by docs/online.html to test it. Then the web tests and web benchmark need to be updated. I think you're headed the right way, though. I might grab the rollup config and play around with the output a little to see how I like it, but I don't think I'll have time for that until tonight (MDT).

[StoneCypher]

we either need to drop the output in browser/peggy.min.js

this is probably the right choice.

 

or change the package.json appropriately.

this is probably also the right choice.

i think the umd should go where the old one was, and we should add a main for the es6 one.

[hildjj]

When you're ready for this to be reviewed, can you please rebase onto main and squash to one or more commits that have your train of thought shown, then tag me for a review?

[StoneCypher]

Rebase: no problem

Squash: actually none of my jobs have ever permitted that. I imagine it won't be a problem? 😅 but in a decade of using git I've never actually learned how

[hildjj]

heh, i had to learn about this on #89. See https://stackoverflow.com/questions/6217156/break-a-previous-commit-into-multiple-commits/6217314#6217314 for a pretty good writeup. Look for the instructions for If you are on a different branch.

[hildjj]

And don't worry that you have to force-push to your branch; GitHub will do the right thing with the PR.

[StoneCypher]

this is going to make me do 40 manual merges

do i have to squash? this is done but that'll take hours and is defect prone

[StoneCypher]

You can see an effective squash here

[hildjj]

What I did was squash down to one commit, then break it apart by file into changes that could layer onto one another. There was only one file that was changed in two different ways, and it wasn't hard to break that file apart. If it's really too much work, I'll take a look at it myself over the next few days.

[StoneCypher]

I'm suggesting to split changes into the following commits:

• upgrade GitHub Actions workflow to use matrix
• replace uglify with terser
• replace mocha with jest
• move to rollup
• add TypeScript stuff
• enable coverage

Please understand that when you say this, you are creating high costs for me without taking the time to understand first.

1. Rollup and Typescript are brutally difficult to add separately.
   1. What you're actually asking me to do is:
      1. Fix the Browserify build
      2. Add typescript and the typescript plugins to the typescript build
      3. Configure typescript in an es5/cjs friendly way
      4. Mash things through without esModuleInterop
      5. Remove browserify and add rollup
      6. Remove the browserify plugins and add the rollup plugins
      7. Convert the source from a no interop approach to an interop approach
   2. What I did instead is:
      1. Rip out the ten year old stuff
      2. Start over

To take this out and re-do it the way you suggest is effectively starting over and tripling the workload, and introducing the opportunity for several layers of bugs.

The benefit appears to be that you don't have to look at a rollup configuration file or a single line in npm scripts when evaluating things.

I respectfully decline.

 

• replace uglify with terser

You're going to ask me to scrub this PR, and make an entire new PR, for a single line?

I respectfully decline.

 

• replace mocha with jest

Mocha won't work with rollup at all in any practical typescript context, because it's ancient. What this means is either I have to replace the packaging and the compiler with no tests, or I have to do what I already did.

This is one line and one configuration file. This doesn't need a separate PR.

 

• enable coverage

This is you requesting an entire PR for a single flag on a thing you want me to do in a different PR.

No, we don't need two PRs for jest --coverage. Thanks for understanding.

Asking for changes like this is very frustrating. These were in the same PR for a reason. I don't generally do that; my PRs are generally small; I've repeatedly expressed that this PR is too large.

 

If these didn't need to be colocated, they wouldn't be. It would be polite to ask why they are, or try to figure it out, before requesting changes.

 

• upgrade GitHub Actions workflow to use matrix

The purpose of the Github Actions workflow is to increase the quality of testing.

I'm sorry, but no, the quality of testing needs to be increased before core build processes are altered.

Because the GHA is reliant on changes in this, either we put the GHA in its own PR then change it in this anyway, or we just do it the way it's already done.

 

Respectfully, the kind of feedback I was hoping for was errors and omissions.

[StoneCypher]

I would like to fix the linting errors, but, to re-introduce linting to the build pass after landing this. The reasoning is ugly.

1. We use "use strict" everywhere.
2. This is actually an error in es6
3. Rollup's config is always an es6 module
4. Getting eslint to handle commonjs and es6 modules at the same time means maintaining several different eslint configurations which can't be done through extend because they're consequential, which would immediately come back out
5. The point of linting is to catch errors. Screenshotting a big 0 should temporarily be enough while I finish rooting out the "use strict" for remove all instances of 'use strict' #127

Would this be acceptable? I just don't want this PR to grow to cover more ground, but it has to to get these things in the right order, unless we just ignore the automation for a few hours

[StoneCypher]

This will not impede the CI lockstep for #1 , which is handled in a different branch

[hildjj]

are node users going to use the code that's been through rollup after we release this way?

[StoneCypher]

are node users going to use the code that's been through rollup after we release this way?

Node users will use code that has been through typescript, then rollup, then terser. Currently there is only one path for this. I would like to branch the rollup step to produce both es6 module and commonjs variants, but am waiting for permission. Downstreams can get "the right one" from the module/main split in package.json, but it does imply a slightly larger npm install.

If we decide to, we can also have one missing the last step on CDN. That's good for people when they're debugging. I haven't done that yet, and it should only be on CDN, not in the module

[StoneCypher]

At this time, that includes typescript users; in the long run there are interesting options around that, but it's way premature to have that discussion

[StoneCypher]

This is considered first-pass review responded.

Outstanding problems that I hope to kick the can down the road for into followup PRs:

1. I need to fix the commonjs/es6 module disparity
2. I need to remove the newly ancillary use stricts
3. Because of 1 and 2, lint is not back in the default build yet (see this).

[StoneCypher]

for some reason it won't let me re-request review from @Mingun. this is not intended as a snub: the button just doesn't work (worked fine for hildjj, not sure what the corned beef is)

[hildjj]

I think I'll have time to do the final review on this today. I don't think we need anymore back-and forth until I'm done with that.

[Mingun]

Please understand that when you say this, you are creating high costs for me without taking the time to understand first.

Creating a good PR is not a simple task, yes. I don't think we should rush to merge unpolished PRs. Clear history has many benefits, for the maintainer in the first place. Polishing, I think this is quite normal practice, which shows respect for all participants in the process for each other. David set a very high bar for the quality of the code and commits in the repository, I would not want to lower it.

Rollup and Typescript are brutally difficult to add separately.

I suggest only general pipeline. You feel free to merge or reorder some steps if you feel that this is necessary. It even doesn't needed to build correctly on all targets if you can't accomplish this in one commit (but, of course, this would be desirable). Just give a clear explanation in the message why this can't be done right now, and fix that in the next commit or two.

What I did instead is:

Even in that case you can leave only two commits, instead of dozens:

• Rip out the old stuff
• Add the new one

You're going to ask me to scrub this PR, and make an entire new PR, for a single line?

No (although this is the point... make changes in small portions). Just that is a pretty isolated thing and that can be done very early and keep the history as clear as possible. I think, it is not depend on the rollup/jest/other changes, right?

Mocha won't work with rollup at all in any practical typescript context, because it's ancient.

Well, it's good that I suggested replacing it before switching to rollup 😄

What this means is either I have to replace the packaging and the compiler with no tests, or I have to do what I already did.

In the end, you do not touch tests at all, so the change of the testing library was smoothly, isn't it?

Asking for changes like this is very frustrating. These were in the same PR for a reason. I don't generally do that; my PRs are generally small;

It seems to me that you are overreacting emotionally and see in the comments what was not there. I do not suggest doing other PRs, I suggest doing rebase this. My apologies.

I've repeatedly expressed that this PR is too large.

Agree, in that current state. Although final changes are small enough

The purpose of the Github Actions workflow is to increase the quality of testing.

I'm sorry, but no, the quality of testing needs to be increased before core build processes are altered.

Hmmm? I said the same thing, in my suggestion, changing the pipeline comes first!

[StoneCypher]

Please understand that when you say this, you are creating high costs for me without taking the time to understand first.

Creating a good PR is not a simple task, yes.

None of what you requested would have led to a good PR, is the problem, and if you'd do your new work yourself, you'd have sufficient experience to understand the problems you're creating, while asking the labor from someone else.

 

I don't think we should rush to merge unpolished PRs.

There's no lack of polish here. You misrepresent your preferences as quality.

Of the entire stack here, the only problems have specific technical reasons for them.

 

Even in that case you can leave only two commits, instead of dozens:

No, I cannot, and I already explained why. I have already concretely told you no several times.

 

No (although this is the point... make changes in small portions). Just that is a pretty isolated thing

No, it isn't.

If and when you try to do this work yourself, you may learn why. I cannot explain a fourth way. I'm sorry none of the existing explanations reached you.

 

Well, it's good that I suggested replacing it before switching to rollup 😄

Well, good luck doing it yourself. You don't seem to understand what's wrong here.

 

In the end, you do not touch tests at all, so the change of the testing library was smoothly, isn't it?

Nope. Sorry you don't understand why. You'll learn when you try to do this work yourself, without me, or end up stuck in tools that are more than a decade old.

 

It seems to me that you are overreacting emotionally

That's nice.

 

Hmmm? I said the same thing, in my suggestion, changing the pipeline comes first!

You've been told no four times. Stop repeating yourself. Do it without me. I'm tired of this.

[StoneCypher]

You guys can have this code. Do with it as you please. It was written MIT. Use it.

I make one recommendation: some of the changes you want to make aren't as easy as they sound. Probably make a new branch and work in that, so that if you change your mind, you can change your mind.

Land this directly if you want.

[StoneCypher]

This went in, largely unmodified, because it's high quality work and you need it.