Skip to content

2026 Beyond testing and type checking JavaScript: Can we be bothered?

Jump to bottom Edit New page
Manuel Rego edited this page Jul 6, 2026 · 1 revision

WEH 2026 - Beyond testing and type checking JavaScript: Can we be bothered?

Jesse (@jessealama): this is for people writing javascript and typescript, not so much c and cpp browsers makers. Some exotic ideas that are a bit outside of the mainstream, but maybe good to keep in mind. We want to test our sotware. IF you’re into that, that you’re in the right room.

Jesse: How do you feel about your tests? How about your tests in the AI era? One of the premises of this talk is that we’re in an interesting time. An opportunity to think about our tests. What to do better. This is not a lecture. Not about switching from some tooling to some other tooling.

Jesse: There are some remote speakers too. Some guest presentations. And then a discussion. WE’ll cover some more concrete things like unit tests and then on to more loose things like static analysis.

Nicolas (@dubzzz): fast-check: Why another technique? Writing tests is hard. Initial position: “it requires seniority”. Coverage. Coverage is not enough, it helps but not a full safety net. Main problem is limited scope of inputs covered. Maybe we missed many cases. Another solution is property-based testing, the idea is going random. But in a reproducible way. We do want new inputs at each run, with shrinking in mind. ... This is it for my presentation.

Nicolas: The official documentation for fast-check is: https://fast-check.dev/. And a few extra links:

?: This is awesome. Q: How do you generate random values?

Nicolas: They are not totally random. If you were to generate ints, chances are you want to check for boundaries. So fc will usually generate extreme values. But generally things are random. We’re biased for some, but generally try as random as possible. shrinking, mostly behind the scenes of going for shorter values in the longer values. To understand how it works I wrote https://dev.to/dubzzz/your-own-property-based-testing-framework-part-1-generators-3afi a long time ago, it goes from generators to shrinkers

?: Yes good answer my only Q is if you want to generate your own values.

Nicolas: You can combine things together which are capable of generating your own ideas. Look for helpers such as record, tuple, stringMatching, .map, .filter, .chain...

Eemily: 8 years ago you helped add fc to my yaml library. Could I cc you on the issue to help generate random YAML?

Jesse: Is it also possible to generate random JSON, HTML, etc?

Eemily: If random YAML is possible, anything is possible.

Philip: For the past few years I worked on Temporarl and had a similar problem trying to test parsing of date and time strings. I had a technique similar to fc to generate infinite valid strings. If you have a grammar you can generate things. Something I struggled with was generating invalid input. That will find bugs in your parser. Does fc have something for that?

Nicolas: I don’t have a good solution. Generating valid is simple. But for buggy values, I don’t have a magic solution. Problem with property-based is generally to find the predicate. As soon as you want to test the faulty case. I don’t have a magical solution, still looking for it.

Jesse: Thank you! Let me take over the screen share. That was one cool idea that I hope got yo fired up your testing game. In many cases things like fc can find some bugs. I used it myself and found couple interesting problems.

Jesse: Another technique that is interesting is mutation testing. The idea is to take the programs you have in your tests, deliberately change them, run the tests, and see what still passes. Think about that. They should break. Maybe there’s something wrong with the tests. Or a bug somewhere. This may sounds hand wavey but try using any of these tools. I found a tool called Stryker (https://stryker-mutator.io). Can I take === and replace it with !==. I burned my battery when I was running this. This is another way of finding issues with your tests. Not just tests in the code themselves, but also bugs in the tests. It’s a fascinating way to think about your programs.

Philip: I haven’t used mutation testing, but I’ve had bugs reported by users using it. When I first read about it, I thought, that sounds really impractical. You’re gonna have so many misses. Most of the things you change will result in a test failure. Is there a way to get the search space smaller?

Jesse: I don’t have any advice. Yes, you’re right. Presumably things fail. It’s impractical if you have a very large test suite or code base. My tip is to look at the thing that you know is important, with this tool you can choose which file, or which line range to focus on.

Matthew: ... Use your own intellgice. Things should filter.

?: Things should break. How do you deal with operator changes, where you have multiple valid options. You use < or <=, in many cases it doesn’t matter.

Jesse: It does require some judgement, something that survives is not nessecarily a bug. It isn’t a technique for all cases. But I had some cases, I had somewhere a with a < or <=, with MAX_SAFE_INTEGER.

Maya: Yes, it could be very useful. Kind of like fuzzing. More of an idea: it sounds expensive, would be good to kind of have it saved between runs, or saved with commits. For the team.

?: Mutation testing really does on thing well, property testing off of your test suite. The other way around. There will be code merged, especially in this day and age, that is sub par. I use mutation to make sure there are no runtime failures. But you have to think about it in that way. It’s a specialised tool for a spezialised case.

Jesse: Keep in mind it’s a bit heavy weight. Make sure your laptop is plugged in. OK, that’s another slice for today. We have another rmeote speaker. Gregory.

Gregory (@gterzian): My talk will be about web engine engineering. Using TLA+ to formally verify some part of web engines. I am also a maintainer of Servo. But I also have a side project (gterzian/formal-web). Let me show this. You can run some WASM, interact with the DOM. You can open a window. In a new tab. Essentially it’s just the basic web engine stuff. Where does it get the formal verification? So you start with a web standard, mostly straightforward. Basically an event loop. Some complicated things but not that much. But there are some exceptions. Here is the navigation algorithm in the HTML spec. It’s very large. Most of it is sequential. For formal verification, you can look for the paralel steps. Like step 24. This algo, what it does, is essentially queue a task and then it will fire right and then ask a user for permission to unload. So essentially you have to wait. This is a great example of fairly complicated thing to formally verify. As you see there was a paper published (Validating traces of distributed programs against TLA+ specifications) a couple years ago. Esepcially designed for concurrency. Purely a mathematical model. In the past it was mostly about writing TLA+, and then you were responsible for your own implementation. But this paper proposes an alternative, you are logging in your code, and these traces can be compared against your spec. Still not complete, but you can check that ?? I am applying ideas from that paper in my project. When you run it with a certain flag, there is a thread running, which reports to the main process, which when the web engine shuts down, it will collect all these traces, and compare the traces and check if they are valid. [...]

Andreas: I am working on this exact code in in Gecko. Can I have this code? My question is [...]

Gregory: There’s a folder tls_specs; if I write a browser in Lean. The proof is only as good as the thing I want to be proved. Up to me what I want to be proven. Same in TLA+. Nothing that says TLA spec models ... other than my own judgement.

Eva: How steep is the learning curve to leanr the spec language, would it be possible to write in a language I am more familiar with, like JS?

Greg: Hmm, in 2020 I came across this spec. I realized I should do something more than just writing code. To be honest, only in 2023 that I really felt like I knew what I was doing. For em it was learning to programm all over again. Having said that. Nowadays AI is pretty good with the language. But of course very dangerous. Unlikely to fit what you want. Maybe you don’t even know what you want. By now if I write I use AI. But of course that requires first know what you want. To answer the questoin, there is something like pluscal (?) which looks a bit like programming and compiles to it, but m advice would be to not use it. Maybe a compiler from JS is possible, but the challenge is more what you want to specify. Learn TLA+. It is fun. But with a steep learnin curve.

Jesse: are there any low hanging fruit, what are the next steps after this navigable thing?

Greg: I have to implement the rest. The whole session history. A really nice thing. Another of my favorites is message port. Just recently i wrote a spec for that. Based on my implementation in servo. Using the spec as a blueprint. And then a couple others. Really a couple percentages of web specs. Like IndexedDB. Maybe service worker. A couple specs have tricky things. But most are simple.

Jesse: a bit more experimental feeling towards these topics: just a moment ago we heard about proofing things. In the mathematical sense. We problably heard about, but didn’t do much with this. One thing I personally experiment with is Thales. Something that compiles to Lean. Which lets you prove in the same file. There is no side car. It’s one thing. Workin on it for some time now. Dipping into a TS compiler for that. What’s interesting is that I worked on it since march, april, but a few days before I anounced it there was something similar: LemmaScript. Which targets Lean and Dafny. Powerful system. Going back to today: AI, not just writing and testing but also proving things are becoming more important. Really difficult in a pre-AI world. Now a bit more realistic. You can add things like loop invariants. A fascinating space. Evolving rapidly. LemmaScript, and my thing, there are probably other players out there. These things are a bit experimental.

Jesse: Static analysis. Let’s talk about something really experimental and open ended. Many of us have probably heard of this. If you work in C or Rust. These are amazing tools. Why can’t we have those tools for JS. Of couse the languages are different. But perhaps there are some assumptions we can make that let us allow such tools. Go forth and test boldly! We saw many things today. Improve your tests. Thank you!

Phil: Clang static analyser is amazing. If you have cpp code base and use any build chain. Drop it in. Out of the box get so many useful checks. Some of these... I guess there’s clang tidy and clang static analyser and does simulated execution of your program. It occured to me that the ease with which you can get these, we don’t have any of that in the JavaScript world. One thing I have looked into. One thing I looked into is syntax query rules in ESLint, you can query the AST, these can be really useful. Other than that, there.

Maya: Thales and Lemma seem interesting. ...

Jesse: The ... folk have been ... their VS Code tools are amazing. I can plug into that. That’s what I am thinking of.

https://www.webengineshackfest.org/

Add a custom sidebar

Clone this wiki locally