Custom utilities functions

[bd82]

This discussion originated here: #1679

Background

Early on Chevrotain used 3rd party utilities library such as lodash/underscore,
But at some point I wanted to remove external dependencies, so I've implemented the basic high order functions as needed.

• https://github.com/Chevrotain/chevrotain/tree/master/packages/uilts

Different Styles

Nowadays the standard library is richer, However it is still not very rich... so this creates two style of invoking high order functions.

arr.foo(() => {})
foo(arr, () => {})
The first:

uses the standard library so one could claim it is more idiomatic.
Requires validating if it is supported in the targeted runtimes
Note the targeted runtimes and ECAMScript spec are both "moving targets".
Would throw exception at runtime if arr is undefined.
Which means we need to be certain the arr can never be undefined in any
The second may be less idiomatic, but also more stable, both in terms of tracking runtime capabilities and possible runtime exceptions.

Consistent Style

But there is another concern, which is consistent style.
The two styles above read very differently, I would rather not mix and match those two styles, particularly not in the same function/file/project. So until some point in the future where ECMAScript has a proper rich standard library which is supported in all runtimes, maybe it is better to stick with custom functions.

[bd82]

Perhaps Chevrotain should use a 3rd party utils library again?

It is also possible to challenge my past decision to implement the custom functions manually.
I don't quite recall all the exact details of why I've done so... ❔

Things to consider:

• minified package size

• Which custom library to pick

  • https://github.com/ramda/ramda
  • https://github.com/sanctuary-js/sanctuary
  • lodash
  • other?

• Performance

  • Generally the parsing runtime code does not use high order functions in performance intensive areas
    But perhaps the choice of library could affect the grammar initialization performance.

• Benefit of reducing amount of code in the library thus reducing TCO

[NaridaL]

I don't think mixing styles is much of an issue. It makes clear which are standard functions and which not.

The main advantage to me for replacing custom functions with built in ones is that it makes it easier to read the code for everyone as most JS developers are going to be familiar with the standard functions.

As for custom-library/not very rich standard lib, almost all the custom functions are covered by the built ins nowadays. E.g. flatten can be replaced by flatMap in most uses in the codebase.

EDIT: Oh yeah and as for null-safety/undefines, I wanted to enable strict in the tsconfig anyway, this should solve that issue.

[bd82]

I'm thinking of trying to use a 3rd party utils library again and completely remove the @chevrotain/utils pakage, so read my reply with that context in mind.

The main advantage to me for replacing custom functions with built in ones is that it makes it easier to read the code for everyone as most JS developers are going to be familiar with the standard functions.

I agree it can make the code easier to read for some (many?), although it can make the code harder to write / maintain e.g:

• May need to check compatibility charts for every built in function used, unless we choose to target only specific node.js versions (which have automated tests for).
• Will still need to implement some custom functions (e.g isEmpty vs arr.length === 0).

As for custom-library/not very rich standard lib, almost all the custom functions are covered by the built ins nowadays. E.g. flatten can be replaced by flatMap in most uses in the codebase.

I'm not sure sure all can be replaced directly, e.g:

• isEmpty
• mapValues (will be replaced with 2 separate functions, Object.values().map)
• first
• last
• drop
• dropRight
• ...

and I'd prefer to write logic in a common generic functional style when possible.

• Note some of these function names (e.g dropRight) are not lodash/ramda specific but are generic high order functions names that can be found in multiple programming languages

[bd82]

My primary objective is to reduce the TCO of developing / maintaining Chevrotain.
I suspect this can be more fully achieved by removing the home-brewed utils package completely and
replacing it with a 3rd party alternative.

This would also support making the code base more standardized if a popular 3rd party library is chosen.

• e.g see: https://www.npmjs.com/package/lodash (~150K dependents) although probably not as much as using only built in functions.

This would also remove any concerns in regards to runtime engine compatibility as long as the 3rd party library chosen targets a lower/equal ECMAScript version than Chevrotain (another reason to stop targeting ES5.1...)

The main disadvantage would be the increased bundle size, but perhaps this can be mitigated nowadays by modern bundling tools

• See: https://lodash.com/per-method-packages

[bd82]

E.g. flatten can be replaced by flatMap in most uses in the codebase.

This is a good example for the compatibility issue with the native built-in functions.
It looks like flatMap is part of the ECMAScript 2019 spec, and while it is supported in most modern runtimes, there are some "live" versions of Safari which do not fully implement the spec for flatMap. (IOS 12 & 12.2)

• https://kangax.github.io/compat-table/es2016plus/

I randomly spotted this while trying to decide which runtime to target in #1272 ...

[NaridaL]

Yeah flatMap was maybe a bad example. Checking charts should not be necessary as TypeScript checks what methods you use (ie flatMap is an error unless "lib": "es2019" is set in the tsconfig).

Are you OK with replacing the util functions which have a direct standard replacement with the standard functions? We can still use a 3rd party lib for the remaining ones.

[bd82]

TypeScript checks what methods you use

Very handy.

Are you OK with replacing the util functions which have a direct standard replacement with the standard functions?

I'll probably would like to make an attempt first to replace everything with a 3rd party library as I would like to keep a consistent style
for the high order functions and not worry about possible (even if small chance) of undefined / null values causing runtime errors.

It is obviously a subjective opinion, but an hybrid approach would not allow me to get rid of (virtually) all of the utilities package
while also retaining the "standard" functional style high order functions, So it would solve part of one problem (code size) but make another problem (inconsistent style for writing logic), more severe...

• There are already two styles, as the parser's runtime logic is implemented in a pure imperative style (loops) for performance reasons.

To be honest I would have liked to play with something that is supposedly more pure functional style like Ramda.

• https://fr.umio.us/why-ramda/
  But I will probably try Lodash first as it should have 1 to 1 drop in replacements for almost all of the utils.
  And I may be an easy win re-introducing it.

[bd82]

Note that introducing a 3rd party lib would likely increase the minified package size for some consumers (web based).
I will try to mitigate it by using smart bundling tools (Tree shaking),
But at the end of the day I have limited time to maintain this free OSS library, so lowering TCO wins out.

• RoadMap / Future Plans #1270