WARC AST

[BubuAnabelas]

I'd bee great to have an Abstract Syntax Tree for WARC files since sometimes can become huge and searching through all the file is tedious.
I would do it myself but have no idea where to start.

[wooorm]

Why do you think a syntax tree would solve this?

To start you could write something like mdast, nlcst, or hast.

[BubuAnabelas]

Because it would be useful to derive some other formats as CDX. Besides it can help to do some statistical analysis and to find some important fields (e.g first and last WARC-Record-ID) to be able to continuously add more fields without having to re-parse the file each time.

I'll take a look to those repos and start my own. Thanks!

[ChristianMurphy]

Thanks for starting the discussion @BubuAnabelas!
We're in the process unifying ideas in with discussions unifiedjs/collective#44
If you'd like to continue this thread, or start a new one https://github.com/syntax-tree/unist/discussions/categories/ideas will be the home for ideas going forward.