Have you seen TcBlack?

[Roald87]

Hi,

I just found your repo when looking for structured text parsers. Your project looks very interesting!

In your readme you mention you want to make a formatter based on Python's Black. I already started such a project: TcBlack.

The current implementation of TcBlack tries to parse structured text using regex. This is not very scalable and therefore I was considering writing my own parser, because I couldn't find anything. But now I see your project and it seems like maybe I could use blark to do the parsing. What is the status of the parser? How much can it parse currently? We could maybe also collaborate?

Roald.

[klauer]

Hi Roald,

Thanks for reaching out - TcBlack is really neat! I expect that with regular expressions it could work reasonably well most of the time for the declarations portion, but maybe less so for the code/implementation itself. But a tool like black should work in all cases (for well-formed input) or it wouldn't be convenient to use, so I can see why you considered going in the direction of a parser. Writing such a parser is an awful lot of work, and any other reference you'll find targets only the official IEC61131-3 standard.

The blark parser is in reasonably good shape at this point, I think. It parses all of the code from my organization (all of the TwinCAT repos here: https://github.com/pcdshub/) along with some others I've found interesting (a few repos from https://github.com/BurksEngineering/). There are definitely things the grammar does not handle at this point, and I've mostly been tackling them on an as-needed basis.

The parser itself is pretty slow, though, as a fair warning. For my purposes, this isn't really a concern.

I would be interested in collaborating. Thinking of some things that could be easy to contribute - I'd be grateful to get code examples, Beckhoff documentation references, or ideally test cases that blark can't handle currently. I'd also be interested in hearing more of your thoughts on what the appropriate resulting style should be (I just saw your document and think it's a great start).

Where blark itself would fit into your tool, I'm not entirely sure. Were yours to be rewritten as a Python-based tool, you could just import and use it as-is. Otherwise, blark might need better command-line output (e.g., a JSON representation of code) for easier interoperability.

In any case, my next upcoming goal on the roadmap-in-my-head is to get the Sphinx documentation tool (#12) working well enough. I'm hoping to use it to offer up better API documentation for our shared libraries.

Ken

[Roald87]

Thanks! Yes for the declaration it still works, but it becomes pretty unreadable as exceptional cases get added over time. Are there any parsers which parse IEC61131-3? I haven't come across any.

Good to hear it works well on a few projects already!

Speed might be a concern, especially if it is integrated into Visual Studio/XAE shell. Is it also slow when formatting a single file?

Let me see if I can get blark running and run some of my tests with it. Maybe something comes up. Although my test cases are only for the declaration part. The document contains my thoughts on the style so far. My expectation was that once I would start implementing it, the exceptions would start to roll in and I would have to think about it as I go.

Yes integrating blark might be challenging. Some part of TcBlack needs to be in a .NET language I think, in order to make the Visual Studio extension.

I'll start to use blark for a few things to get a feel of the possibilities. Will keep you posted!

Roald.

[klauer]

Speed: There are startup costs such as reading the IEC grammar that are expensive (400ms to load up the grammar), then applying it on a file can take significant time as well. Order of 1 second per file is probably a reasonable assumption at the moment.

It's possible it could be improved by simplifying/improving the grammar or digging into the details of the dependency lark, but at least at this point my priority is roughly in the order of - features, correctness, and speed in a distant last place.

A few other projects that might be of interest - no hard feelings if you end up choosing one of these over blark :) -

• https://github.com/nucleron/matiec (older ST compiler with parser)
• https://github.com/thiagoralves/OpenPLC_v3 (uses matiec, may have some customizations)
• https://github.com/PLC-lang/rusty (rust-based ST compiler)
• https://github.com/jubnzv/iec-checker (OCaml grammar + static analysis tool)

[jubnzv]

Hi,

Author of iec-checker here. I was alerted that you mentioned my project here, so I just want to add a few points about repositories listed above that I have worked with.

https://github.com/nucleron/matiec (older ST compiler with parser)

Matiec supports IEC61131-3 second edition, without classes, namespaces and other fancy features. If you decide to use it, I recommend applying the fixes from the company I worked for (see my recent commits). There were some segmentation faults in the parser that we found using fuzzing. After these fixes it seems pretty stable, and there haven't been any crashes in the few months of continuous fuzzing.
You may also be interested in the latest improvements in the parser in the upstream repository. They extended the parser with some constructs from CoDeSys and supported JSON dumps, which may be useful for you.

https://github.com/jubnzv/iec-checker (OCaml grammar + static analysis tool)

iec-checker has the ability to parse ST source code and dump AST and CFG to JSON format, so you can process it with your language of choice. This feature is used in test suite written in Python.
I'm not sure about the parser speed compared to other implementations, but I use menhir for the parser. This is a stable industrial grade LR(1) parser generator for OCaml and Coq.
The grammar completely covers IEC61131-3 3 ed. features used in the company I worked for. If something is missing there, you could create an issue in the repo.

[klauer]

Thanks for the additional information and context, @jubnzv!

[Roald87]

Wow thanks for all the ideas @klauer and @jubnzv! Looks like I have some research to do.