Nifi integration revision

[QuanticPony]

NiFi integration revision

As we aim for a new release I bring a revision of the NiFi integration with a couple of improvements:

• [Bug]: plc4j-tools-connection-cache: broken connections remaing in the cache on timeout #900: this made the processors be able to auto-reconnect with small changes. And allowed to make:

  • Apache NiFi integration should allow Expression Language #593 : this was already done for the addresses. But the connection string had to be constant. Expression Language is now supported on the connection string.
  • Nifi BasePlc4xProcessor define validation logic for PLC_CONNECTION_STRING and PLC_ADDRESS_STRING properties #629 : this was an old TODO. I have added validation for the connection string (if not EL in use) and to the addresses (if not EL in use in neither of them as we need the driver).

• Add FilePropertyAccessStrategy: now we can read the addresses from a file in JSON format. Same as 'Address Text' property but in a file. Expression Language can be used and validation for the tags also works.

• Processors code revision: rewrote the onTrigger methods of the processors so they look more similar between them. To make them more maintainable.

• Complete FlowFlile transfer: some FlowFiles did not transfer to failure correctly. Now the incoming FlowFile is sent to failure and one attribute is added with the localized message in case of any exception.

• Timestamp field name: It was a fixed ts field/attribute we added. I have made a property that defines this field's name.

• Update Readme to reflect changes.

Closes #593
Closes #629

[chrisdutz]

Thanks for this ;-)

Unfortunately I am probably not the one ideal to review NiFi related things, but I would trust you and your changes. So if I don't hear any objections till Monday, I'll merge this PR into develop before cutting the release branch.

[chrisdutz]

By the way ... would be cool, if you could sign up to the dev-list and start communicating with us there ;-)

[ottobackwards]

Thanks for the work!

• There is a lot of duplication of methods between each of the processors. These should be moved so that we don't have multiple copies in each processor
• I would like to see more tests for the new evaluation enabled things
• I would like to see tests around the caching with evaluations
• having properties read from files is not a great pattern for nifi, and there are no tests for it

in short, please add tests for all the new things and things that are now effected by the permutations possible. Please de-duplicate the methods in the various processors.

In the future, smaller pr's would make it easier to review

[QuanticPony]

Sorry for the long PR. I would normally split it, but though posting as early as possible would be best in this case.
Duplicated code removed. Unit tests added.
i added the read tags from a file as I have seen it in some production environments with other OPCUA processors. I think it could be easier to migrate to PLC4X if we have them. If you don't see it clear i can remove it

[QuanticPony]

I'm done. Tell me if anything else is needed. Will have a look at it over the weekend

[TurkerTunali]

So, should we try the NiFi integration now?

[chrisdutz]

Well it should be part of the Release ... so feel free to try.