FlowETL DataPresentSensor fails if file does not exist

[jc-harrison]

When FlowETL is used to ingest CDR data from a file, the first task creates a foreign table for reading the file contents via file_fdw. This can be done before the file arrives - file existence is not checked when creating the foreign table.

The next task is a DataPresentSensor sensor which should check whether the file has arrived and contains data, and keep checking until it does. However, if the file has not yet arrived then the task will encounter a psycopg2 exception and fail (rather than rescheduling, as we'd want it to do). This can still function much like a sensor if appropriate choices of the retries and retry_delay args are used, but this is not the intended purpose of the retry mechanism.

DataPresentSensor should register an unsuccessful "poke" if the file is missing (as is the case if the file is present but empty), instead of failing the entire task.

[jc-harrison]

Should be addressed along with #5090, I think.

[greenape]

Might be more of a fiddle than it first seems, because we're typically using a program to load the file and you want to register a poke only if the file isn't there. I think at the least we need to know which error codes we want to handle here.

[greenape]

In fact we maybe actually want to move the check for the file existing into the create operator, because it'd be tricky to distinguish say, zcat failing because a file doesn't exist from zcat failing because the permissions are wrong?