SFTP event source #2693
SFTP event source #2693
Conversation
dillonstreator
force-pushed
the
sftp-event-source
branch
2 times, most recently
from
e0cc6a0
to
e47ae42
Compare
|
@dillonstreator - thanks! could you run |
dillonstreator
force-pushed
the
sftp-event-source
branch
from
9f654cd
to
5e23b16
Compare
Signed-off-by: dillonstreator <dillonstreator@gmail.com> Signed-off-by: dillon <dillonstreator@gmail.com>
Signed-off-by: dillon <dillonstreator@gmail.com>
Signed-off-by: dillon <dillonstreator@gmail.com>
…roj#2696) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dillon <dillonstreator@gmail.com>
….4.3 (argoproj#2694) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dillon <dillonstreator@gmail.com>
Signed-off-by: dillonstreator <dillonstreator@gmail.com> Signed-off-by: dillon <dillonstreator@gmail.com>
dillonstreator
force-pushed
the
sftp-event-source
branch
from
5e23b16
to
e27aa88
Compare
Signed-off-by: dillonstreator <dillonstreator@gmail.com>
dillonstreator
force-pushed
the
sftp-event-source
branch
from
788eab2
to
7e8e2a8
Compare
| // PollIntervalDuration the interval at which to poll the SFTP server | ||
| // defaults to 10 seconds | ||
| // +optional | ||
| PollIntervalDuration string `json:"pollIntervalDuration" protobuf:"varint,8,opt,name=pollIntervalDuration"` |
There was a problem hiding this comment.
Use *string to avoid parsing empty string?
There was a problem hiding this comment.
Is that preferred? I'm following the pattern defined by HDFSEventSource with CheckInterval
https://github.com/dillonstreator/argo-events/blob/980c63b8576969e67f27e91c65e049b5f9d6e072/pkg/apis/eventsource/v1alpha1/types.go#L1091-L1093
https://github.com/dillonstreator/argo-events/blob/980c63b8576969e67f27e91c65e049b5f9d6e072/eventsources/sources/hdfs/start.go#L96-L104
|
Could you also add a doc in https://github.com/argoproj/argo-events/tree/master/docs/eventsources/setup? |
Signed-off-by: dillon <dillonstreator@gmail.com>
dillonstreator
force-pushed
the
sftp-event-source
branch
from
eb12c04
to
23d588a
Compare
|
|
||
| sftpConfig := &ssh.ClientConfig{ | ||
| User: username, | ||
| Auth: []ssh.AuthMethod{ssh.Password(password)}, |
There was a problem hiding this comment.
would be good to have option for sshkey based auth instead of password based too: https://pkg.go.dev/golang.org/x/crypto/ssh#PublicKeys
There was a problem hiding this comment.
agree with @tooptoop4, the possibility to authenticate with a private key would be very valuable. in our specific case, we're unable to use the sftp event source in production due to our sftp instance being protected by private ssh key.
Signed-off-by: dillonstreator <dillonstreator@gmail.com> Signed-off-by: dillon <dillonstreator@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fixes #1652
This PR introduces an SFTP event source with support for
fsnotify.Removeandfsnotify.Createoperations.A couple of caveats/questions on the implementation:
pollIntervalDurationwindow will not be captured and result in events. This should be understood by consumers given the nature of polling.calendarevent source, if deemed necessary.fsnotify.Opandfsnotify.Eventbut doesn't directly match the fsnotify interface. A file rename, different modification time, or size difference are all captured asfsnotify.Create.Are we okay with these?
If yes, Is there a good place for me to document these caveats?
If no, I'm open to feedback on how we can adjust to support needs.