Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor processor service, processor and node #1354

Merged
merged 73 commits into from
Feb 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
19ca98b
cmd line option
hariso Jan 26, 2024
717db1b
wip
hariso Jan 26, 2024
d71a2f8
yaml
hariso Jan 26, 2024
28e66c3
Merge branch 'main' into haris/wasm-processors-config
hariso Jan 29, 2024
26b2952
use new registry
hariso Jan 29, 2024
30a84bb
compiles
hariso Jan 30, 2024
8c3f2ab
cmd line option
hariso Jan 26, 2024
c5ff6a4
wip
hariso Jan 26, 2024
b4d5769
yaml
hariso Jan 30, 2024
8153859
use new registry
hariso Jan 29, 2024
6e8877f
compiles
hariso Jan 30, 2024
9371444
Merge branch 'haris/wasm-processors-config' of github.com:ConduitIO/c…
hariso Jan 30, 2024
f820472
comments
hariso Jan 30, 2024
7eb5e8a
comments
hariso Jan 30, 2024
c504e45
handle error
hariso Jan 31, 2024
02d60c2
implement conversion methods
hariso Jan 31, 2024
4aa5772
comments, convert opencdc records
hariso Jan 31, 2024
9ea621b
inspectable processor close
hariso Jan 31, 2024
65ae90f
rename Instance.Type to Instance.Plugin
hariso Jan 31, 2024
7d6c2c8
simplify store
hariso Jan 31, 2024
0585753
processor node changes, update tests
hariso Feb 2, 2024
948fa0c
fix parallel_test
hariso Feb 2, 2024
6f2cf9b
move around stuff, fix tests
hariso Feb 5, 2024
eb7235f
inverse init proc instance
hariso Feb 6, 2024
a20bf6d
inverse init proc instance
hariso Feb 6, 2024
7805ac4
fix tests
hariso Feb 6, 2024
d38aaf8
fix tests
hariso Feb 6, 2024
afa0441
fix tests
hariso Feb 6, 2024
0ee7ade
fix stream example
hariso Feb 6, 2024
0a808f7
tests, linter
hariso Feb 6, 2024
72c1368
lint
hariso Feb 6, 2024
b259953
fix orchestrator_test
hariso Feb 6, 2024
9723cf1
rename interface
hariso Feb 6, 2024
2743998
add mock
hariso Feb 6, 2024
acff69b
comments
hariso Feb 6, 2024
c01e945
inspectable_processor_test
hariso Feb 6, 2024
d612a73
fix todos
hariso Feb 6, 2024
5eaf577
fix todos
hariso Feb 6, 2024
4922b76
registry tests
hariso Feb 6, 2024
216fe77
pr feedback
hariso Feb 6, 2024
5972052
fix test
hariso Feb 6, 2024
28bc481
regenerate
hariso Feb 6, 2024
cd60557
remove type
hariso Feb 6, 2024
a21962c
bwc
hariso Feb 7, 2024
942c53d
regenerate
hariso Feb 7, 2024
253029f
linter
hariso Feb 7, 2024
3d969c9
linter, remove type from config
hariso Feb 7, 2024
d11d21c
loop
hariso Feb 7, 2024
2eaf3f3
Revert "loop"
hariso Feb 7, 2024
09e6cf0
refactor runnable processor
lovromazgon Feb 7, 2024
fd5b31c
prevent us from creating a runnable processor twice
lovromazgon Feb 7, 2024
b011564
organize imports
lovromazgon Feb 7, 2024
35276d7
Merge branch 'haris/wasm-processors-config' of github.com:ConduitIO/c…
hariso Feb 7, 2024
c1b8fb4
fix compilation errors
hariso Feb 7, 2024
cec60ab
cleanup old processors, fix errors
hariso Feb 7, 2024
37aad7e
linter
hariso Feb 7, 2024
abf7e2b
go mod tidy
hariso Feb 7, 2024
17f3bb7
comments
hariso Feb 7, 2024
a30f2ed
comments, cleanup
hariso Feb 7, 2024
6eeb95b
loop
hariso Feb 7, 2024
a3b123d
record conversion test
hariso Feb 8, 2024
43dba95
more tests
hariso Feb 8, 2024
a3ee4b2
refactor
hariso Feb 8, 2024
4c24639
tests
hariso Feb 8, 2024
28faa4f
fix test
hariso Feb 8, 2024
bc3ba42
add tests for runnable_processor
hariso Feb 8, 2024
b51cc1d
add tests for runnable_processor
hariso Feb 8, 2024
fc28a50
update deps
hariso Feb 8, 2024
370c9c6
pr feedback
hariso Feb 8, 2024
26b2df5
pr feedback
hariso Feb 9, 2024
e1eb20c
add test
hariso Feb 9, 2024
03e2aa5
rename connector and processor plugin service
lovromazgon Feb 9, 2024
854a292
rename plugin service to connector plugin service in provisioning pac…
lovromazgon Feb 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ dist/

# Standalone connectors
/connectors
# Standalone processors
/processors

/pipelines

Expand Down
23 changes: 4 additions & 19 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ require (
buf.build/gen/go/grpc-ecosystem/grpc-gateway/protocolbuffers/go v1.32.0-20231027202514-3f42134f4c56.1
github.com/Masterminds/semver/v3 v3.2.1
github.com/NYTimes/gziphandler v1.1.1
github.com/antchfx/jsonquery v1.3.3
github.com/bufbuild/buf v1.29.0
github.com/conduitio/conduit-commons v0.0.0-20240112191423-58fcb3055cf2
github.com/conduitio/conduit-connector-file v0.6.0
Expand All @@ -17,35 +16,28 @@ require (
github.com/conduitio/conduit-connector-protocol v0.5.1-0.20240104160905-e9e61586fb8d
github.com/conduitio/conduit-connector-s3 v0.5.1
github.com/conduitio/conduit-connector-sdk v0.8.0
github.com/conduitio/conduit-processor-sdk v0.0.0-20240118151737-a75ea9e86bb8
github.com/conduitio/conduit-processor-sdk v0.0.0-20240208142744-ba2d1942a9bd
github.com/conduitio/yaml/v3 v3.3.0
github.com/dgraph-io/badger/v4 v4.2.0
github.com/dop251/goja v0.0.0-20230531210528-d7324b2d74f7
github.com/dop251/goja_nodejs v0.0.0-20230602164024-804a84515562
github.com/gammazero/deque v0.2.1
github.com/goccy/go-json v0.10.2
github.com/golangci/golangci-lint v1.55.2
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
github.com/gorilla/websocket v1.5.1
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1
github.com/hamba/avro/v2 v2.18.0
github.com/hashicorp/go-hclog v1.6.2
github.com/hashicorp/go-plugin v1.6.0
github.com/jackc/pgx/v5 v5.5.2
github.com/jpillora/backoff v1.0.0
github.com/lovromazgon/franz-go/pkg/sr v0.0.0-20230630140346-bb9ce3f90f4a
github.com/matryer/is v1.4.1
github.com/modern-go/reflect2 v1.0.2
github.com/peterbourgon/ff/v3 v3.4.0
github.com/piotrkowalczuk/promgrpc/v4 v4.1.2
github.com/prometheus/client_golang v1.18.0
github.com/prometheus/client_model v0.5.0
github.com/prometheus/common v0.46.0
github.com/rs/zerolog v1.31.0
github.com/rs/zerolog v1.32.0
github.com/stealthrocket/wazergo v0.19.1
github.com/tetratelabs/wazero v1.6.0
github.com/twmb/go-cache v1.2.0
go.uber.org/goleak v1.3.0
go.uber.org/mock v0.4.0
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
Expand Down Expand Up @@ -82,7 +74,6 @@ require (
github.com/alexkohler/nakedret/v2 v2.0.2 // indirect
github.com/alexkohler/prealloc v1.0.0 // indirect
github.com/alingse/asasalint v0.0.11 // indirect
github.com/antchfx/xpath v1.2.3 // indirect
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
github.com/apache/arrow/go/arrow v0.0.0-20211112161151-bc219186db40 // indirect
github.com/apache/thrift v0.19.0 // indirect
Expand Down Expand Up @@ -119,7 +110,6 @@ require (
github.com/butuzov/mirror v1.1.0 // indirect
github.com/catenacyber/perfsprint v0.2.0 // indirect
github.com/ccojocar/zxcvbn-go v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/charithe/durationcheck v0.0.10 // indirect
github.com/chavacava/garif v0.1.0 // indirect
Expand All @@ -131,7 +121,6 @@ require (
github.com/denis-tingaikin/go-header v0.4.3 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/distribution/reference v0.5.0 // indirect
github.com/dlclark/regexp2 v1.8.0 // indirect
github.com/docker/cli v24.0.7+incompatible // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker v25.0.0+incompatible // indirect
Expand All @@ -153,7 +142,6 @@ require (
github.com/go-critic/go-critic v0.9.0 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sourcemap/sourcemap v2.1.3+incompatible // indirect
github.com/go-toolsmith/astcast v1.1.0 // indirect
github.com/go-toolsmith/astcopy v1.1.0 // indirect
github.com/go-toolsmith/astequal v1.1.0 // indirect
Expand Down Expand Up @@ -212,7 +200,7 @@ require (
github.com/jgautheron/goconst v1.6.0 // indirect
github.com/jingyugao/rowserrcheck v1.1.1 // indirect
github.com/jirfag/go-printf-func-name v0.0.0-20200119135958-7558a9eaa5af // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/jpillora/backoff v1.0.0 // indirect
github.com/julz/importas v0.1.0 // indirect
github.com/kisielk/errcheck v1.6.3 // indirect
github.com/kisielk/gotool v1.0.0 // indirect
Expand Down Expand Up @@ -244,7 +232,6 @@ require (
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/term v0.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/moricho/tparallel v0.3.1 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/nakabonne/nestif v0.3.1 // indirect
Expand Down Expand Up @@ -321,11 +308,9 @@ require (
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.47.0 // indirect
go.opentelemetry.io/otel v1.22.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.22.0 // indirect
go.opentelemetry.io/otel/metric v1.22.0 // indirect
go.opentelemetry.io/otel/sdk v1.22.0 // indirect
go.opentelemetry.io/otel/trace v1.22.0 // indirect
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
go.tmz.dev/musttag v0.7.2 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
Expand All @@ -346,7 +331,7 @@ require (
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.4.6 // indirect
mvdan.cc/gofumpt v0.5.0 // indirect
mvdan.cc/gofumpt v0.6.0 // indirect
mvdan.cc/interfacer v0.0.0-20180901003855-c20040233aed // indirect
mvdan.cc/lint v0.0.0-20170908181259-adc824a0674b // indirect
mvdan.cc/unparam v0.0.0-20221223090309-7455f1af531d // indirect
Expand Down
Loading
Loading