v2.1.0

Streaming Runtime

• Incremental window: for example, calculate average of an 1 hour window, it will calculate incrementally, do not need to save 1 hour data any more. Check incremental computation.
• Add sendNilField option to allow nil output (avoid key changes in csv)
• Support prop func in SQL query and data template
• Add reverse func
• Experimental: rule option blocking mode, in this mode, source/operators will not discard data when buffer full (back pressure) but block and wait. Useful in

IO (source/sink)

• EdgeX V4 support
• MQTT 5 support
• File source supports watch in addition to periodical check
• Support binary format for sink
• Kafka sink streaming mode and supports compression
• HTTP source/sink supports setting header in auth request
• Support XML format
• Encryptor support AES gcm mode

Operation

• Support metrics dump
• Enhance prometheuse metrics for Kafka sink and SQL source

Breaking Changes

• EdgeX supports V4 only
• File source remove parallell property
• Prometheuse metric rename label “instance” to “op_instance”

Change List

• docs(rule): doc of triggered prop by @ngjaying in #3338
• chore: sendError defaults to false by @ngjaying in #3339
• ci: fix doc version by @ngjaying in #3340
• feat(enc): support aes gcm by @ngjaying in #3344
• feat(rule): support send out nil fields by @ngjaying in #3342
• fix(func): object_construct always issue keys by @ngjaying in #3345
• fix: remove tryrun in ruletest by @Yisaer in #3348
• feat: support inc agg window by @Yisaer in #3322
• chore(test): change video by @ngjaying in #3354
• feat: support xml converter by @Yisaer in #3341
• feat(func): add reverse function by @Dream95 in #3357
• feat: support incremental window in having by @Yisaer in #3358
• build: fix make lint error by @Dream95 in #3363
• fix: sql source support null value by @Yisaer in #3365
• fix(conn): conn status nil exception by @ngjaying in #3372
• feat: support kafka sink compression by @Yisaer in #3374
• feat: support auth request headers by @Yisaer in #3376
• feat: support tumbling/sliding inc window by @Yisaer in #3361
• feat: validate confkey by @Yisaer in #3378
• fix(planner): clean subtopo when creation fail by @Yisaer in #3383
• feat: rewrite duration in conf by @Yisaer in #3387
• feat(io): support mqtt5 by @ngjaying in #3384
• feat(mqtt): v5 sink supports properties by @ngjaying in #3390
• feat: support inc window filter by @Yisaer in #3379
• fix(kafka): fix dynamic prop problem by @ngjaying in #3391
• feat: support merge tokens into body by @Yisaer in #3392
• fix(neuron): avoid flooding logs when disconnect by @ngjaying in #3393
• feat: support fileDir notify source by @Yisaer in #3380
• docs: add kafka sink doc by @Yisaer in #3396
• feat: support incremental hopping window by @Yisaer in #3386
• feat(trace): support mqtt 5 trace by @ngjaying in #3394
• feat: support sql lookup template sql by @Yisaer in #3398
• fix: fix replace duration by @Yisaer in #3400
• fix(log): modify nng sock log by @ngjaying in #3403
• feat: support align incremental window by @Yisaer in #3399
• chore: merge exp branch by @ngjaying in #3402
• refactor(neuron): adapt to multi tags by @ngjaying in #3405
• fix: async import data return response by @Yisaer in #3408
• fix: replace props when get by @Yisaer in #3409
• fix: fix flaky TestFileDirSource by @Yisaer in #3412
• fix: fix incrmental sliding window delay function by @Yisaer in #3411
• fix(mqtt): conn subs npe by @ngjaying in #3415
• feat: add source span tag by @Yisaer in #3413
• fix: fix incremental sliding delay window by @Yisaer in #3417
• feat: support incremental window state by @Yisaer in #3406
• feat(converter): support binary encode by @ngjaying in #3420
• fix(window): sliding delay time by @ngjaying in #3424
• feat: support incremental event sliding window by @Yisaer in #3421
• feat: remove promethues tag by @Yisaer in #3427
• fix(io): file source ignore endLine buffer by @ngjaying in #3431
• feat: support incremental event hopping window by @Yisaer in #3429
• fix(state): untriggerd rule always cancel by @ngjaying in #3436
• fix(sql): remove hot path log by @Yisaer in #3438
• feat: revise metrics code by @Yisaer in #3430
• feat: support event tumbling/count window by @Yisaer in #3434
• build(deps): Bump golang.org/x/crypto from 0.27.0 to 0.31.0 by @dependabot in #3439
• feat: support incremental window state by @Yisaer in #3440
• feat: revise grafana panel by @Yisaer in #3452
• docs: import dashboard in grafana by @Yisaer in #3453
• fix: fix sync cache context by @Yisaer in #3451
• fix: converter lose precision by @Yisaer in #3445
• feat(io): file source support inotify subscribe by @ngjaying in #3448
• fix: fix xml converter panic by @Yisaer in #3455
• feat: support event inc window state by @Yisaer in #3446
• feat: support incremental avg by @Yisaer in #3457
• docs: add incremental doc by @Yisaer in #3462
• feat: push down alias decode by @Yisaer in #3460
• chore: merge exp features and fixes by @ngjaying in #3467
• fix(win): sliding delay lost previous input by @ngjaying in #3471
• feat: support inc agg function by @Yisaer in #3465
• feat: upgrade base image to 1.23.4 by @Yisaer in #3479
• feat: support config portable timeout by @Yisaer in #3464
• feat: add alias reference calculate swtich by @Yisaer in #3477
• fix: downgrade sqlite to restore arm7 support by @Yisaer in #3481
• feat(io): merge dirwatch into file source by @ngjaying in #3480
• feat: upgrade go 1.23.4 by @Yisaer in #3484
• feat: support io metrics by @Yisaer in #3443
• feat: support dump metrics by @Yisaer in #3488
• refactor: improve window gc input perf by @Yisaer in #3501
• chore(ci): fix docs deploy ci by @Swilder-M in #3507
• docs: add inc agg function doc by @Yisaer in #3493
• fix(cli): modify nil != nil bug by @maosenlin in #3509
• refactor(onnx): separate impl by @ngjaying in #3521
• chore(ci): fix unable to establish ssl connection by @Swilder-M in #3523
• build(deps): Bump golang.org/x/net from 0.29.0 to 0.33.0 by @dependabot in #3524
• docs: add sendNilField doc by @Yisaer in #3529
• feat: revise sync cache metrics by @Yisaer in #3502
• feat: support disableBufferFullDiscard option by @Yisaer in #3530
• docs: add plugin timeout doc by @Yisaer in #3533
• feat: dump state before stop rule by @Yisaer in #3468
• docs: add incremental docs by @Yisaer in #3531
• fix: revise kafka sink conf/metrics by @Yisaer in #3494
• fix(http): server dynamic update route by @ngjaying in #3336
• feat(edgex): adapt to message v4 by @ngjaying in #3532
• test: add fvt test by @Yisaer in #3535
• docs(file): file source watch by @ngjaying in #3536
• fix: fix kafka sink docs by @Yisaer in #3538
• fix: fix ruletest when shared stream by @Yisaer in #3539
• feat: add kafka sink metrics by @Yisaer in #3540
• fix: enableIncrementalWindow omitempty by @Yisaer in #3542
• refactor(kafka): change to byte collector by @ngjaying in #3545
• fix(state): IncrCounter concurrent bug by @chenrufeng in #3544
• fix: add kfk sink metrics panel by @Yisaer in #3547
• fix: fix kafka sinke metrics by @Yisaer in #3549
• docs: add sql lookup template doc by @Yisaer in #3548
• chore: specify qemu image by @ngjaying in #3556
• fix: fix kafka batchsize by @Yisaer in #3559
• fix(rule): start recover rule start no data by @ngjaying in #3560
• chore(ci): use ubuntu 22-04 by @ngjaying in #3563
• fix: fix kafka sink exception stats by @Yisaer in #3564
• chore(build): update python base image by @ngjaying in #3565
• fix: fix kafka sink panic by @Yisaer in #3566
• refactor(nng): sync dial by @ngjaying in #3568
• fix(rest): missing header problem by @ngjaying in #3569
• fix(edgex): slice type convert issues by @ngjaying in #3572
• docs(edgex): migrate to v4 by @ngjaying in #3577
• fix(kafka): fix kafka source bug by @dbsd11 in #3581
• fix: fix metrics dir by @Yisaer in #3586
• chore(build): base image built on 22.04 by @ngjaying in #3593
• docs: add kafka batch bytes by @Yisaer in #3589
• chore(build): fix incompatible problems by @ngjaying in #3595
• refactor(build): native arm and qemu update by @ngjaying in #3596
• feat: update grafana panel by @Yisaer in #3598
• fix(op): having op update last agg hit time by @ngjaying in #3603
• feat: revise sql source metrics by @Yisaer in #3599
• feat: add metrics dump API by @Yisaer in #3600
• fix: revise drop conn msg by @Yisaer in #3606
• fix(transform): omit empty in send single by @ngjaying in #3605
• fix(sink): sink bufferLength is not applied by @Yisaer in #3608
• chore: update edgex mod by @ngjaying in #3609

New Contributors

• @maosenlin made their first contribution in #3509
• @chenrufeng made their first contribution in #3544
• @dbsd11 made their first contribution in #3581

Full Changelog: v2.0.0...v2.1.0