aljoscha
Follow
aljoscha
Follow
Popular repositories
-
statefun-rust Public
A Rust SDK for StateFun (https://flink.apache.org/stateful-functions.html)
-
An example of using Flink for Fault-Tolerant Stream Processing
-
609 contributions in the last year
Less
More
Activity overview
Contributed to
MaterializeInc/materialize,
apache/flink,
aljoscha/vim-colors-mostly-off
and 5 other
repositories
Contribution activity
January 2022
Created 17 commits in 1 repository
Created a pull request in MaterializeInc/materialize that received 44 comments
persist,sources: allow compaction on persisted sources
The idea behind this is:
keep track of what compaction the coordinator allows in RenderState
received AllowCompaction commands update that compact…
+63
−4
•
44
comments
Opened 10 other pull requests in 1 repository
MaterializeInc/materialize
4
open
6
merged
- coord,persist: fetch latest persist frontiers when rendering source
- dataflow,coord: tweeze apart Response and AllowCompaction messages
- debezium,persist: turn debezium state mutations into differential updates
- dataflow: demonstrate how a halfway fix leads us to reading TOO MANY records from Kafka
- dataflow,sources: fix logic that filters restored (persisted) timstamp bindings
- scratch: add 'sftp' command that connects via msftp
- dataflow,persistence: enable persistence for ENVELOPE NONE
- persist: add allow_compaction() operator
- persist: relax allow_compaction() requirements
- dataflow,sources: only restore timestamp bindings on workers that need them
Reviewed 24 pull requests in 1 repository
MaterializeInc/materialize
24 pull requests
- test: fix test for cdcv2
- persist: remove <K, V> bounds from MultiWriteHandle
- dataflow,coord: tweeze apart Response and AllowCompaction messages
-
persist: lint for
Arc::clone(&foo)vsfoo.clone() - dataflow: use a ProbeHandle to communicate source uppers
- dataflow,persistence: enable persistence for ENVELOPE NONE
- persist: ensure there is a tokio runtime available in BlobCache
- persist: split out a read-only runtime
- persist: handle trace compaction outside of main Indexed loop.
-
Reworking
import_source - dataflow,sources: fix logic that filters restored (persisted) timstamp bindings
- dataflow: extract rendering local source into its own function
- persist,sources: allow compaction on persisted sources
-
Introduce modules into
dataflow-types - dataflow,sources: only restore timestamp bindings on workers that need them
- persist: use parquet as the permanent format for data blobs
- Decouple debezium envelope processing from format decoding
- persist,dataflow: verify expected as_of when replaying streams
- persist: relax allow_compaction() requirements
- persist: pass around Metrics wrapped in an Arc
- persist: take persistence since into account when calculating source since
-
Shore up dataflow |-> coord tail protocol by removing the redundant
Completevariant and changing whenDroppedis sent. - [kafka sink] Remove blocking tx calls from dataflow thread
- dataflow: add a permanent storage format for {Source,Assigned}Timestamp