Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Redundant feature materialization and premature incremental materialization timestamp updates #4

Merged
merged 8 commits into from
Oct 17, 2023
Merged

fix: Redundant feature materialization and premature incremental materialization timestamp updates #4

merged 8 commits into from
Oct 17, 2023

Conversation

KarolisKont
Copy link

This PR mimics this on: feast-dev#3789

james-crabtree-sp and others added 8 commits October 9, 2023 10:59
@james-crabtree-sp
SAASMLOPS-767 wait for jobs to complete
3d85953 
Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
SAASMLOPS-805 Stopgap change to fix duplicate materialization of data
850769e 
Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
SAASMLOPS-805 save BYTEWAX_REPLICAS=1
21a5f05 
Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
SAASMLOPS-809 fix bytewax workers so they only process a single file (#6
49497ec 
)

* SAASMLOPS-809 fix bytewax workers so they only process a single file

* SAASMLOPS-809 fix newlines

Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
SAASMLOPS-833 add configurable job timeout (#7)
5601d5f 
* SAASMLOPS-833 add configurable job timeout

* SAASMLOPS-833 fix whitespace

Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
develop Run large materializations in batches of pods
d44e415 
Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
master Set job_batch_size at least equal to max_parallelism
4b9d2a4 
Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@james-crabtree-sp
master clarity max_parallelism description
3544c6f 
Signed-off-by: James Crabtree <james.crabtree@sailpoint.com>
@KarolisKont KarolisKont merged commit 2949d12 into ml_platfrom_v034 Oct 17, 2023
29 checks passed
@KarolisKont KarolisKont deleted the sailpoint_master branch October 17, 2023 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
lgtm ok-to-test
Projects
None yet
Milestone
No milestone
2 participants
@KarolisKont @james-crabtree-sp