Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: introduce stride in dataset #360

Merged
merged 2 commits into from
Apr 10, 2024
Merged

feat: introduce stride in dataset #360

merged 2 commits into from
Apr 10, 2024

Conversation

ab93
Copy link
Member

@ab93 ab93 commented Apr 9, 2024 •
edited
Loading

Useful for flattened vectored data

@ab93
feat: introduce stride in dataset
ae2eae1 
Signed-off-by: Avik Basu <ab93@users.noreply.github.com>
@ab93 ab93 self-assigned this Apr 9, 2024
@ab93 ab93 marked this pull request as ready for review April 9, 2024 21:43
@ab93 ab93 requested a review from gulshan02 April 9, 2024 21:43
@s0nicboOm
Copy link
Contributor

s0nicboOm commented Apr 10, 2024 •
edited
Loading

I think we need to call this config here too:

numalogic/numalogic/udfs/trainer/_base.py

Line 117 in ae2eae1

train_ds = StreamingDataset(input_, model.seq_len)

I think there are many places where I think we should just add this parameter.

@ab93
chore: add stride from config in udf
5b996f6 
Signed-off-by: Avik Basu <ab93@users.noreply.github.com>
s0nicboOm
s0nicboOm approved these changes Apr 10, 2024
View reviewed changes
Copy link
Contributor

@s0nicboOm s0nicboOm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm!

@ab93 ab93 merged commit 0cbaa52 into main Apr 10, 2024
8 of 9 checks passed
@ab93 ab93 deleted the fix/dataset-stride branch April 10, 2024 22:29
SaiSharathReddy pushed a commit to SaiSharathReddy/numalogic that referenced this pull request Apr 11, 2024
@ab93
feat: introduce stride in dataset (numaproj#360)
9f847f4 
Useful for flattened vectored data

---------

Signed-off-by: Avik Basu <ab93@users.noreply.github.com>
Co-authored-by: Avik Basu <ab93@users.noreply.github.com>
Signed-off-by: skondakindi <saisharathreddy_kondakindi@intuit.com>
SaiSharathReddy pushed a commit to SaiSharathReddy/numalogic that referenced this pull request Apr 29, 2024
@ab93
feat: introduce stride in dataset (numaproj#360)
5ad275b 
Useful for flattened vectored data

---------

Signed-off-by: Avik Basu <ab93@users.noreply.github.com>
Co-authored-by: Avik Basu <ab93@users.noreply.github.com>
Signed-off-by: skondakindi <saisharathreddy_kondakindi@intuit.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@s0nicboOm s0nicboOm s0nicboOm approved these changes

@qhuai qhuai Awaiting requested review from qhuai qhuai is a code owner

@nkoppisetty nkoppisetty Awaiting requested review from nkoppisetty nkoppisetty is a code owner

@gulshan02 gulshan02 Awaiting requested review from gulshan02

Assignees

@ab93 ab93

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ab93 @s0nicboOm