Skip to content

Interactive Dataloader (back)#321

Merged
cristian-tamblay merged 11 commits into
developfrom
feat/interactive-dataloaders
Oct 23, 2025
Merged

Interactive Dataloader (back)#321
cristian-tamblay merged 11 commits into
developfrom
feat/interactive-dataloaders

Conversation

@Riul1999
Copy link
Copy Markdown
Collaborator

@Riul1999 Riul1999 commented Oct 1, 2025

Summary

Add option n_sample to select how many rows to load of a dataset, to enable soft loading of datasets at the moment of select dataloader options

Type of change

  • Back end new feature.
  • Refactoring.

Changes

  • Add n_sample option into dataset flow to avoid loading the whole dataset every time.
  • Get n_sample from job API, set as optional argument to the API. None by default.
  • Use n_sample in dataset job flow, set as optional argument. None by default.
  • Change dataset job flow, now it will reuse previous dataset folder_path if n_sample is given and file_path is not empty string.

How to Test

Run pytest ., check dataloader, api and dataset_job tests as the changes targeted these modules

@Riul1999 Riul1999 self-assigned this Oct 1, 2025
@Riul1999 Riul1999 added the back Backend work label Oct 1, 2025
@Riul1999 Riul1999 marked this pull request as ready for review October 6, 2025 01:42
@Riul1999 Riul1999 requested review from Creylay and Felipedino October 6, 2025 01:50
@cristian-tamblay cristian-tamblay merged commit be9c7d4 into develop Oct 23, 2025
18 checks passed
@cristian-tamblay cristian-tamblay deleted the feat/interactive-dataloaders branch October 23, 2025 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

back Backend work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants