Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated the use cases #2133

Merged

Conversation

jieguangzhou
Copy link
Collaborator

Description

Related Issues

Checklist

  • Is this code covered by new or existing unit tests or integration tests?
  • Did you run make unit_testing and make integration-testing successfully?
  • Do new classes, functions, methods and parameters all have docstrings?
  • Were existing docstrings updated, if necessary?
  • Was external documentation updated, if necessary?

Additional Notes or Comments

@jieguangzhou jieguangzhou force-pushed the feat/usecases-transfer-learning branch from d10336e to a4f731e Compare June 4, 2024 08:39
if self.sample_size is not None and self.sample_size < len(data):
perm = self.random.permutation(len(data)).tolist()
data = [data[perm[i]] for i in range(self.sample_size)]
self.raw_data = pickle_encode([r.encode() for r in data])
self._data = data

Copy link
Collaborator Author

@jieguangzhou jieguangzhou Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don’t need to pin a dataset; instead, the dataset should be the latest data retrieved by the select query.

@jieguangzhou jieguangzhou changed the title Updated the Transfer learning use case Updated the use cases Jun 4, 2024
@jieguangzhou jieguangzhou force-pushed the feat/usecases-transfer-learning branch from ca3d2ee to cca98df Compare June 4, 2024 14:58
@jieguangzhou jieguangzhou linked an issue Jun 4, 2024 that may be closed by this pull request
@blythed blythed merged commit 4a705ca into superduper-io:main Jun 5, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SERIALIZE] Update and check use-cases with new serialize protocol
2 participants