Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/flatten serialization #2049

Closed

Conversation

kartik4949
Copy link
Collaborator

Description

Related Issues

Checklist

  • Is this code covered by new or existing unit tests or integration tests?
  • Did you run make unit-testing and make integration-testing successfully?
  • Do new classes, functions, methods and parameters all have docstrings?
  • Were existing docstrings updated, if necessary?
  • Was external documentation updated, if necessary?

Additional Notes or Comments

@kartik4949 kartik4949 marked this pull request as draft May 10, 2024 17:14
@blythed
Copy link
Collaborator

blythed commented May 13, 2024

Solution for inline-encoding

If a Document has a Schema (e.g. in ibis all Document instances have a Schema) then:

  • bytes are encoded directly (don't need a _BaseEncodable in "_leaves".
  • if Encodable: then the bytes are saved directly in the field (as raw or Base64)
  • if Artifact: then the bytes are referred to with &<file_id>
  • if File: then the file is referred to with file://

@kartik4949 kartik4949 force-pushed the feature/flatten_serialization branch from d710e24 to 09be7a9 Compare May 13, 2024 20:12
@jieguangzhou
Copy link
Collaborator

I rebased the code and created a public branch for collaborative commits.
#2056

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants