Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File.from_stream does not load content into memory #1065

Merged
merged 14 commits into from Nov 4, 2022
Merged

Conversation

shnela
Copy link
Contributor

@shnela shnela commented Oct 27, 2022

No description provided.

@github-actions
Copy link

github-actions bot commented Oct 27, 2022

Test Results

    2 files      2 suites   1m 51s ⏱️
486 tests 486 ✔️ 0 💤 0
972 runs  972 ✔️ 0 💤 0

Results for commit 5ee9e91.

♻️ This comment has been updated with latest results.

@@ -235,7 +235,7 @@ def test_get_html_from_bokeh(self):
result = get_html_content(p)

# then
self.assertTrue(result.startswith('<!DOCTYPE html>\n<html lang="en">'))
self.assertTrue(result.lstrip().startswith('<!DOCTYPE html>\n<html lang="en">'))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pibbyback fixing test which always fails on my local Ubuntu.

self,
path: Optional[str] = None,
content: Optional[bytes] = None,
extension: Optional[str] = None,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, @Herudaio accepted removing extension from this init.

src/neptune/new/exceptions.py Outdated Show resolved Hide resolved

@staticmethod
def from_stream(stream: IOBase, seek: Optional[int] = 0, extension: Optional[str] = None) -> "File":
# TODO: docs (update)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could inform user that from now on, he will be able to use file created with File.from_stream at most once.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shnela shnela changed the title Introduce FileType.STREAM File.from_stream does not load content into memory Oct 28, 2022
@shnela shnela force-pushed the jk/in-memory-stream branch 2 times, most recently from d0554c1 to b2d0002 Compare October 31, 2022 09:38
@@ -0,0 +1,170 @@
#
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove all these _ prefixes and move it to internal package. Everything that is outside of internal should be treated as a user-exposed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

src/neptune/new/types/atoms/file.py Show resolved Hide resolved
@@ -0,0 +1,173 @@
#
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh? Why inside of neptune.internal instead of neptune.new.internal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, fixing

Raalsky
Raalsky previously approved these changes Nov 4, 2022
Raalsky
Raalsky previously approved these changes Nov 4, 2022
@shnela shnela merged commit 6fca974 into master Nov 4, 2022
@shnela shnela deleted the jk/in-memory-stream branch November 4, 2022 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants