Add a File Uploader Widget #488

dcaminos · 2019-10-21T19:14:33Z

Example:

file_png = st.file_uploader("Upload a PNG image", type=([".png"]))

if file_png:
    file_png_bytes = st.file_reader(file_png)
    st.image(file_png_bytes)

tconkling

Hey - this looks really good! I left some mostly nit-picky comments on style.

But the big thing missing is tests :)

FileManager should have its own small test suite
ReportSession should have tests for handle_new_file and handle_file_chunk (alternately, I think these tests could exist in Server_test.py - there are a number of tests in there that simulate a Websocket connection from the client, and send test messages to and from that simulated client.)
The widget itself should have a simple Cypress test

frontend/src/App.tsx

proto/streamlit/proto/FileReader.proto

proto/streamlit/proto/FileUploadStatus.proto

lib/streamlit/ReportSession.py

frontend/src/lib/WidgetStateManager.ts

examples/file_uploader.py

tvst · 2019-10-25T06:02:22Z

Found a bug: the file uploader gets into a "loading forever" state when you pass the wrong kind of file.

Steps to repro:

Run examples/file_uploader.py
In the CSV file uploader, pass in an MP3 file
Expected: the file uploads, but the script throws an exception that appears in the app when it tries to read the MP3 as a CSV
Actual: the file uploader never finishes uploading the file

dcaminos · 2019-10-25T19:52:20Z

@tconkling I fixed all your comments and added a test for FileManager (the most important one). ReportSession and Cypress tests are missing yet.

@tvst I fixed that bug (I check the file type before start the upload) and added a method to cancel the upload.

examples/file_uploader.py

frontend/src/App.tsx

proto/streamlit/proto/BackMsg.proto

proto/streamlit/proto/Element.proto

dcaminos · 2019-11-21T14:59:49Z

@ajinkya933 that source code is old, you should not use file_reader any more (that was my first approach).

Try to run this:

import streamlit as st

the_file = st.file_uploader("Upload a file", type=["pdf"])

if the_file is None:
    'Result:', None
else:
    'Result:', len(the_file)

ajinkya933 · 2019-11-22T06:43:48Z

@ajinkya933 that source code is old, you should not use file_reader any more (that was my first approach).

Try to run this:
import streamlit as st

the_file = st.file_uploader("Upload a file", type=["pdf"])

if the_file is None:
    'Result:', None
else:
    'Result:', len(the_file)

Have you installed streamlit from here: https://github.com/streamlit/streamlit/wiki/Contributing#Ubuntu.

I am having trouble installing develop version of streamlit as pointed out by @tconkling initially I run

ajinkya@ajinkya-H310M-S2:~/Music/streamlit/lib$ pip3 install -e . --user
Obtaining file:///home/ajinkya/Music/streamlit/lib

for which I get a message Successfully installed streamlit

But when I do run $ streamlit --version i get

$ streamlit --version
Traceback (most recent call last):
  File "/home/ajinkya/.local/bin/streamlit", line 11, in <module>
    load_entry_point('streamlit', 'console_scripts', 'streamlit')()
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2852, in load_entry_point
    return ep.load()
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/ajinkya/Music/streamlit/lib/streamlit/__init__.py", line 96, in <module>
    from streamlit.DeltaGenerator import DeltaGenerator as _DeltaGenerator
  File "/home/ajinkya/Music/streamlit/lib/streamlit/DeltaGenerator.py", line 35, in <module>
    from streamlit.proto import Balloons_pb2
ImportError: cannot import name 'Balloons_pb2'

After getting this error I followed instructions here (https://github.com/streamlit/streamlit/wiki/Contributing#Ubuntu) and heres my output error
here is the detailed error log (#728)

tvst · 2019-11-26T23:09:32Z

frontend/src/components/widgets/FileUploader/FileUploader.scss

+  outline: 0;
+}
+
+.stFileUploaderError {


We only add st to the top-level classname in each component.

Also, wrap the whole CSS in .stFileUploader { and then you can just rename this to .uploadError or something. For example:

.stFileUploader { ... .uploadError { ... } ... }

...same with everything else in this file.

That's what we've been doing in the CSS for other elements.

tvst · 2019-11-26T23:13:56Z

frontend/src/components/widgets/FileUploader/FileUploader.scss

+  padding-top: 1rem;
+  outline: 0px;
+  -webkit-box-orient: vertical;
+  -webkit-box-direction: normal;


Why are all these -webkit properties needed? It's best to avoid browser-specific code.

tvst · 2019-11-26T23:15:05Z

frontend/src/lib/WidgetStateManager.ts

+
+    // The default file size allowed by server config is 50MB.
+    // The best message size to don't freeze the server is 10MB
+    const chunkByteLimit = 10 * 1e6 //10MB


What if the user sets the limit to 5MB? Does this break anything?

No, we have a validation to check if the widget accept the file first

tvst · 2019-11-26T23:15:47Z

lib/streamlit/DeltaGenerator.py

+        Examples
+        --------
+        >>> file = st.file_uploader("Upload a image", type=["png"])
+        >>> if file != None:


is not None

tvst · 2019-11-26T23:15:59Z

lib/streamlit/DeltaGenerator.py

+
+        Examples
+        --------
+        >>> file = st.file_uploader("Upload a image", type=["png"])


type="png"

tvst · 2019-11-26T23:16:30Z

lib/streamlit/DeltaGenerator.py

+        ----------
+        label : str or None
+            A short label explaining to the user what this file uploader is for.
+        type : list of string or None


type : str or list of str or None

tvst · 2019-11-26T23:20:27Z

lib/streamlit/folder_black_list.py

@@ -32,6 +33,14 @@
 ]


+# Add the Streamlit lib folder when in dev mode, since otherwise we end up with
+# weird situations where the ID of a class in one run is the same as in another


Ooops, this should be "is not the same" (I forgot the "not")

universewill · 2019-11-30T07:25:58Z

@ajinkya933 Have you installed the file-uploader branch into your Python environment? Simply running demo.py won't work; assuming you had previously run pip install streamlit, then when you import streamlit as st in a source file, you're getting the installed release version of streamlit.
You should probably do something like:
$ pip uninstall streamlit
$ pip install -e <path/to/file-uploader/branch/checkout>
The pip documentation contains more details about local project installs: https://pip.pypa.io/en/stable/reference/pip_install/#local-project-installs
@tconkling I have followed your steps. And have noticed that I have to install streamlit from issue120/file-uploader branch. However there are series of errors I get when I install streamlit develop branch errors:(#728)

I hope someone will help me rectify them

i try to run ' pip install -e path/to/code' command but it tells me no setup.py found.
How to install?

universewill · 2019-11-30T08:00:38Z

@ajinkya933 Have you installed the file-uploader branch into your Python environment? Simply running demo.py won't work; assuming you had previously run pip install streamlit, then when you import streamlit as st in a source file, you're getting the installed release version of streamlit.
You should probably do something like:
$ pip uninstall streamlit
$ pip install -e <path/to/file-uploader/branch/checkout>
The pip documentation contains more details about local project installs: https://pip.pypa.io/en/stable/reference/pip_install/#local-project-installs
@tconkling I have followed your steps. And have noticed that I have to install streamlit from issue120/file-uploader branch. However there are series of errors I get when I install streamlit develop branch errors:(#728)
I hope someone will help me rectify them
i try to run ' pip install -e path/to/code' command but it tells me no setup.py found.
How to install?

solved by 'cd streamlit/lib' first

universewill · 2019-11-30T08:52:29Z

@ajinkya933 that source code is old, you should not use file_reader any more (that was my first approach).
Try to run this:
import streamlit as st

the_file = st.file_uploader("Upload a file", type=["pdf"])

if the_file is None:
    'Result:', None
else:
    'Result:', len(the_file)

Have you installed streamlit from here: https://github.com/streamlit/streamlit/wiki/Contributing#Ubuntu.

I am having trouble installing develop version of streamlit as pointed out by @tconkling initially I run

ajinkya@ajinkya-H310M-S2:~/Music/streamlit/lib$ pip3 install -e . --user
Obtaining file:///home/ajinkya/Music/streamlit/lib

for which I get a message Successfully installed streamlit

But when I do run $ streamlit --version i get

$ streamlit --version
Traceback (most recent call last):
  File "/home/ajinkya/.local/bin/streamlit", line 11, in <module>
    load_entry_point('streamlit', 'console_scripts', 'streamlit')()
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2852, in load_entry_point
    return ep.load()
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/home/ajinkya/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/ajinkya/Music/streamlit/lib/streamlit/__init__.py", line 96, in <module>
    from streamlit.DeltaGenerator import DeltaGenerator as _DeltaGenerator
  File "/home/ajinkya/Music/streamlit/lib/streamlit/DeltaGenerator.py", line 35, in <module>
    from streamlit.proto import Balloons_pb2
ImportError: cannot import name 'Balloons_pb2'

After getting this error I followed instructions here (https://github.com/streamlit/streamlit/wiki/Contributing#Ubuntu) and heres my output error
here is the detailed error log (#728)

get same error

Stamenov · 2019-12-02T13:15:05Z

Hey,
I am looking forward to this feature as a very nice addition to the streamlit widgets.
What is there left to be done? Perhaps I can help.
Best.

ajinkya933 · 2019-12-03T04:04:42Z

Hey,
I am looking forward to this feature as a very nice addition to the streamlit widgets.
What is there left to be done? Perhaps I can help.
Best.

Perhaps you can help me in resolving this issue #728 File uploader feature is available in develop branch but I am having a tough time installing it

jrhone · 2019-12-06T20:44:37Z

lib/streamlit/caching.py

@@ -53,6 +54,18 @@
 to suppress the warning.
 """

+# based on: https://stackoverflow.com/questions/43506378/how-to-get-source-code-of-function-that-is-wrapped-by-a-decorator
+# In Python 2, the @functools.wraps() decorator does not set the convenience __wrapped__
+if sys.version_info[0:2] >= (3, 4):  # Python v3.4+?


Looks like it might be 3.2

New in version 3.2: Automatic addition of the wrapped attribute.

To allow access to the original function for introspection and other purposes (e.g. bypassing a caching decorator such as lru_cache()), this function automatically adds a wrapped attribute to the wrapper that refers to the original function.

Changed in version 3.4: The wrapped attribute now always refers to the wrapped function, even if that function defined a wrapped attribute. (see issue 17482)

Oh I just read the comment on stackoverflow, so I see why you chose 3.4

jrhone · 2019-12-06T20:45:18Z

lib/streamlit/caching.py

+if sys.version_info[0:2] >= (3, 4):  # Python v3.4+?
+    wraps = functools.wraps  # built-in has __wrapped__ attribute
+else:
+    def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS,


Should we add some tests for python 2?

Well, if this doesn't work, a caching test will fail.

All problems have been solved

dancebean · 2019-12-12T23:58:19Z

Is there a way to install the latest version from this branch of repo directly, since this PR has been merged? Since it has not been included in the latest released version, pip install won't get the most updated version

vanpelt · 2019-12-14T21:36:26Z

@dancebean pip install --upgrade git+git://github.com/streamlit/streamlit.git@master#egg=streamlit

RichardOberdieck · 2019-12-16T08:33:32Z

@dancebean pip install --upgrade git+git://github.com/streamlit/streamlit.git@master#egg=streamlit

This breaks when using conda and windows. Just saw this issue with Windows and the develop branch. Is there any indication when this will be released?

treuille · 2020-01-01T01:00:46Z

@dancebean, @vanpelt (👋), and @RichardOberdieck : Please note that file_uploader has been released in 0.52 in experimental form.

Also, you can find instructions to build Streamlit from source here.

Happy New Year! 🎆

universewill · 2020-01-02T11:57:31Z

@dancebean, @vanpelt (👋), and @RichardOberdieck : Please note that file_uploader has been released in 0.52 in experimental form.

Also, you can find instructions to build Streamlit from source here.

Happy New Year! 🎆

Great work! Besides, how to upload an image and write to local as an image file?

universewill · 2020-01-02T12:33:21Z

@dancebean, @vanpelt (👋), and @RichardOberdieck : Please note that file_uploader has been released in 0.52 in experimental form.
Also, you can find instructions to build Streamlit from source here.
Happy New Year! 🎆

Great work! Besides, how to upload an image and write to local as an image file?

This can do:

uploaded_file = st.file_uploader("Choose a image file", type=["png", 'jpg', 'jpeg', 'tiff'])
if uploaded_file is not None:
    im = Image.open(uploaded_file)
    im.save('test.jpg', quality=95)

treuille · 2020-01-03T18:07:17Z

Great, @universewill . Please also know that file_uploader has an experimental API and may change!

fathimanourin · 2021-06-07T04:42:03Z

I get the following error when i run this

import streamlit as st

file_png = st.file_uploader("Upload a PNG image", type=([".png"]))

if file_png:
    file_png_bytes = st.file_reader(file_png)
    st.image(file_png_bytes)

AttributeError:`` module 'streamlit' has no attribute 'file_reader'
Traceback:
File "/home/fathima/.local/lib/python3.8/site-packages/streamlit/script_runner.py", line 338, in _run_script
exec(code, module.dict)
File "/home/fathima/Documents/Internship/frontend.py", line 6, in
file_png_bytes = st.file_reader(file_png)

Daniel Caminos added 7 commits October 21, 2019 16:00

add new FileUploader widget

2c84231

add new FileUploader widget

247ed6d

add new FileUploader widget

6caacc1

save chunks solution

da46d7a

save chunks solution

eb8f340

new server.maxUploadSize config

0b4c8eb

solve lint problems

4f69872

dcaminos requested review from a team and removed request for a team October 21, 2019 19:14

dcaminos added the WIP label Oct 21, 2019

add documentation

62345ad

dcaminos requested a review from a team as a code owner October 22, 2019 14:10

fix and add tests

6a20616

dcaminos removed the WIP label Oct 22, 2019

Daniel Caminos and others added 2 commits October 22, 2019 17:07

Merge branch 'develop' into issue120/file-uploader

f0fc5bb

add missing type

48f11e1

robmarkcole mentioned this pull request Oct 23, 2019

It will be very convenient to have an option for file select. #508

Closed

tconkling previously requested changes Oct 23, 2019

View reviewed changes

improve code

ee7dca5

tvst reviewed Oct 25, 2019

View reviewed changes

examples/file_uploader.py Outdated Show resolved Hide resolved

Daniel Caminos and others added 3 commits October 25, 2019 16:22

add FileManager_test.py and fix some code problems

1445872

Merge branch 'develop' into issue120/file-uploader

a284eae

fix lint problem

0ac4202

dcaminos requested review from tconkling and tvst October 25, 2019 19:52

Daniel Caminos added 2 commits October 28, 2019 12:04

add new error message format in file_uploader widget

5283546

fix lint problem

e0c31c0

tvst reviewed Oct 29, 2019

View reviewed changes

improve proto comments

5fac828

dcaminos requested a review from tvst November 25, 2019 20:51

Add ./lib/streamlit to blacklist when in dev mode

14c005b

tvst approved these changes Nov 26, 2019

View reviewed changes

Daniel Caminos added 3 commits November 27, 2019 15:25

minor fixes

2959ee0

Merge branch 'develop' into issue120/file-uploader

ef01b9d

force to parse config for tests

d0546a5

Daniel Caminos added 3 commits December 4, 2019 14:07

Merge branch 'develop' into issue120/file-uploader

cd5f59d

add code to set __wrapped__ in python 2

7ea7aaf

add comments

3053061

jrhone reviewed Dec 6, 2019

View reviewed changes

merge develop

ac11e4f

dcaminos merged commit 01caeec into streamlit:develop Dec 9, 2019

dcaminos deleted the issue120/file-uploader branch December 9, 2019 22:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a File Uploader Widget #488

Add a File Uploader Widget #488

dcaminos commented Oct 21, 2019 •

edited

tconkling left a comment •

edited

tvst commented Oct 25, 2019

dcaminos commented Oct 25, 2019 •

edited

dcaminos commented Nov 21, 2019 •

edited

ajinkya933 commented Nov 22, 2019

tvst Nov 26, 2019

tvst Nov 26, 2019

tvst Nov 26, 2019

dcaminos Nov 27, 2019

tvst Nov 26, 2019

tvst Nov 26, 2019

tvst Nov 26, 2019

tvst Nov 26, 2019

universewill commented Nov 30, 2019

universewill commented Nov 30, 2019

universewill commented Nov 30, 2019

Stamenov commented Dec 2, 2019

ajinkya933 commented Dec 3, 2019

jrhone Dec 6, 2019

jrhone Dec 6, 2019

jrhone Dec 6, 2019

dcaminos Dec 6, 2019

dancebean commented Dec 12, 2019

vanpelt commented Dec 14, 2019

RichardOberdieck commented Dec 16, 2019 •

edited

treuille commented Jan 1, 2020 •

edited

universewill commented Jan 2, 2020

universewill commented Jan 2, 2020 •

edited

treuille commented Jan 3, 2020

fathimanourin commented Jun 7, 2021

Add a File Uploader Widget #488

Add a File Uploader Widget #488

Conversation

dcaminos commented Oct 21, 2019 • edited

tconkling left a comment • edited

Choose a reason for hiding this comment

tvst commented Oct 25, 2019

dcaminos commented Oct 25, 2019 • edited

dcaminos commented Nov 21, 2019 • edited

ajinkya933 commented Nov 22, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

universewill commented Nov 30, 2019

universewill commented Nov 30, 2019

universewill commented Nov 30, 2019

Stamenov commented Dec 2, 2019

ajinkya933 commented Dec 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dancebean commented Dec 12, 2019

vanpelt commented Dec 14, 2019

RichardOberdieck commented Dec 16, 2019 • edited

treuille commented Jan 1, 2020 • edited

universewill commented Jan 2, 2020

universewill commented Jan 2, 2020 • edited

treuille commented Jan 3, 2020

fathimanourin commented Jun 7, 2021

dcaminos commented Oct 21, 2019 •

edited

tconkling left a comment •

edited

dcaminos commented Oct 25, 2019 •

edited

dcaminos commented Nov 21, 2019 •

edited

RichardOberdieck commented Dec 16, 2019 •

edited

treuille commented Jan 1, 2020 •

edited

universewill commented Jan 2, 2020 •

edited