Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bokeh server: color palette rgb is not recognized #4096

Closed
mdk73 opened this issue Apr 3, 2016 · 20 comments
Closed

bokeh server: color palette rgb is not recognized #4096

mdk73 opened this issue Apr 3, 2016 · 20 comments

Comments

@mdk73
Copy link

@mdk73 mdk73 commented Apr 3, 2016

The following code plots a few lines with changing color (rgb):

from pandas import DataFrame
from bokeh.io import show, output_file, output_server
from bokeh.charts import Line, color
import subprocess
import time

if __name__ == '__main__':   
    output_file('rgb-platte.html')

    df = DataFrame({'1': [0,1],
                    '2': [1,2],
                    '3': [2,3],
                    '4': [3,4],
                    '5': [4,5]})

    # set color palette rgb
    palette_r = [50, 100, 150, 200, 250]     # changing the values for 'red'.
    palette_g = 5 * [0]    # keeping values for 'green' constant. 
    palette_b = 5 * [0]    # keeping values for 'blue' constant. 
    palette = list(zip(palette_r, palette_g, palette_b))

    # define the plot
    p = Line(df, title="palette rgb", plot_width = 300, plot_height = 200, color=palette)

    show(p)

Result:
image

Now I replace output_file('rgb-platte.html') with:

    # start a bokeh server
    args = ['python', '-m', 'bokeh', 'serve']
    server = subprocess.Popen(args) 
    time.sleep(1) # wait for the server to initialize
    output_server('test')

to use the bokeh server.

The resulting image has no colors:
image

@bryevdv bryevdv added type: bug and removed type: discussion labels Apr 9, 2016
@bryevdv bryevdv added this to the short-term milestone Apr 9, 2016
@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

This is clearly a bug. @mdk73 is there any JS console output or server output?

@mdk73
Copy link
Author

@mdk73 mdk73 commented Apr 9, 2016

This is the outout I get on the console:

DEBUG:bokeh.server.tornado:Allowed Host headers: ['localhost:5006']
DEBUG:bokeh.server.tornado:These host origins can connect to the websocket: ['localhost:5006']
DEBUG:bokeh.server.tornado:Patterns are: [('/?', <class 'bokeh.server.views.doc_handler.DocHandler'>, {'bokeh_
websocket_path': '/ws', 'application_context': <bokeh.server.application_context.ApplicationContext object at 
0x0000000006E62908>}), ('/ws', <class 'bokeh.server.views.ws.WSHandler'>, {'bokeh_websocket_path': '/ws', 'app
lication_context': <bokeh.server.application_context.ApplicationContext object at 0x0000000006E62908>}), ('/au
toload.js', <class 'bokeh.server.views.autoload_js_handler.AutoloadJsHandler'>, {'bokeh_websocket_path': '/ws'
, 'application_context': <bokeh.server.application_context.ApplicationContext object at 0x0000000006E62908>}),
 ('/static/(.*)', <class 'bokeh.server.views.static_handler.StaticHandler'>)]
INFO:bokeh.command.subcommands.serve:Starting Bokeh server on port 5006 with applications at paths ['/']
INFO:bokeh.server.views.ws:WebSocket connection opened
DEBUG:bokeh.server.views.ws:Receiver created for Protocol('1.0')
DEBUG:bokeh.server.views.ws:ServerHandler created for Protocol('1.0')
INFO:bokeh.server.views.ws:ServerConnection created
DEBUG:bokeh.server.session:pushing doc to session 'test'
INFO:bokeh.server.views.ws:WebSocket connection closed: code=1000, reason='closed'
INFO:bokeh.client.connection:Connection closed by server
INFO:tornado.access:200 GET /?bokeh-session-id=test (::1) 12.52ms
INFO:bokeh.server.views.ws:WebSocket connection opened
DEBUG:bokeh.server.views.ws:Receiver created for Protocol('1.0')
DEBUG:bokeh.server.views.ws:ServerHandler created for Protocol('1.0')
INFO:bokeh.server.views.ws:ServerConnection created
DEBUG:bokeh.server.session:Sending pull-doc-reply from session 'test'
DEBUG:bokeh.server.tornado:[pid 8704] 1 clients connected
DEBUG:bokeh.server.tornado:[pid 8704]   / has 1 sessions with 0 unused
@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

What about the browser JavaScript console?

@mdk73
Copy link
Author

@mdk73 mdk73 commented Apr 9, 2016

I am not familiar with the browser development tools, I hope I found the
correct console:

bokeh.min.js?v=50ffae6…:1 Bokeh: setting prefix to /
bokeh.min.js?v=50ffae6…:2 Bokeh: setting log level to: 'info'
bokeh.min.js?v=50ffae6…:62 Bokeh: Websocket connection 0 is now open
bokeh.min.js?v=50ffae6…:2 Bokeh items were rendered successfully

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

That is the correct console, if that's all the output there is, this will require more investigation. The first step would be to see whether RGB tuples work in other contexts (i.e. with bokeh.plotting or bokeh.models code) That might inform as to whether this is an issue just in bokeh.charts somehow, or whether it is more general. If you have the time to run some experiments, that is certainly appreciated, but understand completely if you don't

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

Actually, quicker/easier thing. Make a server app (i.e. don't usurp bokeh serve foo.py directly, don't use output_server from a separate python process) and run bokeh json on it, that will spit out the serialized JSON which could be inspected for problems.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

OK, this version, run as a server app, works on current master:

# saved as foo.py 

from pandas import DataFrame
from bokeh.charts import Line
from bokeh.io import curdoc

df = DataFrame({'1': [0,1],
                '2': [1,2],
                '3': [2,3],
                '4': [3,4],
                '5': [4,5]})

# set color palette rgb
palette_r = [50, 100, 150, 200, 250]     # changing the values for 'red'.
palette_g = 5 * [0]    # keeping values for 'green' constant.
palette_b = 5 * [0]    # keeping values for 'blue' constant.
palette = list(zip(palette_r, palette_g, palette_b))

# define the plot
plot = Line(df, title="palette rgb", plot_width = 300, plot_height = 200, color=palette)

curdoc().add_root(plot)

Can you try running bokeh serve on that script to see if works with the version you have installed?


screen shot 2016-04-09 at 10 46 58 am

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

When I run this version of the output_server version:

from pandas import DataFrame
from bokeh.io import show, output_server
from bokeh.charts import Line

if __name__ == '__main__':

    df = DataFrame({'1': [0,1],
                    '2': [1,2],
                    '3': [2,3],
                    '4': [3,4],
                    '5': [4,5]})

    # set color palette rgb
    palette_r = [50, 100, 150, 200, 250]     # changing the values for 'red'.
    palette_g = 5 * [0]    # keeping values for 'green' constant.
    palette_b = 5 * [0]    # keeping values for 'blue' constant.
    palette = list(zip(palette_r, palette_g, palette_b))

    # define the plot
    p = Line(df, title="palette rgb", plot_width = 300, plot_height = 200, color=palette)

    output_server('test')

    show(p)

I get this console output:

screen shot 2016-04-09 at 10 52 52 am

So my best guess is there is something weird going on with output_server and RGB tuples.

My best suggestion for a workaround, and actually my general advice in any case is to use actual bokeh serve style Bokeh apps. The output_server mode has extremely limited capability or usefulness with the new server.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

The problem is definitely that the string "rgb(50, 0, 0)" is being encoded as a field for a column name, rather than a value. This seems to be happening on the python side. But I'm not sure why it would work in a server app, but not output_server context, that's actually troublesome. My best current guess is that serializing the tuple from python works (why a server app works) but that deserializing it in python (ie.e. when the server receives it from a separate script) does not.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

I should add, another workaround if you don't to make a server app is to construct RGBA hex values, e.g. "#aabbcc", I just ran an output_server script that uses this format and it worked as expected.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 9, 2016

OK, I definitely think my above hypothesis was correct. Look at:

https://github.com/bokeh/bokeh/blob/master/bokeh/core/properties.py#L1813-L1837

Color tuples get serialized to strings like "rgb(50, 0, 0)", and if these are sent directly from a server app to the browser this is fine, because the browser knows that to do with that. But in output_server cases there is an extra hop: from the python script, then to the server, then to the browser. Well the first hop turns the tuple into a string, but look at this specifically:

https://github.com/bokeh/bokeh/blob/master/bokeh/core/properties.py#L1832-L1834

In the server it is now a string, which encodes as a field.

There are a few options, any of which you could try to make a PR for, which would be a huge help.

I think the second is safer and easier. Would you be interested in working up a small PR @mdk73?

@mdk73
Copy link
Author

@mdk73 mdk73 commented Apr 10, 2016

I tried to run the script with 'foo.py' with bokeh serve, and I ran once more in the problems I had when reporting #4092 , I updated this issue with a more complete description of steps to reproduce it.

@mdk73
Copy link
Author

@mdk73 mdk73 commented Apr 10, 2016

I tried to replace the rgb values in the original code with hex values, but I get an error log.

The following lines were added to use palette with hex-values (after defining the rgb-palette, before defining the plot:

    # converting into hex colors
    _palette = []
    for item in palette:
        red = item[0]
        green = item[1]
        blue = item[2]
        alpha = item[3]

        hr = format(red, '02x')
        hg = format(green, '02x')
        hb = format(blue, '02x')
        ha = format(int(alpha *255), '02x')
        _palette_ = '#{}{}{}{}'.format(hr, hg, hb, ha) # rgba
        _palette.append(_palette_)
    palette = _palette  
    print(palette) 

And the error log:

['#320000ff', '#640000ff', '#960000ff', '#c80000ff', '#fa0000ff']
Traceback (most recent call last):
  File "C:\Users\mdk\workspace\DemAn\src\qt\bokeh_palette.py", line 48, in <module>
    color=palette)
  File "C:\Python34\lib\site-packages\bokeh\charts\builders\line_builder.py", line 88, in Line
    return create_and_build(LineBuilder, data, **kws)
  File "C:\Python34\lib\site-packages\bokeh\charts\builder.py", line 67, in create_and_build
    chart.add_builder(builder)
  File "C:\Python34\lib\site-packages\bokeh\charts\chart.py", line 149, in add_builder
    builder.create(self)
  File "C:\Python34\lib\site-packages\bokeh\charts\builder.py", line 518, in create
    chart.add_renderers(self, renderers)
  File "C:\Python34\lib\site-packages\bokeh\charts\chart.py", line 144, in add_renderers
    self.renderers += renderers
  File "C:\Python34\lib\site-packages\bokeh\core\property_containers.py", line 18, in wrapper
    result = func(*args, **kwargs)
  File "C:\Python34\lib\site-packages\bokeh\core\property_containers.py", line 77, in __iadd__
    return super(PropertyValueList, self).__iadd__(y)
  File "C:\Python34\lib\site-packages\bokeh\charts\builders\line_builder.py", line 232, in yield_renderers
    **group_kwargs)
  File "C:\Python34\lib\site-packages\bokeh\charts\glyphs.py", line 169, in __init__
    super(LineGlyph, self).__init__(**kwargs)
  File "C:\Python34\lib\site-packages\bokeh\charts\models.py", line 80, in __init__
    super(CompositeGlyph, self).__init__(**properties)
  File "C:\Python34\lib\site-packages\bokeh\core\properties.py", line 701, in __init__
    setattr(self, name, value)
  File "C:\Python34\lib\site-packages\bokeh\core\properties.py", line 714, in __setattr__
    super(HasProps, self).__setattr__(name, value)
  File "C:\Python34\lib\site-packages\bokeh\core\properties.py", line 453, in __set__
    value = self.descriptor.prepare_value(obj.__class__, self.name, value)
  File "C:\Python34\lib\site-packages\bokeh\core\properties.py", line 272, in prepare_value
    raise e
  File "C:\Python34\lib\site-packages\bokeh\core\properties.py", line 265, in prepare_value
    self.validate(value)
  File "C:\Python34\lib\site-packages\bokeh\core\properties.py", line 1298, in validate
    raise ValueError("expected an element of either %s, got %r" % (nice_join(self.type_params), value))
ValueError: expected an element of either Enum('aliceblue', 'antiquewhite', 'aqua', 'aquamarine', 'azure', 'be
ige', 'bisque', 'black', 'blanchedalmond', 'blue', 'blueviolet', 'brown', 'burlywood', 'cadetblue', 'chartreus
e', 'chocolate', 'coral', 'cornflowerblue', 'cornsilk', 'crimson', 'cyan', 'darkblue', 'darkcyan', 'darkgolden
rod', 'darkgray', 'darkgreen', 'darkgrey', 'darkkhaki', 'darkmagenta', 'darkolivegreen', 'darkorange', 'darkor
chid', 'darkred', 'darksalmon', 'darkseagreen', 'darkslateblue', 'darkslategray', 'darkslategrey', 'darkturquo
ise', 'darkviolet', 'deeppink', 'deepskyblue', 'dimgray', 'dimgrey', 'dodgerblue', 'firebrick', 'floralwhite',
 'forestgreen', 'fuchsia', 'gainsboro', 'ghostwhite', 'gold', 'goldenrod', 'gray', 'green', 'greenyellow', 'gr
ey', 'honeydew', 'hotpink', 'indianred', 'indigo', 'ivory', 'khaki', 'lavender', 'lavenderblush', 'lawngreen',
 'lemonchiffon', 'lightblue', 'lightcoral', 'lightcyan', 'lightgoldenrodyellow', 'lightgray', 'lightgreen', 'l
ightgrey', 'lightpink', 'lightsalmon', 'lightseagreen', 'lightskyblue', 'lightslategray', 'lightslategrey', 'l
ightsteelblue', 'lightyellow', 'lime', 'limegreen', 'linen', 'magenta', 'maroon', 'mediumaquamarine', 'mediumb
lue', 'mediumorchid', 'mediumpurple', 'mediumseagreen', 'mediumslateblue', 'mediumspringgreen', 'mediumturquoi
se', 'mediumvioletred', 'midnightblue', 'mintcream', 'mistyrose', 'moccasin', 'navajowhite', 'navy', 'oldlace'
, 'olive', 'olivedrab', 'orange', 'orangered', 'orchid', 'palegoldenrod', 'palegreen', 'paleturquoise', 'palev
ioletred', 'papayawhip', 'peachpuff', 'peru', 'pink', 'plum', 'powderblue', 'purple', 'red', 'rosybrown', 'roy
alblue', 'saddlebrown', 'salmon', 'sandybrown', 'seagreen', 'seashell', 'sienna', 'silver', 'skyblue', 'slateb
lue', 'slategray', 'slategrey', 'snow', 'springgreen', 'steelblue', 'tan', 'teal', 'thistle', 'tomato', 'turqu
oise', 'violet', 'wheat', 'white', 'whitesmoke', 'yellow', 'yellowgreen'), Regex('^#[0-9a-fA-F]{6}$'), Tuple(B
yte(Int, 0, 255), Byte(Int, 0, 255), Byte(Int, 0, 255)) or Tuple(Byte(Int, 0, 255), Byte(Int, 0, 255), Byte(In
t, 0, 255), Percent), got '#320000ff'
DEBUG:bokeh.server.tornado:Allowed Host headers: ['localhost:5006']
DEBUG:bokeh.server.tornado:These host origins can connect to the websocket: ['localhost:5006']
DEBUG:bokeh.server.tornado:Patterns are: [('/?', <class 'bokeh.server.views.doc_handler.DocHandler'>, {'bokeh_
websocket_path': '/ws', 'application_context': <bokeh.server.application_context.ApplicationContext object at 
0x0000000006E60C88>}), ('/ws', <class 'bokeh.server.views.ws.WSHandler'>, {'bokeh_websocket_path': '/ws', 'app
lication_context': <bokeh.server.application_context.ApplicationContext object at 0x0000000006E60C88>}), ('/au
toload.js', <class 'bokeh.server.views.autoload_js_handler.AutoloadJsHandler'>, {'bokeh_websocket_path': '/ws'
, 'application_context': <bokeh.server.application_context.ApplicationContext object at 0x0000000006E60C88>}),
 ('/static/(.*)', <class 'bokeh.server.views.static_handler.StaticHandler'>)]
INFO:bokeh.command.subcommands.serve:Starting Bokeh server on port 5006 with applications at paths ['/']
DEBUG:bokeh.server.tornado:[pid 10380] 0 clients connected
DEBUG:bokeh.server.tornado:[pid 10380]   / has 0 sessions with 0 unused

@mattpap
Copy link
Contributor

@mattpap mattpap commented Apr 10, 2016

to return RGBA hex strings like "#aabb09aa"
I tried to replace the rgb values in the original code with hex values

Hex color syntax doesn't support alpha component (we follow CSS3 here).

@canavandl
Copy link
Contributor

@canavandl canavandl commented Apr 11, 2016

We've added a warning to the docs about this issue at #3904 and proposed a few work-arounds:

.. warning::
    Supplying lists of RGB or RGBA color tuples as color arguments (either
    directly or as a DataSource column reference) doesn't work. You may read a
    discussion of the issue on our project `GitHub`_ page. Suggested
    work-arounds include using lists of:

    * RGB hexadecimal values
    * `bokeh.colors.RGB` objects (i.e. ``[RGB(255, 0, 0), RGB(0, 255, 0)"]``)
    * CSS-format RGB/RGBA strings (i.e. ``["rgb(255, 0, 0)", "rgb(0, 255, 0)"]``)

.. _GitHub: https://github.com/bokeh/bokeh/issues/2622
@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 11, 2016

I am like 99% sure the RGBA hex values did used to work... I thought there was actually a drawn-out issue and PR to make them work.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Apr 11, 2016

Also, I specifically think strings starting with "rgb" are not recognized as values which is the crux of the problem.

@canavandl
Copy link
Contributor

@canavandl canavandl commented Apr 11, 2016

Also, I specifically think strings starting with "rgb" are not recognized as values which is the crux of the problem.

I think it does if you enclose it in a list (similar to how we handle text values). Passing an "rgb..." string directly won't work.

I also removed mention of RGBA hex support in #3904, because it isn't supported in CSS3.

@bryevdv bryevdv modified the milestones: 0.12.7, short-term May 9, 2017
@bryevdv bryevdv modified the milestones: 0.12.7, 0.12.8 Aug 21, 2017
@philippjfr
Copy link
Contributor

@philippjfr philippjfr commented Jun 27, 2018

@bryevdv A number of users have been reporting this because we set the default NaN color to (0, 0, 0, 0) which becomes 'rgb(0, 0, 0, 0)'. I'd be happy to work on a fix here, but I can't quite tell what conclusions to draw from the discussion above. Would it be sufficient to allow rgb strings on the property or is that likely to cause issues elsewhere?

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Jun 27, 2018

@philippjfr There's several intersecting things going on:

  • Narrowly, this issue concerns a problem with bokeh.client and as such, from that perspective, I don't really care about it/it's not important, however

  • incidentally, the problem manifested because strings like "rgb(10, 20, 30, 40)" are not recognized as color values. Offhand, I think making those strings work as color values seems reasonable and useful, and I would support a PR (with tests) that does that

  • doing that should also fix the original issue here, but again, don't really care about that specifically since using bokeh.client like this here is something I'd like to discourage in any case.

  • tangentially, I think there are problems (again?) with handling actual tuples as colors, and it would be nice to fix that too

TLDR; A PR to make CSS "rgb(...)" strings work is welcome. Looking into/fixing literal tuple handling is also welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

6 participants
You can’t perform that action at this time.