Float dtypes no longer work on some hardware (without float textures) as of 0.4.13

[alonyan]

🐛 Bug

Hi All, Just upgraded to 0.4.13 and got some unpleasant bug. If I try to add an image layer and the underlying data is not an integer I get a black napari and the attached traceback. Same code works well on previous versions.

To Reproduce

Steps to reproduce the behavior:
img is a numpy array

1. viewer.add_image(img.astype('float'))

Expected behavior

Show the image

Environment

napari: 0.4.13
Platform: Linux-3.10.0-1160.45.1.el7.x86_64-x86_64-with-glibc2.17
Python: 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:39:48) [GCC 9.3.0]
Qt: 5.12.9
PyQt5: 5.12.3
NumPy: 1.20.3
SciPy: 1.7.1
Dask: 2021.08.1
VisPy: 0.9.4

OpenGL:

• GL version: 2.1 Mesa 18.3.4
• MAX_TEXTURE_SIZE: 8192

Screens:

• screen 1: resolution 1440x847, scale 1.0

Plugins:

• animation: 0.0.2
• cellpose-napari: 0.1.3
• console: 0.0.4
• napari-nikon-nd2: 0.1.3
• scikit-image: 0.4.13
• svg: 0.1.5

[tlambert03]

Tried to copy your env as closely as possible (on a mac though, which is a big caveat), and couldn't reproduce.

Can you perhaps give us a bit more info on your img object? Shape & dtype?
If you haven't already, could you do us a favor and check in a fresh conda environment?

[tlambert03]

one thing I'm wondering: I believe this is the first release using float textures directly on the GPU (instead of downscaling first to 8 bit). So, depending on the size of your volume, it's conceivable that it's exceeding your GPU vram whereas 0.4.12 wasn't. @brisvag, do you know how that error is going to manifest itself? (I don't think it would look like this, but not sure).

[alonyan]

Hi Talley, Thanks for looking into this.

Made a fresh environment with nothing but napari.

img = np.ones((3,3), dtype=float)

still getting the same behavior and traceback.

[tlambert03]

this is on the cluster I assume right?

[alonyan]

Not really. It's a server that's running CentOS but isn't a part of a bigger cluster.

[tlambert03]

gotcha. I guess my suspicion then is perhaps on GL version: 2.1 Mesa 18.3.4 ...
@djhoese, do you have any thoughts? summary: 0.4.13 is the first release to include all the changes to support float textures in vispy. and we do use texture_format='auto' by default (still need a way to expose that to the napari user). @alonyan is now seeing a GL_INVALID_VALUE error when drawing the canvas with a small float array. None of our devs on linux have seen that, so i'm wondering if this has something to do with Mesa (18.3.4) ... but that's a bit out of my depths. Any hints on what we should be checking?

[djhoese]

Yeah that GL version is not promising. Check:

import vispy
print("texture_float"` in vispy.sys_info())

See https://stackoverflow.com/a/1937600/433202

[tlambert03]

thanks! as a bare minimum, it seems like we'll definitely want to expose that texture_format kwarg somehow

[tlambert03]

@djhoese, one more q: do you think it would make sense, either in napari or in vispy, to have texture_format='auto' fall back to the old 8bit method if "texture_float" is not in the GL extensions?

[tlambert03]

in direct communication, can confirm that @alonyan's config doesn't support float textures, here's the full output of vispy.sys_info()

Platform: Linux-3.10.0-1160.45.1.el7.x86_64-x86_64-with-glibc2.17
Python:   3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:40:17)  [GCC 9.4.0]
NumPy:    1.22.1
Backend:  PyQt5
pyqt4:    None
pyqt5:    ('PyQt5', '5.15.6', '5.15.2')
pyqt6:    None
pyside:   None
pyside2:  None
pyside6:  None
pyglet:   None
glfw:     None
sdl2:     None
wx:       None
egl:      EGL 1.4 Mesa Project: OpenGL OpenGL_ES
osmesa:   None
tkinter:  None
jupyter_rfb: None
_test:    None
GL version:  '2.1 Mesa 18.3.4'
MAX_TEXTURE_SIZE: 8192
Extensions: 'GL_ARB_multisample GL_EXT_abgr GL_EXT_bgra GL_EXT_blend_color GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_copy_texture GL_EXT_subtexture GL_EXT_texture_object GL_EXT_vertex_array GL_EXT_compiled_vertex_array GL_EXT_texture GL_EXT_texture3D GL_IBM_rasterpos_clip GL_ARB_point_parameters GL_EXT_draw_range_elements GL_EXT_packed_pixels GL_EXT_point_parameters GL_EXT_rescale_normal GL_EXT_separate_specular_color GL_EXT_texture_edge_clamp GL_SGIS_generate_mipmap GL_SGIS_texture_border_clamp GL_SGIS_texture_edge_clamp GL_SGIS_texture_lod GL_ARB_framebuffer_sRGB GL_ARB_multitexture GL_EXT_framebuffer_sRGB GL_IBM_multimode_draw_arrays GL_IBM_texture_mirrored_repeat GL_ARB_texture_cube_map GL_ARB_texture_env_add GL_ARB_transpose_matrix GL_EXT_blend_func_separate GL_EXT_fog_coord GL_EXT_multi_draw_arrays GL_EXT_secondary_color GL_EXT_texture_env_add GL_EXT_texture_lod_bias GL_INGR_blend_func_separate GL_NV_blend_square GL_NV_light_max_exponent GL_NV_texgen_reflection GL_NV_texture_env_combine4 GL_S3_s3tc GL_SUN_multi_draw_arrays GL_ARB_texture_border_clamp GL_ARB_texture_compression GL_EXT_framebuffer_object GL_EXT_texture_compression_s3tc GL_EXT_texture_env_combine GL_EXT_texture_env_dot3 GL_MESA_window_pos GL_NV_packed_depth_stencil GL_NV_texture_rectangle GL_ARB_depth_texture GL_ARB_occlusion_query GL_ARB_shadow GL_ARB_texture_env_combine GL_ARB_texture_env_crossbar GL_ARB_texture_env_dot3 GL_ARB_texture_mirrored_repeat GL_ARB_window_pos GL_ATI_fragment_shader GL_EXT_stencil_two_side GL_EXT_texture_cube_map GL_NV_depth_clamp GL_NV_fog_distance GL_APPLE_packed_pixels GL_ARB_draw_buffers GL_ARB_fragment_program GL_ARB_fragment_shader GL_ARB_shader_objects GL_ARB_vertex_program GL_ARB_vertex_shader GL_ATI_draw_buffers GL_ATI_texture_env_combine3 GL_EXT_shadow_funcs GL_EXT_stencil_wrap GL_MESA_pack_invert GL_MESA_ycbcr_texture GL_NV_primitive_restart GL_ARB_depth_clamp GL_ARB_fragment_program_shadow GL_ARB_half_float_pixel GL_ARB_occlusion_query2 GL_ARB_point_sprite GL_ARB_shading_language_100 GL_ARB_sync GL_ARB_texture_non_power_of_two GL_ARB_vertex_buffer_object GL_ATI_blend_equation_separate GL_EXT_blend_equation_separate GL_OES_read_format GL_ARB_color_buffer_float GL_ARB_pixel_buffer_object GL_ARB_texture_compression_rgtc GL_ARB_texture_rectangle GL_ATI_texture_compression_3dc GL_EXT_packed_float GL_EXT_pixel_buffer_object GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_rgtc GL_EXT_texture_mirror_clamp GL_EXT_texture_rectangle GL_EXT_texture_sRGB GL_EXT_texture_shared_exponent GL_ARB_framebuffer_object GL_EXT_framebuffer_blit GL_EXT_framebuffer_multisample GL_EXT_packed_depth_stencil GL_ARB_vertex_array_object GL_ATI_separate_stencil GL_ATI_texture_mirror_once GL_EXT_draw_buffers2 GL_EXT_draw_instanced GL_EXT_gpu_program_parameters GL_EXT_texture_array GL_EXT_texture_compression_latc GL_EXT_texture_integer GL_EXT_texture_sRGB_decode GL_EXT_timer_query GL_OES_EGL_image GL_ARB_copy_buffer GL_ARB_depth_buffer_float GL_ARB_draw_instanced GL_ARB_half_float_vertex GL_ARB_instanced_arrays GL_ARB_map_buffer_range GL_ARB_texture_buffer_object GL_ARB_texture_rg GL_ARB_texture_swizzle GL_ARB_vertex_array_bgra GL_EXT_texture_swizzle GL_EXT_vertex_array_bgra GL_NV_conditional_render GL_AMD_conservative_depth GL_AMD_draw_buffers_blend GL_AMD_seamless_cubemap_per_texture GL_AMD_shader_stencil_export GL_ARB_ES2_compatibility GL_ARB_blend_func_extended GL_ARB_debug_output GL_ARB_draw_buffers_blend GL_ARB_draw_elements_base_vertex GL_ARB_explicit_attrib_location GL_ARB_fragment_coord_conventions GL_ARB_provoking_vertex GL_ARB_sampler_objects GL_ARB_seamless_cube_map GL_ARB_shader_stencil_export GL_ARB_shader_texture_lod GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_multisample GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_uniform_buffer_object GL_ARB_vertex_type_2_10_10_10_rev GL_EXT_provoking_vertex GL_EXT_texture_snorm GL_MESA_texture_signed_rgba GL_ARB_draw_indirect GL_ARB_get_program_binary GL_ARB_robustness GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_texture_compression_bptc GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_viewport_array GL_AMD_multi_draw_indirect GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_ARB_base_instance GL_ARB_compressed_texture_pixel_storage GL_ARB_conservative_depth GL_ARB_internalformat_query GL_ARB_map_buffer_alignment GL_ARB_shading_language_420pack GL_ARB_shading_language_packing GL_ARB_texture_storage GL_ARB_transform_feedback_instanced GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_transform_feedback GL_AMD_shader_trinary_minmax GL_ARB_ES3_compatibility GL_ARB_arrays_of_arrays GL_ARB_clear_buffer_object GL_ARB_copy_image GL_ARB_explicit_uniform_location GL_ARB_fragment_layer_viewport GL_ARB_invalidate_subdata GL_ARB_multi_draw_indirect GL_ARB_program_interface_query GL_ARB_stencil_texturing GL_ARB_texture_buffer_range GL_ARB_texture_query_levels GL_ARB_texture_storage_multisample GL_ARB_texture_view GL_ARB_vertex_attrib_binding GL_KHR_debug GL_KHR_texture_compression_astc_ldr GL_ARB_buffer_storage GL_ARB_clear_texture GL_ARB_enhanced_layouts GL_ARB_internalformat_query2 GL_ARB_multi_bind GL_ARB_seamless_cubemap_per_texture GL_ARB_texture_mirror_clamp_to_edge GL_ARB_texture_stencil8 GL_ARB_vertex_type_10f_11f_11f_rev GL_EXT_shader_integer_mix GL_ARB_clip_control GL_ARB_conditional_render_inverted GL_ARB_cull_distance GL_ARB_get_texture_sub_image GL_ARB_pipeline_statistics_query GL_ARB_transform_feedback_overflow_query GL_EXT_polygon_offset_clamp GL_KHR_context_flush_control GL_KHR_no_error GL_KHR_texture_compression_astc_sliced_3d GL_MESA_shader_integer_functions GL_ARB_polygon_offset_clamp '

one solution would be to fall back to texture_format=None if we detect that texture_float is not present in the user's GL extensions, but we should check whether @djhoese thinks that should happen on the vispy side or not

[djhoese]

It wouldn't be terrible for it to be added to vispy, but since the default is already 8-bit maybe this should be a downstream (napari) check? Eh I guess a nice loud warning would work in vispy and force it to legacy 8-bit mode. However, that doesn't really help in the cases where napari might be making the user think that everything is working fine, but their output isn't as they expect. Napari wouldn't show a UserWarning from vispy in its UI anywhere, right?

Edit: To clarify, I would accept a PR that implements this in vispy, but not sure if that helps napari in the long run. Also how many users does this effect (just curious, not saying they shouldn't be considered)?

[tlambert03]

thanks for your input @djhoese, yeah, it's tricky. While texture_format='auto' does imply some degree of magic (implying that auto-avoiding float textures would be ok to do on the vispy side), it doesn't jive with the docstring: "'auto' which will use the data type of the provided image data to determine the internal format of the texture." ...

so I can certainly see an argument either way.

Napari wouldn't show a UserWarning from vispy in its UI anywhere, right?

it certainly could... currently actually all UserWarnings generated anywhere make their way to the napari notification system (though that's potentially changing soon, in which case we'd need to manually forward it)

not sure if that helps napari in the long run. Also how many users does this effect (just curious, not saying they shouldn't be considered)?

probably not many, I agree. but i do think if there's a case that's easy enough to check (e.g. "texture_float" not in vispy.sys_info(), or something lower level) that is guaranteed to fail with float dtypes, then it seems like this little bit of auto-selection can't hurt napari.

tl;dr: I'm torn, so I guess I'm going to add this to napari unless you specifically ask me to make a PR to vispy :)