-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question] - Steps to create new component #44
Comments
Hey @sweco-semtne, the points you listed are accurate. There are two points sides to components. The first one is the output specification (currently documented in The other side is rendering the given Elixir term into that specific output (implementing The output format may be more generic, for example we have the As for the UI, it depends on the case, we currently use JS only for rendering Vega-Lite plots (and that does require a hook). [1/2] |
[2/2] We are definitely open to adding new components, but in cases where it requires integrating an external JS package we need to be careful, because every such package may significantly increase the bundle size. What's the exact idea behind the map component? What kind of Elixir data would that render? Also, see a relevant question here livebook-dev/livebook#305 :) |
Looking at Folium, I can see that they manipulate the map using leaflet.js, so based on the Python code they generate some JavaScript from a template and then let this JavaScript run. So to do something similar we would need either:
|
@jonatanklosko thank you for a very thorough answer. Or a data frame similar to GeoPandas. The reason for having a Server is to be able to push live data on to the map canvas for for instance GPS positions. It could also possible be used to filter tables on geography if a map input component is built. I'm not looking to use leaflet but instead the MapLibre library which is a fork of Mapbox 1.x The library has the concept of data sources that can be updated with a setData function that would map well to hook. Under the hood the input data could be converted to GeoJSON features. I agree that some kind of specification on what background maps, styles etc to use should be formalized but I'm not sure there is any standard worth sticking to. There is the old OGC Web Map Context and several other proprietary means of serializing the map context. I'll try to come up with an example flow to better describe what I'm thinking. Again, thank you for taking the time to respond and also investigate! |
After some discussions with @jonatanklosko and @thbar yesterday, we think it is work adding some support for map rendering in Livebook/Kino. We mentioned VegaLite before but it seems Vega does not have support for tiling yet. My initial suggestion is to support GeoJSON rendering exclusively, so the API would rather evolve around GeoJSON, with some basic configuration for picking the tile provider. Would that be helpful to you @thbar and @sweco-semtne? Also, does anyone have a rundown on the differences or solid preferences towards MapLibre or Leaflet? |
Hi @josevalim On a side note, we've been using |
The Style specification for MapLibre can be used as a basis for a map "context". The GeoJSON specifies the data but there still is a need to specify how to visualize it, ie layer order, colors, shapes etc |
I believe a GeoJSON layer + tile background rendering would be a natural first step, and would probably take us quite far already. It would definitely be useful to the various data explorations we do at https://transport.data.gouv.fr. Let me share extra ideas that could be useful for later (these are things we are also looking at) ; these are currently overkill, but keeping them as options in mind could help design something that can go further later:
I am not familiar at all with MapLibre. At https://transport.data.gouv.fr, we currently use LeafLet and mostly quite happy with it. I'll add a couple of interesting links to the discussion:
|
@thbar MapLibre is the community fork of Mapbox pre 2.0. So it supports vector tiles out of the box. Or did you mean the library should support generating it in Elixir-land? And it would be interesting to wrap a data table with a function to transform it to GeoJSON. |
Neat, thanks for clarifying this!
This is a great question! Yes, I meant "Support /Elixir/ code generating GeoJSON on the fly", definitely. Having "server-side" (Elixir-land) data generation will allow to process large amounts of data, cluster them if needed, clip as needed, and only send what is relevant to the rendering part.
By bounding box I meant: the end user could in the future be able to control the map with arrows, and the resulting viewport (lat/lon top left and bottom right) gives coordinates, which can be "sent back" to the Elixir part, so that it only generates the right bits of data. If needed, I'll emphasise that a first version with no clipping of any sort, "just" rendering an Elixir-generated GeoJSON on a map, and letting the user configure a tiling background where the coordinates would automatically use the same referential and be well aligned with the GeoJSON, would already provide a huge value (the bounding box idea is more fuel for thoughts for later in my head!). |
I'll just add one more link for inspiration: https://geopandas.org/docs/user_guide/interactive_mapping.html The GeoPandas library just introduced the explore function that creates an interactive leaflet map based on a dataframe. |
Very interested in this conversation as I work in the web mapping / data science field where I use and develop mapping components regularly. New to Elixir and loving livebook/kino so I would like to add deck.gl to the conversation. deck.gl supports a declarative system for describing layers and their props, and this declarative API can be trivially exposed via the JSON API. Hoping this may make it easier to integrate. My opinion is a webgl-based mapping library (maplibre, mapbox v1, or deck.gl) would improve livebook more than a leaflet-based solution. see bindings for python (pydeck), Dash Deck , R (mapdeck), Vega(vega-deck.gl) at https://github.com/visgl/deck.gl Brent |
Currently in elixir there are no libraries equivalent to Geopandas, which in its internal development makes use of other mature and stable projects such as shapely to work with geometries, pandas for DataFrame, etc. The first thing would be to create a similar project in elixir that will enhance the use of the platform in this niche. There are two real options: 1- Being Rust a platform that has projects similar to python in this niche, create elixir NIFs with Rustler that brings functionalities to the elixir ecosystem and use them in a transparent way. Creating similar projects from scratch would be very expensive. 2- Find an efficient way to integrate python into livebook and call these functionalities without much cost to the system Later it would be used to display Deck.gl or Maplibrejs, due to its webgl rendering capabilities. |
Being new to Elixir, I have been spending some time getting up-to-speed with the Elixir geospatial libraries. Wow! there are some really good libraries available to build the core of a geospatial data handling/indexing solution. Seems like pkinney would be someone that might have some good ideas into this conversation since he developed quite a few of these libraries that cover core geospatial data formats and storage/indexing requirements. So we already have the libraries above to use within livebook. The missing part is still the map feature and tile rendering - A package that takes a JSON specification and renders the map like @jonatanklosko mentioned above seems like a really good solution when you take into account the existing elixir geo libraries. Brent |
Yes, after a reevaluation, I am thinking that maplibre plus built-in support for the |
Maplibre.js and geo type support would be amazing. This is really great news. |
Also just came across this component which is the beginnings of a simple SVG map renderer that handles OSM map tiles. https://hex.pm/packages/live_map |
Hi everyone! We have pushed an initial implementation of |
Hi!
I'm not sure where to direct my question so apologies if this isn't the correct forum or place.
I'm trying to add a dynamic map component to Kino to interact with geographical data. It is loosely based on Folium
I'm trying to understand the flow and extension points in Kino and Livebook.
My understanding so far:
Kino
add a new type and mapping in Kino.Output
add a defimpl in Kino.Render (to_livebook)
implement a server in Kino
Livebook
introduce a LiveComponent in Livebook for rendering the output
create a hook for the client side JS part of it
add a type in Livebook.Notebook.Cell.Elixir
Are there any other points to consider?
Also I would like to thank all of your building such a fantastic tool! Keep up the good work!
The text was updated successfully, but these errors were encountered: