Move maplibre-rs to the Supported tier #282
Conversation
|
I took the liberty to draft this to get the ball rolling. I've spent some time trying understanding what maplibre-rs is all about, and this is my understanding so far (which I'd love to have corrected if I'm not on point) - @nyurik , looking forward to a review from you before further community feedback, and a formal vote, is requested. |
|
As a goal from a renderer I would expect to see some compliance to the existing spec, otherwise we are building a renderer that is not really related to maplibre. Just my two cents I guess. |
|
Excellent point - Added that bullet to the list. I don't know where maplibre-rs currently stands wrt the style spec, but would love an update on it. |
|
Thank you for taking care of this and writing up this proposal!
This comes up a lot with discussions about maplibre-rs, but I believe that it is not really a goal for 2023-2025. There are a lot of things to solve and actually writing a mapper in the end which supports the maplibre spec should be fairly easy given that the hard parts are solved. Right now I do not plan to follow the spec while implementing maplibre-rs. maplibre-rs should not be developed around the spec. There are inherent limitations in the spec I would rather avoid to implement actually.
Shaders are a very deep implementation detail. The main point is that it is based on WebGPU. rust-gpu is not interesting for now.
I actually do plan to replace native and GL JS. Not now, and not within the next years, but eventually that is a goal of maplibre-rs. |
|
Thanks for clarifying! |
I'll be happy to get a clarification around this statement. I'm not saying the spec is perfect or holy but it is a a major part of maplibre, not one that we plan to ditch in the foreseeable future as I understand. |
@HarelM I spoke with @maxammann about this point before -- there are two aspects:
There is no way This may result in some compatibility layer, dynamic style rewriting, additional compatibility modules, or any other "layer of indirection" methods as needed - but lets first figure out IF any changes are even needed or not. |
|
Okay, so the idea is to lay the foundation for the renderer first, focusing on flexibility, functionality, performance - and this could take years. When it's done, and adoption becomes of interest, a way to load style spec compliant styles will be introduced somehow. Did I get that right? |
Almost - I honestly believe (hope!) it should take less time than that. Also, I hope we can re-use many of the native C++ components as is, at least initially, and eventually migrate them to Rust as needed. This will also be part of the experimentation. |
|
Okay, I understand. So the style-spec isn't so much in focus now, as there are more low-level task to resolve. It's adamant for multiple of us that there is an intention of becoming style spec compliant eventually, even if we don't know how to do it technically yet. If this project will be released for consumption as a MapLibre renderer, as Max mention to compete with GL JS and Native, support for this should certainly be in place before reaching a 1.0 milestone, because until now MapLibre renderer has implicitly has been short for MapLibre Style Spec compliant renderer. In case there isn't an intention to let end-users engage with this through the style spec (it seems like there is though), I'd even suggest to incubate this under a different name (still within the MapLibre org) to avoid confusion, similar to how Martin isn't called @maxammann do I understand you correctly that you see the style-spec more as just one (somewhat limited) public api of potentially multiple that the renderer could support, and that it is more of an after thought if the renderer is built flexible enough? @nyurik , @maxammann does this make sense to you, or do you have a different POV? |
|
Perfect summary, thanks! |
|
I know it's a bit off topic, as it's more of an implementation detail, but it still seems like a quite fundamental question for the project. How come bevy (OSS Rust pendant to i.e. Unity game engine) isn't used directly in this project, and something that looks like a fork of it is? Isn't there a lot of value to be found in having all those continuous improvements in a render pipeline upstream? Is map rendering so different that it's necessary to fork and in-house all that complexity? I read this discussion, and wondered if the status of any of the mentioned points have changed since it was written. Imagine i.e. that performance, mobile/web support etc. has improved significantly over the last year and might slowly become more of a pro than a con - also with the webgpu support announcement of this week. |
R&D efforts to which resources are allocated should in my eyes have the goal to bring some tangible results for the existing projects (with users). When it comes to building knowledge in the community: it's nice to learn Rust, but I don't know any passionate developer who isn't at least curious about the language. What I am trying to say is that there's not really any additional extrinsic motivation needed to get people to learn Rust. Not deep into graphics programming, but from what I can tell WebGPU does indeed look very promising. Note that it does have a C++ API, so after the renderer modularization for MapLibre Native is complete we can also explore that tech by trying to implement a WebGPU backend for MapLibre Native. Someone will have to back me up here, but it's supposed to be quite similar to Metal, so likely no further architectural changes will need to be made for such an effort. Some questions. The 'sponsored tier' allows bounties. Who will decide what work gets bounties attached? I suppose not @maxammann since he is the main developer working on the maplibre-rs project? What are the goals of the project aside from building Rust and WebGPU knowledge in the MapLibre community (already covered above)? If they include feature parity with MapLibre GL JS and MapLibre Native (eventually), how can you ever achieve that if both these libraries continue to evolve (and improve) as well over the next few years? We should also consider the risk that financial incentives actually kill or harm a passion driven project (see this book on that topic). I personally think money is especially useful to get people to commit their time when they wouldn't otherwise be able to, or to give a little push to get stuff done that would not get picked up otherwise. My main question is thus: will bounties help us achieve the goals of the project? |
|
@louwers great point about the bounties (and all the other points too). It's a bit of a conundrum how a system can exist in a Supported tier project, where there are no contracted maintainer associated to it. Because it's a problem inherent to all Supported-tier projects, that we only see now because maplibre-rs is the first in question, did I make a ticket for that specific issue here. |
|
@louwers , regarding adding WebGPU to Native, I don't know much about it, and this is probably not the best place for a discussion of it either, but I think it could be worth exploring. What I can see is that articles like this group Metal/WebGPU as 'concise and simple' with Vulkan described as 'complex', so if we i.e. can get anything near the performance of Vulkan and DX12 etc. out of something simpler than either, that might make for an interesting value proposition. |
|
@louwers , I think you're right that there is an intrinsic motivation amongst many developers to learn Rust, so we don't have to sponsor that per se. To get some data behind your point, it's been the "most loved" and "most wanted" language in StackOverflow Dev Survey for years now https://survey.stackoverflow.co/2022/#most-loved-dreaded-and-wanted-language-love-dread I personally wasn't that sure about the goals of the project, so it became a quite fluffy knowledge-statement with little tangible outcome. It would be good to have someone more involved with maplire-rs state those goals. |
|
What both @wipfli and I have reasoned our way to regarding the bounties for sponsored tiers is that a maintainer who understands what's going on in the project (like @maxammann in this case) could volunteer to manage them and not hunt themselves - and if no such volunteer exist there effectively wouldn't be bounties for now. |
The MapLibre specification is one way of configuring maplibre. In the case of maplibre-gl-js it directly describes how the renderer is working, without a proper abstraction. Supporting the MapLibre specification close to 100% is simple if the rendering core is well architectured. For instance, OpenLayer also supports the maplibre styles to a certain extent. @nyurik
I would replace "styling language" with just "renderer". The architecture of the renderer is where the magic happens and which describes what is possible to render or not. The style specification is kind of a feature specification of what features the renderer supports, without describing how it should implement the rendering.
Yes absolutely! I think aiming for pixel-perfect equivalence with maplibre-gl-js is not really reaslistic. So as long as switching from maplibre-gl-js to maplibre-rs does not fundamentally break users usecase I think it is fine (like not supporting all styling features, rendering items in different order).
Yes, this is a different question, and was already the main concern when you initially reached out to me. I'm still very sceptical about the benefits and viability of the bounties program, also given to its young age. In my opinion it has not yet proven itself in maplibre-gl-js and maplibre-native. Given that I do not want to replace my current development efforts with managing bounties.
I don't fully agree here. R&D could also lead to new projects. But I doubt that it will be limited to that. So I'm not concerned here.
Yes, it should be less than a few days of work to port from Metal to WebGPU. Apple was the initiator of WebGPU. So the idea of WebGPU is to be Metal for the web. It would be called WebMetal if other vendors would have agreed. The thing I'm concerned about with the current goal of implementing Metal, is that it was never considered to target WebGPU instead of Metal. Maybe it was but I never heard of that idea.
Yes and it is hard to find people who are:
I kind of doubt that this will change with bounties.
My personal goal of maplibre-rs is to find enjoyment in its development :)
Yes this is an important point, but also true for maplibre-gl-js and maplibre-native. I don't see how bounties could change the devlopment of maplibre-rs right now, because bounties can not really be used to drive architectural changes. They are nice for implementing certain features or fix bugs, but it does not replace a core team. |
birkskyum
changed the title
|
Thanks for the answers @maxammann. I'm in favor of moving this project to the supported tier (can't approve because I made the PR). My reasoning is that this project is well aligned with our charter and sponsor program, so while it isn't a core tier project, it has the potential to become one, and that is the type of projects I believe we should support. I see the supported-tier as an incubation step. I think we still have some work to do figuring out how best to support supported-tier projects. The bounties were mainly designed for core-tier projects, managed by paid maintainers, and it might simply not be the best fit for all projects at all stages. |
|
One thing which might be interesting is to reserve some budget for students who decide to work on maplibre-rs. In early 2022 a student implemented the basis for the raster tile rendering. During my studies I worked a lot for a non-profit, where I was able to earn some money doing open-source work. Maybe some program like this could be valuable for maplibre-rs. Basically everyone wins, students learn things and get some money, and maplibre-rs gets more eyes on its source code. The connections to universities are already established :) |
|
The ticket will be resolved latest June 1st |
|
A majority has been found in favor of changing the tier of maplibre-rs to Supported. Merging. |
Intro to maplibre-rs
maplibre-rs is an R&D effort that tries to answer the question - If MapLibre Native and MapLibre GL JS didn’t exist today, how would we build such a renderer?
What we want from a renderer is:
So far the project has been exploring the use of Rust as a language, therefore the -rs name, for the reasons that it’s been hailed as a more memory-safe, thus approachable, language than the intimidating C++/C, while maintaining the high performance of a non-garbage collected language. This is secured through extensive compile checks, known as the borrow checker.
The project has also been exploring using the WebGPU API through the wgpu project, and eyes are kept on alternatives like the upcoming Rust-first approach to writing shaders, known as rust-gpu, as well. Having a single language for our shaders would make the repository more maintainable than having metal/opengl/vulkan(?)/webgpu(?) as is the current approach in MapLibre Native.
Browser support of WebAssembly (wasm) opens the door for us to use a native MapLibre implementation like maplibre-rs or MapLibre Native in the browser. This can be done by compiling native code to wasm. For Rust, this can be done using rustwasm or sledgehammer. For C++, this can be done with emscripten. This approach will only become more feasible as both WebAssembly and WebGPU get more capabilities and support in the browsers. The exploration into this area has the potential to one day make our maintenance and feature-parity burdens lower by reducing our offering to a single renderer. The current state of maplibre-rs can already now be executed through this approach in the browser.
WebGPU - Demo - latest Chrome only
WebGL - Demo
Here is a whitepaper with further information
What would be a success for maplibre-rs?
maplibre-rs doesn’t have, or even aim to, replace GL JS and Native to be a success. Even if the only tangible result of this R&D effort is an increased community competence in Rust and nascent graphics APIs, that alone would bring long-term value for both GL JS and Native.
Why does it have to be in the supported tier?
R&D efforts have been voiced as important by our sponsors, because innovation is necessary to stay relevant long-term. This project has come a long way without funding, and putting it in the supported tier is mainly a pre-emptive move to avoid complete stagnation. Lastly, it's a recognition that this effort is likely to bring various kinds of indirect, and maybe even direct, value to our core tier projects Native and GL JS, which our sponsor program is currently centered around.