Suggestion - Introductory Material from the Outside In ("What is gfx-rs?")

[AnthIste]

TL;DR It's difficult to say what gfx-rs does because it is actually just an abstract contract, this should either be embraced or penetrated with some mappings onto concrete implementations.

EDIT: It's worth noting that a cargo doc also goes a long way for discoverability because of the docs on types and modules. Doing this probably goes without saying, but as a Rust newbie I didn't immediately know I could do that. If some of the questions I raise are easily answered by looking in the docs, maybe it's worth making this breadcrumb trail explicit somewhere ("if you don't know what X is, try looking in the docs, they are there").

The Outside In?

So I've been lurking like crazy on #rust-gamedev and gitter, because it's really awesome seeing the birth of what looks to be a seriously cool project. I wanted to see if I can tackle any of the easy issues, but before doing that I wanted to get a good high-level overview of the project so I can see where to dive in and look for what.

What follows is a bit of stream-of-consciousness -- basically my thought process when poking through the code and examples as someone new to Rust and new to the project.

Actually, no, my thought process ended up pretty damn long, so it's at the end as an appendix.

So here are some suggestions as I think of them. I'll probably revise this due to grammar or thinking of more ideas. It's kind of high-level, but I really am noob when it comes to: Rust, Graphics, gfx-rs. So this is very much the paradigm that I am writing in.

Suggestions

Everything seems to be pretty crazily in-flux right now, but when the architecture stabilizes, a diagram will go a long way to show exactly where gfx-rs fits into the bigger picture, and where other tech (eg. raw OpenGL, glfw) fits in.

The problem with creating an abstraction like gfx-rs is that you aren't actually creating a graphics library, you are creating the contract for a graphics library. There just happen to be concrete back-ends like OpenGL shipped with the project that conform to this contract. But the examples all use the abstract front-end (which is the point), but without seeing how this translates to a raw implementation, it can be a bit hard to follow.

I think one of the main issues with this is: as soon as you decouple components, they become easier to look at in isolation, but harder to look at as a cohesive whole.

So maybe it will be valuable to describe gfx-rs in terms of the contract it provides, with a concrete example alongside. Looking at the cube example, there are some features that I discovered:

• It manages shaders
• It manages mesh data
• It manages textures (and can do some stuff to them, not sure what exactly)
• Probably more

So what generally does a mesh look like (in eg OpenGL / DX), and how does gfx-rs abstract this? But a higher level overview of architectural components and their responsibilities will go a long way to figuring things out.

So in terms of actionable 'front-page material' or 'what is gfx-rs and why do you need it' material:

• Pictures, with colors (architectural components)
• Architectural responsibilities
• Concrete feature list
• Contract guarantees
• In terms of what gfx-rs does not do, a list of what it expects to be able to operate

Appending A: My Initial Thought Process Looking at an Example

Sooo, where to begin looking? There is the Piston game engine (or is it an umbrella of projects?), which sits on top of gfx-rs. Which sits on top of OpenGL stuff like glfw and gl-init. Ok, so gfx-rs abstracts graphics calls, but doesn't manage your scene or anything like that (sweet, it says so on the homepage).

Looking at some examples, I see stuff about gfx:: imports (assuming this is gfx-rs) as well as some glfw:: stuff. There is a glfw::Context and a gfx::Device. OK, I need to figure out what a Context does and what a Device does. Also, why are they separate, and why is the one provided by glfw and the other one by gfx-rs?

Looking at the initialization, we start seeing glfw and gfx-rs playing together:

let (w, h) = window.get_framebuffer_size();
let frame = gfx::Frame::new(w as u16, h as u16);

let mut device = gfx::GlDevice::new(|s| glfw.get_proc_address(s));

Woah, ok, let's take a step back here. So there is a gfx::Frame (add that to the list of things to remember to figure out what they do). We initialize a device with some or other function pointer (to do what exactly?). Where do these addresses come from? Is this the standard way that gfx-rs talks to a OpenGL context?

Ok, well, let's just keep going. So there is some mesh data that we use to initialize some sort of typed Mesh abstraction:

let vertex_data = vec![
    // top (0, 0, 1)
    Vertex { pos: [-1, -1,  1], tex_coord: [0, 0] },
    // ...
    Vertex { pos: [ 1, -1, -1], tex_coord: [0, 1] },
];

let mesh = device.create_mesh(vertex_data);

Ok, so this means that gfx-rs abstracts meshes as well as shaders? Why does the mesh come from the device and not.. somewhere? else? If we are submitting draw calls to the device later, maybe this means the mesh instance is actually just a handle, and the device stores the mesh in whatever format it wants?

Same story happens for a TriangleList and TextureInfo. I also see stuff about SamplerInfo. Cool, so gfx-rs gets is input as some sort of standard data structures, and it also allows you to fiddle with textures. Make a mental note of that. What if the texture comes from a file? Does it manage loading, or conversions? Mental note.

Next up we create a whole bunch of new goodies:

let state = gfx::DrawState::new().depth(gfx::state::LessEqual, true);

let mut graphics = gfx::Graphics::new(device);
let batch: CubeBatch = graphics.make_batch(&program, &mesh, slice, &state).unwrap();

Ok, so we have a DrawState, Graphics and CubeBatch. So looking at the shader definitions the CubeBatch is a shader param, but we have to make it using graphics.make_batch(&program, &mesh, slice, &state).unwrap();. Looking at the dependencies of that call (shader program, mesh data, can't remember what slice and state are) it's a fairly complex operation that lies quite 'deep' in the architecture. Make a mental note to find out what batches are because I'm a nub.

Next up is some transform data and ClearData which is simple enough to understand conceptually.

Last up is the actual window loop and drawing:

    graphics.clear(clear_data, &frame);
    graphics.draw(&batch, &data, &frame);
    graphics.end_frame();

    window.swap_buffers();

Ok, so gfx-rs does the drawing, and glfw has the buffers and updates the window. How do they interact again? That getprocaddress thing we did? I can look at another example for using gl-init, but how does everything work here and not crash and burn? I mean, we have to present somewhere, but how does our gfx-rs code interact with the window if it was eg. not double buffered? Just some questions to make note of.

[AnthIste]

Revisiting gfx-rs.github.io/2014/07/25/intro.html actually answers a lot of the questions as to the what and the how, but not neccesarily the why.

the built in code docs help figure out what the library does outside the examples.

these could ppssibly be referenced from another source?

[brendanzab]

I will have to take the time to read this more deeply, but thanks so much for writing this up! It's important we get outside points of view.

On gitter:

@cmr: it seems like we might want to write a "designing a modern graphics api" post that provides the motivation for why gfx-rs is designed the way it is. it's certainly not obvious to someone who isn't deep into all the details

@bjz: we also need to discuss our directions re. new backends

[brendanzab]

Working on the dependency graph:

digraph gfx {
    rankdir=BT;

    subgraph cluster_gfx {
        style=filled;
        color=gray80;

        node [shape=box, style=filled, color=white];
        "gfx" -> "render";
        "gfx" -> "device";
        "render" -> "device";
        "device" -> "gl_device";
    }

    node [shape=none];

    "gl_device" -> "gfx_gl";

    "cube" -> "gfx";
    "cube" -> "glfw";
    "cube" -> "cgmath";

    "terrain" -> "gfx";
    "terrain" -> "glfw";
    "terrain" -> "cgmath";
    "terrain" -> "genmesh";
    "terrain" -> "noise";

    "gl-init" -> "gfx";
    "gl-init" -> "gl_init";

    "performance" -> "gfx";
    "performance" -> "glfw";
    "performance" -> "gfx_gl";
    "performance" -> "cgmath";
}

[kvark]

Revisiting gfx-rs.github.io/2014/07/25/intro.html actually answers a lot of the questions as to the what and the how, but not neccesarily the why.

Reminds me of the "How greater leaders inspire action" talk.

Following the idea, we need to focus on "why" factor, and explain "how"/"what" as secondary. This means not just writing another blog post or wiki page. This means putting all the "why"s in the short README, and then giving links from there to everything else.

[bvssvni]

@AnthIste This is very good.

@kvark Thanks for the link!

[AnthIste]

Something else to consider is "escape hatches". Things like:

• How much architecture does gfx-rs impose on you, that is non-negotiable?
• Where can you make your own decisions as to what sort of pipeline and resource management you want to set up?
• Does gfx-rs provide you with a set of primitive concepts (eg. batches / slices are things I have seen) that you need to adapt into an architecture, or are they simply convenient wrappers?
• Is there any way to bypass gfx-rs and do things manually (even if this is not portable) or is it one of the goals of the project that you are strictly not supposed to do this?

Something else to consider representing visually is data flow:

• Where are things stored in memory?
• What is cached and where?
• What is managed by the user and not the library?

[kvark]

@AnthIste thank you for good input! These questions are important.

[kvark]

Outdated by now. gfx-rs master is basically Vulkan with caveats.