Pixel by pixel rendering

[ktb92677]

Hey, I can't for the life of me figure out how to get the RenderDrawListsFn to work for the graphics engine I'm using. I can't entirely specify why, but I cannot use SDL or OpenGL or DirectX or anything like that. I have access to only some bare bones graphics functions: setColor(c), setPixel(x, y), drawLine(x, y, x1, y1), drawTriangle, fillTriangle etc. How can I implement this library with only these bare bones functions if at all possible?

[ocornut]

You just need to be able to render textured triangles with clipping. Any api from the last decade should be able to do it.

Each ImDrawCmd contains a number of triangles to render with 3 coordinates and 3 uv coordinates for the texture to fetch from the vertex buffer array.

You're not giving much information so I can't tell if the system you are using supports textured triangles.

[ktb92677]

The system I'm using is hand written, so alas if I want to get textured triangles and clipping I have to make these by hand. This is the only GUI toolkit in existience as far as I can see that demands the programmer fill in the render method if you will so I'd really like to get this to work. Could you give me any pointers here as to where I should start looking on how to write these sorts of algorithms? (or even what a textured triangle really is.. where is the texture? What exactly are UV coordinates? I've programmed in OpenGL before but I always just took these terms for granted and never really looked into what they actually meant)

Thank you so much for your assistance in this matter!

[ocornut]

What sort of hardware is your system written on top of ? does it has a 3d gpu ? could you clarify which api you are using ?

If you have a fill trangle function you can give a shot at rendering imgui triangles without textures, using the color of the first vertex of each triangle, and see if you see the shapes appearing (you won't see the text without texture). The texture is obtained when you call ImGui::GetIO()->Fonts->GetTexDataAsRGBA32() etc. but you can start without and see if you can see the general ui shapes even without the text.

I can't really start to explain what a textured triangle is seeing this is very basic and probably covered by thousands of tutorials on the internet. Best trying to clarifying what sort of hardware/api you are using or ask for assistance to whoever is making/using this api on how to render textured triangles.

[ktb92677]

Okay, well I am using the Linux framebuffer device (/dev/fb0). I created on my own a very basic graphics class wrapper to the device. There is no 3D capabilities that I haven't programmed in. Here is what I have for my render method so far, could you let me know where the vertices are? Also if I don't have texture rendering is there an alternative way to render text? Thank you so much!

void RenderDrawLists(ImDrawList** const cmd_lists, int cmd_lists_count)
{
if (cmd_lists_count == 0)
return;

for (int n = 0; n < cmd_lists_count; n++)
{
    const ImDrawList* cmd_list = cmd_lists[n];

    for (int i = 0; i < cmd_list->commands.size(); i++) {
        const ImDrawCmd* pcmd = &cmd_list->commands[i];
        if (pcmd->user_callback)
        {
            pcmd->user_callback(cmd_list, pcmd);
        }
        else
        {
            //graphics.setColor(c);
            //graphics.fillTriagle(x, y, x1, y1, x2, y2)
            //what goes here?
        }
    }
}

}

[ocornut]

what goes here?

It looks like you haven't looked at the provided examples very much.. The ImDrawCmd structure has a 'vtx_count' field that the number of vertices to draw. Divide by 3 to get triangles. The vertices are stored in ImDrawList vtx_buffer, so each command consume 'vtx_count' of them.

Look up for a software implementation of a textured triangle rasterizer. You already have a non-textured triangle rasterizer with your fillTriangle() function so it's only a step up. I would suggest trying to just hook up your non-textured renderer.

There is a chance that it'll be slower than necessary if your software rasterizer is not efficient. There lots of overdraw (painter algorithm) and the library wasn't designed with the assumption that somehow may want to software rasterize those triangles manually. But there's no reason it wouldn't work. 95% of what imgui renders are actually axis aligned quads made of 2 triangles == 6 vertices so you may be able to take advantage of that by detecting them and specialize to just render those with a faster code path for axis-aligned quads (aka sprites).

[ktb92677]

Don't I need a pixel map of some kind to map to the triangle? Otherwise what colors am I mapping? Or do I just make a gradient of all of the colors of the corners of the triangles?

[ocornut]

That's what the texture is. The texture color has to be modulated with the vertex colors. In the current version of imgui all triangles have the same colors in all 3 vertices but future version won't, colors will need to be interpolated between the three vertices of the triangles to allow for anti-aliasing in the future.

[ktb92677]

Okay, cool. Is there a technical term for a texture of a gradient-ed triangle of three colors? Also, you mentioned above something about how text won't work and that I should use "GetTexDataAsRGBA32()". I don't see this referenced in the OpenGL example. Is the method I'm doing here going to show text?

[ocornut]

software rasterization
triangle rasterization
triangle rasterization interpolation

But then again first try without texture and without interpolation and see if you can get shapes appearing but if it's fast enough for you.

The OpenGL2 example uses GetTexDataAsAlpha8() - the texture only require 1 component so I allow the user to get it as 1 (a) or 4 components (rgba).

[ktb92677]

I tried to render it and it worked! Well to some extent. The problem is the text shows up as a bunch of blocks. Here's my final render code:

void RenderDrawLists(ImDrawList** const cmd_lists, int cmd_lists_count) {
if (cmd_lists_count == 0)
return;
int vertex_index = 0;
for (int n = 0; n < cmd_lists_count; n++) {
const ImDrawList* cmd_list = cmd_lists[n];
for (int i = 0; i < cmd_list->commands.size(); i++) {
const ImDrawCmd* pcmd = &cmd_list->commands[i];
if (pcmd->user_callback) {
pcmd->user_callback(cmd_list, pcmd);
} else {
for (int k = vertex_index; k < vertex_index + pcmd->vtx_count; k+=3) {
graphics_t->fillTriangle((int) cmd_list->vtx_buffer[k].pos.x, (int) cmd_list->vtx_buffer[k].pos.y, cmd_list->vtx_buffer[k].col,
(int) cmd_list->vtx_buffer[k + 1].pos.x, (int) cmd_list->vtx_buffer[k + 1].pos.y, cmd_list->vtx_buffer[k + 1].col,
(int) cmd_list->vtx_buffer[k + 2].pos.x, (int) cmd_list->vtx_buffer[k + 2].pos.y, cmd_list->vtx_buffer[k + 2].col);
}
}
vertex_index += pcmd->vtx_count;
}
}
}

[ocornut]

Well this is what we are discussing above, you need textured triangles to see the text.

You'll also need to handle clipping / scissor rectangle for correct fine clipping, but if you don't it will still look mostly ok.

[ktb92677]

I just need text to show up, what's the bare minimum I need to do to get text to show up?

[ocornut]

Render textured triangles. Simplest point filtering ok.

[ktb92677]

I actually am taking into account the gradient-ed triangles, but like you said the color of every corner is the same, so they are all solid colors. Where do I find the textures for the triangles?

[ocornut]

I already said you can get the texture by calling GetTexDataAsAlpha8 or GetTexDataAsRGBA32. Each vertex has uv coordinates referring to positions in the texture (the coordinates are normalized 0.0->1.0 mapping to each texture axis).

[ktb92677]

Okay, I see, but how can I get the blocks to disappear that already show up on the screen?

[ocornut]

I don't understand your question.
The texture include an alpha component so the triangle needs to be alpha blended. Typically the default font will use 0 for transparent pixel and 255 for opaque pixels in the alpha channel.

[ktb92677]

I've been at this for hours now and I can't for the life of me get text to show up. I have literally checked all the pixel transparencies where text should be and they are all the exact same. All I get are a bunch of blocks. Do you think you can point me in the right direction?

[ktb92677]

I don't think I'm loading fonts correctly. How exactly do I load a tty font with just the file and no other libraries? (I used "io.Fonts->AddFontFromFileTTF" as well as "io.Fonts->AddFontDefault()" alone and nothing happened).

[MrSapps]

I don't think your problem is really anything to do with ImGui, its more of a "how to implement a software rendering engine". The loading of default fonts is done in every single ImGui example for every API OpenGL/SDL etc.

[ktb92677]

Okay, so to confirm the one line: io.Fonts->AddFontDefault() is enough to load a font and there is nothing else I need to do?

[ocornut]

That's to load the font but then you need to retrieve the texture. Please follow the examples code.

[ktb92677]

"io.Fonts->AddFontDefault()" isn't referenced in any of the examples (ignoring comments).

I see that these AddFont functions return an ImFont type. The "GetTexDataAsAlpha8" requires 4 parameters.. none of which seem to actually be in the ImFont type. Some pointers as to where to grab the four parameters from ImFont type would help!

[xythobuz]

As has been suggested many times before in this thread, look at the examples!

• OpenGL example using GetTexDataAsAlpha8
• OpenGL 3 example using GetTexDataAsRGBA32
• DirectX 9 example using GetTexDataAsAlpha8
• DirectX 11 example using GetTexDataAsRGBA32

[ktb92677]

Could you please point me to an example that uses: io.Fonts->AddFontDefault(). This is the function I am unsure of how to properly implement

[xythobuz]

You do not need to use this function. Only if you want to load your own font.

[ktb92677]

Hmm, so I don't need to use "io.Fonts->AddFontDefault()" and I can't use any OpenGL or DirectX related functions (as all of the examples you provided use in some fashion to load textures) then what do I use?

[xythobuz]

I thought you're rolling your own hand-written API? You'll have to implement drawing textured triangles yourself. The texture you get from GetTexDataAs... is the texture you need to apply to your triangles, according to the provided UV coordinates. You may want to read something like this, first.

Edit: And these articles go into really great detail.

[ktb92677]

Hmm, okay, so this "io.Fonts->GetTexDataAsAlpha8(&pixels, &width, &height);" fills the array "pixels" with texture data, and then I have to map that to the triangle. Okay I see. Do you think you could give me a basic break down of how to parse the array (pixels)? I mean I don't know what format it is in so it'll be complex to iterate through that data.

[xythobuz]

Well, if you're using Alpha8 every pixel will be one byte, with a greyscale color value from 0 to 255.

If you're using RGBA32, every pixel will be four bytes, first Red, then Green, then Blue and finally the Alpha channel.

[ktb92677]

Okay, well I got that, the array is a linear list of pixels. But what I don't get is where to put each pixel? For example the first pixel where does it go? The second? etc

[xythobuz]

I really can't explain how to do this in an Issue comment. You may want to read some of these articles:

• Basic Texture Mapping
• Texture Mapping
• Texture Mapping

[ocornut]

What Xythobuz said. Read those 3 articles. If it's not enough read other articles and spend time following e.g. basic OpenGL tutorials and software rendering tutorials. For what it worth there's probably dozens of librairies or free piece of code to do exactly what you want, you have to find them.

[ktb92677]

I am still lost here. I did do my research, all that you asked me to do yesterday. I have alpha overlaying working perfectly and the triangles are all gradient-ed properly. The algorithms are not the problem, the texture array is just an array of colors, I understand how to map textures but how do I use the array of pixels? Is it one big texture that I'm cutting out from? If so where do I cut from and what triangle do I map it to? Is it an array of smaller textures? If so how large is each texture so I can find them and once I find them what triangle do I map it to? If the triangles already each have a "col" element how do we apply another color ontop of that?

[ocornut]

There is only one texture it contains all the characters of the font and other things.

struct ImDrawVert
{
    ImVec2  pos;
    ImVec2  uv;
    ImU32   col;
};

The UV coordinates of each vertex you are asked to render are referring to positions inside that texture. Say the texture is 256x64 pixels.
UV (0.5, 0.5) == pixel at 128,32

UV are interpolate between the 3 vertices of a triangle so that each target pixel will have a unique UV coordinate (the same way you would interpolate color). Fetch the pixel at the interpolated UV coordinate and multiply it by the interpolated color.
(You don't need bilinear filtering for the default imgui settings.)

You haven't looked hard enough about texture mapping, uv coordinates and software rendering. I'm not saying it's trivial but if you are doing C++ and dealing with the Linux framebuffer for whatever reason you should be able to figure those things out.

[ktb92677]

Okay, I think I understand! Thank you for the explanation I'll try and implement it. Thank you for going through this hassle for me. I'll post my code if I get it working (perhaps an example using just raw pixel drawing would be good to add to your collection of examples any how considering every graphics engine has at least a draw pixel method, even one's older than decade ;) )

[ocornut]

That code is already available in many forms a google search away, e.g.

http://www.xbdev.net/maths_of_3d/rasterization/texturedtriangle/index.php
http://www.stanleyhayes.com/triangle-rasterizerc.html
http://www.hugi.scene.org/online/coding/hugi%2017%20-%20cotriang.htm

good luck :)

[ktb92677]

I've been looking into doing this and I just can't seem to make it work. Specifically I'm not sure what this means: "Fetch the pixel at the interpolated UV coordinate and multiply it by the interpolated color." How can I multiply two colors together? Do I just multiply each component together? What do I do with overflow?

[ocornut]

Multiply each components together.

0..255 in a single byte correspond to 0..1.0f as floating point, so there can't be overflow.
This topic has nothing to do with imgui so if you have further questions it would be preferable to find a forum dedicated to this topic.

[adminy]

Any update on this? Is it possible to do software pixel Array extraction with ImGui?

[ktb92677]

Nope, I ended up moving to nuklear (https://github.com/vurtun/nuklear). It is a shame that @ocornut will not cater to the many people without access to good graphics libraries (such as those of us doing work on embedded machines or those of us with only access to a framebuffer like /dev/fb0).

[ocornut]

@adminy: this question has nothing to do with imgui. You’ll obtain your pixels via your graphics api eg you render into an offscreen framebuffer and retrieve to data back to cpu. Either way it isn’t within dear imgui scope.

[ocornut]

@ktb92677 I don’t think they asked about software rendering (there is an example sofware renderer on the wiki) but about retrieving their output.

[nicolasnoble]

@ocornut: you may want to activate https://github.com/apps/lock/

[ktb92677]

@ocornut oh wow, that is a new addition since we had this discussion, perhaps I will try and come back to imgui...

[adminy]

Is it possible to integrate it with TinyGL graphics? @ocornut it can load textures onto any shapes as far as I'm aware. It can do many things but It can't compute shader programs.

[LoganDark]

I also only have access to a 32-bit buffer and would have liked to use Dear ImGui... except the fact that it only supports outputting triangles is extremely off-putting. Please reconsider this issue, maybe at least add an official software backend or something to make it easier for those without hardware-accelerated graphics APIs :/

[rokups]

Someone has made a software renderer. Try it: https://github.com/emilk/imgui_software_renderer

[ocornut]

There are 3 software renderer listed on the Wiki
https://github.com/ocornut/imgui/wiki

[ktb92677]

Oh wow looks like there finally is a good software renderer for imgui! Can't believe it has been five years since my original post lol

[MrSapps]

And yet it is still nothing to do with ImGui. ImGui uses a rendering API to output, if you use software, ascii, Morse code, openGL or anything else is up to you.

[ocornut]

https://github.com/ggerganov/imtui

[adminy]

Having the output as triangles was fine for me since I had a GPU that did just that, I don't find it odd anymore that imgui's output is triangles but wow, characters as the output sounds awesome. must try

[ktb92677]

@paulsapps hey!!! it's like a big 5 year reunion!

[MrSapps]

I wonder if you managed to implement software rendering yet @ktb92677