Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added texture type OES #914

Merged
merged 14 commits into from Nov 2, 2018

Conversation

Projects
None yet
2 participants
@0v3rt1r3d
Copy link
Contributor

0v3rt1r3d commented Oct 25, 2018

Alexander Gruzintsev
Show resolved Hide resolved OgreMain/include/OgreTexture.h Outdated

Alexander Gruzintsev added some commits Oct 25, 2018

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 25, 2018

Have you any ideas about my problem? The process of drawing from camera is going on in java code using shaders, maybe an error is in common work ogre with https://github.com/google-ar/arcore-android-sdk/blob/master/samples/hello_ar_java/app/src/main/java/com/google/ar/core/examples/java/common/rendering/BackgroundRenderer.java. I removed part below and added textureId as parameter to constructor:

    // Generate the background texture.
    int[] textures = new int[1];
    GLES20.glGenTextures(1, textures, 0);
    textureId = textures[0];
    int textureTarget = GLES11Ext.GL_TEXTURE_EXTERNAL_OES;
    GLES20.glBindTexture(textureTarget, textureId);
    GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
    GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);
@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 25, 2018

this could be a different opengl context from the one ogre is using. I would rather try drawing the texture inside ogre: http://wiki.ogre3d.org/Displaying+2D+Backgrounds

for this you only need to slightly adapt the shaders to the ogre naming conventions: https://ogrecave.github.io/ogre/api/latest/_high-level-_programs.html#Binding-vertex-attributes

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 26, 2018

I am sure that it is correct opengl context. Android has its own render thread with a context (which is used by system and which I could not access directly), and another context is initialized by ogre. When I run the GAPID tool, I can see both contexts, list of calls for each context (and shaders, and textures etc). So I see my shader and draw commands in the ogre context, also I see texture buffer with correct camera image.

External OES texture buffer: https://yadi.sk/i/tJQmOUoxY3D5GA
Texture loaded to render fish model: https://yadi.sk/i/cSb9390dldUs-g

Maybe the render order is important? In java code I call drawing background and then call renderOneFrame()

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 26, 2018

can you also provide a screenshot of the rendering? it was just black background, right?

renderOneFrame() by default clears the framebuffer, so whatever you render first gets lost. (which is another reason to let ogre render the rectangle)

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 26, 2018

Sure, but as I told before, first several frames (one hundred or more) I see another texture (from fish) https://yadi.sk/i/5K3IF2mOPtA9Pg, but then it replaces by black screen https://yadi.sk/i/YOZ1grlnl1zcLw.
How can prevent ogre to clear buffers? I would like to comment some ogre gl-clear calls to achieve the set of instructions in draw method:

  • Clear buffers by myself
  • draw camera image by myself
  • renderOneFrame() without clearing buffers
@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 26, 2018

How can prevent ogre to clear buffers?

https://ogrecave.github.io/ogre/api/latest/class_ogre_1_1_viewport.html#ad9c172c23b0dcccd929f31d4416f9683

it is strange that

  1. it displays anything in the background at all
  2. it uses the wrong texture content.

can you verify whether the id you get from getTextureId is the one you see in GAPID (i.e. 21 and not 22)?

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 26, 2018

Verified, the textureId is 21. Tried to use setClearEveryFrame() and call every frame glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT|GL_STENCIL_BUFFER_BIT) - nothing changes. (But if I do not clear buffers at all - I stop to see fish).

Maybe I have some errors in creating texture code? (looks strange, because it is kotlin)

val material = MaterialManager.getSingleton().create("PlaneMat", ResourceGroupManager.getDEFAULT_RESOURCE_GROUP_NAME())

val textureSmth = material.techniques[0].passes[0].createTextureUnitState()
material.techniques[0].passes[0].apply {
    depthCheckEnabled = false
    depthWriteEnabled = false
    lightingEnabled = false
}

val texture = TextureManager.getSingleton().createManual("SomeTexture", "General", TextureType.TEX_TYPE_EXTERNAL_OES, 400, 400, 0, PixelFormat.PF_R8G8B8)

textureId = texture.textureId // global variable to use in draw-background method, it is wrapped getter to getTextureId() from kotlin

Timber.d("TEXTUREID: got $textureId") // log, which writes 21 (correct texture name)

textureSmth.setTexture(texture)

val rect = Rectangle2D(true)
rect.setCorners(-1.0f, 1.0f, 1.0f, -1.0f)
rect.material = material
rect.renderQueueGroup = OgreJNI.RENDER_QUEUE_BACKGROUND_get().toShort()
val bb = AxisAlignedBox().apply { setInfinite() }
rect.boundingBox = bb

val node = scnMgr.rootSceneNode.createChildSceneNode()
node.attachObject(rect)
@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 26, 2018

Hm. If I create texture from resources with material.techniques[0].passes[0].createTextureUnitState("Dirt.jpg") it still disappear after some some frames rendered

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 26, 2018

so you are rendering the background using ogre now?

anyway in the "PlaneMat" material will not work because it still needs a "samplerExternalOES" fragment shader.

The default generated shader is currently wrong as the RTSS does not now about that kind of samplers:

mGpuConstTypeMap[GCT_SAMPLER1D] = mIsGLSLES ? "sampler2D" : "sampler1D";
mGpuConstTypeMap[GCT_SAMPLER2D] = "sampler2D";
mGpuConstTypeMap[GCT_SAMPLER2DARRAY] = "sampler2DArray";
mGpuConstTypeMap[GCT_SAMPLER3D] = "sampler3D";
mGpuConstTypeMap[GCT_SAMPLERCUBE] = "samplerCube";
mGpuConstTypeMap[GCT_SAMPLER1DSHADOW] = "sampler1DShadow";
mGpuConstTypeMap[GCT_SAMPLER2DSHADOW] = "sampler2DShadow";

if you provide the right shader, ogre will not be able to correctly parse it:

mTypeEnumMap.insert(StringToEnumMap::value_type("sampler2D", GL_SAMPLER_2D));
mTypeEnumMap.insert(StringToEnumMap::value_type("samplerCube", GL_SAMPLER_CUBE));
mTypeEnumMap.insert(StringToEnumMap::value_type("sampler2DShadow", GL_SAMPLER_2D_SHADOW_EXT));

so you will not be able to change the sampler slot from 0, but that should be fine as it only uses one texture

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 26, 2018

it still disappear after some some frames rendered

I might have an idea about this one. Try making this a member variable

val rect = Rectangle2D(true)

rect is the only reference here. It must be kept somewhere so it does not get garbage collected.

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 26, 2018

using "Dirt.jpg" as a background should be a prerequisite before we continue with TEX_TYPE_EXTERNAL_OES and custom shaders

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 26, 2018

Well, after saving rect as a global variable background image stopped to disappear. Now I try to figure out how to make background rendering using ogre

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 29, 2018

I tried to add shader using code below:

val fragmentShader = HighLevelGpuProgramManager.getSingleton().createProgram("fragment", "General", "glsl", GpuProgramType.GPT_FRAGMENT_PROGRAM)
// fragmentShader.sourceFile = "shaders/screenquad.frag" // doesn't work at all
fragmentShader.source = """
#extension GL_OES_EGL_image_external : require

precision mediump float;
varying vec2 v_TexCoord;
uniform samplerExternalOES sTexture;

void main() {
       gl_FragColor = texture2D(sTexture, v_TexCoord);
}
""".trimIndent()
fragmentShader.load()

val material = MaterialManager.getSingleton().create("PlaneMat", "General")
material.techniques[0].passes[0].setFragmentProgram(fragmentShader.name)

Now I see white screen and nothing else (Replaces by fish, if I comment material.techniques[0].passes[0].setFragmentProgram(fragmentShader.name)). I suppose, the problem is that I make some errors, loading shaders. If I call HighLevelGpuProgramPtr.isSupported() I got false, and HighLevelGpuProgramPtr.hasCompileError() also returns false. How can I debug the shader loading?

Also I cannot load shader from file, its source is empty when I try. (I tried put shader to asset directory and get by relative path, copy ogre-sample directories structure materials/programs/GLSL/myshader.frag and get by filename)
I turned off all gl-calls from java and I see the texture with camera image in GAPID.

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 29, 2018

you have to put it into the GLSLES/ subfolder and use "glsles" language identifier.

Furthermore you also must set a vertex shader. Here you can use the upstream one, but you have to rename the attributes to match ogre convetions as:
https://ogrecave.github.io/ogre/api/latest/_high-level-_programs.html#Accessing-OpenGL-states-in-GLSL

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 29, 2018

Yeah!!! Now I can see changing background, but I looks like one pixel is stretched to full screen

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 29, 2018

please share a screenshot and your vertex shader. It sounds like your uv coordinates are all zero.

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 29, 2018

It looks this way https://yadi.sk/i/xkR3YU8KoFB-RA, but background color is always changing and depends on the light in front of device. Vertex shader is this

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 29, 2018

use this instead:

attribute vec4 vertex;
attribute vec2 uv0;

varying vec2 v_TexCoord;

void main() {
   gl_Position = vertex;
   v_TexCoord = uv0;
}
@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 29, 2018

No, this shader didn't help :(

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 29, 2018

look out for shader compilation errors/ warnings in the log. Also try "visual debugging" by writing

gl_FragColor = vec4(v_TexCoord, 1, 1);

in the FS

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 29, 2018

No shader compile errors happened. And "visual debugging" makes the whole screen blue

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 29, 2018

how about

attribute vec4 vertex;
attribute vec2 uv0;

varying vec2 v_TexCoord;

void main() {
   gl_Position = vertex;
   v_TexCoord = vertex.xy;
}

the issue is that v_TexCoord stays at zero (vec4(0, 0, 1, 1)) which is strange as "Dirt.jpg" w/o shaders worked.

Did you by chance remove "true" from Rectangle2D(true)?

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 30, 2018

Yeah, you are right 😅

Did you by chance remove "true" from Rectangle2D(true)?

Now I looks just rotated https://yadi.sk/i/eVP-MCurloyCcw

Update: Yeah! I did it with this shader:

attribute vec4 vertex;
attribute vec2 uv0;

varying vec2 v_TexCoord;

void main() {
   gl_Position = vertex;
   v_TexCoord = vec2(uv0.y, 1.0 - uv0.x);
}

Thank you so much!!! Let us finish the deal with pr to ogre? Now I will update the pr removing my getter. Also I think to add shaders I used. Will you review it?

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 30, 2018

Also I think to add shaders I used. Will you review it?

yes, the shaders should go into https://github.com/OGRECave/ogre/tree/master/Samples/Media/materials/programs/GLSLES

also you should add a test app similar to
https://github.com/OGRECave/ogre/tree/master/Samples/AndroidJNI

Show resolved Hide resolved RenderSystems/GLES2/src/OgreGLES2RenderSystem.cpp Outdated
Show resolved Hide resolved RenderSystems/GLES2/src/OgreGLES2RenderSystem.cpp Outdated
Show resolved Hide resolved RenderSystems/GLES2/src/OgreGLES2Texture.cpp Outdated
@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Oct 30, 2018

I have some challenges with android test app.

  1. As I figured out you have template for AndroidManifest.xml and generate several different projects using it with cmake. And just copy all other sources, resources and compiled library to the target project. Why don't you just create one gradle-project with different activities and put here your compiled .so library?
  2. Maybe I did a mistake, but after building ogre, I didn't find any source files in SampleBrowserNDK project
@paroj

This comment has been minimized.

Copy link
Member

paroj commented Oct 30, 2018

  1. I did not yet come to port the Android build to gradle. Still using ant here. If it makes your life easier, you can also add another activity to the AndroidJNI sample.
    Also feel free to modernize the Android build, but that would be out of scope for this PR.
  2. SampleBrowserNDK is a pure C++ project and does not contain any Java sources. The TEX_TYPE_EXTERNAL_OES sample would not be part of it, but rather standalone - just like the AndroidJNI sample.

Alexander Gruzintsev added some commits Nov 1, 2018

@0v3rt1r3d 0v3rt1r3d force-pushed the 0v3rt1r3d:overtired/attempt-to-add-oes-texture branch from a589e62 to 65d06c9 Nov 1, 2018

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Nov 1, 2018

Got it. Now I created created sample for oes texture, checked local build and try to build it on CI. Do you think it is ok that TEX_TYPE_EXTERNAL_OES is defined in OgreTexture.h, which means it is common for all render system textures (Direct, GL, Gl 3+, etc) but in practice it is available only for GLES2? I was fixing 'TEX_TYPE_EXTERNAL_OES' not handled in switch errors, I figured out, what it would be better not to consider this type in other render systems at all. Can I do it using preprocessor's directives, like #if <> #endif? And if not, should I treat it as empty swtich-case with a comment or throw an error?

Show resolved Hide resolved RenderSystems/GL/src/OgreGLTexture.cpp Outdated
Show resolved Hide resolved RenderSystems/GLES2/src/OgreGLES2Texture.cpp
Show resolved Hide resolved Samples/AndroidTextureOES/MainActivity.java Outdated
Show resolved Hide resolved .vscode/settings.json Outdated
@paroj

This comment has been minimized.

Copy link
Member

paroj commented Nov 1, 2018

Can I do it using preprocessor's directives, like #if <> #endif? And if not, should I treat it as empty swtich-case with a comment or throw an error?

#ifdef is not an option as the idea of OgreMain is that you can write code that will always compile, but only work on certain rendersystems. (just like GL where some features might not be there)

I agree that TEX_TYPE_EXTERNAL_OES is quite ugly when it comes to passing to to code that does not support it. The only alternative I can think of is TextureUsage::TU_EXTERNAL.
However this will get ugly for code that does support it because it does not map well to the GL spec.

So for now I would stick with TEX_TYPE_EXTERNAL_OES and throw on all other rendersystems.
(as in OGRE_EXCEPT(Exception::ERR_RENDERINGAPI_ERROR, "GL_OES_EGL_image_external unsupported", __FUNCTION__)

@paroj

paroj approved these changes Nov 2, 2018

@paroj

paroj approved these changes Nov 2, 2018

@paroj

This comment has been minimized.

Copy link
Member

paroj commented Nov 2, 2018

feel free to remove the WIP from the title if you are finished.

@0v3rt1r3d

This comment has been minimized.

Copy link
Contributor Author

0v3rt1r3d commented Nov 2, 2018

Yes, will it enough to merge pr?

@0v3rt1r3d 0v3rt1r3d changed the title WIP: Added texture type OES Added texture type OES Nov 2, 2018

@paroj paroj merged commit 69556a6 into OGRECave:master Nov 2, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@paroj

This comment has been minimized.

Copy link
Member

paroj commented Nov 2, 2018

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.