Can't get effect on render-targets working in Metro and WP8 phone projects.

[bonesoul]

I'm not sure if my code/effect is broken or render-target support is, but still I'm using a sample effect from XNA-catalog, so still wanted report this to get you ideas & feedback.

The effect is from the Non-Realisting Rendering sample - the sketch effect.

First of all here's the screenshot of XNA version demonstrating the effect;

And here's output from MonoGame-Metro counterpart;

Actually the effect file included with XNA sample has multiple effects in it, so I trimmed it down as;

sampler SceneSampler : register(s0) = sampler_state
{
    Texture = SceneTexture;

    MinFilter = Linear;
    MagFilter = Linear;

    AddressU = Clamp;
    AddressV = Clamp;
};

texture SketchTexture;

sampler SketchSampler : register(s2) = sampler_state
{
    Texture = SketchTexture;

    AddressU = Wrap;
    AddressV = Wrap;
};

float4 PixelShaderFunction(float2 texCoord : TEXCOORD0) : COLOR0
{
    float3 scene = tex2D(SceneSampler, texCoord);
    float3 saturatedScene = saturate((scene - SketchThreshold) * 2);       
    float3 sketchPattern = tex2D(SketchSampler, texCoord + SketchJitter);    
    float3 negativeSketch = (1 - saturatedScene) * (1 - sketchPattern);    
    float sketchResult = dot(1 - negativeSketch, SketchBrightness);

    scene *= sketchResult;
    return float4(scene, 1);
}

technique ColorSketch
{
    pass P0
    {
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

For monogame-metro, I instead use;

technique ColorSketch
{
    pass P0
    {
        PixelShader = compile ps_4_0_level_9_1 PixelShaderFunction();
    }
}

And i compile it as;

2MGFX.exePostprocessEffect.Metro.fx PostprocessEffect.Metro.mgfxo /DEBUG /DX11

My basic render-target & effect code;

protected override void Draw(GameTime gameTime)
        {
            GraphicsDevice.SetRenderTarget(sceneRenderTarget);
            GraphicsDevice.Clear(Color.CornflowerBlue);

            this.spriteBatch.Begin();
            //this.spriteBatch.Draw(this._sample,this.GraphicsDevice.PresentationParameters.Bounds,Color.White);
            this.spriteBatch.End();

            GraphicsDevice.SetRenderTarget(null);
            ApplyPostprocess();

            base.Draw(gameTime);
        }

        private void ApplyPostprocess()
        {
            EffectParameterCollection parameters = _postprocessEffect.Parameters;

            // Set effect parameters controlling the pencil sketch effect.
            parameters["SketchThreshold"].SetValue(SketchThreshold);
            parameters["SketchBrightness"].SetValue(SketchBrightness);
            parameters["SketchJitter"].SetValue(_sketchJitter);
            parameters["SketchTexture"].SetValue(_sketchTexture);

            // Activate the appropriate effect technique.
            _postprocessEffect.CurrentTechnique = _postprocessEffect.Techniques["ColorSketch"];

            // Draw a fullscreen sprite to apply the postprocessing effect.
            spriteBatch.Begin(0, BlendState.AlphaBlend, null, null, null, _postprocessEffect);
            spriteBatch.Draw(sceneRenderTarget, Vector2.Zero, Color.White);
            spriteBatch.End();
        }

So chances that either render-target support is broken for WP8/Metro or the effect isn't working all good with MonoGame - or I'm doing something horrible with my code :)

• I've uploaded a sample project that both has XNA and MonoGame-Metro version included for easier testing purposes.
• Note: I'm using develop branch (and also tried the stable 3.0 release).

[dellis1972]

I think you need to add the position parameters to the PixelShaderFunction

PixelShaderFunction(float2 texCoord : TEXCOORD0)

to

PixelShaderFunction(float4 position : SV_Position,
float4 color : COLOR0,
float2 texCoord : TEXCOORD0
)

That should do the trick

Tom told me about this earlier this week :)

Dean

On 21 February 2013 23:12, Hüseyin Uslu notifications@github.com wrote:

I'm not sure if my code/effect is broken or render-target support is, but
still I'm using is from XNA-catalog, so still wanted report this to get you
ideas & feedback.

The effect is from the Non-Realisting Renderinghttp://xbox.create.msdn.com/en-US/education/catalog/sample/nonrealistic_renderingsample - the sketch effect.

First of all here's the screenshot of XNA version demonstrating the effect;

[image: xna]https://f.cloud.github.com/assets/73215/182842/2760922e-7c7a-11e2-902a-a8ae8b314821.png

And here's output from MonoGame-Metro counterpart;

[image: metro]https://f.cloud.github.com/assets/73215/182843/404cff52-7c7a-11e2-9483-96192e14bddf.png

Actually the effect file included with XNA sample has multiple effects in
it, so I trimmed it down as;

sampler SceneSampler : register(s0) = sampler_state
{
Texture = SceneTexture;

MinFilter = Linear;
MagFilter = Linear;

AddressU = Clamp;
AddressV = Clamp;

};

texture SketchTexture;

sampler SketchSampler : register(s2) = sampler_state
{
Texture = SketchTexture;

AddressU = Wrap;
AddressV = Wrap;

};

float4 PixelShaderFunction(float2 texCoord : TEXCOORD0) : COLOR0
{
float3 scene = tex2D(SceneSampler, texCoord);
float3 saturatedScene = saturate((scene - SketchThreshold) * 2);
float3 sketchPattern = tex2D(SketchSampler, texCoord + SketchJitter);
float3 negativeSketch = (1 - saturatedScene) * (1 - sketchPattern);
float sketchResult = dot(1 - negativeSketch, SketchBrightness);

scene *= sketchResult;
return float4(scene, 1);

}

technique ColorSketch
{
pass P0
{
PixelShader = compile ps_2_0 PixelShaderFunction();
}
}

For monogame-metro, I instead use;

technique ColorSketch
{
pass P0
{
PixelShader = compile ps_4_0_level_9_1 PixelShaderFunction();
}
}

And i compile it as;

2MGFX.exePostprocessEffect.Metro.fx PostprocessEffect.Metro.mgfxo /DEBUG /DX11

My basic render-target & effect code;

protected override void Draw(GameTime gameTime)
{
GraphicsDevice.SetRenderTarget(sceneRenderTarget);
GraphicsDevice.Clear(Color.CornflowerBlue);

        this.spriteBatch.Begin();
        //this.spriteBatch.Draw(this._sample,this.GraphicsDevice.PresentationParameters.Bounds,Color.White);
        this.spriteBatch.End();

        GraphicsDevice.SetRenderTarget(null);
        ApplyPostprocess();

        base.Draw(gameTime);
    }

    private void ApplyPostprocess()
    {
        EffectParameterCollection parameters = _postprocessEffect.Parameters;

        // Set effect parameters controlling the pencil sketch effect.
        parameters["SketchThreshold"].SetValue(SketchThreshold);
        parameters["SketchBrightness"].SetValue(SketchBrightness);
        parameters["SketchJitter"].SetValue(_sketchJitter);
        parameters["SketchTexture"].SetValue(_sketchTexture);

        // Activate the appropriate effect technique.
        _postprocessEffect.CurrentTechnique = _postprocessEffect.Techniques["ColorSketch"];

        // Draw a fullscreen sprite to apply the postprocessing effect.
        spriteBatch.Begin(0, BlendState.AlphaBlend, null, null, null, _postprocessEffect);
        spriteBatch.Draw(sceneRenderTarget, Vector2.Zero, Color.White);
        spriteBatch.End();
    }

So chances that either render-target support is broken for WP8/Metro or
the effect isn't working all good with MonoGame.

• I've uploaded a sample projecthttp://dl.dropbox.com/u/197820/monogame/SketchSample.rarthat both has XNA and MonoGame-Metro version included for easier testing
  purposes.
• Note: I'm using develop branch (and also tried the stable 3.0
  release).

Any ideas if I'm taking the wrong route with my code/effect is broken, or
something isn't working good within monogame? (didn't have time to check it
myself yet).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1436.

[bonesoul]

@dellis1972 I updated my PixelShaderFunction as your suggestion;

float4 PixelShaderFunction(float4 position : SV_Position, float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0
{
    float3 scene = tex2D(SceneSampler, texCoord);
    float3 saturatedScene = saturate((scene - SketchThreshold) * 2);       
    float3 sketchPattern = tex2D(SketchSampler, texCoord + SketchJitter);    
    float3 negativeSketch = (1 - saturatedScene) * (1 - sketchPattern);    
    float sketchResult = dot(1 - negativeSketch, SketchBrightness);

    scene *= sketchResult;
    return float4(scene, 1);
}

But no luck yet, I still get those erroneous diagonal lines;

@tomspilman Any ideas if my shader is broken?

[tomspilman]

Hard to tell... it looks like you have a clamp texture sampler set to me.

[bonesoul]

Yes, it seems it's all about the shader and render-targets for wp8 and metro just works fine.

I tracked down the problem a bit i guess;

So basically I've pre-rendered sketch texture and effect simply displaces it by an offset (which is randomized frequently) which creates the animated sketch.

    float3 sketchPattern = tex2D(SketchSampler, texCoord + SketchJitter);
---
sampler SketchSampler : register(s2) = sampler_state
{
    Texture = SketchTexture;

    AddressU = Wrap;
    AddressV = Wrap;
};

So from what I understand, although it's set in wrap mode, the sampler doesn't wrap the sketch texture correctly.

I can reproduce the same glitch with XNA based version when I set it to clamp mode explicitliy;

sampler SketchSampler : register(s2) = sampler_state
{
    Texture = SketchTexture;

    AddressU = Clamp;
    AddressV = Clamp;
};

So as a guess-work, 2MGX compiled shader somehow doesn't correctly read Wrap mode and somehow defaults to Clamp mode.

[tomspilman]

Oh... are you using the latest 2MGFX from develop?

There was a bug fixed... #1347

[bonesoul]

Yep, actually had an old develop3d build of 2MGFX -- If I understand correctly, #1347 isn't pull backed to develop yet, so I'll give a try to fixed version in develop3d.

[tomspilman]

If I understand correctly, #1347 isn't pull backed to develop yet

It is in develop... it was merged over already.

[bonesoul]

I discovered a bug in ShaderInfo.cs,

        public TextureAddressMode AddressW
        {
            set
            {
                if (_addressW == value)
                    return;
                _addressW = value;
                _dirty = true;
            }
        }

So if user shader exactly replicates a sampler with exact defaults (as in my case), _dirty will be never set to true which will eventually prevent UpdateSamplerState() getting called -- that means _state staying equal to null. At the end sampler will not even written to compiled output.

[tomspilman]

Hum... you might be right.

The goal was to be sure that if the user didn't specify any state in the sampler to not embed a sampler in the Effect which would then overwrite the sampler state set in game code.

If you set sampler state, but it just happens to be the same as the default... that should still embed sampler state into the Effect. You want the default to overwrite any sampler state set in game code.

You should be able to work around this for now by setting the value twice. I'll get a pull request up to fix this over the weekend.

[bonesoul]

Actually I'm just okay with setting the value twice or whatever other trick to overcome this within my game.

But this is actually something that fights against for XNA 4.0 compability - as I'm already using an XNA catalog provided effect file which XNA seems to be happy to use.

So that was the reason I was tracking it down and seems I hunted a rare bug there :)

Thanks @tomspilman for helping me down the way.

[tomspilman]

We know this isn't how it should work... as we find incompatibilities and bugs we fix them. :)

[bonesoul]

Yes, actually this is not a bug but more of an incompatibility. Tomorrow I'll check if I can come with a fix myself -- if can't will wait for yours :)

[bonesoul]

I further fixed my problem;

sampler SketchSampler : register(s2) = sampler_state
{
    Texture = SketchTexture;

    AddressU = Wrap;
    AddressV = Wrap;
};

My sketch texture sampler was registering for second register, changing it to fixed it for me.

sampler SketchSampler : register(s1) = sampler_state

[tomspilman]

This issue resolved now?