DirectX 11 Overlay #28

Closed
wants to merge 30 commits into
from

Conversation

Projects
None yet
7 participants
@Kissaki
Member

Kissaki commented Apr 5, 2011

mumble_ol.dll increased in size, from 127 to 291 KB
new dependencies: MSVCR, D3DCompiler (DLLs)
because dx11 does not include the effects anymore, the DX SDK sample effects has to be used and adapted, for compilation. By default it compiles as a static library, but thus will use libcmt, which will result in a fatal error, duplicate definition of functions. To fix this, the effects lib has to be compiled with the linker settings “ignore specific default library: LIBCMT”, and “additional dependencies: MSVCRT.lib” (added D for the debug versions).

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Apr 5, 2011

Member

Ok, although it worked on a sample, it crashed BFBC2 without notice. Don’t pull yet. :)

Member

Kissaki commented Apr 5, 2011

Ok, although it worked on a sample, it crashed BFBC2 without notice. Don’t pull yet. :)

@hacst

This comment has been minimized.

Show comment
Hide comment
@hacst

hacst Apr 6, 2011

Member

Wasn't the whole reason pcgod never applied this patch that it didn't work on anything but the samples?

In any case. You should override the author field of the first patch instead of putting the the source in the commit message. Much cleaner that way.

Member

hacst commented Apr 6, 2011

Wasn't the whole reason pcgod never applied this patch that it didn't work on anything but the samples?

In any case. You should override the author field of the first patch instead of putting the the source in the commit message. Much cleaner that way.

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Apr 6, 2011

Member

yeah

Member

Kissaki commented Apr 6, 2011

yeah

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Apr 9, 2011

Member

Updated. Does not crash anymore, but does not work yet neither (in dx samples neither anymore).
Still on it.

Member

Kissaki commented Apr 9, 2011

Updated. Does not crash anymore, but does not work yet neither (in dx samples neither anymore).
Still on it.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Sep 29, 2011

overlay11.hex is actually overlay.hex, and your overlay.pro still has a conflict in it.

overlay11.hex is actually overlay.hex, and your overlay.pro still has a conflict in it.

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Oct 1, 2011

Member

rebased
@nyetwurk: yeah, noticed I had uncommited change to use overlay.hex instead of the 11 one.

Member

Kissaki commented Oct 1, 2011

rebased
@nyetwurk: yeah, noticed I had uncommited change to use overlay.hex instead of the 11 one.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 2, 2011

BTW i got your branch compiling, but it crashes with EmptyProject11

nyetwurk commented Oct 2, 2011

BTW i got your branch compiling, but it crashes with EmptyProject11

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 23, 2011

I was able to get this working by using fxc /Tfx_5_0 to generate overlay.hex

/Tfx_4_0 generates a overlay.hex that crashes CreateEffect

I was able to get this working by using fxc /Tfx_5_0 to generate overlay.hex

/Tfx_4_0 generates a overlay.hex that crashes CreateEffect

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 26, 2011

Works in intro screen of bfbc2, but not the game itself (no sign of overlay)
In BF3, overlay is present, but is corrupted. It looks like the mumble overlay dll isn't getting the right screen size for rendering.
deus ex also shows NO sign of overlay at all (intro or otherwise)
dumb question: how do i get the output of the overlay log messages w/o a debugger attached to the game?

Works in intro screen of bfbc2, but not the game itself (no sign of overlay)
In BF3, overlay is present, but is corrupted. It looks like the mumble overlay dll isn't getting the right screen size for rendering.
deus ex also shows NO sign of overlay at all (intro or otherwise)
dumb question: how do i get the output of the overlay log messages w/o a debugger attached to the game?

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Oct 26, 2011

Member

You don’t need to attach a full-fledged debugger.
Attaching windbg just for log-output should be enough.

Member

Kissaki commented Oct 26, 2011

You don’t need to attach a full-fledged debugger.
Attaching windbg just for log-output should be enough.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 28, 2011

Got everything working on BF3.

Will test out a few other dx11 games then post the patches.

Got everything working on BF3.

Will test out a few other dx11 games then post the patches.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 28, 2011

Here are the issues

  1. DX10 CreateEffect needs fx_4_0, but DX11 Create Effect needs fx_5_0.

Unfortunately, fxc outputs a header that generates a global g_main const.
So we have to have fxc output two different overlay include files, using different name spaces.

  1. Most games are multithreaded, and since we dont have StateBlocks in DX11, and you have to use a DeviceContext, you can't use an ImmediateContext, you have to use a DeferredContext. Fortunately, kissaki already provided the code for it, so i just used that instead (including the CommandList code in draw()) , and bingo, everything works.

Question about the Effects library: i noticed your overlay.pro references the local directory and not the dxsdk directory. Should mumble be just copying the Effects source files as is into the repo? are there licensing issues? if there are licensing issues, then more crazy things have to be added to the build instructions.

Here are the issues

  1. DX10 CreateEffect needs fx_4_0, but DX11 Create Effect needs fx_5_0.

Unfortunately, fxc outputs a header that generates a global g_main const.
So we have to have fxc output two different overlay include files, using different name spaces.

  1. Most games are multithreaded, and since we dont have StateBlocks in DX11, and you have to use a DeviceContext, you can't use an ImmediateContext, you have to use a DeferredContext. Fortunately, kissaki already provided the code for it, so i just used that instead (including the CommandList code in draw()) , and bingo, everything works.

Question about the Effects library: i noticed your overlay.pro references the local directory and not the dxsdk directory. Should mumble be just copying the Effects source files as is into the repo? are there licensing issues? if there are licensing issues, then more crazy things have to be added to the build instructions.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 28, 2011

Works in Deus Ex, but not BFBC2.

DeferredContext creation fails.

Works in Deus Ex, but not BFBC2.

DeferredContext creation fails.

@hacst

This comment has been minimized.

Show comment
Hide comment
@hacst

hacst Oct 28, 2011

Member

@nyetwurk: That sounds great. We would love to see patches for DX11 and, given they are stable, will gladly accept them.

Member

hacst commented Oct 28, 2011

@nyetwurk: That sounds great. We would love to see patches for DX11 and, given they are stable, will gladly accept them.

@pcgod

This comment has been minimized.

Show comment
Hide comment
@pcgod

pcgod Oct 28, 2011

Member

We have to patch the Effects source files (I have a modified version somewhere) to avoid depending on a specific DirectX version, but I don't know if we can do that by checking in the Effects source or just a patch which has to be applied to the Effects source in the SDK (which would make the build instructions even more complicated)...

Member

pcgod commented Oct 28, 2011

We have to patch the Effects source files (I have a modified version somewhere) to avoid depending on a specific DirectX version, but I don't know if we can do that by checking in the Effects source or just a patch which has to be applied to the Effects source in the SDK (which would make the build instructions even more complicated)...

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 28, 2011

https://gist.github.com/1323422

Note that this STILL doesn't help bfbc2, but at least it doesn't crash (for me)

https://gist.github.com/1323422

Note that this STILL doesn't help bfbc2, but at least it doesn't crash (for me)

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 29, 2011

I verified that BFBC2 is creating the pDevice with D3D11_CREATE_DEVICE_SINGLETHREADED, so when we get the pDevice using swapchain->GetDevice(), you can't use it to create a deferred context.

Any ideas? Is there another way to create a DeferredContext other than from the SwapChain's pDevice?

I verified that BFBC2 is creating the pDevice with D3D11_CREATE_DEVICE_SINGLETHREADED, so when we get the pDevice using swapchain->GetDevice(), you can't use it to create a deferred context.

Any ideas? Is there another way to create a DeferredContext other than from the SwapChain's pDevice?

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Oct 29, 2011

Member

Awesome to see some progress here.
Stupid that I was checking against BC2 back when I was trying it out. Maybe with a working game my interest would not have dropped as much back then :P

Member

Kissaki commented Oct 29, 2011

Awesome to see some progress here.
Stupid that I was checking against BC2 back when I was trying it out. Maybe with a working game my interest would not have dropped as much back then :P

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 29, 2011

I could use your help; I'm way out of my depth. Is it really necessary to have the pDevice from the swapchain creation around for creating a deferred context?

Can you just make a pDevice not connected to anything to create a DeferredContext, or do you HAVE to use the one you got when you made the swapchain?

I could use your help; I'm way out of my depth. Is it really necessary to have the pDevice from the swapchain creation around for creating a deferred context?

Can you just make a pDevice not connected to anything to create a DeferredContext, or do you HAVE to use the one you got when you made the swapchain?

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Oct 29, 2011

Member

I was more fooling around than doing what I know. Applying the outdated patch to the (then) current codebase and then trying some things out (deferred context) until time ran short again.
You already figured out more than I did it seems.

Member

Kissaki commented Oct 29, 2011

I was more fooling around than doing what I know. Applying the outdated patch to the (then) current codebase and then trying some things out (deferred context) until time ran short again.
You already figured out more than I did it seems.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 29, 2011

Haha ok good. I thought I was the lone noob :) Hopefully pcgod can figure it out, because I just can't figure out how to do it w/o either deferred context or save blocks.

and i hvae NO idea how to do save blocks in dx11

Haha ok good. I thought I was the lone noob :) Hopefully pcgod can figure it out, because I just can't figure out how to do it w/o either deferred context or save blocks.

and i hvae NO idea how to do save blocks in dx11

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Oct 31, 2011

Ok, on further investigation, the ImmediateContext path is definitely VERY VERY bad unless we can save/restore state. Unless we find a way to implement DX11 SaveBlocks, the ImmediateContext code should be removed...

nyetwurk@174d0db

Ok, on further investigation, the ImmediateContext path is definitely VERY VERY bad unless we can save/restore state. Unless we find a way to implement DX11 SaveBlocks, the ImmediateContext code should be removed...

nyetwurk@174d0db

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Nov 1, 2011

Now that I have BFBC2 support working, how do we move forward with the DX11 Effects code?

More junk in the build instructions, or copy DX11 Effects code into the git repo?

nyetwurk commented Nov 1, 2011

Now that I have BFBC2 support working, how do we move forward with the DX11 Effects code?

More junk in the build instructions, or copy DX11 Effects code into the git repo?

@Hadlock

This comment has been minimized.

Show comment
Hide comment
@Hadlock

Hadlock Nov 2, 2011

Any way you can make a version that works with BF3? I don't think BC2 support is a very high priority any more - my BC2 server has been empty since BF3 was released, and my BF3 servers + mumble server are completely full all the time now.

Hadlock commented Nov 2, 2011

Any way you can make a version that works with BF3? I don't think BC2 support is a very high priority any more - my BC2 server has been empty since BF3 was released, and my BF3 servers + mumble server are completely full all the time now.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Nov 2, 2011

The dx11 branch works in bf3.. its just a bit tricky to build.

nyetwurk commented Nov 2, 2011

The dx11 branch works in bf3.. its just a bit tricky to build.

nyetwurk and others added some commits Nov 11, 2011

manually apply patch
From 0ef1320b7b1bc9839774a89df8e3593d8016a398 Mon Sep 17 00:00:00 2001
From: Benjamin Jemlich <pcgod@users.sourceforge.net>
Date: Thu, 19 Aug 2010 00:37:34 +0200
Subject: [PATCH] Overlay DX 11 support
Use deferred context when possible, generate separate overlay g_mains…
… for dx10 and dx11.

Also a few cosmetic changes
include d3dx11effect.h file, for compilation to work
checkDXGI11Hook loads d3d11core.dll instead of d3d11.dll,
d3d11 uses own filename var from d3d10
add a missing x11 hook-check call
d3d11core.dll -> d3d11.dll,

initialize dwBytesWritten with 0 instead of the desired number of bytes
fix logical error without problematic consequences (sh-mem overlay msg cast used from type in union as src),
rm unused mutex var,
endless loop on gl-errors?

more descriptive var names; make it clearer what they and their differences are,
- rename w and h arguments to more descriptive width and height,
more descriptive/verbose logging
- use uiWidth and uiHeight vars for logging, as they are the ones used for msg initialization,
indent with tab instead of space, fix indents,
add descriptive code comments
Complete DX11 support
Don't compile effects11 if effects11.cpp stub isn't there
@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Nov 16, 2011

Member

I squashed the commits and picked nyets changes.
Diffing with his current HEAD, I will comment the lines maybe nyet can check again, as they are different still!?

Member

Kissaki commented Nov 16, 2011

I squashed the commits and picked nyets changes.
Diffing with his current HEAD, I will comment the lines maybe nyet can check again, as they are different still!?

ignore overlay hex file
fix: rm duplicate declaration (squash-issue)
fix indent in overlay.c
@@ -665,8 +683,9 @@ extern "C" BOOL WINAPI DllMain(HINSTANCE, DWORD fdwReason, LPVOID) {
if (!bBlackListed) {
checkD3D9Hook();
checkDXGIHook();
+ checkDXGI11Hook();

This comment has been minimized.

@Kissaki

Kissaki Nov 16, 2011

Member

This line is not in nyets repo.
But it should be, right? Should be checked at the same places as the others. Like we do above at the other places.

@Kissaki

Kissaki Nov 16, 2011

Member

This line is not in nyets repo.
But it should be, right? Should be checked at the same places as the others. Like we do above at the other places.

This comment has been minimized.

@nyetwurk

nyetwurk Nov 16, 2011

Yup. I missed it. Good eye.

@nyetwurk

nyetwurk Nov 16, 2011

Yup. I missed it. Good eye.

This comment has been minimized.

@nyetwurk

nyetwurk Nov 16, 2011

Just pushed it into my branch. Thanks.

@nyetwurk

nyetwurk Nov 16, 2011

Just pushed it into my branch. Thanks.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Nov 16, 2011

All the remaining things seem cosmetic ... perhaps they can be split out into a different commit and/or branch?

All the remaining things seem cosmetic ... perhaps they can be split out into a different commit and/or branch?

@@ -193,7 +193,7 @@ void Pipe::disconnect() {
bool Pipe::sendMessage(const OverlayMsg &om) {
DWORD dwBytesToWrite = sizeof(OverlayMsgHeader) + om.omh.iLength;
- DWORD dwBytesWritten = dwBytesToWrite;
+ DWORD dwBytesWritten = 0;

This comment has been minimized.

@nyetwurk

nyetwurk Nov 16, 2011

This is the only remaining non-cosmetic change... Is it needed?

@nyetwurk

nyetwurk Nov 16, 2011

This is the only remaining non-cosmetic change... Is it needed?

This comment has been minimized.

@Kissaki

Kissaki Nov 17, 2011

Member

not needed, but better style.
As it's a var for checking written bytes. So it should start with 0, and when we later check for completeness the default is false, instead of the other way around.
(false by default, true only after read / adjusting var)

@Kissaki

Kissaki Nov 17, 2011

Member

not needed, but better style.
As it's a var for checking written bytes. So it should start with 0, and when we later check for completeness the default is false, instead of the other way around.
(false by default, true only after read / adjusting var)

winpaths_default.pri
-VLD_PATH = %PROGPATH%/Visual Leak Detector
-BONJOUR_PATH = %PROGRAMFILES%/Bonjour SDK
+VLD_PATH = /dev/vld
+BONJOUR_PATH = /Program Files/Bonjour SDK

This comment has been minimized.

@nyetwurk

nyetwurk Nov 16, 2011

All of this should be in winpaths_custom.pri, not checked in...

@nyetwurk

nyetwurk Nov 16, 2011

All of this should be in winpaths_custom.pri, not checked in...

This comment has been minimized.

@Kissaki

Kissaki Nov 20, 2011

Member

Mh, to also note out why it’s there here:
The wiki build instructions use /dev path, and not the progpath. Thus, I changed this to the default.

@Kissaki

Kissaki Nov 20, 2011

Member

Mh, to also note out why it’s there here:
The wiki build instructions use /dev path, and not the progpath. Thus, I changed this to the default.

This comment has been minimized.

@hacst

hacst Nov 20, 2011

Member

The wiki instructions are never authorative. What's in the repo is. In this case however the wiki instructions, as far as I can see, agree with winpaths_default.pri that currently is in the repo.

In general winpaths_default.pri really should only be changed if it is actually wrong (new protobuf/ice etc.) or the team agrees that certain defaults make more sense. For everything else we have winpaths_custom.pri.

@hacst

hacst Nov 20, 2011

Member

The wiki instructions are never authorative. What's in the repo is. In this case however the wiki instructions, as far as I can see, agree with winpaths_default.pri that currently is in the repo.

In general winpaths_default.pri really should only be changed if it is actually wrong (new protobuf/ice etc.) or the team agrees that certain defaults make more sense. For everything else we have winpaths_custom.pri.

This comment has been minimized.

@Kissaki

Kissaki Nov 20, 2011

Member

Ok, I reverted the changes.

@Kissaki

Kissaki Nov 20, 2011

Member

Ok, I reverted the changes.

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Nov 20, 2011

Member

Today I tried out nyets installer he compiled, and the overlay worked in BC2.
It didn’t work for my self-compile from this branch though; but I couldn’t even install dx SDK …

Member

Kissaki commented Nov 20, 2011

Today I tried out nyets installer he compiled, and the overlay worked in BC2.
It didn’t work for my self-compile from this branch though; but I couldn’t even install dx SDK …

@yackushevas

This comment has been minimized.

Show comment
Hide comment
@yackushevas

yackushevas Jul 28, 2012

Unfortunately the overlay does not work in the game The Secret World (if you use DirectX11).
Forgive me if I am not writing to the theme with the problem

Unfortunately the overlay does not work in the game The Secret World (if you use DirectX11).
Forgive me if I am not writing to the theme with the problem

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Jul 28, 2012

Member

So you compiled this dx11 branch yourself and tried it?

Member

Kissaki commented Jul 28, 2012

So you compiled this dx11 branch yourself and tried it?

@yackushevas

This comment has been minimized.

Show comment
Hide comment
@yackushevas

yackushevas Jul 28, 2012

Unfortunately, I do not know how to do it. Why can not I enter this code into the mainline?

Unfortunately, I do not know how to do it. Why can not I enter this code into the mainline?

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Jul 28, 2012

My precompiled DX11 binaries are here:

https://github.com/nyetwurk/mumble/downloads

My precompiled DX11 binaries are here:

https://github.com/nyetwurk/mumble/downloads

@Jinx1337

This comment has been minimized.

Show comment
Hide comment
@Jinx1337

Jinx1337 Aug 8, 2012

The file is missing?

I can't download it?
Please reupload, I've been googling high and low for this!

Jinx1337 commented Aug 8, 2012

The file is missing?

I can't download it?
Please reupload, I've been googling high and low for this!

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Aug 8, 2012

Re-uploaded. Try now

nyetwurk commented Aug 8, 2012

Re-uploaded. Try now

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Jul 8, 2013

Member

dD0T: Kissaki: http://msdn.microsoft.com/en-us/library/windows/desktop/bb509633%28v=vs.85%29.aspx#compiling_with_d3dcompilefromfile sounds like the right place to start
dD0T: Kissaki: http://gamedev.stackexchange.com/questions/17602/what-happened-to-fx-files-in-d3d11 << Seems helpful too
dD0T: "A pointer to uncompiled shader data; either ASCII HLSL code or a compiled effect." << I guess .fx is a "compiled effect"?
https://blogs.msdn.com/b/chuckw/archive/2012/05/07/hlsl-fxc-and-d3dcompile.aspx?Redirected=true

Member

Kissaki commented Jul 8, 2013

dD0T: Kissaki: http://msdn.microsoft.com/en-us/library/windows/desktop/bb509633%28v=vs.85%29.aspx#compiling_with_d3dcompilefromfile sounds like the right place to start
dD0T: Kissaki: http://gamedev.stackexchange.com/questions/17602/what-happened-to-fx-files-in-d3d11 << Seems helpful too
dD0T: "A pointer to uncompiled shader data; either ASCII HLSL code or a compiled effect." << I guess .fx is a "compiled effect"?
https://blogs.msdn.com/b/chuckw/archive/2012/05/07/hlsl-fxc-and-d3dcompile.aspx?Redirected=true

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Jul 8, 2013

Member

To keep any followers updated: I plan to commit all non-effects11 changes to master soon(ish) (after some other PRs).
Then we have a useful commited baseline.
Then the necessary adjustments can be done to drop effects11.

Member

Kissaki commented Jul 8, 2013

To keep any followers updated: I plan to commit all non-effects11 changes to master soon(ish) (after some other PRs).
Then we have a useful commited baseline.
Then the necessary adjustments can be done to drop effects11.

@nyetwurk

This comment has been minimized.

Show comment
Hide comment
@nyetwurk

nyetwurk Jul 9, 2013

necro pull request! Glad to see progress again. Thanks, Kissaki. Got tired of pulling/merging/rebuilding quite some time ago...

nyetwurk commented Jul 9, 2013

necro pull request! Glad to see progress again. Thanks, Kissaki. Got tired of pulling/merging/rebuilding quite some time ago...

@Kissaki

This comment has been minimized.

Show comment
Hide comment
@Kissaki

Kissaki Oct 6, 2013

Member

Closed in favor of a rebase with selected commits/changes.

Member

Kissaki commented Oct 6, 2013

Closed in favor of a rebase with selected commits/changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment