Firstly you need to locate the directory (or directories) where the VC++ compiler binaries are located and put copies of the appropriate yasm.exe binary in these directories. Windows Yasm executable binaries that are not named
yasm.exe will need to be renamed
yasm.exe after being placed in the appropriate directories.
On a win32 system the win32 version of Yasm has to be used. On an x64 system either the 32 or the 64 bit versions can be used but the rules file is set up to use the 32 bit version. The win32 Yasm binary should be placed in the 32-bit VC++ binary directory, which is typically located at:
Program Files (x86)\Microsoft Visual Studio 8\VC\bin
If needed the 64-bit Yasm binary should be placed in the 64-bit tools binary directory, which is typically at:
Program Files\Microsoft Visual Studio 8\VC\bin
To use the custom tools facility in Visual Studio 2005, you need to place a copy of the yasm.rules file in the Visual Studio 2005 VC project defaults directory, which is typically located at:
Program Files (x86)\Microsoft Visual Studio 8\VC\VCProjectDefaults
This allows you to configure Yasm as an assembler within the VC++ IDE.
To use Yasm in a project, right click on the project in the Solution Explorer and select "Custom Build Rules..". This will give you a dialog box that allows you to select Yasm as an assembler (note that your assembler files need to have the extension '.asm').
To assemble a file with Yasm, select the Property Page for the file and the select "Yasm Assembler" in the Tool dialog entry. Then click "Apply" and an additional property page entry will appear and enable Yasm settings to be established.
As alternative to placing the yasm.rules files as described above is to set the rules file path in the Visual Studio 2005 settings dialogue.
It is also important to note that the rules file passes the symbols
x64 to Yasm by using the Visual Studio 2005
$(PlatformName) macro in order to obtain either a 32 or a 64 bit assembler mode. This is a recent enhancement to YASM so you will need to be sure that you have Yasm version 0.5.0 or higher to use this facility.
There appears to be a linker bug in the VC++ v8 linker that prevents symbols with absolute addresses being linked in DLL builds. This means, for example, that LEA instructions of the general form:
cannot be used for DLL builds. The following general form has to be used instead:
lea rcx,[symbol wrt rip] lea rax,[rax+rcx]
This limitation may also cause problems with other instructions that use absolute addresses.