Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

capture/replay: implement infrastructure for capture #4372

Merged
merged 5 commits into from
Jun 13, 2024
Merged

capture/replay: implement infrastructure for capture #4372

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d615707
Define api call ID for each being captured methods
kaizhangNV May 17, 2024
3135f89
Add parameter encoder interface
kaizhangNV May 31, 2024
a42a349
Add outputStream and capture manager
kaizhangNV Jun 13, 2024
87c3aac
Fix some Windows & cmake build error
kaizhangNV Jun 13, 2024
19a540d
remove unused headers
kaizhangNV Jun 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions build/visual-studio/slang/slang.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,11 @@ IF EXIST ..\..\..\external\slang-glslang\bin\windows-aarch64\release\slang-glsla
</ItemDefinitionGroup>
<ItemGroup>
<ClInclude Include="..\..\..\slang.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\api_callId.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\capture-manager.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\capture_utility.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\output-stream.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\parameter-encoder.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\slang-composite-component-type.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\slang-entrypoint.h" />
<ClInclude Include="..\..\..\source\slang-capture-replay\slang-filesystem.h" />
Expand Down Expand Up @@ -546,7 +550,10 @@ IF EXIST ..\..\..\external\slang-glslang\bin\windows-aarch64\release\slang-glsla
<ClCompile Include="..\..\..\prelude\slang-cuda-prelude.h.cpp" />
<ClCompile Include="..\..\..\prelude\slang-hlsl-prelude.h.cpp" />
<ClCompile Include="..\..\..\prelude\slang-torch-prelude.h.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\capture-manager.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\capture_utility.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\output-stream.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\parameter-encoder.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\slang-composite-component-type.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\slang-entrypoint.cpp" />
<ClCompile Include="..\..\..\source\slang-capture-replay\slang-filesystem.cpp" />
Expand Down
21 changes: 21 additions & 0 deletions build/visual-studio/slang/slang.vcxproj.filters
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,21 @@
<ClInclude Include="..\..\..\slang.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\source\slang-capture-replay\api_callId.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\source\slang-capture-replay\capture-manager.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\source\slang-capture-replay\capture_utility.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\source\slang-capture-replay\output-stream.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\source\slang-capture-replay\parameter-encoder.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\source\slang-capture-replay\slang-composite-component-type.h">
<Filter>Header Files</Filter>
</ClInclude>
Expand Down Expand Up @@ -722,9 +734,18 @@
<ClCompile Include="..\..\..\prelude\slang-torch-prelude.h.cpp">
<Filter>Header Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\source\slang-capture-replay\capture-manager.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\source\slang-capture-replay\capture_utility.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\source\slang-capture-replay\output-stream.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\source\slang-capture-replay\parameter-encoder.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\source\slang-capture-replay\slang-composite-component-type.cpp">
<Filter>Source Files</Filter>
</ClCompile>
Expand Down
5 changes: 4 additions & 1 deletion source/core/slang-stream.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,10 @@ class OwnedMemoryStream : public MemoryStreamBase
void setContent(const void* contents, size_t contentsSize)
{
m_ownedContents.setCount(contentsSize);
::memcpy(m_ownedContents.getBuffer(), contents, contentsSize);
if (contentsSize > 0)
{
::memcpy(m_ownedContents.getBuffer(), contents, contentsSize);
}
jkwak-work marked this conversation as resolved.
Show resolved Hide resolved
_setContents(m_ownedContents.getBuffer(), m_ownedContents.getCount());
}

Expand Down
147 changes: 147 additions & 0 deletions source/slang-capture-replay/api_callId.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
#ifndef API_CALL_ID_H
#define API_CALL_ID_H

#include <cstdint>

namespace SlangCapture
{
constexpr uint32_t makeApiCallId(uint16_t classId, uint16_t memberFunctionId)
{
return ((static_cast<uint32_t>(classId) << 16) & 0xffff0000) | (static_cast<uint32_t>(memberFunctionId) & 0x0000ffff);
}

constexpr uint16_t getClassId(uint32_t callId)
{
return static_cast<uint16_t>((callId >> 16) & 0x0000ffff);
}

constexpr uint16_t getMemberFunctionId(uint32_t callId)
{
return static_cast<uint16_t>(callId & 0x0000ffff);
}

enum ApiClassId : uint16_t
{
GlobalFunction = 1,
Class_IGlobalSession = 2,
Class_ISession = 3,
Class_IModule = 4,
Class_IEntryPoint = 5,
Class_ICompositeComponentType = 6,
Class_ITypeConformance = 7,
};

typedef uint64_t AddressFormat;

constexpr uint64_t g_globalFunctionHandle = 0;

enum ApiCallId : uint32_t
{
InvalidCallId = 0x00000000,
ICreateGlobalSession = makeApiCallId(GlobalFunction, 0x0000),
IGlobalSession_createSession = makeApiCallId(Class_IGlobalSession, 0x0001),
IGlobalSession_findProfile = makeApiCallId(Class_IGlobalSession, 0x0002),
IGlobalSession_setDownstreamCompilerPath = makeApiCallId(Class_IGlobalSession, 0x0003),
IGlobalSession_setDownstreamCompilerPrelude = makeApiCallId(Class_IGlobalSession, 0x0004),
IGlobalSession_getDownstreamCompilerPrelude = makeApiCallId(Class_IGlobalSession, 0x0005),
IGlobalSession_getBuildTagString = makeApiCallId(Class_IGlobalSession, 0x0006),
IGlobalSession_setDefaultDownstreamCompiler = makeApiCallId(Class_IGlobalSession, 0x0007),
IGlobalSession_getDefaultDownstreamCompiler = makeApiCallId(Class_IGlobalSession, 0x0008),
IGlobalSession_setLanguagePrelude = makeApiCallId(Class_IGlobalSession, 0x0009),
IGlobalSession_getLanguagePrelude = makeApiCallId(Class_IGlobalSession, 0x000A),
IGlobalSession_createCompileRequest = makeApiCallId(Class_IGlobalSession, 0x000B),
IGlobalSession_addBuiltins = makeApiCallId(Class_IGlobalSession, 0x000C),
IGlobalSession_setSharedLibraryLoader = makeApiCallId(Class_IGlobalSession, 0x000D),
IGlobalSession_getSharedLibraryLoader = makeApiCallId(Class_IGlobalSession, 0x000E),
IGlobalSession_checkCompileTargetSupport = makeApiCallId(Class_IGlobalSession, 0x000F),
IGlobalSession_checkPassThroughSupport = makeApiCallId(Class_IGlobalSession, 0x0010),
IGlobalSession_compileStdLib = makeApiCallId(Class_IGlobalSession, 0x0011),
IGlobalSession_loadStdLib = makeApiCallId(Class_IGlobalSession, 0x0012),
IGlobalSession_saveStdLib = makeApiCallId(Class_IGlobalSession, 0x0013),
IGlobalSession_findCapability = makeApiCallId(Class_IGlobalSession, 0x0014),
IGlobalSession_setDownstreamCompilerForTransition = makeApiCallId(Class_IGlobalSession, 0x0015),
IGlobalSession_getDownstreamCompilerForTransition = makeApiCallId(Class_IGlobalSession, 0x0016),
IGlobalSession_getCompilerElapsedTime = makeApiCallId(Class_IGlobalSession, 0x0017),
IGlobalSession_setSPIRVCoreGrammar = makeApiCallId(Class_IGlobalSession, 0x0018),
IGlobalSession_parseCommandLineArguments = makeApiCallId(Class_IGlobalSession, 0x0019),
IGlobalSession_getSessionDescDigest = makeApiCallId(Class_IGlobalSession, 0x001A),

ISession_getGlobalSession = makeApiCallId(Class_ISession, 0x0001),
ISession_loadModule = makeApiCallId(Class_ISession, 0x0002),
ISession_loadModuleFromBlob = makeApiCallId(Class_ISession, 0x0003),
ISession_loadModuleFromIRBlob = makeApiCallId(Class_ISession, 0x0004),
ISession_loadModuleFromSource = makeApiCallId(Class_ISession, 0x0005),
ISession_loadModuleFromSourceString = makeApiCallId(Class_ISession, 0x0006),
ISession_createCompositeComponentType = makeApiCallId(Class_ISession, 0x0007),
ISession_specializeType = makeApiCallId(Class_ISession, 0x0008),
ISession_getTypeLayout = makeApiCallId(Class_ISession, 0x0009),
ISession_getContainerType = makeApiCallId(Class_ISession, 0x000A),
ISession_getDynamicType = makeApiCallId(Class_ISession, 0x000B),
ISession_getTypeRTTIMangledName = makeApiCallId(Class_ISession, 0x000C),
ISession_getTypeConformanceWitnessMangledName = makeApiCallId(Class_ISession, 0x000D),
ISession_getTypeConformanceWitnessSequentialID = makeApiCallId(Class_ISession, 0x000E),
ISession_createTypeConformanceComponentType = makeApiCallId(Class_ISession, 0x000F),
ISession_createCompileRequest = makeApiCallId(Class_ISession, 0x0010),
ISession_getLoadedModuleCount = makeApiCallId(Class_ISession, 0x0011),
ISession_getLoadedModule = makeApiCallId(Class_ISession, 0x0012),
ISession_isBinaryModuleUpToDate = makeApiCallId(Class_ISession, 0x0013),

IModule_findEntryPointByName = makeApiCallId(Class_IModule, 0x0001),
IModule_getDefinedEntryPointCount = makeApiCallId(Class_IModule, 0x0002),
IModule_getDefinedEntryPoint = makeApiCallId(Class_IModule, 0x0003),
IModule_serialize = makeApiCallId(Class_IModule, 0x0004),
IModule_writeToFile = makeApiCallId(Class_IModule, 0x0005),
IModule_getName = makeApiCallId(Class_IModule, 0x0006),
IModule_getFilePath = makeApiCallId(Class_IModule, 0x0007),
IModule_getUniqueIdentity = makeApiCallId(Class_IModule, 0x0008),
IModule_findAndCheckEntryPoint = makeApiCallId(Class_IModule, 0x0009),
IModule_getSession = makeApiCallId(Class_IModule, 0x000A),
IModule_getLayout = makeApiCallId(Class_IModule, 0x000B),
IModule_getSpecializationParamCount = makeApiCallId(Class_IModule, 0x000C),
IModule_getEntryPointCode = makeApiCallId(Class_IModule, 0x000D),
IModule_getResultAsFileSystem = makeApiCallId(Class_IModule, 0x000E),
IModule_getEntryPointHash = makeApiCallId(Class_IModule, 0x000F),
IModule_specialize = makeApiCallId(Class_IModule, 0x0010),
IModule_link = makeApiCallId(Class_IModule, 0x0011),
IModule_getEntryPointHostCallable = makeApiCallId(Class_IModule, 0x0012),
IModule_renameEntryPoint = makeApiCallId(Class_IModule, 0x0013),
IModule_linkWithOptions = makeApiCallId(Class_IModule, 0x0014),

IEntryPoint_getSession = makeApiCallId(Class_IEntryPoint, 0x0001),
IEntryPoint_getLayout = makeApiCallId(Class_IEntryPoint, 0x0002),
IEntryPoint_getSpecializationParamCount = makeApiCallId(Class_IEntryPoint, 0x0003),
IEntryPoint_getEntryPointCode = makeApiCallId(Class_IEntryPoint, 0x0004),
IEntryPoint_getResultAsFileSystem = makeApiCallId(Class_IEntryPoint, 0x0005),
IEntryPoint_getEntryPointHash = makeApiCallId(Class_IEntryPoint, 0x0006),
IEntryPoint_specialize = makeApiCallId(Class_IEntryPoint, 0x0007),
IEntryPoint_link = makeApiCallId(Class_IEntryPoint, 0x0008),
IEntryPoint_getEntryPointHostCallable = makeApiCallId(Class_IEntryPoint, 0x0009),
IEntryPoint_renameEntryPoint = makeApiCallId(Class_IEntryPoint, 0x000A),
IEntryPoint_linkWithOptions = makeApiCallId(Class_IEntryPoint, 0x000B),

ICompositeComponentType_getSession = makeApiCallId(Class_ICompositeComponentType, 0x0001),
ICompositeComponentType_getLayout = makeApiCallId(Class_ICompositeComponentType, 0x0002),
ICompositeComponentType_getSpecializationParamCount = makeApiCallId(Class_ICompositeComponentType, 0x0003),
ICompositeComponentType_getEntryPointCode = makeApiCallId(Class_ICompositeComponentType, 0x0004),
ICompositeComponentType_getResultAsFileSystem = makeApiCallId(Class_ICompositeComponentType, 0x0005),
ICompositeComponentType_getEntryPointHash = makeApiCallId(Class_ICompositeComponentType, 0x0006),
ICompositeComponentType_specialize = makeApiCallId(Class_ICompositeComponentType, 0x0007),
ICompositeComponentType_link = makeApiCallId(Class_ICompositeComponentType, 0x0008),
ICompositeComponentType_getEntryPointHostCallable = makeApiCallId(Class_ICompositeComponentType, 0x0009),
ICompositeComponentType_renameEntryPoint = makeApiCallId(Class_ICompositeComponentType, 0x000A),
ICompositeComponentType_linkWithOptions = makeApiCallId(Class_ICompositeComponentType, 0x000B),

ITypeConformance_getSession = makeApiCallId(Class_ITypeConformance, 0x0001),
ITypeConformance_getLayout = makeApiCallId(Class_ITypeConformance, 0x0002),
ITypeConformance_getSpecializationParamCount = makeApiCallId(Class_ITypeConformance, 0x0003),
ITypeConformance_getEntryPointCode = makeApiCallId(Class_ITypeConformance, 0x0004),
ITypeConformance_getResultAsFileSystem = makeApiCallId(Class_ITypeConformance, 0x0005),
ITypeConformance_getEntryPointHash = makeApiCallId(Class_ITypeConformance, 0x0006),
ITypeConformance_specialize = makeApiCallId(Class_ITypeConformance, 0x0007),
ITypeConformance_link = makeApiCallId(Class_ITypeConformance, 0x0008),
ITypeConformance_getEntryPointHostCallable = makeApiCallId(Class_ITypeConformance, 0x0009),
ITypeConformance_renameEntryPoint = makeApiCallId(Class_ITypeConformance, 0x000A),
ITypeConformance_linkWithOptions = makeApiCallId(Class_ITypeConformance, 0x000B)
};
}
#endif
53 changes: 53 additions & 0 deletions source/slang-capture-replay/capture-manager.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@

#include <string>
#include <sstream>
#include <thread>
#include "capture-manager.h"

namespace SlangCapture
{
CaptureManager::CaptureManager(uint64_t globalSessionHandle)
: m_encoder(&m_memoryStream)
{
std::stringstream ss;
ss << "gs-"<< globalSessionHandle <<"t-"<<std::this_thread::get_id() << ".cap";
m_fileStream = std::make_unique<FileOutputStream>(ss.str());
jkwak-work marked this conversation as resolved.
Show resolved Hide resolved
}

void CaptureManager::clearWithHeader(const ApiCallId& callId, uint64_t handleId)
{
m_memoryStream.flush();
FunctionHeader header {};
header.callId = callId;
header.handleId = handleId;

// write header to memory stream
m_memoryStream.write(&header, sizeof(FunctionHeader));
}

ParameterEncoder* CaptureManager::beginMethodCapture(const ApiCallId& callId, uint64_t handleId)
{
clearWithHeader(callId, handleId);
return &m_encoder;
}

void CaptureManager::endMethodCapture()
{
FunctionHeader* pHeader = const_cast<FunctionHeader*>(
reinterpret_cast<const FunctionHeader*>(m_memoryStream.getData()));

pHeader->dataSizeInBytes = m_memoryStream.getSizeInBytes() - sizeof(FunctionHeader);

std::hash<std::thread::id> hasher;
pHeader->threadId = hasher(std::this_thread::get_id());

// write capture data to file
m_fileStream->write(m_memoryStream.getData(), m_memoryStream.getSizeInBytes());

// take effect of the write
m_fileStream->flush();

// clear the memory stream
m_memoryStream.flush();
}
}
30 changes: 30 additions & 0 deletions source/slang-capture-replay/capture-manager.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#ifndef CAPTURE_MANAGER_H
#define CAPTURE_MANAGER_H

#include "parameter-encoder.h"
#include "api_callId.h"
namespace SlangCapture
{
class CaptureManager
{
public:
CaptureManager(uint64_t globalSessionHandle);
ParameterEncoder* beginMethodCapture(const ApiCallId& callId, uint64_t handleId);
void endMethodCapture();
private:
void clearWithHeader(const ApiCallId& callId, uint64_t handleId);

struct FunctionHeader
{
ApiCallId callId {InvalidCallId};
uint64_t handleId {0};
uint64_t dataSizeInBytes {0};
uint64_t threadId {0};
};

MemoryStream m_memoryStream;
std::unique_ptr<FileOutputStream> m_fileStream;
ParameterEncoder m_encoder;
};
} // namespace SlangCapture
#endif // CAPTURE_MANAGER_H
52 changes: 52 additions & 0 deletions source/slang-capture-replay/output-stream.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#include "output-stream.h"
#include "capture_utility.h"

namespace SlangCapture
{
FileOutputStream::FileOutputStream(const std::string& filename, bool append)
{
Slang::String path(filename.c_str());
Slang::FileMode fileMode = append ? Slang::FileMode::Append : Slang::FileMode::Create;
Slang::FileAccess fileAccess = Slang::FileAccess::Write;
Slang::FileShare fileShare = Slang::FileShare::None;

SlangResult res = m_fileStream.init(path, fileMode, fileAccess, fileShare);

if (res != SLANG_OK)
{
SlangCapture::slangCaptureLog(SlangCapture::LogLevel::Error, "Failed to open file %s\n", filename.c_str());
std::abort();
}
}

FileOutputStream::~FileOutputStream()
{
m_fileStream.close();
}

void FileOutputStream::write(const void* data, size_t len)
{
SLANG_CAPTURE_ASSERT(m_fileStream.write(data, len));
jkwak-work marked this conversation as resolved.
Show resolved Hide resolved
}

MemoryStream::MemoryStream()
: m_memoryStream(Slang::FileAccess::Write)
{ }

void FileOutputStream::flush()
{
SLANG_CAPTURE_ASSERT(m_fileStream.flush());
}

void MemoryStream::write(const void* data, size_t len)
{
SLANG_CAPTURE_ASSERT(m_memoryStream.write(data, len));
}

void MemoryStream::flush()
{
// This call will reset the underlying buffer to size 0,
// and reset the write position to 0.
m_memoryStream.setContent(nullptr, 0);
}
}
Loading
Loading