Skip to content

Latest commit

 

History

History
335 lines (268 loc) · 15.1 KB

ProgrammingGuideDLSS.md

File metadata and controls

335 lines (268 loc) · 15.1 KB
Raw

Streamline - DLSS

The focus of this guide is on using Streamline to integrate DLSS into an application. For more information about DLSS itself, please visit the NVIDIA Developer DLSS Page For information on user interface considerations when using the DLSS plugin, please see the "RTX UI Developer Guidelines.pdf" document included with this SDK.

Version 2.4.0

1.0 INITIALIZE AND SHUTDOWN

Call slInit as early as possible (before any dxgi/d3d11/d3d12 APIs are invoked)

#include <sl.h>
#include <sl_consts.h>
#include <sl_dlss.h>

sl::Preferences pref{};
pref.showConsole = true; // for debugging, set to false in production
pref.logLevel = sl::eLogLevelDefault;
pref.pathsToPlugins = {}; // change this if Streamline plugins are not located next to the executable
pref.numPathsToPlugins = 0; // change this if Streamline plugins are not located next to the executable
pref.pathToLogsAndData = {}; // change this to enable logging to a file
pref.logMessageCallback = myLogMessageCallback; // highly recommended to track warning/error messages in your callback
pref.applicationId = myId; // Provided by NVDA, required if using NGX components (DLSS 2/3)
pref.engineType = myEngine; // If using UE or Unity
pref.engineVersion = myEngineVersion; // Optional version
pref.projectId = myProjectId; // Optional project id
if(SL_FAILED(res, slInit(pref)))
{
    // Handle error, check the logs
    if(res == sl::Result::eErrorDriverOutOfDate) { /* inform user */}
    // and so on ...
}

For more details please see preferences

Call slShutdown() before destroying dxgi/d3d11/d3d12/vk instances, devices and other components in your engine.

if(SL_FAILED(res, slShutdown()))
{
    // Handle error, check the logs
}

1.1 SET THE CORRECT DEVICE

Once the main device is created call slSetD3DDevice or slSetVulkanInfo:

if(SL_FAILED(res, slSetD3DDevice(nativeD3DDevice)))
{
    // Handle error, check the logs
}

2.0 CHECK IF DLSS IS SUPPORTED

As soon as SL is initialized, you can check if DLSS is available for the specific adapter you want to use:

Microsoft::WRL::ComPtr<IDXGIFactory> factory;
if (SUCCEEDED(CreateDXGIFactory(__uuidof(IDXGIFactory), (void**)&factory)))
{
    Microsoft::WRL::ComPtr<IDXGIAdapter> adapter{};
    uint32_t i = 0;
    while (factory->EnumAdapters(i, &adapter) != DXGI_ERROR_NOT_FOUND)
    {
        DXGI_ADAPTER_DESC desc{};
        if (SUCCEEDED(adapter->GetDesc(&desc)))
        {
            sl::AdapterInfo adapterInfo{};
            adapterInfo.deviceLUID = (uint8_t*)&desc.AdapterLuid;
            adapterInfo.deviceLUIDSizeInBytes = sizeof(LUID);
            if (SL_FAILED(result, slIsFeatureSupported(sl::kFeatureDLSS, adapterInfo)))
            {
                // Requested feature is not supported on the system, fallback to the default method
                switch (result)
                {
                    case sl::Result::eErrorOSOutOfDate:         // inform user to update OS
                    case sl::Result::eErrorDriverOutOfDate:     // inform user to update driver
                    case sl::Result::eErrorNoSupportedAdapter:  // cannot use this adapter (older or non-NVDA GPU etc)
                    // and so on ...
                };
            }
            else
            {
                // Feature is supported on this adapter!
            }
        }
        i++;
    }
}

3.0 CHECK DLSS SETTINGS AND SETUP VIEWPORT RENDERING SIZE

Next, we need to find out the rendering resolution and the optimal sharpness level based on DLSS settings:

// Using helpers from sl_dlss.h

sl::DLSSOptimalSettings dlssSettings;
sl::DLSSOptions dlssOptions;
// These are populated based on user selection in the UI
dlssOptions.mode = myUI->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
dlssOptions.outputWidth = myUI->getOutputWidth();    // e.g 1920;
dlssOptions.outputHeight = myUI->getOutputHeight(); // e.g. 1080;
// Now let's check what should our rendering resolution be
if(SL_FAILED(result, slDLSSGetOptimalSettings(dlssOptions, dlssSettings))
{
    // Handle error here
}
// Setup rendering based on the provided values in the sl::DLSSSettings structure
myViewport->setSize(dlssSettings.renderWidth, dlssSettings.renderHeight);

Note that the structure sl::DLSSOptimalSettings will upon return from slDLSSGetOptimalSettings contain information pertinent to DLSS dynamic resolution min and max source image sizes (if dynamic resolution is supported).

4.0 TAG ALL REQUIRED RESOURCES

DLSS requires depth, motion vectors, render-res input color and final-res output color buffers.

// IMPORTANT: Make sure to mark resources which can be deleted or reused for other purposes within a frame as volatile

// Prepare resources (assuming d3d11/d3d12 integration so leaving Vulkan view and device memory as null pointers)
sl::Resource colorIn = {sl::ResourceType::Tex2d, myTAAUInput, nullptr, nullptr, nullptr};
sl::Resource colorOut = {sl::ResourceType::Tex2d, myTAAUOutput, nullptr, nullptr, nullptr};
sl::Resource depth = {sl::ResourceType::Tex2d, myDepthBuffer, nullptr, nullptr, nullptr};
sl::Resource mvec = {sl::ResourceType::Tex2d, myMotionVectorsBuffer, nullptr, nullptr, nullptr};
sl::Resource exposure = {sl::ResourceType::Tex2d, myExposureBuffer, nullptr, nullptr, nullptr};

sl::ResourceTag colorInTag = sl::ResourceTag {&colorIn, sl::kBufferTypeScalingInputColor, sl::ResourceLifecycle::eOnlyValidNow, &myExtent };
sl::ResourceTag colorOutTag = sl::ResourceTag {&colorOut, sl::kBufferTypeScalingOutputColor, sl::ResourceLifecycle::eOnlyValidNow, &myExtent };
sl::ResourceTag depthTag = sl::ResourceTag {&depth, sl::kBufferTypeDepth, sl::ResourceLifecycle::eValidUntilPresent, &fullExtent };
sl::ResourceTag mvecTag = sl::ResourceTag {&mvec, sl::kBufferTypeMvec, sl::ResourceLifecycle::eOnlyValidNow, &fullExtent };
sl::ResourceTag exposureTag = sl::ResourceTag {&exposure, sl::kBufferTypeExposure, sl::ResourceLifecycle::eOnlyValidNow, &my1x1Extent};

// Tag in group
sl::Resource inputs[] = {colorInTag, colorOutTag, depthTag, mvecTag};
slSetTag(viewport, inputs, _countof(inputs), cmdList);

NOTE: If dynamic resolution is used then please specify the extent for each tagged resource. Please note that SL manages resource states so there is no need to transition tagged resources.

NOTE: If sl::kBufferTypeExposure is NOT provided or dlssOptions.useAutoExposure is set to be true then DLSS will be in auto-exposure mode (NVSDK_NGX_DLSS_Feature_Flags_AutoExposure will be set automatically)

5.0 PROVIDE DLSS OPTIONS

DLSS options must be set so that the DLSS plugin can track any changes made by the user:

sl::DLSSOptions dlssOptions = {};
// Set preferred Render Presets per Perf Quality Mode. These are typically set one time
// and established while evaluating DLSS SR Image Quality for your Application.
// It will be set to DSSPreset::eDefault if unspecified.
// Please Refer to section 3.12 of the DLSS Programming Guide for details.
dlssOptions.dlaaPreset = sl::DLSSPreset::ePresetA;
dlssOptions.qualityPreset = sl::DLSSPreset::ePresetD;
dlssOptions.balancedPreset = sl::DLSSPreset::ePresetD;
dlssOptions.performancePreset = sl::DLSSPreset::ePresetD;
dlssOptions.ultraPerformancePreset = sl::DLSSPreset::ePresetA;
// These are populated based on user selection in the UI
dlssOptions.mode = myUI->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
dlssOptions.outputWidth = myUI->getOutputWidth();    // e.g 1920;
dlssOptions.outputHeight = myUI->getOutputHeight(); // e.g. 1080;
dlssOptions.sharpness = dlssSettings.sharpness; // optimal sharpness
dlssOptions.colorBuffersHDR = sl::Boolean::eTrue; // assuming HDR pipeline
dlssOptions.useAutoExposure = sl::Boolean::eFalse; // autoexposure is not to be used if a proper exposure texture is available
dlssOptions.alphaUpscalingEnabled = sl::Boolean::eFalse; // experimental alpha upscaling, enable to upscale alpha channel of color texture
if(SL_FAILED(result, slDLSSSetOptions(viewport, dlssOptions)))
{
    // Handle error here, check the logs
}

NOTE: To turn off DLSS set sl::DLSSOptions.mode to sl::DLSSMode::eOff, note that this does NOT release any resources, for that please use slFreeResources

NOTE: Set the DLSSOptions.useAutoExposure boolean to be true only if you want DLSS to be in in auto-exposure mode. Also, it is strongly advised to provide exposure if a proper exposure texture is available.

NOTE: Alpha upscaling (DLSSOptions::alphaUpscalingEnabled) is experimental, and will impact performace. This feature should be used only if the alpha channel of the color texture needs to be upscaled (if eFalse, only RGB channels will be upscaled).

6.0 PROVIDE COMMON CONSTANTS

Various per frame camera related constants are required by all Streamline features and must be provided if any SL feature is active and as early in the frame as possible. Please keep in mind the following:

  • All SL matrices are row-major and should not contain any jitter offsets
  • If motion vector values in your buffer are in {-1,1} range then motion vector scale factor in common constants should be {1,1}
  • If motion vector values in your buffer are NOT in {-1,1} range then motion vector scale factor in common constants must be adjusted so that values end up in {-1,1} range
sl::Constants consts = {};
// Set motion vector scaling based on your setup
consts.mvecScale = {1,1}; // Values in eMotionVectors are in [-1,1] range
consts.mvecScale = {1.0f / renderWidth,1.0f / renderHeight}; // Values in eMotionVectors are in pixel space
consts.mvecScale = myCustomScaling; // Custom scaling to ensure values end up in [-1,1] range
// Set all other constants here
if(SL_FAILED(result, slSetConstants(consts, *frameToken, myViewport))) // constants are changing per frame so frame index is required
{
    // Handle error, check logs
}

For more details please see common constants

7.0 ADD DLSS TO THE RENDERING PIPELINE

On your rendering thread, call slEvaluateFeature at the appropriate location where up-scaling is happening. Please note that when using slSetTag, slSetConstants and slDLSSSetOptions the frameToken and myViewport used in slEvaluateFeature must match across all API calls.

// Make sure DLSS is available and user selected this option in the UI
if(useDLSS) 
{
    // NOTE: We can provide all inputs here or separately using slSetTag, slSetConstants or slDLSSSetOptions

    // Inform SL that DLSS should be injected at this point for the specific viewport
    const sl::BaseStructure* inputs[] = {&myViewport};
    if(SL_FAILED(result, slEvaluateFeature(sl::kFeatureDLSS, *frameToken, inputs, _countof(inputs), myCmdList)))
    {
        // Handle error
    }
    else
    {
        // IMPORTANT: Host is responsible for restoring state on the command list used
        restoreState(myCmdList);
    }
}
else
{
    // Default up-scaling pass like for example TAAU goes here
}

IMPORTANT: Plase note that host is responsible for restoring the command buffer(list) state after calling slEvaluate. For more details on which states are affected please see restore pipeline section

8.0 MULTIPLE VIEWPORTS

Here is a code snippet showing one way of handling two viewports with explicit resource allocation and de-allocation:

// Viewport1
{
    // We need to setup our constants first so sl.dlss plugin has enough information
    sl::DLSSOptions dlssOptions = {};
    dlssOptions.mode = viewport1->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
    dlssOptions.outputWidth = viewport1->getOutputWidth();    // e.g 1920;
    dlssOptions.outputHeight = viewport1->getOutputHeight(); // e.g. 1080;
    // Note that we are passing viewport id 1
    slDLSSSetOptions(viewport1->id, dlssOptions);
    
    // Set our tags, note that we are passing viewport id
    setTag(viewport1->id, &tags2, numTags2);
    // and so on ...

    // Now we can allocate our feature explicitly, again passing viewport id
    slAllocateResources(sl::kFeatureDLSS, viewport1->id);

    // Evaluate DLSS on viewport1, again passing viewport id so we can map tags, constants correctly
    //
    // NOTE: If slAllocateResources is not called DLSS resources would be initialized at this point
    slEvaluateFeature(sl::kFeatureDLSS, myFrameIndex, viewport1->id, nullptr, 0, myCmdList);

    // Assuming the above evaluate call is still pending on the CL, make sure to flush it before releasing resources
    flush(myCmdList);

    // When we no longer need this viewport
    slFreeResources(sl::kFeatureDLSS, viewport1->id);
}

// Viewport2
{
    // We need to setup our constants first so sl.dlss plugin has enough information
    sl::DLSSOptions dlssOptions = {};
    dlssOptions.mode = viewport2->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
    dlssOptions.outputWidth = viewport2->getOutputWidth();    // e.g 1920;
    dlssOptions.outputHeight = viewport2->getOutputHeight(); // e.g. 1080;
    // Note that we are passing viewport id 2
    slDLSSSetOptions(viewport2->id, dlssOptions);

    // Set our tags, note that we are passing viewport id
    setTag(viewport2->id, &tags2, numTags2);
    // and so on ...

    // Now we can allocate our feature explicitly, again passing viewport id
    slAllocateResources(sl::kFeatureDLSS, viewport2->id);
    
    // Evaluate DLSS on viewport2, again passing viewport id so we can map tags, constants correctly
    //
    // NOTE: If slAllocateResources is not called DLSS resources would be initialized at this point
    slEvaluateFeature(sl::kFeatureDLSS, myFrameIndex, viewport2->id, nullptr, 0, myCmdList);

    // Assuming the above evaluate call is still pending on the CL, make sure to flush it before releasing resources
    flush(myCmdList);

    // When we no longer need this viewport
    slFreeResources(sl::kFeatureDLSS, viewport2->id);
}

9.0 CHECK STATE AND VRAM USAGE

To obtain current state for a given viewport the following API can be used:

sl::DLSSState dlssState{};
if(SL_FAILED(result, slDLSSGetState(viewport, dlssState))
{
    // Handle error here
}
// Check how much memory DLSS is using for this viewport
dlssState.estimatedVRAMUsageInBytes

10.0 TROUBLESHOOTING

If the DLSS output does not look right please check the following:

  • If your motion vectors are in pixel space then scaling factors sl::Constants::mvecScale should be {1 / render width, 1 / render height}
  • If your motion vectors are in normalized -1,1 space then scaling factors sl::Constants::mvecScale should be {1, 1}
  • Make sure that jitter offset values are in pixel space
  • NVSDK_NGX_Parameter_FreeMemOnRelease is replaced with slFreeResources
  • NVSDK_NGX_DLSS_Feature_Flags_MVLowRes is handled automatically based on tagged motion vector buffer's size and extent.