C# bindings: GDAL_FILENAME_IS_UTF8 config option doesn't work for cyrillic symbols

[Gigas002]

Expected behavior and actual behavior.

Gdal's c# bindings seems not to work with GDAL_FILENAME_IS_UTF8 config option.
If I don't set this option in code, methods Gdal.wrapper_GDALTranslate, Gdal.Open and Gdal.wrapper_GDALWarpDestName works fine with paths with spaces and cyrillic symbols, even though Gdal.GetConfigOption("GDAL_FILENAME_IS_UTF8", "YES") returns YES. Yet the Gdal.wrapper_GDALBuildVRT_names doesn’t work and also doesn’t throw any exceptions, it just doesn’t do the job, if path contains cyrillic symbols.

If I explicitly set this option to NO this way: Gdal.SetConfigOption("GDAL_FILENAME_IS_UTF8", "NO"), all the methods, written above (including Gdal.wrapper_GDALBuildVRT_names) doesn’t work with cyrillic paths at all.

I tried to reproduce error with console gdalbuildvrt.exe app, but it creates .vrt file as expected (with --config GDAL_FILENAME_IS_UTF8 NO string, and doesn’t create anything without it), so it seems like the problem is only in bindings. Probably, @szekerest knows something about it?

Steps to reproduce the problem.

Here's ready for tests c# code. Just create .NET Framework project, add GDAL and GDAL.Native nuget packages and set active platform to x64. Then change paths in code and run.

Click to expand

using System;
using System.IO;
using System.Linq;
using OSGeo.GDAL;
namespace Test
{
    internal static class Program
    {
        private static void Main()
        {
            //Configure Gdal's paths before using it. Don't forget to change target system to x64.
            GdalConfiguration.ConfigureGdal();

            //Check the default state of "GDAL_FILENAME_IS_UTF8" config option.
            string currentState = Gdal.GetConfigOption("GDAL_FILENAME_IS_UTF8", "YES");
            Console.WriteLine($"GDAL_FILENAME_IS_UTF8 is set to {currentState} by default.");
    
            //Paths local variables.
            string inputFilePath, inputDirectoryPath, outputFilePath;
    
            #region Test 1
            //Test 1 - Passing. Paths doesn't contain any cyrillic symbols.
            inputFilePath = "D:/Test/Input data/Test.tif";
            inputDirectoryPath = "D:/Test/Input data";
            outputFilePath = "D:/Test/Test1.vrt";
            string test1Result = RunTest(inputFilePath, inputDirectoryPath, outputFilePath) ? "passed" : "failed";
            Console.WriteLine($"Test 1 {test1Result}.");
    
            //Clean up.
            if (File.Exists(outputFilePath)) File.Delete(outputFilePath);
            #endregion
    
            #region Test 2
            //Test 2 - Gdal.Open pass, BuildVrt fails (doesn't throw errors/exceptions, but no output file), writes "warning" in console.
            inputFilePath = "D:/Тест/Исходные данные/Тест.tif";
            inputDirectoryPath = "D:/Тест/Исходные данные";
            outputFilePath = "D:/Тест/Тест2.vrt";
            string test2Result = RunTest(inputFilePath, inputDirectoryPath, outputFilePath) ? "passed" : "failed";
            Console.WriteLine($"Test 2 {test2Result}.");
    
            //Clean up.
            if (File.Exists(outputFilePath)) File.Delete(outputFilePath);
            #endregion
    
            #region Test 3
            //Test 3 - Gdal.Open pass, BuildVrt fails (doesn't throw errors/exceptions, but no output file), writes "warning" in console.
            //Change "GDAL_FILENAME_IS_UTF8" value to "NO" and check, if it was changed correctly.
            Gdal.SetConfigOption("GDAL_FILENAME_IS_UTF8", "NO");
            currentState = Gdal.GetConfigOption("GDAL_FILENAME_IS_UTF8", "YES");
            Console.WriteLine($"GDAL_FILENAME_IS_UTF8 is set to {currentState} before the test.");
            inputFilePath = "D:/Test/Input data/Test.tif";
            inputDirectoryPath = "D:/Test/Input data";
            outputFilePath = "D:/Test/Test3.vrt";
            string test3Result = RunTest(inputFilePath, inputDirectoryPath, outputFilePath) ? "passed" : "failed";
            Console.WriteLine($"Test 3 {test3Result}.");
    
            //Clean up.
            if (File.Exists(outputFilePath)) File.Delete(outputFilePath);
            #endregion
    
            #region Test 4
            //Test 4 - all fails, Gdal.Open throws exception, BuildVrt writes "" warning.
            Console.WriteLine($"GDAL_FILENAME_IS_UTF8 is set to {currentState} before the test.");
            inputFilePath = "D:/Тест/Исходные данные/Тест.tif";
            inputDirectoryPath = "D:/Тест/Исходные данные";
            outputFilePath = "D:/Тест/Тест4.vrt";
            string test4Result = RunTest(inputFilePath, inputDirectoryPath, outputFilePath) ? "passed" : "failed";
            Console.WriteLine($"Test 4 {test4Result}.");
    
            //Clean up.
            if (File.Exists(outputFilePath)) File.Delete(outputFilePath);
            #endregion
    
            Console.ReadKey();
        }
    
        private static bool GdalBuildVrt(string[] inputFilesPaths,
                                        string outputFilePath,
                                        string[] options,
                                        Gdal.GDALProgressFuncDelegate callback)
        {
            try
            {
                //"using" statement or explicit "result?.Dispose()" method is required for immidiate disposing of the dataset.
                using (Dataset result = Gdal.wrapper_GDALBuildVRT_names(outputFilePath, inputFilesPaths, new GDALBuildVRTOptions(options), callback, null)) { }
            }
            catch (Exception exception)
            {
                Console.WriteLine(exception.Message);
                return false;
            }
    
            return true;
        }
    
        private static bool OpenDataset(string inputFilePath)
        {
            try
            {
                //"using" statement or explicit "inputDataset?.Dispose()" method is required for immidiate disposing of the dataset.
                using (Dataset inputDataset = Gdal.Open(inputFilePath, Access.GA_ReadOnly)) { }
            }
            catch (Exception exception)
            {
                Console.WriteLine(exception.Message);
                return false;
            }
    
            return true;
        }
    
        private static bool RunTest(string inputFilePath, string inputDirectoryPath, string outputFilePath)
        {
            bool isTestSuccessful = OpenDataset(inputFilePath);
    
            string[] inputFilesPaths = new DirectoryInfo(inputDirectoryPath)
                                      .EnumerateFiles().Select(fileInfo => fileInfo.FullName).ToArray();
            if (!GdalBuildVrt(inputFilesPaths, outputFilePath, null, null))
                isTestSuccessful = false;
    
            //Check if .vrt file was created, because GdalBuildVrt doesn't throw exceptions in that case.
            if (!new FileInfo(outputFilePath).Exists)
                isTestSuccessful = false;
    
            return isTestSuccessful;
        }
    }
}

Operating system

Windows 8.1 x64

GDAL version and provenance

GDAL and GDAL.Native packages from nuget, ver. 2.3.3.

[Gigas002]

Updated packages to the newest 2.4.1 and tested again. Seems like 3rd test is passing now, but still cyrillic paths doesn't work with test 2 and test 4.
In case of test 2, Gdal.Open works correctly, but Gdal.wrapper_GDALBuildVRT_names fails to create .vrt file.
In case of test 4, they both fail.