Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to open a netcdf file which path or file name contains unicode characters? #1568

Closed
wants to merge 3 commits into from
Closed

Conversation

kenash0625
Copy link

@kenash0625 kenash0625 commented May 24, 2019

my os is windows 10 pro N english edition,download prebuilt binaries from
http://www.gisinternals.com/query.html?content=filelist&file=release-1911-gdal-3-0-0-mapserver-7-4-0.zip
trying to open a file, file path is D:\日\1.nc, gdal returns nullptr.
i tried CPLSetConfigOption("GDAL_FILENAME_IS_UTF8", "YES"); ,but did not work.
i copied file D:\日\1.nc to D:\1\1.nc, everything is fine.

#include <ogrsf_frmts.h>
#pragma comment(lib,"gdal_i.lib")
#include <string>

#include<Windows.h>
std::string utf16ToUtf8(const wchar_t* lpszSrc, int cp = 65001)
{
std::string sResult;
int nUTF8Len = WideCharToMultiByte(cp, 0, lpszSrc, -1, NULL, 0, NULL, NULL);
char* pUTF8 = new char[nUTF8Len + 1];
ZeroMemory(pUTF8, nUTF8Len + 1);
auto r = WideCharToMultiByte(cp, 0, lpszSrc, -1, pUTF8, nUTF8Len, NULL, NULL);
sResult = pUTF8;
delete[] pUTF8;

return sResult;

}
int main()
{
CPLSetConfigOption("GDAL_DRIVER_PATH", "D:\release-1911-gdal-3-0-0-mapserver-7-4-0\bin\gdal\plugins");
GDALAllRegister();
OGRRegisterAll();
const char* pszFormat = "netCDF";
GDALDriver* poDriver;
poDriver = GetGDALDriverManager()->GetDriverByName(pszFormat);
if (poDriver != nullptr) printf("drv ok\n");

std::wstring wFile(L"d:\\日\\1.nc");
std::string sFile(utf16ToUtf8(wFile.c_str()));
CPLSetConfigOption("GDAL_FILENAME_IS_UTF8", "YES");
GDALDataset* tmFile = ((GDALDataset*)GDALOpen(sFile.c_str(), GA_ReadOnly));
if (tmFile != nullptr) printf("datasource-u8 ok\n");
GDALClose(tmFile);


GDALDataset* tmFile2 = ((GDALDataset*)GDALOpen("d:\\1\\1.nc", GA_ReadOnly));
if (tmFile2 != nullptr) printf("datasource ok\n");
GDALClose(tmFile2);
return 0;

}

output:

drv ok
Warning 1: One or several characters could not be translated to CP0. This warning will not be emitted anymore.
ERROR 4: `d:\µùÑ\1.nc' not recognized as a supported file format.
Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute
datasource ok

D:\release-1911-gdal-3-0-0-mapserver-7-4-0\bin\Project1.exe (process 1040) exited with code 0.
Press any key to close this window . . .

What does this PR do?

What are related issues/pull requests?

Tasklist

  • ADD YOUR TASKS HERE
  • Add test case(s)
  • Review
  • Adjust for comments
  • All CI builds and checks have passed

Environment

Provide environment details, if relevant:

  • OS:
  • Compiler:

with windows 10 pro N english edition,
trying to open a file, file path is D:\1961\日\CN05.1_Tm_1961_2012_daily_05x05.nc, gdal returns nullptr.
tried CPLSetConfigOption("GDAL_FILENAME_IS_UTF8", "YES"); ,but did not work.
the following piece of code prints 22222not null,so i guess we could use setlocale instead of widechartomultibyte and pass path to fopen?

        wstring testwstr = L"D:\\1961\\降雨温度统计\\CN05.1_Pre_1961_2012_daily_05x05.nc";
	FILE* fp2 = _wfopen(testwstr.c_str(),L"r");
	fclose(fp2);


	string teststr = utf16ToUtf8(testwstr.c_str());

	FILE *fp=fopen(teststr.c_str(),"r");

	if (fp)
	{
		cout << "1111not null" << endl;
	}
	auto setr = setlocale(LC_ALL, "en_US.UTF-8");
	fp = fopen(teststr.c_str(), "r");

	if (fp)
	{
		cout << "22222not null" << endl;
	}  




std::string utf16ToUtf8(const wchar_t* lpszSrc,int cp=65001)
{
	std::string sResult;
	int  nUTF8Len = WideCharToMultiByte(cp, 0, lpszSrc, -1, NULL, 0, NULL, NULL);
	char* pUTF8 = new char[nUTF8Len + 1];
	ZeroMemory(pUTF8, nUTF8Len + 1);
	auto r = WideCharToMultiByte(cp, 0, lpszSrc, -1, pUTF8, nUTF8Len, NULL, NULL);
	sResult = pUTF8;
	delete[] pUTF8;

	return sResult;
}
@rouault
Copy link
Member

rouault commented May 24, 2019

c:\projects\gdal\gdal\frmts\netcdf\netcdfdataset.cpp(6101) : warning C4701: potentially uninitialized local variable 'oldLocale' used
c:\projects\gdal\gdal\frmts\netcdf\netcdfdataset.cpp(6101) : warning C4703: potentially uninitialized local pointer variable 'oldLocale' used
gmtdataset.cpp
c:\projects\gdal\gdal\frmts\netcdf\netcdfdataset.cpp(7033) : warning C4701: potentially uninitialized local variable 'oldLocale' used
c:\projects\gdal\gdal\frmts\netcdf\netcdfdataset.cpp(7033) : warning C4703: potentially uninitialized local pointer variable 'oldLocale' used

@kenash0625
Copy link
Author

pushed one more commit.
maybe i should pull a request to lib netcdf , request to use _wfopen on windows instead of fopen?

@rouault
Copy link
Member

rouault commented May 28, 2019

your fix doesn't look right. The initial version was almost good. You just needed to initialize the oldLocale variable to nullptr

@kenash0625
Copy link
Author

your fix doesn't look right. The initial version was almost good. You just needed to initialize the oldLocale variable to nullptr

msdn page https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=vs-2019 says,A null pointer that's passed as the locale argument tells setlocale to query instead of to set the international environment.
initialized oldLocale variable using setlocale.

@kenash0625 kenash0625 changed the title cannot open netcdf file path contain non CP_ACP characters? How to open a netcdf file which path or file name contains unicode characters? Jul 10, 2019
@stale
Copy link

stale bot commented Jan 30, 2020

The GDAL project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last 21 days and is being automatically marked as "stale". If you think this pull request should be merged, please check - that all unit tests are passing - that all comments by reviewers have been addressed - that there is enough information for reviewers, in particular

  • link to any issues which this pull request fixes
  • add a description of workflows which this pull request fixes
  • add screenshots if applicable
  • that you have written unit tests where possible In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this pull request. If there is no further activity on this pull request, it will be closed in a week.

@stale stale bot added the stale label Jan 30, 2020
@stale
Copy link

stale bot commented Feb 6, 2020

While we hate to see this happen, this PR has been automatically closed because it has not had any activity in the last 28 days. If this pull request should be reconsidered, please follow the guidelines in the previous comment and reopen this pull request. Or, if you have any further questions, just ask! We love to help, and if there's anything the GDAL project can do to help push this PR forward please let us know how we can assist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants