Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support long file names on Windows #28

Closed
wants to merge 4 commits into from

Conversation

softworkz
Copy link
Collaborator

@softworkz softworkz commented May 13, 2022

This patchset adds support for long file and directory paths on Windows.
The implementation follows the same logic that .NET is using internally,
with the only exception that it doesn't expand short path components
in 8.3 format. .NET does this as the same function is also used for other
purposes, but in our case, that's not required. Short (8.3) paths are working
as well with the extended path prefix, even when longer than 260.

Successfully tested:

  • Regular paths wth drive letter
  • Regular UNC paths
  • Long paths wth drive letter
  • Long paths wth drive letter and forward slashes
  • Long UNC paths
  • Prefixed paths wth drive letter
  • Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it
easy to compare with the .NET impl.
(compilers should inlinie those anyway)

v2

  • wchar_filename: Improve comments and function documentation
  • os_support: adjust defines to use win32_stat

v3

  • removed length check in path_is_extended()
  • added path_is_device_path() check in add_extended_prefix()
  • add_extended_prefix(): clarified doc and add checks
  • clarified string allocation length calculation
  • replaced 260 with MAX_PATH
  • removed redundant checks after normalization

v4

  • rebased. no changes

v5

  • resolved the ugly struct duplication
  • compatible with _USE_32BIT_TIME_T

v6

  • wchar_filename.h: added links to .NET source code
  • wchar_filename.h: free allocations on error
  • os_support.hs: use clean and safe way to redirect stat() calls

v7

  • os_support.h: remapped fstat with win32_stat structure
  • os_support.h: use int64_t for some members
  • avformat/file: remove _WIN32 condition

v8

  • os_support.h: documented win32_stat structure
  • os_support.h: renamed function parameters

cc: Soft Works softworkz@hotmail.com
cc: Hendrik Leppkes h.leppkes@gmail.com
cc: Martin Storsjö martin@martin.st

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.ffstaging.FFmpeg.1652435595.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v1

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v1:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v1

@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, Soft Works wrote (reply to this):



> -----Original Message-----
> From: ffmpegagent <ffmpegagent@gmail.com>
> Sent: Friday, May 13, 2022 11:53 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: softworkz <softworkz@hotmail.com>
> Subject: [PATCH 0/2] Support long file names on Windows
> 
> This patchset adds support for long file and directory paths on
> Windows. The
> implementation follows the same logic that .NET is using internally,
> with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other
> purposes,
> but in our case, that's not required. Short (8.3) paths are working as
> well
> with the extended path prefix, even when longer than 260.
> 
> Successfully tested:
> 
>  * Regular paths wth drive letter
>  * Regular UNC paths
>  * Long paths wth drive letter
>  * Long paths wth drive letter and forward slashes
>  * Long UNC paths
>  * Prefixed paths wth drive letter
>  * Prefixed UNC paths
> 
> I have kept the individual functions separate on purpose, to make it
> easy to
> compare with the .NET impl. (compilers should inlinie those anyway)

Forgot to mention that I tested this with builds from both, MSVC and 
MinGW/MSYS2 (gcc 10.3).

sw


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

User Soft Works <softworkz@hotmail.com> has been added to the cc: list.

libavutil/file_open.c Outdated Show resolved Hide resolved
libavformat/os_support.h Outdated Show resolved Hide resolved
@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, nil-admirari@mailo.com wrote (reply to this):

> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)

Calling add_extended_prefix without pre-validation results in error,
since it does check for \\?\, \\.\, or \??\; only it's wrapper get_extended_win32_path does.
And it's not private, it's in the header.

path_normalize is a do nothing function.

Keeping the comments about where the code originated from may be useful.
Copying the structure of .NET results in problems.



These patches are very difficult to review. E.g. stat is not covered:
https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296448.html.

Parts of FFmpeg still use fopen instead of av_fopen_utf8. Some of these uses are
in examples or tests or inside #ifdef DEBUG blocks.
Others aren't: dvdsubdec.c:620 uses fopen and it is not covered by your
previous patchset https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296189.html.

So please check that all uses of CRT functions that do I/O:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/stream-i-o?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/low-level-i-o?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/directory-control?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/file-handling?view=msvc-170
are actually covered (plus WinAPI functions as well).



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, Soft Works wrote (reply to this):



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Sunday, May 15, 2022 9:36 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on
> Windows
> 
> > I have kept the individual functions separate on purpose, to make it
> easy to
> > compare with the .NET impl. (compilers should inlinie those anyway)
> 
> Calling add_extended_prefix without pre-validation results in error,
> since it does check for \\?\, \\.\, or \??\; only it's wrapper
> get_extended_win32_path does.
> And it's not private, it's in the header.


> path_normalize is a do nothing function.
> 
> Keeping the comments about where the code originated from may be
> useful.
> Copying the structure of .NET results in problems.

I can squash them together if that would be a common desire.


> These patches are very difficult to review. E.g. stat is not covered:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296448.html.

I'll look into stat in a moment.

> 
> Parts of FFmpeg still use fopen instead of av_fopen_utf8. Some of
> these uses are
> in examples or tests or inside #ifdef DEBUG blocks.
> Others aren't: dvdsubdec.c:620 uses fopen and it is not covered by
> your
> previous patchset https://ffmpeg.org/pipermail/ffmpeg-devel/2022-
> May/296189.html.

I have left those out by intention because they are pending removal
and are only for debugging.


Thanks again,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.v2.ffstaging.FFmpeg.1652653071.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v2

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v2:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v2

libavutil/file_open.c Outdated Show resolved Hide resolved
@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, nil-admirari@mailo.com wrote (reply to this):

> I have left those out by intention because they are pending removal
> and are only for debugging.

Is dvdsubdec.c parse_ifo_palette pending removal? What about
- vf_pnsr.c init()
- vf_vidstabdetect.c config_input()
- vf_vidstabtransform.c config_input()?



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.v3.ffstaging.FFmpeg.1652736203.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v3

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v3:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v3

@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, Soft Works wrote (reply to this):



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Monday, May 16, 2022 10:45 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on
> Windows
> 
> > I have left those out by intention because they are pending removal
> > and are only for debugging.
> 
> Is dvdsubdec.c parse_ifo_palette pending removal?

Just the debug part. Seems I missed the other one.

> - vf_pnsr.c init()
> - vf_vidstabdetect.c config_input()
> - vf_vidstabtransform.c config_input()?

I don't have the latter two included due to missing
lib. No idea how I have missed the other two.

Thanks for pointing out. Update will follow.

Kind regards,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.v4.ffstaging.FFmpeg.1653305371.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v4

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v4:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v4

@ffmpeg-codebot
Copy link

User Hendrik Leppkes <h.leppkes@gmail.com> has been added to the cc: list.

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.v5.ffstaging.FFmpeg.1653381808.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v5

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v5:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v5

libavutil/file_open.c Outdated Show resolved Hide resolved
@ffmpeg-codebot
Copy link

User Martin Storsjö <martin@martin.st> has been added to the cc: list.

libavformat/os_support.h Outdated Show resolved Hide resolved
Signed-off-by: softworkz <softworkz@hotmail.com>
@softworkz
Copy link
Collaborator Author

/submit

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.v7.ffstaging.FFmpeg.1653430851.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v7

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v7:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v7

@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, Martin Storsjö wrote (reply to this):

On Tue, 24 May 2022, ffmpegagent wrote:

> This patchset adds support for long file and directory paths on Windows. The
> implementation follows the same logic that .NET is using internally, with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other purposes,
> but in our case, that's not required. Short (8.3) paths are working as well
> with the extended path prefix, even when longer than 260.
>
> Successfully tested:
>
> * Regular paths wth drive letter
> * Regular UNC paths
> * Long paths wth drive letter
> * Long paths wth drive letter and forward slashes
> * Long UNC paths
> * Prefixed paths wth drive letter
> * Prefixed UNC paths
>
> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)
>
> v2
>
> * wchar_filename: Improve comments and function documentation
> * os_support: adjust defines to use win32_stat
>
> v3
>
> * removed length check in path_is_extended()
> * added path_is_device_path() check in add_extended_prefix()
> * add_extended_prefix(): clarified doc and add checks
> * clarified string allocation length calculation
> * replaced 260 with MAX_PATH
> * removed redundant checks after normalization
>
> v4
>
> * rebased. no changes
>
> v5
>
> * resolved the ugly struct duplication
> * compatible with _USE_32BIT_TIME_T
>
> v6
>
> * wchar_filename.h: added links to .NET source code
> * wchar_filename.h: free allocations on error
> * os_support.hs: use clean and safe way to redirect stat() calls
>
> v7
>
> * os_support.h: remapped fstat with win32_stat structure
> * os_support.h: use int64_t for some members
> * avformat/file: remove _WIN32 condition
>
> softworkz (3):
>  avutil/wchar_filename,file_open: Support long file names on Windows
>  avformat/os_support: Support long file names on Windows
>  avformat/file: remove _WIN32 condition

Thanks, I think this iteration of the patchset is fine - I don't have 
anything else to add to it at the moment. Let's wait for others to chime 
in still before proceeding further with it, though.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

libavformat/os_support.h Outdated Show resolved Hide resolved
Signed-off-by: softworkz <softworkz@hotmail.com>
stat is now re-mapped with long path support
in os_support.h

Signed-off-by: softworkz <softworkz@hotmail.com>
@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

Submitted as pull.28.v8.ffstaging.FFmpeg.1653557330.ffmpegagent@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v8

To fetch this version to local tag pr-ffstaging-28/softworkz/submit_long_filenames-v8:

git fetch --no-tags https://github.com/ffstaging/FFmpeg tag pr-ffstaging-28/softworkz/submit_long_filenames-v8

@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, Martin Storsjö wrote (reply to this):

On Thu, 26 May 2022, ffmpegagent wrote:

> This patchset adds support for long file and directory paths on Windows. The
> implementation follows the same logic that .NET is using internally, with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other purposes,
> but in our case, that's not required. Short (8.3) paths are working as well
> with the extended path prefix, even when longer than 260.
>
> Successfully tested:
>
> * Regular paths wth drive letter
> * Regular UNC paths
> * Long paths wth drive letter
> * Long paths wth drive letter and forward slashes
> * Long UNC paths
> * Prefixed paths wth drive letter
> * Prefixed UNC paths
>
> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)
>
> v2
>
> * wchar_filename: Improve comments and function documentation
> * os_support: adjust defines to use win32_stat
>
> v3
>
> * removed length check in path_is_extended()
> * added path_is_device_path() check in add_extended_prefix()
> * add_extended_prefix(): clarified doc and add checks
> * clarified string allocation length calculation
> * replaced 260 with MAX_PATH
> * removed redundant checks after normalization
>
> v4
>
> * rebased. no changes
>
> v5
>
> * resolved the ugly struct duplication
> * compatible with _USE_32BIT_TIME_T
>
> v6
>
> * wchar_filename.h: added links to .NET source code
> * wchar_filename.h: free allocations on error
> * os_support.hs: use clean and safe way to redirect stat() calls
>
> v7
>
> * os_support.h: remapped fstat with win32_stat structure
> * os_support.h: use int64_t for some members
> * avformat/file: remove _WIN32 condition
>
> v8
>
> * os_support.h: documented win32_stat structure
> * os_support.h: renamed function parameters
>
> softworkz (3):
>  avutil/wchar_filename,file_open: Support long file names on Windows
>  avformat/os_support: Support long file names on Windows
>  avformat/file: remove _WIN32 condition
>
> libavformat/file.c         |   4 -
> libavformat/os_support.h   | 116 ++++++++++++++++++------
> libavutil/file_open.c      |   2 +-
> libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 272 insertions(+), 30 deletions(-)

This is still ok with me, fwiw.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, Martin Storsjö wrote (reply to this):

On Thu, 26 May 2022, ffmpegagent wrote:

> This patchset adds support for long file and directory paths on Windows. The
> implementation follows the same logic that .NET is using internally, with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other purposes,
> but in our case, that's not required. Short (8.3) paths are working as well
> with the extended path prefix, even when longer than 260.
>
> Successfully tested:
>
> * Regular paths wth drive letter
> * Regular UNC paths
> * Long paths wth drive letter
> * Long paths wth drive letter and forward slashes
> * Long UNC paths
> * Prefixed paths wth drive letter
> * Prefixed UNC paths
>
> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)
>
> v2
>
> * wchar_filename: Improve comments and function documentation
> * os_support: adjust defines to use win32_stat
>
> v3
>
> * removed length check in path_is_extended()
> * added path_is_device_path() check in add_extended_prefix()
> * add_extended_prefix(): clarified doc and add checks
> * clarified string allocation length calculation
> * replaced 260 with MAX_PATH
> * removed redundant checks after normalization
>
> v4
>
> * rebased. no changes
>
> v5
>
> * resolved the ugly struct duplication
> * compatible with _USE_32BIT_TIME_T
>
> v6
>
> * wchar_filename.h: added links to .NET source code
> * wchar_filename.h: free allocations on error
> * os_support.hs: use clean and safe way to redirect stat() calls
>
> v7
>
> * os_support.h: remapped fstat with win32_stat structure
> * os_support.h: use int64_t for some members
> * avformat/file: remove _WIN32 condition
>
> v8
>
> * os_support.h: documented win32_stat structure
> * os_support.h: renamed function parameters
>
> softworkz (3):
>  avutil/wchar_filename,file_open: Support long file names on Windows
>  avformat/os_support: Support long file names on Windows
>  avformat/file: remove _WIN32 condition
>
> libavformat/file.c         |   4 -
> libavformat/os_support.h   | 116 ++++++++++++++++++------
> libavutil/file_open.c      |   2 +-
> libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 272 insertions(+), 30 deletions(-)

This looks fine to me, and the discussion seems to have converged, so I'll 
go ahead and push this.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, nil-admirari@mailo.com wrote (reply to this):

> This looks fine to me, and the discussion seems to have converged, so I'll 
> go ahead and push this.

Build is now failing: https://github.com/yt-dlp/FFmpeg-Builds/runs/6819319105?check_suite_focus=true.

In file included from /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/dshow.h:33,
                 from libavcodec/mf_utils.h:32,
                 from libavcodec/mfenc.c:26:
./libavutil/wchar_filename.h: In function 'add_extended_prefix':
./libavutil/wchar_filename.h:211:9: error: 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use in this function)
  211 |         wcscpy(temp_w, unc_prefix);
      |         ^~~~~~
./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is reported only once for each function it appears in

wchar.h is indeed not included in wchar_filename.h.



Incompatible pointer types warning is still there as well:

In file included from ./libavformat/internal.h:30,
                 from libavdevice/alldevices.c:21:
./libavformat/os_support.h: In function 'win32_stat':
./libavformat/os_support.h:241:36: warning: passing argument 2 of '_wstat64' from incompatible pointer type [-Wincompatible-pointer-types]
  241 |         ret = _wstat64(filename_w, &crtstat);
      |                                    ^~~~~~~~
      |                                    |
      |                                    struct _stati64 *
In file included from ./libavformat/os_support.h:32,
                 from ./libavformat/internal.h:30,
                 from libavdevice/alldevices.c:21:
/opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note: expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
  129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64 *_Stat);
      |                                                     ~~~~~~~~~~~~~~~~^~~~~





_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, Soft Works wrote (reply to this):



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Thursday, June 9, 2022 9:37 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
> 
> > This looks fine to me, and the discussion seems to have converged, so I'll
> > go ahead and push this.
> 
> Build is now failing: https://github.com/yt-dlp/FFmpeg-
> Builds/runs/6819319105?check_suite_focus=true.
> 
> In file included from /opt/ct-ng/i686-w64-
> mingw32/sysroot/mingw/include/dshow.h:33,
>                  from libavcodec/mf_utils.h:32,
>                  from libavcodec/mfenc.c:26:
> ./libavutil/wchar_filename.h: In function 'add_extended_prefix':
> ./libavutil/wchar_filename.h:211:9: error:
> 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use in
> this function)
>   211 |         wcscpy(temp_w, unc_prefix);
>       |         ^~~~~~
> ./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is
> reported only once for each function it appears in
> 
> wchar.h is indeed not included in wchar_filename.h.

wcscpy is defined in corecrt_wstring.h, included in string.h, 
included in winnt.h included in windef.h, included in windows.h


> Incompatible pointer types warning is still there as well:
> 
> In file included from ./libavformat/internal.h:30,
>                  from libavdevice/alldevices.c:21:
> ./libavformat/os_support.h: In function 'win32_stat':
> ./libavformat/os_support.h:241:36: warning: passing argument 2 of '_wstat64'
> from incompatible pointer type [-Wincompatible-pointer-types]
>   241 |         ret = _wstat64(filename_w, &crtstat);
>       |                                    ^~~~~~~~
>       |                                    |
>       |                                    struct _stati64 *
> In file included from ./libavformat/os_support.h:32,
>                  from ./libavformat/internal.h:30,
>                  from libavdevice/alldevices.c:21:
> /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note:
> expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
>   129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64
> *_Stat);
>       |
> ~~~~~~~~~~~~~~~~^~~~~

Yea, right. We need to get rid of the 'i' in the struct
types for getting fully independent of USE_32BIT_TIME_T.

Will send an update in a minute.

Thanks for reporting.

sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, Martin Storsjö wrote (reply to this):

On Thu, 9 Jun 2022, nil-admirari@mailo.com wrote:

>> This looks fine to me, and the discussion seems to have converged, so I'll
>> go ahead and push this.
>
> Build is now failing: https://github.com/yt-dlp/FFmpeg-Builds/runs/6819319105?check_suite_focus=true.
>
> In file included from /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/dshow.h:33,
>                 from libavcodec/mf_utils.h:32,
>                 from libavcodec/mfenc.c:26:
> ./libavutil/wchar_filename.h: In function 'add_extended_prefix':
> ./libavutil/wchar_filename.h:211:9: error: 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use in this function)
>  211 |         wcscpy(temp_w, unc_prefix);
>      |         ^~~~~~
> ./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is reported only once for each function it appears in

This error isn't reproducible in git master - it's triggered by your 
yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).

> Incompatible pointer types warning is still there as well:
>
> In file included from ./libavformat/internal.h:30,
>                 from libavdevice/alldevices.c:21:
> ./libavformat/os_support.h: In function 'win32_stat':
> ./libavformat/os_support.h:241:36: warning: passing argument 2 of '_wstat64' from incompatible pointer type [-Wincompatible-pointer-types]
>  241 |         ret = _wstat64(filename_w, &crtstat);
>      |                                    ^~~~~~~~
>      |                                    |
>      |                                    struct _stati64 *
> In file included from ./libavformat/os_support.h:32,
>                 from ./libavformat/internal.h:30,
>                 from libavdevice/alldevices.c:21:
> /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note: expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
>  129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64 *_Stat);
>      |                                                     ~~~~~~~~~~~~~~~~^~~~~

This I can indeed reproduce now. I did build these patches in a couple 
other environments (both mingw and msvc) where those warnings didn't 
appear, but now I managed to find one that shows those warnings.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, nil-admirari@mailo.com wrote (reply to this):

> wcscpy is defined in corecrt_wstring.h, included in string.h, 
> included in winnt.h included in windef.h, included in windows.h

Not so easy, it's the same as https://trac.ffmpeg.org/ticket/999:

> In file included from /opt/ct-ng/i686-w64-
> mingw32/sysroot/mingw/include/dshow.h:33

NO_DSHOW_STRSAFE must be defined somewhere.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, nil-admirari@mailo.com wrote (reply to this):

> This error isn't reproducible in git master - it's triggered by your 
> yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).

Ok. It can be fixed by either
- defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
- or by migrating os_support.h to StrSafe.h functions.

Which way is preferable?



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, Martin Storsjö wrote (reply to this):

On Thu, 9 Jun 2022, nil-admirari@mailo.com wrote:

>> This error isn't reproducible in git master - it's triggered by your
>> yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).
>
> Ok. It can be fixed by either
> - defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
> - or by migrating os_support.h to StrSafe.h functions.
>
> Which way is preferable?

I think avoiding wcscat (with whatever usable alternative, not necessarily 
from strsafe.h) is the more robust solution, instead of having to play 
whack-a-mole with silencing such warnings. The 10 year old trac you 
referenced mentioned that the strsafe.h alternative functions weren't 
available in all toolchains used at that time though.

Or if we'd add the define projectwide in e.g. configure it probably 
wouldn't be that bad? Kinda like how we already add 
"-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS" in MSVC builds. 
Then we wouldn't need to worry about missing it somewhere accidentally.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, Soft Works wrote (reply to this):



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Thursday, June 9, 2022 10:58 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
> 
> > This error isn't reproducible in git master - it's triggered by your
> > yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).
> 
> Ok. It can be fixed by either
> - defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
> - or by migrating os_support.h to StrSafe.h functions.
> 
> Which way is preferable?

I think the def is at least better for an instant fix.
Also, nowhere in ffmpeg are those "secure" functions being used, so it
aligns to current practice.

sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

On the FFmpeg mailing list, Soft Works wrote (reply to this):



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft Works
> Sent: Thursday, June 9, 2022 10:16 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> > admirari@mailo.com
> > Sent: Thursday, June 9, 2022 9:37 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on
> Windows
> >
> > > This looks fine to me, and the discussion seems to have converged, so
> I'll
> > > go ahead and push this.
> >
> > Build is now failing: https://github.com/yt-dlp/FFmpeg-
> > Builds/runs/6819319105?check_suite_focus=true.
> >
> > In file included from /opt/ct-ng/i686-w64-
> > mingw32/sysroot/mingw/include/dshow.h:33,
> >                  from libavcodec/mf_utils.h:32,
> >                  from libavcodec/mfenc.c:26:
> > ./libavutil/wchar_filename.h: In function 'add_extended_prefix':
> > ./libavutil/wchar_filename.h:211:9: error:
> > 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use
> in
> > this function)
> >   211 |         wcscpy(temp_w, unc_prefix);
> >       |         ^~~~~~
> > ./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is
> > reported only once for each function it appears in
> >
> > wchar.h is indeed not included in wchar_filename.h.
> 
> wcscpy is defined in corecrt_wstring.h, included in string.h,
> included in winnt.h included in windef.h, included in windows.h
> 
> 
> > Incompatible pointer types warning is still there as well:
> >
> > In file included from ./libavformat/internal.h:30,
> >                  from libavdevice/alldevices.c:21:
> > ./libavformat/os_support.h: In function 'win32_stat':
> > ./libavformat/os_support.h:241:36: warning: passing argument 2 of
> '_wstat64'
> > from incompatible pointer type [-Wincompatible-pointer-types]
> >   241 |         ret = _wstat64(filename_w, &crtstat);
> >       |                                    ^~~~~~~~
> >       |                                    |
> >       |                                    struct _stati64 *
> > In file included from ./libavformat/os_support.h:32,
> >                  from ./libavformat/internal.h:30,
> >                  from libavdevice/alldevices.c:21:
> > /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note:
> > expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
> >   129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64
> > *_Stat);
> >       |
> > ~~~~~~~~~~~~~~~~^~~~~
> 
> Yea, right. We need to get rid of the 'i' in the struct
> types for getting fully independent of USE_32BIT_TIME_T.
> 
> Will send an update in a minute.

Here's a patch for this part:

https://github.com/ffstaging/FFmpeg/pull/34

https://github.com/ffstaging/FFmpeg/pull/34.patch

Can't send it right now because Google decided to no longer
username/password login (which affects ffmpegagent). It's
said to work with an "app password" which in turn can only
be used when 2step verification is set up, and that again
requires either a phone or a security key :-(

Does anybody know a "security key" emulator.

softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

@softworkz
Copy link
Collaborator Author

/submit

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

Error: e13c6b0 was already submitted

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

Invalid author email in bd0a2ca: "4985349+softworkz@users.noreply.github.com"

@ffmpeg-codebot
Copy link

ffmpeg-codebot bot commented Jun 9, 2022

There is a merge commit in this Pull Request:

bd0a2ca280752765ad21656cd5d22facb7675ceb

Please rebase the branch and force-push.

@softworkz
Copy link
Collaborator Author

Merged into master

@softworkz softworkz closed this Jun 9, 2022
@nanake
Copy link

nanake commented Jun 9, 2022

@softworkz this causes build failed on win32 https://0x0.st/oMIX.log

@softworkz
Copy link
Collaborator Author

@softworkz this causes build failed on win32 0x0.st/oMIX.log

@nanake - yes, when USE_32BIT_TIME_T is specifified. To fix, you also need to apply this: #35

@nanake
Copy link

nanake commented Jun 10, 2022

@nanake - yes, when USE_32BIT_TIME_T is specifified. To fix, you also need to apply this: #35

thanks, build passed locally.
waiting for #35 to get merged.

@ffmpeg-codebot
Copy link

On the FFmpeg mailing list, nil-admirari@mailo.com wrote (reply to this):

> ...
> > - defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
> ...
> Or if we'd add the define projectwide in e.g. configure it probably 
> wouldn't be that bad? Kinda like how we already add 
> "-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS" in MSVC builds. 
> Then we wouldn't need to worry about missing it somewhere accidentally.

Ended up defining NO_DSHOW_STRSAFE in mf_utils.h,
just like dshow_capture.h does (these are the only two uses).

-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS
are defined only for MVSC:

elif test_${pfx}cpp_condition crtversion.h "defined _VC_CRT_MAJOR_VERSION"; then

NO_DSHOW_STRSAFE should cover MinGW as well, and probably others.
If you still want global NO_DSHOW_STRSAFE, please point where to add it exactly.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants