Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLAM-1535: Long file path support on Windows #229

Merged
merged 1 commit into from Aug 5, 2021

Conversation

kang-grace
Copy link
Contributor

via clam.manifest in win32/res. Opts into new Windows behavior that
does not have file path limitations.
Only works on Windows 10. In addition, you must set the registry key
"LongPathsEnabled" to 1.
(as described here: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=powershell)

via clam.manifest in win32/res. Opts into new Windows behavior that
does not have file path limitations.
Only works on Windows 10. In addition, you must set the registry key
"LongPathsEnabled" to  1.
(as described here: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=powershell)
@micahsnyder
Copy link
Contributor

I hadn't realized earlier that both the application (manifest) and the system (registry) need to opt-in for it to work. It still seems worthwhile, but less exciting if we can't make it work automatically*.

I did some basic testing using --leave-temps and scanning our source tarball. I found that this change + enabling the registry setting resolves most but not all issues with creating temporary directories and temporary files.

Before:

clamav-micah/build/install on  PR-229 took 6s
❯ .\clamscan.exe -d ..\unit_tests\input\clamav.hdb C:\Users\micasnyd\Downloads\clamav-0.104.0-rc.tar.gz --gen-json --leave-temps --tempdir=$home\tmp --max-filesize=1000M --max-scansize=1000M
Loading:     0s, ETA:   0s [========================>]        1/1 sigs
Compiling:   0s, ETA:   0s [========================>]       10/10 tasks

LibClamAV Error: cli_scanhtml: Can't create temporary directory C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\phish-test-ssl.35c4f2940b\clamav-3dd0be01179b2f52f4376cbcb4275001.tmp.c51f703942\html-tmp.a709ce8f65
LibClamAV Error: cli_scanhtml: Can't create temporary directory C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\phish-test-clean.2baea8658f\clamav-0a9ebb1e67b45d5367a660cdde1b6883.tmp.52e96bcf93\html-tmp.019174cdd0
LibClamAV Error: cli_gentempfd_with_prefix: Can't create temporary file C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\has_png_and_jpeg.xls.d1c9931756\[Content_Types].xml.507d844e70\clamav-6d1b6e467d81b0509a8691b3984f0be4.tmp: No such file or directory
LibClamAV Error: cli_gentempfd_with_prefix: Can't create temporary file C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\has_png_and_jpeg.xls.d1c9931756\themeManager.xml.rels.89ee9f98a7\clamav-b4aaee9edeae214f003ad7c7347c82d3.tmp: No such file or directory
LibClamAV Error: cli_scanhtml: Can't create temporary directory C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\phish-test-cloak.2bede906e6\clamav-ce6eea7bdeae45de86198f0b7a1141bb.tmp.fcddfcea42\html-tmp.af1a1ec9e2
LibClamAV Error: cli_magic_scan: Can't create tmp sub-directory for scan: C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\normal_255c_filename.cab.32b80d2d0f\HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello!.txt.546ccbefdb.
LibClamAV Error: cli_magic_scan: Can't create tmp sub-directory for scan: C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\cve-2017-6419-lzx-negative-spaninfo.chm.5bad207483\clamav-1a2df57ca987773cdef87c28345f8c63.tmp.
LibClamAV Error: cli_gentempfd_with_prefix: Can't create temporary file C:\Users\micasnyd\tmp\20210805_105646-clamav-0.104.0-rc.tar.gz.dc6f73a610\clamav-0.104.0-rc.tar.gz.f57f2cd37b\clamav-a1ca9db5a4c8caa42b90a9ebfd67b70d.tmp\cve-2015-4472-namelen-bounds.chm.e3fe7614a5\#SYSTEM.06bf468c6e\clamav-ba1761a3bc58e84974459da2ae87a771.tmp: No such file or directory
C:\Users\micasnyd\Downloads\clamav-0.104.0-rc.tar.gz: OK

After:

clamav-micah/build/install on  PR-229 took 8s
❯ .\clamscan.exe -d ..\unit_tests\input\clamav.hdb C:\Users\micasnyd\Downloads\clamav-0.104.0-rc.tar.gz --gen-json --leave-temps --tempdir=$home\tmp --max-filesize=1000M --max-scansize=1000M
Loading:     0s, ETA:   0s [========================>]        1/1 sigs
Compiling:   1s, ETA:   0s [========================>]       10/10 tasks

LibClamAV Error: cli_magic_scan: Can't create tmp sub-directory for scan: C:\Users\micasnyd\tmp\20210805_105732-clamav-0.104.0-rc.tar.gz.7aaf0d029f\clamav-0.104.0-rc.tar.gz.a27f17c30f\clamav-1c7cf9fd23947aa0d4fd16fb5732403c.tmp\normal_255c_filename.cab.0ec3efc103\HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello!.txt.d569591020.
C:\Users\micasnyd\Downloads\clamav-0.104.0-rc.tar.gz: OK

I think the reason that last directory didn't work is because Windows won't allow the ! character in the path. That's not really related to this change. I think we could resolve it by having a feature to strip invalid characters from file paths in cli_sanitize_filepath() This would be a separate task.

*I'm dreaming a little bit here about a follow-up task to improve the user experience. What would be really neat is if we could check if the registry key is set, maybe when starting freshclam.exe, clamd.exe, or clamscan.exe. If not set, we could print a message saying something along the lines of "Support for long paths is not enabled on this computer. Please run: clamconf --enable-long-paths to enable long paths.". Then we could add such an option to our clamconf.exe program which would notify the user with a message like "Your system may prompt you to for Administrator access permissions in order to make this change." after which it would attempt to get User Account Control (UAC) and enable that "LongPathsEnabled" registry value. Disclaimer: I've never actually made a program that prompts for UAC permissions so I don't really know how it's done, or if we can do it from a command line program.

@micahsnyder
Copy link
Contributor

I created two Jira tasks for these followup ideas: CLAM-1541 & CLAM-1542

@micahsnyder micahsnyder merged commit 657a8e0 into Cisco-Talos:main Aug 5, 2021
21 of 24 checks passed
@micahsnyder
Copy link
Contributor

I also cherry-picked this into rel/0.104 so we get long filepath support in 0.104 rc2 & the 0.104 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants