Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What about make a new one that just like asepsis? #25

Open
xiaozhuai opened this issue Dec 4, 2019 · 43 comments
Open

What about make a new one that just like asepsis? #25

xiaozhuai opened this issue Dec 4, 2019 · 43 comments

Comments

@xiaozhuai
Copy link

I am wondering if we can make a new one that just like asepsis. And how much work will it takes?
For some reason that asepsis not work well and lose maintainance. All we need is a easy to install and easy to use tool to prevent the fucking .DS_Store.

@JK3Y
Copy link
Owner

JK3Y commented Dec 5, 2019

honestly, that is the ideal solution. the issue is that asepsis and the underlying mach_override library hook into running system processes, which Apple has steadily been taking steps to prevent.
I have discovered that there is an 'official' way to get this functionality using system extensions. I'm unsure if the necessary apis would be available though.
I'm not the original developer of Asepsis, so I don't know how long it would take to reengineer a second one and I'm still running Mojave so I haven't actually messed with Catalina compatibility yet.

@xiaozhuai
Copy link
Author

Many evidences prove that Finder create and use the .DS_Store, am I right? So I am going to reverse enginering Finder to see what I can do.

@JK3Y
Copy link
Owner

JK3Y commented Dec 5, 2019 via email

@xiaozhuai
Copy link
Author

That's true!

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 5, 2019

@JK3Y Ok, I've spend all my day to work with it. Finally, I found something.
What if I attach to Finder process and hook the open syscall? If the file is .DS_Store then remove it and return -1. I've make an early prototype to prove my conjecture.

https://github.com/xiaozhuai/odourless

It works on my Mac. What's your suggestion?
And it's possible to redirect the file to another place that just like asepsis does. I am gona try it.

@darwin
Copy link

darwin commented Dec 5, 2019

Well, if you just wanted to delete .DS_Store files you could as well write a background daemon which would be watching filesystem changes and delete .DS_Store files after creation. This solution would not be Finder-specific and would not require SIP disabled.

@xiaozhuai
Copy link
Author

@darwin Yes, I know. You did a good job, Thank you!
May I ask a question, is the .DS_Store created by only Finder? It seems that's not always true. What I missed?

@JK3Y
Copy link
Owner

JK3Y commented Dec 6, 2019

@darwin thanks for jumping in the thread! Personally, I really like the DS cage. If my folder views didn't persist when I changed them, it would be pretty frustrating.
I've tried to find information on developing something like Asepsis but either my searches are bad, or the documentation is scarce. Do you have any resources you used during development? I'd like to keep this project running as long as possible (it's the first thing I install on a fresh system lol).

@xiaozhuai
Copy link
Author

@JK3Y It's the first thing I install on a fresh system tooooooo! Of course, @darwin has his own reason to drop maintainance. But I just want to do some thing to keep this project going on.

@JK3Y
Copy link
Owner

JK3Y commented Dec 6, 2019

@xiaozhuai interesting! I've never heard of Frida before. I admit, I was confused seeing a js and python file in there lol. Haven't tried it yet, but I'm glad you're making progress so quickly, that's awesome.
Asepsis makes some changes to the DesktopServicesPrivFramework, which I think is what you're looking for specifically instead of just the Finder process.
From what I remember, that framework is not documented because and it's a private framework it's not supposed to be accessible. The https://github.com/JK3Y/asepsis/blob/master/ctl/cmd/install_wrapper.rb file shows how it's done + has helpful comments

@xiaozhuai
Copy link
Author

@JK3Y I will take a look at this file. Frida depends on frida-gum which is a low-level code instrumentation library. The odourless I made is just a prototype. I believe I can do the same thing in c/c++ with frida-gum and mach_inject. I use python and js just for prototyping until I find a good way.

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 6, 2019

@JK3Y I look into DesktopServicesPriv.framework and I did find something!
image

image

image

image

If we can override this function __int64 __fastcall TCFURLInfo::Initialize(TCFURLInfo *this, const TCFURLInfo *a2, const TUString *a3) in DesktopServicesPriv.framework, then we can return a fake path of .DS_Store, just like Asepsis's DS cage.

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 6, 2019

@darwin What Asepsis does is overriding open, openx_np, getattrlist, setattrlist in DesktopServicesPriv.framework and change the path to a fake one with DS cage prefix.
Of course it check the process exeutable path to see if it was one of /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder and /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdwrite

Am I right? What is this mdwrite process?

Seems like Finder load DesktopServicesPriv but mdwrite not load it. So strange. I am running Majove 10.14.6.

xiaozhuai@xiaozhuai-master ~/Desktop/finder (master) $ ps -ax | grep Finder
17709 ??         0:06.58 /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
17866 ttys009    0:00.00 grep Finder

xiaozhuai@xiaozhuai-master ~/Desktop/finder (master) $ otool -L /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
/System/Library/CoreServices/Finder.app/Contents/MacOS/Finder:
	/System/Library/PrivateFrameworks/SafariFoundation.framework/Versions/A/SafariFoundation (compatibility version 1.0.0, current version 607.2.3)
	/System/Library/Frameworks/SafariServices.framework/Versions/A/SafariServices (compatibility version 535.0.0, current version 607.2.3)
	/System/Library/PrivateFrameworks/TimeMachine.framework/Versions/A/TimeMachine (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 1671.50.103)
	/System/Library/Frameworks/ContactsUI.framework/Versions/A/ContactsUI (compatibility version 1.0.0, current version 1894.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1575.12.0)
	/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox (compatibility version 1.0.0, current version 918.4.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 944.3.0)
	/System/Library/Frameworks/Quartz.framework/Versions/A/Frameworks/QuickLookUI.framework/Versions/A/QuickLookUI (compatibility version 1.0.0, current version 0.0.0)
	/System/Library/PrivateFrameworks/FileProvider.framework/Versions/A/FileProvider (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/SystemMigration.framework/Versions/A/SystemMigration (compatibility version 1.0.0, current version 866.0.0)
	/System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Versions/A/DesktopServicesPriv (compatibility version 1.0.0, current version 0.0.0)
	/System/Library/Frameworks/CloudKit.framework/Versions/A/CloudKit (compatibility version 1.0.0, current version 736.221.0)
	/System/Library/Frameworks/CoreMedia.framework/Versions/A/CoreMedia (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/AVFoundation.framework/Versions/A/AVFoundation (compatibility version 1.0.0, current version 2.0.0)
	/System/Library/Frameworks/AVKit.framework/Versions/A/AVKit (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/UserActivity.framework/Versions/A/UserActivity (compatibility version 1.0.0, current version 232.0.0)
	/System/Library/PrivateFrameworks/IconServices.framework/Versions/A/IconServices (compatibility version 1.0.0, current version 379.0.0)
	/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 963.250.1)
	/System/Library/PrivateFrameworks/Bom.framework/Versions/A/Bom (compatibility version 2.0.0, current version 195.0.0)
	/System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 161.9.16)
	/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/CloudDocs.framework/Versions/A/CloudDocs (compatibility version 0.0.0, current version 575.255.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/System/Library/PrivateFrameworks/SystemAdministration.framework/Versions/A/SystemAdministration (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/CoreUI (compatibility version 1.0.0, current version 499.10.0)
	/System/Library/Frameworks/ImageCaptureCore.framework/Versions/A/ImageCaptureCore (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)
	/System/Library/Frameworks/Quartz.framework/Versions/A/Quartz (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/Backup.framework/Versions/A/Backup (compatibility version 1.0.0, current version 0.0.0)
	/System/Library/PrivateFrameworks/AOSAccounts.framework/Versions/A/AOSAccounts (compatibility version 1.0.0, current version 1.9.95)
	/System/Library/Frameworks/QuickLook.framework/Versions/A/QuickLook (compatibility version 1.0.0, current version 0.0.0)
	/usr/lib/libicucore.A.dylib (compatibility version 1.0.0, current version 62.1.0)
	/System/Library/Frameworks/NetFS.framework/Versions/A/NetFS (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 50.1.0)
	/System/Library/PrivateFrameworks/Tourist.framework/Versions/A/Tourist (compatibility version 1.0.0, current version 1.0.61)
	/System/Library/PrivateFrameworks/Suggestions.framework/Versions/A/Suggestions (compatibility version 1.0.0, current version 226.0.0)
	/System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox (compatibility version 1.0.0, current version 492.0.0)
	/System/Library/PrivateFrameworks/DiskImages.framework/Versions/A/DiskImages (compatibility version 1.0.8, current version 480.255.1)
	/usr/lib/libDiagnosticMessagesClient.dylib (compatibility version 1.0.0, current version 107.0.0)
	/usr/lib/libCoreStorage.dylib (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/DiskManagement.framework/Versions/A/DiskManagement (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libcsfde.dylib (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/PrivateFrameworks/PerformanceAnalysis.framework/Versions/A/PerformanceAnalysis (compatibility version 1.0.0, current version 218.2.0)
	/System/Library/Frameworks/Quartz.framework/Versions/A/Frameworks/ImageKit.framework/Versions/A/ImageKit (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/SecurityInterface.framework/Versions/A/SecurityInterface (compatibility version 1.0.0, current version 55109.200.8)
	/System/Library/Frameworks/Contacts.framework/Versions/A/Contacts (compatibility version 0.0.0, current version 0.0.0)
	/System/Library/PrivateFrameworks/AppleSystemInfo.framework/Versions/A/AppleSystemInfo (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/Collaboration.framework/Versions/A/Collaboration (compatibility version 1.0.0, current version 81.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 58286.255.16)
	/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 23.0.0)
	/System/Library/PrivateFrameworks/Sharing.framework/Versions/A/Sharing (compatibility version 1.0.0, current version 1288.38.0)
	/System/Library/PrivateFrameworks/SidecarUI.framework/Versions/A/SidecarUI (compatibility version 1.0.0, current version 38.2.0)
	/System/Library/PrivateFrameworks/BackgroundTaskManagement.framework/Versions/A/BackgroundTaskManagement (compatibility version 1.0.0, current version 57.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
	/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork (compatibility version 1.0.0, current version 978.0.7)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1575.12.0)
	/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1251.12.0)
	/System/Library/Frameworks/CoreImage.framework/Versions/A/CoreImage (compatibility version 1.0.1, current version 5.0.0)
	/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation (compatibility version 1.0.0, current version 55185.255.1)

xiaozhuai@xiaozhuai-master ~/Desktop/finder (master) $ lsof -p 17709
COMMAND   PID      USER   FD   TYPE             DEVICE SIZE/OFF     NODE NAME
Finder  17709 xiaozhuai  cwd    DIR                1,4      928        2 /
Finder  17709 xiaozhuai  txt    REG                1,4  9441664 20352973 /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
Finder  17709 xiaozhuai  txt    REG                1,4  4134800 20728757 /System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Versions/A/DesktopServicesPriv
Finder  17709 xiaozhuai  txt    REG                1,4    21104 20385123 /Library/Preferences/Logging/.plist-cache.1VdnWDaz
Finder  17709 xiaozhuai  txt    REG                1,4   239648 13821703 /private/var/db/timezone/tz/2019c.1.0/icutz/icutz44l.dat
Finder  17709 xiaozhuai  txt    REG                1,4    28200 17910282 /Library/Application Support/CrashReporter/SubmitDiagInfo.domains
Finder  17709 xiaozhuai  txt    REG                1,4  4186528   313025 /System/Library/CoreServices/SystemAppearance.bundle/Contents/Resources/SystemAppearance.car
Finder  17709 xiaozhuai  txt    REG                1,4  3992488   313009 /System/Library/CoreServices/SystemAppearance.bundle/Contents/Resources/DarkAquaAppearance.car
Finder  17709 xiaozhuai  txt    REG                1,4   108308 20387519 /private/var/folders/0d/3gkdnd3d5vq64msqhnbt2glh0000gn/C/com.apple.IntlDataCache.le.kbdx
Finder  17709 xiaozhuai  txt    REG                1,4   889636   464993 /System/Library/Keyboard Layouts/AppleKeyboardLayouts.bundle/Contents/Resources/AppleKeyboardLayouts-L.dat
Finder  17709 xiaozhuai  txt    REG                1,4    87744   706430 /usr/lib/libobjc-trampolines.dylib
Finder  17709 xiaozhuai  txt    REG                1,4 27154336 10398505 /usr/share/icu/icudt62l.dat
Finder  17709 xiaozhuai  txt    REG                1,4 10010624 20727916 /private/var/folders/0d/3gkdnd3d5vq64msqhnbt2glh0000gn/0/com.apple.LaunchServices-231-v2.csstore
Finder  17709 xiaozhuai  txt    REG                1,4  2520960   313007 /System/Library/CoreServices/SystemAppearance.bundle/Contents/Resources/DarkAppearance.car
Finder  17709 xiaozhuai  txt    REG                1,4  3479856   313001 /System/Library/CoreServices/SystemAppearance.bundle/Contents/Resources/Assets.car
Finder  17709 xiaozhuai  txt    REG                1,4   212200   313013 /System/Library/CoreServices/SystemAppearance.bundle/Contents/Resources/FunctionRowAppearance.car
Finder  17709 xiaozhuai  txt    REG                1,4  2320268   334573 /System/Library/Fonts/Helvetica.ttc
Finder  17709 xiaozhuai  txt    REG                1,4  1797492   334723 /System/Library/Fonts/SFNSText.ttf
Finder  17709 xiaozhuai  txt    REG                1,4  1602736 10361561 /System/Library/CoreServices/Finder.app/Contents/Resources/Assets.car
Finder  17709 xiaozhuai  txt    REG                1,4  1332240 20358507 /System/Library/Frameworks/AppKit.framework/Versions/C/Resources/Assets.car
Finder  17709 xiaozhuai  txt    REG                1,4   262164 18655395 /Library/Caches/com.apple.iconservices.store/A2887C41-BFE2-738D-0875-D0F9D5EFA5D9.isdata
Finder  17709 xiaozhuai  txt    REG                1,4   299960   334658 /System/Library/Fonts/SFCompactText-Regular.otf
xiaozhuai@xiaozhuai-master ~/Desktop/finder (master) $ ps -ax | grep mdwrite
  534 ??         0:00.63 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdwrite
17859 ttys009    0:00.00 grep mdwrite

xiaozhuai@xiaozhuai-master ~/Desktop/finder (master) $ otool -L /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdwrite
/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdwrite:
	/System/Library/PrivateFrameworks/MetadataUtilities.framework/Versions/A/MetadataUtilities (compatibility version 1.0.0, current version 1191.57.0)
	/System/Library/PrivateFrameworks/SpotlightServerKit.framework/Versions/A/SpotlightServerKit (compatibility version 1.0.0, current version 1191.57.0)
	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 50.1.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1575.22.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 946.0.0)
	/System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Versions/A/DesktopServicesPriv (compatibility version 1.0.0, current version 0.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1575.22.0)
	/System/Library/Frameworks/QuickLook.framework/Versions/A/QuickLook (compatibility version 1.0.0, current version 0.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 58286.255.3)
	/System/Library/PrivateFrameworks/Bom.framework/Versions/A/Bom (compatibility version 2.0.0, current version 195.0.0)
	/System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation (compatibility version 1.0.0, current version 55185.255.1)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

xiaozhuai@xiaozhuai-master ~/Desktop/finder (master) $ lsof -p 534
COMMAND PID      USER   FD   TYPE DEVICE   SIZE/OFF     NODE NAME
mdwrite 534 xiaozhuai  cwd    DIR    1,4        928        2 /
mdwrite 534 xiaozhuai  txt    REG    1,4     129872 20359783 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mdwrite
mdwrite 534 xiaozhuai  txt    REG    1,4      21104 20385123 /Library/Preferences/Logging/.plist-cache.1VdnWDaz
mdwrite 534 xiaozhuai  txt    REG    1,4      32768 20387025 /private/var/db/mds/messages/501/se_SecurityMessages
mdwrite 534 xiaozhuai  txt    REG    1,4     239648 13821703 /private/var/db/timezone/tz/2019c.1.0/icutz/icutz44l.dat
mdwrite 534 xiaozhuai  txt    REG    1,4     973824 20379360 /usr/lib/dyld
mdwrite 534 xiaozhuai  txt    REG    1,4   27154336 10398505 /usr/share/icu/icudt62l.dat
mdwrite 534 xiaozhuai  txt    REG    1,4 1446039552 20384711 /private/var/db/dyld/dyld_shared_cache_x86_64h
mdwrite 534 xiaozhuai    0r   CHR    3,2        0t0      308 /dev/null
mdwrite 534 xiaozhuai    1u   CHR    3,2        0t0      308 /dev/null
mdwrite 534 xiaozhuai    2u   CHR    3,2        0t0      308 /dev/null

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 6, 2019

@JK3Y @darwin Ok, guys. I was able to make my project odourless redirect all .DS_Store files to a cage directory that just like asepsis does. Check https://github.com/xiaozhuai/odourless/blob/master/agent.js.
I am gona see if I can do the same thing with overriding TCFURLInfo::Initialize.

@darwin
Copy link

darwin commented Dec 6, 2019

@xiaozhuai Nice job!

Yes, asepsis had a whitelist of processes. In the end it was effective for Finder and mdwrite. Maybe mdwrite changed at some point or it uses DesktopServices indirectly via some of its dependencies.

I don't really remember the details. But some additional info can be obtained from the old homepage: https://asepsis.binaryage.com

Additionally Asepsis implements a system-wide daemon asepsisd whose purpose is to monitor system-wide folder renames (or deletes) and mirror those operations in the prefix folder. This is probably the best we can do. This way you don't lose your settings after renaming folders because rename is also executed on folder structure in the prefix directory.

It used to be that this "folder rename watching" was implemented as kernel extension. Later when they introduced effective filesystem events watching APIs I was able to rewrite it in user space: https://github.com/JK3Y/asepsis/tree/master/daemon

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 6, 2019

@darwin Yes,mdwrite do link against DesktopServicesPriv, but not load it. That made me confused.
I already see https://github.com/JK3Y/asepsis/tree/master/daemon.
But there is distance to reach this. I am still looking for a key function to override with.
I've ask a question in apple community to see if is possible to impl it in a system extension, so that we can do this with SIP enabled.

@JK3Y
Copy link
Owner

JK3Y commented Dec 7, 2019

@xiaozhuai great work! have you seen if this solution works on the latest macOS? I'm amazed at how quickly you're making advancements

@darwin thanks for pointing me in the right direction. I appreciate your help with this asepsis modernization :)

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 13, 2019

@JK3Y @darwin Ok, guys. I achieved this. Check https://github.com/xiaozhuai/odourless for more infomation. Tested on Mojave, not tested on Catalina yet.

@xiaozhuai
Copy link
Author

Tested on Catalina~ It works well.

@dactylo
Copy link

dactylo commented Dec 17, 2019

Hey @xiaozhuai thanks for your work on this. That came out of nowhere. I was starting to worry I might have to accept the .DS_Store when/if I move to Catalina and beyond.

If I understand correctly, this is using a similar method as asepsis, but it's a simplification?

@xiaozhuai
Copy link
Author

@dactylo Yes, as for now, odourless do not have a feature that @darwin mentioned before. That feature is on my plan.

Additionally Asepsis implements a system-wide daemon asepsisd whose purpose is to monitor system-wide folder renames (or deletes) and mirror those operations in the prefix folder. This is probably the best we can do. This way you don't lose your settings after renaming folders because rename is also executed on folder structure in the prefix directory.

Odourless do use a similar method as asepsis, and it works well on Catalina. But I'd recommend you backup your whole system via TimeMachine before upgrading to Catalina.

And if you got a good idea, issue and pr is welcome.

@torarnv
Copy link

torarnv commented Dec 23, 2019

@xiaozhuai Does Odourless require SIP-disabling only for installing, or permanently?

@xiaozhuai
Copy link
Author

@torarnv
Permanently.
There's a way to make Odourless only needs to turn off SIP during installation. Odourless make no changes to system files, it uses mach_inject to inject Finder during runtime, so it require SIP-disabling permanently. There is a way to use insert_dylib to do injecting.
Unfortunately, it seems not working on latest Catalina. Asepsis also needs to disable SIP permanently on Catalina too, am I right? If you got some thing, please share it to me.

@torarnv
Copy link

torarnv commented Dec 23, 2019

Thanks for the clarification!

Do I understand it correctly that an agent that watches added .DS_Store files and just removes them would also work, but you would loose any Finder customizations you do since those are what trigger the .DS_Store creation?

Would it be possible to persist the same Finder customizations in an extended attribute instead?

@xiaozhuai
Copy link
Author

@torarnv Yes, you are right.
I don't understand what you menu by storing in an extended attribute.

@torarnv
Copy link

torarnv commented Dec 23, 2019

I don't understand what you menu by storing in an extended attribute

Some properties are stored in extended attributes in the filesystem, such as file comments and tags, eg:

❯ xattr -l README.md
com.apple.FinderInfo:
00000000  00 00 00 00 00 00 00 00 00 0C 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
00000020
com.apple.metadata:_kMDItemUserTags:
00000000  62 70 6C 69 73 74 30 30 A1 01 55 52 65 64 0A 36  |bplist00..URed.6|
00000010  08 0A 00 00 00 00 00 00 01 01 00 00 00 00 00 00  |................|
00000020  00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
00000030  00 10                                            |..|
00000032

I was thinking that perhaps the info in the .DS_Store file could be turned into extended attributes first, and then removing the file. This assumes of course that there's a matching extended attribute (FinderInfo?) for all the info in the .DS_Store file, and that Finder will read those attributes in lieu of the .DS_Store file.

Some cursory googling suggests that it's not the case, but it might have changed in recent OS releases, so worth a shot?

@xiaozhuai
Copy link
Author

@torarnv That may be possible, but what matters is code injection. Code injection needs SIP disabled. If you want to change the behivaor of Finder, you need code injection.
It doesn't matter where you store it. Also we can design a new file format and store all these data to one single file. Odourless currently do it the way like Asepesis. Store all .DS_Store in a cage directory.

@torarnv
Copy link

torarnv commented Dec 23, 2019

If there's a matching extended attribute for the .DS_Store then there's no need to disable SIP. Just run a file watcher agent (like described in #25 (comment)) that looks for .DS_Store creation, copy the content over to a extended attribute for the given folder, and remove the .DS_Store file.

@darwin
Copy link

darwin commented Dec 23, 2019

There are probably some reasons why Apple didn't change implementation of .DS_Store to extended attributes. Most likely legacy reasons. One reason which comes to mind is that extended attributes are not supported on all possible filesystems. But I agree that it would be a more elegant solution (for a typical user who uses HFS+/APFS and does not care about external drives that much). And it would not require background daemon for monitoring folder renames. The downside is that cage directory approach has very simple implementation, but using extended attributes we would have to somehow extract DS_Store and then delete it or something like that.

@torarnv
Copy link

torarnv commented Dec 23, 2019

But I agree that it would be a more elegant solution (for a typical user who uses HFS+/APFS and does not care about external drives that much)

Right, this seems like the primary (only) use-case here (for us), so extended attributed would be a good fit. If it is possible, then it seems like a safer and more future proof solution than disabling SIP and interposing Finder, even if it requires a bit more code 😊

@darwin
Copy link

darwin commented Dec 23, 2019

I found this interesting article about xattr support on different filesystems:
https://eclecticlight.co/2018/01/12/which-file-systems-and-cloud-services-preserve-extended-attributes/

@torarnv I don't understand how you could implement it using xattr without interposing. When Finder wants to access a .DS_Store file there must be a mechanism which would read it from xattr and present it to Finder as fake .DS_Store.

@xiaozhuai
Copy link
Author

@torarnv But how can you implement it without interposing?

@torarnv
Copy link

torarnv commented Dec 23, 2019

@darwin The assumption is that Finder natively supports reading the info it needs also from extended attributes, but that it's creating .DS_Store files instead for compatibility, as mentioned earlier:

This assumes of course that there's a matching extended attribute (FinderInfo?) for all the info in the .DS_Store file, and that Finder will read those attributes in lieu of the .DS_Store file.

If that doesn't hold then yes, we'd need interposing, and there's no point in using extended attributes for the sidecar data over a file system jail like Odourless uses.

@xiaozhuai
Copy link
Author

@torarnv
This assumption needs to be verified on Catalina. If it's true, many people will be happy including me. Will you please give it a try?

@torarnv
Copy link

torarnv commented Dec 23, 2019

I don't know if it's possible, I was just brainstorming 😊 Looking at the disassembly of Finder or its frameworks might reveal what extended attributes it supports, and whether they (com.apple.FinderInfo) are able to contain the same info as the .DS_Store file.

@xiaozhuai
Copy link
Author

xiaozhuai commented Dec 23, 2019

@torarnv You may want to take a look at /System/Library/PrivateFrameworks/DesktopServicesPriv.framework. #25 (comment)
And I will go back to check it again on Catalina.

@darwin
Copy link

darwin commented Dec 23, 2019

I pretty much doubt com.apple.FinderInfo stores the same info as .DS_Store.

This[1] commenter claims he knows what is stored in com.apple.FinderInfo. Currently the structures are in latest SDK defined as:

struct FileInfo {
  OSType              fileType;               /* The type of the file */
  OSType              fileCreator;            /* The file's creator */
  UInt16              finderFlags;            /* ex: kHasBundle, kIsInvisible... */
  Point               location;               /* File's location in the folder */
                                              /* If set to {0, 0}, the Finder will place the item automatically */
  UInt16              reservedField;          /* (set to 0) */
};
struct ExtendedFileInfo {
  SInt16              reserved1[4];           /* Reserved (set to 0) */
  UInt16              extendedFinderFlags;    /* Extended flags (custom badge, routing info...) */
  SInt16              reserved2;              /* Reserved (set to 0). Comment ID if high-bit is clear */
  SInt32              putAwayFolderID;        /* Put away folder ID */
};

My observations:

  1. those structures are fixed-size and speak about a single file. I believe .DS_Store is dynamically-sized and speak about folder's content, e.g. which subfolders are expanded and what are current folder view-settings, icon positions, etc. see[2][3]
  2. Finder is not directly aware of .DS_Store files, it uses DesktopServicesPriv.framework which is responsible for managing this metadata. In general this framework can be used by different applications, not only Finder, the wiki page mentions Spotlight as well.

[1] https://eclecticlight.co/2017/12/19/xattr-com-apple-finderinfo-information-for-the-finder/#comment-16241
[2] https://en.wikipedia.org/wiki/.DS_Store
[3] https://0day.work/parsing-the-ds_store-file-format/

@torarnv
Copy link

torarnv commented Dec 23, 2019

@darwin Interesting, thanks for the insight! I didn't have high hopes for it to be possible, but was worth a shot!

@dactylo
Copy link

dactylo commented Dec 27, 2019

Even if it was possible, deleting the .DS_Store would change the modification date of the enclosing folder. That would be undesirable IMO, so that would have to be saved/restored, which I assume would introduce a race condition.

@davidkaufman
Copy link

Interesting conversation! I found this issue because I just upgraded to Catalina and (of course) Asepsis stopped working.
And of course I'd installed this fork (thanks JK3Y!) last time it broke, which was when I upgraded to Mojave :-) but I'd totally forgotten I'd had to do that last time, till I found this cloned REPO in my dev dir lol

And Big Sur is coming soon! So, for the record, I'll be happy switching to @xiaozhuai's odourless fork right now, even without the directory-watching-renaming daemon. I really could not care less if my folder metadata is lost when I rename, move or delete them! In fact, I'd be happy to use an even simpler fork that just /dev/null'ed all the damn .DS files, not saving them anywhere! No need for the complexities of a daemon OR extended attributes then! Uber-simplification and at what cost? What "feature" is lost? Where I left the icons on my desktop?? That's is the only one folder I even remotely ever care about where I left the icons: my desktop, and I hardly ever look at that one!

I keep my finder in Details view. I DO like that folders "remember" whether I left them sorted by date, or by name, or size -- but I could live without that too! The .DS file pollution is way more aggravating than losing that would be. Rather than a daemon, I'd be thrilled to run a cleanup/pruning script from time to time that compares the caged directories to "reality" and deletes ay that are no longer found, just so the cage doesn't grow in size endlessly -- In fact, I'll contribute that script :-)

Long-term, I would pay good $$ for a solution that:

  • had a simple installer (even if I had to run it after every upgrade),
  • was reasonably well-maintained (i don't upgrade the MOMENT a new macOS release comes out, but not too long after)
  • and that didn't even TRY to preserve the "features" these folder metadata turds were designed to implement!

Please -- somebody start a GoFundMe and save me!

Oh and it would be nice if the old Asepsis page had a link near the top that pointed (here, for Mojave users, and) to whatever the current workaround is that all of us old Asepsis addicts need to keep us going, since we are so old now that we tend to forget... :-)

thanks!

-dave

@Joshfindit
Copy link

I’m with dave: All my files are cross-platform so I don’t make use of any “MacOS specific” metadata, and as far as I’m concerned all finder windows can be in detail view sorted by name. If I want another view it’s for a specific use.

.DS_Store and all the other “dotfiles” can simply not exist and I’m perfectly happy.

@Joshfindit
Copy link

In fact, I prefer it.

@xiaozhuai
Copy link
Author

@davidkaufman If you want deleted .DS_Store anyway than keep it in a cage. There are several ways. It's much easier.

  1. Inject. You can fork my repo and make a little changes. It requires SIP disabled.
  2. A Daemon watches file created event,delete the file when matched. This one do not requires SIP disabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants