Whitelisted files need to be manually relisted #583

wblommaert · 2020-12-04T09:20:03Z

Sequel Ace Version: 2.3.2 build 2121
macOS Version: 11.0.1 Big Sur
Processor Type (Intel/Apple): Intel Core i7 - MacBook Pro (16-inch, 2019)
MySQL Version: 5.6, 5.7 and 8.0

Description
Every time I start the program (cold restart) I need to explicitly whitelist my key files and config again or I can't connect to any server. Additionally, the dropdown to select my SSH config always reverts back to the Sequel Ace default option after browsing to the config file on my Mac. Whitelisting files on the Files tab of the settings also remains blank, even after adding files multiple times and after each restart.

Additionally (might be related to the same issue), whenever I try to export a database to an existing selected directory (such as my Desktop directory) it gives me a warning that I can't access that directory. When I browse to it in the export dialog window and select the exact same directory again it will export just fine. Again a case of having to reselect things I've selected before to trigger "permission to access".

I've even gone as far as giving Sequel Ace "Full Disk Access" in macOS, but to no avail.

Steps to reproduce whitelisting issue

Cold start the program
Try to connect to any server
Verbose logs show that it can't access my config
Try to whitelist config
List of whitelisted files does not update or show anything
Dropdown of config selection still shows Sequel Ace default
Connection does work so it seems to have worked for this session
Fully close program and restart
Connecting to any server fails again

Expected Behaviour
Whitelisted files to remain whitelisted after restarts of the program.

Is Issue Present in Latest Beta?
Yes, my latest attempt I've tried the pre-release of 3.0.0 and this still has the same problem.

Additional Context
Absolutely love what you guys have done picking up from Sequel Pro. I've installed the app using the App Store, already tried reinstalling multiple times, giving full disk access, trying several betas over the months, but nothing seems to resolve my problem. The app doesn't crash so I don't have any crash logs, it just seems to always lose its sandbox access to whitelisted files after restarting.

Kaspik · 2020-12-04T14:13:53Z

I'm trying to repro this, but I have no luck...

The preferences persist, the access persists, everything works fine.

Could you screenshot the "Files" list so I can see how your path looks, if there are spaces, etc? It might be related to NSUserDefaults.

jamesstout · 2020-12-06T17:21:23Z

Hmmm ... there's a param we are not using, it might affect big sur.

bookmarkDataIsStale - I'll check it out.

wblommaert · 2020-12-07T14:21:03Z

I'm trying to repro this, but I have no luck...

The preferences persist, the access persists, everything works fine.

Could you screenshot the "Files" list so I can see how your path looks, if there are spaces, etc? It might be related to NSUserDefaults.

Hi Kaspik, thanks for checking it out. I made a short video recording of what happens as well as the screenshot. Unfortunately the path to my .ssh directory does not contain any spaces, so I'm afraid that might not be the issue.

Video:
https://streamable.com/kwcxml

Image of path to my .ssh directory:

Should you require anything else, I'd be more than happy to provide it. Thanks!

Kaspik · 2020-12-07T16:34:41Z

Hmm, interesting, sorry to hear that.

Could you please try one more thing (if @jamesstout doesn't have any other idea)?

export your favorites and other stuff so you don't loose it
completely wipe Sequel Ace (ideally via AppCleaner)
install either AppStore or latest beta again and go through all the prompts etc again

I feel like there is something wrong with the instance/your mac and Sequel Ace... even selecting the config doesn't wok which is weird.

wblommaert · 2020-12-07T20:00:14Z

I've tried both your suggestions using AppCleaner to remove everything related to Sequel Ace, but neither the AppStore version or the beta downloaded via GitHub have resolved the issue. Both of them still behave exactly like the video I captured earlier today. I'm not receiving any of the prompts that you mention either, which is something that worries me and makes me believe it is not fully removed despite using AppCleaner and removing everything it has flagged.

For what it's worth as extra info for @jamesstout, this issue didn't start in Big Sur, I also experienced it in Catalina. I probably inherited it when I upgraded to Big Sur last month.

Kaspik · 2020-12-07T21:14:20Z

Can you try resetting the prompts and the privacy when app is killed? tccutil reset All com.sequel-ace.sequel-ace (or tccutil reset All com.sequel-ace.sequel-ace-beta if you use the beta version)

It should reset everything privacy related for Sequel Ace.

wblommaert · 2020-12-08T09:46:42Z

I tried your suggestion but I can't seem to trigger the prompts anymore. What I did is:

Remove the app using AppCleaner
Used tccutil reset All com.sequel-ace.sequel-ace-beta
Moved the app (beta) back into my Applications directory
Launched the app and tried to whitelist files/change my SSH config setting

I didn't encounter any prompts during that process, and the behaviour of the whitelisting hasn't changed either. The tccutil command did provide this feedback Successfully reset All approval status for com.sequel-ace.sequel-ace-beta so I'm assuming it did its thing.

Kaspik · 2020-12-08T10:12:01Z

All right, thanks. We will rewrite the file manager to Swift and more modern code, but it might not be for 3.0.0 release. :)

wblommaert · 2020-12-08T10:20:17Z

I'll keep my eyes open for a future release then, thanks for the follow up. If there's anything I can provide (system settings, logs, etc.) I'd be more than happy to help.

jamesstout · 2020-12-08T10:24:30Z

Video:
https://streamable.com/kwcxml

Just watched this ... very weird. Could we provide a custom build with logging switched on for @wblommaert?

wblommaert · 2020-12-08T13:29:45Z

Video:
https://streamable.com/kwcxml

Just watched this ... very weird. Could we provide a custom build with logging switched on for @wblommaert?

I would be more than happy to try out a custom build and provide you with all the logs it would generate. I'm assuming if that were the case I'd have to do a clean install of the app (using the steps provided by Kaspik)?

jamesstout · 2020-12-11T08:48:39Z

@wblommaert any chance you could try out RC1 please?

wblommaert · 2020-12-11T15:18:11Z

@wblommaert any chance you could try out RC1 please?

Hi @jamesstout, I tried out the RC1 you linked but unfortunately I'm still experiencing the same problems. Here's what I did when installing the RC1:

exported my favourites
quit Sequel Ace
used AppCleaner to remove everything related to Sequel Ace
emptied recycle bin
downloaded and installed the RC1

I then tried the things shown in the video I linked earlier this week, but the behaviour hasn't changed. I'd try a clean install of my Mac but since this is my work laptop that I need on a daily basis that is unfortunately not an option (not until I'm on leave at least).

Kaspik · 2020-12-11T15:29:15Z

That's expected, we just added tons of logging so we will take a look at them. If you keep using it, it might give us even more info. Thanks! :)

jamesstout · 2020-12-11T23:58:02Z

@wblommaert what's the size of your prefs file:

/Users/<user>/Library/Containers/com.sequel-ace.sequel-ace/Data/Library/Preferences/com.sequel-ace.sequel-ace.plist

wblommaert · 2020-12-14T10:56:04Z

@wblommaert what's the size of your prefs file:

/Users/<user>/Library/Containers/com.sequel-ace.sequel-ace/Data/Library/Preferences/com.sequel-ace.sequel-ace.plist

@jamesstout the file is 2kb in size, I don't seem to have the specific file you mention, I have the beta version instead (I assume AppCleaner got rid of the non-beta one)

jamesstout · 2020-12-15T07:37:51Z

Hmmm, so it's not size (max size of a prefs file is 4MB).

jamesstout · 2020-12-15T07:48:26Z

We got some crashlytics logs. I won't post the screenshots here. But basically

-[SPFilePreferencePane loadBookmarks] line 103 $ Could not load SPSecureBookmarks from prefs

There's no key in prefs for SPSecureBookmarks and Sequel Ace can't create a bookmark:

-[SPFilePreferencePane chooseFile]_block_invoke_2 line 290 $ Problem creating bookmark - file:///Users/xxxxxxxxxx The file couldn’t be opened

This line fails. Gives us something to go on...

NSData *tmpAppScopedBookmark = [url 
    bookmarkDataWithOptions:(NSURLBookmarkCreationWithSecurityScope|NSURLBookmarkCreationSecurityScopeAllowOnlyReadAccess) 
    includingResourceValuesForKeys:nil 
    relativeToURL:nil 
    error:&error];

Sequel-Ace/Source/Controllers/Preferences/Panes/SPFilePreferencePane.m

Lines 257 to 291 in d1d94b7

    
           if ([self->_currentFilePanel.URL startAccessingSecurityScopedResource] == YES) { 
        
           	NSLog(@"got access to: %@", url.absoluteString); 
        
           	BOOL __block beenHereBefore = NO; 
        
           	[self.bookmarks enumerateObjectsUsingBlock:^(NSDictionary *dict, NSUInteger idx, BOOL *stop) { 
        
           		// check, if a bookmark already exists 
        
           		if (dict[url.absoluteString] != nil) { 
        
           			beenHereBefore = YES; 
        
           			*stop = YES; 
        
           		} 
        
           	}]; 
        
           	// if no bookmark exist, create on 
        
           	if (beenHereBefore == NO) { 
        
           		NSError *error = nil; 
        
           		NSData *tmpAppScopedBookmark = [url 
        
           										bookmarkDataWithOptions:(NSURLBookmarkCreationWithSecurityScope 
        
           																 | 
        
           																 NSURLBookmarkCreationSecurityScopeAllowOnlyReadAccess) 
        
           										includingResourceValuesForKeys:nil 
        
           										relativeToURL:nil 
        
           										error:&error]; 
        
           		// save the bookmark to the preferences in order to access 
        
           		// them later in the SPConnectionController 
        
           		if (tmpAppScopedBookmark && !error) { 
        
           			[self->bookmarks addObject:@{url.absoluteString : tmpAppScopedBookmark}]; 
        
           			[self->prefs setObject:self->bookmarks forKey:SPSecureBookmarks]; 
        
           		} 
        
           		else{ 
        
           			SPLog(@"Problem creating bookmark - %@ : %@",url.absoluteString, [error localizedDescription]); 
        
           			CLS_LOG(@"Problem creating bookmark - %@ : %@",url.absoluteString, [error localizedDescription]); 
        
           		}

jamesstout · 2020-12-15T08:01:26Z

Hmm, we're using a different url object from the one selected in the panel:

(lldb) p self->_currentFilePanel.URL
(NSURL *) $0 = 0x0000600002689aa0 @"file:///Users/james/.ssh/ssh_config_inc"
(lldb) p url
(NSURL *) $1 = 0x00006000026894a0 @"file:///Users/james/.ssh/ssh_config_inc"

This definitely caused problems for me during my initial write of the secure bookmarks code.

jamesstout · 2020-12-15T08:15:26Z

@wblommaert could you show the output of

codesign -d --entitlements :- /Applications/Sequel\ Ace.app/

please

or probably

codesign -d --entitlements :- /Applications/Sequel\ Ace\ Beta.app/

for you.

wblommaert · 2020-12-15T13:53:19Z

Hi @jamesstout, it gave me the output below:

Executable=/Applications/Sequel Ace Beta.app/Contents/MacOS/Sequel Ace Beta
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.application-identifier</key>
	<string>NKQ4HJ66PX.com.sequel-ace.sequel-ace-beta</string>
	<key>com.apple.developer.team-identifier</key>
	<string>NKQ4HJ66PX</string>
	<key>com.apple.security.app-sandbox</key>
	<true/>
	<key>com.apple.security.application-groups</key>
	<array>
		<string>NKQ4HJ66PX.sequel-ace</string>
	</array>
	<key>com.apple.security.automation.apple-events</key>
	<true/>
	<key>com.apple.security.files.bookmarks.app-scope</key>
	<true/>
	<key>com.apple.security.files.user-selected.read-write</key>
	<true/>
	<key>com.apple.security.network.client</key>
	<true/>
	<key>com.apple.security.network.server</key>
	<true/>
	<key>com.apple.security.print</key>
	<true/>
	<key>keychain-access-groups</key>
	<array>
		<string>NKQ4HJ66PX.com.sequel-ace.sequel-ace</string>
	</array>
</dict>
</plist>

Jason-Morcos · 2020-12-19T04:51:22Z

@wblommaert Can you please try the latest beta/release (3.0.1) and let us know if it's fixed now? There were a few tweaks aimed at possibly addressing these issues! May still not be fixed yet, however, but worth a check

https://github.com/Sequel-Ace/Sequel-Ace/releases

wblommaert · 2020-12-21T17:44:34Z

Hi @Jason-Morcos, thank you for linking the latest release. This seems to have fixed all of the problems mentioned in this issue. Both the selection of my own ssh config and the selection of whitelisted files is now working correctly and showing the correct files. For safe measure I restarted the application several times to try and reproduce my problems, but I haven't encountered any issues anymore.

Big thanks to @jamesstout and @Kaspik as well for the follow-up. Thanks guys!

wblommaert · 2021-01-22T09:14:37Z

Unfortunately I'm afraid this issue has returned after updating to the latest version (3.1.1 build 3016) today. I'm not sure whether it was the combination of a macOS update and a Sequel Ace update, but I'm unable to whitelist files again and my connections fail due to not having access to the files I had whitelisted before. I've already tried uninstalling using AppCleaner and reinstalling from the AppStore but the issue persists.

Is there anything I can provide to help investigate? The presented behaviour is exactly the same as in the video I provided earlier in this issue, the only difference being that in the original issue I coud still whitelist things and it would just not show visually while it seemed to whitelist them in the background. Currently I can't whitelist anything at all which means I can't use the application for any connections using SSH.

jamesstout · 2021-01-22T09:21:43Z

Checking.

Would you be happy to send me your prefs file? (/Users/<user>/Library/Containers/com.sequel-ace.sequel-ace/Data/Library/Preferences/com.sequel-ace.sequel-ace.plist)

You can edit to remove any personal info.

Or just send me the values for:

SPStaleSecureBookmarks
SPSecureBookmarksOldFormat
SPSecureBookmarks
SPSecureBookmarksHaveBeenMigrated

wblommaert · 2021-01-22T10:20:24Z

Hi @jamesstout, none of the keys you mention seem to be present in my prefs file. I've attached it in a zip so you can have a look (I didn't see any personal info in there, so it should be fine to share this I hope).

prefs file.zip

jamesstout · 2021-01-22T10:30:25Z

Thanks. So haven't managed to create bookmark since you uninstalled/cleaned. Any chance your old prefs file is still in trash?

Anyway, I think, I've identified an issue. I can't replicate your ssh config issue (might need to reboot into Big Sur), but I can't add multiple files on the files pane which is obviously a bug.

wblommaert · 2021-01-22T10:41:46Z

I hadn't created any new bookmarks since the uninstall/clean. I tried doing so now, but none of the keys you mentioned were added to the prefs file afterwards. Unfortunately my old prefs file is no longer in trash, so sadly I can't provide that one anymore.

I tried adding the files individually to work around the bug you mention, but that doesn't seem to work either. I'll try the reboot and see if that changes anything.

wblommaert · 2021-01-22T10:56:51Z

@jamesstout I rebooted and noticed the following:

I tried adding the files individually to work around the bug you mention, but that doesn't seem to work either.

This now works after rebooting. I can add files individually, but not multiple at the same time.

I'll try the reboot and see if that changes anything.

This seems to have fixed the whitelisting and selection of my ssh config issue as well. Guess the good old turning it off and on again did something.

jamesstout · 2021-01-22T11:01:30Z

Ahhh, great, cos I've fixed the multiple file thing and was trying to figure out your config issue.

wblommaert · 2021-01-22T11:49:58Z

Thanks for looking into it @jamesstout , I hope I didn't take up too much valuable time. Do you recommend restarting after every Sequel Ace update?

I'll close the issue again since everything seems to be working again since the reboot. I'll take that rebooting into account if I run into anything in the future.

Kaspik · 2021-01-22T12:00:36Z

I hope we won't be touching this between every release (we made some changes, that could cause it) so it shouldn't be necessary. :)

and added logging and alert #583 (again)

Kaspik added Bug Something isn't working Sandboxing labels Dec 4, 2020

Kaspik added Cannot Reproduce Help wanted Extra attention is needed Need More Info and removed Bug Something isn't working Sandboxing labels Dec 4, 2020

jamesstout mentioned this issue Dec 12, 2020

#temp: swizzle initFileURLWithPath: isDirectory #615

Merged

7 tasks

jamesstout mentioned this issue Dec 15, 2020

some fixes re secure bookmark generation and logging #fixed #624

Merged

7 tasks

wblommaert closed this as completed Dec 21, 2020

wblommaert reopened this Jan 22, 2021

wblommaert closed this as completed Jan 22, 2021

jamesstout added a commit that referenced this issue Jan 22, 2021

fix file url selection

8afb2cf

and added logging and alert #583 (again)

jamesstout mentioned this issue Jan 22, 2021

#fixed Bookmarks not being generated correctly #819

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Whitelisted files need to be manually relisted #583

Whitelisted files need to be manually relisted #583

wblommaert commented Dec 4, 2020

Kaspik commented Dec 4, 2020

jamesstout commented Dec 6, 2020

wblommaert commented Dec 7, 2020

Kaspik commented Dec 7, 2020

wblommaert commented Dec 7, 2020

Kaspik commented Dec 7, 2020 •

edited

Loading

wblommaert commented Dec 8, 2020

Kaspik commented Dec 8, 2020

wblommaert commented Dec 8, 2020

jamesstout commented Dec 8, 2020

wblommaert commented Dec 8, 2020

jamesstout commented Dec 11, 2020

wblommaert commented Dec 11, 2020

Kaspik commented Dec 11, 2020

jamesstout commented Dec 11, 2020

wblommaert commented Dec 14, 2020

jamesstout commented Dec 15, 2020

jamesstout commented Dec 15, 2020

jamesstout commented Dec 15, 2020

jamesstout commented Dec 15, 2020

wblommaert commented Dec 15, 2020

Jason-Morcos commented Dec 19, 2020

wblommaert commented Dec 21, 2020

wblommaert commented Jan 22, 2021

jamesstout commented Jan 22, 2021

wblommaert commented Jan 22, 2021

jamesstout commented Jan 22, 2021 •

edited

Loading

wblommaert commented Jan 22, 2021

wblommaert commented Jan 22, 2021

jamesstout commented Jan 22, 2021

wblommaert commented Jan 22, 2021

Kaspik commented Jan 22, 2021

Whitelisted files need to be manually relisted #583

Whitelisted files need to be manually relisted #583

Comments

wblommaert commented Dec 4, 2020

Kaspik commented Dec 4, 2020

jamesstout commented Dec 6, 2020

wblommaert commented Dec 7, 2020

Kaspik commented Dec 7, 2020

wblommaert commented Dec 7, 2020

Kaspik commented Dec 7, 2020 • edited Loading

wblommaert commented Dec 8, 2020

Kaspik commented Dec 8, 2020

wblommaert commented Dec 8, 2020

jamesstout commented Dec 8, 2020

wblommaert commented Dec 8, 2020

jamesstout commented Dec 11, 2020

wblommaert commented Dec 11, 2020

Kaspik commented Dec 11, 2020

jamesstout commented Dec 11, 2020

wblommaert commented Dec 14, 2020

jamesstout commented Dec 15, 2020

jamesstout commented Dec 15, 2020

jamesstout commented Dec 15, 2020

jamesstout commented Dec 15, 2020

wblommaert commented Dec 15, 2020

Jason-Morcos commented Dec 19, 2020

wblommaert commented Dec 21, 2020

wblommaert commented Jan 22, 2021

jamesstout commented Jan 22, 2021

wblommaert commented Jan 22, 2021

jamesstout commented Jan 22, 2021 • edited Loading

wblommaert commented Jan 22, 2021

wblommaert commented Jan 22, 2021

jamesstout commented Jan 22, 2021

wblommaert commented Jan 22, 2021

Kaspik commented Jan 22, 2021

Kaspik commented Dec 7, 2020 •

edited

Loading

jamesstout commented Jan 22, 2021 •

edited

Loading