Pinned notes and folders order

[podviaznikov]

It's possible to Pin note and it's possible to rearrange folders order.
I assume this information is somewhere in sqlite db. Would it be possible to add this to the library?

[threeplanetssoftware]

I have considered adding some of the more "meta" information such as this (primarily the pinned note and the paper display on the note), but hadn't yet come up with a real use case. Could you please describe the value this would add for you?

[podviaznikov]

I'm trying to generate proper accurate html representation of my folder apple notes folder/notes structure.

Can also add PR if you help me to understand where this pin attribute (and order) info is stored.

[threeplanetssoftware]

Pinning is indicated by the ZICCLOUDSYNCINGOBJECT.ZISPINNED column. This is now added to the HTML and CSV output. In HTML, you will see a unicode pin next to the notes that are pinned.

[podviaznikov]

Oh, yeah - I saw that column too when inspected sqlite. Thank you for adding this! Couldn't find the column that stores the folder/notes order though.
Probably it's in one of those bplist columns?

[threeplanetssoftware]

Ok, I took a look and I suspect I know how the folder and note ordering is being stored, the folder aspect is the more annoying part since notes are just stored by modification date. I will try to work it in, but this will need a bit of testing so it may be a few days.

[threeplanetssoftware]

I just pushed a partial solution to this issue in 2df48b6. The note order is easy, so check out the -r option for your next run. I'll reorder the folders when I can time to test. This will show the notes in each folder in the right (I believe) order with pinned notes on top and everything sorted by modification time, newest on top.

[podviaznikov]

thank you so much! pinned notes work. Happy to test folders order when it's available.

[threeplanetssoftware]

Ok, thanks! It might be a bit, I opened a few other issues while poking at folders that I'd like to handle before pushing it out.

[threeplanetssoftware]

I have this at roughly an 80% solution, but there are some odd edge cases in my test data that simply aren't fitting. This isn't a bplist, it is another protobuf, similar to how tables are represented.

[podviaznikov]

thank you for the update!

[threeplanetssoftware]

I figured out where folder order is stored, but before I push this code out I want to handle a few other updates to folders nicely, rather than leave them on the todo list. Hopefully won't be too long, though.

[threeplanetssoftware]

Ok, check out 03265c1 on the order-folders-and-notes branch. This is a test that retains folder order, per account. Still a few things I'd like to clean up, but I think that the bulk of the work is done.

[podviaznikov]

got this error

Storing the results in ./output/notes_rip

Created a new AppleBackup from single file: NoteStore.sqlite
Guessed Notes Version: 15
Starting Apple Notes Parser at Thu Oct  6 23:17:38 2022
Storing the results in ./output/notes_rip

Created a new AppleBackup from single file: NoteStore.sqlite
Guessed Notes Version: 15
/Users/m1/app/parser/lib/AppleNoteStore.rb:442:in `inflate': buffer error (Zlib::BufError)
	from /Users/m1/app/parser/lib/AppleNoteStore.rb:442:in `block (2 levels) in rip_account'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:204:in `block (2 levels) in execute'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/resultset.rb:134:in `each'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:203:in `block in execute'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:156:in `prepare'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:198:in `execute'
	from /Users/m1/app/parser/lib/AppleNoteStore.rb:437:in `block in rip_account'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:204:in `block (2 levels) in execute'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/resultset.rb:134:in `each'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:203:in `block in execute'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:156:in `prepare'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:198:in `execute'
	from /Users/m1/app/parser/lib/AppleNoteStore.rb:405:in `rip_account'
	from /Users/m1/app/parser/lib/AppleNoteStore.rb:347:in `block in rip_accounts'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:204:in `block (2 levels) in execute'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/resultset.rb:134:in `each'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:203:in `block in execute'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:156:in `prepare'
	from /Library/Ruby/Gems/2.6.0/gems/sqlite3-1.4.4/lib/sqlite3/database.rb:198:in `execute'
	from /Users/m1/app/parser/lib/AppleNoteStore.rb:344:in `rip_accounts'
	from /Users/m1/app/parser/lib/AppleNoteStore.rb:202:in `rip_all_objects'
	from /Users/m1/app/parser/lib/AppleBackup.rb:169:in `block in rip_notes'
	from /Users/m1/app/parser/lib/AppleBackup.rb:166:in `each'
	from /Users/m1/app/parser/lib/AppleBackup.rb:166:in `rip_notes'
	from notes_cloud_ripper.rb:152:in `<main>'

and the debug logs show this

D, [2022-10-06T23:17:39.047193 #67015] DEBUG -- : Rip Account: Using server_record_column of ZSERVERRECORDDATA
D, [2022-10-06T23:17:39.047203 #67015] DEBUG -- : Rip Account: Query is SELECT ZICCLOUDSYNCINGOBJECT.ZNAME, ZICCLOUDSYNCINGOBJECT.Z_PK, ZICCLOUDSYNCINGOBJECT.ZSERVERRECORDDATA, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER, ZICCLOUDSYNCINGOBJECT.ZSERVERSHAREDATA, ZICCLOUDSYNCINGOBJECT.ZUSERRECORDNAME, ZICCLOUDSYNCINGOBJECT.ZACCOUNTDATA FROM ZICCLOUDSYNCINGOBJECT WHERE ZICCLOUDSYNCINGOBJECT.Z_PK=?

[threeplanetssoftware]

Thanks, just pushed 8833d10 which should hopefully fix that.