»Open Timeline« shows blank grey screen

[mourique]

Hi! I wanted to try out the app and clicked »Start Remembering«, use my Computer for like 10 Minutes, clicked »Stop Remembering« and then »Open Timeline«. It open a fullscreen, blank grey window with no controls and no content. Is there any way i can send you debug information?

Also clicking on »Search« results in the screen i attached, it seems the screenshots are not correctly loaded?

I'd love to be able to help more on this!

[jasonjmcghee]

I've seen your issue- that's not good! I will look into it.

[jasonjmcghee]

I believe this is working properly now… locally. I’m getting ready to create another build but I’m trying to do an official one with Apple (non-app store) so figuring out code signing. If it takes too long I’ll make another unofficial

[jasonjmcghee]

I'm hoping this release works better for you - still plenty of issues: https://github.com/jasonjmcghee/rem/releases/tag/v0.1.2

[jasonjmcghee]

@mourique Could you check ~/Library/Application\ Support/today.jason.rem and see if there are any mp4s? I'm thinking ffmpeg might have failed to run. We're flying blind as there is no logging- at all.

[mourique]

i tried 0.1.2 and 0.1.3 but both did not work unfortunatly. Weirdly the folder you wanted me to checkout lives at ~/Library/Application\ Scripts/today.jason.rem but there is nothing in it, so you could be right and ffmpeg is not running.

edit: installing ffmpeg via homebrew did nothing, i did not think it would, but i tried it anyway :)

[jasonjmcghee]

No sqlite db either! I will investigate. Out of curiosity, do you have a folder at ~/Library/Containers/today.jason.rem/Data/Library/Application Support/today.jason.rem - the hardened app sandbox may be putting it there now instead (going to add to readme as well...)

[jarrenrocks]

@jasonjmcghee This is also happening for me as well. I'm still able to paste previous context, so it seems like it's still capturing. I have .mp4 files in ~/Library/Application\ Support/today.jason.rem

M2 air, macOS Sonoma 14.0 - did a full restart as well

installed via v0.1.5dmg

[jasonjmcghee]

mind trying https://github.com/jasonjmcghee/rem/releases/tag/v0.1.6?

[lotas]

I have similar issue, and the reason for me why that page is blank is because it seems like no images have been taken. All output-dd.mp4 files are zero bytes.
Search works but i believe it only works on a window/tab titles/urls so far, so content is not visible.
Running Sonoma 14.2.1.
Permissions to record screen were granted and I even restarted few times.

console app show this kind of errors from imagenet process, might be relevant ?

Error communicating with persistent store service proxy: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction.}

Also, seems like 0.1.6 is more resource-intensive, up to making whole system sluggish. 0.1.5 was running smoother but also not recording images :)

Thanks! Great efforts 👏

[jasonjmcghee]

Oh no! Worse? I will add a note to the release...

0.1.6 should only be changed during search / timeline. It's "sluggish" feeling- is that during scrubbing in the timeline? If so, I can just turn down how aggressive the throttling is

[jasonjmcghee]

contactsd issue... I was really trying to follow all Apple security best practices, but concerned it's not letting it write to its own folder.

This seems relevant: https://discussions.apple.com/thread/254659981?sortBy=best

[jasonjmcghee]

@lotas were you streaming with the console app when you ran 0.1.6? Because that will absolutely cause performance issues

[jasonjmcghee]

Just had a thought - in order to get things signed properly I created ffmpegX. I'm wondering if somehow, when the app isn't running on my computer (ugh) that ffmpegX doesn't have access to rem's Application Support location, due to sandboxing.

Crazy that it could create the files, just not, write to them?

[jasonjmcghee]

If someone runs into this, and is willing to try xattr -c rem.app that will remove gatekeeper stuff which should remove "sandbox". If it does, that will be pretty strong evidence of where we need to investigate / debug

[lotas]

So I did apply xattr -c, I have ffmpeg also installed on the system, and I see sqlite db growing and empty videos at /Users/yaraslaukurmyza/Library/Containers/today.jason.rem/Data/Library/Application Support/today.jason.rem

It's "sluggish" feeling- is that during scrubbing in the timeline

Everything.. but I think after few restarts it got better, although Activity shows %cpu floats around 15-20% but Energy Impact shows crazy values around 380-490 (whatever units those are)
(I tried both with and without console running)

Crazy that it could create the files, just not, write to them?

Is there a way to debug this?

upd: (I keep streaming console messages)
I see entries like this during recording:

default	19:20:08.908675+0100	rem	Calling completion handler For Request ID:13
Total Processing Time 1164.47ms
Has Analysis: YES
TextLength: 5201 DD: 8, MRC: 0, VS:0 
request: <private>
Error: (null)
default	19:20:10.126691+0100	rem	Processing Request: 14 - <private>
default	19:20:10.127588+0100	rem	Assigned madId: 14 to vkId: 14
default	19:20:11.230825+0100	rem	Completed MRC Parsing of 0 elements in 0.000002 seconds.
default	19:20:11.327843+0100	rem	Beginning tableProcessing with cellCount: 250
default	19:20:11.330559+0100	rem	Table processing complete with cellCount: 250
default	19:20:11.339406+0100	rem	Request completed: <private>

So I guess it is able to extract text from screenshot, but not write any image/video

Ok, those messages are accompanied with those:

default	19:24:55.921626+0100	mediaanalysisd	assetsForQuery: <query: com.apple.MobileAsset.LinguisticData, locO: 1, iO: 1, latO: 1, cO: 0, <filter: {
    AssetLocale = "{(\n    fr\n)}";
    LinguisticAssetType = "{(\n    Optional\n)}";
}>> final result: (
    "fr-Optional::65a0942448fecab0f4e9114258df6d1970091ab3.asset"
) was cached: 1, cachedOnly: 0
default	19:24:55.923937+0100	mediaanalysisd	Successfully loaded default inline completions lexicon
error	19:24:55.929491+0100	taskgated	cannot open file at line 49259 of [1b37c146ee]
error	19:24:55.929550+0100	taskgated	os_unix.c:49259: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
default	19:24:55.929943+0100	taskgated	MacOS error: -67062

stackoverflow says that macos error -67062 might be a signing issues after all.. but not sure which executable it is.

upd: taskgated ones coming from com.apple.libsqlite3 subsystem.. so maybe not related to the images/videos directly.. no idea

[jasonjmcghee]

Yeah that medianalysisd is ocr...

Things I'd try:

• trying to execute the ffmpeg command manually with the embedded ffmpeg
• swapping out the embedded ffmpeg with your homebrew one
• Try building it yourself with my script!

[lotas]

No luck so far, but I'm pretty sure the problem is somewhere around ffmpeg for me. Tried replacing with homebrew version but nothing's happening.

Although, when I tried to mimic the png stuff i got some codec errors like below, where my ffmpeg would run fine, but embedded one would. (maybe doing something wrong here also)

Btw, between 0.1.5 and 0.1.6 UI was "broken":

• when toggling timeline just gray window appears, in 0.1.5 it had a message in the center "nothing to remember.."
• search also stopped showing images initially, just all blank space and input is not focused, window not active.

I'm not sure how helpful that is though :)

[jasonjmcghee]

The errors here seems like it wants MP4 extension. Did you try specifying the codec as h264_videotoolbox?

[jasonjmcghee]

Managed to repro this, but mp4 etc. all working fine, search working etc. but getting gray timeline, until quitting and reopening. i think this is a potentially unrelated issue?

Unrelated and fixed in 0.1.8

[jasonjmcghee]

when toggling timeline just gray window appears, in 0.1.5 it had a message in the center "nothing to remember.."

Made a ton of progress on this. still working on it, but things are making a lot more sense. there were some bound variables that were updating and causing repeated re-renders

[jasonjmcghee]

Managed to repro this, but mp4 etc. all working fine, search working etc. but getting gray timeline, until quitting and reopening. i think this is a potentially unrelated issue?

when toggling timeline just gray window appears, in 0.1.5 it had a message in the center "nothing to remember.."

Both should be fixed in 0.1.8

---

The inability for some users to write data at all is still the main issue here and not fixed AFAIK.

[jasonjmcghee]

@lotas

Is there a way to debug this?

Absolutely there is, if you spin up xcode and step through the code, you should get a better sense of why it's failing

[lotas]

Absolutely there is, if you spin up xcode and step through the code, you should get a better sense of why it's failing

my xcode-fu is not sufficient apparently.. couldn't make debug build work .. complains about sqlite library missing, although it was fetched, build succesfully. Archive also worked.
Will have a look next year

[jasonjmcghee]

complains about sqlite library missing

Did you make sure it was marked as macos instead of iOS? (as per screenshot in readme)

I'd also make sure to do clean build too. I have to do it pretty regularly.

[lotas]

Ha! So I managed to make it work finally, it had to do with the fact that I don't have a developer account.. so I had to disable all that signing and tell "Disable Library Validation Entitlement" for sqlite .. anyways.. so it runs just fine and in that debug builds ffmpeg works flawlessly 😂
Not sure what's not right with the release package.

[mourique]

unfortunatly my skilllevel is not high enough yet to really grasp building apps with xcode, so i am no help on that front. but i did install 0.1.8 just now and while it still shows the blank grey screens (with the error message of missing frames), there is data in today.jason.rem, the sqlite db and the mp4-files (0 byte though).

[jasonjmcghee]

@lotas thank you- yeah that tells me there’s still something to do with either hardened runtime / app sandbox that’s preventing it from working for people who are downloading the release.

Will continue investigating…

[jasonjmcghee]

Alright. I did a bunch of... stuff... to do with entitlements etc.

Would love if someone having this issue try https://github.com/jasonjmcghee/rem/releases/tag/v0.1.9

Sorry for all the failures 🤦 - I'm trying. Never released an app through apple before and all the security / codesigning stuff is throwing me for a loop.

cc @mourique @lotas

[jasonjmcghee]

Also happy to invite one of you to the TestFlight, and see if that works:

[mourique]

installing the app via the pkg from 0.1.9 results in a corrupted app for me. can not be opened.
How would i go about installing the testflight version? — Can you send me an invite link?

[jasonjmcghee]

@mourique I need to invite you via email. I can send to the one posted on your personal site? If not, feel free to email me whichever preferred email to rem@jason.today

[mourique]

The one on my website is perfectly fine!

[jasonjmcghee]

Oof, I have to wait for external testflight app review. I'll add the latest dmg to 0.1.9 - it should have the same fixes? 🤞

[lotas]

I think 0.1.9 is finally recording videos and have screenshots! 🎉 Great job, thanks!

[jasonjmcghee]

If @mourique signs off on 0.1.9 dmg, I’ll close!

[mourique]

good news! »Search« works now, with screenshots and everything. »Toggle Timeline« is now a transparent screens. I can completely see the window behind and pressing escape returns there.

but i'll watch this more closely as i only tested it now for 5 minutes, you can close the issue for sure. thanks for being so responsive and nice! <3

[jasonjmcghee]

it never ends! 😂 Thank YOU for reporting and helping debug! It was huge.