Freeze after cloning big repo

[lordi]

Bug Description

The app freezes in step 3. "Try Cloning..." after reaching 100%. UI is unresponsive for at least 30 minutes, maybe longer or indefinite.

Expected Result

The cloning completes successfully.

Steps to Reproduce

I can reproduce it with my big private repo.

Possible Solutions (optional)

• Gather timestamps only for relevant or configured files (.md?)
• Provide a progress bar

Possible Additional Information (optional)

The last item in the log is "getTimestamps".

App version

25.09

Logs

The last item in the log is "getTimestamps".

[wiiznokes]

this is the function that collect timestamp:

gitnote/app/src/main/rust/src/libgit2/mod.rs

Line 289 in 1cbe5d4

pub fn get_timestamps() -> Result<HashMap<String, i64>, Error> {

I will try to make a test apk to see if it's really the bottleneck. But it could surely be optimized.

Just curious, how many files and commits is there in your repo ?

[lordi]

• Tracked files: 14123
• Total commits: 6608

[wiiznokes]

Ty.
Can you test this this apk ? https://github.com/wiiznokes/gitnote/actions/runs/18243325588
It's doesn't go in the timestamps function. This will help to know if there are other bottleneck in the code.

[lordi]

Thanks for the effort, this build crashed after cloning.

log.txt

Is the repo just too big? Would shallow cloning be an option?

[wiiznokes]

Idk. I think i will test on a repo with a similar size. Supporting this kind of repo will require massive change i believe. For example, i'm currently using a sql database to store the "notes", but this was unnecessary. The error seems to come from this: SQLiteQuery io.github.wiiznokes.gitnote.nightly E exception: Row too big to fit into CursorWindow.

[wiiznokes]

Looks like you have image encoded in base64, so theses files are written in the db. But when reading it, we can not go above 4mb. One solution could be to ignore files > x mb. Or store them on the fs.

https://stackoverflow.com/questions/57107489/sqliteblobtoobigexception-row-too-big-to-fit-into-cursorwindow-while-writing-to

[lordi]

Thanks for digging into it!

Aren't the files already stored in git and don't need to be stored a second time?

I'm not aware of any huge base64-encoded images in that repo.

[wiiznokes]

Aren't the files already stored in git and don't need to be stored a second time?

Yes. I originally done it this way because it was simplier, that's all

[wiiznokes]

Hey,

So i made some optimization to support huge repository like yours. Would you mind testing this artifact ? https://github.com/wiiznokes/gitnote/actions/runs/18907663448

They might be some bugs, i haven't tested it a lot, and i don't have a repo to test it (i tried the linux repo, but it take forever to clone with my connection).

[lordi]

Thanks, I will try

[wiiznokes]

You should try the latest run https://github.com/wiiznokes/gitnote/actions/workflows/nightly.yml