Add support for f-droids new repository format #287
Comments
|
I will start working on this tomorrow. |
Signed-off-by: LooKeR <mohit2002ss@gmail.com>
|
This is now the first priority. I will work on other topics once this is over |
|
Have you considered making use of the official F-Droid libraries for this? At least the minimal index library should be helpful. |
|
I looked into it but thought I will write my own implementation |
|
Here's an early draft of a blog post about those libraries. Maybe it can convince you to not re-invent the wheel: |
|
Cool I will read it and get back. But I just wanted to tell you something, your best locale selector is probably not working as intended. I wrote my own and I think it is passing all the tests that I wrote which are inspired by your tests. Maybe you can run those test once and check? I could be very wrong. |
Can you please point me to the test case(s) that fail for our implementation? Due to the differences in your code, it isn't obvious to spot.
Even just the index library? It should e the easiest to integrate into an existing project that doesn't use v2, yet. Also remember that code is copyrighted and if you copy it into your project, there are certain conditions that apply such as crediting the original auther, etc. This is also true for test code. |
This is one of the test which fails with the current implementation of LocaleChooser
Yes, I haven't looked into it thoroughly but I will reach back if I fail to implement it this time.
I have actually not copied it, I just took inspiration, you can see my Tests and Localization code here. I will give credits anyways. Thanks for notifying because I am not that knowledgeable about copyright rules. |
This test seems to be copied one-to-one from F-Droid's source code (again without attribution and license declaration): It is passing just fine. So maybe you were thinking of other tests that don't pass for F-Droid's locale chooser implementation? |
|
@grote I added credits to FDroid in latest commit 3ee1064c40d8caeac5ba5719bbd86ea6d5cd8fe0. You can tell me if anything needs to be changed |
Idk it was failing for me. I will try to replicate it can share a ss to you here |
|
What device do you run it on? Which Android version? Maybe there's differences on older versions. |
I am not using any devices, I am just running the test on Android Studio |
|
This is an instrumentation test. It won't work as a unit test which probably explains the failure you've been seeing. Roboelectric most likely doesn't provide the real implementation and just mocks those classes. |
😵😵 My bad |
|
It still downloads whole 1.2 mb of index from izzyondroid's repo. I wonder if it's a bug or sth else. |
Index v2 implementation has not been completed |
|
@grote There seems to be a reflection error due to mismatch in ktor library version in the Maybe this can be fixed on the library side? I am encountring this issue when I add Logs |
|
Also I am getting the following error when I have both my This occurs when I try to download a file. It works fine if I dont have data module(Which contains Logs |
|
does it work when using the same ktor library version? |
Nope it still fails |
|
Does it work when excluding the ktor dependency from fdroid's downloader lib? is your kotlin-reflect dependency on the same kotlin version as your project? |
|
I am not using
but it was still available to use |
|
Your stacktrace shows that you do use kotlin-reflect, so please as a first thing add it as an explicit dependency and ensure that it uses the same kotlin version as your project does. This is a common source of such errors. |
That seems to have fixed it. I digged a little deeper and it seemed like Android Studio G comes with Either-way thanks 😄 |
|
I am facing a similar issue again, this time I have the latest reflect and kotlin version but still. Details |
|
ProGuard 🤔 |
In debug builds tho |
|
Not select text for copy |
Btw this happens when I am downloading a v1 format index. Sorry @grote for tagging again |
|
|
Yes, it happens with Izzy and FDroid index v1.
Removing |
|
You can test it in this branch, you just need to build and click sync button on main screen |
|
Tested with repo The repo was generated by Thank you for looking into this! |
|
Any update on this? |
|
@machiav3lli As we are both working with same stuff here, should we both work on it here? Or are you planning to use FDroid's library completely through-and-through? |
Only support for diff parsing is missingOtherwise everything seems to be working the way I intend 🤔 I will add license and other references in few day to code similar to FDroid Library. |
|
I would like some help with Diff Parsing to speed up the process, I won't be able to allocate any time to this for some time now. |
|
Started checking & navigating the codes & docs. |
|
@Iamlooker So, just reporting: the most complexity seems in keeping the automated parsing and still using diff. Specially considering v2 diffs, the archived versions get
Another (mostly implausible) option is to use F-Droid's libs, but then it'll get pretty complex adding extensions (also considering the effort you put in this). So I'll wait your opinion, we can also discuss things in a call/chat. |
|
@machiav3lli Thanks for going through this messy code. About automated parsing, we can switch to old school Jackson parsing if we wanted to, its just rewriting these two classes: V1Parser & V2Parser Both the options sound good to me, but if I had to choose one it would be the second one, just duplicate the models, make some edits and its almost good to go (I am running away from reflection, they scare me now) Also about extensions, we can simply extend these classes like ExodusSyncable or ExodusParser in future to add extensions and make them optional based on user preferences, this would be really simple as far as I can think. You know me, my dm's are always open 😄 . We can even arrange a call at your preferable time. |
|
Aah and also I think switching to old school parsing will allow us to not create a so many models and just rely on some builders or something, idk. Or or or wait... we can write our own json parser, the fastest one yet using json-schema which is automated but also maps to current models and is just perfect? NVM 😅 |
Hehe, nah nah not that complicated. It's just defining Diff-Variations of the mentioned 3 classes and declaring patchers.
I've already written most of the logic locally and should be able to complete it and PR it before the weekend. It seems that sync:fdroid will have to be an android.library instead of jvm.library. I actually don't mind this, as I'd need anyway another implementation for Workers. Maybe we can in the future streamline things and make an independent lib out of this, but not for now.
It'd be even easier just adding the extensions to the respective serializable classes with empty/null defaults.
Actually Foxy-Droid's old-school parser is still the one used in NS, which is afair the last part of Foxy-Droid still existing in NS, but will be replaced to one using kotlinx.serialization in 1.1.0, which is part of my plan for Index-V2+. We'll talk about it in more details when I start officially working on it. |
|
PRed in #825 |
|
When parsing the JSON diff as JSON, how do you differentiate a missing value from an explicit |
|
@grote I solved this in a pretty hacky way (for now), defaulting values of the duplicate classes ( |
Right, but what if an app or a version got removed by setting it to |
Exactly.
in the current implementation it'll be kept :p (the scenario wasn't included in the diffs I tested, so it slipped through). So I guess the patchers will have to go one level deeper. Lists are one of the things I needed more information on (and didn't find). For example what diff we get when a mirror get updated: do we get the whole list updated? or only the updated mirror? The first one would increase the size of the diff (not relevant tbh), the second will lead to conflicts on the logic of patching. |
|
You may want to add test cases. Please observe licensing and attribution in case you copy test cases from the official v2 libs. |
The official F-Droid client recently gained support for delta repository index updates:
https://f-droid.org/2023/03/01/new-repo-format-faster-smaller-updates.html
It would be nice if Droid-ify could also use the entry.json file to speedup index updates.
The text was updated successfully, but these errors were encountered: