Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Convert APIs to Kotlin DSL #298

Draft
wants to merge 46 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
3760cb2
feat: Convert patches to Kotlin DSL
oSumAtrIX Mar 7, 2024
d0cbbac
Merge branch 'dev' into feat/dsl-api
oSumAtrIX Apr 24, 2024
8a64ccf
feat: Add fingerprints to DSL and finish patches DSL
oSumAtrIX Apr 25, 2024
c5f02e8
chore: Add tests, fix running patches, add DSL for patch loader
oSumAtrIX Apr 26, 2024
4088609
fix: Load functional patches correctly
oSumAtrIX Apr 27, 2024
e4e4c2a
feat: Use a builder for declaring compatible packages
oSumAtrIX Apr 27, 2024
347cda6
feat: Use a builder for declaring dependencies
oSumAtrIX Apr 27, 2024
c84c0e0
feat: Simplify options API
oSumAtrIX Apr 27, 2024
cf59950
fix: Move annotation to DSL api
oSumAtrIX Apr 27, 2024
85c83e0
fix: Modernize some APIs
oSumAtrIX Apr 28, 2024
2503283
docs: Update docs to match new changes
oSumAtrIX Apr 28, 2024
25d1b65
docs: Fix mistakes
oSumAtrIX Apr 28, 2024
c2b4f2c
docs: Fix mistakes
oSumAtrIX Apr 28, 2024
ea5b76e
docs: Improve wording
oSumAtrIX Apr 28, 2024
d6012a4
docs: Add note
oSumAtrIX Apr 28, 2024
7fee767
feat: Use vararg for access flags
oSumAtrIX Apr 28, 2024
503078e
use vararg instead of builder
oSumAtrIX Apr 29, 2024
21c2a86
sync docs
oSumAtrIX Apr 29, 2024
58c61f7
fix navigator
oSumAtrIX Apr 30, 2024
55bba4b
fix proxy
oSumAtrIX Apr 30, 2024
59be336
Add new string package overload
oSumAtrIX May 26, 2024
875ea5b
docs: Recommend a better structure for patches
oSumAtrIX May 27, 2024
a5060b5
feat: Retain generic option type parameter
oSumAtrIX May 29, 2024
3b448d3
feat: Do not overwrite compatible packages or dependencies when build…
oSumAtrIX May 30, 2024
74fa410
feat: Add ability to navigate using predicate
oSumAtrIX Jun 1, 2024
d747c98
feat: Convert extension functions to properties, add extensions for i…
oSumAtrIX Jun 1, 2024
7221f9d
fix docs mistake
oSumAtrIX Jun 10, 2024
27794c9
fix docs mistake
oSumAtrIX Jun 10, 2024
0403287
Fix docs on how to add fingerprints while delegating result
oSumAtrIX Jun 16, 2024
e36e959
dump api
oSumAtrIX Jun 16, 2024
eb4db5a
refactor fingerprint apis
oSumAtrIX Jun 17, 2024
37fbef6
add todo
oSumAtrIX Jun 17, 2024
40926b8
docs: Simplify
oSumAtrIX Jun 17, 2024
1caf522
fix test by mocking missing property
oSumAtrIX Jun 17, 2024
fe5bcb3
simplify patcher api
oSumAtrIX Jun 17, 2024
a4670dc
simplify docs
oSumAtrIX Jun 17, 2024
e2bc302
fix docs link
oSumAtrIX Jun 17, 2024
3776599
add field to patch loader to get patches by bundle
oSumAtrIX Jun 18, 2024
4affb65
simplify api
oSumAtrIX Jun 18, 2024
dd7a5bd
Merge branch 'refs/heads/feat/separate-bundle-in-loader' into feat/in…
oSumAtrIX Jun 18, 2024
44fa0ce
Add per patch extensions
oSumAtrIX Jun 18, 2024
e473e59
Merge branch 'refs/heads/feat/separate-bundle-in-loader' into feat/ds…
oSumAtrIX Jun 18, 2024
d79012d
Merge branch 'refs/heads/feat/integrations-merge' into feat/dsl-api
oSumAtrIX Jun 18, 2024
1d16c88
docs: Use new extension
oSumAtrIX Jun 18, 2024
dfbb70f
simplfiy function name
oSumAtrIX Jun 18, 2024
8821b6b
Make accessing delegated match safer
oSumAtrIX Jun 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
575 changes: 304 additions & 271 deletions api/revanced-patcher.api

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ dependencies {
// Exclude, otherwise the org.w3c.dom API breaks.
exclude(group = "xerces", module = "xmlParserAPIs")
}

testImplementation(libs.kotlin.test)
testImplementation(libs.mockk)
}

kotlin {
Expand Down
53 changes: 28 additions & 25 deletions docs/1_patcher_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,40 +60,43 @@

# 💉 Introduction to ReVanced Patcher

In order to create patches for Android applications, you first need to understand the fundamentals of ReVanced Patcher.
To create patches for Android apps, it is recommended to know the basic concept of ReVanced Patcher.

## 📙 How it works

ReVanced Patcher is a library that allows you to modify Android applications by applying patches to their APKs. It is built on top of [Smali](https://github.com/google/smali) for bytecode manipulation and [Androlib (Apktool)](https://github.com/iBotPeaches/Apktool) for resource decoding and encoding.
ReVanced Patcher accepts a list of patches and integrations, and applies them to a given APK file. It then returns the modified components of the APK file, such as modified dex files and resources, that can be repackaged into a new APK file.
ReVanced Patcher is a library that allows modifying Android apps by applying patches.
It is built on top of [Smali](https://github.com/google/smali) for bytecode manipulation and [Androlib (Apktool)](https://github.com/iBotPeaches/Apktool)
for resource decoding and encoding.

ReVanced Patcher has a simple API that allows you to load patches and integrations from JAR files and apply them to an APK file.
Later on, you will learn how to create patches.
ReVanced Patcher receives a list of patches and applies them to a given APK file.
It then returns the modified components of the APK file, such as modified dex files and resources,
that can be repackaged into a new APK file.

ReVanced Patcher has a simple API that allows you to load patches from RVP (JAR or DEX container) files
and apply them to an APK file. Later on, you will learn how to create patches.

```kt
// Executed patches do not necessarily reset their state.
// For that reason it is important to create a new instance of the PatchBundleLoader
// once the patches are executed instead of reusing the same instance of patches loaded by PatchBundleLoader.
val patches: PatchSet /* = Set<Patch<*>> */ = PatchBundleLoader.Jar(File("revanced-patches.jar"))
val integrations = setOf(File("integrations.apk"))
val patches = loadPatchesFromJar(setOf(File("revanced-patches.rvp")))

val patcherResult = Patcher(PatcherConfig(apkFile = File("some.apk"))).use { patcher ->
// Here you can access metadata about the APK file through patcher.context.packageMetadata
// such as package name, version code, version name, etc.

// Instantiating the patcher will decode the manifest of the APK file to read the package and version name.
val patcherConfig = PatcherConfig(apkFile = File("some.apk"))
val patcherResult = Patcher(patcherConfig).use { patcher ->
patcher.apply {
acceptIntegrations(integrations)
acceptPatches(patches)
// Add patches.
patcher += patches

// Execute patches.
runBlocking {
patcher.apply(returnOnError = false).collect { patchResult ->
if (patchResult.exception != null)
println("${patchResult.patchName} failed:\n${patchResult.exception}")
else
println("${patchResult.patchName} succeeded")
}
// Execute the patches.
runBlocking {
patcher().collect { patchResult ->
if (patchResult.exception != null)
logger.info("\"${patchResult.patch}\" failed:\n${patchResult.exception}")
else
logger.info("\"${patchResult.patch}\" succeeded")
}
}.get()
}

// Compile and save the patched APK file components.
patcher.get()
}

// The result of the patcher contains the modified components of the APK file that can be repackaged into a new APK file.
Expand Down
3 changes: 2 additions & 1 deletion docs/2_1_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,8 @@ Throughout the documentation, [ReVanced Patches](https://github.com/revanced/rev
3. Open the project in your IDE

> [!TIP]
> It is a good idea to set up a complete development environment for ReVanced, so that you can also test your patches by following the [ReVanced documentation](https://github.com/ReVanced/revanced-documentation).
> It is a good idea to set up a complete development environment for ReVanced, so that you can also test your patches
> by following the [ReVanced documentation](https://github.com/ReVanced/revanced-documentation).

## ⏭️ What's next

Expand Down
198 changes: 101 additions & 97 deletions docs/2_2_1_fingerprinting.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,10 @@

# 🔎 Fingerprinting

In the context of ReVanced, fingerprinting is primarily used to resolve methods with a limited amount of known information.
In the context of ReVanced, fingerprinting is primarily used to match methods with a limited amount of known information.
Methods with obfuscated names that change with each update are primary candidates for fingerprinting.
The goal of fingerprinting is to uniquely identify a method by capturing various attributes, such as the return type, access flags, an opcode pattern, strings, and more.
The goal of fingerprinting is to uniquely identify a method by capturing various attributes, such as the return type,
access flags, an opcode pattern, strings, and more.

## ⛳️ Example fingerprint

Expand All @@ -72,14 +73,14 @@ Throughout the documentation, the following example will be used to demonstrate

package app.revanced.patches.ads.fingerprints

object ShowAdsFingerprint : MethodFingerprint(
returnType = "Z",
accessFlags = AccessFlags.PUBLIC or AccessFlags.FINAL,
parameters = listOf("Z"),
opcodes = listOf(Opcode.RETURN),
strings = listOf("pro"),
customFingerprint = { (methodDef, classDef) -> methodDef.definingClass == "Lcom/some/app/ads/AdsLoader;" }
)
fingerprint {
accessFlags(AccessFlags.PUBLIC, AccessFlags.FINAL)
returns("Z")
parameters("Z")
opcodes(Opcode.RETURN)
strings("pro")
custom { (method, classDef) -> method.definingClass == "Lcom/some/app/ads/AdsLoader;" }
}
```

## 🔎 Reconstructing the original code from a fingerprint
Expand All @@ -91,22 +92,22 @@ The fingerprint contains the following information:
- Method signature:

```kt
returnType = "Z",
access = AccessFlags.PUBLIC or AccessFlags.FINAL,
parameters = listOf("Z"),
accessFlags(AccessFlags.PUBLIC, AccessFlags.FINAL)
returns("Z")
parameters("Z")
```

- Method implementation:

```kt
opcodes = listOf(Opcode.RETURN)
strings = listOf("pro"),
opcodes(Opcode.RETURN)
strings("pro")
```

- Package and class name:

```kt
customFingerprint = { (methodDef, classDef) -> methodDef.definingClass == "Lcom/some/app/ads/AdsLoader;"}
custom = { (method, classDef) -> method.definingClass == "Lcom/some/app/ads/AdsLoader;"}
```

With this information, the original code can be reconstructed:
Expand All @@ -129,144 +130,147 @@ With this information, the original code can be reconstructed:

> [!TIP]
> A fingerprint should contain information about a method likely to remain the same across updates.
> A method's name is not included in the fingerprint because it is likely to change with each update in an obfuscated app. In contrast, the return type, access flags, parameters, patterns of opcodes, and strings are likely to remain the same.
> A method's name is not included in the fingerprint because it will likely change with each update in an obfuscated app.
> In contrast, the return type, access flags, parameters, patterns of opcodes, and strings are likely to remain the same.

## 🔨 How to use fingerprints

After creating a fingerprint, add it to the constructor of a `BytecodePatch`:
Fingerprints can be added to a patch by directly creating and adding them or by invoking them manually.
Fingerprints added to a patch are matched by ReVanced Patcher before the patch is executed.

```kt
object DisableAdsPatch : BytecodePatch(
setOf(ShowAdsFingerprint)
) {
val fingerprint = fingerprint {
// ...
}
```
}

> [!NOTE]
> Fingerprints passed to the constructor of `BytecodePatch` are resolved by ReVanced Patcher before the patch is executed.
val patch = bytecodePatch {
// Directly create and add a fingerprint.
fingerprint {
// ...
}

> [!TIP]
> Multiple patches can share fingerprints. If a fingerprint is resolved once, it will not be resolved again.
// Add a fingerprint manually by invoking it.
fingerprint()
}
```

> [!TIP]
> If a fingerprint has an opcode pattern, you can use the `FuzzyPatternScanMethod` annotation to fuzzy match the pattern.
> Opcode pattern arrays can contain `null` values to indicate that the opcode at the index is unknown.
> Any opcode will match to a `null` value.

> [!WARNING]
> If the fingerprint can not be resolved because it does not match any method, the result of a fingerprint is `null`.
> Multiple patches can share fingerprints. If a fingerprint is matched once, it will not be matched again.

Once the fingerprint is resolved, the result can be used in the patch:
> [!TIP]
> If a fingerprint has an opcode pattern, you can use the `fuzzyPatternScanThreshhold` parameter of the `opcode`
> function to fuzzy match the pattern.
> `null` can be used as a wildcard to match any opcode:
>
> ```kt
> fingerprint(fuzzyPatternScanThreshhold = 2) {
> opcodes(
> Opcode.ICONST_0,
> null,
> Opcode.ICONST_1,
> Opcode.IRETURN,
> )
>}
> ```

Once the fingerprint is matched, the match can be used in the patch:

```kt
object DisableAdsPatch : BytecodePatch(
setOf(ShowAdsFingerprint)
) {
override fun execute(context: BytecodeContext) {
val result = ShowAdsFingerprint.result
?: throw PatchException("ShowAdsFingerprint not found")

val patch = bytecodePatch {
// Add a fingerprint and delegate its match to a variable.
val match by showAdsFingerprint()
val match2 by fingerprint {
// ...
}

execute {
val method = match.method
val method2 = match2.method
}
}
```

The result of a fingerprint that resolved successfully contains mutable and immutable references to the method and the class it is defined in.
> [!WARNING]
> If the fingerprint can not be matched to any method, the match of a fingerprint is `null`. If such a match is delegated
> to a variable, accessing it will raise an exception.

The match of a fingerprint contains mutable and immutable references to the method and the class it matches to.

```kt
class MethodFingerprintResult(
class Match(
val method: Method,
val classDef: ClassDef,
val scanResult: MethodFingerprintScanResult,
val patternMatch: Match.PatternMatch?,
val stringMatches: List<Match.StringMatch>?,
// ...
) {
val mutableClass by lazy { /* ... */ }
val mutableMethod by lazy { /* ... */ }

// ...
}

class MethodFingerprintScanResult(
val patternScanResult: PatternScanResult?,
val stringsScanResult: StringsScanResult?,
) {
class StringsScanResult(val matches: List<StringMatch>) {
class StringMatch(val string: String, val index: Int)
}

class PatternScanResult(
val startIndex: Int,
val endIndex: Int,
// ...
) {
// ...
}
}
```

## 🏹 Manual resolution of fingerprints
## 🏹 Manual matching of fingerprints

Unless a fingerprint is added to the constructor of `BytecodePatch`, the fingerprint will not be resolved automatically by ReVanced Patcher before the patch is executed.
Instead, the fingerprint can be resolved manually using various overloads of the `resolve` function of a fingerprint.
Unless a fingerprint is added to a patch, the fingerprint will not be matched automatically by ReVanced Patcher
before the patch is executed.
Instead, the fingerprint can be matched manually using various overloads of a fingerprint's `match` function.

You can resolve a fingerprint in the following ways:
You can match a fingerprint the following ways:

- On a **list of classes**, if the fingerprint can resolve on a known subset of classes
- In a **list of classes**, if the fingerprint can match in a known subset of classes

If you have a known list of classes you know the fingerprint can resolve on, you can resolve the fingerprint on the list of classes:
If you have a known list of classes you know the fingerprint can match in,
you can match the fingerprint on the list of classes:

```kt
override fun execute(context: BytecodeContext) {
val result = ShowAdsFingerprint.also { it.resolve(context, context.classes) }.result
?: throw PatchException("ShowAdsFingerprint not found")

// ...
}
execute { context ->
val match = showAdsFingerprint.apply {
match(context, context.classes)
}.match ?: throw PatchException("No match found")
}
```

- On a **single class**, if the fingerprint can resolve on a single known class
- In a **single class**, if the fingerprint can match in a single known class

If you know the fingerprint can resolve to a method in a specific class, you can resolve the fingerprint on the class:
If you know the fingerprint can match a method in a specific class, you can match the fingerprint in the class:

```kt
override fun execute(context: BytecodeContext) {
execute { context ->
val adsLoaderClass = context.classes.single { it.name == "Lcom/some/app/ads/Loader;" }

val result = ShowAdsFingerprint.also { it.resolve(context, adsLoaderClass) }.result
?: throw PatchException("ShowAdsFingerprint not found")

// ...
val match = showAdsFingerprint.apply {
match(context, adsLoaderClass)
}.match ?: throw PatchException("No match found")
}
```

- On a **single method**, to extract certain information about a method
- Match a **single method**, to extract certain information about it

The result of a fingerprint contains useful information about the method, such as the start and end index of an opcode pattern or the indices of the instructions with certain string references.
The match of a fingerprint contains useful information about the method, such as the start and end index of an opcode pattern
or the indices of the instructions with certain string references.
A fingerprint can be leveraged to extract such information from a method instead of manually figuring it out:

```kt
override fun execute(context: BytecodeContext) {
val adsFingerprintResult = ShowAdsFingerprint.result
?: throw PatchException("ShowAdsFingerprint not found")

val proStringsFingerprint = object : MethodFingerprint(
strings = listOf("free", "trial")
) {}

proStringsFingerprint.also {
it.resolve(context, adsFingerprintResult.method)
}.result?.let { result ->
result.scanResult.stringsScanResult!!.matches.forEach { match ->
execute { context ->
val proStringsFingerprint = fingerprint {
strings("free", "trial")
}

proStringsFingerprint.apply {
match(context, adsFingerprintMatch.method)
}.match?.let { match ->
match.stringMatches.forEach { match ->
println("The index of the string '${match.string}' is ${match.index}")
}

} ?: throw PatchException("pro strings fingerprint not found")
} ?: throw PatchException("No match found")
}
```

> [!TIP]
> To see real-world examples of fingerprints, check out the [ReVanced Patches](https://github.com/revanced/revanced-patches) repository.
> To see real-world examples of fingerprints,
> check out the repository for [ReVanced Patches](https://github.com/revanced/revanced-patches).

## ⏭️ What's next

Expand Down
Loading
Loading