Repository navigation
Looking for gaps and disagreements... #1
Replies: 14 comments 16 replies
|
Repeating the enthusiasm for this as an effort that I expressed in the Briefcase forum - +1 to the general idea this is proposing; I need to find some time to dig into the specifics. I hope to be able to do that next week; my colleague @mhsmith is on leave at the moment, but should be able to provide more substantial Android feedback when he returns in early September. |
|
@freakboy3742 Instead of: The sidecar contains everything necessary for a build tool to correctly integrate the package. The updated target is: The sidecar automates the portable, repeatable parts of integration and tells the application author exactly what remains to be done. Effectively it is a way to put TODOs into the app's pyproject.toml. The sidecar simplifies to:
My expectation is this will make the spec easier to understand and maintain. I was jumping through too many hoops to add things the app author was better positioned to do. This will reduce the complexity of authoring a sidecar. I'll let you know when the spec is ready for review. Thanks for your patience. |
|
@freakboy3742 The spec is now ready for review. The readme and spec have both been updated. I look forward to your feedback. |
|
Thanks for looking into this. The specification definitely addresses some important use cases, but I think it's a bit over-ambitious with its scope, and overlaps several things which are better dealt with in other ways:
So the main thing that leaves is app-level configuration items in various files. As Russell mentioned here, there are a large number of these and they're frequently changing, so maintaining a mapping of them to TOML format would be a significant maintenance burden. Instead, the approach we've taken with Briefcase is to provide extension points in the relevant parts of each file, where arbitrary string content can be inserted in its original format. We use this most extensively on Android: see the "extra_content" settings. The set of extension points is likely to change much more slowly than the set of things that can be inserted there, so I think it would make sense for us to try and standardize that across multiple app build tools. |
|
@mhsmith So I currently see the roles as:
There is some conceptual overlaps around declaring non-Python dependencies, but I don’t yet see a PEP 725 mechanism that replaces the application’s integration metadata or materially simplifies it. PEP 725 provides standardized metadata for a Python project’s external build and runtime dependencies. Its build/host distinction in is describes what must be present to build a Python distribution, including the cross-compilation case. Native Integration describes a different relationship: what a Python package contributes to, requires from, or asks of the native application that contains it. For example, KivMob needs the Google Mobile Ads Maven dependency, but that is only one part of its native integration. It also includes the Python code that exposes that native functionality, contributes Java source and Android permissions, requires the application to supply an AdMob application ID at a particular manifest location, and may require actions from the application author. Those things collectively form the package's integration contract with the application. There is clearly some overlap at the edges — a Maven or SwiftPM dependency is both an external dependency and a native application contribution. But I don't yet see how PEP 725 addresses the broader Native Integration use cases such as permissions, contributed native source, manifest/plist contributions, application-supplied values, ownership/collision handling, or application actions. If you have specific ideas of how PEP 725 can be used to represent those cases, I'd be interested in understanding it. That might help identify whether there is useful overlap I'm missing. |
|
Looking at The goal of Native Integration is for those requirements to travel with the Python package itself, so a consumer such as Briefcase or another application builder can discover them and compose them into the application. The application author should only need to deal with the parts that actually require application-specific decisions or values. A Native Integration |
|
@mhsmith The floors exist in the sidecar because of what a build tool does with them, not because of anything pip needs. Once a compliant built tool resolves the dependency closure, it configures one Gradle build for every native SDK the closure pulled in — reading min_sdk, compile_sdk, target_sdk, and deployment_target off each sidecar and comparing them against what the app author configured. If the app is set too low, it fails and names the specific package that needed more, before Gradle or Xcode fails two layers down with an error that names no package at all. That's why min_sdk/deployment_target aren't redundant with a wheel tag: a wheel tag describes the package's own compiled code, if it has any; these floors describe what its native dependency needs, resolved later by Gradle or SwiftPM, in a process pip never sees. Your feedback had me review these elements - but I'm left with them staying in place. Do let me know if I have misunderstood something. EDIT: I'm removing |
|
Thank you for your patience and tenacity. TL;DR: I agree with the direction, and I'll update the proposal to integrate your input. It will take me a few weeks, and I'll let you know when it's ready. Below is a short summary of each point and why I came around.
Given all of this, the config a package author writes goes into the package's pyproject.toml, and the sidecar disappears. The content is generated into The key element that remains is Thanks again. |
|
Firstly - sincere apologies for the delay in responding to this. Responding to the three specific questions you've raised in the original post:
It may be better to look at this with PEP 621 as an analog - an initial spec that describes the basic shape of the metadata (e.g., where it's defined, and the expectation for how it will be consumed), and some basic metadata that everyone can agree on - maybe something as simple as permissions in the first instance - but also including a "safe" namspace for tools to experiment further, and an understanding of how shared improvements can be added over time.
In terms of what seems right - it makes sense for me for a library to have concrete requirements that an app must satisfy, such as the library needing the app to declare a permission or an intent for Android purposes. However, there's limits to what a library can contribute to that conversation. A library can't provide a meaningful "reason" for a permission; it can only declare that the permission requirement exists. The requirements for things like an intent declaration are both platform specific, and likely version specific within that platform. I'm less certain about the flow of information the other way. True, an advertising library can't specify an AdMob API key. However that sort of data could be provided by a Python package (the package for "the app"); in the context of an "Android/iOS project that incorporates some Python code", it could be a "meta" package that publishes that data into a "package-like" format as part of the packaging process. It's also unclear to me if the S2.4 "obligations on the consumer's bootstrap" are appropriate in this context - or, at least, that they point at a much more fundamental issue that needs to be addressed. As a practical manifestation of this - Kivy-focussed libraries use PyJNIus for Android integration and PyObjUs for iOS; BeeWare library tend to use Chaquopy and Rubicon ObjC for the same platforms respectively. Having hard specifications for how code and interfaces "will" be used when it's not possible to import Kivy and Chaquopy using libraies into the same namespace seems premature. There's a separate piece of work here as well around ensuring we have platform compatibility; without that piece, making hard declarations about how an app will or won't integrate with app lifecycle seems premature.
Outside of those specific questions, I also share some of the concerns @mhsmith's has already raised aroud the use of entry points, including source, and PEP 725 compatibility. Based on your most recent comment, it sounds like you agree as well; so I guess I'll wait for your next revision. The tl;dr of all this - I applaud the ambition, but I think it's a little too ambitious to succeed (at least in its current form, anyway). I feel that, as a standardization effort, it would benefit from picking a much smaller target with specific immediate goals and a path for future expansion (and a clear convention for experimentation) - and possibly being broken up in to a series of smaller specs, rather than one huge spec. |
|
@mhsmith @freakboy3742 Before I put more effort into the full proposal, I'd like to know whether this direction looks right to you. ObjectiveLet a Python package that wraps an Android or iOS SDK tell the tool building the application which native dependencies it needs and what the application author must do. Updated ProposalA package that wraps an Android or iOS SDK declares two things in the
# pyadmob's pyproject.toml
[tool.native-integration.android.gradle]
dependencies = ["com.google.android.gms:play-services-ads:24.0.0"]
[[tool.native-integration.android.application_action]]
id = "admob-app-id"
description = """\
Set the manifest meta-data com.google.android.gms.ads.APPLICATION_ID to your \
AdMob app ID."""
reason = 'Without this tag the app crashes with "Missing application ID".'
acceptance = ["The app starts without that crash."]
[tool.native-integration.ios.swift_packages]
GoogleMobileAds = { url = "https://github.com/googleads/swift-package-manager-google-mobile-ads", requirement = { from = "12.0.0" }, products = ["GoogleMobileAds"] }
[[tool.native-integration.ios.application_action]]
id = "admob-app-id"
description = "Set GADApplicationIdentifier in Info.plist to your AdMob app ID."
reason = "Google's iOS setup requires this key, and only you have the ID."
acceptance = ["Info.plist has GADApplicationIdentifier set to your iOS app ID."]A build tool (Briefcase, kivyforge) reads this from each installed package and passes the dependencies to Gradle or SwiftPM, much as an installer handles Requires-Dist. How it does that is its choice, including whether it records them in pyproject.toml or a lock file. The build tool also appends each new action to the application's # Added by kivyforge. Requested by pyadmob.
# Set the manifest meta-data com.google.android.gms.ads.APPLICATION_ID to your AdMob app ID.
# Why: Without this tag the app crashes with "Missing application ID".
# Acceptance: The app starts without that crash.
[tool.native-integration.ack.pyadmob.android.admob-app-id]
response = "Set in the AndroidManifest template."The response is an acknowledgement, not a claim that the action was taken. "Not using the scanner, so no camera access is needed" is a valid answer. The tool reports every empty response on every build. Whether an empty response fails the build (or warns) Every action is manual in v1. No tool writes on an Permissions are not in this metadata. If a package needs one its SDK doesn't declare, it uses an This proposal follows Russell's PEP 621 suggestion: it defines where the data lives and how it is read, and keeps the initial metadata small. A change on PEP 725Previously, I wrote "Agreed and adopted" about PEP 725. I'm walking that back. I tried converting the 18 example packages, 15 of them hypothetical wrappers around real SDKs, and it is better to leave these mobile dependencies in their native format. DepURLs are meant to be portable across packaging ecosystems, and virtual packages let each ecosystem choose a provider. That fits system dependencies. Here each platform has one resolver, and a Gradle coordinate or Swift URL names exactly one artifact. A tool would only convert the DepURL back again. All 17 Gradle dependencies convert, but gain nothing. A Swift dependency has no complete DepURL: SwiftPM links products, and PURL's I take PEP 725's flow from Where I'd like your view
And of course any other feedback is welcomed. None of this has been built. |
|
@freakboy3742 Thanks for your continued engagement and inputs.
I'll keep this in mind as this progresses from "scope" to a fleshed out proposal.
Agreed. There is value in the layout of the metadata in I'll render a maven and swift "coordinates" in a follow up message. Perhaps an update to 725 that better supports mobile and addresses the issue with swift is a better path.
Conflict resolution is not covered in PEP 725. Gradle and Xcode each have a way of dealing with conflicts. The app build tool (Briefcase, kivyforge, etc.) can choose to detect conflicts before handing off, or leave them to Gradle/Xcode. I’d suggest the spec recommend the former, because the app build tool knows which Python distribution declared each dependency and can name it, where a Gradle or Xcode error can’t. To be clear, the dependencies aren’t passed through as raw build-file text. Each is a structured native coordinate (Maven group/artifact/version; Swift package URL, version requirement and product), so the app build tool can recognize when two distributions declare the same artifact, merge them into a single declaration, and report the conflict against the distributions that caused it. Resolving the versions themselves is still left to Gradle and SwiftPM.
I like this framing a lot. It turns "dependencies plus a TODO list" into a container with a clear path to grow. As I understand it:
Does that match what you had in mind? If so, I'll restructure the proposal around it, with permissions as the worked example.
reason explains why this application_action is being requested. I was thinking about this in the context of the plyer library https://github.com/kivy/plyer. plyer is an abstraction layer that covers a broad set of capabilities (camera, bluetooth, vibrator...) rather than just request all of the supported capabilities, each would have an associated reason. This is a text field that explains why a library is requesting a capability from the app author. acceptance The concept here was to provide a coding agent the acceptance criteria to support an automated test. The acceptance lists what can be checked to ensure the request was satisfied when examining the built artifact or runtime behavior. This is an optional field. response This is for any requirement the tool cannot verify on its own. This is effectively a Boolean with documentation. An empty entry causes the build tool to react to the entry as an unsatisfied request. The text describes how the app author decided to satisfy the request and provide that as documentation. I understand your pushback on having the build tool modifying the pyproject.toml, I'll change that. The idea was the response to the request needs to be there - but how it gets there does not need to be in the spec, and it is better to have the author remain the editor. I've run out of time this morning... Thanks again! |
PEP 725 Discussion@freakboy3742 @mhsmith Here are the renderings I promised, using PEP 725 DepURLs. Gradle coordinates fit, except BOMs (below). All 17 in the examples convert: I use Swift dependencies do not fit. The URL and version convert, but SwiftPM links products, and PURL's What would need to changeThrough DepURL changes or a native expression in PEP 725:
I'd prefer a native expression. Authors can copy coordinates from the vendor's documentation and tools can pass them on unchanged, whereas converting by hand only adds chances for error. PEP 725 lets later PEPs add fields to |
|
#Follow up on PEP 725 Versioning
Guava shows the effect. It specifies its flavor in the Maven version, so an Android app needs The last one is accepted but no longer says Android, and Maven Central has no plain PURL isn't the limit: So writing this dependency as a DepURL needs a change to PEP 725. |
Updated scope and worked example@freakboy3742 @mhsmith Here is the updated scope from our discussion so far, and a worked example. None of this is built, except where I say I tested something. Scopev1 is intentionally small. It covers four things:
An action needs Worked exampleAn app uses plyer for GPS and pyadmob for ads, and builds for Android, then iOS. Both packages are hypothetical here: plyer is real but has no such metadata, so this shows how it could adopt the spec, and pyadmob is a wrapper from my examples. It shows how:
Assumed workflow
1. plyer: features and permissionsplyer calls platform APIs directly and ships no AAR, so nothing carries its permissions into the app. Its GPS and vibrator notes list what the app needs, so it could declare: # pyproject.toml for a plyer that adopts the spec
[[tool.native-integration.android.application_action]]
id = "gps"
feature = "gps"
revision = 1
kind = "permission"
names = [
"android.permission.ACCESS_FINE_LOCATION",
"android.permission.ACCESS_COARSE_LOCATION",
]
description = "Declare these permissions in the manifest."
[[tool.native-integration.android.application_action]]
id = "vibrator"
feature = "vibrator"
revision = 1
kind = "permission"
names = ["android.permission.VIBRATE"]
description = "Declare this permission in the manifest."The tool reads the feature names from the actions. Until the app has decided, the tool reports what each feature brings and a block to paste: The author pastes the block and keeps what the app uses: # The app's pyproject.toml
[tool.native-integration.features]
plyer = ["gps"]The app didn't select
For the features the app selected, the tool adds the permissions and says so: A tool that doesn't implement 2. pyadmob: dependencies, and what only the app can do# pyadmob's pyproject.toml
[tool.native-integration.android.gradle]
dependencies = ["com.google.android.gms:play-services-ads:[24.0,25.0[!!24.0.0"]
[[tool.native-integration.android.application_action]]
id = "admob-app-id"
revision = 1
description = "Set the manifest meta-data com.google.android.gms.ads.APPLICATION_ID to your AdMob app ID."The Gradle version is Gradle's own strict syntax: the range the binding supports, and the version it prefers. The tool passes it on unchanged. pyadmob needs no permission action, because Google's AAR already declares the permissions it uses. This action has no The author does the work with what the tool already provides, then pastes the block, and can add a comment: # The app's pyproject.toml
[tool.native-integration.ack.pyadmob.android]
# The value is the revision of the action being acknowledged.
admob-app-id = 1 # Set in the AndroidManifest template.The author's comment is for whoever maintains the app later. 3. UpgradesIf a new pyadmob changes what the action asks for, it sets The report stays until the author changes the value to 4. Tool-defined kindsA tool can define its own kinds under [[tool.native-integration.android.application_action]]
id = "sdk-flag"
revision = 1
kind = "tool.briefcase.manifest_application_content"
content = '<meta-data android:name="com.example.sdk.FLAG" android:value="true" />'
description = 'Add <meta-data android:name="com.example.sdk.FLAG" android:value="true" /> to the manifest application element.'Briefcase could insert the content. Any other tool reports the 5. Two packages, one native dependencyThe tool can give each Python distribution its own Gradle module, named after the distribution, holding only that distribution's coordinates. Gradle does all the resolving, and its errors name the distributions. I tested two modules with ranges that don't overlap: // pkg-a/build.gradle
dependencies { api 'com.example:foobar:[1.0,2.0[!!1.4' }
// pkg-b/build.gradle
dependencies { api 'com.example:foobar:[2.0,3.0[!!2.5' }Here With This was Gradle 8.11.1 with plain Java modules. I haven't tested it with the Android Gradle Plugin. 6. What's different on iOSActions, features, acknowledgements and revisions work the same way. A selection applies to both platforms, so Permissions: an iOS permission needs a purpose string, and only the app can write it. So the tool checks that the key is set rather than adding it: # pyproject.toml for a plyer that adopts the spec
[[tool.native-integration.ios.application_action]]
id = "gps"
feature = "gps"
revision = 1
kind = "permission"
names = ["NSLocationWhenInUseUsageDescription"]
description = "Add this key to Info.plist, saying why your app uses location."The tool prints a block for its own configuration, here kivyforge's: Briefcase would print its Dependencies: a Swift package names the products to link. Its # pyadmob's pyproject.toml
[tool.native-integration.ios.swift_packages]
GoogleMobileAds = { url = "https://github.com/googleads/swift-package-manager-google-mobile-ads", requirement = { from = "12.0.0" }, products = ["GoogleMobileAds"] }
[[tool.native-integration.ios.application_action]]
id = "admob-app-id"
revision = 1
description = "Set GADApplicationIdentifier in Info.plist to your AdMob app ID."The tool reports this action as it does on Android: The author pastes the block once the work is done: # The app's pyproject.toml
[tool.native-integration.ack.pyadmob.ios]
# The value is the revision of the action being acknowledged.
admob-app-id = 1 # Set in Info.plist.Where this growsAreas to explore during spec development:
FeedbackThe example adds:
Does this match what you had in mind, and what would you change? |
Uh oh!
There was an error while loading. Please reload this page.
I've been working on a mobile build tool for python. I have discovered a gap.
There is currently no standard way to integrate native mobile content (Java, swift...)
into a python wheel - but there is a need. This spec is my attempt at closing this gap.
It's a draft, nothing is implemented, and it's deliberately not one tool's
feature — a convention only one tool reads isn't worth defining.
Reading order
Don't start with the spec — it's ~17k words.
README first, then
one example:
pystripe
to see both halves of one integration,
Appendix D
is a one-line description of every key if you'd rather scan the surface.
What I'd genuinely like torn apart
1. Is the declaration set right — what's missing, what's unnecessary?
2. Does the "app keeps authority" split land in the right place?
A package may request a permission or register a component, but only the app
may mark a feature
requiredor a componentexported, and the app can suppressany contributed permission. A package can never write an entitlement, and never
writes an iOS purpose string — that text is App Store reviewed and the app
answers for it.
3. If you maintain a build tool: is this something you'd plausibly read?
There are 26 numbered obligations on a consuming tool in §8. If one of them is
unimplementable in your architecture, that's the most valuable thing you could
tell me.
All reactions