ARKit for Godot 3.2 #24227
ARKit for Godot 3.2 #24227
Conversation
BastiaanOlij
requested review from
leonkrause,
karroffel,
reduz and
vnen
as code owners
|
I'm not sure why Github decided to auto request all the reviews btw :) |
|
We use Github's file owners feature. It automatically asks review for PR that changes a of "owned" files. |
BastiaanOlij
force-pushed
the
arkit31
branch
2 times, most recently
from
04a8c76
to
ae21d9f
Compare
|
Loads of thanks to Arjen van Staalduinen who's been testing this on an iPad and emailing me patches to apply. I've not been able to test it out myself due to some hardware issues but this is really starting to work well by the looks of what Arjen has been doing. |
|
Todos that are left:
|
BastiaanOlij
force-pushed
the
arkit31
branch
2 times, most recently
from
bcc8dc7
to
94699ea
Compare
|
Ok, important bit of information we'll eventually need to add into the docs :) Godot has the ability to fix the orientation for mobile apps with the default set to landscape: ArKit does work best with a fixed orientation so this default works best but you can change it to sensor to have the 2D part of the UI match the orientation of the iPhone/iPad. Whichever you chose, you have to make sure that your xcode setting match else Godot will fix the orientation but ArKit will keep adjusting its orientation with some really funky results. So if you stick to the landscape default make sure your xcode project settings are as follows: |
BastiaanOlij
force-pushed
the
arkit31
branch
2 times, most recently
from
fbe2873
to
33df9b8
Compare
|
Ok, I'm moving ARKit into a module, easier to turn it on/off. Haven't finished moving it completely yet. Still WIP :) |
BastiaanOlij
force-pushed
the
arkit31
branch
3 times, most recently
from
d46046a
to
11585a7
Compare
| // Note, this also contains a scale factor which gives us an idea of the size of the anchor | ||
| // We may extract that in our ARVRAnchor class | ||
| Basis b; | ||
| matrix_float4x4 m44 = anchor.transform; |
There was a problem hiding this comment.
These lines are begging for a static inline helper
There was a problem hiding this comment.
lol, yeah it can be cleaned up a bit, I'll be doing that at the end when everything is working fine. I have a feeling there will be more changes here :)
|
Just rebased this and made a few small changes. I'm trying to make the focus in/focus out logic for pausing ARKit go through _notification as the notification system is being called for this but unsuccessfully so far. I'm guessing because ARVR Interfaces aren't nodes but more basic objects. Could use some help with solving this better (also impacts ARCore). @reduz any suggestions here? |
There was a problem hiding this comment.
I reviewed the relevant iOS portions. Everything looks good, but I can't quite understand the camera session stuff. It seems like we're duplicating some work that already exists in a more optimized fashion. Would love your input on this as I probably missed something.
| if (plane_detection_is_enabled != p_enable) { | ||
| plane_detection_is_enabled = p_enable; | ||
|
|
||
| // Restart our session (this will be ignore if we're not initialised) |
There was a problem hiding this comment.
The comment says initalised but the code reads session_was_started. Which is correct?
There was a problem hiding this comment.
Let me come back to you on that one once I have a moment to catch up :)
There was a problem hiding this comment.
Ok, so this is some weirdness around how phones work, it doesn't work fully yet because I haven't got notifications to work properly just yet.
Anyway, the problem lies around an app becoming dormant when you switch away from it. Basically ARKit needs to be doing its thing when you've initialized it AND we have focus.
So if ARKit is initialized before we obtain focus, we initialize ARKit but we don't start the session. Then when we obtain focus start_session is called and we start our ARKit session.
If we obtain focus before ARKit is initialized start_session is called but initialize is false so we don't do anything more, the session_was_started ensures that once we initialize we also setup the ARKit session.
If at any time after this has all been setup we loose focus stop_session is called and we simply pause our ARKit session and unpause it in start_session once focus is regained (the unpauze is actually a reinitialize as settings on the phone may have changed).
There was a problem hiding this comment.
I wonder if we should either hook the appDidResignActive and appDidBecomeActive events from iOS, or if we should hoist them up to Godot and provide other notifications for the window.
There was a problem hiding this comment.
They are now hooked up, these result in FOCUS IN and FOCUS OUT notifications and after I managed to get notifications to work with the ARVRServer the other day and that change was merged in, should now work with ARKit :)
| b.elements[1].z = m44.columns[2][1]; | ||
| b.elements[2].z = m44.columns[2][2]; | ||
| tracker->set_orientation(b); | ||
| tracker->set_rw_position(Vector3(m44.columns[3][0], m44.columns[3][1], m44.columns[3][2])); |
There was a problem hiding this comment.
Since we're missing the scale in the transform here, Won't that prevent ARKit from resizing the generated surface to match the environment?
There was a problem hiding this comment.
That is why we have set_rw_position (rw as in real world), the scale is applied later. This so that if the position of an anchor doesn't change, but you change the scale, we still get everything positioned correctly
There was a problem hiding this comment.
Hmm, now that I think of it, we do need to apply our scale to the mesh somehow or it will have the wrong size, need to think that one through... That might need to be solved in the anchor.tscn
|
Hey sam,
Already working on that, they are already hooked up, the problem is getting
the notifications send to the interface:)
On Fri, 19 Apr 2019 at 4:57 am, Sam Green ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In modules/arkit/arkit_interface.mm
<#24227 (comment)>:
> + stop_session();
+ }; break;
+ default:
+ break;
+ }
+}
+
+bool ARKitInterface::get_anchor_detection_is_enabled() const {
+ return plane_detection_is_enabled;
+}
+
+void ARKitInterface::set_anchor_detection_is_enabled(bool p_enable) {
+ if (plane_detection_is_enabled != p_enable) {
+ plane_detection_is_enabled = p_enable;
+
+ // Restart our session (this will be ignore if we're not initialised)
I wonder if we should either hook the appDidResignActive and
appDidBecomeActive events from iOS, or if we should hoist them up to
Godot and provide other notifications for the window.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#24227 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAO262K37JZQGXEIDJFXXX3PRDABVANCNFSM4GJGK3HA>
.
|
|
Forgot my dongle to attach my phone so I can't test this latest rebase but it compiles fine. Will test on Monday or hear back from one of the other if it still works. I've removed WIP from this because I feel this is pretty much done but bug fixing and polish. Before we can merge this however we do need to first approve and merge PR #10643 The only real question mark for this PR is around iOS versions, ARKit has a minimum of iOS 11 and a few of the newer functions in ARKit 2.0 need an even more up to date version of iOS. The other thing that needs to improve but which we might want to handle separately is including the ARKit framework automatically on export. |
|
Just rebased this, things run on my phone but having some issues still with the shadow thing, not sure why yet, still looking into it |
|
Just rebased, need to spend a few more hours cleaning stuff up and fixing the export so the arkit framework is added into the build settings. |
|
This compiled successfully for me, but I can't export to iOS as Godot is reporting damaged export templates. How to fix that? |
|
@form-follows-function I did a video on the process: https://www.youtube.com/watch?v=6NEonfH1ME0 That said, I had reports from someone else it does currently crash, I may have damaged something in the camera server while rebasing this but I haven't had time to investigate yet. |
|
Ok, this morning I spend a bit of time making sure this compiles and runs again. Somehow I messed something up when it doesn't find any cameras because we weren't looking for all types. It should also export ARKit projects properly when the ARKit tick is ticked. What I don't know is how well things behave if you're not using ARkit and not including it. That said if ARKit is unwanted to be included you can always turn off the module when compiling. I vote we start working towards merging this. There are plenty of improvements to be made but I think this is better served if its merged and more people can start picking things up. |
|
Same comment as for #10643 (comment), could you make the commit log a bit more descriptive so that it's ready to merge? Also it probably shouldn't refer to Godot 3.1 :) |
This PR introduces support for ARKit to the iOS version of Godot. ARKit is Apples Augmented Reality platform. This PR brings in support for ARKit 1.0 and implements a few ARKit 2.0 features. It requires iOS 11 to run but should not prevent Godot from running on older versions as long as ARKit remains unused.
|
Thanks a ton! |
|
What is the status on this? I saw there is a plugin that is kind of compatible with Godot 4, but this seems to be a native implementation, is this already inside Godot 4? Thanks! |
This PR sees the introduction of our ARKit interface on iOS. All the basic functionality now works. There is some polish needed most likely but....
Anyway this does need a proper example project, there is a simple test project that you can find here: https://github.com/BastiaanOlij/godot3_test_projects/tree/ARKit/ARKit
Since the class is only available on iOS its also not part of the documentation, I need to have a look into working around that. I might just include the class but strip out the code when not compiling for iOS
Todos:
Note that this PR includes the Camera server PR and BzztBombs PR. They should probably be merged first before merging this.