602.srt

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:00:00.506 --> 00:00:10.706 A:middle
[ Silence ]

00:00:11.206 --> 00:00:11.886 A:middle
>> Good morning everyone.

00:00:12.496 --> 00:00:14.316 A:middle
My name is Tony Guetta
and I'm the Manager

00:00:14.316 --> 00:00:15.576 A:middle
in the Core Audio
group at Apple.

00:00:15.716 --> 00:00:17.186 A:middle
And today, I'm going
to talk to you about,

00:00:17.186 --> 00:00:21.356 A:middle
What's New in Core
Audio for iOS.

00:00:21.476 --> 00:00:23.706 A:middle
We're going to begin with a
very high level overview of some

00:00:23.756 --> 00:00:25.466 A:middle
of the new audio
features in iOS 7.

00:00:25.626 --> 00:00:28.106 A:middle
And for the majority of this
session, we're going to talk--

00:00:28.106 --> 00:00:30.556 A:middle
spend our time focused on one
new technology in particular

00:00:30.766 --> 00:00:32.305 A:middle
that we think you're going
to be very excited about.

00:00:32.625 --> 00:00:34.896 A:middle
So, let's dive in to the
list of new features.

00:00:35.906 --> 00:00:39.756 A:middle
First is Audio Input Selection
and with input selection,

00:00:39.886 --> 00:00:41.986 A:middle
your application now has
the ability to specify

00:00:41.986 --> 00:00:44.696 A:middle
which audio input it would like
to use in certain situations.

00:00:44.736 --> 00:00:48.506 A:middle
So for example, if the user had
a wired headset plugged into his

00:00:48.506 --> 00:00:50.706 A:middle
or her device, but your
app wanted to continue

00:00:50.706 --> 00:00:52.396 A:middle
to use the built-in
microphone for input,

00:00:52.636 --> 00:00:55.506 A:middle
you now have the
capability to control that.

00:00:56.166 --> 00:00:58.436 A:middle
With input selection, you can
also choose which microphone

00:00:58.436 --> 00:01:00.456 A:middle
that you'd like to use on
our multi-mic platforms

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:00:58.436 --> 00:01:00.456 A:middle
that you'd like to use on
our multi-mic platforms

00:01:00.996 --> 00:01:03.426 A:middle
and on devices that support
it such as the iPhone 5.

00:01:03.846 --> 00:01:04.616 A:middle
You can take advantage

00:01:04.616 --> 00:01:06.136 A:middle
of microphone beam
forming processing

00:01:06.636 --> 00:01:08.846 A:middle
to set an effective
microphone directivity

00:01:08.976 --> 00:01:12.096 A:middle
by specifying a polar pattern
such as cardioid or subcardioid.

00:01:14.716 --> 00:01:17.176 A:middle
We've made some enhancements
to multichannel audio on iOS 7.

00:01:17.556 --> 00:01:19.536 A:middle
And through the use of
the AVAudioSession API,

00:01:20.236 --> 00:01:22.266 A:middle
you can now discover the
maximum number of input

00:01:22.266 --> 00:01:23.446 A:middle
and output channels
that are supported

00:01:23.446 --> 00:01:25.656 A:middle
by the current audio route
as well as being able

00:01:25.656 --> 00:01:29.096 A:middle
to specify your preferred number
of input and output channels.

00:01:29.096 --> 00:01:31.006 A:middle
For audio outputs
supported such as HDMI,

00:01:31.906 --> 00:01:33.786 A:middle
you can obtain audio
channel labels

00:01:33.786 --> 00:01:35.806 A:middle
which associate a
particular audio channel

00:01:36.126 --> 00:01:38.196 A:middle
with the description of a
physical speaker location

00:01:38.196 --> 00:01:41.486 A:middle
such as front left, front
right, center and so on.

00:01:43.236 --> 00:01:44.916 A:middle
We've added some
extensions to Open AL

00:01:44.916 --> 00:01:47.886 A:middle
to enhance the gaming audio
experience in iOS 7 starting

00:01:47.886 --> 00:01:51.036 A:middle
with the ability to specify a
spatialization rendering quality

00:01:51.086 --> 00:01:52.376 A:middle
on a per-sound source basis.

00:01:53.126 --> 00:01:54.046 A:middle
Now, you might use this

00:01:54.046 --> 00:01:56.196 A:middle
to specify a very high
quality rendering algorithm

00:01:56.196 --> 00:01:58.146 A:middle
for the important sound
sources in your game.

00:01:58.396 --> 00:02:00.066 A:middle
But a less CPU intensive
algorithm

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:01:58.396 --> 00:02:00.066 A:middle
But a less CPU intensive
algorithm

00:02:00.066 --> 00:02:02.036 A:middle
for those less importance
sound sources in your game.

00:02:03.576 --> 00:02:04.816 A:middle
We've also made some
improvements

00:02:04.816 --> 00:02:06.996 A:middle
to our high quality
spatialization

00:02:06.996 --> 00:02:07.676 A:middle
rendering algorithm.

00:02:08.326 --> 00:02:10.366 A:middle
And also added the ability
to support rendering

00:02:10.366 --> 00:02:12.356 A:middle
to multichannel output
hardware when it's available.

00:02:13.326 --> 00:02:15.016 A:middle
Finally, we've added
an extension

00:02:15.016 --> 00:02:16.116 A:middle
to allow capturing the output

00:02:16.116 --> 00:02:18.166 A:middle
of the current Open AL
3D rendering context.

00:02:18.666 --> 00:02:22.666 A:middle
We've added time-pitch
capabilities to Audio Queue.

00:02:22.916 --> 00:02:25.496 A:middle
So your application can now
control the speed up and slow

00:02:25.496 --> 00:02:27.266 A:middle
down of Audio Queue
playback both in terms

00:02:27.266 --> 00:02:28.576 A:middle
of time and in frequency.

00:02:31.276 --> 00:02:34.176 A:middle
We've enhanced the security
around audio recording in iOS 7

00:02:34.546 --> 00:02:37.456 A:middle
and we now require explicit user
approval before your application

00:02:37.456 --> 00:02:38.336 A:middle
can do audio input.

00:02:38.966 --> 00:02:40.166 A:middle
Now, the reason for
doing this is

00:02:40.166 --> 00:02:42.146 A:middle
to prevent a malicious
application from being able

00:02:42.146 --> 00:02:43.876 A:middle
to record a user without
him or her knowing it.

00:02:45.046 --> 00:02:46.976 A:middle
The way that this works
is very similar to the way

00:02:46.976 --> 00:02:49.876 A:middle
that the location service's
permission mechanism works.

00:02:49.876 --> 00:02:50.976 A:middle
In that the user is presented

00:02:50.976 --> 00:02:53.206 A:middle
with a model dialog
requesting his or her permission

00:02:53.206 --> 00:02:54.096 A:middle
to use the audio input.

00:02:54.486 --> 00:02:58.006 A:middle
The decision is made on
per-application basis

00:02:58.076 --> 00:02:59.196 A:middle
and it is a one-time decision.

00:02:59.726 --> 00:03:01.976 A:middle
However, if you'd like to
go in and change your mind

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:02:59.726 --> 00:03:01.976 A:middle
However, if you'd like to
go in and change your mind

00:03:01.976 --> 00:03:03.206 A:middle
at a later time,
you can always go

00:03:03.206 --> 00:03:04.816 A:middle
into the Settings
application to do that.

00:03:06.666 --> 00:03:08.776 A:middle
Until the user has given
your application permission

00:03:08.776 --> 00:03:11.186 A:middle
to use audio input, you
will get silence so you need

00:03:11.186 --> 00:03:12.276 A:middle
to be prepared to handle that.

00:03:12.276 --> 00:03:15.876 A:middle
Now, what actually triggers
the dialog from being presented

00:03:15.876 --> 00:03:17.976 A:middle
to the user is an attempt
by your application

00:03:17.976 --> 00:03:20.626 A:middle
to use an audio session
category that would enable input

00:03:20.626 --> 00:03:22.546 A:middle
such as the record
category or play and record.

00:03:23.596 --> 00:03:26.076 A:middle
However, if you'd
like to have control

00:03:26.076 --> 00:03:27.856 A:middle
over when the user is
presented with his dialog

00:03:28.016 --> 00:03:29.966 A:middle
so that it can happen
at a more opportune time

00:03:29.966 --> 00:03:32.256 A:middle
for your application,
we've added some API

00:03:32.446 --> 00:03:35.426 A:middle
and AVAudioSession for
you to be able to do that.

00:03:38.226 --> 00:03:39.976 A:middle
Finally, just a note on
the AudioSession API.

00:03:40.426 --> 00:03:42.766 A:middle
As we mentioned at last year's
conference, the C version

00:03:42.766 --> 00:03:45.316 A:middle
of the AudioSession API is
officially being deprecated

00:03:45.316 --> 00:03:46.086 A:middle
in iOS 7.

00:03:46.326 --> 00:03:48.386 A:middle
So, we hope that over the
course of the past year,

00:03:48.576 --> 00:03:50.416 A:middle
you've all had the opportunity
to move your applications

00:03:50.416 --> 00:03:52.286 A:middle
over to using the
AVAudioSession API.

00:03:55.696 --> 00:03:57.976 A:middle
So, here is a summary of the
features that we just discussed.

00:03:57.976 --> 00:04:00.116 A:middle
We're not going to spend anymore
time today going over any

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:03:57.976 --> 00:04:00.116 A:middle
We're not going to spend anymore
time today going over any

00:04:00.116 --> 00:04:01.446 A:middle
of these topics in
any more detail.

00:04:01.616 --> 00:04:03.326 A:middle
So, if you have any
questions about these

00:04:03.416 --> 00:04:05.636 A:middle
or like a more detailed
overview of any of these items,

00:04:06.026 --> 00:04:08.036 A:middle
we encourage you to come by
our labs either later today

00:04:08.316 --> 00:04:10.106 A:middle
or tomorrow morning and
we'd be happy to discuss

00:04:10.106 --> 00:04:10.936 A:middle
with you in more detail.

00:04:11.666 --> 00:04:14.046 A:middle
I'd also encourage you to have
a look at the documentation

00:04:14.046 --> 00:04:16.046 A:middle
in the various header files
that I outlined in the course

00:04:16.046 --> 00:04:17.276 A:middle
of going through
each of these topics.

00:04:18.276 --> 00:04:21.526 A:middle
So for the remainder of this
session, we're going to focus

00:04:21.526 --> 00:04:23.546 A:middle
on one new technology in
particular that again,

00:04:23.856 --> 00:04:25.406 A:middle
we think you're going
to be very excited about

00:04:25.406 --> 00:04:28.046 A:middle
and that's Inter-App Audio.

00:04:28.516 --> 00:04:29.926 A:middle
So what is Inter-App Audio?

00:04:30.286 --> 00:04:32.856 A:middle
Well, as the name implies,

00:04:32.856 --> 00:04:34.996 A:middle
Inter-App Audio provides
the ability to stream audio

00:04:34.996 --> 00:04:36.406 A:middle
between applications
in real-time.

00:04:36.756 --> 00:04:39.326 A:middle
So, if you have a really cool
effects application and you want

00:04:39.326 --> 00:04:40.806 A:middle
to integrate that into
your DAW application,

00:04:40.906 --> 00:04:42.726 A:middle
you now have the
ability to do that.

00:04:43.266 --> 00:04:46.146 A:middle
We've built Inter-App Audio on
top of existing Core Audio APIs

00:04:46.146 --> 00:04:47.986 A:middle
so it should be very
easy for you to integrate

00:04:47.986 --> 00:04:49.776 A:middle
into your existing applications

00:04:49.776 --> 00:04:52.616 A:middle
and deploy quickly
to the app store.

00:04:52.616 --> 00:04:54.326 A:middle
Because it's built into
the operating system,

00:04:54.526 --> 00:04:56.776 A:middle
the solution is very efficient
with zero additional latency

00:04:57.296 --> 00:05:00.026 A:middle
and should provide for a stable
platform for the evolution

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:04:57.296 --> 00:05:00.026 A:middle
and should provide for a stable
platform for the evolution

00:05:00.026 --> 00:05:00.796 A:middle
of the feature over time.

00:05:01.706 --> 00:05:03.476 A:middle
Now, before we get into any
of the technical details

00:05:03.476 --> 00:05:06.406 A:middle
of how Inter-App Audio works,
I'd like to invite up Alec

00:05:06.636 --> 00:05:09.106 A:middle
from the GarageBand
team to give you a demo.

00:05:09.181 --> 00:05:11.181 A:middle
[ Applause ]

00:05:11.256 --> 00:05:16.566 A:middle
>> Thanks Tony.

00:05:17.056 --> 00:05:17.476 A:middle
Am I up?

00:05:17.766 --> 00:05:17.986 A:middle
>> Yeah.

00:05:18.416 --> 00:05:22.996 A:middle
>> My name is Alec, I am a
product designer for GarageBand

00:05:22.996 --> 00:05:26.296 A:middle
and Logic and I'm going to
switch over here to my iPad.

00:05:27.106 --> 00:05:30.496 A:middle
So, what I want to do today
is give a quick demonstration

00:05:30.546 --> 00:05:33.876 A:middle
about how we have been working
with the development version,

00:05:33.876 --> 00:05:36.016 A:middle
kind of a sneak peek into
a development version

00:05:36.016 --> 00:05:38.976 A:middle
of GarageBand and how we're
doing some experiments

00:05:39.096 --> 00:05:40.096 A:middle
with Inter-App Audio.

00:05:40.936 --> 00:05:43.946 A:middle
So, what I have up here is
just a simple FourTrack song

00:05:43.946 --> 00:05:45.736 A:middle
in GarageBand, I'm going
to play a little bit

00:05:45.736 --> 00:05:47.396 A:middle
so you can get an idea
of what it sounds like.

00:05:47.396 --> 00:05:47.936 A:middle
[ Music ]

00:05:47.936 --> 00:05:56.386 A:middle
OK. So the first thing
I want to do is I want

00:05:56.386 --> 00:05:58.556 A:middle
to add a little keyboard
part to this.

00:05:59.056 --> 00:06:01.736 A:middle
But instead of using one
of the built-in instruments

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:05:59.056 --> 00:06:01.736 A:middle
But instead of using one
of the built-in instruments

00:06:01.736 --> 00:06:05.946 A:middle
in GarageBand, I want to use
an instrument, on the system

00:06:05.946 --> 00:06:07.216 A:middle
that is not part of GarageBand.

00:06:07.216 --> 00:06:08.806 A:middle
So to do that, I'm going to go

00:06:08.806 --> 00:06:10.666 A:middle
out to the GarageBand
instrument browser.

00:06:10.776 --> 00:06:13.356 A:middle
Now, what we see here are
the instruments that ship

00:06:13.356 --> 00:06:15.096 A:middle
with GarageBand,
part of GarageBand.

00:06:15.576 --> 00:06:17.976 A:middle
And then we have a new
icon here, Music Apps.

00:06:18.266 --> 00:06:22.536 A:middle
I'm going to tap on that and
we see the icons of other apps

00:06:22.536 --> 00:06:26.466 A:middle
on the system which
are audio apps.

00:06:26.516 --> 00:06:30.116 A:middle
So, I'm going to click on
sampler one here and we'll see

00:06:30.116 --> 00:06:32.556 A:middle
that the sampler launches
in the background.

00:06:33.126 --> 00:06:36.976 A:middle
Now here it with the UI in the
foreground and we can hear it.

00:06:37.936 --> 00:06:39.906 A:middle
Now, you see there's a
transport here and that's,

00:06:39.906 --> 00:06:42.626 A:middle
this transport is remotely
controlling the transport

00:06:42.626 --> 00:06:43.386 A:middle
of GarageBand.

00:06:43.556 --> 00:06:45.456 A:middle
So, when I press the record
button, what we're going

00:06:45.456 --> 00:06:48.716 A:middle
to hear is a count off from
GarageBand and then the track

00:06:48.716 --> 00:06:53.856 A:middle
that I just played and I'll
record over the top of it.

00:06:54.356 --> 00:07:01.376 A:middle
[ Music ]

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:06:54.356 --> 00:07:01.376 A:middle
[ Music ]

00:07:01.876 --> 00:07:03.236 A:middle
Brilliant musical passage.

00:07:04.816 --> 00:07:07.996 A:middle
So now, if I-- if you look up at
this transport again you'll see

00:07:07.996 --> 00:07:09.736 A:middle
that there's a GarageBand icon.

00:07:10.366 --> 00:07:12.916 A:middle
When I tap on that
icon, I switch back

00:07:12.916 --> 00:07:16.756 A:middle
to the GarageBand application
and now, in the tracks view,

00:07:17.126 --> 00:07:20.456 A:middle
a new track has been added
with this little keyboard part

00:07:20.456 --> 00:07:26.846 A:middle
that I played we
can listen to it.

00:07:26.936 --> 00:07:30.346 A:middle
[Background Music] And
add some keyboard to it.

00:07:30.346 --> 00:07:30.896 A:middle
[ Music ]

00:07:30.896 --> 00:07:33.746 A:middle
So, that was bringing audio
from another application,

00:07:33.746 --> 00:07:35.986 A:middle
controlling that
application in its interface,

00:07:35.986 --> 00:07:37.486 A:middle
and recording that
in GarageBand.

00:07:38.096 --> 00:07:39.626 A:middle
The next thing I
want to do is I want

00:07:39.626 --> 00:07:43.066 A:middle
to process an input
from GarageBand.

00:07:43.066 --> 00:07:47.736 A:middle
So, I'm going to put on my
little guitar here and we'll go

00:07:47.736 --> 00:07:50.896 A:middle
to the guitar amp in GarageBand.

00:07:51.426 --> 00:07:53.516 A:middle
Now, this guitar
amp is part of--

00:07:53.516 --> 00:07:57.466 A:middle
one of the instruments built
in the GarageBand and I'm going

00:07:57.466 --> 00:07:59.856 A:middle
to turn on input monitoring
so I can hear myself.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:08:00.616 --> 00:08:04.886 A:middle
[Background Music] You
guys got it out there?

00:08:07.456 --> 00:08:10.076 A:middle
It's a little phase switch.

00:08:13.456 --> 00:08:15.586 A:middle
OK. Or we're going to--

00:08:15.586 --> 00:08:18.176 A:middle
there you go, that's
more rock and roll.

00:08:18.686 --> 00:08:21.186 A:middle
OK. So, that's a
good sound right?

00:08:21.186 --> 00:08:23.316 A:middle
That's using the guitar
app from GarageBand.

00:08:23.316 --> 00:08:25.456 A:middle
What I want to do though
is I want to process it

00:08:25.626 --> 00:08:27.476 A:middle
with another effect
on my system.

00:08:27.926 --> 00:08:31.426 A:middle
So again, I'm going to go into
the input settings in GarageBand

00:08:31.426 --> 00:08:32.416 A:middle
and if you see about halfway

00:08:32.416 --> 00:08:34.466 A:middle
down this list, it
says Effect App.

00:08:34.466 --> 00:08:38.076 A:middle
I'm going to tap on that and
we can see a list of apps

00:08:38.076 --> 00:08:40.525 A:middle
on my system that are
effects so I'm going

00:08:40.525 --> 00:08:42.046 A:middle
to click on this Audio Delay.

00:08:42.905 --> 00:08:45.606 A:middle
[Music] So, there is the delay

00:08:45.656 --> 00:08:47.366 A:middle
but it's not really
the settings I want.

00:08:47.496 --> 00:08:51.276 A:middle
So, I'm going to tap on
the Effect icon and switch

00:08:51.276 --> 00:08:52.546 A:middle
to the Effects Interface.

00:08:53.086 --> 00:08:54.636 A:middle
I'm going to take the feedback

00:08:54.636 --> 00:08:58.356 A:middle
down here a little
bit and the mix.

00:08:59.016 --> 00:08:59.196 A:middle
[Music] OK.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:09:00.266 --> 00:09:04.836 A:middle
So that's a little bit better.

00:09:04.836 --> 00:09:06.726 A:middle
So now, what I'm doing
is I'm taking the input

00:09:07.296 --> 00:09:09.316 A:middle
through GarageBand, sending
it out to this effect

00:09:09.686 --> 00:09:11.476 A:middle
and bringing it back
on to GarageBand.

00:09:12.226 --> 00:09:13.676 A:middle
Then I can hit record.

00:09:13.676 --> 00:09:14.256 A:middle
[ Music ]

00:09:14.256 --> 00:09:54.026 A:middle
Now, if we switch back
to the tracks view,

00:09:55.626 --> 00:09:57.406 A:middle
we can see that new
region has been recorded

00:09:57.406 --> 00:09:59.606 A:middle
in GarageBand and if I play.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:10:00.106 --> 00:10:03.576 A:middle
[ Music ]

00:10:04.076 --> 00:10:08.566 A:middle
There is the source
with the delay added

00:10:08.676 --> 00:10:10.676 A:middle
to it recorded in
the GarageBand.

00:10:14.816 --> 00:10:17.626 A:middle
So that's just the
quick overview

00:10:17.626 --> 00:10:19.536 A:middle
of how we're doing some
experiments inside this

00:10:19.536 --> 00:10:21.096 A:middle
development version
of GarageBand

00:10:21.186 --> 00:10:24.056 A:middle
with the new Inter-App
Audio APIs.

00:10:24.056 --> 00:10:25.976 A:middle
And next, we're going
to bring up Doug

00:10:25.976 --> 00:10:27.996 A:middle
to give you a little more
detail about how some

00:10:28.136 --> 00:10:36.726 A:middle
of the stuff works
under the hood.

00:10:36.726 --> 00:10:36.793 A:middle
[ Applause ]

00:10:36.793 --> 00:10:39.286 A:middle
>> Thank you Alec.

00:10:39.286 --> 00:10:42.096 A:middle
Hi, my name is Doug Wyatt, I'm a
plumber in the Core Audio group.

00:10:42.686 --> 00:10:44.896 A:middle
I'd like to present to
you some of the details

00:10:44.896 --> 00:10:46.776 A:middle
of the Inter-App Audio APIs.

00:10:48.376 --> 00:10:52.956 A:middle
So, conceptually here, we
have two kinds of applications

00:10:52.956 --> 00:10:54.746 A:middle
which we call the
host application

00:10:55.146 --> 00:10:56.476 A:middle
and the node application.

00:10:57.296 --> 00:10:58.536 A:middle
The fundamental distinction

00:10:58.536 --> 00:11:01.556 A:middle
between these two
applications is that the host is

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:10:58.536 --> 00:11:01.556 A:middle
between these two
applications is that the host is

00:11:01.556 --> 00:11:04.066 A:middle
where we ultimately
want the audio coming

00:11:04.066 --> 00:11:06.296 A:middle
from the node application
to end up.

00:11:06.296 --> 00:11:09.766 A:middle
So, GarageBand in this
example was a host application.

00:11:09.766 --> 00:11:12.626 A:middle
It was receiving audio from
the sampler application

00:11:13.066 --> 00:11:15.256 A:middle
and from the delay
effect application.

00:11:15.816 --> 00:11:18.396 A:middle
So, given these two
kinds of applications,

00:11:18.916 --> 00:11:20.256 A:middle
we're going to look at APIs

00:11:20.256 --> 00:11:22.776 A:middle
for how node applications
can register themselves

00:11:22.776 --> 00:11:26.306 A:middle
with the system and how host
applications can discover those

00:11:26.306 --> 00:11:27.916 A:middle
registered node applications.

00:11:28.816 --> 00:11:32.206 A:middle
We'll look at how
host applications can

00:11:32.206 --> 00:11:35.766 A:middle
of initiate connections through
the system to node applications.

00:11:35.766 --> 00:11:38.476 A:middle
And once those connections
are established,

00:11:38.846 --> 00:11:41.476 A:middle
the two applications can stream
audio between each other.

00:11:41.846 --> 00:11:45.986 A:middle
But, again, primarily the
destination has to be the host

00:11:46.036 --> 00:11:47.826 A:middle
that could optionally
send the audio to the node

00:11:47.826 --> 00:11:49.886 A:middle
if the node is providing
an effect.

00:11:50.506 --> 00:11:54.266 A:middle
We'll look at how furthermore
host applications can send MIDI

00:11:54.266 --> 00:11:57.576 A:middle
events to node applications to
control their audio rendering.

00:11:58.106 --> 00:12:01.656 A:middle
So for example, with
that sampler application,

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:11:58.106 --> 00:12:01.656 A:middle
So for example, with
that sampler application,

00:12:01.656 --> 00:12:04.106 A:middle
the host could have been
actually sending the MIDI nodes

00:12:04.556 --> 00:12:08.946 A:middle
to the sampler and receiving
the rendered audio back.

00:12:09.556 --> 00:12:11.766 A:middle
We'll look at some interfaces

00:12:11.766 --> 00:12:14.226 A:middle
where the host can
express information

00:12:14.226 --> 00:12:17.016 A:middle
about its transport
controls and transport state

00:12:17.016 --> 00:12:19.826 A:middle
and timeline position
to node applications.

00:12:20.036 --> 00:12:21.306 A:middle
And finally, we'll look

00:12:21.306 --> 00:12:23.866 A:middle
at how node applications
can remotely control

00:12:23.866 --> 00:12:24.976 A:middle
host applications.

00:12:24.976 --> 00:12:28.416 A:middle
So, let's look inside
host applications.

00:12:28.956 --> 00:12:31.636 A:middle
So, this is your
basic standalone music

00:12:31.636 --> 00:12:33.516 A:middle
or audio application on iOS.

00:12:34.106 --> 00:12:38.366 A:middle
We have AURemoteIO audio unit
and its function is to connect

00:12:38.466 --> 00:12:41.606 A:middle
to the audio input and
output system with zero--

00:12:41.606 --> 00:12:45.876 A:middle
well, very low latency using
pretty much the same mechanisms

00:12:45.876 --> 00:12:49.426 A:middle
as on the desktop but always
through the RemoteIO audio unit.

00:12:49.786 --> 00:12:53.506 A:middle
So, feeding the audio unit, we
have the host's audio engine

00:12:53.506 --> 00:12:55.696 A:middle
and that can be constructed
in a number of ways,

00:12:55.696 --> 00:12:57.316 A:middle
we'll show some examples later.

00:12:58.016 --> 00:13:00.196 A:middle
But for the-- the purposes
of Inter-App Audio,

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:12:58.016 --> 00:13:00.196 A:middle
But for the-- the purposes
of Inter-App Audio,

00:13:01.176 --> 00:13:04.206 A:middle
the host engine connects
to a node application

00:13:04.786 --> 00:13:06.806 A:middle
by instantiating
a node audio unit.

00:13:07.216 --> 00:13:10.976 A:middle
This is another Apple supplied
audio unit that, in effect,

00:13:11.426 --> 00:13:14.546 A:middle
creates the bridge to the
remote node application

00:13:14.606 --> 00:13:16.546 A:middle
to communicate audio with it.

00:13:17.916 --> 00:13:21.256 A:middle
Now, on the node side, a
node application is also

00:13:21.686 --> 00:13:25.796 A:middle
by default a normal application
with an AURemoteIO that can play

00:13:25.796 --> 00:13:29.366 A:middle
and record as always and it's
got its own audio engine.

00:13:30.386 --> 00:13:33.086 A:middle
What's a little different here
is in the Inter-App scenario,

00:13:33.086 --> 00:13:36.976 A:middle
the node application has its
input and output redirected

00:13:36.976 --> 00:13:40.236 A:middle
from the mic and speaker
to the host application.

00:13:40.866 --> 00:13:44.646 A:middle
So, that's the node application.

00:13:45.636 --> 00:13:48.996 A:middle
So, you see then we're
implementing this API

00:13:48.996 --> 00:13:50.966 A:middle
as a series of extensions

00:13:50.966 --> 00:13:54.276 A:middle
to the existing
AudioUnit.framework APIs.

00:13:54.276 --> 00:13:57.496 A:middle
The host sees the node
application as an audio unit

00:13:57.496 --> 00:13:58.546 A:middle
that it communicates with

00:13:59.206 --> 00:14:04.536 A:middle
and the nodes AURemoteIO unit
gets redirected to the host

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:13:59.206 --> 00:14:04.536 A:middle
and the nodes AURemoteIO unit
gets redirected to the host

00:14:04.596 --> 00:14:06.226 A:middle
so the node's communication

00:14:06.226 --> 00:14:09.226 A:middle
to the host application
is through that IO unit.

00:14:11.696 --> 00:14:15.486 A:middle
So, to express the capabilities
of these node applications

00:14:15.486 --> 00:14:17.346 A:middle
and to distinguish them a bit

00:14:17.406 --> 00:14:20.246 A:middle
from the existing
audio unit types,

00:14:20.336 --> 00:14:22.126 A:middle
we have these four new types.

00:14:23.306 --> 00:14:26.196 A:middle
They all are the same in that
they produce audio output

00:14:27.046 --> 00:14:30.136 A:middle
but they differ in what input
they receive from the host.

00:14:30.546 --> 00:14:33.116 A:middle
We have remote generators
which require no input.

00:14:33.796 --> 00:14:36.456 A:middle
We have remote instruments
which take MIDI input

00:14:36.456 --> 00:14:38.436 A:middle
to produce output, audio output.

00:14:39.156 --> 00:14:41.306 A:middle
We have effects which
are audio in and out.

00:14:41.776 --> 00:14:45.086 A:middle
And finally, we have music
effects which take both audio

00:14:45.086 --> 00:14:47.546 A:middle
and MIDI input and
produce audio.

00:14:48.876 --> 00:14:52.546 A:middle
So, node applications
use these component types

00:14:52.546 --> 00:14:54.286 A:middle
to describe their capabilities.

00:14:54.726 --> 00:14:57.776 A:middle
And furthermore one node
application may actually have

00:14:58.316 --> 00:15:01.966 A:middle
multiple sets of capabilities
and may wish to present itself

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:14:58.316 --> 00:15:01.966 A:middle
multiple sets of capabilities
and may wish to present itself

00:15:02.096 --> 00:15:03.996 A:middle
in multiple ways to hosts.

00:15:04.156 --> 00:15:05.486 A:middle
As a simple example,

00:15:05.486 --> 00:15:09.106 A:middle
a node application may produce
audio just fine on its own,

00:15:09.576 --> 00:15:10.876 A:middle
in which case, it's a generator.

00:15:11.316 --> 00:15:15.006 A:middle
It may optionally be
able to respond to MIDI

00:15:15.006 --> 00:15:16.936 A:middle
in producing that audio.

00:15:17.616 --> 00:15:19.566 A:middle
So, it can also be a generator.

00:15:19.566 --> 00:15:22.436 A:middle
So, such a node application
could publish itself

00:15:22.436 --> 00:15:24.326 A:middle
as two different
audio components

00:15:24.566 --> 00:15:25.996 A:middle
with separate capabilities.

00:15:26.856 --> 00:15:29.036 A:middle
Another example of
that is an application

00:15:29.036 --> 00:15:33.336 A:middle
like a guitar amp simulator
where the application appears

00:15:33.336 --> 00:15:36.816 A:middle
to the user as an effect
because audio is going in,

00:15:37.136 --> 00:15:39.636 A:middle
it's being processed in some
way and then it comes out.

00:15:40.206 --> 00:15:41.746 A:middle
But from the host's
point of view,

00:15:42.156 --> 00:15:45.616 A:middle
this application can appear
either as a generator an effect

00:15:45.776 --> 00:15:48.056 A:middle
and the node can publish
itself either way.

00:15:48.566 --> 00:15:52.676 A:middle
For example, if a node says I'm
a generator, it can continue

00:15:52.676 --> 00:15:57.906 A:middle
to receive microphone or a line
input from a guitar directly

00:15:57.976 --> 00:16:02.736 A:middle
from the underlying AURemoteIO
while only sending the audio

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:15:57.976 --> 00:16:02.736 A:middle
from the underlying AURemoteIO
while only sending the audio

00:16:02.736 --> 00:16:03.736 A:middle
output to the host.

00:16:04.066 --> 00:16:05.446 A:middle
So again, that's generator mode.

00:16:05.986 --> 00:16:07.906 A:middle
Or if a host application

00:16:07.906 --> 00:16:10.686 A:middle
like GarageBand might have
a prerecorded guitar track

00:16:11.176 --> 00:16:13.846 A:middle
and want to process that through
the guitar amp simulator.

00:16:14.306 --> 00:16:17.226 A:middle
The guitar amp simulator can
function fully as an effect,

00:16:17.786 --> 00:16:20.036 A:middle
not communicate with the
audio hardware at all,

00:16:20.556 --> 00:16:22.896 A:middle
and just communicate
the two audio streams

00:16:23.276 --> 00:16:27.366 A:middle
between itself and the host.

00:16:27.966 --> 00:16:30.416 A:middle
Let's move on and look at
some of the requirements

00:16:30.416 --> 00:16:33.276 A:middle
for the Inter-App Audio
feature, it's available

00:16:33.276 --> 00:16:35.826 A:middle
on most iOS 7 compatible
devices,

00:16:35.866 --> 00:16:37.886 A:middle
the exception being
the iPhone 4.

00:16:38.196 --> 00:16:41.476 A:middle
And on the iPhone 4, you
don't really have to deal

00:16:41.476 --> 00:16:43.786 A:middle
with this specially because
what will happen is node

00:16:43.786 --> 00:16:46.916 A:middle
applications, if they
attempt to register themselves

00:16:46.916 --> 00:16:50.276 A:middle
with the system, those calls
will just fail silently,

00:16:50.276 --> 00:16:51.396 A:middle
the system will ignore them.

00:16:51.396 --> 00:16:55.776 A:middle
And on the host side, the
host will simply see no node

00:16:55.776 --> 00:16:58.626 A:middle
applications on the system.

00:16:58.626 --> 00:17:00.676 A:middle
Both host and node
applications need

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:16:58.626 --> 00:17:00.676 A:middle
Both host and node
applications need

00:17:00.676 --> 00:17:04.185 A:middle
to have a new entitlement called
"inter-app-audio" and this--

00:17:04.656 --> 00:17:06.566 A:middle
you can set this
for your application

00:17:06.566 --> 00:17:08.396 A:middle
in the Xcode Capabilities tab.

00:17:10.156 --> 00:17:15.296 A:middle
Furthermore, most applications
will want to have audio

00:17:15.296 --> 00:17:17.076 A:middle
in their UIBackgroundModes.

00:17:17.806 --> 00:17:21.036 A:middle
Most especially hosts
for obvious reasons

00:17:21.086 --> 00:17:23.925 A:middle
because hosts will keep
running their engines

00:17:23.925 --> 00:17:25.226 A:middle
when nodes are on
the foreground.

00:17:25.736 --> 00:17:29.436 A:middle
Also, nodes like the guitar amp
simulator I just mentioned may

00:17:29.436 --> 00:17:32.206 A:middle
want to continue accessing the
mic and to be able to do that,

00:17:32.256 --> 00:17:34.786 A:middle
they too need to have the
audio background mode.

00:17:35.766 --> 00:17:37.686 A:middle
One final requirement for nodes

00:17:37.686 --> 00:17:39.846 A:middle
in particular is
the MixWithOthers

00:17:39.846 --> 00:17:41.616 A:middle
AudioSessionCategoryOption.

00:17:42.276 --> 00:17:45.556 A:middle
Hosts can go either
way on this one.

00:17:45.666 --> 00:17:47.576 A:middle
We'll get into that
in more detail later.

00:17:48.176 --> 00:17:52.086 A:middle
OK. Getting in to the nuts
and bolts of the APIs here,

00:17:52.086 --> 00:17:54.276 A:middle
let's look at how node
applications can register

00:17:54.276 --> 00:17:57.876 A:middle
themselves with the system.

00:17:57.876 --> 00:18:01.656 A:middle
So, there's two pieces
of registering one's self

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:17:57.876 --> 00:18:01.656 A:middle
So, there's two pieces
of registering one's self

00:18:01.656 --> 00:18:02.766 A:middle
for a node application.

00:18:03.246 --> 00:18:07.186 A:middle
The first is an Info.plist
entry called AudioComponents.

00:18:07.626 --> 00:18:11.306 A:middle
So, the presence of this
Info.plist entry makes the app

00:18:11.586 --> 00:18:15.096 A:middle
discoverable and
launchable to the system.

00:18:15.396 --> 00:18:16.896 A:middle
The system knows,
oh I've got one

00:18:16.896 --> 00:18:18.706 A:middle
of this node applications
installed.

00:18:19.226 --> 00:18:22.866 A:middle
The second part of registration
is for the node application

00:18:22.866 --> 00:18:26.286 A:middle
to call AudioOutputUnitPublish
which checks

00:18:26.286 --> 00:18:30.106 A:middle
in that registration that it
advertised in its Info.plist.

00:18:30.286 --> 00:18:33.146 A:middle
It says, I've been launched and
here I am ready to communicate.

00:18:33.366 --> 00:18:36.316 A:middle
So let's look at those two
pieces in a little detail.

00:18:37.096 --> 00:18:41.286 A:middle
So here is the AudioComponents
entry in the Info.plist.

00:18:41.286 --> 00:18:43.756 A:middle
Its value is an array
and in that array,

00:18:43.756 --> 00:18:46.436 A:middle
there is a dictionary
for every AudioComponent

00:18:46.436 --> 00:18:47.836 A:middle
that the node wants to register.

00:18:48.576 --> 00:18:50.986 A:middle
And in that dictionary,
if you're familiar

00:18:50.986 --> 00:18:53.056 A:middle
with AudioComponentDescriptions
already,

00:18:53.056 --> 00:18:55.276 A:middle
you'll see some familiar
fields there.

00:18:55.276 --> 00:18:59.946 A:middle
There is the type, subtype and
manufacturer along with the name

00:18:59.946 --> 00:19:01.056 A:middle
and the version number.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:18:59.946 --> 00:19:01.056 A:middle
and the version number.

00:19:01.656 --> 00:19:04.186 A:middle
So, that completely
describes the AudioComponent

00:19:04.566 --> 00:19:07.376 A:middle
that the node application
is advertising.

00:19:07.976 --> 00:19:13.026 A:middle
So, moving on to the second
part of the registration here,

00:19:13.636 --> 00:19:16.306 A:middle
this is when the node
application launches,

00:19:16.806 --> 00:19:21.676 A:middle
the first piece of code here
is the node's normal process

00:19:21.676 --> 00:19:24.416 A:middle
for creating its
AURemoteIO when it launches.

00:19:24.846 --> 00:19:26.856 A:middle
It creates an
AudioComponentDescription

00:19:26.856 --> 00:19:30.176 A:middle
describing the Apple
AURemoteIO instance,

00:19:30.176 --> 00:19:32.146 A:middle
it uses AudioComponentFindNext

00:19:32.146 --> 00:19:35.476 A:middle
to go find the AudioComponent
for the AURemoteIO.

00:19:35.996 --> 00:19:38.896 A:middle
And finally, it creates an
instance of the AURemoteIO

00:19:38.896 --> 00:19:43.326 A:middle
and this is something just about
every audio and music app on--

00:19:43.556 --> 00:19:47.586 A:middle
today will do for creating
a low latency IO channel.

00:19:48.106 --> 00:19:52.166 A:middle
What's new is that the node
application, to participate

00:19:52.166 --> 00:19:56.696 A:middle
in Inter-App Audio is now
going to connect that IO Unit

00:19:56.736 --> 00:20:00.086 A:middle
that it just created with
the component description

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:19:56.736 --> 00:20:00.086 A:middle
that it just created with
the component description

00:20:00.086 --> 00:20:02.356 A:middle
that was published in
the Info.plist entry.

00:20:02.976 --> 00:20:06.126 A:middle
So, to do that, we're
seeing this code here

00:20:06.126 --> 00:20:08.506 A:middle
that that node creates an
AudioComponentDescription

00:20:08.506 --> 00:20:11.736 A:middle
which matches the one in the
Info.plist we saw a moment ago.

00:20:12.356 --> 00:20:15.736 A:middle
It supplies the name and version
number and passes all that along

00:20:15.736 --> 00:20:17.456 A:middle
with the AURemoteIO instance

00:20:17.456 --> 00:20:20.806 A:middle
to a new API called
AudioOutputUnitPublish.

00:20:20.976 --> 00:20:24.476 A:middle
So again, that connects what
was advertised in the Info.plist

00:20:25.126 --> 00:20:29.476 A:middle
with the actual RemoteIO
instance in the application

00:20:29.806 --> 00:20:32.106 A:middle
to which the host
application will connect

00:20:32.106 --> 00:20:34.216 A:middle
as we'll see in a little bit.

00:20:34.216 --> 00:20:36.556 A:middle
So, to make this all
work, a requirement

00:20:36.556 --> 00:20:40.726 A:middle
of the node application is
to publish that RemoteIO unit

00:20:40.836 --> 00:20:44.426 A:middle
when it launches because the
node application is going

00:20:44.426 --> 00:20:46.986 A:middle
to get launched by host
applications when--

00:20:46.986 --> 00:20:48.896 A:middle
at times when the user
what's to use them.

00:20:49.936 --> 00:20:53.366 A:middle
And so, the node application
basically has to acknowledge,

00:20:53.366 --> 00:20:54.526 A:middle
I'm here, I've been launched.

00:20:55.666 --> 00:21:00.306 A:middle
And so, you can see then why the
Info.plist entry and the call

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:20:55.666 --> 00:21:00.306 A:middle
And so, you can see then why the
Info.plist entry and the call

00:21:00.306 --> 00:21:04.636 A:middle
to AudioOutputUnitPublish
must have the same component

00:21:04.636 --> 00:21:06.426 A:middle
descriptions, names,
and versions.

00:21:07.426 --> 00:21:09.416 A:middle
One note here is
that by convention,

00:21:09.826 --> 00:21:13.106 A:middle
the component name should
contain your manufacture name

00:21:13.106 --> 00:21:17.236 A:middle
and application name and that
lets host applications sort the

00:21:17.236 --> 00:21:20.416 A:middle
available node applications by
manufacture name if they like.

00:21:20.416 --> 00:21:26.516 A:middle
So, that's the registration
process for node applications,

00:21:26.516 --> 00:21:29.416 A:middle
let's look at how host
applications can discover

00:21:29.416 --> 00:21:30.556 A:middle
those registrations.

00:21:33.836 --> 00:21:37.426 A:middle
So again, if you've used the
AudioComponent calls before,

00:21:37.626 --> 00:21:39.346 A:middle
this should look
fairly familiar.

00:21:39.726 --> 00:21:43.456 A:middle
What we here-- have here is a
loop where we want to iterate

00:21:43.526 --> 00:21:46.126 A:middle
through all of the components on
the system because we're looking

00:21:46.126 --> 00:21:48.656 A:middle
for nodes and there
are multiple types.

00:21:48.786 --> 00:21:50.286 A:middle
So, the simplest
way to do that is

00:21:50.286 --> 00:21:52.506 A:middle
to create a wild card
component description

00:21:52.506 --> 00:21:54.916 A:middle
and that's the searchDesc,
it's full of zeros.

00:21:55.586 --> 00:21:58.906 A:middle
And so then, this loop will
call AudioComponentFindNext

00:21:58.906 --> 00:22:02.936 A:middle
repeatedly and that
will yield in turn each

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:21:58.906 --> 00:22:02.936 A:middle
repeatedly and that
will yield in turn each

00:22:02.936 --> 00:22:05.166 A:middle
of the AudioComponents
on the system which are

00:22:05.526 --> 00:22:06.886 A:middle
in the local variable comp.

00:22:07.516 --> 00:22:10.486 A:middle
When we find null, then we've
gotten to the end of the list

00:22:10.486 --> 00:22:14.006 A:middle
of all the components in
the system and we're done

00:22:14.006 --> 00:22:15.286 A:middle
with our loop, we'll
have found them all.

00:22:16.126 --> 00:22:18.256 A:middle
Now, for each component on
the system, what we want

00:22:18.256 --> 00:22:20.716 A:middle
to do is call
AudioComponentGetDescription

00:22:21.596 --> 00:22:24.486 A:middle
and this will supply to us
the AudioComponent description

00:22:24.576 --> 00:22:27.446 A:middle
of the actual unit as
opposed to that wild card

00:22:27.646 --> 00:22:28.786 A:middle
that we used for searching.

00:22:29.436 --> 00:22:32.816 A:middle
So now, in foundDesc, we can
look at its component type

00:22:32.816 --> 00:22:37.706 A:middle
and see if it's one of the
four inter-app audio unit types

00:22:37.706 --> 00:22:40.466 A:middle
that we're interested in, the
RemoteEffects, RemoteGenerator,

00:22:40.466 --> 00:22:42.196 A:middle
RemoteInstrument, and
RemoteMusicEffect.

00:22:42.676 --> 00:22:46.596 A:middle
If we see one of those, then
we know we found the node.

00:22:47.136 --> 00:22:49.336 A:middle
OK. So the host has
found a node.

00:22:49.926 --> 00:22:52.496 A:middle
So now, I'm going to
walk through a little bit

00:22:52.496 --> 00:22:54.746 A:middle
of code here from one
of our sample apps.

00:22:55.296 --> 00:22:59.126 A:middle
It creates an objective C
object of its own just as a way

00:22:59.126 --> 00:23:01.486 A:middle
of storing information about
the nodes that it's found.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:22:59.126 --> 00:23:01.486 A:middle
of storing information about
the nodes that it's found.

00:23:02.436 --> 00:23:05.686 A:middle
And it calls this class
RemoteAU and it stores away

00:23:05.686 --> 00:23:08.266 A:middle
into it the component
description that was found

00:23:08.806 --> 00:23:10.866 A:middle
and the AudioComponent
that was found.

00:23:11.886 --> 00:23:14.516 A:middle
It also fetches the
component's name and stores

00:23:14.516 --> 00:23:17.806 A:middle
that in the field of
the RemoteAU object.

00:23:18.696 --> 00:23:22.806 A:middle
It sets the image from
AudioComponentGetIcon

00:23:22.806 --> 00:23:25.536 A:middle
which is a new API call which
works with inter-app audio.

00:23:26.186 --> 00:23:28.046 A:middle
This gives you the
application of--

00:23:28.376 --> 00:23:30.966 A:middle
I'm sorry, the icon of
the node application.

00:23:32.836 --> 00:23:36.996 A:middle
We can also discover the time at
which the user last interacted

00:23:37.086 --> 00:23:41.656 A:middle
with the node app and this
can be useful if we want

00:23:41.696 --> 00:23:46.286 A:middle
to sort a list of available
node applications by time

00:23:46.286 --> 00:23:47.926 A:middle
of when they were
most recently used,

00:23:48.336 --> 00:23:50.016 A:middle
the way the home screen does.

00:23:50.686 --> 00:23:52.936 A:middle
So we've gathered up
all this information

00:23:52.936 --> 00:23:56.146 A:middle
about the node application,
and now we've built an array

00:23:56.146 --> 00:24:00.216 A:middle
from which we can drive a
table view and present the user

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:23:56.146 --> 00:24:00.216 A:middle
from which we can drive a
table view and present the user

00:24:00.216 --> 00:24:05.306 A:middle
with a choice of node
applications to deal with.

00:24:06.046 --> 00:24:09.606 A:middle
One wrinkle though is having
cached all that information

00:24:09.606 --> 00:24:13.986 A:middle
in an array, it can become stale
and out of sync with the system.

00:24:14.436 --> 00:24:17.606 A:middle
Most notably, when apps
are installed and deleted

00:24:17.606 --> 00:24:21.656 A:middle
so if you find yourself caching
list of components like this,

00:24:21.656 --> 00:24:25.456 A:middle
you should probably also
listen to this new notification

00:24:25.456 --> 00:24:28.966 A:middle
that we supply, its name is
AudioComponentRegistrations

00:24:28.966 --> 00:24:30.196 A:middle
ChangedNotification.

00:24:30.586 --> 00:24:35.916 A:middle
So, you can pass that
to NSNotification center

00:24:35.916 --> 00:24:37.846 A:middle
to register for a notification.

00:24:38.136 --> 00:24:42.216 A:middle
In this example, we're supplying
a block to be called and then

00:24:42.216 --> 00:24:45.426 A:middle
that block which is called when
the notification or rather,

00:24:45.426 --> 00:24:48.166 A:middle
when the registration has
changed, we can refresh

00:24:48.236 --> 00:24:50.366 A:middle
that cached list of
audio units we built.

00:24:51.016 --> 00:24:55.346 A:middle
So, that's the process of
discovering node apps for host.

00:24:55.696 --> 00:24:57.656 A:middle
So now, we've built up a table

00:24:58.256 --> 00:25:01.656 A:middle
and maybe the user has
selected one of them

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:24:58.256 --> 00:25:01.656 A:middle
and maybe the user has
selected one of them

00:25:01.656 --> 00:25:03.246 A:middle
and in the host application now,

00:25:03.246 --> 00:25:05.116 A:middle
we want to actually
establish a connection

00:25:05.516 --> 00:25:08.836 A:middle
to the node application so
let's look at how that works.

00:25:09.886 --> 00:25:13.716 A:middle
The first step is very
simple because we held

00:25:13.716 --> 00:25:16.646 A:middle
on to the AudioComponent
of that node.

00:25:17.476 --> 00:25:20.356 A:middle
Now, all we have to do is create
an instance of that component

00:25:20.436 --> 00:25:22.266 A:middle
and now, we have an audio unit

00:25:22.386 --> 00:25:24.296 A:middle
through which we can
communicate with the node.

00:25:24.756 --> 00:25:27.746 A:middle
It's worth mentioning
that this is the moment

00:25:27.746 --> 00:25:30.556 A:middle
at which the node
application will get launched

00:25:30.556 --> 00:25:32.506 A:middle
into the background if it
it's not already running.

00:25:33.796 --> 00:25:36.006 A:middle
And we'll look it all the
mechanics of what happens

00:25:36.006 --> 00:25:37.486 A:middle
on the node side of that later.

00:25:37.826 --> 00:25:39.736 A:middle
Right now, we're just going
to focus on the host side.

00:25:40.446 --> 00:25:44.826 A:middle
So, the host has to do a fair--
a few steps here to get ready

00:25:44.906 --> 00:25:48.046 A:middle
to steam audio between
itself and the node.

00:25:49.396 --> 00:25:53.406 A:middle
Most importantly, the
host must be communicating

00:25:53.406 --> 00:25:57.376 A:middle
with the node using the same
hardware sample rate as--

00:25:57.466 --> 00:25:59.146 A:middle
or the same sample
rate as the hardware.

00:25:59.806 --> 00:26:04.036 A:middle
So, to be absolutely sure the
hardware sample rate is what

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:25:59.806 --> 00:26:04.036 A:middle
So, to be absolutely sure the
hardware sample rate is what

00:26:04.036 --> 00:26:07.156 A:middle
it's supposed to be, we should
be making our audio session

00:26:07.156 --> 00:26:08.666 A:middle
active if we haven't already.

00:26:09.996 --> 00:26:11.876 A:middle
So, once having done that,

00:26:11.876 --> 00:26:14.786 A:middle
then we can specify the audio
stream basic description

00:26:15.236 --> 00:26:19.516 A:middle
which is a detailed
description of the audio format

00:26:20.466 --> 00:26:23.116 A:middle
that the host wishes to use
to communicate with the node.

00:26:24.286 --> 00:26:26.936 A:middle
So, we can choose
mono or stereo.

00:26:26.936 --> 00:26:28.756 A:middle
In this example,
I've chosen stereo.

00:26:28.756 --> 00:26:32.136 A:middle
Here is where we're using
the hardware sample rate

00:26:32.756 --> 00:26:38.166 A:middle
And these lines of code here
are basically specifying 32 bit

00:26:38.166 --> 00:26:40.206 A:middle
floating-point, non-interleaved.

00:26:40.596 --> 00:26:43.866 A:middle
Now, the host application can
choose any format it likes here

00:26:44.006 --> 00:26:47.356 A:middle
and the system will perform
whatever conversions are

00:26:47.356 --> 00:26:50.926 A:middle
necessary as long as there's
not a sample rate conversion

00:26:51.086 --> 00:26:51.786 A:middle
being requested.

00:26:51.786 --> 00:26:56.196 A:middle
Again, you must use the sample
rate that matches the hardware.

00:26:57.426 --> 00:26:58.136 A:middle
All right.

00:26:58.136 --> 00:27:00.896 A:middle
Now, we have built up an
audio stream basic description

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:26:58.136 --> 00:27:00.896 A:middle
Now, we have built up an
audio stream basic description

00:27:00.896 --> 00:27:02.906 A:middle
and we can use
AudioUnitSetProperty

00:27:02.906 --> 00:27:06.496 A:middle
on the node AudioUnit for
the stream format property

00:27:07.366 --> 00:27:10.476 A:middle
and this is specifying-- since
it's in the output scope,

00:27:10.476 --> 00:27:14.596 A:middle
this is specifying the output
format of the audio we need

00:27:14.596 --> 00:27:15.746 A:middle
to receive from the node.

00:27:15.866 --> 00:27:18.856 A:middle
If we're working with a
generator or instrument

00:27:18.856 --> 00:27:21.276 A:middle
which don't take audio
input, that's it, we've--

00:27:21.426 --> 00:27:25.786 A:middle
we're done, we've just
specified the output format.

00:27:26.146 --> 00:27:27.566 A:middle
But if we're dealing
with an effect,

00:27:27.686 --> 00:27:29.966 A:middle
then we should also
specify the input format.

00:27:30.426 --> 00:27:32.856 A:middle
And in many cases, it's
going to be identical

00:27:32.956 --> 00:27:35.396 A:middle
to the output format and so,

00:27:35.396 --> 00:27:39.486 A:middle
we can make that same call
using the input scope just

00:27:39.516 --> 00:27:42.936 A:middle
that the input format that we're
going to supply to the node.

00:27:44.076 --> 00:27:49.806 A:middle
So, having specified formats,
we can look how we're going

00:27:49.806 --> 00:27:52.426 A:middle
to get audio from the
host into the node

00:27:52.926 --> 00:27:56.496 A:middle
and this is starting it get into
the details of multiple ways

00:27:56.496 --> 00:27:58.776 A:middle
that your host may
be interacting

00:27:58.776 --> 00:27:59.866 A:middle
with the node AudioUnit.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:28:00.216 --> 00:28:03.256 A:middle
Now, since we're connecting
input, this is only for effects

00:28:04.616 --> 00:28:08.426 A:middle
and the host at this point
can supply input to a node

00:28:08.426 --> 00:28:12.926 A:middle
from another audio unit using
AUGraphConnectNodeInput.

00:28:13.056 --> 00:28:15.836 A:middle
AUGraph is a higher level API
which I'm just going to touch

00:28:15.836 --> 00:28:20.586 A:middle
on a few times today but you can
use AUGraph to build up graphs

00:28:20.586 --> 00:28:23.236 A:middle
or a series of connections
between audio units.

00:28:24.316 --> 00:28:27.626 A:middle
The other way to make a
connection to the node's input

00:28:27.916 --> 00:28:30.096 A:middle
from some other audio unit is

00:28:30.096 --> 00:28:32.336 A:middle
with the AudioUnitProperty
MakeConnection.

00:28:33.606 --> 00:28:36.956 A:middle
Alternatively, a host can simply
supply a callback function

00:28:37.396 --> 00:28:39.236 A:middle
with the SetRenderCallback
property.

00:28:39.646 --> 00:28:42.066 A:middle
This callback function
gets called at render time

00:28:42.066 --> 00:28:46.016 A:middle
and the host supplies the audio
samples to be given to the node.

00:28:46.426 --> 00:28:49.036 A:middle
Now, as far as connecting
the output of the node,

00:28:49.116 --> 00:28:52.646 A:middle
this too depends on the way
you built your host engine.

00:28:53.196 --> 00:28:54.996 A:middle
If you're using audio
units, you want to connect

00:28:54.996 --> 00:28:57.366 A:middle
to the node output to
some other audio unit.

00:28:57.796 --> 00:29:00.416 A:middle
You can use the MakeConnection
property again.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:28:57.796 --> 00:29:00.416 A:middle
You can use the MakeConnection
property again.

00:29:01.146 --> 00:29:04.846 A:middle
If however you're pulling
audio into a custom engine,

00:29:04.846 --> 00:29:06.426 A:middle
then you would call
AudioUnitRender

00:29:06.996 --> 00:29:09.946 A:middle
but there's no setup
at this time for that.

00:29:11.276 --> 00:29:12.956 A:middle
We'll look at the
rendering process

00:29:12.956 --> 00:29:14.356 A:middle
in more detail a little later.

00:29:14.956 --> 00:29:22.006 A:middle
OK. One last a bit of mechanics
here that a host needs to do

00:29:22.006 --> 00:29:25.486 A:middle
to establish a reliable
connection to a node or actually

00:29:25.486 --> 00:29:29.706 A:middle
to reliably handle bad things
happening with the node is

00:29:30.166 --> 00:29:32.866 A:middle
to look out for what happens
when nodes become disconnected.

00:29:33.196 --> 00:29:35.996 A:middle
This could happen automatically
if the node app crashes,

00:29:36.326 --> 00:29:39.556 A:middle
if the system ejects it from
this memory before being

00:29:39.936 --> 00:29:41.726 A:middle
under memory pressure.

00:29:42.456 --> 00:29:44.196 A:middle
Also, if the host fails

00:29:44.196 --> 00:29:47.836 A:middle
to render the node
application regularly enough,

00:29:47.836 --> 00:29:49.526 A:middle
the system will evict it from--

00:29:49.866 --> 00:29:51.746 A:middle
or I'm sorry, will
break the connection.

00:29:52.206 --> 00:29:56.196 A:middle
When these things happen, then
the node AudioUnit becomes,

00:29:56.256 --> 00:29:59.466 A:middle
in effect of zombie,
meaning that it's--

00:29:59.596 --> 00:30:01.326 A:middle
there's still an
audio unit there.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:29:59.596 --> 00:30:01.326 A:middle
there's still an
audio unit there.

00:30:01.426 --> 00:30:05.356 A:middle
You can make API calls on
it but they won't crash

00:30:06.256 --> 00:30:08.606 A:middle
but you will get errors
back and that's the error

00:30:08.606 --> 00:30:10.746 A:middle
that you'll get back, the
InstanceInvalidated error.

00:30:11.996 --> 00:30:16.136 A:middle
The mechanics of establishing
that disconnection callback -

00:30:16.456 --> 00:30:20.836 A:middle
we call
AudioUnitAddPropertyListener

00:30:20.956 --> 00:30:23.556 A:middle
for this new property
IsInterAppConnected.

00:30:24.696 --> 00:30:26.876 A:middle
Here is what you would do
in the connection listener,

00:30:26.876 --> 00:30:30.996 A:middle
you can fetch current value of
the property and see if it is 0

00:30:31.146 --> 00:30:35.446 A:middle
and if the local variable here
connected has become zero,

00:30:35.946 --> 00:30:38.136 A:middle
then you know the node
application has become

00:30:38.576 --> 00:30:40.646 A:middle
disconnected and you
should react accordingly.

00:30:41.716 --> 00:30:46.806 A:middle
So, all of that prep work
has led us up to the point

00:30:46.806 --> 00:30:49.066 A:middle
or we're ready to actually
initialize the node.

00:30:49.296 --> 00:30:53.736 A:middle
Now, the AudioUnit
initialize call basically says

00:30:53.736 --> 00:30:55.986 A:middle
to the system and
the other AudioUnit.

00:30:56.286 --> 00:30:57.436 A:middle
Here, allocate all

00:30:57.436 --> 00:30:59.406 A:middle
of the resources you
need for rendering.

00:30:59.986 --> 00:31:01.496 A:middle
In the case of inter-app audio,

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:30:59.986 --> 00:31:01.496 A:middle
In the case of inter-app audio,

00:31:01.886 --> 00:31:05.146 A:middle
the system at this point is
also allocating some resources

00:31:05.146 --> 00:31:07.626 A:middle
on behalf of that
connection such as the buffers

00:31:07.626 --> 00:31:10.706 A:middle
between the applications and
the real-time rendering thread

00:31:10.706 --> 00:31:11.966 A:middle
in the node application.

00:31:12.956 --> 00:31:14.656 A:middle
So, it's important to realize.

00:31:14.656 --> 00:31:18.756 A:middle
This is a point at which you are
beginning to consume resources

00:31:19.046 --> 00:31:21.526 A:middle
and as such, you have
the responsibility now

00:31:22.296 --> 00:31:24.706 A:middle
of calling AudioUnitRender
regularly

00:31:25.746 --> 00:31:27.846 A:middle
on this node audio unit.

00:31:29.296 --> 00:31:32.506 A:middle
So, that's the process
of setting up a host

00:31:32.506 --> 00:31:33.666 A:middle
to communicate with a node.

00:31:33.806 --> 00:31:36.896 A:middle
You activate your audio session,
you set your stream formats,

00:31:37.026 --> 00:31:39.966 A:middle
you connect your audio input,
add a disconnection listener,

00:31:40.026 --> 00:31:42.256 A:middle
and finally call
AudioUnitInitialize.

00:31:42.256 --> 00:31:46.666 A:middle
So having done that, you're at
the point now where you're ready

00:31:46.666 --> 00:31:50.116 A:middle
to begin streaming audio
between the two applications.

00:31:51.246 --> 00:31:54.466 A:middle
Let's look inside a host
application's engine

00:31:54.466 --> 00:31:55.206 A:middle
in more detail.

00:31:55.656 --> 00:31:59.036 A:middle
This is kind of a wonderfully
simple way to do things

00:31:59.036 --> 00:32:03.166 A:middle
if you can get your work
done using Apple AudioUnits.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:31:59.036 --> 00:32:03.166 A:middle
if you can get your work
done using Apple AudioUnits.

00:32:03.166 --> 00:32:06.506 A:middle
So, the green box, the
green dotted lines--

00:32:07.256 --> 00:32:09.116 A:middle
box represents a host engine

00:32:09.116 --> 00:32:12.606 A:middle
but those red boxes inside are
all Apple supplied audio units.

00:32:12.876 --> 00:32:15.136 A:middle
So, there is the AURemoteIO,

00:32:15.136 --> 00:32:17.176 A:middle
we have a mixer AudioUnite
feeding that.

00:32:17.586 --> 00:32:20.356 A:middle
In feeding the mixer, we
have a file player AudioUnit

00:32:20.856 --> 00:32:22.836 A:middle
and the node AudioUnit.

00:32:23.736 --> 00:32:27.116 A:middle
But of course, there are many
things you would want to do

00:32:27.116 --> 00:32:29.586 A:middle
with audio that Apple doesn't
give you AudioUnits for.

00:32:29.586 --> 00:32:31.876 A:middle
If you want to that, then
you're going to write some code

00:32:31.876 --> 00:32:34.956 A:middle
of your own represented
by the green box

00:32:34.956 --> 00:32:36.556 A:middle
with the squiggly brackets.

00:32:36.946 --> 00:32:41.336 A:middle
So here, your engine is
feeding the AURemoteIO

00:32:41.536 --> 00:32:44.136 A:middle
and if you've written
an app like this before,

00:32:44.136 --> 00:32:47.386 A:middle
you know the way to provide
input to an AURemoteIO

00:32:47.386 --> 00:32:50.526 A:middle
from your own engine is with
the SetRenderCallback property.

00:32:51.266 --> 00:32:53.666 A:middle
And now in this case,
to fetch the audio

00:32:53.666 --> 00:32:57.316 A:middle
from the node AudioUnit, you
would call AudioUnitRender.

00:32:57.916 --> 00:33:03.126 A:middle
OK. So, that's a bunch
of stuff about how we--

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:32:57.916 --> 00:33:03.126 A:middle
OK. So, that's a bunch
of stuff about how we--

00:33:03.286 --> 00:33:06.116 A:middle
a host application
interact with a node.

00:33:06.116 --> 00:33:09.676 A:middle
One final nice thing to
do for the user here is

00:33:09.676 --> 00:33:11.896 A:middle
to provide a way for the user

00:33:12.666 --> 00:33:15.806 A:middle
to bring the node
application to the foreground.

00:33:16.176 --> 00:33:22.296 A:middle
So, we can do this by asking
the audio unit for a PeerURL

00:33:22.296 --> 00:33:25.436 A:middle
and this URL is only
valid during the life

00:33:25.436 --> 00:33:26.276 A:middle
of the connection.

00:33:26.276 --> 00:33:28.166 A:middle
You don't want to hold on to it

00:33:28.166 --> 00:33:30.266 A:middle
because it's not going
to be useful later.

00:33:30.716 --> 00:33:33.576 A:middle
But right before the user
wants to switch in response

00:33:33.606 --> 00:33:35.016 A:middle
to that Icon tap or whatever,

00:33:35.556 --> 00:33:38.966 A:middle
you can fetch the PeerURL then
pass that to UIApplication

00:33:39.026 --> 00:33:43.196 A:middle
and ask it to open that URL and
that will accomplish the switch

00:33:43.446 --> 00:33:45.936 A:middle
of bringing the node
application to the foreground.

00:33:50.936 --> 00:33:54.446 A:middle
So, let's go back just
a little bit and look

00:33:54.516 --> 00:33:57.156 A:middle
at how node applications
see the process

00:33:57.156 --> 00:33:58.866 A:middle
of becoming connected to hosts.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:34:01.556 --> 00:34:04.956 A:middle
So, the most important thing to
think about here as the author

00:34:04.956 --> 00:34:06.336 A:middle
of a node application is

00:34:06.416 --> 00:34:10.936 A:middle
that when the user opens
your application explicitly

00:34:10.936 --> 00:34:12.496 A:middle
from the home screen,
you're launched

00:34:12.496 --> 00:34:15.366 A:middle
into the foreground state,
you're ready start making music.

00:34:16.576 --> 00:34:19.306 A:middle
But if you're being
launched from the context

00:34:19.306 --> 00:34:22.545 A:middle
of a host application, you're
actually going to get launched

00:34:22.545 --> 00:34:25.045 A:middle
into the background state and
there are some limitations

00:34:25.045 --> 00:34:26.326 A:middle
about what you can do at this--

00:34:26.446 --> 00:34:28.436 A:middle
in this state and there's
also a requirement here.

00:34:29.096 --> 00:34:33.366 A:middle
You can't start running from the
background but you must create

00:34:33.366 --> 00:34:35.646 A:middle
and publish your I/O
unit as I showed earlier.

00:34:36.116 --> 00:34:40.126 A:middle
So, it's probably going
to be necessary and useful

00:34:40.456 --> 00:34:41.696 A:middle
in your node application

00:34:42.016 --> 00:34:44.906 A:middle
to ask UIApplication
what's the state here,

00:34:45.056 --> 00:34:47.196 A:middle
Am I in the background or
am I in the foreground?

00:34:47.496 --> 00:34:48.556 A:middle
and proceed accordingly.

00:34:50.976 --> 00:34:53.315 A:middle
So, node applications to find

00:34:53.315 --> 00:34:55.235 A:middle
out when they're
becoming connected

00:34:55.235 --> 00:34:57.966 A:middle
and disconnected can also listen

00:34:58.036 --> 00:35:01.166 A:middle
for the IsInterAppConnected
property just as I described

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:34:58.036 --> 00:35:01.166 A:middle
for the IsInterAppConnected
property just as I described

00:35:01.526 --> 00:35:03.086 A:middle
for host applications earlier.

00:35:04.596 --> 00:35:07.106 A:middle
For a node application,
you listen to this property

00:35:07.526 --> 00:35:10.036 A:middle
on your AURemoteIO instance.

00:35:11.456 --> 00:35:15.576 A:middle
So, in your property listener,
you can notice the transitions

00:35:15.576 --> 00:35:17.916 A:middle
of this property value
from zero to one.

00:35:18.236 --> 00:35:20.406 A:middle
When you see it becoming
true, then you know

00:35:20.406 --> 00:35:22.936 A:middle
that you're output unit has
been initialized underneath you

00:35:23.166 --> 00:35:26.076 A:middle
and that you should set
your audio session active

00:35:26.126 --> 00:35:27.816 A:middle
if you're going to
access the microphone.

00:35:29.076 --> 00:35:31.886 A:middle
You should at this time start
running because that's kind

00:35:31.886 --> 00:35:35.516 A:middle
of your final step of consent
saying, My engine is all hooked

00:35:35.516 --> 00:35:39.426 A:middle
up and ready to render,
start pulling on me.

00:35:39.986 --> 00:35:43.206 A:middle
You can, at this time,
start running even

00:35:43.206 --> 00:35:44.346 A:middle
if you are in the background.

00:35:44.346 --> 00:35:45.676 A:middle
This is the exception
to the rule

00:35:45.676 --> 00:35:46.826 A:middle
about running in the background.

00:35:47.466 --> 00:35:48.966 A:middle
When you are connected
to the host,

00:35:48.966 --> 00:35:52.066 A:middle
you can start running
in the background.

00:35:52.066 --> 00:35:53.306 A:middle
One further note, if you want

00:35:53.306 --> 00:35:55.476 A:middle
to draw an icon representing
the host

00:35:55.476 --> 00:35:56.856 A:middle
that you've become connected to,

00:35:57.186 --> 00:36:00.286 A:middle
there's a new API called
AudioOutputUnitGetHostIcon.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:35:57.186 --> 00:36:00.286 A:middle
there's a new API called
AudioOutputUnitGetHostIcon.

00:36:03.546 --> 00:36:07.036 A:middle
Pertaining further to the
IsInterAppConnected property,

00:36:07.036 --> 00:36:10.016 A:middle
you also want to watch
for the transition to zero

00:36:10.016 --> 00:36:13.166 A:middle
or false meaning that the host
has disconnected from you.

00:36:13.656 --> 00:36:17.466 A:middle
What you want to do at this
point is understand your output

00:36:17.466 --> 00:36:20.186 A:middle
unit has been uninitialized
and stopped for--

00:36:20.226 --> 00:36:21.296 A:middle
out from underneath you.

00:36:22.636 --> 00:36:25.646 A:middle
Now, if you were
accessing the microphone,

00:36:25.926 --> 00:36:28.206 A:middle
you should set your session
inactive at this time.

00:36:28.906 --> 00:36:31.476 A:middle
However, you might,
in some situations,

00:36:31.476 --> 00:36:35.336 A:middle
find yourself disconnected
while in the foreground.

00:36:35.336 --> 00:36:37.076 A:middle
Maybe the host application
crashed

00:36:37.076 --> 00:36:39.496 A:middle
or the system didn't have enough
memory to keep it running.

00:36:39.886 --> 00:36:42.666 A:middle
So, if that happens,
you probably do want

00:36:42.716 --> 00:36:45.196 A:middle
to start running and keep
your audio session active

00:36:45.196 --> 00:36:47.606 A:middle
or make it active
if it isn't already.

00:36:48.926 --> 00:36:50.746 A:middle
But again, you can only start--

00:36:50.836 --> 00:36:53.906 A:middle
you can only make your session
active and start running

00:36:53.906 --> 00:36:54.906 A:middle
when you're in the foreground.

00:36:55.506 --> 00:36:59.576 A:middle
So, just to reemphasize that.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:37:00.586 --> 00:37:04.246 A:middle
Your node application can
start if you've been connected

00:37:04.246 --> 00:37:05.956 A:middle
to the host or you're
in the foreground

00:37:06.386 --> 00:37:08.326 A:middle
but you can keep running
in to the background

00:37:08.946 --> 00:37:11.366 A:middle
if you're connected, of
course, or if you are

00:37:11.366 --> 00:37:15.156 A:middle
in some other standalone
non inter-app scenario

00:37:15.316 --> 00:37:17.556 A:middle
where your app wants to keep
running in to the background.

00:37:21.506 --> 00:37:25.916 A:middle
Let's look again now at a few
different scenarios involving

00:37:25.946 --> 00:37:27.476 A:middle
how nodes render audio.

00:37:28.276 --> 00:37:30.486 A:middle
This is your normal
standalone mode

00:37:30.486 --> 00:37:31.686 A:middle
when the user has launched you.

00:37:32.346 --> 00:37:34.486 A:middle
You've got your engine
connected to the RemoteIO,

00:37:34.486 --> 00:37:37.036 A:middle
connected to the
audio I/O system.

00:37:37.446 --> 00:37:40.066 A:middle
If you're a generator
or instrument,

00:37:41.406 --> 00:37:44.406 A:middle
you may have your output
completely redirected

00:37:44.406 --> 00:37:45.706 A:middle
to the host.

00:37:47.156 --> 00:37:49.626 A:middle
But if you leave your
input bus enabled

00:37:49.766 --> 00:37:52.986 A:middle
but you advertise yourself
as a generator or instrument,

00:37:53.676 --> 00:37:56.566 A:middle
then you've continued
to receive input

00:37:56.566 --> 00:38:00.126 A:middle
from the microphone even while
your output has been redirected

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:37:56.566 --> 00:38:00.126 A:middle
from the microphone even while
your output has been redirected

00:38:00.126 --> 00:38:01.346 A:middle
to the host application.

00:38:02.186 --> 00:38:04.386 A:middle
Now, this doesn't
add any extra latency

00:38:05.406 --> 00:38:07.436 A:middle
because the system
is smart enough

00:38:07.436 --> 00:38:11.866 A:middle
to deliver your application, the
microphone input first and then

00:38:11.866 --> 00:38:15.936 A:middle
in that same I/O cycle, the
host application will pull

00:38:15.936 --> 00:38:16.646 A:middle
your output.

00:38:17.256 --> 00:38:21.466 A:middle
In the final node
rendering scenario -

00:38:21.866 --> 00:38:24.456 A:middle
is you have in effect
both your input

00:38:24.456 --> 00:38:26.796 A:middle
and output streams are
connected to the host rather

00:38:26.796 --> 00:38:29.386 A:middle
than the audio I/O system.

00:38:29.956 --> 00:38:34.026 A:middle
Node applications can also use

00:38:34.026 --> 00:38:36.506 A:middle
that PeerURL property
I described earlier

00:38:36.666 --> 00:38:40.496 A:middle
to show an icon as
Alec did in his demo.

00:38:41.286 --> 00:38:44.606 A:middle
He showed-- the Garageband
icon in the sampler app.

00:38:45.216 --> 00:38:48.446 A:middle
So, you can fetch that
icon from your remote--

00:38:48.446 --> 00:38:51.046 A:middle
your AURemoteIO instance
in this case.

00:38:51.046 --> 00:38:52.666 A:middle
You can-- I'm sorry.

00:38:52.666 --> 00:38:55.176 A:middle
You can fetch that URL
to accomplish the switch.

00:38:58.936 --> 00:39:00.926 A:middle
OK. Back on the host
side of things,

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:38:58.936 --> 00:39:00.926 A:middle
OK. Back on the host
side of things,

00:39:01.706 --> 00:39:04.496 A:middle
there are a few considerations
about stopping audio rendering.

00:39:05.476 --> 00:39:08.886 A:middle
The normal API calls for this
doing are AudioOutputUnitStop

00:39:08.886 --> 00:39:12.566 A:middle
or AUGraphStop and
what you want to do

00:39:12.566 --> 00:39:16.236 A:middle
at this point is promptly
uninitialize your AudioUnit

00:39:16.236 --> 00:39:17.276 A:middle
representing the node.

00:39:17.976 --> 00:39:20.696 A:middle
That releases the
resources that were allocated

00:39:20.756 --> 00:39:24.446 A:middle
when you initialized it and it
releases you from the promise

00:39:24.496 --> 00:39:25.836 A:middle
to keep rendering frequently.

00:39:27.076 --> 00:39:29.806 A:middle
You can turn around and
reinitialize when the user wants

00:39:29.806 --> 00:39:32.766 A:middle
to start communicating again
or if you're completely done

00:39:32.766 --> 00:39:33.996 A:middle
with that node AudioUnit,

00:39:33.996 --> 00:39:36.346 A:middle
you can call
AudioComponentInstanceDispose

00:39:37.066 --> 00:39:39.746 A:middle
and that's what you would do
the if the user, for example,

00:39:40.006 --> 00:39:43.176 A:middle
explicitly breaks the
connection or if you discover

00:39:43.176 --> 00:39:45.816 A:middle
that the node application
has become invalidated.

00:39:46.426 --> 00:39:50.326 A:middle
So, that's the process
of audio rendering.

00:39:51.056 --> 00:39:54.516 A:middle
Next, I'd like to look at how
we can communicate MIDI events

00:39:54.516 --> 00:39:57.256 A:middle
from host applications
to node applications.

00:39:57.726 --> 00:40:01.746 A:middle
Now, this of course, is
for remote instrument

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:39:57.726 --> 00:40:01.746 A:middle
Now, this of course, is
for remote instrument

00:40:01.816 --> 00:40:03.806 A:middle
and remote music effect nodes.

00:40:05.606 --> 00:40:07.986 A:middle
You would want to use this
if you have MIDI events

00:40:07.986 --> 00:40:11.486 A:middle
that are tightly coupled to your
audio that's being rendered.

00:40:12.126 --> 00:40:16.186 A:middle
It lets you sample-accurately
schedule MIDI note-ons,

00:40:16.186 --> 00:40:19.506 A:middle
control events, pitch-bends,
et cetera.

00:40:19.506 --> 00:40:23.436 A:middle
But this is not recommended as
a way of communicating clock

00:40:23.436 --> 00:40:24.876 A:middle
and time code information.

00:40:25.256 --> 00:40:28.516 A:middle
That's sort of a funny
way to communicate

00:40:28.516 --> 00:40:31.936 A:middle
that you're using seven
bit numbers to break

00:40:31.936 --> 00:40:33.596 A:middle
up timing information.

00:40:33.596 --> 00:40:35.456 A:middle
We actually have a
better way to do that.

00:40:35.916 --> 00:40:39.746 A:middle
I should also mention that this
does not replace the coreMIDI

00:40:39.746 --> 00:40:41.526 A:middle
framework which still has a role

00:40:41.926 --> 00:40:46.786 A:middle
when you're dealing USB MIDI
input and output devices or,

00:40:46.786 --> 00:40:48.326 A:middle
for example, the
MIDI network driver.

00:40:49.056 --> 00:40:51.046 A:middle
You might also be
dealing with applications

00:40:51.046 --> 00:40:53.126 A:middle
that don't support inter-app
audio and you still want

00:40:53.126 --> 00:40:57.236 A:middle
to communicate with them.

00:40:57.416 --> 00:41:00.276 A:middle
So, let's look at how a
host application can send

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:40:57.416 --> 00:41:00.276 A:middle
So, let's look at how a
host application can send

00:41:00.276 --> 00:41:01.246 A:middle
MIDI events.

00:41:01.526 --> 00:41:03.796 A:middle
You might do something
like this in this--

00:41:03.886 --> 00:41:06.526 A:middle
like the sampler demo
app, Alex showed.

00:41:06.686 --> 00:41:08.536 A:middle
It had an on-screen keyboard.

00:41:09.026 --> 00:41:11.966 A:middle
So, whenever the user touches
the key, you send a note-on.

00:41:11.966 --> 00:41:13.946 A:middle
When the key is released,
you send a note-off.

00:41:14.476 --> 00:41:17.816 A:middle
So, the APIs for
sending MIDI events are

00:41:17.816 --> 00:41:22.826 A:middle
in the header file MusicDevice.h
and there is a function

00:41:22.826 --> 00:41:24.966 A:middle
in there called
MusicDeviceMIDIEvent.

00:41:25.846 --> 00:41:28.466 A:middle
Here, you pass the
node AudioUnit,

00:41:28.956 --> 00:41:31.106 A:middle
the three byte MIDI--
MIDI message.

00:41:31.696 --> 00:41:34.826 A:middle
And here, offsetSampleFrames,
the final parameter,

00:41:35.386 --> 00:41:37.716 A:middle
that would be used for
sample-accurate scheduling

00:41:37.716 --> 00:41:42.226 A:middle
but since we're doing this
in kind of a UI context,

00:41:42.446 --> 00:41:45.956 A:middle
we don't really know how to have
that kind of sample accuracy.

00:41:46.256 --> 00:41:49.066 A:middle
I'll get into how
we do in a moment.

00:41:49.066 --> 00:41:51.966 A:middle
So, we just passed this sample
offset frames of zero that--

00:41:52.036 --> 00:41:54.176 A:middle
at that note-on will
appear at the beginning

00:41:54.176 --> 00:41:55.526 A:middle
of the next rendered buffer.

00:41:55.526 --> 00:41:59.446 A:middle
Now, if we do want to
do sample-accurate,

00:41:59.446 --> 00:42:03.006 A:middle
scheduling then we have to
schedule our MIDI events

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:41:59.446 --> 00:42:03.006 A:middle
scheduling then we have to
schedule our MIDI events

00:42:03.006 --> 00:42:05.536 A:middle
on the same thread that
were rendering the audio,

00:42:05.976 --> 00:42:08.036 A:middle
because in that thread context,

00:42:08.136 --> 00:42:11.376 A:middle
we can say where the MIDI
events need to land relative

00:42:11.376 --> 00:42:12.806 A:middle
to the beginning of
that audio buffer.

00:42:13.346 --> 00:42:14.996 A:middle
For instance if that MIDI buffer

00:42:14.996 --> 00:42:19.416 A:middle
or audio buffer rather is 1,024
frames, we might do some math

00:42:19.416 --> 00:42:23.396 A:middle
and figure out, oh, that note-on
needs to land at 412 samples

00:42:23.396 --> 00:42:27.576 A:middle
in to that sample buffer and
we can specify that in our call

00:42:27.636 --> 00:42:29.966 A:middle
to MusicDeviceMIDIEvent.

00:42:30.416 --> 00:42:33.306 A:middle
Now, of course, we can call
MusicDeviceMIDIEvent any number

00:42:33.306 --> 00:42:36.716 A:middle
of times to schedule any number
of events for one render cycle.

00:42:36.716 --> 00:42:40.036 A:middle
I just put these next to each
other to emphasize that you have

00:42:40.036 --> 00:42:43.076 A:middle
to be in the rendering
thread context to be able

00:42:43.116 --> 00:42:45.466 A:middle
to schedule sample-accurately.

00:42:46.076 --> 00:42:48.616 A:middle
Now, for you're using
AUGraph and you want

00:42:48.616 --> 00:42:51.356 A:middle
to schedule sample-accurately,
it's similar

00:42:51.356 --> 00:42:54.566 A:middle
but a little different because
you're not calling AUGgraph--

00:42:54.566 --> 00:42:56.586 A:middle
I'm sorry, you're not
calling AudioUnitRender,

00:42:56.946 --> 00:42:58.776 A:middle
the graph is doing
that on your behalf.

00:42:59.846 --> 00:43:03.306 A:middle
So, the way to do this,
there's an AUGraph API

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:42:59.846 --> 00:43:03.306 A:middle
So, the way to do this,
there's an AUGraph API

00:43:03.306 --> 00:43:06.296 A:middle
that lets you get called
back in the render context

00:43:06.296 --> 00:43:08.716 A:middle
and that's
AUGraphAddRenderNotify.

00:43:09.886 --> 00:43:12.926 A:middle
That gives you a callback
function that the graph calls

00:43:13.206 --> 00:43:15.876 A:middle
at the beginning of the render
cycle before actually pulling

00:43:15.876 --> 00:43:16.866 A:middle
audio from the node.

00:43:17.316 --> 00:43:20.456 A:middle
And that turns out to be
the precisely corrects time

00:43:20.866 --> 00:43:23.956 A:middle
to call MusicDeviceMIDIEvent
to schedule events

00:43:24.376 --> 00:43:25.666 A:middle
for that render cycle.

00:43:26.266 --> 00:43:29.926 A:middle
So, that's the process
of sending MIDIEvents,

00:43:30.586 --> 00:43:33.826 A:middle
let's look at how nodes
receive MIDI Events.

00:43:33.946 --> 00:43:35.756 A:middle
So, we have two basic
functions for sending

00:43:35.756 --> 00:43:37.416 A:middle
and then there's
MusicDeviceMIDIEvent

00:43:37.416 --> 00:43:38.836 A:middle
and MusicDeviceSysEx.

00:43:39.316 --> 00:43:42.846 A:middle
And we have two corresponding
callback functions for the use

00:43:42.846 --> 00:43:45.136 A:middle
of the node application,
the MIDIEventProc

00:43:45.136 --> 00:43:46.496 A:middle
and the MIDISysExProc.

00:43:48.826 --> 00:43:50.636 A:middle
So, in the node application,

00:43:50.636 --> 00:43:52.866 A:middle
here we have an example
of MIDIEventProc.

00:43:52.866 --> 00:43:55.456 A:middle
Well, it doesn't
do much but here is

00:43:55.456 --> 00:43:58.366 A:middle
where you receive each event
that's coming from the host

00:43:58.526 --> 00:44:02.376 A:middle
and typically, you would just
save it up in a local structure

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:43:58.526 --> 00:44:02.376 A:middle
and typically, you would just
save it up in a local structure

00:44:02.976 --> 00:44:06.616 A:middle
and use it the next
time you render a buffer

00:44:08.276 --> 00:44:11.236 A:middle
because this function will
get called at the beginning

00:44:11.236 --> 00:44:14.046 A:middle
of each render cycle
with new events

00:44:14.046 --> 00:44:15.576 A:middle
that apply to that render cycle.

00:44:16.186 --> 00:44:18.586 A:middle
So, having created
that callback function,

00:44:18.586 --> 00:44:21.276 A:middle
we can populate a
structure of callbacks.

00:44:21.666 --> 00:44:23.906 A:middle
You can notice I left
the SysExProc null,

00:44:23.906 --> 00:44:25.876 A:middle
that just means I'm
not going to get called

00:44:25.936 --> 00:44:27.956 A:middle
if there is any SysEx.

00:44:28.036 --> 00:44:32.376 A:middle
We use AudioUnitSetProperty to
install those callbacks and now,

00:44:32.956 --> 00:44:34.486 A:middle
on the node application side,

00:44:34.486 --> 00:44:37.686 A:middle
I'm going to receive each
MIDIEvent as it arrives.

00:44:39.256 --> 00:44:43.256 A:middle
So, that's how hosts
can sent MIDI to nodes.

00:44:43.756 --> 00:44:47.016 A:middle
Let's look now at how host can
communicate their transport

00:44:47.016 --> 00:44:48.656 A:middle
and timeline information
to nodes.

00:44:49.226 --> 00:44:53.476 A:middle
So, the important thing
about this model is

00:44:53.476 --> 00:44:55.546 A:middle
that the host is
always the master here.

00:44:56.256 --> 00:44:58.846 A:middle
The nodes can just find
out where the host is

00:44:58.846 --> 00:45:00.016 A:middle
and synchronize to that.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:44:58.846 --> 00:45:00.016 A:middle
and synchronize to that.

00:45:00.266 --> 00:45:04.976 A:middle
We'll look at how the host can
communicate its musical position

00:45:05.866 --> 00:45:08.126 A:middle
as well as the state
of its transport.

00:45:08.126 --> 00:45:12.366 A:middle
And all of this is highly
precise and it's called

00:45:12.366 --> 00:45:15.246 A:middle
and pertains to the
render context.

00:45:15.596 --> 00:45:21.116 A:middle
So here too, we have a structure
full of callback functions,

00:45:21.226 --> 00:45:22.496 A:middle
we'll look at each of these.

00:45:23.466 --> 00:45:26.496 A:middle
So, this is probably
the most common one

00:45:26.496 --> 00:45:27.816 A:middle
that a host will implement,

00:45:28.206 --> 00:45:30.346 A:middle
this is called the
BeatAndTempo callback.

00:45:31.166 --> 00:45:33.666 A:middle
Here, the host can
say for the beginning

00:45:33.666 --> 00:45:37.136 A:middle
of the current audio buffer,
Where am I in the track

00:45:37.136 --> 00:45:40.826 A:middle
and that could be
in between beats.

00:45:40.826 --> 00:45:44.756 A:middle
The host can also communicate
what the current tempo is.

00:45:44.866 --> 00:45:48.116 A:middle
And so with these two pieces
of information, even--

00:45:48.116 --> 00:45:50.126 A:middle
even only these two
pieces information,

00:45:50.126 --> 00:45:52.776 A:middle
the node can do beat
synchronized effects

00:45:52.776 --> 00:45:54.546 A:middle
from the host for instance.

00:45:55.116 --> 00:46:01.016 A:middle
There's also some more detailed
musical location information

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:45:55.116 --> 00:46:01.016 A:middle
There's also some more detailed
musical location information

00:46:01.016 --> 00:46:03.816 A:middle
supplied by the host such as
the current time signature.

00:46:03.896 --> 00:46:08.556 A:middle
And finally, the host
can communicate some bits

00:46:08.556 --> 00:46:11.126 A:middle
of transport state, most
notably whether it's playing

00:46:11.126 --> 00:46:11.806 A:middle
or recording.

00:46:11.806 --> 00:46:14.546 A:middle
There's also a facility
for the host

00:46:14.546 --> 00:46:17.576 A:middle
to express whether it's
cycling or looping.

00:46:20.636 --> 00:46:23.546 A:middle
So here too, we're installing
a set of callback from--

00:46:23.606 --> 00:46:26.876 A:middle
callback functions
on an audio unit.

00:46:26.876 --> 00:46:29.576 A:middle
The host populates the host
callback info structure,

00:46:29.926 --> 00:46:32.766 A:middle
installs the callback
functions that it implements

00:46:32.956 --> 00:46:34.886 A:middle
and calls AudioUnitSetProperty.

00:46:35.416 --> 00:46:39.576 A:middle
So, once the host does this, the
system will call those callbacks

00:46:39.576 --> 00:46:42.466 A:middle
at the beginning of each
render cycle and communicate

00:46:42.466 --> 00:46:45.436 A:middle
that over-- that information
over to the node process

00:46:46.456 --> 00:46:50.256 A:middle
where the node application
will have access to them.

00:46:50.586 --> 00:46:54.906 A:middle
And the way the node
application gets that access is

00:46:54.906 --> 00:46:56.936 A:middle
by fetching the host
callback property.

00:46:57.926 --> 00:47:01.076 A:middle
It will receive that structure
full of function pointers.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:46:57.926 --> 00:47:01.076 A:middle
It will receive that structure
full of function pointers.

00:47:01.076 --> 00:47:02.656 A:middle
They won't actually
point to functions

00:47:02.656 --> 00:47:05.326 A:middle
in the host process, of course.

00:47:05.526 --> 00:47:08.496 A:middle
We can't make a cross process
call there, but the information,

00:47:08.496 --> 00:47:11.506 A:middle
as I just said, has been
communicated over to the node.

00:47:11.506 --> 00:47:16.006 A:middle
And it can access them there
within its own process.

00:47:17.016 --> 00:47:21.016 A:middle
There are some considerations
of thread safety here.

00:47:21.256 --> 00:47:25.856 A:middle
Most people importantly, since
this information is accurate

00:47:25.856 --> 00:47:28.736 A:middle
as of the beginning of the
render cycle, if you call it

00:47:28.736 --> 00:47:33.446 A:middle
in some other context, you
might get inconsistent results.

00:47:33.826 --> 00:47:38.856 A:middle
It's easiest if you fetch this
information on the render thread

00:47:39.236 --> 00:47:41.466 A:middle
but of course, there are
some cases where you want

00:47:41.466 --> 00:47:43.936 A:middle
to observe a transport
state for instance.

00:47:44.716 --> 00:47:49.986 A:middle
So, we give you a better
way to receive notifications

00:47:49.986 --> 00:47:54.146 A:middle
of transport state changes on
a non-render thread context.

00:47:54.576 --> 00:47:56.236 A:middle
You can install this
property listener

00:47:56.236 --> 00:47:59.176 A:middle
for the HostTransportState
and get a callback

00:47:59.526 --> 00:48:00.796 A:middle
on a non-render thread.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:47:59.526 --> 00:48:00.796 A:middle
on a non-render thread.

00:48:01.456 --> 00:48:05.176 A:middle
Okay, so that's the
process of transport

00:48:05.176 --> 00:48:06.386 A:middle
and timeline information.

00:48:06.386 --> 00:48:09.136 A:middle
Finally, I'd like to look
at the whole mechanism

00:48:09.136 --> 00:48:12.366 A:middle
by which node applications
can send remote control events

00:48:12.606 --> 00:48:13.856 A:middle
to host applications.

00:48:14.016 --> 00:48:16.966 A:middle
To accomplish that, we
have something called

00:48:17.206 --> 00:48:19.476 A:middle
AudioUnitRemoteControlEvents.

00:48:19.646 --> 00:48:21.916 A:middle
Now, there's something
called RemoteControlEvents

00:48:21.916 --> 00:48:23.116 A:middle
in UIKit as well.

00:48:23.116 --> 00:48:26.386 A:middle
Those are kind of in
a different world.

00:48:27.036 --> 00:48:30.286 A:middle
These are more specific to the
needs of audio applications.

00:48:30.776 --> 00:48:33.896 A:middle
So with these events, the
node can control the host

00:48:33.896 --> 00:48:35.366 A:middle
application's transport.

00:48:35.896 --> 00:48:38.566 A:middle
And for now, we have these
three events to find.

00:48:38.716 --> 00:48:42.186 A:middle
You can toggle-- you being a
node application-can toggle the

00:48:42.806 --> 00:48:49.116 A:middle
host's play or pause state, its
recording state and the node,

00:48:49.246 --> 00:48:51.066 A:middle
through an event, can
send the host back

00:48:51.066 --> 00:48:54.096 A:middle
to the beginning of
the song or track.

00:48:54.636 --> 00:48:56.666 A:middle
We do have some sample
applications

00:48:56.666 --> 00:49:00.116 A:middle
where our node applications
have some standard looking

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:48:56.666 --> 00:49:00.116 A:middle
where our node applications
have some standard looking

00:49:00.116 --> 00:49:01.166 A:middle
transport controls.

00:49:01.166 --> 00:49:03.486 A:middle
And we'd like to encourage you
to check those out and use them

00:49:03.486 --> 00:49:06.836 A:middle
in your application so that
we can have a consistent look

00:49:06.836 --> 00:49:08.146 A:middle
and feel for these controls.

00:49:09.796 --> 00:49:14.436 A:middle
So, looking at how node
applications can send

00:49:15.046 --> 00:49:17.356 A:middle
RemoteControlEvents,
first, we want to find

00:49:17.356 --> 00:49:19.736 A:middle
out whether the host actually
is listening and is going

00:49:19.736 --> 00:49:21.416 A:middle
to support them because
if it doesn't,

00:49:21.416 --> 00:49:22.326 A:middle
maybe we don't even want

00:49:22.326 --> 00:49:24.486 A:middle
to bother drawing the
transport controls at all.

00:49:24.996 --> 00:49:26.836 A:middle
So to do that, we can
fetch this property

00:49:26.836 --> 00:49:28.946 A:middle
HostReceivesRemoteControlEvents.

00:49:29.826 --> 00:49:32.186 A:middle
And to actually send
the RemoteControlEvent,

00:49:32.756 --> 00:49:35.916 A:middle
the node calls
AudioUnitSetProperty using the

00:49:35.916 --> 00:49:37.996 A:middle
remote control to
event or I'm sorry,

00:49:37.996 --> 00:49:39.476 A:middle
remote control to host event.

00:49:39.716 --> 00:49:43.066 A:middle
And the value of that
property is the actual control

00:49:43.066 --> 00:49:45.766 A:middle
to be sent, toggle, or
record on this example.

00:49:46.806 --> 00:49:49.386 A:middle
So, there's a node sending
a RemoteControlEvent.

00:49:50.286 --> 00:49:55.146 A:middle
Here is a host receiving
one or rather preparing

00:49:55.146 --> 00:49:56.266 A:middle
to receive them, I should say.

00:49:57.006 --> 00:50:00.046 A:middle
So, to do that, the host creates
a block called the listenerBlock

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:49:57.006 --> 00:50:00.046 A:middle
So, to do that, the host creates
a block called the listenerBlock

00:50:01.766 --> 00:50:06.016 A:middle
and in that block, the host
simply takes the incoming

00:50:06.016 --> 00:50:09.246 A:middle
AudioUnitRemoteControlEvent
and passes it to one

00:50:09.246 --> 00:50:11.886 A:middle
of its own methods called
handleRemoteControlEvent.

00:50:12.626 --> 00:50:16.426 A:middle
Now, that block is in
turn a property value

00:50:16.426 --> 00:50:18.576 A:middle
for the
RemoteControlEventListener

00:50:18.576 --> 00:50:22.266 A:middle
property so the host only
has to set that property

00:50:22.596 --> 00:50:27.836 A:middle
on the node AudioUnit and that
accomplishes the installation

00:50:27.866 --> 00:50:29.856 A:middle
of the listener for
RemoteControlEvents.

00:50:30.306 --> 00:50:33.656 A:middle
Next, I'd like to bring up
my colleague Harry Tormey

00:50:33.656 --> 00:50:35.996 A:middle
to show you about some
of these other aspects

00:50:35.996 --> 00:50:38.096 A:middle
of the inter-app
audio API in action.

00:50:38.886 --> 00:50:42.136 A:middle
>> Thanks Doug.

00:50:42.486 --> 00:50:46.586 A:middle
Hey everybody, my name is Harry
Tormey and I work with Doug

00:50:46.586 --> 00:50:47.946 A:middle
in the Core Audio
Group at Apple.

00:50:48.626 --> 00:50:51.626 A:middle
And today, I'm going to be
giving you a demonstration

00:50:51.626 --> 00:50:53.526 A:middle
of some of the sample
applications we're going

00:50:53.526 --> 00:50:55.136 A:middle
to be releasing on
the developer portal

00:50:55.206 --> 00:50:57.616 A:middle
to illustrate how
inter-app audio works.

00:50:59.066 --> 00:51:00.936 A:middle
The first demo I'm going
to be giving you is

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:50:59.066 --> 00:51:00.936 A:middle
The first demo I'm going
to be giving you is

00:51:00.936 --> 00:51:05.616 A:middle
of a host application connecting
to a sampler node application

00:51:05.896 --> 00:51:07.686 A:middle
and sending it some MIDI events.

00:51:08.506 --> 00:51:11.616 A:middle
So what you see in the screen
up there is a host application

00:51:11.616 --> 00:51:13.276 A:middle
and I'm going to bring up a list

00:51:13.276 --> 00:51:16.996 A:middle
of all the remote instrument
node applications installed

00:51:16.996 --> 00:51:18.186 A:middle
on this device and
I'm going to do

00:51:18.186 --> 00:51:19.776 A:middle
that by touching the
add instrument button.

00:51:21.026 --> 00:51:24.016 A:middle
So, none of these applications
are currently running.

00:51:24.476 --> 00:51:25.886 A:middle
They have just published
themselves

00:51:25.886 --> 00:51:27.506 A:middle
with their audio
component descriptions.

00:51:27.806 --> 00:51:30.516 A:middle
When I select one of these
applications from the list,

00:51:30.746 --> 00:51:33.596 A:middle
it will launch into the
background and connect

00:51:33.596 --> 00:51:35.076 A:middle
to the host application.

00:51:35.456 --> 00:51:37.636 A:middle
So I'm going to do that, I'm
going to select the sampler.

00:51:38.756 --> 00:51:41.786 A:middle
OK. So, you can see
the sampler's icon

00:51:41.786 --> 00:51:44.056 A:middle
up there underneath
the instrument label.

00:51:44.276 --> 00:51:46.246 A:middle
That means it's connected
to the host application.

00:51:46.576 --> 00:51:49.106 A:middle
So, I'm going to bring
up a keyboard in the host

00:51:49.106 --> 00:51:52.166 A:middle
by touching the show
keyboard button and I'm going

00:51:52.166 --> 00:51:55.466 A:middle
to send some MIDI
events from the host

00:51:55.466 --> 00:52:00.556 A:middle
to the sampler by
playing the keys.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:51:55.466 --> 00:52:00.556 A:middle
to the sampler by
playing the keys.

00:52:00.686 --> 00:52:01.436 A:middle
[Music] Totally awesome.

00:52:02.336 --> 00:52:06.866 A:middle
OK. So, what if I want
to change the sample bank

00:52:06.866 --> 00:52:08.056 A:middle
that the sampler is using?

00:52:08.386 --> 00:52:10.616 A:middle
Well, I'm going to have to do
to the sampler and do that.

00:52:11.106 --> 00:52:13.216 A:middle
I'm going to do that by
touching the sampler's icon.

00:52:14.016 --> 00:52:16.816 A:middle
We're now in a separate
application and I'm going

00:52:16.816 --> 00:52:19.826 A:middle
to select a different
sample bank to use so how

00:52:19.826 --> 00:52:21.596 A:middle
about something nice
like a harpsichord?

00:52:22.026 --> 00:52:24.256 A:middle
Let me just do that there.

00:52:24.256 --> 00:52:26.026 A:middle
OK. So now, we're
in harpsichord,

00:52:26.026 --> 00:52:28.546 A:middle
I'm going to touch the
host icon there and go back

00:52:28.546 --> 00:52:29.666 A:middle
to the host application.

00:52:30.356 --> 00:52:36.736 A:middle
Touch the show keyboard
again and listen for it.

00:52:36.736 --> 00:52:36.803 A:middle
[ Music ]

00:52:36.803 --> 00:52:38.576 A:middle
That's a harpsichord.

00:52:39.746 --> 00:52:42.186 A:middle
OK. So, the next thing that
I'm going to show you is how

00:52:42.186 --> 00:52:45.716 A:middle
to use the callbacks that the
host application has published

00:52:45.996 --> 00:52:49.676 A:middle
to get the time code of the
host application when it records

00:52:49.676 --> 00:52:50.616 A:middle
and plays back things.

00:52:50.896 --> 00:52:53.446 A:middle
So one again, I'm going to go
the sampler by touching its icon

00:52:53.446 --> 00:52:57.286 A:middle
and I'm going to touch the
record button and I'm going

00:52:57.286 --> 00:52:59.346 A:middle
to record some audio in the host

00:52:59.346 --> 00:53:01.816 A:middle
so I'm sending a remote
message to the host.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:52:59.346 --> 00:53:01.816 A:middle
so I'm sending a remote
message to the host.

00:53:02.876 --> 00:53:06.836 A:middle
[Music] I'm going
to stop recoding

00:53:06.836 --> 00:53:07.966 A:middle
by touching record button again.

00:53:08.216 --> 00:53:09.366 A:middle
Now, what I want
you to pay attention

00:53:09.366 --> 00:53:12.196 A:middle
to is the blue text
over the play button.

00:53:12.806 --> 00:53:15.286 A:middle
This text is going to be
updated with the callbacks

00:53:15.286 --> 00:53:17.396 A:middle
that the host application
has published and were going

00:53:17.396 --> 00:53:20.686 A:middle
to use this to display a
time code indicating how far

00:53:20.686 --> 00:53:21.786 A:middle
into the recording we are.

00:53:21.786 --> 00:53:28.126 A:middle
So, I'm going to touch the play
button and watch that text.

00:53:28.126 --> 00:53:33.766 A:middle
[Music] So, if I do
that again and I go

00:53:33.766 --> 00:53:36.596 A:middle
to the host application, you'll
see the time code is consistent

00:53:36.596 --> 00:53:38.856 A:middle
across both applications
so let me do that.

00:53:38.856 --> 00:53:41.666 A:middle
Let me press play again and go
to-- back to host application.

00:53:42.116 --> 00:53:50.986 A:middle
[Music] Okay, so
for my grand finale,

00:53:51.326 --> 00:53:54.756 A:middle
I'm going to add an effect
and that effect is going

00:53:54.756 --> 00:53:56.426 A:middle
to be the delay effect
that you saw.

00:53:56.466 --> 00:53:59.046 A:middle
So once again, I touched
the add effect button.

00:53:59.316 --> 00:54:00.786 A:middle
It shows you all of the effects

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:53:59.316 --> 00:54:00.786 A:middle
It shows you all of the effects

00:54:00.786 --> 00:54:02.746 A:middle
that are installed
on this device.

00:54:02.746 --> 00:54:04.876 A:middle
I'm going to select the delay
one, it's going to launch it

00:54:04.876 --> 00:54:07.446 A:middle
and connect to the host.

00:54:07.446 --> 00:54:11.486 A:middle
OK. So in the host, if I touched
the show keyboard button again

00:54:11.486 --> 00:54:13.636 A:middle
and play a note, it's
going to be delayed.

00:54:18.236 --> 00:54:18.836 A:middle
[Music] How about that?

00:54:18.966 --> 00:54:20.776 A:middle
Much cooler than
remote controlled cars.

00:54:21.256 --> 00:54:24.546 A:middle
Okay everyone, that's
me, these demos are all

00:54:24.546 --> 00:54:28.296 A:middle
up on the developer portal and
I'm done with my demo so back

00:54:28.296 --> 00:54:29.746 A:middle
over to you Doug and
thank you very much.

00:54:30.246 --> 00:54:32.806 A:middle
[Applause]

00:54:33.306 --> 00:54:33.896 A:middle
>> Thank you Harry.

00:54:33.896 --> 00:54:37.556 A:middle
Hey, I found the right button.

00:54:38.006 --> 00:54:44.726 A:middle
So, back to some more
mundane matters here.

00:54:44.826 --> 00:54:47.516 A:middle
Dealing with audio session
interruptions, both host

00:54:47.516 --> 00:54:49.346 A:middle
and node applications
need to deal

00:54:49.346 --> 00:54:50.706 A:middle
with audio session
interruptions.

00:54:51.296 --> 00:54:53.996 A:middle
Here, the usual rules
apply namely

00:54:53.996 --> 00:54:56.696 A:middle
that your AURemoteIO gets
stopped underneath you.

00:54:56.696 --> 00:54:59.296 A:middle
But furthermore, in
a host application,

00:54:59.296 --> 00:55:02.096 A:middle
the system will uninitialize
any node AudioUnits

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:54:59.296 --> 00:55:02.096 A:middle
the system will uninitialize
any node AudioUnits

00:55:02.096 --> 00:55:02.956 A:middle
that you have open.

00:55:03.366 --> 00:55:06.126 A:middle
This will reclaim the
resources I've been talking

00:55:06.126 --> 00:55:10.506 A:middle
about that you acquire when you
initialize the node AudioUnit.

00:55:12.076 --> 00:55:14.376 A:middle
One other bit of
housekeeping here,

00:55:14.836 --> 00:55:16.866 A:middle
you can make your
application more robust

00:55:16.866 --> 00:55:19.886 A:middle
if you handle a media
services reset correctly.

00:55:19.946 --> 00:55:25.506 A:middle
It's a little bit hard to test
this sometimes-- oops, but--

00:55:25.506 --> 00:55:28.946 A:middle
let me find my way back.

00:55:29.796 --> 00:55:33.146 A:middle
But if you implement this,

00:55:33.146 --> 00:55:37.476 A:middle
your application will
survive calamities.

00:55:38.276 --> 00:55:40.506 A:middle
So, when this happens, you can--

00:55:40.506 --> 00:55:42.256 A:middle
you will find out that all

00:55:42.256 --> 00:55:44.646 A:middle
of your inter-app audio
connections have been broken,

00:55:44.646 --> 00:55:46.826 A:middle
the component instances
have been invalidated.

00:55:47.236 --> 00:55:50.086 A:middle
So, in a host audio--
host application,

00:55:50.086 --> 00:55:53.376 A:middle
you should dispose your node
AudioUnit and your AURemoteIO.

00:55:53.906 --> 00:55:55.486 A:middle
And in a node application,

00:55:55.636 --> 00:55:57.836 A:middle
you should also dispose
your AURemoteIO.

00:55:58.346 --> 00:56:00.016 A:middle
So in general, it's simplest

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:55:58.346 --> 00:56:00.016 A:middle
So in general, it's simplest

00:56:00.016 --> 00:56:03.026 A:middle
to dispose your entire audio
engine including those Apple

00:56:03.026 --> 00:56:03.936 A:middle
Audio objects.

00:56:04.716 --> 00:56:07.166 A:middle
And then, start over
from scratch

00:56:07.166 --> 00:56:09.246 A:middle
as if your app has
just been launched

00:56:09.476 --> 00:56:10.786 A:middle
and that's the simplest way

00:56:10.786 --> 00:56:14.316 A:middle
to robustly handle the
media services being reset.

00:56:16.856 --> 00:56:19.976 A:middle
Some questions that have come
up in showing this feature

00:56:19.976 --> 00:56:21.626 A:middle
to people, in talking with them,

00:56:21.986 --> 00:56:24.506 A:middle
can you have multiple
host applications?

00:56:24.906 --> 00:56:26.456 A:middle
Yes, if they are all mixable.

00:56:26.996 --> 00:56:29.056 A:middle
If one is unmixable, of course,

00:56:29.056 --> 00:56:32.126 A:middle
it will interrupt everything
else as it takes control.

00:56:32.896 --> 00:56:36.336 A:middle
Also, if you were to have
multiple host that are mixable

00:56:36.336 --> 00:56:41.686 A:middle
and one node application,
only one host can connect

00:56:41.776 --> 00:56:42.936 A:middle
to that node at a time.

00:56:43.686 --> 00:56:45.536 A:middle
Can you have multiple
node applications?

00:56:45.536 --> 00:56:48.646 A:middle
Yes, Harry just showed us that
that's more than possible.

00:56:49.246 --> 00:56:53.016 A:middle
A couple of debugging
tips here you may find

00:56:53.106 --> 00:56:55.076 A:middle
when creating a node application

00:56:55.076 --> 00:56:57.356 A:middle
that you're having
trouble getting it to show

00:56:57.356 --> 00:56:59.236 A:middle
up in host applications.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:57:00.346 --> 00:57:03.116 A:middle
If you see that happening, you
should watch the system log.

00:57:03.246 --> 00:57:05.896 A:middle
We try to leave some
clues for you there

00:57:06.316 --> 00:57:07.696 A:middle
in the form of error messages.

00:57:08.016 --> 00:57:10.376 A:middle
If you see a problem with
your Info.plist entry

00:57:10.376 --> 00:57:13.076 A:middle
which is a little bit
easy to do unfortunately

00:57:13.076 --> 00:57:15.356 A:middle
but if you do see a problem
there, we'll tell you that

00:57:15.846 --> 00:57:18.786 A:middle
and I would recommend going
and comparing your Info.plist

00:57:19.036 --> 00:57:21.656 A:middle
with the one-- in one of
our example applications.

00:57:21.896 --> 00:57:26.556 A:middle
I should also mention here
the infamous error of 12,985

00:57:26.586 --> 00:57:29.666 A:middle
which many people stub
their toes on in a lot

00:57:29.666 --> 00:57:30.786 A:middle
of different contexts.

00:57:31.256 --> 00:57:34.786 A:middle
I can tell you that what it
means is operation denied.

00:57:35.436 --> 00:57:40.256 A:middle
And in the context of inter-app
audio, you're likely to hit it

00:57:40.416 --> 00:57:41.986 A:middle
if you start playing
from the background.

00:57:42.886 --> 00:57:47.136 A:middle
We do hope to in an upcoming
release give that a proper name

00:57:47.136 --> 00:57:50.036 A:middle
and maybe another
value but in any case,

00:57:50.036 --> 00:57:51.546 A:middle
if you do see it,
that's what it means.

00:57:52.636 --> 00:57:55.866 A:middle
So we've looked at how node
applications register themselves

00:57:55.866 --> 00:57:58.156 A:middle
with the system,
hosts discover them.

00:57:58.846 --> 00:58:01.856 A:middle
Hosts create connections
to node applications.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:57:58.846 --> 00:58:01.856 A:middle
Hosts create connections
to node applications.

00:58:03.166 --> 00:58:06.946 A:middle
Once that connection is up, host
and node apps can stream audio

00:58:06.946 --> 00:58:08.206 A:middle
to and from each other.

00:58:08.526 --> 00:58:11.066 A:middle
Host apps can send MIDI
to node applications.

00:58:11.736 --> 00:58:13.646 A:middle
Hosts can communicate
their transport

00:58:13.646 --> 00:58:14.916 A:middle
and timeline information.

00:58:15.416 --> 00:58:19.176 A:middle
And finally, we have seen how
nodes can remotely control hosts

00:58:20.396 --> 00:58:23.246 A:middle
so I think if you
have an existing music

00:58:23.246 --> 00:58:26.086 A:middle
or audio application,
it's not that much work

00:58:26.086 --> 00:58:27.466 A:middle
to convert it to a node.

00:58:27.996 --> 00:58:30.206 A:middle
It's mostly adding a
little bit of code to deal

00:58:30.206 --> 00:58:32.966 A:middle
with the transitions to and
from the connected state

00:58:33.016 --> 00:58:34.596 A:middle
and you can look how that works

00:58:34.966 --> 00:58:37.286 A:middle
in the example apps
we have posted.

00:58:37.896 --> 00:58:40.156 A:middle
Creating a host application
is a bit more work

00:58:40.156 --> 00:58:43.016 A:middle
but you're using existing
API for audio units

00:58:43.016 --> 00:58:45.726 A:middle
and there's a lot of history
there as well as powerful--

00:58:46.046 --> 00:58:47.706 A:middle
there's a lot of
power and flexibility.

00:58:49.186 --> 00:58:50.276 A:middle
We also like you to--

00:58:50.726 --> 00:58:53.786 A:middle
we encourage you to look
at our sample applications.

00:58:54.336 --> 00:58:56.646 A:middle
They'll help you with a lot
of the little ins and outs

00:58:56.996 --> 00:58:58.156 A:middle
and we're really looking forward

00:58:58.156 --> 00:59:01.296 A:middle
to the great music apps
you're going to make.

WEBVTT
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000

00:58:58.156 --> 00:59:01.296 A:middle
to the great music apps
you're going to make.

00:59:01.846 --> 00:59:05.416 A:middle
On to some housekeeping matters
here, if you wish to talk

00:59:05.416 --> 00:59:07.656 A:middle
to an Apple Evangelist,
there's John Geleynse.

00:59:07.956 --> 00:59:11.056 A:middle
Here are some links
to some documentation

00:59:11.056 --> 00:59:12.546 A:middle
and our developer forums.

00:59:13.176 --> 00:59:15.346 A:middle
This is the only Core
Audio Session this year

00:59:15.806 --> 00:59:18.756 A:middle
but here are some other media
sessions later this week

00:59:18.756 --> 00:59:19.926 A:middle
that you might be interested in.

00:59:20.766 --> 00:59:21.806 A:middle
Thank you very much.

00:59:22.306 --> 00:59:29.690 A:middle
[ Silence ]