Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WebGPU] MTLResource allocations should be attributed to the respective web process #11787

Conversation

mwyrzykowski
Copy link
Contributor

@mwyrzykowski mwyrzykowski commented Mar 22, 2023

d984488

[WebGPU] MTLResource allocations should be attributed to the respective web process
https://bugs.webkit.org/show_bug.cgi?id=254219
<radar://107001772>

Reviewed by Tadeu Zagallo and Myles C. Maxfield.

Attribute memory used by IOSurfaces and MTLResource instances to the
corresponding web process, so we do not exceed jetsam limits on iOS.

Mostly this involves piping the web process ID which already is
accessible via RemoteGPU down to WebGPU.framework and then calling the
appropriate IOSurface and Metal SPIs.

* Source/WebCore/PAL/pal/graphics/WebGPU/Impl/WebGPUCreateImpl.cpp:
(PAL::WebGPU::create):
* Source/WebCore/PAL/pal/graphics/WebGPU/Impl/WebGPUCreateImpl.h:
* Source/WebGPU/WebGPU/Buffer.mm:
(WebGPU::Device::safeCreateBuffer const):
* Source/WebGPU/WebGPU/Device.h:
* Source/WebGPU/WebGPU/Device.mm:
(WebGPU::setOwnerWithIdentity):
(WebGPU::Device::setOwnerWithIdentity const):
(WebGPU::Device::newBufferWithBytes const):
(WebGPU::Device::newTextureWithDescriptor const):
* Source/WebGPU/WebGPU/Instance.h:
* Source/WebGPU/WebGPU/Instance.mm:
(WebGPU::Instance::create):
(WebGPU::Instance::Instance):
(WebGPU::m_webProcessID):
(WebGPU::Instance::createSurface):
(WebGPU::Instance::webProcessID const):
* Source/WebGPU/WebGPU/PresentationContext.h:
* Source/WebGPU/WebGPU/PresentationContext.mm:
(WebGPU::PresentationContext::create):
* Source/WebGPU/WebGPU/PresentationContextIOSurface.h:
* Source/WebGPU/WebGPU/PresentationContextIOSurface.mm:
(WebGPU::PresentationContextIOSurface::create):
(WebGPU::PresentationContextIOSurface::PresentationContextIOSurface):
(WebGPU::PresentationContextIOSurface::renderBuffersWereRecreated):
(WebGPU::PresentationContextIOSurface::configure):
* Source/WebGPU/WebGPU/Queue.mm:
(WebGPU::Queue::writeBuffer):
(WebGPU::Queue::writeTexture):
* Source/WebGPU/WebGPU/Texture.mm:
(WebGPU::Device::createTexture):
* Source/WebGPU/WebGPU/WebGPUExt.h:
* Source/WebKit/GPUProcess/graphics/WebGPU/RemoteGPU.cpp:
(WebKit::webProcessID):
(WebKit::RemoteGPU::workQueueInitialize):

* Source/WebGPU/WebGPU/MetalSPI.h:
Add SPI header.

Canonical link: https://commits.webkit.org/276332@main

9d70059

Misc iOS, tvOS & watchOS macOS Linux Windows
βœ… πŸ§ͺ style βœ… πŸ›  ios βœ… πŸ›  mac βœ… πŸ›  wpe βœ… πŸ›  wincairo
βœ… πŸ§ͺ bindings βœ… πŸ›  ios-sim βœ… πŸ›  mac-AS-debug   πŸ§ͺ wpe-wk2
βœ… πŸ§ͺ webkitperl βœ… πŸ§ͺ ios-wk2 βœ… πŸ§ͺ api-mac βœ… πŸ§ͺ api-wpe
βœ… πŸ§ͺ ios-wk2-wpt βœ… πŸ§ͺ mac-wk1 βœ… πŸ›  wpe-skia
⏳ πŸ›  πŸ§ͺ jsc βœ… πŸ§ͺ api-ios βœ… πŸ§ͺ mac-wk2 βœ… πŸ›  gtk
⏳ πŸ›  πŸ§ͺ jsc-arm64 βœ… πŸ›  tv βœ… πŸ§ͺ mac-AS-debug-wk2 βœ… πŸ§ͺ gtk-wk2
βœ… πŸ›  tv-sim   πŸ§ͺ api-gtk
βœ… πŸ›  πŸ§ͺ merge loading πŸ›  watch
βœ… πŸ›  watch-sim

@mwyrzykowski mwyrzykowski self-assigned this Mar 22, 2023
@mwyrzykowski mwyrzykowski added the WebGPU For bugs in WebGPU label Mar 22, 2023
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@mwyrzykowski mwyrzykowski removed the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from c73c13e to 19866b4 Compare March 22, 2023 01:08
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@@ -189,6 +198,24 @@ static void captureFrame(id<MTLDevice> captureObject)
m_isLost = true;
}

static void setOwnerWithIdentity(id<MTLResourceSPI> resource, auto webProcessID)
{
ASSERT([resource respondsToSelector:@selector(setOwnerWithIdentity:)]);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The delegate method is optional hence the respondsToSelector check. But from what I can tell all resources we create implement it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can avoid the double respondsToSelector by structuring the statement as WebKit-style early return + ASSERT_NOT_REACHED()

@mwyrzykowski mwyrzykowski removed the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from 19866b4 to a47903d Compare March 22, 2023 01:26
@@ -96,7 +96,7 @@ class RemoteGPUProxy final : public PAL::WebGPU::GPU, private IPC::Connection::C
}
IPC::Connection& connection() const { return m_gpuProcessConnection->connection(); }

void requestAdapter(const PAL::WebGPU::RequestAdapterOptions&, CompletionHandler<void(RefPtr<PAL::WebGPU::Adapter>&&)>&&) final;
void requestAdapter(const PAL::WebGPU::RequestAdapterOptions&, CompletionHandler<void(RefPtr<PAL::WebGPU::Adapter>&&)>&&, unsigned = 0) final;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the meaning of this? It needs a name. (And it seems unused in the .cpp file)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this up

@@ -118,7 +118,7 @@ void RemoteGPUProxy::waitUntilInitialized()
m_lost = true;
}

void RemoteGPUProxy::requestAdapter(const PAL::WebGPU::RequestAdapterOptions& options, CompletionHandler<void(RefPtr<PAL::WebGPU::Adapter>&&)>&& callback)
void RemoteGPUProxy::requestAdapter(const PAL::WebGPU::RequestAdapterOptions& options, CompletionHandler<void(RefPtr<PAL::WebGPU::Adapter>&&)>&& callback, unsigned)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem used?

This indicates a leaky abstraction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to factor this away to avoid this issue

auto remoteCompositorIntegration = RemoteCompositorIntegration::create(compositorIntegration, m_objectHeap, *m_streamConnection, identifier);
m_objectHeap->addObject(identifier, remoteCompositorIntegration);
}

unsigned RemoteGPU::webProcessID() const
{
return m_gpuConnectionToWebProcess->webProcessIdentity().taskIdToken();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use the identity throughout our code, and only call taskIdToken() at the latest possible moment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ProcessIdentity is in WebCore/platform so unfortunately I think this is the latest possible moment.

We can also forward declare ProcessIdentity and pass it throughout in PAL but then where do we use it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:/ ok

@@ -42,7 +42,7 @@ class GPU : public RefCounted<GPU> {
public:
virtual ~GPU() = default;

virtual void requestAdapter(const RequestAdapterOptions&, CompletionHandler<void(RefPtr<Adapter>&&)>&&) = 0;
virtual void requestAdapter(const RequestAdapterOptions&, CompletionHandler<void(RefPtr<Adapter>&&)>&&, unsigned = 0) = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there a better way to do this? Every GPU in the GPU process will come from a different web process, can't this information be in the GPU object (and not passed to requestAdapter())?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could store it in the GPU object instead. This would require from what I can tell storing the GPU object in the Adapter or the Device so we can get the web process id.

Do you think storing a GPU& in the adapter is a good idea? I can do that if so

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's the design we have already - child objects hold strong references to their parent object

Copy link
Contributor

@litherum litherum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It generally seems good, but I think we should rejigger the handshake between the GPU and the Device, so that we don't have to modify pal/graphics/WebGPU/WebGPU.h.

@@ -44,6 +44,7 @@ class CompositorIntegration : public RefCounted<CompositorIntegration> {

#if PLATFORM(COCOA)
virtual Vector<MachSendRight> recreateRenderBuffers(int width, int height) = 0;
virtual void setWebProcessID(unsigned) { }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, this is a perfect example of why the Compositor Integration object exists - it's our own object with our own API that we fully control.

@@ -64,16 +64,17 @@ class Adapter : public WGPUAdapterImpl, public RefCounted<Adapter> {
void makeInvalid() { m_device = nil; }

Instance& instance() const { return m_instance; }

unsigned webProcessID() const { return m_webProcessID; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this belongs here :/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh maybe it should go in Instance? The name of that had me thinking there was only one Instance instance per GPU process. But it seems there is one Instance instance in the GPU process for each web process?

#if USE(INTERNAL_APPLE_SDK)
#import <Metal/MTLResource_Private.h>
#else
@protocol MTLResourceSPI <MTLResource>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we put these in their own MetalSPI.h file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we introduce a new MetalSPI.h for WebGPU.framework? I don't think one exists yet

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add one since I don't think a single SPI usage warrants its own file

Copy link
Contributor

@litherum litherum Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should make a MetalSPI.h. It's important we keep accounting of SPI clear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SPI header added :)

return [m_device newBufferWithLength:std::max(static_cast<NSUInteger>(1), length) options:resourceOptions];
id<MTLBuffer> buffer = [m_device newBufferWithLength:std::max(static_cast<NSUInteger>(1), length) options:resourceOptions];
ASSERT(buffer);
setOwnerWithIdentity(buffer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand, the setOwnerWithIdentity() function is on Device but I don't see a m_device. here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are inside Device::safeCreateBuffer

id<MTLBuffer> Device::newBufferWithBytes(const void *pointer, NSUInteger length, MTLResourceOptions options) const
{
id<MTLBuffer> buffer = [m_device newBufferWithBytes:pointer length:length options:options];
ASSERT(buffer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really right to ASSERT() success? Don't we need to handle the case where allocation failed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should handle that case. I'll file a bug to track that. The debug assert is to catch the failure when it happens as opposed to letting it go unnoticed.

Copy link
Contributor

@litherum litherum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm switching to approve because I think it's generally good, but I still think we should rejigger the handshake.

void Device::setOwnerWithIdentity(id<MTLResource> resource) const
{
auto webProcessID = m_adapter->webProcessID();
if (!webProcessID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this deserves a comment saying "WebKit1" or "In-process WebGPU" or something

#endif

WTF::Function<void(CompletionHandler<void()>&&)> m_onSubmittedWorkScheduledCallback;

RefPtr<PresentationContextImpl> m_presentationContext;
Ref<ConvertToBackingContext> m_convertToBackingContext;
unsigned m_webProcessID { 0 };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there a mach_foo_t type we can use instead of unsigned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is mach_port_t which is typedef as unsigned int but I don't think that is available in PAL. I could wrap all the usage of it in #ifdef however but this defeats the purpose of having a PAL

I'm going to see if I can refactor some of these changes to have minimal impact on PAL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switched to mach_port_t as most PAL changes were removed. I think it will be fine, if there is a build issue on some platform, I will add a typedef

@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from a47903d to 83eaccb Compare March 22, 2023 03:25
@mwyrzykowski
Copy link
Contributor Author

@litherum it's much simpler now :) in case you want to take a second look

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@mwyrzykowski mwyrzykowski removed the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from 83eaccb to e3cd683 Compare March 22, 2023 04:18
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 22, 2023
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from 7f52e17 to db34b15 Compare March 12, 2024 04:54
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 12, 2024
@mwyrzykowski mwyrzykowski removed the merging-blocked Applied to prevent a change from being merged label Mar 12, 2024
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from db34b15 to 11a3d18 Compare March 12, 2024 16:04
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 12, 2024
@mwyrzykowski mwyrzykowski removed the merging-blocked Applied to prevent a change from being merged label Mar 12, 2024
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from 11a3d18 to d117ecf Compare March 12, 2024 18:38
@mwyrzykowski mwyrzykowski added safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks and removed safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks labels Mar 13, 2024
@webkit-ews-buildbot
Copy link
Collaborator

Failed api-ios checks. Please resolve failures and re-apply safe-merge-queue label.

Rejecting #11787 from merge queue.

@webkit-ews-buildbot webkit-ews-buildbot added merging-blocked Applied to prevent a change from being merged and removed safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks labels Mar 18, 2024
@webkit-ews-buildbot
Copy link
Collaborator

Safe-Merge-Queue: Build #15310.

@mwyrzykowski mwyrzykowski added safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks and removed merging-blocked Applied to prevent a change from being merged safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks labels Mar 18, 2024
@mwyrzykowski mwyrzykowski force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from d117ecf to 9d70059 Compare March 18, 2024 23:28
@mwyrzykowski mwyrzykowski added safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks merge-queue Applied to send a pull request to merge-queue and removed safe-merge-queue Applied to automatically send a pull-request to merge-queue after passing EWS checks labels Mar 19, 2024
…ve web process

https://bugs.webkit.org/show_bug.cgi?id=254219
<radar://107001772>

Reviewed by Tadeu Zagallo and Myles C. Maxfield.

Attribute memory used by IOSurfaces and MTLResource instances to the
corresponding web process, so we do not exceed jetsam limits on iOS.

Mostly this involves piping the web process ID which already is
accessible via RemoteGPU down to WebGPU.framework and then calling the
appropriate IOSurface and Metal SPIs.

* Source/WebCore/PAL/pal/graphics/WebGPU/Impl/WebGPUCreateImpl.cpp:
(PAL::WebGPU::create):
* Source/WebCore/PAL/pal/graphics/WebGPU/Impl/WebGPUCreateImpl.h:
* Source/WebGPU/WebGPU/Buffer.mm:
(WebGPU::Device::safeCreateBuffer const):
* Source/WebGPU/WebGPU/Device.h:
* Source/WebGPU/WebGPU/Device.mm:
(WebGPU::setOwnerWithIdentity):
(WebGPU::Device::setOwnerWithIdentity const):
(WebGPU::Device::newBufferWithBytes const):
(WebGPU::Device::newTextureWithDescriptor const):
* Source/WebGPU/WebGPU/Instance.h:
* Source/WebGPU/WebGPU/Instance.mm:
(WebGPU::Instance::create):
(WebGPU::Instance::Instance):
(WebGPU::m_webProcessID):
(WebGPU::Instance::createSurface):
(WebGPU::Instance::webProcessID const):
* Source/WebGPU/WebGPU/PresentationContext.h:
* Source/WebGPU/WebGPU/PresentationContext.mm:
(WebGPU::PresentationContext::create):
* Source/WebGPU/WebGPU/PresentationContextIOSurface.h:
* Source/WebGPU/WebGPU/PresentationContextIOSurface.mm:
(WebGPU::PresentationContextIOSurface::create):
(WebGPU::PresentationContextIOSurface::PresentationContextIOSurface):
(WebGPU::PresentationContextIOSurface::renderBuffersWereRecreated):
(WebGPU::PresentationContextIOSurface::configure):
* Source/WebGPU/WebGPU/Queue.mm:
(WebGPU::Queue::writeBuffer):
(WebGPU::Queue::writeTexture):
* Source/WebGPU/WebGPU/Texture.mm:
(WebGPU::Device::createTexture):
* Source/WebGPU/WebGPU/WebGPUExt.h:
* Source/WebKit/GPUProcess/graphics/WebGPU/RemoteGPU.cpp:
(WebKit::webProcessID):
(WebKit::RemoteGPU::workQueueInitialize):

* Source/WebGPU/WebGPU/MetalSPI.h:
Add SPI header.

Canonical link: https://commits.webkit.org/276332@main
@webkit-commit-queue webkit-commit-queue force-pushed the eng/WebGPU-MTLResource-allocations-should-be-attributed-to-the-respective-web-process branch from 9d70059 to d984488 Compare March 19, 2024 05:47
@webkit-commit-queue
Copy link
Collaborator

Committed 276332@main (d984488): https://commits.webkit.org/276332@main

Reviewed commits have been landed. Closing PR #11787 and removing active labels.

@webkit-commit-queue webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Mar 19, 2024
@webkit-commit-queue webkit-commit-queue merged commit d984488 into WebKit:main Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
WebGPU For bugs in WebGPU
Projects
None yet
8 participants