Skip to content

Commit

Permalink
Stop using delayedPreviewProviderForDroppingItem to update drop previ…
Browse files Browse the repository at this point in the history 
…ews asynchronously

https://bugs.webkit.org/show_bug.cgi?id=263983
rdar://114331921

Reviewed by Aditya Keerthi.

Stop relying on this private `UIDropInteraction` delegate method. Instead, adopt a new method,
`-[UIDragItem _setNeedsDropPreviewUpdate]`, to inform UIKit that a given item's associated drop
preview is changing. The drop session will then query the normal API again
(`-[UIDropInteractionDelegate dropInteraction:previewForDroppingItem:withDefault:]`), and
automatically retarget the drop animation based on the new result preview.

See below for more details.

* Source/WebKit/Platform/spi/ios/UIKitSPI.h:
* Source/WebKit/UIProcess/ios/DragDropInteractionState.h:
* Source/WebKit/UIProcess/ios/DragDropInteractionState.mm:
(WebKit::createTargetedDragPreview):
(WebKit::DragDropInteractionState::addDefaultDropPreview):
(WebKit::DragDropInteractionState::defaultDropPreview const):
(WebKit::DragDropInteractionState::finalDropPreview const):

Refactor the drag and drop interaction state to accomodate the new way of updating previews on drop.
In particular, replace `m_delayedItemPreviewProviders` with `m_finalDropPreviews` instead. We no
longer need to save a list of preview-provider blocks, and can instead:

1.  Stash away any incoming asynchronously-delivered previews on the interaction state…
2.  Call `-_setNeedsDropPreviewUpdate` on the drag item, and finally…
3.  Return the final drag preview in `-dropInteraction:previewForDroppingItem:withDefault:`, for
    that drag item.

(WebKit::dragItemSupportsAsynchronousUpdates):
(WebKit::DragDropInteractionState::deliverDelayedDropPreview):
(WebKit::DragDropInteractionState::dragAndDropSessionsDidBecomeInactive):
(WebKit::DragDropInteractionState::setDefaultDropPreview): Deleted.
(WebKit::BlockPtr<void): Deleted.
(WebKit::DragDropInteractionState::prepareForDelayedDropPreview): Deleted.
(WebKit::DragDropInteractionState::clearAllDelayedItemPreviewProviders): Deleted.
(WebKit::DragDropInteractionState::dragAndDropSessionsDidEnd): Deleted.
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView cleanUpDragSourceSessionState]):
(-[WKContentView _handleDropByInsertingImagePlaceholders:session:]):
(-[WKContentView dropInteraction:concludeDrop:]):
(-[WKContentView dropInteraction:previewForDroppingItem:withDefault:]):

Return the final drop preview, if available. See above for more details.

(-[WKContentView _dropInteraction:delayedPreviewProviderForDroppingItem:previewProvider:]): Deleted.
* Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm:
(-[UIDragItem _dragAndDropSimulator]):
(-[UIDragItem _setDragAndDropSimulator:]):
(-[UIDragItem _setNeedsDropPreviewUpdate]):
(-[DragAndDropSimulator _resetSimulatedState]):
(-[DragAndDropSimulator runFrom:to:additionalItemRequestLocations:]):
(-[DragAndDropSimulator defaultDropPreviewForItemAtIndex:]):
(-[DragAndDropSimulator _setNeedsDropPreviewUpdate:]):
(-[DragAndDropSimulator _concludeDropAndPerformOperationIfNecessary]):

Update our drag and drop test infrastructure to handle calls to `-_setNeedsDropPreviewUpdate` by
having the drop simulator query `-dropInteraction:previewForDroppingItem:withDefault:` again and
save the result, which aligns with platform behavior support for this new API.

Canonical link: https://commits.webkit.org/270290@main
  • Loading branch information
@whsieh
whsieh committed Nov 6, 2023
1 parent c84d6bc commit 6930358
Show file tree
Hide file tree
Showing 5 changed files with 101 additions and 107 deletions.
4 changes: 4 additions & 0 deletions Source/WebKit/Platform/spi/ios/UIKitSPI.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1196,6 +1196,10 @@ typedef NS_ENUM(NSUInteger, _UIScrollDeviceCategory) {
@property (nonatomic, readonly) CGRect _selectionClipRect;
@end

@interface UIDragItem (Staging_117702233)
- (void)_setNeedsDropPreviewUpdate;
@end

@interface UIDevice ()
@property (nonatomic, setter=_setBacklightLevel:) float _backlightLevel;
@end
Expand Down
24 changes: 6 additions & 18 deletions Source/WebKit/UIProcess/ios/DragDropInteractionState.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,7 @@ struct DragSourceState {
NSInteger itemIdentifier { 0 };
};

struct ItemAndPreview {
RetainPtr<UIDragItem> item;
RetainPtr<UITargetedDragPreview> preview;
};

struct ItemAndPreviewProvider {
RetainPtr<UIDragItem> item;
BlockPtr<void(UITargetedDragPreview *)> provider;
};
using DragItemToPreviewMap = HashMap<RetainPtr<UIDragItem>, RetainPtr<UITargetedDragPreview>>;

enum class AddPreviewViewToContainer : bool;

Expand All @@ -89,13 +81,11 @@ class DragDropInteractionState {
bool shouldRequestAdditionalItemForDragSession(id <UIDragSession>) const;
void dragSessionWillRequestAdditionalItem(void (^completion)(NSArray <UIDragItem *> *));

// These helper methods are unique to UIDropInteraction.
void dropSessionDidEnterOrUpdate(id <UIDropSession>, const WebCore::DragData&);
void dropSessionDidExit() { m_dropSession = nil; }
void dropSessionWillPerformDrop() { m_isPerformingDrop = true; }

// This is invoked when both drag and drop interactions are no longer active.
void dragAndDropSessionsDidEnd();
void dragAndDropSessionsDidBecomeInactive();

CGPoint adjustedPositionForDragEnd() const { return m_adjustedPositionForDragEnd; }
bool didBeginDragging() const { return m_didBeginDragging; }
Expand All @@ -106,17 +96,15 @@ class DragDropInteractionState {
BlockPtr<void(NSArray<UIDragItem *> *)> takeAddDragItemCompletionBlock() { return WTFMove(m_addDragItemCompletionBlock); }
RetainPtr<UIView> takePreviewViewForDragCancel() { return std::exchange(m_previewViewForDragCancel, { }); }

void setDefaultDropPreview(UIDragItem *, UITargetedDragPreview *);
void prepareForDelayedDropPreview(UIDragItem *, void(^provider)(UITargetedDragPreview *preview));
void addDefaultDropPreview(UIDragItem *, UITargetedDragPreview *);
UITargetedDragPreview *finalDropPreview(UIDragItem *) const;
void deliverDelayedDropPreview(UIView *contentView, UIView *previewContainer, const WebCore::TextIndicatorData&);
void deliverDelayedDropPreview(UIView *contentView, CGRect unobscuredContentRect, NSArray<UIDragItem *> *, const Vector<WebCore::IntRect>& placeholderRects);
void clearAllDelayedItemPreviewProviders();

private:
void updatePreviewsForActiveDragSources();
std::optional<DragSourceState> activeDragSourceForItem(UIDragItem *) const;
UITargetedDragPreview *defaultDropPreview(UIDragItem *) const;
BlockPtr<void(UITargetedDragPreview *)> dropPreviewProvider(UIDragItem *);

RetainPtr<UITargetedDragPreview> createDragPreviewInternal(UIDragItem *, UIView *contentView, UIView *previewContainer, AddPreviewViewToContainer, const std::optional<WebCore::TextIndicatorData>&) const;

Expand All @@ -132,8 +120,8 @@ class DragDropInteractionState {

std::optional<DragSourceState> m_stagedDragSource;
Vector<DragSourceState> m_activeDragSources;
Vector<ItemAndPreviewProvider> m_delayedItemPreviewProviders;
Vector<ItemAndPreview> m_defaultDropPreviews;
DragItemToPreviewMap m_defaultDropPreviews;
DragItemToPreviewMap m_finalDropPreviews;
};

} // namespace WebKit
Expand Down
76 changes: 27 additions & 49 deletions Source/WebKit/UIProcess/ios/DragDropInteractionState.mm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,11 @@
static RetainPtr<UITargetedDragPreview> createTargetedDragPreview(UIImage *image, UIView *rootView, UIView *previewContainer, const FloatRect& frameInRootViewCoordinates, const Vector<FloatRect>& clippingRectsInFrameCoordinates, UIColor *backgroundColor, UIBezierPath *visiblePath, AddPreviewViewToContainer addPreviewViewToContainer)
{
if (frameInRootViewCoordinates.isEmpty() || !image || !previewContainer.window)
return nullptr;
return nil;

FloatRect frameInContainerCoordinates = [rootView convertRect:frameInRootViewCoordinates toView:previewContainer];
if (frameInContainerCoordinates.isEmpty())
return nullptr;
return nil;

FloatSize scalingRatio = frameInContainerCoordinates.size() / frameInRootViewCoordinates.size();
auto clippingRectValuesInFrameCoordinates = createNSArray(clippingRectsInFrameCoordinates, [&] (auto rect) {
Expand Down Expand Up @@ -198,76 +198,64 @@ static bool canUpdatePreviewForActiveDragSource(const DragSourceState& source)
updatePreviewsForActiveDragSources();
}

void DragDropInteractionState::setDefaultDropPreview(UIDragItem *item, UITargetedDragPreview *preview)
void DragDropInteractionState::addDefaultDropPreview(UIDragItem *item, UITargetedDragPreview *preview)
{
m_defaultDropPreviews.append({ item, preview });
m_defaultDropPreviews.add(item, preview);
}

UITargetedDragPreview *DragDropInteractionState::defaultDropPreview(UIDragItem *item) const
{
auto matchIndex = m_defaultDropPreviews.findIf([&] (auto& itemAndPreview) {
return itemAndPreview.item == item;
});
return matchIndex == notFound ? nil : m_defaultDropPreviews[matchIndex].preview.get();
return m_defaultDropPreviews.get(item).get();
}

BlockPtr<void(UITargetedDragPreview *)> DragDropInteractionState::dropPreviewProvider(UIDragItem *item)
UITargetedDragPreview *DragDropInteractionState::finalDropPreview(UIDragItem *item) const
{
auto matchIndex = m_delayedItemPreviewProviders.findIf([&] (auto& itemAndProvider) {
return itemAndProvider.item == item;
});

if (matchIndex == notFound)
return nil;

return m_delayedItemPreviewProviders[matchIndex].provider;
return m_finalDropPreviews.get(item).get();
}

void DragDropInteractionState::prepareForDelayedDropPreview(UIDragItem *item, void(^provider)(UITargetedDragPreview *preview))
inline static bool dragItemSupportsAsynchronousUpdates()
{
m_delayedItemPreviewProviders.append({ item, provider });
static bool hasSupport = [UIDragItem instancesRespondToSelector:@selector(_setNeedsDropPreviewUpdate)];
return hasSupport;
}

void DragDropInteractionState::deliverDelayedDropPreview(UIView *contentView, UIView *previewContainer, const WebCore::TextIndicatorData& indicator)
{
if (m_delayedItemPreviewProviders.isEmpty())
return;

auto textIndicatorImage = uiImageForImage(indicator.contentImage.get());
auto preview = createTargetedDragPreview(textIndicatorImage.get(), contentView, previewContainer, indicator.textBoundingRectInRootViewCoordinates, indicator.textRectsInBoundingRectCoordinates, cocoaColor(indicator.estimatedBackgroundColor).get(), nil, AddPreviewViewToContainer::No);
for (auto& itemAndPreviewProvider : m_delayedItemPreviewProviders)
itemAndPreviewProvider.provider(preview.get());
m_delayedItemPreviewProviders.clear();
if (!preview)
return;

for (auto item : m_defaultDropPreviews.keys()) {
m_finalDropPreviews.add(item, preview.get());
if (dragItemSupportsAsynchronousUpdates())
[item _setNeedsDropPreviewUpdate];
}
}

void DragDropInteractionState::deliverDelayedDropPreview(UIView *contentView, CGRect unobscuredContentRect, NSArray<UIDragItem *> *items, const Vector<IntRect>& placeholderRects)
{
if (items.count != placeholderRects.size()) {
RELEASE_LOG(DragAndDrop, "Failed to animate image placeholders: number of drag items (%tu) does not match number of placeholders (%tu)", items.count, placeholderRects.size());
clearAllDelayedItemPreviewProviders();
RELEASE_LOG_ERROR(DragAndDrop, "Failed to animate image placeholders: number of drag items (%tu) does not match number of placeholders (%tu)", items.count, placeholderRects.size());
return;
}

for (size_t i = 0; i < placeholderRects.size(); ++i) {
UIDragItem *item = [items objectAtIndex:i];
auto& placeholderRect = placeholderRects[i];
auto provider = dropPreviewProvider(item);
if (!provider)
continue;

auto defaultPreview = defaultDropPreview(item);
auto defaultPreviewSize = [defaultPreview size];
if (!defaultPreview || defaultPreviewSize.width <= 0 || defaultPreviewSize.height <= 0 || placeholderRect.isEmpty()) {
provider(nil);
if (!defaultPreview || defaultPreviewSize.width <= 0 || defaultPreviewSize.height <= 0 || placeholderRect.isEmpty())
continue;
}

FloatRect previewIntersectionRect = enclosingIntRect(CGRectIntersection(unobscuredContentRect, placeholderRect));
if (previewIntersectionRect.isEmpty()) {
// If the preview rect is completely offscreen, don't bother trying to clip out or scale the default preview;
// simply retarget the default preview.
auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:contentView center:placeholderRect.center()]);
provider([defaultPreview retargetedPreviewWithTarget:target.get()]);
m_finalDropPreviews.add(item, [defaultPreview retargetedPreviewWithTarget:target.get()]);
if (dragItemSupportsAsynchronousUpdates())
[item _setNeedsDropPreviewUpdate];
continue;
}

Expand All @@ -288,18 +276,10 @@ static bool canUpdatePreviewForActiveDragSource(const DragSourceState& source)
auto transform = CGAffineTransformMakeScale(placeholderRect.width() / defaultPreviewSize.width, placeholderRect.height() / defaultPreviewSize.height);
auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:contentView center:previewIntersectionRect.center() transform:transform]);
[defaultPreview parameters].visiblePath = [UIBezierPath bezierPathWithRect:insetPreviewBounds];
auto newPreview = adoptNS([[UITargetedDragPreview alloc] initWithView:[defaultPreview view] parameters:[defaultPreview parameters] target:target.get()]);
provider(newPreview.get());
m_finalDropPreviews.add(item, adoptNS([[UITargetedDragPreview alloc] initWithView:[defaultPreview view] parameters:[defaultPreview parameters] target:target.get()]));
if (dragItemSupportsAsynchronousUpdates())
[item _setNeedsDropPreviewUpdate];
}

m_delayedItemPreviewProviders.clear();
}

void DragDropInteractionState::clearAllDelayedItemPreviewProviders()
{
for (auto& itemAndPreviewProvider : m_delayedItemPreviewProviders)
itemAndPreviewProvider.provider(nil);
m_delayedItemPreviewProviders.clear();
}

UITargetedDragPreview *DragDropInteractionState::previewForLifting(UIDragItem *item, UIView *contentView, UIView *previewContainer, const std::optional<WebCore::TextIndicatorData>& indicator) const
Expand Down Expand Up @@ -395,10 +375,8 @@ static bool canUpdatePreviewForActiveDragSource(const DragSourceState& source)
m_stagedDragSource = std::nullopt;
}

void DragDropInteractionState::dragAndDropSessionsDidEnd()
void DragDropInteractionState::dragAndDropSessionsDidBecomeInactive()
{
clearAllDelayedItemPreviewProviders();

if (auto previewView = takePreviewViewForDragCancel())
[previewView removeFromSuperview];

Expand Down
17 changes: 5 additions & 12 deletions Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9366,7 +9366,7 @@ - (void)cleanUpDragSourceSessionState
[self _removeDropCaret];
_shouldRestoreEditMenuAfterDrop = NO;

_dragDropInteractionState.dragAndDropSessionsDidEnd();
_dragDropInteractionState.dragAndDropSessionsDidBecomeInactive();
_dragDropInteractionState = { };
}

Expand Down Expand Up @@ -9981,21 +9981,18 @@ - (BOOL)_handleDropByInsertingImagePlaceholders:(NSArray<NSItemProvider *> *)ite
auto& state = protectedSelf->_dragDropInteractionState;
if (!data || !protectedSelf->_dropAnimationCount) {
RELEASE_LOG(DragAndDrop, "Failed to animate image placeholders: missing text indicator data.");
state.clearAllDelayedItemPreviewProviders();
return;
}

auto snapshotWithoutSelection = data->contentImageWithoutSelection;
if (!snapshotWithoutSelection) {
RELEASE_LOG(DragAndDrop, "Failed to animate image placeholders: missing unselected content image.");
state.clearAllDelayedItemPreviewProviders();
return;
}

auto unselectedSnapshotImage = snapshotWithoutSelection->nativeImage();
if (!unselectedSnapshotImage) {
RELEASE_LOG(DragAndDrop, "Failed to animate image placeholders: could not decode unselected content image.");
state.clearAllDelayedItemPreviewProviders();
return;
}

Expand Down Expand Up @@ -10362,13 +10359,15 @@ - (void)dropInteraction:(UIDropInteraction *)interaction concludeDrop:(id <UIDro
[self _removeContainerForDropPreviews];
[std::exchange(_visibleContentViewSnapshot, nil) removeFromSuperview];
[std::exchange(_unselectedContentSnapshot, nil) removeFromSuperview];
_dragDropInteractionState.clearAllDelayedItemPreviewProviders();
_page->didConcludeDrop();
}

- (UITargetedDragPreview *)dropInteraction:(UIDropInteraction *)interaction previewForDroppingItem:(UIDragItem *)item withDefault:(UITargetedDragPreview *)defaultPreview
{
_dragDropInteractionState.setDefaultDropPreview(item, defaultPreview);
if (auto preview = _dragDropInteractionState.finalDropPreview(item))
return preview;

_dragDropInteractionState.addDefaultDropPreview(item, defaultPreview);

CGRect caretRect = _page->currentDragCaretRect();
if (CGRectIsEmpty(caretRect))
Expand All @@ -10382,12 +10381,6 @@ - (UITargetedDragPreview *)dropInteraction:(UIDropInteraction *)interaction prev
return [defaultPreview retargetedPreviewWithTarget:target.get()];
}

- (void)_dropInteraction:(UIDropInteraction *)interaction delayedPreviewProviderForDroppingItem:(UIDragItem *)item previewProvider:(void(^)(UITargetedDragPreview *preview))previewProvider
{
// FIXME: This doesn't currently handle multiple items in a drop session.
_dragDropInteractionState.prepareForDelayedDropPreview(item, previewProvider);
}

- (void)dropInteraction:(UIDropInteraction *)interaction sessionDidEnd:(id <UIDropSession>)session
{
RELEASE_LOG(DragAndDrop, "Drop session ended: %p (performing operation: %d, began dragging: %d)", session, _dragDropInteractionState.isPerformingDrop(), _dragDropInteractionState.didBeginDragging());
Expand Down
Loading

0 comments on commit 6930358

Please sign in to comment.