Add secondary actions when swiping down on iPadOS

[danielsaidi]

The native iOS keyboard has swipe support for its buttons, which means that you can have two actions on the same button. This should be added to KeyboardKit as well.

[nikans]

You mean for iPad? I'm currently thinking on how to create a feasible native-like iPad style for your keyboard. Any ideas?

[danielsaidi]

I was thinking about the secondary actions that you have on iPad, where you can swipe down to perform another action. I think I'll wait until SwiftUI matures enough to make it easier to achieve.

After using SwiftUI to build a Keyboard, I'm also not sure if KeyboardKit should provide a lot of UI stuff. Things that are really hard in UIKit are really easy in SwiftUI, so perhaps the library should be more focused on the model and ignore the UI layer?

[digitalheir]

I've made a quick prototype for this behaviour in SwiftUI -- maybe it's useful to you?

It's not perfect: it performs both a tap and a secondary action when the finger has not dragged far enough to cancel tap. But I feels quite nice already.

import SwiftUI 

public struct KeyWithSuper : View {
    public init(keyHeight: CGFloat, fontSize: CGFloat, label: String, output: String, alt: String, altOutput: String?, onKeyPress: @escaping (String) -> Void) {
        self.keyHeight = keyHeight
        self.fontSize = fontSize
        self.label = label
        self.output = output
        self.alt = alt
        self.altOutput = altOutput
        self.onKeyPress = onKeyPress
    }
    
    private let keyHeight: CGFloat 
    private let fontSize: CGFloat
    
    private let label: String
    private let output: String
    
    private let alt: String
    private let altOutput: String?
    
    public let onKeyPress: (String) -> Void
    
    @GestureState private var dragProgress: Double = -1.0

    private var dragThreshold: CGFloat { keyHeight * 0.7 } 
    
    private var dragPerc: Double {
        max(0.0, dragProgress)
    }
    
    private var altOpacity: Double {
        return 0.4 + 0.6 * (dragPerc * Double((1 - altLabelAlpha) + altLabelAlpha))
    }
    private var altOpacityMain: Double {
        return (1.0 - dragPerc)
    }
    private var scaleFactor: CGFloat {
        CGFloat((CGFloat(dragPerc) * (1 - altLabelScale)) + altLabelScale)
    }
    
    private var scaleFactorMain: CGFloat {
        CGFloat(0.4 + (0.5 * (1.0 - dragPerc)))
    }
    
    private var offsetY: CGFloat {
        (CGFloat(1.0 - dragPerc) * keyHeight) * -0.3
    }
    
    private var offsetYMain: CGFloat {
        ((CGFloat(dragPerc) * 0.2) + 0.1) * keyHeight
    }
    
    public var body: some View {
        ZStack(alignment: .center) { 
                    RoundedRectangle(
                        cornerRadius: 8)
                        .foregroundColor(Color.gray)
                        .padding(2)
            Text(alt)
                .opacity(altOpacity)
                .scaleEffect(scaleFactor)
                .offset(y: offsetY)
            Text(label)
                .opacity(altOpacityMain)
                .scaleEffect(scaleFactorMain)
                .offset(y: offsetYMain)
                .font(.system(size: fontSize))
        }
        .gesture(
            DragGesture(minimumDistance: 0, coordinateSpace: .local)
                    .updating($dragProgress) { value, p, transaction in
                        let dragDelta = value.translation.height
                        // print("\(label) drag progress \(dragDelta) / \(dragThreshold) = \(dragDelta / dragThreshold)")
                        p = max(0.0, Double(min(1.0, dragDelta / dragThreshold)))
                    }
                    .onEnded { (gesture: DragGesture.Value) in
                        dragEnd(gesture.translation.height)
                    }
        )
        .simultaneousGesture(TapGesture() 
          .onEnded({
            if(dragProgress < 1.0) {
                onKeyPress(output)
            }
            // todo how to determine whether the slide was active and tap should be disabled?
        }))    
    }
         
    private func gestureEnded(_ d: CGFloat) {
        if (d >= dragThreshold) {
           onKeyPress(altOutput ?? alt)
        }
    }

    private func dragEnd(_ gesture: CGFloat) {
        gestureEnded(gesture)
    } 
}
 

struct KeyWithSuper_Previews: PreviewProvider {
    
    static var previews: some View {
        KeyWithSuper(
            keyHeight: 48,
            fontSize: 32,
            label: "a", output: "a",
            alt: "c", altOutput: "c"
        ) { _ in            print("hehe")        }
            .foregroundColor(.blue)
            .padding()
            .background(Color.red)
            .frame(width: 200, height: 200)
    }
}

[danielsaidi]

Wow, cool! I’ll give it a try, thanks!

Not to get personal here, but just want to give a heads up that I’m currently recovering from surgery and may not be on this project for a few days or so.

[digitalheir]

Oh, hope everything went well and you are healthy! (Can't resist the urge to go off-topic, sorry.)

[danielsaidi]

Thanks! I shattered my shoulder last week and underwent surgery two days ago, so I am currently struggling with typing on my computer. I get easily tired, but I'm so relieved that I can type anything at all.

[lvandal]

+1 for adding this feature.

[danielsaidi]

This should be doable since we now have a nice swipe gesture for each key.

I think the feature should be broken up into the following steps:

• Specify an optional secondary action for each input in the input set
• Trigger that action on swipe down (iPad only?)
• Update the button design if a secondary action exists
• Implement the transition on swipe

Let's implement these features individually and mark this issue as done once they're all done.

[danielsaidi]

I realized that I will have to plan this after the new gesture engine is done.

[f-person]

Trigger that action on swipe down (iPad only?)

I think adding a swipe down trigger for the action would be useful not only on iPad but also on iPhone. It could serve as an alternative method to trigger keys with a single callout option in certain scenarios (e.g., a letter is only available as a callout character). For me personally, any default behavior is fine as long as it's customizable :)