Skip to content

Commit

Permalink
Only access ControlInteractivity through the projection (#10051)
Browse files Browse the repository at this point in the history
## Summary of the Pull Request

This forces the `TermControl` to only use `ControlCore` and `ControlInteractivity` via their WinRT projections. We want this, because WinRT projections can be used across process boundaries. In the future, `ControlCore` and `ControlInteractivity` are going to be living in a different process entirely from `TermControl`. By enforcing this boundary now, we can make sure that they will work seamlessly in the future.

## References
* Tear-out: #1256
* Megathread: #5000
* Project: https://github.com/microsoft/terminal/projects/5

## PR Checklist
* [x] Closes https://github.com/microsoft/terminal/projects/5#card-50760270
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated

## Detailed Description of the Pull Request / Additional comments

Most all this was just converting pure c++ types to winrt types when possible. I've added a couple helper projections with `til` converters, which made most of this really easy.

The "`MouseButtonState` needs to be composed of `Int32`s instead of `bool`s" is MENTAL. I have no idea why this is, but when I had the control OOP in the sample, that would crash when trying to de-marshal the bools. BODGY.

The biggest changes are in the way the UIA stuff is hooked up. The UiaEngine needs to be attached directly to the `Renderer`, and it can't be easily projected, so it needs to live next to the `ControlCore`. But the `TermControlAutomationPeer` needed the `UiaEngine` to help implement some interfaces.

Now, there's a new layer we've introduced. `InteractivityAutomationPeer` does the `ITextProvider`, `IControlAccessibilityInfo` and the `IUiaEventDispatcher` thing. `TermControlAutomationPeer` now has a 
`InteractivityAutomationPeer` stashed inside itself, so that it can ask the interactivity layer to do the real work. We still need the `TermControlAutomationPeer` though, to be able to attach to the real UI tree.

## Validation Steps Performed

The terminal behaves basically the same as before.

Most importantly, I whipped out Accessibility Insights, and the Terminal looks the same as before.
  • Loading branch information
zadjii-msft committed Jul 19, 2021
1 parent 8947909 commit 7f3bc3c
Show file tree
Hide file tree
Showing 27 changed files with 968 additions and 349 deletions.
4 changes: 2 additions & 2 deletions src/cascadia/TerminalApp/AppLogic.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1129,7 +1129,7 @@ namespace winrt::TerminalApp::implementation
// - Gets the taskbar state value from the last active control
// Return Value:
// - The taskbar state of the last active control
size_t AppLogic::GetLastActiveControlTaskbarState()
uint64_t AppLogic::GetLastActiveControlTaskbarState()
{
if (_root)
{
Expand All @@ -1142,7 +1142,7 @@ namespace winrt::TerminalApp::implementation
// - Gets the taskbar progress value from the last active control
// Return Value:
// - The taskbar progress of the last active control
size_t AppLogic::GetLastActiveControlTaskbarProgress()
uint64_t AppLogic::GetLastActiveControlTaskbarProgress()
{
if (_root)
{
Expand Down
4 changes: 2 additions & 2 deletions src/cascadia/TerminalApp/AppLogic.h
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ namespace winrt::TerminalApp::implementation

void WindowCloseButtonClicked();

size_t GetLastActiveControlTaskbarState();
size_t GetLastActiveControlTaskbarProgress();
uint64_t GetLastActiveControlTaskbarState();
uint64_t GetLastActiveControlTaskbarProgress();

winrt::Windows::Foundation::IAsyncOperation<winrt::Windows::UI::Xaml::Controls::ContentDialogResult> ShowDialog(winrt::Windows::UI::Xaml::Controls::ContentDialog dialog);

Expand Down
8 changes: 4 additions & 4 deletions src/cascadia/TerminalApp/TerminalPage.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2020,11 +2020,11 @@ namespace winrt::TerminalApp::implementation
// - Gets the taskbar state value from the last active control
// Return Value:
// - The taskbar state of the last active control
size_t TerminalPage::GetLastActiveControlTaskbarState()
uint64_t TerminalPage::GetLastActiveControlTaskbarState()
{
if (auto control{ _GetActiveControl() })
{
return gsl::narrow_cast<size_t>(control.TaskbarState());
return control.TaskbarState();
}
return {};
}
Expand All @@ -2033,11 +2033,11 @@ namespace winrt::TerminalApp::implementation
// - Gets the taskbar progress value from the last active control
// Return Value:
// - The taskbar progress of the last active control
size_t TerminalPage::GetLastActiveControlTaskbarProgress()
uint64_t TerminalPage::GetLastActiveControlTaskbarProgress()
{
if (auto control{ _GetActiveControl() })
{
return gsl::narrow_cast<size_t>(control.TaskbarProgress());
return control.TaskbarProgress();
}
return {};
}
Expand Down
4 changes: 2 additions & 2 deletions src/cascadia/TerminalApp/TerminalPage.h
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ namespace winrt::TerminalApp::implementation
winrt::TerminalApp::IDialogPresenter DialogPresenter() const;
void DialogPresenter(winrt::TerminalApp::IDialogPresenter dialogPresenter);

size_t GetLastActiveControlTaskbarState();
size_t GetLastActiveControlTaskbarProgress();
uint64_t GetLastActiveControlTaskbarState();
uint64_t GetLastActiveControlTaskbarProgress();

void ShowKeyboardServiceWarning();
winrt::hstring KeyboardServiceDisabledText();
Expand Down
53 changes: 44 additions & 9 deletions src/cascadia/TerminalControl/ControlCore.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,16 @@ namespace winrt::Microsoft::Terminal::Control::implementation
// - Updates last hovered cell, renders / removes rendering of hyper-link if required
// Arguments:
// - terminalPosition: The terminal position of the pointer
void ControlCore::UpdateHoveredCell(const std::optional<til::point>& terminalPosition)
void ControlCore::SetHoveredCell(Core::Point pos)
{
_updateHoveredCell(std::optional<til::point>{ pos });
}
void ControlCore::ClearHoveredCell()
{
_updateHoveredCell(std::nullopt);
}

void ControlCore::_updateHoveredCell(const std::optional<til::point> terminalPosition)
{
if (terminalPosition == _lastHoveredCell)
{
Expand Down Expand Up @@ -477,7 +486,7 @@ namespace winrt::Microsoft::Terminal::Control::implementation
return winrt::hstring{ _terminal->GetHyperlinkAtPosition(pos) };
}

winrt::hstring ControlCore::GetHoveredUriText() const
winrt::hstring ControlCore::HoveredUriText() const
{
auto lock = _terminal->LockForReading(); // Lock for the duration of our reads.
if (_lastHoveredCell.has_value())
Expand All @@ -487,9 +496,9 @@ namespace winrt::Microsoft::Terminal::Control::implementation
return {};
}

std::optional<til::point> ControlCore::GetHoveredCell() const
Windows::Foundation::IReference<Core::Point> ControlCore::HoveredCell() const
{
return _lastHoveredCell;
return _lastHoveredCell.has_value() ? Windows::Foundation::IReference<Core::Point>{ _lastHoveredCell.value() } : nullptr;
}

// Method Description:
Expand Down Expand Up @@ -895,6 +904,24 @@ namespace winrt::Microsoft::Terminal::Control::implementation
return _actualFont;
}

winrt::Windows::Foundation::Size ControlCore::FontSize() const noexcept
{
const auto fontSize = GetFont().GetSize();
return {
::base::saturated_cast<float>(fontSize.X),
::base::saturated_cast<float>(fontSize.Y)
};
}
winrt::hstring ControlCore::FontFaceName() const noexcept
{
return winrt::hstring{ GetFont().GetFaceName() };
}

uint16_t ControlCore::FontWeight() const noexcept
{
return static_cast<uint16_t>(GetFont().GetWeight());
}

til::size ControlCore::FontSizeInDips() const
{
const til::size fontSize{ GetFont().GetSize() };
Expand Down Expand Up @@ -1077,10 +1104,18 @@ namespace winrt::Microsoft::Terminal::Control::implementation
return _settings.CopyOnSelect();
}

std::vector<std::wstring> ControlCore::SelectedText(bool trimTrailingWhitespace) const
Windows::Foundation::Collections::IVector<winrt::hstring> ControlCore::SelectedText(bool trimTrailingWhitespace) const
{
// RetrieveSelectedTextFromBuffer will lock while it's reading
return _terminal->RetrieveSelectedTextFromBuffer(trimTrailingWhitespace).text;
const auto internalResult{ _terminal->RetrieveSelectedTextFromBuffer(trimTrailingWhitespace).text };

auto result = winrt::single_threaded_vector<winrt::hstring>();

for (const auto& row : internalResult)
{
result.Append(winrt::hstring{ row });
}
return result;
}

::Microsoft::Console::Types::IUiaData* ControlCore::GetUiaData() const
Expand Down Expand Up @@ -1124,7 +1159,7 @@ namespace winrt::Microsoft::Terminal::Control::implementation
}
}

void ControlCore::SetBackgroundOpacity(const float opacity)
void ControlCore::SetBackgroundOpacity(const double opacity)
{
if (_renderEngine)
{
Expand Down Expand Up @@ -1176,15 +1211,15 @@ namespace winrt::Microsoft::Terminal::Control::implementation
}
}

HANDLE ControlCore::GetSwapChainHandle() const
uint64_t ControlCore::SwapChainHandle() const
{
// This is called by:
// * TermControl::RenderEngineSwapChainChanged, who is only registered
// after Core::Initialize() is called.
// * TermControl::_InitializeTerminal, after the call to Initialize, for
// _AttachDxgiSwapChainToXaml.
// In both cases, we'll have a _renderEngine by then.
return _renderEngine->GetSwapChainHandle();
return reinterpret_cast<uint64_t>(_renderEngine->GetSwapChainHandle());
}

void ControlCore::_rendererWarning(const HRESULT hr)
Expand Down
18 changes: 12 additions & 6 deletions src/cascadia/TerminalControl/ControlCore.h
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,19 @@ namespace winrt::Microsoft::Terminal::Control::implementation
void UpdateAppearance(const IControlAppearance& newAppearance);
void SizeChanged(const double width, const double height);
void ScaleChanged(const double scale);
HANDLE GetSwapChainHandle() const;
uint64_t SwapChainHandle() const;

void AdjustFontSize(int fontSizeDelta);
void ResetFontSize();
FontInfo GetFont() const;
til::size FontSizeInDips() const;

winrt::Windows::Foundation::Size FontSize() const noexcept;
winrt::hstring FontFaceName() const noexcept;
uint16_t FontWeight() const noexcept;

til::color BackgroundColor() const;
void SetBackgroundOpacity(const float opacity);
void SetBackgroundOpacity(const double opacity);

void SendInput(const winrt::hstring& wstr);
void PasteText(const winrt::hstring& hstr);
Expand All @@ -67,10 +71,11 @@ namespace winrt::Microsoft::Terminal::Control::implementation
void ResumeRendering();

void UpdatePatternLocations();
void UpdateHoveredCell(const std::optional<til::point>& terminalPosition);
void SetHoveredCell(Core::Point terminalPosition);
void ClearHoveredCell();
winrt::hstring GetHyperlink(const til::point position) const;
winrt::hstring GetHoveredUriText() const;
std::optional<til::point> GetHoveredCell() const;
winrt::hstring HoveredUriText() const;
Windows::Foundation::IReference<Core::Point> HoveredCell() const;

::Microsoft::Console::Types::IUiaData* GetUiaData() const;

Expand Down Expand Up @@ -119,7 +124,7 @@ namespace winrt::Microsoft::Terminal::Control::implementation

bool HasSelection() const;
bool CopyOnSelect() const;
std::vector<std::wstring> SelectedText(bool trimTrailingWhitespace) const;
Windows::Foundation::Collections::IVector<winrt::hstring> SelectedText(bool trimTrailingWhitespace) const;
void SetSelectionAnchor(til::point const& position);
void SetEndSelectionPoint(til::point const& position);

Expand Down Expand Up @@ -232,6 +237,7 @@ namespace winrt::Microsoft::Terminal::Control::implementation
void _raiseReadOnlyWarning();
void _updateAntiAliasingMode(::Microsoft::Console::Render::DxEngine* const dxEngine);
void _connectionOutputHandler(const hstring& hstr);
void _updateHoveredCell(const std::optional<til::point> terminalPosition);

friend class ControlUnitTests::ControlCoreTests;
friend class ControlUnitTests::ControlInteractivityTests;
Expand Down
88 changes: 88 additions & 0 deletions src/cascadia/TerminalControl/ControlCore.idl
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,97 @@ import "EventArgs.idl";
namespace Microsoft.Terminal.Control
{

// This is a mirror of
// ::Microsoft::Console::VirtualTerminal::TerminalInput::MouseButtonState,
// but projectable.
// !! LOAD BEARING !! If you make this a struct with Booleans (like they
// make the most sense as), then the app will crash trying to toss one of
// these across the process boundary. I haven't the damndest idea why.
[flags]
enum MouseButtonState
{
IsLeftButtonDown = 0x1,
IsMiddleButtonDown = 0x2,
IsRightButtonDown = 0x4
};

[default_interface] runtimeclass ControlCore : ICoreState
{
ControlCore(IControlSettings settings,
Microsoft.Terminal.TerminalConnection.ITerminalConnection connection);

Boolean Initialize(Double actualWidth,
Double actualHeight,
Double compositionScale);

void UpdateSettings(IControlSettings settings);
void UpdateAppearance(IControlAppearance appearance);

UInt64 SwapChainHandle { get; };

Windows.Foundation.Size FontSize { get; };
String FontFaceName { get; };
UInt16 FontWeight { get; };

Boolean TrySendKeyEvent(Int16 vkey,
Int16 scanCode,
Microsoft.Terminal.Core.ControlKeyStates modifiers,
Boolean keyDown);
Boolean SendCharEvent(Char ch,
Int16 scanCode,
Microsoft.Terminal.Core.ControlKeyStates modifiers);
void SendInput(String text);
void PasteText(String text);

void SetHoveredCell(Microsoft.Terminal.Core.Point terminalPosition);
void ClearHoveredCell();

void ResetFontSize();
void AdjustFontSize(Int32 fontSizeDelta);
void SizeChanged(Double width, Double height);
void ScaleChanged(Double scale);

void ToggleShaderEffects();
void ToggleReadOnlyMode();

Microsoft.Terminal.Core.Point CursorPosition { get; };
void ResumeRendering();
void BlinkAttributeTick();
void UpdatePatternLocations();
void Search(String text, Boolean goForward, Boolean caseSensitive);
void SetBackgroundOpacity(Double opacity);
Microsoft.Terminal.Core.Color BackgroundColor { get; };

Boolean HasSelection { get; };
IVector<String> SelectedText(Boolean trimTrailingWhitespace);

String HoveredUriText { get; };
Windows.Foundation.IReference<Microsoft.Terminal.Core.Point> HoveredCell { get; };

void Close();
void BlinkCursor();
Boolean IsInReadOnlyMode { get; };
Boolean CursorOn;
void EnablePainting();

event FontSizeChangedEventArgs FontSizeChanged;

event Windows.Foundation.TypedEventHandler<Object, CopyToClipboardEventArgs> CopyToClipboard;
event Windows.Foundation.TypedEventHandler<Object, TitleChangedEventArgs> TitleChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> WarningBell;
event Windows.Foundation.TypedEventHandler<Object, Object> TabColorChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> BackgroundColorChanged;
event Windows.Foundation.TypedEventHandler<Object, ScrollPositionChangedArgs> ScrollPositionChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> CursorPositionChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> TaskbarProgressChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> ConnectionStateChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> HoveredHyperlinkChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> RendererEnteredErrorState;
event Windows.Foundation.TypedEventHandler<Object, Object> SwapChainChanged;
event Windows.Foundation.TypedEventHandler<Object, RendererWarningArgs> RendererWarning;
event Windows.Foundation.TypedEventHandler<Object, NoticeEventArgs> RaiseNotice;
event Windows.Foundation.TypedEventHandler<Object, TransparencyChangedEventArgs> TransparencyChanged;
event Windows.Foundation.TypedEventHandler<Object, Object> ReceivedOutput;

};
}
Loading

0 comments on commit 7f3bc3c

Please sign in to comment.