diff --git a/CHANGELOG.md b/CHANGELOG.md index e476d88..eb2636e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,29 @@ -### v2.0.2 released 04/14/2019: +### v2.1.0 released 06/02/2020: + +#### Enhancements +* Upgraded dependency to Device SDK 1.19.1 +* Provided a reference implementation for rendering of captions +* Support for APL 1.3 +* Added support for visual metrics +* Moved APL Core integration out of SampleApp into a standalone library + +#### Bug fixes +* Fixed browser warnings caused by short-hand definition of "transition" property +* Fixed potentially invalid and corrupted data due to using multiple rapidjson allocators + +#### Known issues +* SampleApp screen cut off at the bottom on fullscreen when using emulateDisplayDimensions configuration +* Display of captions is not in sync with Alexa audio and is as expected +* Memory leak issue with resetDevice member function in GUIManager +* Notification cues are not playing +* Potential memory leak issue with executeCommands directive +* Skill session remains alive for a short amount of time after exit +* Control buttons on the Flash briefing page is disabled +* Buttons on Now Playing display card do not update their visual states when updated by voice +* Scrolling on General Knowledge display card does not work when the content does not fit into screen +* Progress bar on Now Playing display card does not reflect the actual audio offset when such information is not provided in RenderPlayerInfo directive + +### v2.0.2 released 04/14/2020: #### Bug fixes * Added support for Raspbian Buster. diff --git a/CMakeLists.txt b/CMakeLists.txt index 3a0b22f..4d0de86 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,6 +1,6 @@ cmake_minimum_required(VERSION 3.1 FATAL_ERROR) -project(AlexaSmartScreenSDK VERSION 2.0.2 LANGUAGES CXX) +project(AlexaSmartScreenSDK VERSION 2.1 LANGUAGES CXX) set(PROJECT_BRIEF "A cross-platform, modular SDK for multi modal interaction with the Alexa Voice Service") configure_file ( "${PROJECT_SOURCE_DIR}/modules/Alexa/Utils/include/Utils/SmartScreenSDKVersion.h.in" diff --git a/NOTICE.txt b/NOTICE.txt index 9f8fa9d..eb28523 100644 --- a/NOTICE.txt +++ b/NOTICE.txt @@ -1,5 +1,5 @@ Alexa Smart Screen SDK -Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. +Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. ********************************* Alexa Smart Screen SDK COMPONENTS diff --git a/README.md b/README.md index 28341c1..f083e8e 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,6 @@ The Alexa Smart Screen SDK depends on the following additional GitHub repos: You can set up the Alexa Smart Screen SDK by using the following Quick Start Guides: * [MacOS Quick Start Guide](https://github.com/alexa/alexa-smart-screen-sdk/wiki/MacOS-Quick-Start-Guide) -* [Ubuntu Quick Start Guide](https://github.com/alexa/alexa-smart-screen-sdk/wiki/Ubuntu-Quick-Start-Guide) * [Raspberry Pi Quick Start Guide](https://github.com/alexa/alexa-smart-screen-sdk/wiki/Raspberry-Pi-Quick-Start-Guide) (Raspbian Stretch) You can also create your device prototype by using an [Amazon-qualified development kit](https://developer.amazon.com/en-US/alexa/alexa-voice-service/dev-kits) that supports the Smart Screen SDK, such as: @@ -64,20 +63,28 @@ All Alexa products should adopt the [Security Best Practices for Alexa](https:// ## Optional Configurations +### Add voice chrome + +The default implementation provides information on [Alexa state](https://github.com/alexa/alexa-smart-screen-sdk/blob/master/modules/GUI/SDK-GUI-API.md#alexastatechanged), which you can use to create voice chrome. Be sure to follow the [AVS Voice Chrome guidelines](https://developer.amazon.com/docs/alexa-voice-service/ux-design-attention.html#chrome). + ### Run the GUI client with predefined device visual characteristics and GUI client configurations -We provide 4 different sample configuration files of predefined device visual characteristics and GUI client configurations. This can be found under `modules/GUI/config/guiConfigSamples`. To enable a predefined configuration, -You can pass any of them as an extra config file argument after the main Smart Screen SDK config file argument when running the Sample App. +We provide four different sample configuration files containing predefined device visual characteristics and GUI client configurations. These can be found under `modules/GUI/config/guiConfigSamples`. +You can pass any of them as an extra config file argument after the main Smart Screen SDK config file argument when running the Sample App, for example: +``` + cd /ss-build + ./modules/Alexa/SampleApp/src/SampleApp + -C /sdk-build/Integration/AlexaClientSDKConfig.json + -C /alexa-smart-screen-sdk/modules/GUI/config/SmartScreenSDKConfig.json + -C /alexa-smart-screen-sdk/modules/GUI/config/guiConfigSamples/GuiConfigSample_TvOverlayPortrait.json + -L INFO +``` ### Remote control support -Exit and back remote control functionality is minimally supported by the Smart Screen SDK. The following behaviors are expected to occur: +Functionality for Exit and Back buttons (as found on a device's physical remote control) is minimally supported by the Smart Screen SDK. The following behaviors are expected to occur on execution of either a `BACK` or `EXIT` [navigationEvent](./modules/GUI/SDK-GUI-API#navigationevent): * Clear the rendering screen - Exit fully out of the Alexa-presented display so that no static image or layout is left. * Release the focus management - Release any focus management that might be held. -Using the predefined device visual characteristics, Esc and B are mapped to Exit and Back respectively. - -## Release Notes and Known Issues - -For a list of enhancements, bug fixes, and known issues, refer to [CHANGELOG.md](https://github.com/alexa/alexa-smart-screen-sdk/blob/master/CHANGELOG.md). +Using the default gui client configuration's [device keys](./modules/GUI/config/SmartScreenSDKConfig#device-keys-parameters), `Esc` and `B` are mapped to `EXIT` and `BACK` respectively. diff --git a/issue_template.md b/issue_template.md index da0db3b..0bb580e 100644 --- a/issue_template.md +++ b/issue_template.md @@ -31,3 +31,7 @@ - [ ] Raspbian Stretch - [ ] Raspbian Jessy - [ ] Other - tell us more: + +### Have you tried the same use case with AVS Device SDK SampleApp? +- [ ] The issue is not applicable to the AVS Device SDK SampleApp. +- [ ] The issue is not reproducible on the AVS Device SDK SampleApp. \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/CMakeLists.txt b/modules/Alexa/ApplicationUtilities/APLClient/CMakeLists.txt new file mode 100644 index 0000000..3326687 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/CMakeLists.txt @@ -0,0 +1,6 @@ +cmake_minimum_required(VERSION 3.1 FATAL_ERROR) +project(APLClient LANGUAGES CXX) + +include(../../build/BuildDefaults.cmake) + +add_subdirectory("src") diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplClientBinding.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplClientBinding.h new file mode 100644 index 0000000..734d493 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplClientBinding.h @@ -0,0 +1,115 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCLIENTBINDING_H_ +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCLIENTBINDING_H_ + +#include +#include +#include +#include "AplCoreConnectionManager.h" +#include "AplCoreGuiRenderer.h" +#include "AplOptionsInterface.h" + +namespace APLClient { + +/** + * AplClientBinding abstracts away many of the implementation details of integrating with the APLCoreEngine and exposes + * a smaller interface to allow rendering of APL documents on a remote view host through a client provided IPC layer. + */ +class AplClientBinding { +public: + /** + * Constructor + */ + AplClientBinding(AplOptionsInterfacePtr options); + + virtual ~AplClientBinding() = default; + + /** + * Pass a message received from the viewhost to the @c AplClientBinding, this should be called before + * @c handleMessage and on a different thread to @c renderDocument. + * @note This is a workaround to allow support for devices which do not support synchronous sends + * + * @param message + * @return true if the message should be passed onwards to handleMessage, false if handling is complete + */ + bool shouldHandleMessage(const std::string& message); + + /** + * Pass a message received from the viewhost to the @c AplClientBinding, should only be called if + * @c shouldHandleMessage returns true and must be run on the same thread as @c renderDocument + * @param message The message from the viewhost + */ + void handleMessage(const std::string& message); + + /** + * Render an APL document + * @param document The document json payload + * @param data The document data + * @param viewports The supported viewports + * @param token The APL document token + */ + void renderDocument( + const std::string& document, + const std::string& data, + const std::string& viewports, + const std::string& token); + + /** + * Clears the current APL document + */ + void clearDocument(); + + /** + * Execute an APL command sequence + * @param jsonPayload The JSON APL command payload + * @param token The APL document token + */ + void executeCommands(const std::string& jsonPayload, const std::string& token); + + /** + * Interrupts the currently executing command sequence + */ + void interruptCommandSequence(); + + /** + * Requests the visual context + */ + void requestVisualContext(unsigned int stateRequestToken); + + /** + * Updates the data source + * @param sourceType The source type + * @param jsonPayload The json payload containing the new data + * @param token The APL token + */ + void dataSourceUpdate(const std::string& sourceType, const std::string& jsonPayload, const std::string& token); + + /** + * Updates the rendered document + * @note Ideally this function should should be called once for each screen refresh (e.g. 60 times per second) + */ + void onUpdateTick(); + +private: + AplOptionsInterfacePtr m_aplOptions; + + AplCoreConnectionManagerPtr m_aplConnectionManager; + + std::unique_ptr m_aplGuiRenderer; +}; +} // namespace APLClient +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCLIENTBINDING_H_ diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreConnectionManager.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreConnectionManager.h new file mode 100644 index 0000000..275dbbe --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreConnectionManager.h @@ -0,0 +1,380 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCORECONNECTIONMANAGER_H +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCORECONNECTIONMANAGER_H + +#include + +#include +#include +// TODO: Tidy up core to prevent this (ARC-917) +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wreorder" +#pragma push_macro("DEBUG") +#pragma push_macro("TRUE") +#pragma push_macro("FALSE") +#undef DEBUG +#undef TRUE +#undef FALSE +#include +#pragma pop_macro("DEBUG") +#pragma pop_macro("TRUE") +#pragma pop_macro("FALSE") +#pragma GCC diagnostic pop + +#include "AplCoreViewhostMessage.h" +#include "AplCoreMetrics.h" +#include "AplOptionsInterface.h" + +namespace APLClient { + +/** + * Interacts with the APL Core Engine handling the event loop, updates etc. and passes messages between the core + * and the viewhost. + */ +class AplCoreConnectionManager : public std::enable_shared_from_this { +public: + /** + * Constructor + * @param guiClientInterface Pointer to the GUI Client interface + */ + AplCoreConnectionManager(const AplOptionsInterfacePtr aplOptions); + + virtual ~AplCoreConnectionManager() = default; + +public: + /** + * Sets the APL Content to be rendered by the APL Core + * @param content + * @param token APL Presentation token for this content + */ + void setContent(const apl::ContentPtr content, const std::string& token); + + /** + * Sets the APL ScalingOptions + * @param supportedViewports The JSON Payload + */ + void setSupportedViewports(const std::string& jsonPayload); + + /** + * Receives messages from the APL view host and identifies if it will require further handling + * @note This function does not need to be handled on the same execution thread as other function calls + * @param message The JSON Payload + * @return true if the message should be passed to @c handleMessage, false if message + */ + bool shouldHandleMessage(const std::string& message); + + /** + * Receives messages from the APL view host + * @param message The JSON Payload + */ + void handleMessage(const std::string& message); + + /** + * Executes an APL command + * @param command The command to execute + * @param token Directive token to bind result processing + */ + void executeCommands(const std::string& command, const std::string& token); + + /** + * Execute DataSource updates. + * @param sourceType DataSource type. + * @param jsonPayload The payload of the directive in structured JSON format. + * @param token Directive token used to bind result processing. + */ + void dataSourceUpdate(const std::string& sourceType, const std::string& jsonPayload, const std::string& token); + + /** + * Interrupts the currently executing APL command sequence + */ + void interruptCommandSequence(); + + /** + * Send a message to the view host and block until you get a reply + * @param message The message to send + * @return The resultant message or a NULL object if a response was not received. + */ + rapidjson::Document blockingSend( + AplCoreViewhostMessage& message, + const std::chrono::milliseconds& timeout = std::chrono::milliseconds(2000)); + + void provideState(unsigned int stateRequestToken); + + AplCoreMetricsPtr aplCoreMetrics() const { + return m_AplCoreMetrics; + } + + /** + * Schedules an update on the root context and runs the update loop - this may result in the viewhost being + * updated and any events currently pending will be processed. If nothing is currently being displayed calling + * this method will result in no action being taken. For the best rendering experience it is recommended that this + * method is scheduled to be called at a rate which matches the refresh rate of the users display. + */ + void onUpdateTick(); + + /** + * Resets the connection manager to remove the current document + */ + void reset(); + +private: + /** + * Sends document theme information to the client + */ + void sendDocumentThemeMessage(); + + /** + * Sends document background information to the client + * @param background + */ + void sendDocumentBackgroundMessage(const apl::Object& background); + + /** + * Sends screenLock state to the client + * @param screenLock + */ + void sendScreenLockMessage(bool screenLock); + + /** + * Handles the build message received from the view host, builds the component hierarchy. + * @param message + */ + void handleBuild(const rapidjson::Value& message); + + /** + * Handle an update message from the view host of the form: + * + * { "id": COMPONENT_ID, "type": EventType(int), "value": Integer } + * + * @param update + */ + void handleUpdate(const rapidjson::Value& update); + + /** + * Handle an media update message from the view host of the form: + * + * { "id": COMPONENT_ID, "mediaState": apl::MediaState, "fromEvent": boolean } + * + * @param update + */ + void handleMediaUpdate(const rapidjson::Value& update); + + /** + * Handle an graphic update message from the view host of the form: + * + * { "id": COMPONENT_ID, "avg": json } + * + * @param update + */ + void handleGraphicUpdate(const rapidjson::Value& update); + + /** + * Handles the ensureLayout message received from the viewhost + * @param payload + */ + void handleEnsureLayout(const rapidjson::Value& payload); + + /** + * Handle the scrollToRectInComponent message received from the viewhost + * @param payload + */ + void handleScrollToRectInComponent(const rapidjson::Value& payload); + + /** + * Handle the handleKeyboard message received from the viewhost + * @param payload + */ + void handleHandleKeyboard(const rapidjson::Value& payload); + + /** + * Handle the updateCursorPosition message received from the viewhost + * @param payload + */ + void handleUpdateCursorPosition(const rapidjson::Value& payload); + + /** + * Process responses to events with action references. The payload should be of the form: + * + * { "event": EVENT_NUMBER, "argument": VALUE } + * + * @param response + */ + void handleEventResponse(const rapidjson::Value& response); + + /** + * Check for screenLock condition and process it accordingly. + */ + void handleScreenLock(); + + /** + * Execute the event. + * ActionRefs have to be stored while we are waiting for a response. + * Terminates have to be sent up if the action is cancelled. + * Resolves/Rejects sent down have to be acted up. + * @param event requested core event. + */ + void processEvent(const apl::Event& event); + + /** + * Process set of dirty components and send out dirty properties as required. + * @param dirty dirty components set. + */ + void processDirty(const std::set& dirty); + + /** + * APL Core relies on operations to be performed in particular way. + * Order and set of operations in this method should be preserved. + * Order is the following: + * * Update time and adjust TimeZone if required. + * * Call **clearPending** method on RootConfig to give Core possibility to execute all pending actions and updates. + * * Process requested events. * * Process dirty properties. + * * Check and set screenlock if required. + */ + void coreFrameUpdate(); + + /** + * Send a message to the view host + * @param message The message to send + * @return The sequence number of this message + */ + unsigned int send(AplCoreViewhostMessage& message); + + /** + * Sends an error message to the view host + * @param message The message to send to the view hsot + */ + void sendError(const std::string& message); + + /** + * Retrieves the current time + * @return The time + */ + std::chrono::milliseconds getCurrentTime() { + return std::chrono::duration_cast( + std::chrono::system_clock::now().time_since_epoch()); + } + + /** + * Get optional value from Json. + * @param jsonNode json data + * @param key Member name + * @param defaultValue Default value + * @return Value from json or default if not found. + */ + double getOptionalValue(const rapidjson::Value& jsonNode, const std::string& key, double defaultValue); + + /** + * Get optional value from Json. + * @param jsonNode json data + * @param key Member name + * @param defaultValue Default value + * @return Value from json or default if not found. + */ + std::string getOptionalValue( + const rapidjson::Value& jsonNode, + const std::string& key, + const std::string& defaultValue); + + /** + * Get optional bool value from Json. + * @param jsonNode json data + * @param key Member name + * @param defaultValue Default value + * @return Value from json or default if not found. + */ + bool getOptionalBool(const rapidjson::Value& jsonNode, const std::string& key, bool defaultValue); + + /** + * Get optional integer value from Json. + * @param jsonNode json data + * @param key Member name + * @param defaultValue Default value + * @return Value from json or default if not found. + */ + int getOptionalInt(const rapidjson::Value& jsonNode, const std::string& key, int defaultValue); + + /** + * Gets a rect from Json and converts it to an apl::Rect + * @param jsonNode json data + * @return An apl rect with the values from the json + */ + apl::Rect convertJsonToScaledRect(const rapidjson::Value& jsonNode); + + /** + * Check if any errors returned from any of loaded datasources and report them. + */ + void checkAndSendDataSourceErrors(); + + /// View host message type to handler map + std::map> m_messageHandlers; + + /// Shared pointer to the APL Content + apl::ContentPtr m_Content; + + /// The APL presentation token for the currently rendered document + std::string m_aplToken; + + /// The APL Metrics object - received from the view host and used for generating the apl root context + apl::Metrics m_Metrics; + + /** + * The Viewport Size Specifications object - created on directive processing and passed to core in + * order to calculate scaling. + */ + std::vector m_ViewportSizeSpecifications; + + /** + * Scaling calculation object. + */ + AplCoreMetricsPtr m_AplCoreMetrics; + + /// Pointer to the APL Root Context + apl::RootContextPtr m_Root; + + /// Map of pending APL Core events + std::map m_PendingEvents; + + /// The start time used to calculate the update time used by APL Core + std::chrono::milliseconds m_StartTime; + + /// Pointer to APL Options + AplOptionsInterfacePtr m_aplOptions; + + /// Screen lock flag + bool m_ScreenLock; + + /// Next packet sequence number + unsigned int m_SequenceNumber; + + /// The sequence number which a blockingSend is waiting for + unsigned int m_replyExpectedSequenceNumber; + + /// Whether we are expecting a reply to a blockingSend + bool m_blockingSendReplyExpected; + + /// The pending promise from a call to blockingSend + std::promise m_replyPromise; + + /// The mutex protecting blockingSend + std::mutex m_blockingSendMutex; +}; + +using AplCoreConnectionManagerPtr = std::shared_ptr; + +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCORECONNECTIONMANAGER_H diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreEngineLogBridge.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreEngineLogBridge.h new file mode 100644 index 0000000..2d0765a --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreEngineLogBridge.h @@ -0,0 +1,44 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREENGINELOGBRIDGE_H +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREENGINELOGBRIDGE_H + +#include + +// TODO: Tidy up core to prevent this (ARC-917) +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wreorder" +#pragma push_macro("DEBUG") +#undef DEBUG +#include +#pragma pop_macro("DEBUG") +#pragma GCC diagnostic pop +#include "AplOptionsInterface.h" + +namespace APLClient { +/// Wrapper around the Alexa Client SDK logger for use by APLCoreEngine +class AplCoreEngineLogBridge : public apl::LogBridge { +public: + AplCoreEngineLogBridge(AplOptionsInterfacePtr aplOptions); + + void transport(apl::LogLevel level, const std::string& log) override; + +private: + AplOptionsInterfacePtr m_aplOptions; +}; +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREENGINELOGBRIDGE_H \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreGuiRenderer.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreGuiRenderer.h new file mode 100644 index 0000000..ae112c9 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreGuiRenderer.h @@ -0,0 +1,110 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREGUIRENDERER_H_ +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREGUIRENDERER_H_ + +// TODO: Tidy up core to prevent this (ARC-917) +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wreorder" +#pragma push_macro("DEBUG") +#pragma push_macro("TRUE") +#pragma push_macro("FALSE") +#undef DEBUG +#undef TRUE +#undef FALSE +#include +#pragma pop_macro("DEBUG") +#pragma pop_macro("TRUE") +#pragma pop_macro("FALSE") +#pragma GCC diagnostic pop + +#include "AplCoreConnectionManager.h" +#include "AplOptionsInterface.h" + +namespace APLClient { + +/** + * Handles the initial creation of the APL content and retrieves package dependencies, also handles interaction with + * the @c AplCoreConnectionManager + */ +class AplCoreGuiRenderer { +public: + /** + * Constructor. + * + * @param aplOptions The AplOptionsInterface object + * @param aplCoreConnectionManager Pointer to the APL Core connection manager + */ + AplCoreGuiRenderer(AplOptionsInterfacePtr aplOptions, AplCoreConnectionManagerPtr aplCoreConnectionManager); + + /** + * Renders the given template document and data payload through Apl Core + * @param document Template + * @param data Payload + * @param supportedViewports SupportedViewports + * @param token The token for APL payload, empty string otherwise + */ + void renderDocument( + const std::string& document, + const std::string& data, + const std::string& supportedViewports, + const std::string& token); + + /** + * Clears the currently rendered document + */ + void clearDocument(); + + /** + * Executes the given sequence of APL commands + * @param jsonPayload The APL commands to execute + * @param token The APL token + */ + void executeCommands(const std::string& jsonPayload, const std::string& token); + + /** + * For lazy loading - updates the data source which is used by the currently rendered document + * @param sourceType The data source type + * @param jsonPayload The new data source payload + * @param token The APL token + */ + void dataSourceUpdate(const std::string& sourceType, const std::string& jsonPayload, const std::string& token); + + /** + * Interrupts the currently executing command sequence + */ + void interruptCommandSequence(); + +private: + /** + * A flag indicating if the document has been cleared. + * Used to cover the gap in time between request to render and any incoming clear events. + */ + bool m_isDocumentCleared; + + /** + * A reference to the AplOptionsInterface object + */ + AplOptionsInterfacePtr m_aplOptions; + + /** + * A reference to the APL Core connection manager to forward APL messages to + */ + AplCoreConnectionManagerPtr m_aplCoreConnectionManager; +}; +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREGUIRENDERER_H_ diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreMetrics.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreMetrics.h new file mode 100644 index 0000000..df0dd95 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreMetrics.h @@ -0,0 +1,66 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREMETRICS_H +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREMETRICS_H + +#include + +namespace APLClient { + +/** + * Extends apl MetricsTransform to transform units between viewhost and core dimensions + */ +class AplCoreMetrics : public apl::MetricsTransform { +public: + explicit AplCoreMetrics(apl::Metrics& metrics) : MetricsTransform(metrics) { + } + AplCoreMetrics(apl::Metrics& metrics, apl::ScalingOptions& options) : MetricsTransform(metrics, options) { + } + + virtual ~AplCoreMetrics() = default; + + /** + * Converts dp units into px units + * @param value dp unit + * @return px unit + */ + float toViewhost(float value) const override; + + /** + * Converts px units into dp units + * @param value px unit + * @return dp unit + */ + float toCore(float value) const override; + + /** + * Return the viewport width in pixels + * @return pixel width + */ + float getViewhostWidth() const override; + + /** + * Return the viewport height in pixels + * @return pixel height + */ + float getViewhostHeight() const override; +}; + +using AplCoreMetricsPtr = std::shared_ptr; + +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREMETRICS_H \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreTextMeasurement.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreTextMeasurement.h new file mode 100644 index 0000000..f1e9b09 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreTextMeasurement.h @@ -0,0 +1,76 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCORETEXTMEASUREMENT_H +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCORETEXTMEASUREMENT_H + +// TODO: Tidy up core to prevent this (ARC-917) +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wreorder" +#pragma push_macro("DEBUG") +#pragma push_macro("TRUE") +#pragma push_macro("FALSE") +#undef DEBUG +#undef TRUE +#undef FALSE +#include +#pragma pop_macro("DEBUG") +#pragma pop_macro("TRUE") +#pragma pop_macro("FALSE") +#pragma GCC diagnostic pop + +#include "AplCoreConnectionManager.h" +#include "AplOptionsInterface.h" + +namespace APLClient { + +/** + * Provides the ability to retrieve text measurements from a remote viewhost + */ +class AplCoreTextMeasurement : public apl::TextMeasurement { +public: + /** + * Constructor + * + * @param aplCoreConnectionManager Pointer to the APL Core connection manager + */ + AplCoreTextMeasurement( + const AplCoreConnectionManagerPtr aplCoreConnectionManager, + const AplOptionsInterfacePtr aplOptions) : + m_aplCoreConnectionManager(aplCoreConnectionManager), + m_aplOptions(aplOptions) { + } + + /// @name apl::TextMeasurement Functions + /// @{ + virtual YGSize measure( + apl::TextComponent* component, + float width, + YGMeasureMode widthMode, + float height, + YGMeasureMode heightMode) override; + + virtual float baseline(apl::TextComponent* component, float width, float height) override; + /// @} + +private: + std::weak_ptr m_aplCoreConnectionManager; + + AplOptionsInterfacePtr m_aplOptions; +}; + +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCORETEXTMEASUREMENT_H diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreViewhostMessage.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreViewhostMessage.h new file mode 100644 index 0000000..6a9e73c --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplCoreViewhostMessage.h @@ -0,0 +1,115 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREVIEWHOSTMESSAGE_H +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREVIEWHOSTMESSAGE_H + +#include +#include +#include +#include + +namespace APLClient { + +/// The root GUI message type +const char MSG_TYPE_TAG[] = "type"; + +/// The seqno json key in the message. +const char MSG_SEQNO_TAG[] = "seqno"; + +/// The payload json key in the message. +const char MSG_PAYLOAD_TAG[] = "payload"; + +/** + * The @c AplCoreViewhostMessage base class for messages sent to AplViewHost. + * + * { "type": STRING, "seqno": NUMBER, "payload": ANY } + */ +class AplCoreViewhostMessage { +private: + rapidjson::Document mDocument; + +public: + /** + * Constructor + * @param type The type from this message + */ + AplCoreViewhostMessage(const std::string& type) : mDocument(rapidjson::kObjectType) { + auto& alloc = mDocument.GetAllocator(); + mDocument.AddMember(MSG_TYPE_TAG, rapidjson::Value(type.c_str(), alloc).Move(), alloc); + } + + /** + * Sets the sequence number for this message + * @param sequenceNumber + * @return this + */ + AplCoreViewhostMessage& setSequenceNumber(unsigned sequenceNumber) { + mDocument.AddMember(MSG_SEQNO_TAG, sequenceNumber, mDocument.GetAllocator()); + return *this; + } + + /** + * Sets the json payload for this message + * @param payload The payload to send + * @return this + */ + AplCoreViewhostMessage& setPayload(rapidjson::Value&& payload) { + mDocument.AddMember(MSG_PAYLOAD_TAG, std::move(payload), mDocument.GetAllocator()); + return *this; + } + + /** + * Sets the json payload for this message + * @param payload The payload to send + * @return this + */ + AplCoreViewhostMessage& setPayload(const std::string& payload) { + mDocument.AddMember( + MSG_PAYLOAD_TAG, + rapidjson::Value(payload.c_str(), mDocument.GetAllocator()).Move(), + mDocument.GetAllocator()); + return *this; + } + + /** + * Retrieves the json string representing this message + * @return json string representation of message + */ + std::string get() { + rapidjson::StringBuffer buffer; + buffer.Clear(); + rapidjson::Writer< + rapidjson::StringBuffer, + rapidjson::UTF8<>, + rapidjson::UTF8<>, + rapidjson::CrtAllocator, + rapidjson::kWriteNanAndInfFlag> + writer(buffer); + mDocument.Accept(writer); + return std::string(buffer.GetString(), buffer.GetSize()); + } + + /** + * Retrieves the rapidjson allocator + * @return The allocator + */ + auto alloc() -> decltype(mDocument.GetAllocator()) { + return mDocument.GetAllocator(); + }; +}; +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLCOREVIEWHOSTMESSAGE_H diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplOptionsInterface.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplOptionsInterface.h new file mode 100644 index 0000000..3f9760f --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplOptionsInterface.h @@ -0,0 +1,153 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLOPTIONSINTERFACE_H_ +#define ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLOPTIONSINTERFACE_H_ + +#include +#include "AplRenderingEvent.h" + +namespace APLClient { +/// Enumeration of log levels sent by the APL client binding (DBG used to avoid conflicts with compiler defined macros) +enum class LogLevel { CRITICAL, ERROR, WARN, INFO, DBG, TRACE }; + +/** + * The @c AplOptionsInterface defines the set of callbacks which users of the APL client library must provide, it will + * be used to inform the consumer of certain state changes as well as requests for data or to pass messages to the + * APL Viewhost. + * @note This class is named "Options" to match the implementation of other APLCore integration methods, it does not + * imply that this is only used for settings, or that it is an optional interface. + */ +class AplOptionsInterface { +public: + /** + * Virtual destructor + */ + virtual ~AplOptionsInterface() = default; + + /** + * Send the given payload to the APL Viewhost + * @param payload + */ + virtual void sendMessage(const std::string& payload) = 0; + + /** + * Requests that the APL viewhost is reset to render a new APL document + * @param token The APL token + */ + virtual void resetViewhost(const std::string& token) = 0; + + /** + * Download the given resource + * @param source The URI for the resource + * @return The content of the resource + */ + virtual std::string downloadResource(const std::string& source) = 0; + + /** + * Retrieve the current timezone offset + * @return The offset in milliseconds + */ + virtual std::chrono::milliseconds getTimezoneOffset() = 0; + + /** + * The given activity has started + * @param source The activity type + */ + virtual void onActivityStarted(const std::string& source) = 0; + + /** + * The given activity has ended + * @param source The activity type + */ + virtual void onActivityEnded(const std::string& source) = 0; + + /** + * An APL send event command was executed + * @param event The event + */ + virtual void onSendEvent(const std::string& event) = 0; + + /** + * Command execution has completed + * @param token The APL token + * @param result Whether the command executed to completion successfully + */ + virtual void onCommandExecutionComplete(const std::string& token, bool result) = 0; + + /** + * Rendering the APL document has completed + * @param token The APL token + * @param result The result of the rendering + * @param error The error message if a failure occurred + */ + virtual void onRenderDocumentComplete(const std::string& token, bool result, const std::string& error) = 0; + + /** + * Called as a response to a @c requestVisualContext request + * @param stateRequestToken The token which was passed during the call to @c requestVisualContext + * @param context The visual context + */ + virtual void onVisualContextAvailable(unsigned int stateRequestToken, const std::string& context) = 0; + + /** + * Called when the document idle timeout is set + * @param timeout The timeout value + */ + virtual void onSetDocumentIdleTimeout(const std::chrono::milliseconds& timeout) = 0; + + /** + * Called when an event occurs during APL rendering, generally used for metrics + * @param event The event + */ + virtual void onRenderingEvent(AplRenderingEvent event) = 0; + + /** + * A finish event occurred, the APL document should be removed + */ + virtual void onFinish() = 0; + + /** + * A data source fetch request for lazy loading + * @param type The type of the fetch request + * @param payload The payload for the fetch request + */ + virtual void onDataSourceFetchRequestEvent(const std::string& type, const std::string& payload) = 0; + + /** + * Handles a RuntimeError event + * @param payload The payload for the error event + */ + virtual void onRuntimeErrorEvent(const std::string& payload) = 0; + + /** + * Called when a message should be logged + * @param level The log level + * @param source The source of the message + * @param message + */ + virtual void logMessage(LogLevel level, const std::string& source, const std::string& message) = 0; + + /** + * Returns the maximum number of concurrent downloads from the configs. + */ + virtual int getMaxNumberOfConcurrentDownloads() = 0; +}; + +/// Convenience typedef +using AplOptionsInterfacePtr = std::shared_ptr; + +} // namespace APLClient +#endif // ALEXA_SMART_SCREEN_SDK_APPLICATIONUTILITIES_APL_APLOPTIONSINTERFACE_H_ diff --git a/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplRenderingEvent.h b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplRenderingEvent.h new file mode 100644 index 0000000..b3f5ddf --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/include/APLClient/AplRenderingEvent.h @@ -0,0 +1,35 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_SMARTSCREENSDKINTERFACES_INCLUDE_SMARTSCREENSDKINTERFACES_APLEVENT_H_ +#define ALEXA_SMART_SCREEN_SDK_SMARTSCREENSDKINTERFACES_INCLUDE_SMARTSCREENSDKINTERFACES_APLEVENT_H_ + +namespace APLClient { + +/// Enumeration of APL events that could be sent from GUI to @c AlexaPresentation. +enum class AplRenderingEvent { + /// APL Core Engine started document inflation + INFLATE_BEGIN, + + /// APL Core Engine started document inflation + INFLATE_END, + + /// Text measure event was called + TEXT_MEASURE +}; + +} // namespace APLClient + +#endif // ALEXA_SMART_SCREEN_SDK_SMARTSCREENSDKINTERFACES_INCLUDE_SMARTSCREENSDKINTERFACES_APLEVENT_H_ diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/AplClientBinding.cpp b/modules/Alexa/ApplicationUtilities/APLClient/src/AplClientBinding.cpp new file mode 100644 index 0000000..c5d4a5c --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/AplClientBinding.cpp @@ -0,0 +1,69 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#include "APLClient/AplCoreEngineLogBridge.h" +#include "APLClient/AplClientBinding.h" + +namespace APLClient { + +AplClientBinding::AplClientBinding(AplOptionsInterfacePtr options) : m_aplOptions{options} { + m_aplConnectionManager = std::make_shared(options); + m_aplGuiRenderer.reset(new AplCoreGuiRenderer(options, m_aplConnectionManager)); + apl::LoggerFactory::instance().initialize(std::make_shared(options)); +} + +bool AplClientBinding::shouldHandleMessage(const std::string& message) { + return m_aplConnectionManager->shouldHandleMessage(message); +} + +void AplClientBinding::handleMessage(const std::string& message) { + m_aplConnectionManager->handleMessage(message); +} + +void AplClientBinding::renderDocument( + const std::string& document, + const std::string& data, + const std::string& viewports, + const std::string& token) { + m_aplGuiRenderer->renderDocument(document, data, viewports, token); +} + +void AplClientBinding::clearDocument() { + m_aplGuiRenderer->clearDocument(); +} + +void AplClientBinding::executeCommands(const std::string& jsonPayload, const std::string& token) { + m_aplConnectionManager->executeCommands(jsonPayload, token); +} + +void AplClientBinding::interruptCommandSequence() { + m_aplGuiRenderer->interruptCommandSequence(); +} + +void AplClientBinding::requestVisualContext(unsigned int stateRequestToken) { + m_aplConnectionManager->provideState(stateRequestToken); +} + +void AplClientBinding::dataSourceUpdate( + const std::string& sourceType, + const std::string& jsonPayload, + const std::string& token) { + m_aplConnectionManager->dataSourceUpdate(sourceType, jsonPayload, token); +} + +void AplClientBinding::onUpdateTick() { + m_aplConnectionManager->onUpdateTick(); +} +} // namespace APLClient \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreConnectionManager.cpp b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreConnectionManager.cpp new file mode 100644 index 0000000..f33523f --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreConnectionManager.cpp @@ -0,0 +1,947 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#include "APLClient/AplCoreTextMeasurement.h" +#include "APLClient/AplCoreConnectionManager.h" +#include "APLClient/AplCoreViewhostMessage.h" + +#include + +namespace APLClient { + +/// The keys used in ProvideState. +static const char TOKEN_KEY[] = "token"; +static const char VERSION_KEY[] = "version"; +static const char CONTEXT_KEY[] = "componentsVisibleOnScreen"; +/// The value used in ProvideState. +// TODO: need to get version number from APLCoreEngine: ARC-858 +static const char VERSION_VALUE[] = "AplRenderer-1.2"; + +// Key used in messaging +static const char SEQNO_KEY[] = "seqno"; + +/// APL Scaling bias constant +static const float SCALING_BIAS_CONSTANT = 10.0f; +/// APL Scaling cost override +static const bool SCALING_SHAPE_OVERRIDES_COST = true; + +/// The keys used in APL context creation. +static const char HEIGHT_KEY[] = "height"; +static const char WIDTH_KEY[] = "width"; +static const char DPI_KEY[] = "dpi"; +static const char MODE_KEY[] = "mode"; +static const char SHAPE_KEY[] = "shape"; +static const char SCALING_KEY[] = "scaling"; +static const char SCALE_FACTOR_KEY[] = "scaleFactor"; +static const char VIEWPORT_WIDTH_KEY[] = "viewportWidth"; +static const char VIEWPORT_HEIGHT_KEY[] = "viewportHeight"; +static const char HIERARCHY_KEY[] = "hierarchy"; +static const char X_KEY[] = "x"; +static const char Y_KEY[] = "y"; +static const char DOCTHEME_KEY[] = "docTheme"; +static const char BACKGROUND_KEY[] = "background"; +static const char SCREENLOCK_KEY[] = "screenLock"; +static const char COLOR_KEY[] = "color"; +static const char GRADIENT_KEY[] = "gradient"; +static const char ENSURELAYOUT_KEY[] = "ensureLayout"; +static const char AGENTNAME_KEY[] = "agentName"; +static const char AGENTVERSION_KEY[] = "agentVersion"; +static const char ALLOWOPENURL_KEY[] = "allowOpenUrl"; +static const char DISALLOWVIDEO_KEY[] = "disallowVideo"; +static const char ANIMATIONQUALITY_KEY[] = "animationQuality"; + +/// The keys used in APL event execution. +static const char ERROR_KEY[] = "error"; +static const char EVENT_KEY[] = "event"; +static const char EVENT_TERMINATE_KEY[] = "eventTerminate"; +static const char DIRTY_KEY[] = "dirty"; + +/// SendEvent keys +static const char PRESENTATION_TOKEN_KEY[] = "presentationToken"; +static const char SOURCE_KEY[] = "source"; +static const char ARGUMENTS_KEY[] = "arguments"; +static const char COMPONENTS_KEY[] = "components"; + +/// RuntimeError keys +static const char ERRORS_KEY[] = "errors"; + +/// Media update keys +static const char MEDIA_STATE_KEY[] = "mediaState"; +static const char FROM_EVENT_KEY[] = "fromEvent"; +static const char TRACK_INDEX_KEY[] = "trackIndex"; +static const char TRACK_COUNT_KEY[] = "trackCount"; +static const char CURRENT_TIME_KEY[] = "currentTime"; +static const char DURATION_KEY[] = "duration"; +static const char PAUSED_KEY[] = "paused"; +static const char ENDED_KEY[] = "ended"; + +/// Activity tracking sources +static const std::string APL_COMMAND_EXECUTION{"APLCommandExecution"}; +static const std::string APL_SCREEN_LOCK{"APLScreenLock"}; +static const char RENDERING_OPTIONS_KEY[] = "renderingOptions"; + +static const char LEGACY_KARAOKE_KEY[] = "legacyKaraoke"; + +static apl::Bimap AVS_VIEWPORT_MODE_MAP = { + {"HUB", apl::ViewportMode::kViewportModeHub}, + {"TV", apl::ViewportMode::kViewportModeTV}, + {"MOBILE", apl::ViewportMode::kViewportModeMobile}, + {"AUTO", apl::ViewportMode::kViewportModeAuto}, + {"PC", apl::ViewportMode::kViewportModePC}, +}; + +static apl::Bimap AVS_VIEWPORT_SHAPE_MAP = { + {"ROUND", apl::ScreenShape::ROUND}, + {"RECTANGLE", apl::ScreenShape::RECTANGLE}, +}; + +AplCoreConnectionManager::AplCoreConnectionManager(const AplOptionsInterfacePtr aplOptions) : + m_aplOptions{aplOptions}, + m_ScreenLock{false}, + m_SequenceNumber{0}, + m_replyExpectedSequenceNumber{0}, + m_blockingSendReplyExpected{false} { + m_StartTime = getCurrentTime(); + m_messageHandlers.emplace("build", [this](const rapidjson::Value& payload) { handleBuild(payload); }); + m_messageHandlers.emplace("update", [this](const rapidjson::Value& payload) { handleUpdate(payload); }); + m_messageHandlers.emplace("updateMedia", [this](const rapidjson::Value& payload) { handleMediaUpdate(payload); }); + m_messageHandlers.emplace( + "updateGraphic", [this](const rapidjson::Value& payload) { handleGraphicUpdate(payload); }); + m_messageHandlers.emplace("response", [this](const rapidjson::Value& payload) { handleEventResponse(payload); }); + m_messageHandlers.emplace("ensureLayout", [this](const rapidjson::Value& payload) { handleEnsureLayout(payload); }); + m_messageHandlers.emplace( + "scrollToRectInComponent", [this](const rapidjson::Value& payload) { handleScrollToRectInComponent(payload); }); + m_messageHandlers.emplace( + "handleKeyboard", [this](const rapidjson::Value& payload) { handleHandleKeyboard(payload); }); + m_messageHandlers.emplace( + "updateCursorPosition", [this](const rapidjson::Value& payload) { handleUpdateCursorPosition(payload); }); +} + +void AplCoreConnectionManager::setContent(const apl::ContentPtr content, const std::string& token) { + m_Content = content; + m_aplToken = token; + m_aplOptions->resetViewhost(token); +} + +void AplCoreConnectionManager::setSupportedViewports(const std::string& jsonPayload) { + rapidjson::Document doc; + if (doc.Parse(jsonPayload.c_str()).HasParseError()) { + m_aplOptions->logMessage(LogLevel::ERROR, "setSupportedViewportsFailed", "Failed to parse json payload"); + return; + } + + if (doc.GetType() != rapidjson::Type::kArrayType) { + m_aplOptions->logMessage(LogLevel::ERROR, "setSupportedViewportsFailed", "Unexpected json document type"); + return; + } + + m_ViewportSizeSpecifications.clear(); + for (auto& spec : doc.GetArray()) { + double minWidth = getOptionalValue(spec, "minWidth", 1); + double maxWidth = getOptionalValue(spec, "maxWidth", INT_MAX); + double minHeight = getOptionalValue(spec, "minHeight", 1); + double maxHeight = getOptionalValue(spec, "maxHeight", INT_MAX); + std::string mode = getOptionalValue(spec, "mode", "HUB"); + std::string shape = spec.FindMember("shape")->value.GetString(); + + m_ViewportSizeSpecifications.emplace_back( + minWidth, + maxWidth, + minHeight, + maxHeight, + AVS_VIEWPORT_MODE_MAP.at(mode), + AVS_VIEWPORT_SHAPE_MAP.at(shape) == apl::ScreenShape::ROUND); + } +} + +double AplCoreConnectionManager::getOptionalValue( + const rapidjson::Value& jsonNode, + const std::string& key, + double defaultValue) { + double value = defaultValue; + const auto& valueIt = jsonNode.FindMember(key); + if (valueIt != jsonNode.MemberEnd()) { + value = valueIt->value.GetDouble(); + } + + return value; +} + +std::string AplCoreConnectionManager::getOptionalValue( + const rapidjson::Value& jsonNode, + const std::string& key, + const std::string& defaultValue) { + std::string value = defaultValue; + const auto& valueIt = jsonNode.FindMember(key); + if (valueIt != jsonNode.MemberEnd()) { + value = valueIt->value.GetString(); + } + + return value; +} + +bool AplCoreConnectionManager::getOptionalBool( + const rapidjson::Value& jsonNode, + const std::string& key, + bool defaultValue) { + bool value = defaultValue; + const auto& valueIt = jsonNode.FindMember(key); + if (valueIt != jsonNode.MemberEnd()) { + value = valueIt->value.GetBool(); + } + + return value; +} + +int AplCoreConnectionManager::getOptionalInt( + const rapidjson::Value& jsonNode, + const std::string& key, + int defaultValue) { + if (jsonNode.HasMember(key) && jsonNode[key].IsInt()) { + return jsonNode[key].GetInt(); + } + + return defaultValue; +} + +bool AplCoreConnectionManager::shouldHandleMessage(const std::string& message) { + if (m_blockingSendReplyExpected) { + rapidjson::Document doc; + if (doc.Parse(message.c_str()).HasParseError()) { + m_aplOptions->logMessage(LogLevel::ERROR, "shouldHandleMessageFailed", "Error whilst parsing message"); + return false; + } + + if (doc.HasMember(SEQNO_KEY) && doc[SEQNO_KEY].IsNumber()) { + unsigned int seqno = doc[SEQNO_KEY].GetUint(); + if (seqno == m_replyExpectedSequenceNumber) { + m_blockingSendReplyExpected = false; + m_replyPromise.set_value(message); + return false; + } + } + } + + return true; +} + +void AplCoreConnectionManager::handleMessage(const std::string& message) { + rapidjson::Document doc; + if (doc.Parse(message.c_str()).HasParseError()) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleMessageFailed", "Error whilst parsing message"); + return; + } + + if (!doc.HasMember("type")) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleMessageFailed", "Unable to find type in message"); + return; + } + std::string type = doc["type"].GetString(); + + auto payload = doc.FindMember("payload"); + if (payload == doc.MemberEnd()) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleMessageFailed", "Unable to find payload in message"); + return; + } + + auto fit = m_messageHandlers.find(type); + if (fit != m_messageHandlers.end()) { + fit->second(payload->value); + } else { + m_aplOptions->logMessage(LogLevel::ERROR, "handleMessageFailed", "Unrecognized message type: " + type); + } +} + +void AplCoreConnectionManager::executeCommands(const std::string& command, const std::string& token) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "executeCommandsFailed", "Root context is missing"); + return; + } + + rapidjson::Document* document = new rapidjson::Document(); + if (document->Parse(command).HasParseError()) { + m_aplOptions->logMessage(LogLevel::ERROR, "executeCommandsFailed", "Parse commands failed"); + return; + } + + auto it = document->FindMember("commands"); + if (it == document->MemberEnd() || it->value.GetType() != rapidjson::Type::kArrayType) { + m_aplOptions->logMessage(LogLevel::ERROR, "executeCommandsFailed", "Missing commands, or is not array"); + return; + } + + apl::Object object{it->value}; + auto action = m_Root->executeCommands(object, false); + if (!action) { + m_aplOptions->logMessage(LogLevel::ERROR, "executeCommandsFailed", "Execute commands failed"); + return; + } + + m_aplOptions->onActivityStarted(APL_COMMAND_EXECUTION); + + action->setUserData(document); + + action->then([this, document, token](const apl::ActionPtr& action) { + m_aplOptions->logMessage(LogLevel::DBG, "executeCommands", "Command sequence complete"); + if (action->getUserData()) { + delete document; + action->setUserData(nullptr); + } + m_aplOptions->onCommandExecutionComplete(token, true); + + m_aplOptions->onActivityEnded(APL_COMMAND_EXECUTION); + }); + action->addTerminateCallback([this, action, document, token](const apl::TimersPtr&) { + m_aplOptions->logMessage(LogLevel::DBG, "executeCommandsFailed", "Command sequence failed"); + if (action->getUserData()) { + delete document; + action->setUserData(nullptr); + } + m_aplOptions->onCommandExecutionComplete(token, false); + + m_aplOptions->onActivityEnded(APL_COMMAND_EXECUTION); + }); +} + +void AplCoreConnectionManager::dataSourceUpdate( + const std::string& sourceType, + const std::string& jsonPayload, + const std::string& token) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "dataSourceUpdateFailed", "Root context is missing"); + return; + } + + auto provider = m_Root->context().getRootConfig().getDataSourceProvider(sourceType); + if (!provider) { + m_aplOptions->logMessage(LogLevel::ERROR, "dataSourceUpdateFailed", "Unknown provider requested."); + return; + } + + bool result = provider->processUpdate(jsonPayload); + if (!result) { + m_aplOptions->logMessage(LogLevel::ERROR, "dataSourceUpdateFailed", "Update is not processed."); + checkAndSendDataSourceErrors(); + } +} + +void AplCoreConnectionManager::provideState(unsigned int stateRequestToken) { + if (!m_Content) { + m_aplOptions->logMessage(LogLevel::WARN, "provideStateFailed", "Root context is null"); + sendError("Root context is null"); + return; + } + + rapidjson::Document state(rapidjson::kObjectType); + rapidjson::Document::AllocatorType& allocator = state.GetAllocator(); + // Add presentation token info + state.AddMember(TOKEN_KEY, m_aplToken, allocator); + // Add version info + state.AddMember(VERSION_KEY, VERSION_VALUE, allocator); + rapidjson::Value arr(rapidjson::kArrayType); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + if (m_Root && m_Root->topComponent()) { + auto context = m_Root->topComponent()->serializeVisualContext(allocator); + arr.PushBack(context, allocator); + } else { + m_aplOptions->logMessage(LogLevel::ERROR, "provideStateFailed", "Unable to get visual context"); + rapidjson::Value emptyObj(rapidjson::kObjectType); + // add an empty visual context + arr.PushBack(emptyObj, allocator); + } + // Add visual context info + state.AddMember(CONTEXT_KEY, arr, allocator); + state.Accept(writer); + m_aplOptions->onVisualContextAvailable(stateRequestToken, buffer.GetString()); +} + +void AplCoreConnectionManager::interruptCommandSequence() { + if (m_Root) { + m_Root->cancelExecution(); + } +} + +void AplCoreConnectionManager::handleBuild(const rapidjson::Value& message) { + /* APL Document Inflation started */ + m_aplOptions->onRenderingEvent(AplRenderingEvent::INFLATE_BEGIN); + + auto renderingOptionsMsg = AplCoreViewhostMessage(RENDERING_OPTIONS_KEY); + rapidjson::Value renderingOptions(rapidjson::kObjectType); + renderingOptions.AddMember(LEGACY_KARAOKE_KEY, m_Content->getAPLVersion() == "1.0", renderingOptionsMsg.alloc()); + send(renderingOptionsMsg.setPayload(std::move(renderingOptions))); + + if (!m_Content) { + m_aplOptions->logMessage(LogLevel::WARN, "handleBuildFailed", "No content to build"); + sendError("No content to build"); + return; + } + + std::string agentName = getOptionalValue(message, AGENTNAME_KEY, "wssHost"); + std::string agentVersion = getOptionalValue(message, AGENTVERSION_KEY, "1.0"); + bool allowOpenUrl = getOptionalBool(message, ALLOWOPENURL_KEY, false); + bool disallowVideo = getOptionalBool(message, DISALLOWVIDEO_KEY, false); + int animationQuality = + getOptionalInt(message, ANIMATIONQUALITY_KEY, apl::RootConfig::AnimationQuality::kAnimationQualityNormal); + + // TODO: Imports on CDN got wrong APL spec versions. Should be fixed for everyone. + auto config = apl::RootConfig() + .agent(agentName, agentVersion) + .allowOpenUrl(allowOpenUrl) + .disallowVideo(disallowVideo) + .animationQuality(static_cast(animationQuality)) + .measure(std::make_shared(shared_from_this(), m_aplOptions)) + .utcTime(getCurrentTime().count()) + .localTimeAdjustment(m_aplOptions->getTimezoneOffset().count()) + .enforceAPLVersion(apl::APLVersion::kAPLVersionIgnore) + .sequenceChildCache(5); + + // Data Sources + config.dataSourceProvider( + apl::DynamicIndexListConstants::DEFAULT_TYPE_NAME, std::make_shared()); + + m_PendingEvents.clear(); + + // Release the activity tracker + m_aplOptions->onActivityEnded(APL_COMMAND_EXECUTION); + + if (m_ScreenLock) { + m_aplOptions->onActivityEnded(APL_SCREEN_LOCK); + m_ScreenLock = false; + } + + // Handle metrics data + m_Metrics.size(message[WIDTH_KEY].GetInt(), message[HEIGHT_KEY].GetInt()) + .dpi(message[DPI_KEY].GetInt()) + .shape(AVS_VIEWPORT_SHAPE_MAP.at(message[SHAPE_KEY].GetString())) + .mode(AVS_VIEWPORT_MODE_MAP.at(message[MODE_KEY].GetString())); + + do { + apl::ScalingOptions scalingOptions = { + m_ViewportSizeSpecifications, SCALING_BIAS_CONSTANT, SCALING_SHAPE_OVERRIDES_COST}; + if (!scalingOptions.getSpecifications().empty()) { + m_AplCoreMetrics = std::make_shared(m_Metrics, scalingOptions); + } else { + m_AplCoreMetrics = std::make_shared(m_Metrics); + } + + // Send scaling metrics out to viewhost + auto reply = AplCoreViewhostMessage(SCALING_KEY); + rapidjson::Value scaling(rapidjson::kObjectType); + scaling.AddMember(SCALE_FACTOR_KEY, m_AplCoreMetrics->toViewhost(1.0f), reply.alloc()); + scaling.AddMember(VIEWPORT_WIDTH_KEY, m_AplCoreMetrics->getViewhostWidth(), reply.alloc()); + scaling.AddMember(VIEWPORT_HEIGHT_KEY, m_AplCoreMetrics->getViewhostHeight(), reply.alloc()); + send(reply.setPayload(std::move(scaling))); + + m_StartTime = getCurrentTime(); + m_Root = apl::RootContext::create(m_AplCoreMetrics->getMetrics(), m_Content, config); + if (m_Root) { + break; + } else if (!m_ViewportSizeSpecifications.empty()) { + m_aplOptions->logMessage( + LogLevel::WARN, __func__, "Unable to inflate document with current chosen scaling."); + } + + auto it = m_ViewportSizeSpecifications.begin(); + for (; it != m_ViewportSizeSpecifications.end(); it++) { + if (*it == m_AplCoreMetrics->getChosenSpec()) { + m_ViewportSizeSpecifications.erase(it); + break; + } + } + if (it == m_ViewportSizeSpecifications.end()) { + // Core returned specification that is not in list. Something went wrong. Prevent infinite loop. + break; + } + } while (!m_ViewportSizeSpecifications.empty()); + + /* APL Core Inflation ended */ + m_aplOptions->onRenderingEvent(AplRenderingEvent::INFLATE_END); + + if (m_Root) { + sendDocumentThemeMessage(); + + sendDocumentBackgroundMessage(m_Content->getBackground(m_AplCoreMetrics->getMetrics(), config)); + + auto reply = AplCoreViewhostMessage(HIERARCHY_KEY); + send(reply.setPayload(m_Root->topComponent()->serialize(reply.alloc()))); + + auto idleTimeout = std::chrono::milliseconds(m_Root->settings().idleTimeout()); + m_aplOptions->onSetDocumentIdleTimeout(idleTimeout); + m_aplOptions->onRenderDocumentComplete(m_aplToken, true, ""); + } else { + m_aplOptions->logMessage(LogLevel::ERROR, "handleBuildFailed", "Unable to inflate document"); + sendError("Unable to inflate document"); + m_aplOptions->onRenderDocumentComplete(m_aplToken, false, "Unable to inflate document"); + // Send DataSource errors if any + checkAndSendDataSourceErrors(); + } +} + +void AplCoreConnectionManager::sendDocumentThemeMessage() { + if (m_Root) { + auto themeMsg = AplCoreViewhostMessage(DOCTHEME_KEY); + auto& alloc = themeMsg.alloc(); + rapidjson::Value payload(rapidjson::kObjectType); + std::string docTheme = "dark"; + if (m_Root->contextPtr()) { + docTheme = m_Root->contextPtr()->getTheme(); + } + payload.AddMember(DOCTHEME_KEY, rapidjson::Value(docTheme.c_str(), alloc).Move(), alloc); + themeMsg.setPayload(std::move(payload)); + send(themeMsg); + } +} + +void AplCoreConnectionManager::sendDocumentBackgroundMessage(const apl::Object& background) { + auto backgroundMsg = AplCoreViewhostMessage(BACKGROUND_KEY); + auto& alloc = backgroundMsg.alloc(); + rapidjson::Value payload(rapidjson::kObjectType); + rapidjson::Value backgroundValue(rapidjson::kObjectType); + if (background.isColor()) { + backgroundValue.AddMember(COLOR_KEY, background.asString(), alloc); + } else if (background.isGradient()) { + backgroundValue.AddMember(GRADIENT_KEY, background.getGradient().serialize(alloc), alloc); + } else { + backgroundValue.AddMember(COLOR_KEY, apl::Color().asString(), alloc); + } + payload.AddMember(BACKGROUND_KEY, backgroundValue, alloc); + backgroundMsg.setPayload(std::move(payload)); + send(backgroundMsg); +} + +void AplCoreConnectionManager::sendScreenLockMessage(bool screenLock) { + auto screenLockMsg = AplCoreViewhostMessage(SCREENLOCK_KEY); + auto& alloc = screenLockMsg.alloc(); + rapidjson::Value payload(rapidjson::kObjectType); + payload.AddMember(SCREENLOCK_KEY, screenLock, alloc); + screenLockMsg.setPayload(std::move(payload)); + send(screenLockMsg); +} + +void AplCoreConnectionManager::handleUpdate(const rapidjson::Value& update) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleUpdateFailed", "Root context is null"); + return; + } + + auto id = update["id"].GetString(); + auto component = m_Root->context().findComponentById(id); + if (!component) { + m_aplOptions->logMessage( + LogLevel::ERROR, "handleUpdateFailed", std::string("Unable to find component with id: ") + id); + sendError("Unable to find component"); + return; + } + + auto type = static_cast(update["type"].GetInt()); + auto value = update["value"].GetFloat(); + if (type == apl::UpdateType::kUpdateScrollPosition) { + value = m_AplCoreMetrics->toCore(value); + } + + component->update(type, value); +} + +void AplCoreConnectionManager::handleMediaUpdate(const rapidjson::Value& update) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleMediaUpdateFailed", "Root context is null"); + return; + } + + auto id = update["id"].GetString(); + auto component = m_Root->context().findComponentById(id); + if (!component) { + m_aplOptions->logMessage( + LogLevel::ERROR, "handleMediaUpdateFailed", std::string("Unable to find component with id: ") + id); + sendError("Unable to find component"); + return; + } + + if (!update.HasMember(MEDIA_STATE_KEY) || !update.HasMember(FROM_EVENT_KEY)) { + m_aplOptions->logMessage( + LogLevel::ERROR, "handleMediaUpdateFailed", "State update object is missing parameters"); + sendError("Can't update media state."); + return; + } + auto& state = update[MEDIA_STATE_KEY]; + auto fromEvent = update[FROM_EVENT_KEY].GetBool(); + + if (!state.HasMember(TRACK_INDEX_KEY) || !state.HasMember(TRACK_COUNT_KEY) || !state.HasMember(CURRENT_TIME_KEY) || + !state.HasMember(DURATION_KEY) || !state.HasMember(PAUSED_KEY) || !state.HasMember(ENDED_KEY)) { + m_aplOptions->logMessage( + LogLevel::ERROR, "handleMediaUpdateFailed", "Can't update media state. MediaStatus structure is wrong"); + sendError("Can't update media state."); + return; + } + + // numeric parameters are sometimes converted to null during stringification, set these to 0 + const int trackIndex = getOptionalInt(state, TRACK_INDEX_KEY, 0); + const int trackCount = getOptionalInt(state, TRACK_COUNT_KEY, 0); + const int currentTime = getOptionalInt(state, CURRENT_TIME_KEY, 0); + const int duration = getOptionalInt(state, DURATION_KEY, 0); + + const apl::MediaState mediaState( + trackIndex, trackCount, currentTime, duration, state[PAUSED_KEY].GetBool(), state[ENDED_KEY].GetBool()); + component->updateMediaState(mediaState, fromEvent); +} + +void AplCoreConnectionManager::handleGraphicUpdate(const rapidjson::Value& update) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleGraphicUpdateFailed", "Root context is null"); + return; + } + + auto id = update["id"].GetString(); + auto component = m_Root->context().findComponentById(id); + if (!component) { + m_aplOptions->logMessage( + LogLevel::ERROR, "handleGraphicUpdateFailed", std::string("Unable to find component with id:") + id); + sendError("Unable to find component"); + return; + } + + auto json = apl::GraphicContent::create(update["avg"].GetString()); + component->updateGraphic(json); +} + +void AplCoreConnectionManager::handleEnsureLayout(const rapidjson::Value& payload) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleEnsureLayoutFailed", "Root context is null"); + return; + } + + auto id = payload["id"].GetString(); + auto component = m_Root->context().findComponentById(id); + if (!component) { + m_aplOptions->logMessage( + LogLevel::ERROR, "handleEnsureLayoutFailed", std::string("Unable to find component with id:") + id); + sendError("Unable to find component"); + return; + } + + component->ensureLayout(true); + auto msg = AplCoreViewhostMessage(ENSURELAYOUT_KEY); + send(msg.setPayload(id)); +} + +void AplCoreConnectionManager::handleScrollToRectInComponent(const rapidjson::Value& payload) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleScrollToRectInComponentFailed", "Root context is null"); + return; + } + + auto id = payload["id"].GetString(); + auto component = m_Root->context().findComponentById(id); + if (!component) { + m_aplOptions->logMessage( + LogLevel::ERROR, + "handleScrollToRectInComponentFailed", + std::string("Unable to find component with id:") + id); + sendError("Unable to find component"); + return; + } + + apl::Rect rect = convertJsonToScaledRect(payload); + m_Root->scrollToRectInComponent(component, rect, static_cast(payload["align"].GetInt())); +} + +void AplCoreConnectionManager::handleHandleKeyboard(const rapidjson::Value& payload) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleHandleKeyboardFailed", "Root context is null"); + return; + } + + auto keyType = payload["keyType"].GetInt(); + auto code = payload["code"].GetString(); + auto key = payload["key"].GetString(); + auto repeat = payload["repeat"].GetBool(); + auto altKey = payload["altKey"].GetBool(); + auto ctrlKey = payload["ctrlKey"].GetBool(); + auto metaKey = payload["metaKey"].GetBool(); + auto shiftKey = payload["shiftKey"].GetBool(); + apl::Keyboard keyboard(code, key); + keyboard.repeat(repeat); + keyboard.alt(altKey); + keyboard.ctrl(ctrlKey); + keyboard.meta(metaKey); + keyboard.shift(shiftKey); + m_Root->handleKeyboard(static_cast(keyType), keyboard); +} + +void AplCoreConnectionManager::handleUpdateCursorPosition(const rapidjson::Value& payload) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleUpdateCursorPositionFailed", "Root context is null"); + return; + } + + const float x = payload[X_KEY].GetFloat(); + const float y = payload[Y_KEY].GetFloat(); + apl::Point cursorPosition(m_AplCoreMetrics->toCore(x), m_AplCoreMetrics->toCore(y)); + m_Root->updateCursorPosition(cursorPosition); +} + +void AplCoreConnectionManager::handleEventResponse(const rapidjson::Value& response) { + if (!m_Root) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleEventResponseFailed", "Root context is null"); + return; + } + + if (!response["event"].IsInt()) { + m_aplOptions->logMessage(LogLevel::ERROR, "handleEventResponseFailed", "Invalid event response"); + sendError("Invalid event response"); + return; + } + auto event = response["event"].GetInt(); + auto it = m_PendingEvents.find(event); + if (it != m_PendingEvents.end()) { + auto rectJson = response.FindMember("rectArgument"); + if (rectJson != response.MemberEnd()) { + apl::Rect rect = convertJsonToScaledRect(rectJson->value); + it->second.resolve(rect); + } else { + auto arg = response.FindMember("argument"); + if (arg != response.MemberEnd()) { + it->second.resolve(arg->value.GetInt()); + } else { + it->second.resolve(); + } + } + m_PendingEvents.erase(it); + } +} + +unsigned int AplCoreConnectionManager::send(AplCoreViewhostMessage& message) { + unsigned int seqno = ++m_SequenceNumber; + m_aplOptions->sendMessage(message.setSequenceNumber(seqno).get()); + return seqno; +} + +rapidjson::Document AplCoreConnectionManager::blockingSend( + AplCoreViewhostMessage& message, + const std::chrono::milliseconds& timeout) { + std::lock_guard lock{m_blockingSendMutex}; + m_replyPromise = std::promise(); + m_replyExpectedSequenceNumber = send(message); + m_blockingSendReplyExpected = true; + + auto future = m_replyPromise.get_future(); + auto status = future.wait_for(timeout); + if (status != std::future_status::ready) { + m_blockingSendReplyExpected = false; + // Under the situation that finish command destroys the renderer, there is no response. + m_aplOptions->logMessage(LogLevel::WARN, "blockingSendFailed", "Did not receive response"); + return rapidjson::Document(rapidjson::kNullType); + } + + rapidjson::Document doc; + if (doc.Parse(future.get()).HasParseError()) { + m_aplOptions->logMessage(LogLevel::ERROR, "blockingSendFailed", "parsingFailed"); + return rapidjson::Document(rapidjson::kNullType); + } + + return doc; +} + +void AplCoreConnectionManager::sendError(const std::string& message) { + auto reply = AplCoreViewhostMessage(ERROR_KEY); + send(reply.setPayload(message)); +} + +void AplCoreConnectionManager::handleScreenLock() { + if (m_Root->screenLock() && !m_ScreenLock) { + m_aplOptions->onActivityStarted(APL_SCREEN_LOCK); + m_ScreenLock = true; + } else if (!m_Root->screenLock() && m_ScreenLock) { + m_aplOptions->onActivityEnded(APL_SCREEN_LOCK); + m_ScreenLock = false; + } else { + return; + } + sendScreenLockMessage(m_ScreenLock); +} + +void AplCoreConnectionManager::processEvent(const apl::Event& event) { + if (apl::EventType::kEventTypeFinish == event.getType()) { + m_aplOptions->onFinish(); + return; + } + + if (apl::EventType::kEventTypeSendEvent == event.getType()) { + rapidjson::Document userEventPayloadJson(rapidjson::kObjectType); + auto& allocator = userEventPayloadJson.GetAllocator(); + auto source = event.getValue(apl::EventProperty::kEventPropertySource); + auto components = event.getValue(apl::EventProperty::kEventPropertyComponents); + auto arguments = event.getValue(apl::EventProperty::kEventPropertyArguments); + + userEventPayloadJson.AddMember( + PRESENTATION_TOKEN_KEY, rapidjson::Value(m_aplToken.c_str(), allocator).Move(), allocator); + userEventPayloadJson.AddMember(SOURCE_KEY, source.serialize(allocator).Move(), allocator); + userEventPayloadJson.AddMember(ARGUMENTS_KEY, arguments.serialize(allocator).Move(), allocator); + userEventPayloadJson.AddMember(COMPONENTS_KEY, components.serialize(allocator).Move(), allocator); + + rapidjson::StringBuffer sb; + rapidjson::Writer writer(sb); + userEventPayloadJson.Accept(writer); + + m_aplOptions->onSendEvent(sb.GetString()); + return; + } + + if (apl::EventType::kEventTypeDataSourceFetchRequest == event.getType()) { + rapidjson::Document fetchRequestPayloadJson(rapidjson::kObjectType); + auto& allocator = fetchRequestPayloadJson.GetAllocator(); + auto type = event.getValue(apl::EventProperty::kEventPropertyName); + auto payload = event.getValue(apl::EventProperty::kEventPropertyValue); + + apl::ObjectMap fetchRequest(payload.getMap()); + fetchRequest.emplace(PRESENTATION_TOKEN_KEY, m_aplToken); + + auto fetch = apl::Object(std::make_shared(fetchRequest)).serialize(allocator); + rapidjson::StringBuffer sb; + rapidjson::Writer writer(sb); + fetch.Accept(writer); + + m_aplOptions->onDataSourceFetchRequestEvent(type.asString(), sb.GetString()); + return; + } + + auto msg = AplCoreViewhostMessage(EVENT_KEY); + auto token = send(msg.setPayload(event.serialize(msg.alloc()))); + + // If the event had an action ref, stash the reference for future use + auto ref = event.getActionRef(); + if (!ref.isEmpty()) { + m_PendingEvents.emplace(token, ref); + ref.addTerminateCallback([this, token](const apl::TimersPtr&) { + auto it = m_PendingEvents.find(token); + if (it != m_PendingEvents.end()) { + m_PendingEvents.erase(it); // Remove the pending event + + auto msg = AplCoreViewhostMessage(EVENT_TERMINATE_KEY); + rapidjson::Value payload(rapidjson::kObjectType); + payload.AddMember("token", token, msg.alloc()); + send(msg.setPayload(std::move(payload))); + } else { + m_aplOptions->logMessage(LogLevel::WARN, __func__, "Event was not pending"); + } + }); + } +} + +void AplCoreConnectionManager::processDirty(const std::set& dirty) { + std::map tempDirty; + auto msg = AplCoreViewhostMessage(DIRTY_KEY); + + for (auto& component : dirty) { + if (component->getDirty().count(apl::kPropertyNotifyChildrenChanged)) { + auto notify = component->getCalculated(apl::kPropertyNotifyChildrenChanged); + const auto& changed = notify.getArray(); + // Whenever we get NotifyChildrenChanged we get 2 types of action + // Either insert or delete. The delete will happen on the viewhost level + // However, insert needs the full serialized component from core & will be initalized + // on apl-client side + for (size_t i = 0; i < changed.size(); i++) { + auto newChildId = changed.at(i).get("uid").asString(); + auto newChildIndex = changed.at(i).get("index").asInt(); + auto action = changed.at(i).get("action").asString(); + if (action == "insert") { + tempDirty[newChildId] = component->getChildAt(newChildIndex)->serialize(msg.alloc()); + } + } + } + if (tempDirty.find(component->getUniqueId()) == tempDirty.end()) { + tempDirty.emplace(component->getUniqueId(), component->serializeDirty(msg.alloc())); + } + } + + rapidjson::Value array(rapidjson::kArrayType); + for (auto rit = tempDirty.rbegin(); rit != tempDirty.rend(); rit++) { + auto uid = rit->first; + auto& update = rit->second; + + array.PushBack(update.Move(), msg.alloc()); + } + send(msg.setPayload(std::move(array))); +} + +void AplCoreConnectionManager::coreFrameUpdate() { + auto now = getCurrentTime() - m_StartTime; + m_Root->updateTime(now.count(), getCurrentTime().count()); + m_Root->setLocalTimeAdjustment(m_aplOptions->getTimezoneOffset().count()); + + m_Root->clearPending(); + + while (m_Root->hasEvent()) { + processEvent(m_Root->popEvent()); + } + + if (m_Root->isDirty()) { + processDirty(m_Root->getDirty()); + m_Root->clearDirty(); + } + + handleScreenLock(); +} + +void AplCoreConnectionManager::onUpdateTick() { + if (m_Root) { + coreFrameUpdate(); + // Check regularly as something like timed-out fetch requests could come up. + checkAndSendDataSourceErrors(); + } +} + +apl::Rect AplCoreConnectionManager::convertJsonToScaledRect(const rapidjson::Value& jsonNode) { + const float scale = m_AplCoreMetrics->toCore(1.0f); + const float x = jsonNode[X_KEY].IsNumber() ? jsonNode[X_KEY].GetFloat() : 0.0f; + const float y = jsonNode[Y_KEY].IsNumber() ? jsonNode[Y_KEY].GetFloat() : 0.0f; + const float width = jsonNode[WIDTH_KEY].IsNumber() ? jsonNode[WIDTH_KEY].GetFloat() : 0.0f; + const float height = jsonNode[HEIGHT_KEY].IsNumber() ? jsonNode[HEIGHT_KEY].GetFloat() : 0.0f; + + return apl::Rect(x * scale, y * scale, width * scale, height * scale); +} + +void AplCoreConnectionManager::checkAndSendDataSourceErrors() { + // TODO: Single provider supported as of now. + auto provider = + m_Root->context().getRootConfig().getDataSourceProvider(apl::DynamicIndexListConstants::DEFAULT_TYPE_NAME); + if (provider) { + auto errors = provider->getPendingErrors(); + if (!errors.empty() && errors.isArray()) { + auto errorEvent = std::make_shared(); + errorEvent->emplace(PRESENTATION_TOKEN_KEY, m_aplToken); + errorEvent->emplace(ERRORS_KEY, errors); + + rapidjson::Document runtimeErrorPayloadJson(rapidjson::kObjectType); + auto& allocator = runtimeErrorPayloadJson.GetAllocator(); + auto runtimeError = apl::Object(errorEvent).serialize(allocator); + + rapidjson::StringBuffer sb; + rapidjson::Writer writer(sb); + runtimeError.Accept(writer); + + m_aplOptions->onRuntimeErrorEvent(sb.GetString()); + } + } +} + +void AplCoreConnectionManager::reset() { + m_aplToken = ""; + m_Root.reset(); + m_Content.reset(); +} + +} // namespace APLClient diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreEngineLogBridge.cpp b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreEngineLogBridge.cpp new file mode 100644 index 0000000..e38caa1 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreEngineLogBridge.cpp @@ -0,0 +1,51 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#include "APLClient/AplCoreEngineLogBridge.h" + +static const std::string TAG("AplCoreEngine"); + +namespace APLClient { + +AplCoreEngineLogBridge::AplCoreEngineLogBridge(AplOptionsInterfacePtr aplOptions) : m_aplOptions{aplOptions} { +} + +void AplCoreEngineLogBridge::transport(apl::LogLevel level, const std::string& log) { + switch (level) { + case apl::LogLevel::TRACE: + m_aplOptions->logMessage(LogLevel::TRACE, TAG, log); + break; + // TODO: Same problem as in AplCoreGuiRenderer.h but not solved by undef by some reason. + case static_cast(1): + m_aplOptions->logMessage(LogLevel::DBG, TAG, log); + break; + case apl::LogLevel::INFO: + m_aplOptions->logMessage(LogLevel::INFO, TAG, log); + break; + case apl::LogLevel::WARN: + m_aplOptions->logMessage(LogLevel::WARN, TAG, log); + break; + case apl::LogLevel::ERROR: + m_aplOptions->logMessage(LogLevel::ERROR, TAG, log); + break; + case apl::LogLevel::CRITICAL: + m_aplOptions->logMessage(LogLevel::CRITICAL, TAG, log); + break; + default: + m_aplOptions->logMessage(LogLevel::ERROR, "AplCoreEngineUnknownLogLevel", log); + return; + } +} +} // namespace APLClient \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreGuiRenderer.cpp b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreGuiRenderer.cpp new file mode 100644 index 0000000..3115a0e --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreGuiRenderer.cpp @@ -0,0 +1,151 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ +#include + +#include + +#include "APLClient/AplCoreGuiRenderer.h" + +namespace APLClient { + +/// CDN for alexa import packages (styles/resources/etc) +/// (https://developer.amazon.com/en-US/docs/alexa/alexa-presentation-language/apl-document.html#import) +static const char* ALEXA_IMPORT_PATH = "https://d2na8397m465mh.cloudfront.net/packages/%s/%s/document.json"; +/// The number of bytes read from the attachment with each read in the read loop. +static const size_t CHUNK_SIZE(1024); +/// Name of the mainTemplate parameter to which avs datasources binds to. +static const std::string DEFAULT_PARAM_BINDING = "payload"; +/// Default string to attach to mainTemplate parameters. +static const std::string DEFAULT_PARAM_VALUE = "{}"; + +AplCoreGuiRenderer::AplCoreGuiRenderer( + AplOptionsInterfacePtr aplOptions, + AplCoreConnectionManagerPtr aplCoreConnectionManager) : + m_isDocumentCleared{false}, + m_aplOptions{aplOptions}, + m_aplCoreConnectionManager{aplCoreConnectionManager} { +} + +void AplCoreGuiRenderer::executeCommands(const std::string& jsonPayload, const std::string& token) { + m_aplCoreConnectionManager->executeCommands(jsonPayload, token); +} + +void AplCoreGuiRenderer::dataSourceUpdate( + const std::string& sourceType, + const std::string& jsonPayload, + const std::string& token) { + m_aplCoreConnectionManager->dataSourceUpdate(sourceType, jsonPayload, token); +} + +void AplCoreGuiRenderer::interruptCommandSequence() { + m_aplCoreConnectionManager->interruptCommandSequence(); +} + +void AplCoreGuiRenderer::renderDocument( + const std::string& document, + const std::string& data, + const std::string& supportedViewports, + const std::string& token) { + m_isDocumentCleared = false; + + auto content = apl::Content::create(std::move(document)); + if (!content) { + m_aplOptions->logMessage(LogLevel::ERROR, "renderByAplCoreFailed", "Unable to create content"); + + m_aplOptions->onRenderDocumentComplete(token, false, "Unable to create content"); + return; + } + + std::map params; + apl::JsonData sourcesData(data); + if (sourcesData.get().IsObject()) { + for (auto objIt = sourcesData.get().MemberBegin(); objIt != sourcesData.get().MemberEnd(); objIt++) { + params.emplace(objIt->name.GetString(), objIt->value); + } + } + + for (size_t idx = 0; idx < content->getParameterCount(); idx++) { + auto parameterName = content->getParameterAt(idx); + if (parameterName == DEFAULT_PARAM_BINDING) { + content->addData(parameterName, data); + } else if (params.find(parameterName) != params.end()) { + content->addData(parameterName, params.at(parameterName).toString()); + } else { + content->addData(parameterName, DEFAULT_PARAM_VALUE); + } + } + + std::unordered_map> packageContentByRequestId; + std::unordered_map packageRequestByRequestId; + while (content->isWaiting() && !content->isError()) { + auto packages = content->getRequestedPackages(); + unsigned int count = 0; + for (auto& package : packages) { + auto name = package.reference().name(); + auto version = package.reference().version(); + auto source = package.source(); + + if (source.empty()) { + char sourceBuffer[CHUNK_SIZE]; + snprintf(sourceBuffer, CHUNK_SIZE, ALEXA_IMPORT_PATH, name.c_str(), version.c_str()); + source = sourceBuffer; + } + + auto packageContentPromise = async(std::launch::async, &AplOptionsInterface::downloadResource, m_aplOptions, source); + packageContentByRequestId.insert(std::make_pair(package.getUniqueId(), std::move(packageContentPromise))); + packageRequestByRequestId.insert(std::make_pair(package.getUniqueId(), package)); + count++; + + // if we reach the maximum number of concurrent downloads or already go through all packages, wait for them to finish + if (count % m_aplOptions->getMaxNumberOfConcurrentDownloads() == 0 || packages.size() == count) { + for (auto& kvp : packageContentByRequestId) { + auto packageContent = kvp.second.get(); + if (packageContent.empty()) { + m_aplOptions->logMessage( + LogLevel::ERROR, "renderByAplCoreFailed", "Could not be retrieve requested import"); + + m_aplOptions->onRenderDocumentComplete(token, false, "Unresolved import"); + return; + } + content->addPackage(packageRequestByRequestId.at(kvp.first), packageContent); + } + packageContentByRequestId.clear(); + packageRequestByRequestId.clear(); + } + } + } + + if (!content->isReady()) { + m_aplOptions->logMessage(LogLevel::ERROR, "renderByAplCoreFailed", "Content is not ready"); + + m_aplOptions->onRenderDocumentComplete(token, false, "Content is not ready"); + return; + } + + if (!m_isDocumentCleared) { + /** + * Only set the content if we haven't been cleared while building. + */ + m_aplCoreConnectionManager->setSupportedViewports(supportedViewports); + m_aplCoreConnectionManager->setContent(content, token); + } +} + +void AplCoreGuiRenderer::clearDocument() { + m_isDocumentCleared = true; + m_aplCoreConnectionManager->reset(); +} + +} // namespace APLClient diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreMetrics.cpp b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreMetrics.cpp new file mode 100644 index 0000000..6b50d16 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreMetrics.cpp @@ -0,0 +1,38 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#include "APLClient/AplCoreMetrics.h" + +namespace APLClient { + +static const float DEFAULT_BASE_DPI = 160.0f; + +float AplCoreMetrics::toViewhost(float value) const { + return value * getScaleToViewhost() * getDpi() / DEFAULT_BASE_DPI; +} + +float AplCoreMetrics::toCore(float value) const { + return value * getScaleToCore() * DEFAULT_BASE_DPI / getDpi(); +} + +float AplCoreMetrics::getViewhostWidth() const { + return toViewhost(getWidth()); +} + +float AplCoreMetrics::getViewhostHeight() const { + return toViewhost(getHeight()); +} + +} // namespace APLClient \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreTextMeasurement.cpp b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreTextMeasurement.cpp new file mode 100644 index 0000000..0238207 --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/AplCoreTextMeasurement.cpp @@ -0,0 +1,135 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#include "APLClient/AplCoreViewhostMessage.h" +#include "APLClient/AplCoreTextMeasurement.h" + +namespace APLClient { + +/// The keys used in APL text measurement. +static const char MEASURE_KEY[] = "measure"; +static const char BASELINE_KEY[] = "baseline"; + +/** + * Request a text measurement. + * + * { "type": "measure", + * "payload": { + * "id": UNIQUE_ID, + * "width": FLOAT, + * "height": FLOAT, + * "widthMode": INT, + * "heightMode": INT + * .... + * }} + * + * The response: + * + * { "type": "measure", + * "payload": { + * "width": FLOAT, + * "height": FLOAT + * }} + * + * @param component + * @param width + * @param widthMode + * @param height + * @param heightMode + * @return + */ +YGSize AplCoreTextMeasurement::measure( + apl::TextComponent* component, + float width, + YGMeasureMode widthMode, + float height, + YGMeasureMode heightMode) { + /* Notify about the text measurement event */ + m_aplOptions->onRenderingEvent(AplRenderingEvent::TEXT_MEASURE); + + if (auto aplCoreConnectionManager = m_aplCoreConnectionManager.lock()) { + auto msg = AplCoreViewhostMessage(MEASURE_KEY); + auto& alloc = msg.alloc(); + + auto aplCoreMetrics = aplCoreConnectionManager->aplCoreMetrics(); + + rapidjson::Value payload(component->serialize(alloc)); + payload.AddMember("width", aplCoreMetrics->toViewhost(std::isnan(width) ? INT_MAX : width), alloc); + payload.AddMember("height", aplCoreMetrics->toViewhost(std::isnan(height) ? INT_MAX : height), alloc); + payload.AddMember("widthMode", widthMode, alloc); + payload.AddMember("heightMode", heightMode, alloc); + msg.setPayload(std::move(payload)); + + auto result = aplCoreConnectionManager->blockingSend(msg); + + if (result.IsObject()) { + auto measuredWidth = aplCoreMetrics->toCore(result["payload"]["width"].GetFloat()); + auto measuredHeight = aplCoreMetrics->toCore(result["payload"]["height"].GetFloat()); + + return {measuredWidth, measuredHeight}; + } + + m_aplOptions->logMessage(LogLevel::WARN, __func__, "Didn't get a valid reply. Returning generic size."); + return {aplCoreMetrics->toCore(100), aplCoreMetrics->toCore(100)}; + } else { + m_aplOptions->logMessage(LogLevel::WARN, __func__, "ConnectionManager does not exist. Returning generic size."); + return {0, 0}; + } +} + +/** + * Send a message to the view host asking for a baseline calculation: + * + * { "type": "baseline", + * "payload": { + * "id": UNIQUE_ID, + * "width": FLOAT, + * "height": FLOAT }} + * + * The result should look like: + * + * { "type": "baseline", + * "payload": FLOAT } + * + * @param component + * @param width + * @param height + * @return + */ +float AplCoreTextMeasurement::baseline(apl::TextComponent* component, float width, float height) { + if (auto aplCoreConnectionManager = m_aplCoreConnectionManager.lock()) { + auto msg = AplCoreViewhostMessage(BASELINE_KEY); + auto& alloc = msg.alloc(); + + auto aplCoreMetrics = aplCoreConnectionManager->aplCoreMetrics(); + + rapidjson::Value payload(rapidjson::kObjectType); + payload.AddMember("id", rapidjson::Value(component->getUniqueId().c_str(), alloc).Move(), alloc); + payload.AddMember("width", aplCoreMetrics->toViewhost(width), alloc); + payload.AddMember("height", aplCoreMetrics->toViewhost(height), alloc); + msg.setPayload(std::move(payload)); + + auto result = aplCoreConnectionManager->blockingSend(msg); + + if (result.IsObject()) { + auto it = result.FindMember("payload"); + if (it != result.MemberEnd()) return aplCoreMetrics->toCore(it->value.GetFloat()); + } + } + m_aplOptions->logMessage(LogLevel::WARN, __func__, "Got invalid result from baseline calculation. Returning 0."); + return 0; +} + +} // namespace APLClient \ No newline at end of file diff --git a/modules/Alexa/ApplicationUtilities/APLClient/src/CMakeLists.txt b/modules/Alexa/ApplicationUtilities/APLClient/src/CMakeLists.txt new file mode 100644 index 0000000..926fb1b --- /dev/null +++ b/modules/Alexa/ApplicationUtilities/APLClient/src/CMakeLists.txt @@ -0,0 +1,39 @@ +cmake_minimum_required(VERSION 3.1 FATAL_ERROR) + +add_library(APLClient SHARED + AplClientBinding.cpp + AplCoreConnectionManager.cpp + AplCoreEngineLogBridge.cpp + AplCoreGuiRenderer.cpp + AplCoreMetrics.cpp + AplCoreTextMeasurement.cpp + ) + +target_include_directories(APLClient PUBLIC + "${ASDK_INCLUDE_DIRS}" + "${APLClient_SOURCE_DIR}/include") + +if(NOT APLCORE_INCLUDE_DIR) + message(FATAL_ERROR "APLCore Include Dir is required") +endif() + +if(NOT APLCORE_LIB_DIR) + message(FATAL_ERROR "APLCore Lib Dir is required") +endif() + +if(APL_CORE) + target_include_directories(APLClient PUBLIC + "${APLCORE_INCLUDE_DIR}" + "${YOGA_INCLUDE_DIR}") + + target_link_libraries(APLClient ${APLCORE_LIB_DIR}/libapl.a) + target_link_libraries(APLClient ${YOGA_LIB_DIR}/libyogacore.a) + + set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-error -Wno-reorder") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error -Wno-reorder") + + add_definitions(-DAPL_CORE) +endif() + +# install target +asdk_install() diff --git a/modules/Alexa/ApplicationUtilities/CMakeLists.txt b/modules/Alexa/ApplicationUtilities/CMakeLists.txt index 9e507d0..34f15ac 100644 --- a/modules/Alexa/ApplicationUtilities/CMakeLists.txt +++ b/modules/Alexa/ApplicationUtilities/CMakeLists.txt @@ -3,6 +3,9 @@ project(ApplicationUtilities LANGUAGES CXX) include(../build/BuildDefaults.cmake) -add_subdirectory("Communication") +add_subdirectory("APLClient") +if (NOT MSVC) + add_subdirectory("Communication") +endif () add_subdirectory("SmartScreenClient") diff --git a/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketConfig.h b/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketConfig.h index b2bfa25..1a20552 100644 --- a/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketConfig.h +++ b/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketConfig.h @@ -1,5 +1,5 @@ /* - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketSDKLogger.h b/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketSDKLogger.h index 5a9d6d2..c2382cc 100644 --- a/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketSDKLogger.h +++ b/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketSDKLogger.h @@ -1,5 +1,5 @@ /* - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketServer.h b/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketServer.h index 1cee2df..6687d2a 100644 --- a/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketServer.h +++ b/modules/Alexa/ApplicationUtilities/Communication/include/Communication/WebSocketServer.h @@ -1,5 +1,5 @@ /* - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketSDKLogger.cpp b/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketSDKLogger.cpp index 13a3199..3051cf0 100644 --- a/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketSDKLogger.cpp +++ b/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketSDKLogger.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketServer.cpp b/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketServer.cpp index d42e03e..342acf1 100644 --- a/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketServer.cpp +++ b/modules/Alexa/ApplicationUtilities/Communication/src/WebSocketServer.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/DeviceSettingsManagerBuilder.h b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/DeviceSettingsManagerBuilder.h index 170269e..9c7f412 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/DeviceSettingsManagerBuilder.h +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/DeviceSettingsManagerBuilder.h @@ -1,5 +1,5 @@ /* - * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. @@ -146,7 +146,22 @@ class DeviceSettingsManagerBuilder */ std::unique_ptr build() override; + /** + * Gets Device Time Zone Offset. + * + * @return Timezone offset in milliseconds. + */ + std::chrono::milliseconds getDeviceTimezoneOffset(); + private: + /** + * Calculates Device Timezone Offset. + * + * @param timeZone Timezone string + * @return Timezone offset in milliseconds. + */ + std::chrono::milliseconds calculateDeviceTimezoneOffset(const std::string& timeZone); + /** * Builds a setting that follows the given synchronization protocol. * @@ -177,6 +192,9 @@ class DeviceSettingsManagerBuilder /// Flag that indicates if there was any configuration error. bool m_foundError; + + /// Device Timezone Offset in milliseconds. + std::chrono::milliseconds m_deviceTimeZoneOffset; }; template diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/EqualizerRuntimeSetup.h b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/EqualizerRuntimeSetup.h index da2ba13..bdd9139 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/EqualizerRuntimeSetup.h +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/EqualizerRuntimeSetup.h @@ -1,5 +1,5 @@ /* - * Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/ExternalCapabilitiesBuilderInterface.h b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/ExternalCapabilitiesBuilderInterface.h index 3ee84b3..467df1e 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/ExternalCapabilitiesBuilderInterface.h +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/ExternalCapabilitiesBuilderInterface.h @@ -1,5 +1,5 @@ /* - * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/SmartScreenClient.h b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/SmartScreenClient.h index a5e1cbd..a64a18e 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/SmartScreenClient.h +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/SmartScreenClient.h @@ -1,5 +1,5 @@ /* - * Copyright 2017-2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. @@ -54,6 +54,7 @@ #include #include #include +#include #include #include #include @@ -78,6 +79,7 @@ #include #include +#include #include #include #include @@ -115,6 +117,7 @@ #include #include #include +#include #ifdef ENABLE_REVOKE_AUTH #include @@ -764,6 +767,19 @@ class SmartScreenClient */ void sendUserEvent(const std::string& payload); + /** + * Sends a DataSourceFetchRequest event. + * @param type DataSource type. + * @param payload event. + */ + void sendDataSourceFetchRequestEvent(const std::string& type, const std::string& payload); + + /** + * Sends a RuntimeError event. + * @param payload event. + */ + void sendRuntimeErrorEvent(const std::string& payload); + /** * Handle visual context. * @param token The visual context token. @@ -804,6 +820,20 @@ class SmartScreenClient */ void setDocumentIdleTimeout(std::chrono::milliseconds timeout); + /** + * Add an observer to SpeechSynthesizer. + * @param observer The SpeechSynthesizer observer. + */ + void addSpeechSynthesizerObserver( + std::shared_ptr observer); + + /** + * Remove an observer from SpeechSynthesizer. + * @param observer The SpeechSynthesizer observer. + */ + void removeSpeechSynthesizerObserver( + std::shared_ptr observer); + /** * Clear all execute commands. */ @@ -815,6 +845,31 @@ class SmartScreenClient */ void setDeviceWindowState(const std::string& payload); + /** + * Gets Device Time Zone Offset. + */ + std::chrono::milliseconds getDeviceTimezoneOffset(); + + /** + * handle the renderComplete notification for APL Document + * @param isAlexaPresentationPresenting Whether the AlexaPresentation agent is presenting or not. + */ + void handleRenderComplete(bool isAlexaPresentationPresenting); + + /** + * handle the displayMetrics notification for APL Document + * @param dropFrameCount Count of the number of frames dropped. + * @param isAlexaPresentationPresenting Whether the AlexaPresentation agent is presenting or not. + */ + void handleDropFrameCount(uint64_t dropFrameCount, bool isAlexaPresentationPresenting); + + /** + * handle the event raised by APL core engine + * @param event APL Event in context + * @param isAlexaPresentationPresenting Whether the AlexaPresentation agent is presenting or not. + */ + void handleAPLEvent(APLClient::AplRenderingEvent event, bool isAlexaPresentationPresenting); + /** * Destructor. */ @@ -1142,6 +1197,8 @@ class SmartScreenClient /// The @c AVSGatewayManager instance used in the AVS Gateway connection sequence. std::shared_ptr m_avsGatewayManager; + /// Device Timezone Offset in milliseconds. + std::chrono::milliseconds m_deviceTimeZoneOffset; #ifdef ENABLE_COMMS_AUDIO_PROXY /// The CallAudioDeviceProxy used to work with audio proxy audio driver of CommsLib. std::shared_ptr m_callAudioDeviceProxy; diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/CMakeLists.txt b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/CMakeLists.txt index 6196bd5..d80cb2e 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/CMakeLists.txt +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/CMakeLists.txt @@ -4,14 +4,15 @@ add_definitions("-DACSDK_LOG_MODULE=smartScreenClient") add_library(SmartScreenClient SHARED SmartScreenClient.cpp EqualizerRuntimeSetup.cpp - DeviceSettingsManagerBuilder.cpp - ) + DeviceSettingsManagerBuilder.cpp) target_include_directories(SmartScreenClient PUBLIC "${SmartScreenClient_SOURCE_DIR}/include" "${AlexaPresentation_SOURCE_DIR}/include" - "${TemplateRuntime_SOURCE_DIR}/include" + "${SmartScreenTemplateRunTime_SOURCE_DIR}/include" "${VisualCharacteristics_SOURCE_DIR}/include" - "${ASDK_INCLUDE_DIRS}") + "${ASDK_INCLUDE_DIRS}" + "${SQLITE_INCLUDE_DIRS}") + if(BLUETOOTH_BLUEZ) target_link_libraries(SmartScreenClient diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/DeviceSettingsManagerBuilder.cpp b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/DeviceSettingsManagerBuilder.cpp index dd2a7d1..94952d3 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/DeviceSettingsManagerBuilder.cpp +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/DeviceSettingsManagerBuilder.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. @@ -22,7 +22,6 @@ #include #include #include - #include "SmartScreenClient/DeviceSettingsManagerBuilder.h" /// String to identify log entries originating from this file. @@ -133,12 +132,39 @@ DeviceSettingsManagerBuilder& DeviceSettingsManagerBuilder::withTimeZoneSetting( settingsConfig.getString(DEFAULT_TIMEZONE_CONFIGURATION_KEY, &defaultTimezone, DEFAULT_TIMEZONE); } + m_deviceTimeZoneOffset = calculateDeviceTimezoneOffset(defaultTimezone); + return withSynchronizedSetting( alexaClientSDK::capabilityAgents::system::TimeZoneHandler::getTimeZoneMetadata(), defaultTimezone, applyFunction); } +std::chrono::milliseconds DeviceSettingsManagerBuilder::getDeviceTimezoneOffset() { + return m_deviceTimeZoneOffset; +} + +std::chrono::milliseconds DeviceSettingsManagerBuilder::calculateDeviceTimezoneOffset(const std::string& timeZone) { +#ifdef _MSC_VER + TIME_ZONE_INFORMATION TimeZoneInfo; + GetTimeZoneInformation(&TimeZoneInfo); + auto offsetInMinutes = - TimeZoneInfo.Bias - TimeZoneInfo.DaylightBias; + ACSDK_DEBUG9(LX(__func__).m(std::to_string(offsetInMinutes))); + return std::chrono::minutes(offsetInMinutes); +#else + char *prevTZ = getenv("TZ"); + setenv("TZ", timeZone.c_str(), 1); + time_t t = time(NULL); + struct tm *structtm = localtime(&t); + if (prevTZ) { + setenv("TZ", prevTZ, 1); + } else { + unsetenv("TZ"); + } + return std::chrono::milliseconds(structtm->tm_gmtoff * 1000); +#endif +} + DeviceSettingsManagerBuilder& DeviceSettingsManagerBuilder::withLocaleSetting( std::shared_ptr localeAssetsManager) { // TODO: LocaleWakeWordsSetting should accept nullptr as wakeWordEventSender if wake words are disabled. diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/EqualizerRuntimeSetup.cpp b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/EqualizerRuntimeSetup.cpp index b929cde..2d177cb 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/EqualizerRuntimeSetup.cpp +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/EqualizerRuntimeSetup.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. diff --git a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/SmartScreenClient.cpp b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/SmartScreenClient.cpp index 46b82dd..01ad0d4 100644 --- a/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/SmartScreenClient.cpp +++ b/modules/Alexa/ApplicationUtilities/SmartScreenClient/src/SmartScreenClient.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2017-2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. @@ -20,6 +20,7 @@ #include #include #include +#include #ifdef ACSDK_ENABLE_METRICS_RECORDING #include @@ -555,6 +556,8 @@ bool SmartScreenClient::initialize( return false; } + m_deviceTimeZoneOffset = settingsManagerBuilder.getDeviceTimezoneOffset(); + /* * Creating the Audio Activity Tracker - This component is responsibly for * reporting the audio channel focus @@ -1057,7 +1060,12 @@ bool SmartScreenClient::initialize( */ m_alexaPresentation = alexaSmartScreenSDK::smartScreenCapabilityAgents::alexaPresentation::AlexaPresentation::create( - m_visualFocusManager, m_exceptionSender, m_connectionManager, contextManager, visualStateProvider); + m_visualFocusManager, + m_exceptionSender, + metricRecorder, + m_connectionManager, + contextManager, + visualStateProvider); if (!m_alexaPresentation) { ACSDK_ERROR(LX("initializeFailed").d("reason", "unableToCreateAlexaPresentationCapabilityAgent")); return false; @@ -1424,6 +1432,7 @@ void SmartScreenClient::onFocusChanged( alexaClientSDK::avsCommon::avs::MixingBehavior behavior) { if (newFocus == alexaClientSDK::avsCommon::avs::FocusState::FOREGROUND) { stopForegroundActivity(); + m_audioInputProcessor->resetState(); clearCard(); } } @@ -1776,6 +1785,14 @@ void SmartScreenClient::sendUserEvent(const std::string& payload) { m_alexaPresentation->sendUserEvent(payload); } +void SmartScreenClient::sendDataSourceFetchRequestEvent(const std::string& type, const std::string& payload) { + m_alexaPresentation->sendDataSourceFetchRequestEvent(type, payload); +} + +void SmartScreenClient::sendRuntimeErrorEvent(const std::string& payload) { + m_alexaPresentation->sendRuntimeErrorEvent(payload); +} + void SmartScreenClient::handleVisualContext(uint64_t token, std::string payload) { m_alexaPresentation->onVisualContextAvailable(token, payload); } @@ -1811,6 +1828,46 @@ void SmartScreenClient::setDeviceWindowState(const std::string& payload) { m_visualCharacteristics->setDeviceWindowState(payload); } +void SmartScreenClient::addSpeechSynthesizerObserver( + std::shared_ptr observer) { + if (!m_speechSynthesizer) { + ACSDK_ERROR(LX("addSpeechSynthesizerObserverFailed").d("reason", "speechSynthesizerNotSupported")); + return; + } + m_speechSynthesizer->addObserver(observer); +}; + +void SmartScreenClient::removeSpeechSynthesizerObserver( + std::shared_ptr observer) { + if (!m_speechSynthesizer) { + ACSDK_ERROR(LX("addSpeechSynthesizerObserverFailed").d("reason", "speechSynthesizerNotSupported")); + return; + } + m_speechSynthesizer->removeObserver(observer); +} + +std::chrono::milliseconds SmartScreenClient::getDeviceTimezoneOffset() { + return m_deviceTimeZoneOffset; +} + +void SmartScreenClient::handleRenderComplete(bool isAlexaPresentationPresenting) { + if (isAlexaPresentationPresenting) { + m_alexaPresentation->recordRenderComplete(); + } +} + +void SmartScreenClient::handleDropFrameCount(uint64_t dropFrameCount, bool isAlexaPresentationPresenting) { + if (isAlexaPresentationPresenting) { + m_alexaPresentation->recordDropFrameCount(dropFrameCount); + } +} + +void SmartScreenClient::handleAPLEvent(APLClient::AplRenderingEvent event, bool isAlexaPresentationPresenting) { + if (isAlexaPresentationPresenting) { + m_alexaPresentation->recordAPLEvent(event); + } +} + SmartScreenClient::~SmartScreenClient() { ACSDK_DEBUG3(LX(__func__)); diff --git a/modules/Alexa/CMakeLists.txt b/modules/Alexa/CMakeLists.txt index cb8629a..e389612 100644 --- a/modules/Alexa/CMakeLists.txt +++ b/modules/Alexa/CMakeLists.txt @@ -2,8 +2,53 @@ cmake_minimum_required(VERSION 3.1 FATAL_ERROR) project(AlexaModule LANGUAGES CXX) -find_package(PkgConfig) -pkg_check_modules(ASDK REQUIRED AlexaClientSDK=1.19.0) +if (NOT MSVC) + find_package(PkgConfig) + pkg_check_modules(ASDK REQUIRED AlexaClientSDK=1.19.1) +else() + # No package manager support in MSVC yet. Emulating the same results + set(ASDK_INCLUDE_DIRS "${ASDK_INSTALL_PATH}/include") + set(ASDK_LIB_DIR "${ASDK_INSTALL_PATH}/lib") + set(SQLITE_INCLUDE_DIRS "${DEPS_PATH}/include") + + message (WARNING "SQLITE_INCLUDE_DIRS ${SQLITE_INCLUDE_DIRS} ASDK_INCLUDE_DIRS ${ASDK_INCLUDE_DIRS}" ) + set(ASDK_LDFLAGS + "${ASDK_LIB_DIR}/ACL.lib" + "${ASDK_LIB_DIR}/ADSL.lib" + "${ASDK_LIB_DIR}/AFML.lib" + "${ASDK_LIB_DIR}/AIP.lib" + "${ASDK_LIB_DIR}/Alerts.lib" + "${ASDK_LIB_DIR}/Alexa.lib" + "${ASDK_LIB_DIR}/ApiGateway.lib" + "${ASDK_LIB_DIR}/AudioPlayer.lib" + "${ASDK_LIB_DIR}/AudioResources.lib" + "${ASDK_LIB_DIR}/AVSCommon.lib" + "${ASDK_LIB_DIR}/AVSSystem.lib" + "${ASDK_LIB_DIR}/Bluetooth.lib" + "${ASDK_LIB_DIR}/CapabilitiesDelegate.lib" + "${ASDK_LIB_DIR}/CBLAuthDelegate.lib" + "${ASDK_LIB_DIR}/CertifiedSender.lib" + "${ASDK_LIB_DIR}/ContextManager.lib" + "${ASDK_LIB_DIR}/DefaultClient.lib" + "${ASDK_LIB_DIR}/DeviceSettings.lib" + "${ASDK_LIB_DIR}/DoNotDisturbCA.lib" + "${ASDK_LIB_DIR}/Endpoints.lib" + "${ASDK_LIB_DIR}/Equalizer.lib" + "${ASDK_LIB_DIR}/EqualizerImplementations.lib" + "${ASDK_LIB_DIR}/ExternalMediaPlayer.lib" + "${ASDK_LIB_DIR}/InteractionModel.lib" + "${ASDK_LIB_DIR}/KWD.lib" + "${ASDK_LIB_DIR}/MRM.lib" + "${ASDK_LIB_DIR}/Notifications.lib" + "${ASDK_LIB_DIR}/PlaybackController.lib" + "${ASDK_LIB_DIR}/RegistrationManager.lib" + "${ASDK_LIB_DIR}/SpeakerManager.lib" + "${ASDK_LIB_DIR}/SpeechEncoder.lib" + "${ASDK_LIB_DIR}/SpeechSynthesizer.lib" + "${ASDK_LIB_DIR}/SQLiteStorage.lib" + "${ASDK_LIB_DIR}/SystemSoundPlayer.lib") + +endif() include(build/BuildDefaults.cmake) include(tools/Testing.cmake) @@ -16,9 +61,13 @@ add_subdirectory("ThirdParty") add_subdirectory("SmartScreenCapabilityAgents") add_subdirectory("SmartScreenSDKInterfaces") add_subdirectory("ApplicationUtilities") -add_subdirectory("SampleApp") -add_subdirectory("doc") +if (NOT MSVC) + # For UWP SampleApp files are built as part of UWPSampleApp + add_subdirectory("SampleApp") + add_subdirectory("doc") +endif() add_subdirectory("Utils") +add_subdirectory("SSSDKCommon") # Create .pc pkg-config file include(build/cmake/GeneratePkgConfig.cmake) diff --git a/modules/Alexa/SSSDKCommon/CMakeLists.txt b/modules/Alexa/SSSDKCommon/CMakeLists.txt new file mode 100644 index 0000000..7d67e9b --- /dev/null +++ b/modules/Alexa/SSSDKCommon/CMakeLists.txt @@ -0,0 +1,6 @@ +cmake_minimum_required(VERSION 3.1) +project(SSSDKCommon LANGUAGES CXX) + +include(../build/BuildDefaults.cmake) + +add_subdirectory("src") \ No newline at end of file diff --git a/modules/Alexa/SSSDKCommon/include/SSSDKCommon/AudioFileUtil.h b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/AudioFileUtil.h new file mode 100644 index 0000000..cebbde1 --- /dev/null +++ b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/AudioFileUtil.h @@ -0,0 +1,35 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_AUDIOFILEUTIL_H_ +#define ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_AUDIOFILEUTIL_H_ + +#pragma once + +#include +#include + +namespace alexaSmartScreenSDK { +namespace sssdkCommon { + +class AudioFileUtil { +public: + static std::vector readAudioFromFile(const std::string& fileName, bool& errorOccurred); +}; + +} // namespace sssdkCommon +} // namespace alexaSmartScreenSDK + +#endif // ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_AUDIOFILEUTIL_H_ diff --git a/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMediaPlayer.h b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMediaPlayer.h new file mode 100644 index 0000000..fe37c01 --- /dev/null +++ b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMediaPlayer.h @@ -0,0 +1,82 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMEDIAPLAYER_H_ +#define ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMEDIAPLAYER_H_ + +#pragma once + +#include +#include +#include + +namespace alexaSmartScreenSDK { +namespace sssdkCommon { + +using namespace alexaClientSDK; +using SourceId = avsCommon::utils::mediaPlayer::MediaPlayerInterface::SourceId; + +class NullMediaPlayer + : public avsCommon::utils::mediaPlayer::MediaPlayerInterface + , public avsCommon::sdkInterfaces::audio::EqualizerInterface + , public avsCommon::utils::RequiresShutdown { +public: + NullMediaPlayer(); + + /// @name EqualizerInterface methods. + ///@{ + void setEqualizerBandLevels(avsCommon::sdkInterfaces::audio::EqualizerBandLevelMap bandLevelMap) override; + + int getMinimumBandLevel() override; + + int getMaximumBandLevel() override; + ///}@ + + /// @name MediaPlayerInterface methods. + ///@{ + SourceId setSource( + std::shared_ptr attachmentReader, + const avsCommon::utils::AudioFormat* format) override; + + SourceId setSource(const std::string& url, std::chrono::milliseconds offset, bool repeat) override; + + SourceId setSource(std::shared_ptr stream, bool repeat) override; + + bool play(SourceId id) override; + + bool stop(SourceId id) override; + + bool pause(SourceId id) override; + + bool resume(SourceId id) override; + + std::chrono::milliseconds getOffset(SourceId id) override; + + uint64_t getNumBytesBuffered() override; + + void setObserver( + std::shared_ptr playerObserver) override; + ///@} + +protected: + /// @name RequiresShutdown methods. + /// @{ + void doShutdown() override; + /// @} +}; +} // namespace sssdkCommon +} // namespace alexaSmartScreenSDK + +#endif // ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMEDIAPLAYER_H_ diff --git a/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMediaSpeaker.h b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMediaSpeaker.h new file mode 100644 index 0000000..90a5993 --- /dev/null +++ b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMediaSpeaker.h @@ -0,0 +1,41 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMEDIASPEAKER_H_ +#define ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMEDIASPEAKER_H_ + +#pragma once + +#include + +namespace alexaSmartScreenSDK { +namespace sssdkCommon { + +using namespace alexaClientSDK; + +class NullMediaSpeaker : public avsCommon::sdkInterfaces::SpeakerInterface { +public: + NullMediaSpeaker(); + /// @name SpeakerInterface methods. + ///@{ + bool setVolume(int8_t volume) override; + bool setMute(bool mute) override; + bool getSpeakerSettings(SpeakerSettings* settings) override; + ///@} +}; +} // namespace sssdkCommon +} // namespace alexaSmartScreenSDK + +#endif // ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMEDIASPEAKER_H_ \ No newline at end of file diff --git a/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMicrophone.h b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMicrophone.h new file mode 100644 index 0000000..0894207 --- /dev/null +++ b/modules/Alexa/SSSDKCommon/include/SSSDKCommon/NullMicrophone.h @@ -0,0 +1,44 @@ +/* + * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"). + * You may not use this file except in compliance with the License. + * A copy of the License is located at + * + * http://aws.amazon.com/apache2.0/ + * + * or in the "license" file accompanying this file. This file is distributed + * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either + * express or implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +#ifndef ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMICROPHONE_H_ +#define ALEXA_SMART_SCREEN_SDK_SSSDKCOMMON_INCLUDE_SSSDKCOMMON_NULLMICROPHONE_H_ + +#pragma once +#include