The Product overview provides conceptual information about the Digital Wave product. This book defines terminology and includes implementation scenarios.
Digital Wave is a high-performance, in-memory MPP (Massively Parallel Processing) database specifically designed for analytics. From business-critical data applications to advanced analytics, Digital Wave helps you analyze large volumes of data faster than ever before, helping you to accelerate your reporting, build intelligence applications, and to turn data into value. The system is designed to automate the pricing process, which includes the formation of recommendations based on price rules, competitor prices, forecasts and elasticity. The system is a comprehensive solution based on a microservice architecture that allows you to generate recommendations at regular prices with a speed close to online. The system is built on the basis of the ELT (Extract, Load, Transform) process, which includes:
- extracting data from external sources - extract;
- loading them into the data warehouse - load;
- transformation and cleanup to fit the needs of the business model - transform.
Digital Wave is a suite of SaaS applications for category managers, pricing analysts, trade marketing, and merchandising executives. With Dynamic and Rules-Based Pricing, Digital Wave automates your re-pricing and executes the pricing strategy. By giving real-time, intra-day, and online pricing recommendations at scale and speed, Digital Wave will help you maintain a competitive position, protect your margin, and adhere to product relationships and other pricing policies.
Dynamic pricing is a pricing strategy where fixed prices are replaced with variable prices. Typically, a business will set prices at a certain amount when going into a specific season. In contrast, dynamic pricing will involve multiple price changes within a day.
The system, developed by Digital Wave, predicts at what price items are most profitable to sell and boosts sales by analyzing demand elasticity. A low-code platform, it flexibly adjusts to users without long development cycles. To do this, the model focuses on the main demand drivers: seasonality, promotional activity, inventory levels, price elasticity of demand, and assortment rotation. The system considers the costs of store re-evaluation, more than 40 pricing rules, and competitor data.
The pricing module is used to calculate price recommendations based on price rules and established optimization principles. Price rules can be formed in a special interface from basic rules. The module accepts json files as input with the combined entity and additional parameters that are generated by the Retail microservice, such as price zones.
It analyzes the intersecting rules and breaks the data into independent blocks, on which the optimization module performs a parallel calculation. At the output, the optimization module returns csv files. The input and output files of the optimization module are stored in the /gp/retail/cache folder. The user interface shows the proposed price and optimization curves (if the latter has been added).
There are several types of price rules processed by the optimizer. Based on them, system administrators can create price rules that are available for users to add to scripts and create instances of price rules. The rules include a common set of settings for all rules called <header> and individual settings depending on the rule type. All types of price rules have a common set of parameters listed below: Common <header> parameters used in rules:
- grouper: defines independent product groups within which the rule will work.
- filter: Filters the data for the rule, including the filter results.
- filter_not: Filters the data for the rule, discarding the filter results.
- id: id of the rule.
- name: the name of the rule.
- weight: weight of the rule for the optimizer. In the event of a conflict between non-strict rules, a price will be chosen such that the sum of the products of weight times the price deviation from the allowable price in the rule will be minimal for all non-strict rules.
- strict: Flag for the strictness of the rule.
- text: The text of the rule that the user will see. Customizable options are marked with curly braces.
- number: The sequence number of the rule. If there is a conflict between strict rules, the rule with the lower sequence number will take precedence.
IMPORTANT!: If you enter {parameter name} in the text parameter, then the parameter becomes available for editing by the user in the interface.
For more information of about the pricing rules, please check Existing & Custom Pricing rules.
Strict optimization is another rule configured through the pricing module interface. You can assign weights to it and it can violate other rules (except strict ones). Convenient for setting various goals (revenue, number of sales, etc.) with setting weights for each goal.
Balanced optimization works in the same way as strict optimization, but it cannot violate other rules, giving price recommendations only in the price range that does not violate any other rule.
The interface of the pricing module is represented by the pages root page and several adjacent pages, which will be listed below. It serves for compiling price rules, generating reports on various metrics, optimizing prices and compiling price recommendations.
For more details, please check the UI Guide.
The forecasting module is used to predict various indicators: elasticity, demand (including seasonal demand), whether it is necessary to buy additional goods. The output of the module can be used either for direct evaluation by the client or in an optimization process in the pricing module. The module accepts historical data and a set of hyperparameters as input.
The module launches a virtual machine in the cloud, copies the combined files to it using rsync, then cuts them into categories, while using the last mapping of the category products. Then the module gets the last attribute, starts modeling, segmentation and kvi. Each process creates files that are copied back to the server using rsync. The model trims the data at the end of the week, so the module runs once a week.
Elasticity is calculated inside the optimization module at different levels: product, product line, brand, class, etc. This can be configured in the configuration file in /retail The model automatically calculates the validity of each of these levels and combines them with the assigned weights for each level. The weights are entered manually or by a special hyperparameter selection module.
Custom hyperparameters:
- level of elasticity calculation - elasticity calculation level
- weights of each level - weight of each level
- elast_min, elast_max - elasticity value limits
- reest_horizon - number of days to reestimate simulation parameters to capture local trends
- reest_kernel - form of reestimation of modeling parameters: "triangle" (linear growth of the contribution of information from days, the closer to the present time, the stronger the influence) or "threshold" (the same contribution from all days). The angle of the right triangle is calculated automatically.
- min_basedemand - restrictions on the minimum demand, so as not to spoil the statistics with data when the product was not bought.
This submodule calculates seasonal curves and holidays, detecting the latter automatically based on anomalies in the statistics. Floating holidays, such as Easter, are also taken into account if their deviation is within four weeks.
List of configurable hyperparameters:
- season_rolling_window - size of the rolling window in days
- covid - special periods that are excluded from statistics
- covidRightDays, covidLeftDays - size of smoothing barriers on the left and right in days
- fine tune holidays detection parameters - parameters for calculating special holidays:
- special_events_rolling_window, special_events_grubbs_alpha
Configuration example:
"CATEGORY_NAME": {
"season": {
"season_rolling_window": 63
"covid": [("2020-02-23", "2020-06-01")]
"special_events_rolling_window": 21,
"special_events_grubbs_alpha": 2,
"covidLeftDays": 15,
"covidRightDays": 120,
}
}
The hyperparameter fitting module uses the engine tuning technology described below to fit or adjust a set of hyperparameters to calculate a category prediction.
The following input data are required for the selection of hyperparameters:
- Category for which the selection is made
- Dates on which the simulation is run
- Hyperparameters and their possible values
Due to the large volume of products in the category, modeling is performed on truncated data: a randomly selected 5% of products. Engine tuning can be run in serial or parallel mode.
In sequential mode, the system runs the simulation in turn on the selected hyperparameters, taking into account the quality of optimization from previous runs. This makes it possible to find the optimal hyperparameter values in fewer iterations, but takes more time. In parallel mode, the system starts the simulation on the entire range of hyperparameters at once, choosing the optimal one. It is faster, but requires more computing power.
After finding the optimum, an additional check is carried out: modeling is carried out on the entire category, if as a result of this the quality of the forecast has improved, then a new set of hyperparameters is written into the system.
The unified view combined combines information about the cost of goods, sales, stocks, promotional activity in a historical perspective. It is one of the main sources of data in application, modeling and price optimization.
When new transactional data arrives, the platform can automatically start a recalculation for the last seven (by default) days to take into account “lagging” data - transaction trailing. The number of days for recalculation can be configured separately. At the stage of forming the combined view, complex logics can be applied, for example, by calculating the cost of goods.
The service allows you to upload files via the web user interface to the server. It supports uploading files with xlsx, csv extension. It is possible to upload files in bulk, for this to move the selected files to the Drag&Drop area at the same time.
The file placed in the Drag&Drop area is uploaded using the POST method to the microservice inside the Retail module, which uploads the file to the system server into /data/lenta/retail directory.
This module is responsible for exporting data from the system. By default, it can be found at https:///pages/new_price_count/. The following options are available:
- Export to file - export data in txt format. The file name has the following mask: _<today's date in YYYYMMDD format>.txt
- Export to SFTP - export data to the sftp server.
A script is launched that generates a POST request /export_prices to the Retail microservice. This request to Retail is open and can be executed from the outside, by a script, according to a schedule, and so on. The parameters of the name of the uploaded file, its attribute composition and the SFTP server are written inside the Retail microservice.
The system assumes several types of data loading. Loading data into the system is implemented through file import The following import methods are provided:
- Loading from the integration area on the server
- Drag&drop in the front module of the system
- upload via API
- upload via ODBC and kafka
The system stores all initial data, including historical and irrelevant cuts, to save the possibility of recalculating the results when changing calculation algorithms or models.
IMPORTANT!:For the basic functionality (reports, scripts, config2) to work, all data must be linked to combined
For uploaded files, there is an integration area on the server (directory ../inbox). It contains all integration files and directories.
IMPORTANT!: When an uploaded file is deleted from the integration area, the data from this file will be deleted from the system.
- The system supports any type of file that can be converted to a column format (csv, xls, xlsx, xlsm, etc.) csv files must comply with the rfc4180 standard and be encoded in UTF-8.
- In decimal fractions, the separator must be a period.
- Dates must be in YYYY-MM-DD format
- Excel files should not have merged cells.
- The Excel file you upload must have only one table per sheet.
- When downloading large amounts of data, you can use GZIP to compress the downloaded file.
- When loading a new version of data for an already existing report, the location of the attributes must match the previously loaded files.
To implement the price recommendation process in the system, you need to take the following steps:
- to process and transform data to present them in the structure necessary for the system
- upload data to the system
- customize user interfaces
- set up the main entity for the pricing interface
- customize pricing module interface
- set up price rules
Think about additional features (May be creating pages and some other objects)
Users in the system can be created both locally and imported through external authorization systems. The system supports integration with MIRACL, Active Directory. Passwords and roles of external users are managed in an external system.
- Config
- Streams
- Reports
- Sources
- Solutions
- Users
- Flow
Basic interface - a web interface for users to work with the application. It is a set of pre-configured pages with static addresses and user-created pages. The root page for custom pages is the /pages page. List of preconfigured pages:
- pages
- pages/config2
- admin
- users
- graphiQL
User-created pages are created blank by default. If, after creation, the content of the page is not edited, then it is not saved in the system.
Reports are entities that exist on top of the data entered into the system and serve for aggregation, unification, and application of other business logics. They can use each other.
You can use the Graphiql console to view the report result. Reports are linked to streams, so they can also be accessed (via streams in the frontend).
IMPORTANT!: When switching to another tab, the system does not save the manual changes inside the admin panel. To save the changes, click Submit changes.
- Log in using https://<instance name>/admin.
- Go to the Reports tab.
- Press Create report.
- Enter the name of the report in the input window.
| Item | Color | Gov ruler | Brand | Purchase price | Selling price |
|---|---|---|---|---|---|
| Saucepan | Red | Color | A | 50 | 100 |
| Saucepan | Blue | Color | A | 50 | 120 |
| Saucepan | Green | Color | B | 50 | 140 |
| Saucepan | White | Color | B | 50 | 160 |
Let's set up for these products the rules of the horizontal line (SamePrice), brand price ratios (Relation), and margins (PctChange). A new rule is created in the page editor (design mode) on the Settings tab (Elements)
What is test server IP?
For more details, please check:
- Product Requirement documentation
- UI style guide & product map
- Software architecture design
- API documentation
- FAQ
- Our Support Portal
For information about legal notices, trademarks, disclaimers, warranties, export and other use restrictions, patent policy, and FIPS compliance, see Digital Wave - Legal Page.