A set of tools for finite-state Markov processes (MP) state estimation and control with applications to data transmission (TCP) performance evaluation and control.
Algorithms for MP optimal (suboptimal) state estimation and control, all theory may be found in [1-2].
Classes for the controllable Markov chain, counting process with controllable intensity, controllable continuous process.
Model with state dynamics given by a controllable Markov process and indirect observations given by a counting process with intensity, which depends on the state of the observable MP. Suboptimal filter for the MP state estimation as described in [1]].
- Model with additional continuous observations given by Wiener process with drift and variance dependant on the state of the MP.
- Model with additional continuous observations and possibility of simultaneous jumps of the controlled MP and the counting process observations. Optimal filter from [3]
A model for an Unmanned aerial vehicle (UAV) transmitting data through the set of base stations (BS) from [2].
The transmitter chooses how to allocate the traffic, by choosing the intensity of the data flow to each transmission channel. The transmission channel to each BS is described by a controllable MP, the transitions intensity depends on the distance between the transmitter and the BS (the farther, the worse states are more possible). The transmitter estimates the channel states having only the information about the packets lost during the transmission (TCP-like protocol). These observations are given by the controllable counting process with intensity, which depends on the state of the channel (the worse the channel state is, the more packets are being lost). Different control policies are compared with the one based on the suboptimal channel state estimation.
TCP Illonoise channel model with states given by a controllable MP, ack flow approximated by continuous Wiener process and losses/timeouts represented by two counting processes [3]. The key feature is that the losses/timeouts may occur at the same time as the MP transitions.
[1] Miller B., Miller G., Siemenikhin K. Optimal control problem regularization for the Markov process with finite number of states and constraints // Automation and Remote Control, 2016, vol. 77, pp. 1589-1611. DOI: 10.1134/S0005117916090071. Researchgate
[2] Miller B., Miller G., Siemenikhin K. Optimization of the Data Transmission Flow from Moving Object to Nonhomogeneous Network of Base Stations // In Preprints of the 20th World Congress The International Federation of Automatic Control, July 2017, Toulouse, France, IFAC-PapersOnLine, vol 50(1), pp. 6160-6165. DOI: 10.1016/j.ifacol.2017.08.981. Researchgate
[3] A. Borisov, G. Miller, A. Stefanovich, Controllable Markov Jump Processes. II. Monitoring and Optimization of TCP Connections // Journal of Computer and Systems Sciences International. January 2019. 58(1):12-28 DOI: 10.1134/S1064230719010040. Researchgate
[4] A. Borisov, A. Bosov, G. Miller, I. Sokolov, Partial Diffusion Markov Model of Heterogeneous TCP Link: Optimization with Incomplete Information // Mathematics 2021, 9(14), 1632, 2021. DOI: 10.3390/math9141632