Skip to content

Improve user experience for streaming large objects (delta encoding) #475

@brokenhappy

Description

@brokenhappy

Is your feature request related to a problem? Please describe.
A common use case for users is to send some state using RPC as a Flow<MyState>

@Rpc interface MyRpcService { fun state(): Flow<MyState> }

Now the problem might be that the MyState is a large object, and the overhead of serializing and deserializing the entire object each time can be high, even though the real changes are small.

Now what a user would need to do in this case is commonly referred to as delta encoding.
This would mean that the user creates some representation of a change instead of the entire object.
One such example is:

data class MyState(val lotsOfData: List<Data>, val smallData: String)

// instead of: @Rpc interface MyRpcService { fun state(): Flow<MyState> }
// One would:
@Rpc interface MyRpcService {
    /** You probably want to use [MyRpcService.state] instead */
    suspend fun getInitialState(): MyState
    /** You probably want to use [MyRpcService.state] instead */
    fun stateDeltas(): Flow<MyStateChange>
}

sealed class MyStateChange {
    data class SmallDataChanged(val newSmallData: String) : MyStateChange()
    // ...
}

fun MyState.after(change: MyStateChange): MyState = when (change) {
    is SmallDataChanged -> this.copy(smallData = change.newSmallData)
    // ...
}

fun MyRpcService.state(): Flow<MyState> {
    var state = getInitialState()
    emit(state)
    stateDeltas().collect {
        state = state.after(change)
        emit(state)
    }
}

Now the real problem is that this is a lot of code to write with a lot of boilerplate

Describe the solution you'd like
I would love for kotlinx-rpc to help with this. Imagine something along the lines of:

@Rpc interface MyRpcService { @DeltaEncoded fun state(): Flow<MyState> }

There's a lot of questions that arise. But I don't care about the actual implementation/exact UX. I just would love not to have to write this boilerplate myself.

Describe alternatives you've considered
I am quite confident that there was a related KT issue related to this. But I couldn't find it :(

Additional context

Metadata

Metadata

Assignees

Labels

featureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions