Querying the state does not work in Azure Containers App

[sergiibielskyi]

Hi team,
I am trying to find the solution to how to get results from state storage by using a specific query, not just getting item by id. I found a good explanation here https://docs.dapr.io/developing-applications/building-blocks/state-management/howto-state-query-api/. And I prepared method to use query state.

CancellationTokenSource source = new CancellationTokenSource();
CancellationToken cancellationToken = source.Token;

var query = "{" +
        "\"filter\": {" +
            "\"EQ\": { \"value.style\": \""+ style +"\" }" +
        "}" +
    "}";

var queryResponse = await daprClient.QueryStateAsync(storeName, query, cancellationToken: cancellationToken);

Locally, it works. But when I try to publish that into the Azure Containers App I see 500 error.

I assume, probably Azure Containers App service doesn't have the right Dapr runtime but I can not prove it.
Any ideas?

App logs info below,
Dapr.DaprException: Query state operation failed: the Dapr endpointed indicated a failure.
See InnerException for details. ---> Grpc.Core.RpcException: Status(StatusCode="Unknown",
Detail="cannot proxy request: missing dapr-app-id metadata")
at Dapr.Client.DaprClientGrpc.QueryStateAsync[TValue](String storeName, String jsonQuery,
IReadOnlyDictionary2 metadata, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Dapr.Client.DaprClientGrpc.QueryStateAsync[TValue](String storeName, String jsonQuery, IReadOnlyDictionary2 metadata, CancellationToken cancellationToken)
at mart.cosmosdb.ObjectController.GetObjectsByQuery(String style)
in /cosmosdb/mart-api-cosmos-db/Controllers/ObjectController.cs:line 38
at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.
TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper,
ObjectMethodExecutor executor, Object controller, Object[] arguments)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.
g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask1 actionResultValueTask) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker. g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker) at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication1 application)

[sergiibielskyi]

Also, I just added metadata but still have an error.
`CancellationTokenSource source = new CancellationTokenSource();
CancellationToken cancellationToken = source.Token;
var metadata = new Dictionary<string, string>
{
{ "dapr-app-id", "cosmosdbapp" }
};

    var query = "{" +
            "\"filter\": {" +
                "\"EQ\": { \"value.style\": \""+ style +"\" }" +
            "}" +
        "}";
    
    var queryResponse =  await daprClient.QueryStateAsync<dynamic>(storeName, query, metadata, cancellationToken: cancellationToken);`

[ahmelsayed]

@sergiibielskyi can you share what your configuration section in the ContainerApp resource looks like as well as what your dapr component looks like.

[sergiibielskyi]

@sergiibielskyi can you share what your configuration section in the ContainerApp resource looks like as well as what your dapr component looks like.

Sure,
resource cosmosDBComponent 'Microsoft.App/managedEnvironments/daprComponents@2022-01-01-preview' = {
name: '${environmentName}/cosmosdb'
properties: {
componentType: 'state.azure.cosmosdb'
version: 'v1'
secrets: [
{
name: 'cosmos-key'
value: cosmosKey
}
]
metadata: [
{
name: 'url'
value: cosmosdbUrl
}
{
name: 'database'
value: cosmosDBName
}
{
name: 'collection'
value: collectionName
}
{
name: 'masterkey'
secretRef: 'cosmos-key'
}
]
scopes: [
cosmosdbapp.name
]
}
}

resource cosmosdbapp 'Microsoft.App/containerapps@2022-01-01-preview' = {
name: 'cosmosdbapp'
location: location
properties: {
managedEnvironmentId: environmentId
configuration: {
ingress: {
external: false
targetPort: 80
}
registries: [
{
server: containerRegistry
username: containerRegistryUsername
passwordSecretRef: 'registry-key'
}
]
secrets: [
{
name: 'registry-key'
value: registrySecretRefName
}
]
dapr: {
enabled: true
appPort: 80
appId: 'cosmosdbapp'
}
}
template: {
containers: [
{
image: imagePath
name: 'cosmosdbapi'
resources: {
cpu: '0.5'
memory: '1Gi'
}
}
]
scale: {
minReplicas: 1
maxReplicas: 1
}
}
}
}

[sergiibielskyi]

Also, tested by adding apiVersion: 'dapr.io/v1alpha1' to properties but it was nothing changed

[sergiibielskyi]

Hey @ahmelsayed, any updates from your side? What I did wrong?

[tjoudeh]

The problem is still there even though the Container Apps is using dapr v1.7.3.

I can't query Cosmos DB store when deploying to Azure container apps, while query cosmos db works fine when I self-host dapr on my machine with no issues. Other operations such as GET/PUT/Delete/Post are working fine in container apps with cosmos state store.

This is the exception I'm receiving

Dapr.DaprException: Query state operation failed: the Dapr endpointed indicated a failure. See InnerException for details.
 ---> Grpc.Core.RpcException: Status(StatusCode="Unimplemented", Detail="Not Implemented")
 at Dapr.Client.DaprClientGrpc.QueryStateAsync[TValue](String storeName, String jsonQuery, IReadOnlyDictionary`2 metadata, CancellationToken cancellationToken)
 --- End of inner exception stack trace ---

My query is very simple, like the below

            var query = "{" +
                   "\"filter\": {" +
                       "\"EQ\": { \"taskCreatedBy\": \"" + createdBy + "\" }" +
                   "}}";

            var queryResponse = await _daprClient.QueryStateAsync<TaskModel>(STORE_NAME, query);

            var tasksList = queryResponse.Results.Select(q => q.Data).OrderByDescending(o=>o.TaskCreatedOn);

            return tasksList.ToList();

Here is my state store yaml used for azure container apps deployment

componentType: state.azure.cosmosdb
version: v1
metadata:
- name: url
  value: https://<url>.documents.azure.com:443/
- name: masterkey
  secretRef: cosmoskey
- name: database
  value: tasksmanagerdb
- name: collection
  value: taskscollection
secrets:
- name: cosmoskey
  value: <value>
scopes:
- tasksmanager-backend-api

Here is my state store yaml used locally to test dapr on my machine

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: statestore
spec:
  type: state.azure.cosmosdb
  version: v1
  metadata:
  - name: url
    value: https://<url>.documents.azure.com:443/
  - name: masterKey
    value: <value>
  - name: database
    value: tasksmanagerdb
  - name: collection
    value: taskscollection
scopes:
- tasksmanager-backend-api   

FYI @ahmelsayed @torosent

[torosent]

@ItalyPaleAle Can you help with this issue?

[ItalyPaleAle]

@tjoudeh Thanks for the report.

Can you confirm if while running in Container Apps you can connect to Cosmos DB? I understand that querying doesn't work, but are you able to perform operations to get or set a state?

[tjoudeh]

Thanks @ItalyPaleAle for reaching out, as I stated in my comment, all other operations such as GET by id, POST, DELETE, etc.. are working fine with no issues on container apps. Only query the collection is throwing this exception. Yet when running the app self hosted on my machine the query works fine with no issues, so I believe it's something related to container apps hosting. I'm using the latest .net core dapr sdks too.

[ItalyPaleAle]

Thanks for confirming @tjoudeh . As you confirm that, I believe we have identified the root cause of the issue and are working on a fix. The issue in upstream Dapr is here: dapr/dapr#5008 I am working with @torosent and his team to understand how we can ship this to CApps as soon as feasible.

[tjoudeh]

Perfect, thank you so much, happy to early test this of there is a preview version which will be shipped soon, please consider looking into the order by part too as its not working on self hosted dapr application and most probably on CApps too 😀

[ItalyPaleAle]

Can you please open an issue on for that in http://github.com/dapr/components-contrib and we'll look into it :)

[tjoudeh]

Sure thing, will do on Sunday 👊

[akshat172]

the issue is still there

@ItalyPaleAle any updates or workarounds

[ItalyPaleAle]

@akshat172 Yes, the rollout of the fix hasn't been completed yet. It may take another week or two.

[akshat172]

@akshat172 Yes, the rollout of the fix hasn't been completed yet. It may take another week or two.

Ok Great thanks for the update. Will wait for the fix to be released.

[akshat172]

Hi @ruslany @ItalyPaleAle , Has this issue been resolved.

[ItalyPaleAle]

I believe the updated version of Dapr should be available in all Azure regions now. If it’s still not working for you, can you please let us know?

[akshat172]

I believe the updated version of Dapr should be available in all Azure regions now. If it’s still not working for you, can you please let us know?

@ItalyPaleAle I deployed my code this morning, I was still getting that gRPC unimplemented error.

[akshat172]

I redeployed my code and still getting this error

This is how I am calling the Dapr Query Async function

and the component looks like

Location - Australia East

[ItalyPaleAle]

You are right, I just checked and the fix hasn’t rolled out yet. It will be in Dapr version 1.8.4-msft-2 (you’re still running 1.8.4-msft-1). Checking with the release team what the status is. Sorry about that

[akshat172]

Hi, @ItalyPaleAle PLease let me know once it has been rolled out.
Do I need to make any changes to my CA environment or in my code when fix is released.

[ItalyPaleAle]

Do I need to make any changes to my CA environment or in my code when fix is released.

No, it should “just work” once it’s been rolled out

[tjoudeh]

Hello @ItalyPaleAle Has this been rolled out now? Thank you.

[tjoudeh]

I tried it out right now on East US, and I can confirm that query API is working as expected on ACA with the latest dapr update. thanks @ItalyPaleAle & @torosent

[sergiibielskyi]

Hey, happy to hear that it should work. But after testing I see response 200 but without content in response. But if we are talking about executing the same piece of code locally, I do not see such a problem. What will be and how to trace that? My example is following below,
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken cancellationToken = source.Token;

    var query = "{" +
            "\"filter\": {" +
                "\"EQ\": { \"value.style\": \""+ style +"\" }" +
            "}" +
        "}";
    
    var queryResponse =  await daprClient.QueryStateAsync<ObjectModel>(storeName, query, cancellationToken: cancellationToken);

[mehmetaltuntas]

Hi, do we know this has been fixed entirely in all Azure regions? thanks

[ItalyPaleAle]

Hi, do we know this has been fixed entirely in all Azure regions? thanks

Yes should be fixed in all regions