Azure Cosmos Emulator docker image is broken

[soenneker]

The Azure Cosmos DB emulator docker image is broken

Hi Azure Cosmos team,
Hopefully this can get some visibility and reach the people who can fix this.

I filed the first bug on this almost two years ago:
microsoft/azure-pipelines-tasks#15865

To boil it down: on Ubuntu 20 and later, agents that have an Intel processor fail to start with the error: This is an evaluation version. There are [165] days left in the evaluation period. Ubuntu 18 worked, but those images are no longer available on the major platforms. Since an action may or may not receive an agent with an Intel processor, the error doesn't occur 100% of the time.

This issue is currently blocking anyone who wants to use the emulator in a Linux action.

I know that the Windows runners have the emulator built into them, but one issue is containers on Windows aren't able to be run within GitHub. I know that we can run a custom agent, but that shouldn't be necessary.

Please, let me know how I can help!

Here are many, many issues talking about this problem, spanning all types of projects, stackoverflow, etc... This isn't an exhaustive list, there are so many more:

#85
#84
#81
#77
#66
#64
#60
#45
#87
#56

https://github.com/MicrosoftDocs/visualstudio-docs/issues/9970
Azure/azure-cosmos-dotnet-v3#3010
actions/runner-images#5036
actions/runner-images#6111

https://learn.microsoft.com/en-us/answers/questions/1089628/azure-cosmos-emulator-the-evaluation-period-has-ex
https://stackoverflow.com/questions/68187668/cosmos-db-emulator-for-linux-docker-this-is-an-evaluation-version-there-are-1
https://stackoverflow.com/questions/50333215/cosmos-db-emulator
https://stackoverflow.com/questions/74440386/azure-cosmos-db-emulator-linux-image-does-not-start-error-the-evaluation-perio
https://stackoverflow.com/questions/76282676/health-check-failing-for-cosmos-db-emulator
https://learn.microsoft.com/en-us/answers/questions/1088903/unable-to-start-cosmosdb-emulator-for-linux-docker
https://learn.microsoft.com/en-us/answers/questions/1393449/while-installing-cosmosdb-emulator-locally-with-do

Tagging to increase visibility:
@ealsur @sajeetharan @TheovanKraay @markjbrown @jaydestro @suvishodcitus @satyanadella

[kirankumarkolli]

@sajeetharan, @v1k1 can you please look into it?
Also please feel free to move the issue to the right appropriate target repo/place.

[sajeetharan]

@soenneker

Thank you for highlighting the issues and we apologize for the inconvenience. The Linux emulator has been in an extended preview, but we've recently addressed crucial bugs, including support for Cosmos DB for Mongo API, in our 2023 November release.

Regarding the Ubuntu 20 support, We are looking forward to upgrading the version and will update this forum soon on the dates and our progress to all the valuable asks.

We acknowledge the importance of a seamless developer experience and are actively working to improve. Enhancing the emulator is a top priority for us as we strive to deliver a superior developer-first database experience. Your feedback is invaluable, and we are committed to meeting and exceeding expectations.

[soenneker]

Thanks @sajeetharan

Please put a priority on this as a team; we're doing all sorts of non-optimal solutions to get our tests going with Cosmos, and it's souring our perception of Cosmos.

If you could let us know an estimated date when you'll be able to address this that would be greatly appreciated.

[samuel-kogler-AP]

Just wanted to add that we are dealing with so many issues here as well, the amount of time and money that has been burned due to issues with this emulator is starting to make us look for alternatives to CosmosDB, just because the CI / CD situation is not really solvable with CosmosDB.

[pacorreia]

Want to add on top this issue, the startup script for docker container really needs a re-work, I had to take it, place my fizes and improvements to make the self-signed certificate actually match the exact desired hostname, rather than ips or localhost.
Also having kind of a luancher/wrapper to launch a windows binay does not seem the right way to do it.

Logging is also missing, we can't see what's going on if it can't start properly.

I do hope that when taking the next fixes, the team can stop, re-think it as if they were actually using it like us and provide the flexibility we need when working locally.

I'm happy to provide further details to Product Group if desired.

[zacutti]

Same here, is frustating trying to engage the team to use Cosmos Db.
Im getting this error, the emulator sometimes start but its unusable.

This is an evaluation version. There are [83] days left in the evaluation period.
Starting
Started 1/11 partitions
Started 2/11 partitions
Started 3/11 partitions
Started 4/11 partitions
Started 5/11 partitions
Started 6/11 partitions
Started 7/11 partitions
Started 8/11 partitions
Started 9/11 partitions
Started 10/11 partitions
Started 11/11 partitions
Started
This program has encountered a fatal error and cannot continue running at Mon Feb 5 21:21:45 2024
The following diagnostic information is available:

     Reason: OS Fatal Error (0x00000006)
    Message: Last Chance
    Address: 0x3fff84111082
 Parameters: 0x3fff8855cf0b
Stack Trace:
             file://package1/windows/system32/sqlpal.dll+0x0000000000298F1C
             file://package1/windows/system32/sqlpal.dll+0x0000000000296EC1
             file://package1/windows/system32/sqlpal.dll+0x0000000000266430
             file://package1/windows/system32/sqlpal.dll+0x00000000002660ED
             file://package1/windows/system32/sqlpal.dll+0x00000000003944F3
             file:///windows/System32/Drivers/netio.sys+0x0000000000001082
             file:///windows/System32/Drivers/Http.sys+0x0000000000069526
             file:///windows/System32/Drivers/Http.sys+0x000000000006AAA4
             file:///windows/System32/Drivers/Http.sys+0x0000000000004B79
             file:///windows/System32/Drivers/Http.sys+0x00000000000A3E27
             file://package1/windows/system32/sqlpal.dll+0x000000000024A57C
             file://package1/windows/system32/sqlpal.dll+0x0000000000394491
             <unknown>+0x00000000B1D45C00
    Process: 20 - cosmosdb-emulator
     Thread: 274 (application thread 0x114)
Instance Id: 063d91f5-bdcf-488b-a9bb-884fd9ece736
   Crash Id: 00d7a76c-b2bb-40df-8213-addeb5e3a9b9
Build stamp: (null)

Distribution: Ubuntu 20.04.6 LTS
Processors: 8
Total Memory: 16707002368 bytes
Timestamp: Mon Feb 5 21:21:45 2024
Last errno: -34938881
Last errno text: Unknown error -34938881
*********** PAL PANIC CORE DUMP GENERATION FAILED **********
Unable to locate handle-crash.sh. Error: File: signals.cpp:483 [Status: 0xC0000034 Object name not found errno = 0x2(2) No such file or directory]
Executing: /usr/local/bin/cosmos/handle-crash.sh with parameters
handle-crash.sh
/usr/local/bin/cosmos/cosmosdb-emulator
20
/usr/local/bin/cosmos
/tmp/cosmos/appdata/log/

 063d91f5-bdcf-488b-a9bb-884fd9ece736
 00d7a76c-b2bb-40df-8213-addeb5e3a9b9

[soenneker]

@sajeetharan Any update here? It's been ~3 weeks. Thanks

[soenneker]

@sajeetharan Checking in on any updates, thank you

[soenneker]

@sajeetharan An update would be appreciated; we're almost at the 3 month mark here with radio silence from Microsoft. Just giving us a ballpark would be helpful...

Tagging again hopefully this can get some attention
@ealsur @TheovanKraay @markjbrown @jaydestro @suvishodcitus @satyanadella

[adamzest]

@sajeetharan An update would be appreciated; we're almost at the 3 month mark here with radio silence from Microsoft. Just giving us a ballpark would be helpful...

Tagging again hopefully this can get some attention @ealsur @TheovanKraay @markjbrown @jaydestro @suvishodcitus @satyanadella

@sajeetharan any news please? Another month with our local test deployment crippled.

[docHologram]

We are unable to run mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest because once again, the evaluation period has expired. Why is there an evaluation period at all?

[kntajus]

We are unable to run mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest because once again, the evaluation period has expired. Why is there an evaluation period at all?

A new version was released about 3 weeks ago, make sure you've pulled down the latest image.

[pacorreia]

TBF this doesn't make sense anymore, to have an evaluation period.

The image already limits the RU/s, no one would use as live systemgiven it's limitations, better to remove that obsolete evaluation period and spare both sides with annoyances.