Error in initialization of client with macOS

[ChingEn-Lin]

ValueError: invalid literal for int() with base 10: b'2022-04-21T17:34:12.398+0200 I CONTROL [js] machdep.cpu.extfeatures unavailable\n0\n'
when trying to instantiate client with client = MongoClient()

To Reproduce

from pymongo_inmemory import MongoClient
client = MongoClient()

gives the error
ValueError: invalid literal for int() with base 10: b'2022-04-21T17:34:12.398+0200 I CONTROL [js] machdep.cpu.extfeatures unavailable\n0\n'

Logs

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "./lib/python3.7/site-packages/pymongo_inmemory/_pim.py", line 9, in __init__
    self._mongod.start()
  File "./python3.7/site-packages/pymongo_inmemory/mongod.py", line 101, in start
    while not self.is_healthy:./python3.7/site-packages/pymongo_inmemory/mongod.py", line 140, in is_healthy
    if int(uptime) > 0:
ValueError: invalid literal for int() with base 10: b'2022-04-21T17:34:12.398+0200 I CONTROL  [js] machdep.cpu.extfeatures unavailable\n0\n'

Context:

• macOS Monterey with M1 chip
• pytest-6.2.2
• mongo4.2

[ekarademir]

Hi @ChingEn-Lin I'm going to look into this.

[ekarademir]

Hi @ChingEn-Lin could you please post the output of

python -c "import platform; print(platform.uname())"

I don't have an M1 machine, so I need your help debugging this. The version of MongoDB is determined by the output of a similar Python function. I wonder if the output has changed for M1 chips.

[ChingEn-Lin]

Dear Ertugrul, Thank you for looking into this. The output is as follow:

>> import platform; print(platform.uname())

uname_result(system='Darwin', node=‘xxxx-MacBook-Pro.local', release='21.4.0', version='Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000', machine='x86_64', processor='i386’) Regards, Lin Ertugrul Karademir ***@***.******@***.***>> 於 2022年5月1日 下午1:08 寫道： python -c "import platform; print(platform.uname())"

[ekarademir]

Hi @ChingEn-Lin

Thank you for the output. I think it's not about selecting the version, but how the code decides if MongoDB server is running or not. So the debugging will be harder. Below steps is basically what pymongo_inmemory does. The aim is to see if the commands it's using are producing the output that it's expecting. When you have time could you do the following please:

• Download the MongoDB server directly, and untar it to a suitable place. You can use this link: https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-4.2.18.tgz or a suitable one from https://www.mongodb.com/download-center/community/releases/archive
• Once you untar it, open up a terminal to the bin folder.
• See that mongod executable is there.
• Open up another terminal on the same path
• See that mongo executable is there.
• On one terminal start mongod with

mkdir data
./mongod --storageEngine ephemeralForTest --dbpath ./data

• Once the server is alive, then on the other terminal start the REPL

./mongo

• Once REPL starts, on the REPL terminal, could you run this command and send on the output?

db.serverStatus().uptime

• Once you get the output, you can exit the REPL with exit and you can stop the server with Ctrl + C.

[ChingEn-Lin]

Dear Ertugrul, I followed the instructions, The output is as follow

db.serverStatus().uptime

68 I believe this is sth that is expected? Regards, Lin Ertugrul Karademir ***@***.******@***.***>> 於 2022年5月2日 上午9:10 寫道： Hi @ChingEn-Lin<https://github.com/ChingEn-Lin> Thank you for the output. I think it's not about selecting the version, but how the code decides if MongoDB server is running or not. So the debugging will be harder. Below steps is basically what pymongo_inmemory does. The aim is to see if the commands it's using are producing the output that it's expecting. When you have time could you do the following please: * Download the MongoDB server directly, and untar it to a suitable place. You can use this link: https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-4.2.18.tgz or a suitable one from https://www.mongodb.com/download-center/community/releases/archive * Once you untar it, open up a terminal to the bin folder. * See that mongod executable is there. * Open up another terminal on the same path * See that mongo executable is there. * On one terminal start mongod with mkdir data ./mongod --storageEngine ephemeralForTest --dbpath ./data * Once the server is alive, then on the other terminal start the REPL ./mongo * Once REPL starts, on the REPL terminal, could you run this command and send on the output? db.serverStatus().uptime * Once you get the output, you can exit the REPL with exit and you can stop the server with Ctrl + C. — Reply to this email directly, view it on GitHub<#56 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARWAPJ6KWXW25GLJQP3Y7KDVH55VZANCNFSM5T7UY2QA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ekarademir]

Indeed, that is the expected output. However this might be related to your particular case:
https://jira.mongodb.org/browse/SERVER-56416

Apparently MongoDB server outputs an extra line of logging with CPU features summary.

It looks like it has been fixed in 4.4.10 and 5.0.4, I'm guessing also in other minor versions. I'm going to publish a new version of pymongo_inmemory which will include option to download these versions. Could you try again after the next release?

If you are tied to this version, I will try to blindly create a patch and I might need your help testing it.

[ChingEn-Lin]

Yes I’ll be gladly to try again after the next release, as well as to help test it. Ertugrul Karademir ***@***.******@***.***>> 於 2022年5月3日 上午9:31 寫道： Indeed, that is the expected output. However this might be related to your particular case: https://jira.mongodb.org/browse/SERVER-56416 Apparently MongoDB server outputs an extra line of logging with CPU features summary. It looks like it has been fixed in 4.4.10 and 5.0.4, I'm guessing also in other minor versions. I'm going to publish a new version of pymongo_inmemory which will include option to download these versions. Could you try again after the next release? If you are tied to this version, I will try to blindly create a patch and I might need your help testing it. — Reply to this email directly, view it on GitHub<#56 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARWAPJ2HNBQGDTHUQGPWVPDVIDI3XANCNFSM5T7UY2QA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ekarademir]

I pushed a new release few ours ago. Could you try if the problem is resolved with 4.4 or 5.0?

[ChingEn-Lin]

I am using MongoDB shell version v5.0.7 and pymongo-inmemory version: 0.2.7, still get the same

 Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "./lib/python3.7/site-packages/pymongo_inmemory/_pim.py", line 9, in __init__
    self._mongod.start()
  File "./lib/python3.7/site-packages/pymongo_inmemory/mongod.py", line 112, in start
    while not self.is_healthy:
  File "./lib/python3.7/site-packages/pymongo_inmemory/mongod.py", line 154, in is_healthy
    if int(uptime) > 0:
ValueError: invalid literal for int() with base 10: b'2022-05-03T15:25:41.744+0200 I CONTROL  [js] machdep.cpu.extfeatures unavailable\n0\n'

for the code

[ekarademir]

Ok, thanks for trying out. I'll try creating a patch on a branch then I might ask you to test it. It will take me few days :)

[ekarademir]

Hi @ChingEn-Lin ,

As mentioned above, I created a fix for this issue. However now we need to see if it solves your issue.

Could you do the following please?

1. Reproduce the issue, or, make sure the issue is still there. For this you'd need at least poetry installed on the system. Or, you can install the only runtime dependency, pymongo via pip. The choice is yours.

git clone https://github.com/kaizendorks/pymongo_inmemory.git
cd pymongo_inmemory
# Only if you have poetry
poetry install --no-root
poetry shell
# If you don't have poetry
pip install pymongo

Then,

python -m pymongo_inmemory.mongod

Or you can create the same test file you've mentioned in this PR and see if the problem is still there.

2. Now we observed that the problem is there, we checkout the fix and run the code again, in the same repository directory

git checkout Issue56

and run the same test and see if the problem is still there.

[ChingEn-Lin]

Hello, After the testing, I had a weird finding. This

>> from pymongo_inmemory import MongoClient >> client = MongoClient()

actually works if I run it under the repo of pymongo_inmemory, regardless of the fix branch or not. Ertugrul Karademir ***@***.******@***.***>> 於 2022年5月9日 下午3:11 寫道： Hi @ChingEn-Lin<https://github.com/ChingEn-Lin> , As mentioned above, I created a fix for this issue. However now we need to see if it solves your issue. Could you do the following please? 1. Reproduce the issue, or, make sure the issue is still there. For this you'd need at least poetry<https://python-poetry.org/> installed on the system. Or, you can install the only runtime dependency, pymongo via pip. The choice is yours. git clone https://github.com/kaizendorks/pymongo_inmemory.git cd pymongo_inmemory # Only if you have poetry poetry install --no-root poetry shell # If you don't have poetry pip install pymongo Then, python -m pymongo_inmemory.mongod Or you can create the same test file you've mentioned in this PR and see if the problem is still there. 1. Now we observed that the problem is there, we checkout the fix and run the code again, in the same repository directory git checkout Issue56 and run the same test and see if the problem is still there. — Reply to this email directly, view it on GitHub<#56 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARWAPJ3UL6LETLOCO5YGM63VJEFIPANCNFSM5T7UY2QA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ekarademir]

Ok. I'll create a new version anyway, because I think similar problems arose in Windows as well. After that you can update the version and see if the problem is still there.

[ekarademir]

Hi @ChingEn-Lin I published a new version. Could you update the version you are using and try to see if your problem is solved?

[ChingEn-Lin]

Hello, the problem is solved, thank you very much!

[ekarademir]

Great to hear!