Unable to connect to docker simulator image

[mikehale]

ISSUE TYPE

• Bug Report

COMPONENT NAME

docker simulator image

CLOUDSTACK VERSION

4.17.2.0

CONFIGURATION

N/A

OS / ENVIRONMENT

Docker

SUMMARY

The instructions for using the simulator don't seem to work.

STEPS TO REPRODUCE

From the [README](tools/docker/README.md) I followed these instructions:

```
docker run --name simulator -p 8080:5050 -d apache/cloudstack-simulator:4.17.2.0
```

Waited for the logs to settle down by monitoring `docker logs simulator -f`, then attempted to access the UI, but the connection times out:

```
➜ curl --max-time 5 http://localhost:8080/
curl: (28) Operation timed out after 5004 milliseconds with 0 bytes received
```

EXPECTED RESULTS

see above

ACTUAL RESULTS

see above

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[mikehale]

I tried exposing all ports like this:

docker run --name simulator -p 8080:8080 -p 5050:5050 -p 8096:8096 -d apache/cloudstack-simulator:4.17.2.0

And I was able to connect however, I'm not sure if something still isn't setup correctly, or if I'm interacting with the API incorrectly:

➜ curl --max-time 3 -u admin:password http://localhost:8080/client/api/listVirtualMachinesList
<?xml version="1.0" encoding="UTF-8"?><errorresponse cloud-stack-version="4.17.2.0"><errorcode>401</errorcode><errortext>unable to verify user credentials and/or request signature</errortext></errorresponse>%

[mikehale]

Additionally I tried to setup a basic cloud, again by following the readme instructions but got an error:

➜ docker exec -it simulator python /root/tools/marvin/marvin/deployDataCenter.py -i /root/setup/dev/basic.cfg

==== Log Folder Path: /tmp/MarvinLogs/DeployDataCenter__Jan_17_2024_22_00_55_INWHCR All logs will be available here ====

=== TestClient Creation Failed===

[weizhouapache]

@mikehale
can you try with 4.18.1.0 ?

I just tried , it looks ok

docker run --name simulator -p 8081:5050 -d apache/cloudstack-simulator:4.18.1.0

curl http://[IP]:8081/

(port 8080 is already used, so I use port 8081)

[mikehale]

I tried running that exact command, and I'm still getting a timeout trying to connect over http. That's really strange to me since docker should be using a completely encapsulated environment. I do see a lot of things being setup at boot time though, so maybe my container is actually different than yours somehow?

After it finishes booting I see the following repeating in the logs:

ERROR [c.c.c.ClusterServiceServletContainer] (Thread-19:null) (logid:) Unexpected exception
java.net.SocketTimeoutException: Accept timed out
        at java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:474)
        at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
        at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
        at com.cloud.cluster.ClusterServiceServletContainer$ListenerThread.run(ClusterServiceServletContainer.java:131)

[mikehale]

So, running that docker command on my Apple Silicon laptop I never see:

App running at: http://localhost:5050

However, when I tried it on my Synology linux box I do see that message, and I'm able to access the dashboard.

[mikehale]

For reference I'm attaching the log file from my apple machine: apple.log

[weizhouapache]

@mikehale
can you try with a port less than 1024 ?

[mikehale]

Same result. I think for some reason when running on apple silicon the app is never starting on port 5050.

[weizhouapache]

Same result. I think for some reason when running on apple silicon the app is never starting on port 5050.

5050 is the port for UI
the port for mgmt server is 8080

can you try with privileged mode (and both smaller port than 1024) ?

[mikehale]

Just for grins I ran an nginx container and confirmed I could connect to it:

$ docker run --name nginx-test -d -p 8080:80 nginx:stable-alpine-slim
$ curl --max-time 3 -I http://localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.24.0
Date: Thu, 18 Jan 2024 17:25:12 GMT

Running the simulator in privileged mode with a low port, still results in the same outcome:

$ docker run --privileged --name simulator -p 1023:5050 -d apache/cloudstack-simulator:4.17.2.0
$ curl --max-time 3 -I http://localhost:1023
curl: (28) Operation timed out after 3002 milliseconds with 0 bytes received

Looking at the logs I don't see anything listening on port 5050 inside the container, only port 8080:

$ docker logs simulator | egrep '5050|8080'
[INFO] Started ServerConnector@40d5ec9c{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}

[weizhouapache]

@mikehale
can you check if there are errors in /var/log/supervisor/cloudstack-ui-stderr*log in the container ?

[mikehale]

I just started the container again and suddenly now it is working without any changes on my end!? It's a bit unsettling to have the container change behavior even if it is for the better:

➜ docker logs simulator | egrep '5050|8080'
[INFO] Started ServerConnector@40d5ec9c{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
[INFO] Started ServerConnector@5a0479f{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
  - Local:   http://localhost:5050
  - Network: http://172.17.0.2:5050

➜ curl -I http://localhost:1023
HTTP/1.1 200 OK
X-Powered-By: Express
Date: Thu, 18 Jan 2024 19:16:46 GMT

[mikehale]

Tangential, but do you know offhand if there is a way to just boot the container, and not have it download and configure a bunch of stuff when it starts?

[mikehale]

I guess it may be that I just wasn't waiting long enough? I just tried running again without --privileged, and using a high port (8081) and it worked.

I tail'd /var/log/supervisor/cloudstack-ui-stderr*log and watched it run through webpack.Progress till it got to 100% and then the UI was available.

It still seems like something isn't working because I'm seeing errors like like this via docker logs:

ERROR [c.c.s.StatsCollector] (StatsCollector-2:ctx-d1d92dd4) (logid:5f11466e) Error trying to retrieve management server host statistics
java.lang.NullPointerException
        at com.cloud.server.StatsCollector$ManagementServerCollector.getSystemCpuCyclesTotal(StatsCollector.java:981)
        at com.cloud.server.StatsCollector$ManagementServerCollector.getProcFileSystemData(StatsCollector.java:950)
        at com.cloud.server.StatsCollector$ManagementServerCollector.getDataFrom(StatsCollector.java:816)
        at com.cloud.server.StatsCollector$ManagementServerCollector.runInContext(StatsCollector.java:760)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Also the deploy script is having issues:

➜ docker exec -it simulator python /root/tools/marvin/marvin/deployDataCenter.py -i /root/setup/dev/basic.cfg

==== Log Folder Path: /tmp/MarvinLogs/DeployDataCenter__Jan_18_2024_19_31_52_FXMVM3 All logs will be available here ====

=== TestClient Creation Failed===

[weizhouapache]

Tangential, but do you know offhand if there is a way to just boot the container, and not have it download and configure a bunch of stuff when it starts?

if you asked about general containers, you can update the entrypoint command/config, or export the container and recreate with a new entrypoint if the first option does not work.

if you talked about the simulator container, the start process is quite simple (1) start mysql, (2) start cloudstack; (3) start cloudstack UI. no need to do above.

[weizhouapache]

I guess it may be that I just wasn't waiting long enough? I just tried running again without --privileged, and using a high port (8081) and it worked.

that's good to know.

I tail'd /var/log/supervisor/cloudstack-ui-stderr*log and watched it run through webpack.Progress till it got to 100% and then the UI was available.

It still seems like something isn't working because I'm seeing errors like like this via docker logs:

ERROR [c.c.s.StatsCollector] (StatsCollector-2:ctx-d1d92dd4) (logid:5f11466e) Error trying to retrieve management server host statistics
java.lang.NullPointerException
        at com.cloud.server.StatsCollector$ManagementServerCollector.getSystemCpuCyclesTotal(StatsCollector.java:981)
        at com.cloud.server.StatsCollector$ManagementServerCollector.getProcFileSystemData(StatsCollector.java:950)
        at com.cloud.server.StatsCollector$ManagementServerCollector.getDataFrom(StatsCollector.java:816)
        at com.cloud.server.StatsCollector$ManagementServerCollector.runInContext(StatsCollector.java:760)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

this issue can be ignored. it should be because you run cloudstack in a container.

Also the deploy script is having issues:

➜ docker exec -it simulator python /root/tools/marvin/marvin/deployDataCenter.py -i /root/setup/dev/basic.cfg

==== Log Folder Path: /tmp/MarvinLogs/DeployDataCenter__Jan_18_2024_19_31_52_FXMVM3 All logs will be available here ====

=== TestClient Creation Failed===

retry with advanced.cfg ?

[mikehale]

Tangential, but do you know offhand if there is a way to just boot the container, and not have it download and configure a bunch of stuff when it starts?

if you asked about general containers, you can update the entrypoint command/config, or export the container and recreate with a new entrypoint if the first option does not work.

Thanks, I'm familiar with this.

if you talked about the simulator container, the start process is quite simple (1) start mysql, (2) start cloudstack; (3) start cloudstack UI. no need to do above.

I guess maybe my question is more along the lines of is there a way to speed up the startup time of the simulator container? Currently it takes about 6 minutes to go from docker start simulator (on a previously run container), to the UI being available. It seems that a large amount of that time is spent running webpack for the UI. Maybe the way things are structured none of that work is cached between runs?

[mikehale]

Running with advanced.cfg yields the same error:

➜ docker exec -it simulator python /root/tools/marvin/marvin/deployDataCenter.py -i /root/setup/dev/advanced.cfg

==== Log Folder Path: /tmp/MarvinLogs/DeployDataCenter__Jan_18_2024_20_19_02_OVT40E All logs will be available here ====

=== TestClient Creation Failed===

[weizhouapache]

Running with advanced.cfg yields the same error:

➜ docker exec -it simulator python /root/tools/marvin/marvin/deployDataCenter.py -i /root/setup/dev/advanced.cfg

==== Log Folder Path: /tmp/MarvinLogs/DeployDataCenter__Jan_18_2024_20_19_02_OVT40E All logs will be available here ====

=== TestClient Creation Failed===

@mikehale
it worked in my testing

root@buildbox-u22:~ # docker exec -it simulator python /root/tools/marvin/marvin/deployDataCenter.py -i /root/setup/dev/advanced.cfg

==== Log Folder Path: /tmp/MarvinLogs/DeployDataCenter__Jan_19_2024_07_34_09_I58832 All logs will be available here ====

==== Deploy DC Started ====

=== Data Center Settings are dumped to /tmp/MarvinLogs/DeployDataCenter__Jan_19_2024_07_34_09_I58832/dc_entries.obj===

====Deploy DC Successful=====

I suspect it is caused by some settings with the docker image.
I would be good to test it by someone else who use Mac,

[mikehale]

@weizhouapache thanks for all your time and help on this. I'm guessing that this is related to Apple Silicon and the rosetta compatibility layer that docker uses.

I'm going to double check that by running on an old Intel Mac.

[mikehale]

I just confirmed everything works just fine on my Intel Mac, so this seems to be an issue with Apple Silicon.

[weizhouapache]

@mikehale
I am closing this issue as it seems not to be a cloudstack issue.
please feel free to reopen it