Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

phost stop cause by os error 104 #231

Closed
maybeTomorrow opened this issue Jun 22, 2021 · 14 comments
Closed

phost stop cause by os error 104 #231

maybeTomorrow opened this issue Jun 22, 2021 · 14 comments

Comments

@maybeTomorrow
Copy link

[2021-06-22T06:55:07Z INFO phost::chain_client] - Stats Online Workers: 20
[2021-06-22T06:55:07Z INFO phost::chain_client] - Stats Compute Workers: 5
[2021-06-22T06:55:07Z INFO phost::pruntime_client] Response: 200 OK
[2021-06-22T06:55:07Z INFO phost::pruntime_client] Response: 200 OK
[2021-06-22T06:55:07Z INFO phost] sending a batch of 222 headers (last: 210900, change: None)
[2021-06-22T06:55:07Z INFO phost] bridge() exited with result: Err(connection error: Connection reset by peer (os error 104)

Caused by:
    Connection reset by peer (os error 104))
@h4x3rotab
Copy link
Contributor

I believe that's because pruntime has crashed. Checking the log from pruntime should be helpful.

@maybeTomorrow
Copy link
Author

maybeTomorrow commented Jun 23, 2021 •
edited

@h4x3rotab
this is the pruntime logs,what's wrong?

[2021-06-22T06:55:07Z INFO rocket::rocket] Outcome: Success
[2021-06-22T06:55:07Z INFO rocket::rocket] Response succeeded.
[2021-06-22T06:55:07Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-22T06:55:07Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-22T06:55:07Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-22T06:55:07Z INFO rocket::rocket] Outcome: Failure
[2021-06-22T06:55:07Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-22T06:55:07Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.
[2021-06-22T08:15:13Z INFO rocket::rocket] GET /:
[2021-06-22T08:15:13Z ERROR rocket::rocket] No matching routes for GET /.
[2021-06-22T08:15:13Z WARN rocket::rocket] Responding with 404 Not Found catcher.
[2021-06-22T08:15:13Z INFO rocket::rocket] Response succeeded.

@maybeTomorrow
Copy link
Author

[2021-06-23T09:26:20Z INFO rocket::rocket] POST /dispatch_block application/json:
[2021-06-23T09:26:20Z INFO rocket::rocket] Matched: POST /dispatch_block application/json (dispatch_block)
[2021-06-23T09:26:20Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-23T09:26:20Z INFO rocket::rocket] Outcome: Failure
[2021-06-23T09:26:20Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-23T09:26:20Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.

@maybeTomorrow
Copy link
Author

some one help?

@jasl
Copy link
Collaborator

jasl commented Jun 25, 2021

whats your hardware and OS? we didn't see this error before

@maybeTomorrow
Copy link
Author

whats your hardware and OS? we didn't see this error before

azure vm,Linux (ubuntu 20.04), DC2s_v2 (2 vcpu，8 GiB 内存)

@jasl
Copy link
Collaborator

jasl commented Jun 25, 2021

whats your hardware and OS? we didn't see this error before

azure vm,Linux (ubuntu 20.04), DC2s_v2 (2 vcpu，8 GiB 内存)

We're not test any cloud VM yet, from started phost, how long will you meet this problem?

I suspect EPC too small on 2 vcpu VM, can you try 4 vcpu?

@maybeTomorrow
Copy link
Author

whats your hardware and OS? we didn't see this error before

azure vm,Linux (ubuntu 20.04), DC2s_v2 (2 vcpu，8 GiB 内存)

We're not test any cloud VM yet, from started phost, how long will you meet this problem?

I suspect EPC too small on 2 vcpu VM, can you try 4 vcpu?

about 2 minutes. before this error,there is many log show
[2021-06-25T03:49:59Z INFO rocket::rocket] POST /get_info application/json:
[2021-06-25T03:49:59Z INFO rocket::rocket] Matched: POST /get_info application/json (get_info)
[2021-06-25T03:49:59Z INFO enclaveapp] {"payload":"{"blocknum":481913,"dev_mode":false,"ecdh_public_key":"044b537a99c69e9f6658cc01f22906d7b497a4b9dcd0b7081cf4b5305e11bdae4049643050b627dbb6986ecca1b5343fec52df0f96bc3bfef2a6e46deeb4cc7467","headernum":481913,"initialized":true,"machine_id":[182,160,133,212,75,11,247,25,10,153,107,187,23,200,73,12],"public_key":"02255db5612b6495f668cf1dd24e20a481540942632b6263a95a1b4f0cbe75d094","system_egress":{"len":1681,"sequence":1681}}","signature":"a58be74386b614b2b0c53337920e122c36048ad2c400bf32b981529520d0107f31b1acc4ef08e23aa482f0667bb082bca69df00ce1a4f64f7f2956f07be7db36","status":"ok"}
[2021-06-25T03:49:59Z INFO rocket::rocket] Outcome: Success
[2021-06-25T03:49:59Z INFO rocket::rocket] Response succeeded.
[2021-06-25T03:50:01Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-25T03:50:01Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-25T03:50:01Z INFO enclaveapp::light_validation] ancestor_hash: 0xdc55…94bc

@jasl
Copy link
Collaborator

jasl commented Jun 25, 2021

whats your hardware and OS? we didn't see this error before

azure vm,Linux (ubuntu 20.04), DC2s_v2 (2 vcpu，8 GiB 内存)

We're not test any cloud VM yet, from started phost, how long will you meet this problem?
I suspect EPC too small on 2 vcpu VM, can you try 4 vcpu?

about 2 minutes. before this error,there is many log show
[2021-06-25T03:49:59Z INFO rocket::rocket] POST /get_info application/json:
[2021-06-25T03:49:59Z INFO rocket::rocket] Matched: POST /get_info application/json (get_info)
[2021-06-25T03:49:59Z INFO enclaveapp] {"payload":"{"blocknum":481913,"dev_mode":false,"ecdh_public_key":"044b537a99c69e9f6658cc01f22906d7b497a4b9dcd0b7081cf4b5305e11bdae4049643050b627dbb6986ecca1b5343fec52df0f96bc3bfef2a6e46deeb4cc7467","headernum":481913,"initialized":true,"machine_id":[182,160,133,212,75,11,247,25,10,153,107,187,23,200,73,12],"public_key":"02255db5612b6495f668cf1dd24e20a481540942632b6263a95a1b4f0cbe75d094","system_egress":{"len":1681,"sequence":1681}}","signature":"a58be74386b614b2b0c53337920e122c36048ad2c400bf32b981529520d0107f31b1acc4ef08e23aa482f0667bb082bca69df00ce1a4f64f7f2956f07be7db36","status":"ok"}
[2021-06-25T03:49:59Z INFO rocket::rocket] Outcome: Success
[2021-06-25T03:49:59Z INFO rocket::rocket] Response succeeded.
[2021-06-25T03:50:01Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-25T03:50:01Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-25T03:50:01Z INFO enclaveapp::light_validation] ancestor_hash: 0xdc55…94bc

these are normal

@maybeTomorrow
Copy link
Author

maybeTomorrow commented Jun 25, 2021 •
edited

i reboot the vm.now it's running,and 5 minutes stop
[2021-06-25T08:28:54Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-25T08:28:54Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-25T08:28:54Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-25T08:28:54Z INFO rocket::rocket] Outcome: Failure
[2021-06-25T08:28:54Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-25T08:28:54Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.

@jasl
Copy link
Collaborator

jasl commented Jun 25, 2021

i reboot the vm.now it's running,and 5 minutes stop
[2021-06-25T08:28:54Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-25T08:28:54Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-25T08:28:54Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-25T08:28:54Z INFO rocket::rocket] Outcome: Failure
[2021-06-25T08:28:54Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-25T08:28:54Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.

Do you change vCPU to 4?

@maybeTomorrow
Copy link
Author

i reboot the vm.now it's running,and 5 minutes stop
[2021-06-25T08:28:54Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-25T08:28:54Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-25T08:28:54Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-25T08:28:54Z INFO rocket::rocket] Outcome: Failure
[2021-06-25T08:28:54Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-25T08:28:54Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.

Do you change vCPU to 4?

no,i'm changing

@maybeTomorrow
Copy link
Author

maybeTomorrow commented Jun 28, 2021 •
edited

i reboot the vm.now it's running,and 5 minutes stop
[2021-06-25T08:28:54Z INFO rocket::rocket] POST /sync_header application/json:
[2021-06-25T08:28:54Z INFO rocket::rocket] Matched: POST /sync_header application/json (sync_header)
[2021-06-25T08:28:54Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-25T08:28:54Z INFO rocket::rocket] Outcome: Failure
[2021-06-25T08:28:54Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-25T08:28:54Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.

Do you change vCPU to 4?

i chnage to vcpu 4,same error
标准 DC4s_v2 (4 vcpu，16 GiB 内存)

[2021-06-28T02:25:30Z INFO enclaveapp::system] System::handle_reward_seed(438600, BlockRewardInfo { seed: 26927906142820954459526734866955509585022124177009442592143407157399007829398, online_target: 385973630791053984745236616695626359510899948885468546798191946693043760000, compute_target: 385819241338737563151338522048948108967095588905914359379472669914366547494 })
[2021-06-28T02:25:30Z INFO enclaveapp] handle_events: skipped for miners
[2021-06-28T02:25:30Z INFO enclaveapp] {"payload":"{"dispatched_to":438600}","signature":"1c5ddef9fff4b4829f11f8a82d8619ead63e8583f44be73a4dc1c1aeb75ac20250cc71bfa990dfcc831d968164fa589b49bf77a312235c57a1436d5c64d17fc6","status":"ok"}
[2021-06-28T02:25:30Z INFO rocket::rocket] Outcome: Success
[2021-06-28T02:25:30Z INFO rocket::rocket] Response succeeded.
[2021-06-28T02:25:30Z INFO rocket::rocket] POST /dispatch_block application/json:
[2021-06-28T02:25:30Z INFO rocket::rocket] Matched: POST /dispatch_block application/json (dispatch_block)
[2021-06-28T02:25:30Z WARN rocket::data::data_stream] Data left unread. Force closing network stream.
[2021-06-28T02:25:30Z INFO rocket::rocket] Outcome: Failure
[2021-06-28T02:25:30Z WARN rocket::rocket] Responding with 400 Bad Request catcher.
[2021-06-28T02:25:30Z ERROR rocket::rocket] Failed to write response: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }.

@h4x3rotab
Copy link
Contributor

Obsolete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jasl @h4x3rotab @maybeTomorrow