Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

env.verify_rsa_sha256_sig unresolveable #28

Closed
cc32d9 opened this issue Aug 18, 2019 · 38 comments

Comments

@cc32d9
Copy link
Contributor

@cc32d9 cc32d9 commented Aug 18, 2019

Some servers under "wax-1.8.1-1.0.0-rc2-19-ge2187f190" cannot get past the block 9135392, with the error as follows:

Aug 18 08:32:43 wax nodeos[18746]: error 2019-08-18T08:32:43.883 nodeos    producer_plugin.cpp:400       on_incoming_block    ] 3070000 wasm_exception: WASM Exception
Aug 18 08:32:43 wax nodeos[18746]: env.verify_rsa_sha256_sig unresolveable
Aug 18 08:32:43 wax nodeos[18746]:     {"module":"env","export":"verify_rsa_sha256_sig"}
Aug 18 08:32:43 wax nodeos[18746]:     nodeos  wasm_interface.hpp:59 resolve
Aug 18 08:32:43 wax nodeos[18746]:     {"mod_name":"env","export_name":"verify_rsa_sha256_sig"}
Aug 18 08:32:43 wax nodeos[18746]:     nodeos  wasm_interface.hpp:61 resolve
Aug 18 08:32:43 wax nodeos[18746]: pending console output:
Aug 18 08:32:43 wax nodeos[18746]:     {"console":""}
Aug 18 08:32:43 wax nodeos[18746]:     nodeos  apply_context.cpp:113 exec_one
Aug 18 08:32:43 wax nodeos[18746]:     {}
Aug 18 08:32:43 wax nodeos[18746]:     nodeos  controller.cpp:1765 apply_block
Aug 18 08:32:43 wax nodeos[18746]: rethrow
Aug 18 08:32:43 wax nodeos[18746]:     {}
Aug 18 08:32:43 wax nodeos[18746]:     nodeos  controller.cpp:1821 push_block

Other servers with the same software version work fine. This failing server has only one peer that is on a server that made it through. Michael at EOS USA has also a machine that fails in the same way, but another machine in the same environment works fine.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Aug 18, 2019

I noticed that was from "develop" branch, because Git picked it up as default. Now recompiled from "master", and result is the same, with exactly the same error.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Aug 18, 2019

Build procedure:

mkdir /opt/src/
cd /opt/src
git clone -b master https://github.com/worldwide-asset-exchange/wax-blockchain.git
cd wax-blockchain/
git submodule update --init --recursive
./wax_build.sh -P -y
./wax_install.sh

Now trying without the -P flag

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Aug 18, 2019

it went through without the -P flag. So it's probably an issue with clang optimizer

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Aug 19, 2019

Hi @cc32d9! Please use our release version (wax-1.8.1-1.0.0 instead of wax-1.8.1-1.0.0-rc2). Be aware that release candidate versions are not ready for production (they are released for testing purposes and may have bugs).

@ankh2054

This comment has been minimized.

Copy link

@ankh2054 ankh2054 commented Aug 19, 2019

Same issue for me on 1.8.1-1.0.0

@igorls

This comment has been minimized.

Copy link
Contributor

@igorls igorls commented Aug 19, 2019

Same issue here on 1.8.1-1.0.0 with state-history enabled (build with -P)

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Aug 19, 2019

Hi guys. We are investigating the problem. In the meantime, could you post some useful information for us?

  1. How did you get the code? ($ git clone -b wax-1.8.1-1.0.0 https://github.com/worldwide-asset-exchange/wax-blockchain.git is recommended)
  2. How did you compile the code? (updated instructions are in develop branch: README.md)
  3. Operating system used (Ubuntu 18.04 is recommended, but not necessary)
  4. Result of: $ nodeos --version (expected wax-1.8.1-1.0.0)
  5. Result of: $ cleos get info
  6. Logs from nodeos (especially the first 50 lines)

Thanks

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Aug 20, 2019

Here's what I used:

mkdir /opt/src/
cd /opt/src
git clone -b master https://github.com/worldwide-asset-exchange/wax-blockchain.git
cd wax-blockchain/
git submodule update --init --recursive
./wax_build.sh -P -y
./wax_install.sh

later on, I removed -P and it started working. On another machine, this worked with -P too. The machines have different generations of intel Xeon CPU

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Aug 20, 2019

Hi @cc32d9, so you cloned from master, but according to your first post, your version is wax-1.8.1-1.0.0-rc2. That's not possible, rc2 was never merged to master (actually we never merge release candidate versions to master). Could you please check the version of your nodeos binary? (step 4 of my previous post). Last but not least, yesterday we recompiled, from scratch, version wax-1.8.1-1.0.0 and performed a full sync without any issue.

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Aug 27, 2019

Due to our policy we have to close this issue because we waited a week for an answer. Don't hesitate to open a new issue if you still have problems.

Remember, use our updated installation instructions README.md

@jalbiero jalbiero closed this Aug 27, 2019
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Aug 27, 2019

no, wait, I mentioned above that I cloned later from proper branch and the problem remained. Also other users confirm the problem. Please reopen it, as the issue is still unresolved.

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Aug 28, 2019

But none of you provided the information we asked in order to help us with the resolution (6 items). We cannot reproduce the problem and without those items we cannot be sure that you are running the same version that we have in production. I will reopen the issue, but you will have to provide the information I asked you.

@jalbiero jalbiero reopened this Aug 28, 2019
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Aug 28, 2019

the problem is that it's shaky. The same setup works on one machine and fails on the other. I'll try to reproduce it in a test environment. The case was opened when I was trying to repair the production environment, so some logs were lost in the rush.

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Sep 3, 2019

Please be aware that if no one can provide more information (especially the one we asked) this issue will be closed tomorrow. Thanks.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 3, 2019

sorry I was taken away by urgent work. This is a critical issue confirmed by several teams, so it cannot be closed and ignored.

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Sep 3, 2019

Hi @cc32d9 the official way is to use our docker image. If you can, please try with that, it's really easy (more information here: README.md).

I am sorry, I don't want to be rude, but we need evidence and more information to confirm the problem. We are willing to help you, actually, we try, but we couldn't reproduce it, and people here that said that they experienced the same didn't provide any information that lead us to confirm the problem.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 3, 2019

I don't have docker in our production infrastructure. The difficulty is that the same installation procedure on two different servers gave different results, and one was failing while the other was OK. I will try to find the time to reproduce it within a couple of days.

@matthewdarwin

This comment has been minimized.

Copy link

@matthewdarwin matthewdarwin commented Sep 3, 2019

I ran into this problem as well, however I am not runnnig docker. It would get stuck at that block with sync from network, replaying blocks from a pre-existing block log worked fine.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

@jalbiero your installation instructions don't have the -P option for the build script, and I have already got a working server without this option. But -P is important because it fixates the compiler version, so that subsequent upgrades are compatible with the data files. This error occurred when I used the -P option.

@matthewdarwin

This comment has been minimized.

Copy link

@matthewdarwin matthewdarwin commented Sep 4, 2019

I used "-P" as well.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

I'm setting up a test instance now, compiling with -P

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

here we go, I reproduced it on the same physical machine, in a separate LXC container.

root@waxtest:~# cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Build procedure:

mkdir /opt/src/
cd /opt/src
git clone -b wax-1.8.1-1.0.0 https://github.com/worldwide-asset-exchange/wax-blockchain.git
cd wax-blockchain/
git submodule update --init --recursive
./wax_build.sh -P -y 
./wax_install.sh

config.ini:

root@waxtest:~# cat /srv/testnode/etc/config.ini 
chain-state-db-size-mb = 65536
reversible-blocks-db-size-mb = 2048
wasm-runtime = wavm
http-server-address = 0.0.0.0:8888
p2p-listen-endpoint = 0.0.0.0:9876
access-control-allow-origin = *
max-clients = 100
plugin = eosio::chain_plugin
plugin = eosio::chain_api_plugin
validation-mode = light
sync-fetch-span = 500
p2p-peer-address = peers.wax.io:9876
p2p-peer-address = br.eosrio.io:35668
p2p-peer-address = p2p.waxsweden.org:35777
p2p-peer-address = wax.blockmatrix.network:13546
p2p-peer-address = waxp2p.eoscafeblock.com:9090
p2p-peer-address = waxpeer.eos42.io:9879
p2p-peer-address = wax.zenblocks.io:13975
p2p-peer-address = peer.wax.alohaeos.com:9876
p2p-peer-address = peer1-wax.eosphere.io:9876
p2p-peer-address = wax-p2p.infinitybloc.io:9876
p2p-peer-address = wax.greymass.com:35777
p2p-peer-address = wax-p2p.eos.barcelona:9876
p2p-peer-address = wax.eosusa.news:9879

get info:

root@waxtest:~# /root/eosio/1.8/bin/cleos get info
{
  "server_version": "2fc665e9",
  "chain_id": "1064487b3cd1a897ce03ae5b6a865651747e2e152090f99c1d19d44e01aea5a4",
  "head_block_num": 9135392,
  "last_irreversible_block_num": 9135065,
  "last_irreversible_block_id": "008b63d999601bf346088db9a2a7a47edcf408ad2865248eeeaa7107149ada42",
  "head_block_id": "008b6520e9aab1d849205c7ae9e43934636dc70eddb9139122208f0cf100a5fc",
  "head_block_time": "2019-08-16T19:43:01.000",
  "head_block_producer": "mining.wax",
  "virtual_block_cpu_limit": 500000000,
  "virtual_block_net_limit": 1048576000,
  "block_cpu_limit": 500000,
  "block_net_limit": 1048576,
  "server_version_string": "wax-1.8.1-1.0.0",
  "fork_db_head_block_num": 9135392,
  "fork_db_head_block_id": "008b6520e9aab1d849205c7ae9e43934636dc70eddb9139122208f0cf100a5fc"
}

The node started from state and blocks log at about block 9083000. The whole log is attached.
screenlog.txt

The whole log

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

get info was taken after the node got stuck.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

/proc/cpuinfo shows:

model name	: Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz

On a different physical machine where the sync went smoothly,

model name	: Intel(R) Xeon(R) E-2186G CPU @ 3.80GHz
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

The whole content of data and config directories at block 9083000 is available at https://snapshots.eosamsterdam.net/wax/wax_issue_28_testnode.tar.gz

@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Sep 4, 2019

Excellent @cc32d9, your help is really appreciated. We are analyzing the information that you provided. Thanks!

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 4, 2019

just doing my job :)
The log is best to view in raw mode:

less -r screenlog.txt
@jalbiero

This comment has been minimized.

Copy link
Member

@jalbiero jalbiero commented Sep 16, 2019

Hi @cc32d9. We try to reproduce the problem, but we couldn't. Our production node with you config file, a node compiled with -P (pinned compiler) with our config file, and with your config file, all of them synchronized correctly.

FYI: The problem was present in wax-1.8.1-1.0.0-rc1 (because the function verify_rsa_sha256_sig was not exported correctly). rc2 and final release don't have the mentioned issue (at least for us)

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 16, 2019

thanks, I will try to re-check with 1.8.3 during the week, on the same machine.

ernestopskins pushed a commit that referenced this issue Sep 26, 2019
# This is the 1st commit message:

various improvements

# This is the commit message #2:

new hash

# This is the commit message #3:

fix for script path

# This is the commit message #4:

fixes

# This is the commit message #5:

fixes

# This is the commit message #6:

fixes

# This is the commit message #7:

fixes

# This is the commit message #8:

fixes

# This is the commit message #9:

fixes

# This is the commit message #10:

fixes

# This is the commit message #11:

fixes

# This is the commit message #12:

fixes

# This is the commit message #13:

fixes

# This is the commit message #14:

fixes

# This is the commit message #15:

fixes

# This is the commit message #16:

fixes

# This is the commit message #17:

fixes

# This is the commit message #18:

fixes

# This is the commit message #19:

fixes

# This is the commit message #20:

fixes

# This is the commit message #21:

fixes

# This is the commit message #22:

fixes

# This is the commit message #23:

fixes

# This is the commit message #24:

fixes

# This is the commit message #25:

fixes

# This is the commit message #26:

testing

# This is the commit message #27:

testing

# This is the commit message #28:

testing

# This is the commit message #29:

testing

# This is the commit message #30:

testing

# This is the commit message #31:

testing

# This is the commit message #32:

testing

# This is the commit message #33:

testing

# This is the commit message #34:

testing

# This is the commit message #35:

testing

# This is the commit message #36:

testing

# This is the commit message #37:

testing

# This is the commit message #38:

testing

# This is the commit message #39:

testing

# This is the commit message #40:

testing

# This is the commit message #41:

testing

# This is the commit message #42:

testing

# This is the commit message #43:

testing

# This is the commit message #44:

fixes

# This is the commit message #45:

fixes

# This is the commit message #46:

fixes

# This is the commit message #47:

fixes

# This is the commit message #48:

fixes

# This is the commit message #49:

fixes

# This is the commit message #50:

fixes

# This is the commit message #51:

fixes

# This is the commit message #52:

fixes

# This is the commit message #53:

fixes

# This is the commit message #54:

fixes

# This is the commit message #55:

fixes

# This is the commit message #56:

fixes

# This is the commit message #57:

fixes

# This is the commit message #58:

fixes

# This is the commit message #59:

fixes

# This is the commit message #60:

fixes

# This is the commit message #61:

fixes

# This is the commit message #62:

fixes

# This is the commit message #63:

fixes

# This is the commit message #64:

fixes

# This is the commit message #65:

fixes

# This is the commit message #66:

fixes

# This is the commit message #67:

fixes

# This is the commit message #68:

fixes

# This is the commit message #69:

fixes

# This is the commit message #70:

fixes

# This is the commit message #71:

fixes

# This is the commit message #72:

fixes

# This is the commit message #73:

fixes

# This is the commit message #74:

fixes

# This is the commit message #75:

fixes

# This is the commit message #76:

fixes

# This is the commit message #77:

fixes

# This is the commit message #78:

fixes

# This is the commit message #79:

more testing

# This is the commit message #80:

testing

# This is the commit message #81:

fixes

# This is the commit message #82:

fixes

# This is the commit message #83:

fixes

# This is the commit message #84:

fixes

# This is the commit message #85:

fixes

# This is the commit message #86:

fixes

# This is the commit message #87:

fixes

# This is the commit message #88:

fixes

# This is the commit message #89:

fixes

# This is the commit message #90:

fixes

# This is the commit message #91:

fixes

# This is the commit message #92:

fixes

# This is the commit message #93:

propagate-environment for buildkite-agent

# This is the commit message #94:

propagate-environment for buildkite-agent

# This is the commit message #95:

propagate-environment for buildkite-agent

# This is the commit message #96:

propagate-environment for buildkite-agent

# This is the commit message #97:

fixes

# This is the commit message #98:

fixes

# This is the commit message #99:

fixes

# This is the commit message #100:

fixes

# This is the commit message #101:

fixes

# This is the commit message #102:

fixes

# This is the commit message #103:

fixes

# This is the commit message #104:

fixes

# This is the commit message #105:

fixes

# This is the commit message #106:

fixes

# This is the commit message #107:

fixes

# This is the commit message #108:

fixes

# This is the commit message #109:

fixes

# This is the commit message #110:

fixes

# This is the commit message #111:

fixes

# This is the commit message #112:

fixes

# This is the commit message #113:

fixes

# This is the commit message #114:

fixes

# This is the commit message #115:

fixes

# This is the commit message #116:

fixes

# This is the commit message #117:

fixes

# This is the commit message #118:

fixes

# This is the commit message #119:

fixes

# This is the commit message #120:

fixes

# This is the commit message #121:

fixes

# This is the commit message #122:

fixes

# This is the commit message #123:

fixes

# This is the commit message #124:

fixes

# This is the commit message #125:

fixes

# This is the commit message #126:

fixes

# This is the commit message #127:

fixes

# This is the commit message #128:

fixes

# This is the commit message #129:

fixes

# This is the commit message #130:

fixes

# This is the commit message #131:

fixes

# This is the commit message #132:

fixes

# This is the commit message #133:

fixes

# This is the commit message #134:

fixes

# This is the commit message #135:

fixes

# This is the commit message #136:

fixes

# This is the commit message #137:

fixes

# This is the commit message #138:

fixes

# This is the commit message #139:

fixes

# This is the commit message #140:

fixes

# This is the commit message #141:

fixes

# This is the commit message #142:

fixes

# This is the commit message #143:

fixes

# This is the commit message #144:

fixes

# This is the commit message #145:

fixes

# This is the commit message #146:

fixes

# This is the commit message #147:

fixes

# This is the commit message #148:

fixes

# This is the commit message #149:

fixes

# This is the commit message #150:

fixes

# This is the commit message #151:

fixes

# This is the commit message #152:

fixes

# This is the commit message #153:

testing

# This is the commit message #154:

fixes

# This is the commit message #155:

fixes

# This is the commit message #156:

fixes

# This is the commit message #157:

fixes

# This is the commit message #158:

fixes

# This is the commit message #159:

fixes

# This is the commit message #160:

fixes

# This is the commit message #161:

fixes

# This is the commit message #162:

fixes

# This is the commit message #163:

fixes

# This is the commit message #164:

fixes

# This is the commit message #165:

fixes

# This is the commit message #166:

fixes

# This is the commit message #167:

fixes

# This is the commit message #168:

fixes

# This is the commit message #169:

fixes

# This is the commit message #170:

fixes

# This is the commit message #171:

fixes

# This is the commit message #172:

fixes

# This is the commit message #173:

fixes

# This is the commit message #174:

fixes

# This is the commit message #175:

fixes

# This is the commit message #176:

fixes

# This is the commit message #177:

fixes

# This is the commit message #178:

fixes

# This is the commit message #179:

fixes

# This is the commit message #180:

fixes

# This is the commit message #181:

fixes

# This is the commit message #182:

fixes

# This is the commit message #183:

fixes

# This is the commit message #184:

fixes

# This is the commit message #185:

fixes

# This is the commit message #186:

fixes
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 30, 2019

I made a fresh build of 1.8.4 and the issue is still reproduceable on this machine.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 30, 2019

root@waxtest:~# /root/eosio/1.8/bin/cleos get info
{
  "server_version": "7328c2db",
  "chain_id": "1064487b3cd1a897ce03ae5b6a865651747e2e152090f99c1d19d44e01aea5a4",
  "head_block_num": 9135392,
  "last_irreversible_block_num": 9135065,
  "last_irreversible_block_id": "008b63d999601bf346088db9a2a7a47edcf408ad2865248eeeaa7107149ada42",
  "head_block_id": "008b6520e9aab1d849205c7ae9e43934636dc70eddb9139122208f0cf100a5fc",
  "head_block_time": "2019-08-16T19:43:01.000",
  "head_block_producer": "mining.wax",
  "virtual_block_cpu_limit": 500000000,
  "virtual_block_net_limit": 1048576000,
  "block_cpu_limit": 500000,
  "block_net_limit": 1048576,
  "server_version_string": "wax-1.8.4-1.0.0",
  "fork_db_head_block_num": 9135392,
  "fork_db_head_block_id": "008b6520e9aab1d849205c7ae9e43934636dc70eddb9139122208f0cf100a5fc"
}
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Sep 30, 2019

I copied the nodeos binary from a server that was working well onto this test machine, and it produces the same error. So it's not a compiler problem, but a runtime problem. Probably some cryptographic primitives are not implemented in this CPU model?

The one failing:

vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz
stepping        : 7
microcode       : 0x718
cpu MHz         : 1292.526
cache size      : 15360 KB
physical id     : 0
siblings        : 12
core id         : 0
cpu cores       : 6
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm arat pln pts md_clear flush_l1d
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips        : 3790.87
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

The one working fine:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 158
model name      : Intel(R) Xeon(R) E-2186G CPU @ 3.80GHz
stepping        : 10
microcode       : 0xb4
cpu MHz         : 901.072
cache size      : 12288 KB
physical id     : 0
siblings        : 12
core id         : 0
cpu cores       : 6
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_act_window hwp_epp md_clear flush_l1d
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs
bogomips        : 7584.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:
@matthewdarwin

This comment has been minimized.

Copy link

@matthewdarwin matthewdarwin commented Oct 1, 2019

I tried my own binary on 3 different servers, at least 1 of which didn't work on earlier version. All work now.

processor       : 31
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz
stepping        : 7
microcode       : 0x714
cpu MHz         : 3300.134
cache size      : 20480 KB
physical id     : 1
siblings        : 16
core id         : 7
cpu cores       : 8
apicid          : 47
initial apicid  : 47
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts flush_l1d
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs
bogomips        : 5801.74
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:
processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 158
model name      : Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz
stepping        : 9
microcode       : 0x84
cpu MHz         : 3699.999
cache size      : 8192 KB
physical id     : 0
siblings        : 4
core id         : 3
cpu cores       : 4
apicid          : 6
initial apicid  : 6
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs
bogomips        : 7008.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:
processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 158
model name      : Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
stepping        : 12
microcode       : 0xa2
cpu MHz         : 4653.017
cache size      : 16384 KB
physical id     : 0
siblings        : 8
core id         : 7
cpu cores       : 8
apicid          : 14
initial apicid  : 14
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp flush_l1d arch_capabilities
bugs            : spectre_v1 spectre_v2 spec_store_bypass mds swapgs
bogomips        : 7200.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Oct 4, 2019

Reproduced with i9:

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 158
model name	: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
stepping	: 13
microcode	: 0xb8
cpu MHz		: 4850.207
cache size	: 16384 KB
physical id	: 0
siblings	: 16
core id		: 7
cpu cores	: 8
apicid		: 15
initial apicid	: 15
fpu		: yes
fpu_exception	: yes
cpuid level	: 22
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
bugs		: spectre_v1 spectre_v2 spec_store_bypass swapgs
bogomips	: 7200.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:
@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Oct 4, 2019

and now, the most interesting part. Same error without -P flag. The server is with i9 CPU as in previous message.

Building steps:

mkdir /srv/wax/src/
cd /srv/wax/src/
git clone -b master https://github.com/worldwide-asset-exchange/wax-blockchain.git
cd wax-blockchain/
git submodule update --init --recursive
./wax_build.sh -i /srv/wax/opt -y
./wax_install.sh

Downloaded Eric's snapshot from right before the bad block: https://snapshots.waxsweden.org/snapshots/57abfa/1.8.eosio/snapshot-9123753.bin.tar.gz

/srv/wax/opt/bin/nodeos --data-dir /srv/wax/data --config-dir /srv/wax/etc --snapshot tmp/snapshot-9123753.bin 

it results in the same error, "WASM Exception env.verify_rsa_sha256_sig unresolveable"

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Oct 4, 2019

ok that snapshpot was made with vanilla eosio. If starting from a snapshot which was created by wax software, it goes through the bad block

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Oct 4, 2019

if started from a snapshot made with Wax software that has run from the genesis, both with and without -P nodes go over this block without issues. I will retest on the machine that was originally used for reporting.

@cc32d9

This comment has been minimized.

Copy link
Contributor Author

@cc32d9 cc32d9 commented Oct 4, 2019

tested on the other machine, with the same results. So, the outcome is, that the state produced by vanilla nodeos produces this error after switching to WAX software. If WAX software has replayed from genesis, the error does not pop up with or without -P option.

@cc32d9 cc32d9 closed this Oct 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.