Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[求助/Help]Ceph rbd 作为后端存储，创建虚拟机报：no enough backend "rbd" #18380

Closed
chenjacken opened this issue Oct 21, 2023 · 28 comments
Closed

[求助/Help]Ceph rbd 作为后端存储，创建虚拟机报：no enough backend "rbd" #18380

chenjacken opened this issue Oct 21, 2023 · 28 comments
Labels
question Further information is requested

Comments

@chenjacken
Copy link

chenjacken commented Oct 21, 2023 •
edited
Loading

一、版本
v3.10.6

二、root+ceph

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# ceph osd status
+----+-------+-------+-------+--------+---------+--------+---------+-----------+
| id |  host |  used | avail | wr ops | wr data | rd ops | rd data |   state   |
+----+-------+-------+-------+--------+---------+--------+---------+-----------+
| 0  | node6 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 1  | node5 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 2  | node7 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 3  | node4 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 4  | node4 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 5  | node9 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 6  | node8 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
+----+-------+-------+-------+--------+---------+--------+---------+-----------+

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# ceph osd pool ls detail
pool 2 'hwcloudpool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 274 pgp_num 262 pg_num_target 32 pgp_num_target 32 autoscale_mode on last_change 937 lfor 0/937/935 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd
        removed_snaps [1~3]

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# ceph osd tree
ID  CLASS WEIGHT  TYPE NAME      STATUS REWEIGHT PRI-AFF 
 -1       3.18352 root default                           
 -9       0.90958     host node4                         
  3  nvme 0.45479         osd.3      up  1.00000 1.00000 
  4  nvme 0.45479         osd.4      up  1.00000 1.00000 
 -5       0.45479     host node5                         
  1  nvme 0.45479         osd.1      up  1.00000 1.00000 
 -3       0.45479     host node6                         
  0  nvme 0.45479         osd.0      up  1.00000 1.00000 
 -7       0.45479     host node7                         
  2  nvme 0.45479         osd.2      up  1.00000 1.00000 
-13       0.45479     host node8                         
  6  nvme 0.45479         osd.6      up  1.00000 1.00000 
-11       0.45479     host node9                         
  5  nvme 0.45479         osd.5      up  1.00000 1.00000

三、平台存储关联rbd存储：
image

四、创建虚拟机时候报的信息：

image

五、命令创建镜像，是可以的

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd ls hwcloudpool
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd create --size 10240 hwcloudpool/test-image01
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd ls hwcloudpool
test-image01
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd info  hwcloudpool/test-image01
rbd image 'test-image01':
        size 10 GiB in 2560 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: e79d896e56fd
        block_name_prefix: rbd_data.e79d896e56fd
        format: 2
        features: layering
        op_features: 
        flags: 
        create_timestamp: Sun Oct 22 00:59:09 2023
        access_timestamp: Sun Oct 22 00:59:09 2023
        modify_timestamp: Sun Oct 22 00:59:09 2023

六、在web端创建硬盘，出现：调度失败的错误，日志：

[
    "No matching resources",
    "{\"error\":{\"code\":400,\"details\":\"Error #01: GetCandidatesByIds [b877f88b-5d8c-48bd-8d03-efd269104188]: object b877f88b-5d8c-48bd-8d03-efd269104188 not found\\n\",\"request\":{\"body\":\"{\\\"backup\\\":false,\\\"best_effort\\\":false,\\\"change_config\\\":false,\\\"count\\\":1,\\\"details\\\":false,\\\"disks\\\":[{\\\"backe...u_count\\\":0,\\\"vmem_size\\\":0}\",\"headers\":{\"Content-Length\":\"1156\",\"Content-Type\":\"application/json\",\"User-Agent\":\"yunioncloud-go/201708\",\"X-Auth-Token\":\"*\",\"X-Request-Id\":\"607ab5-52f075\",\"X-Yunion-Lang\":\"zh-CN\",\"X-Yunion-Parent-Id\":\"0.0\",\"X-Yunion-Peer-Service-Name\":\"compute_v2\",\"X-Yunion-Remote-Addr\":\"default-scheduler:30887\",\"X-Yunion-Span-Id\":\"0.0.0\",\"X-Yunion-Span-Name\":\"\",\"X-Yunion-Strace-Debug\":\"true\",\"X-Yunion-Strace-Id\":\"ae8e5d5a\"},\"method\":\"POST\",\"url\":\"https://default-scheduler:30887/scheduler\"}}}"
]
@chenjacken chenjacken added the question Further information is requested label Oct 21, 2023
@chenjacken
Copy link
Author

六、default-scheduler的日志[root@master1 ~]# kubectl -n onecloud logs default-scheduler-84bc8c87f4-p2fjs --since 10m -f

[info 2023-10-22 02:50:58 candidate.generalGetUpdateFunc.func1(default.go:93)] FullUpdateCounter: 1, update all baremetals.
[info 2023-10-22 02:50:58 cache.(*schedulerCache).loadObjects(cache.go:155)] Start load CandidateCache - Baremetals, period: 30s, ttl: 0s
[info 2023-10-22 02:50:58 cache.(*schedulerCache).loadObjects.func1(cache.go:159)] End load CandidateCache - Baremetals, elapsed 1.387069ms
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:239)] original schedtag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:263)] updated sched tag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:239)] original schedtag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:263)] updated sched tag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:239)] original schedtag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:263)] updated sched tag map[string]*compute.SchedtagConfig{}
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "fe071d8d-46d3-4266-835d-93365960ec18", filter: "host_status", is_ok: false, reason: "enable_status is 'false', expected 'true'", error: <nil>
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "9eb94b52-f91c-4082-8014-3709c1920fd7", filter: "host_status", is_ok: false, reason: "enable_status is 'false', expected 'true'", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.7","access_mac":"6c:0b:84:d7:1c:df","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2802,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T14:29:25.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node6-172-16-1-7","id":"91018292-660b-498f-8d8e-984afcd66625","imported_at":"2023-10-19T14:29:25.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:29:13.000000Z","manager_uri":"https://172.16.1.7:8885","mem_cmtbound":1,"mem_reserved":13155,"mem_size":64355,"name":"node6-172-16-1-7","node_count":2,"ovn_mapped_ip_addr":"100.64.0.131","ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC0DWDVW","source":"local","status":"running","storage_driver":"Linux","storage_size":443695,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":50,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20221208.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC0DWDVW","storage_type":"ssd","topology":{"architecture":"numa","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":1,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":11,"index":2,"logical_processors":[10,38],"total_threads":2},{"id":12,"index":3,"logical_processors":[11,39],"total_threads":2},{"id":13,"index":4,"logical_processors":[12,40],"total_threads":2},{"id":14,"index":5,"logical_processors":[13,41],"total_threads":2},{"id":2,"index":6,"logical_processors":[2,30],"total_threads":2},{"id":3,"index":7,"logical_processors":[3,31],"total_threads":2},{"id":4,"index":8,"logical_processors":[32,4],"total_threads":2},{"id":5,"index":9,"logical_processors":[33,5],"total_threads":2},{"id":6,"index":10,"logical_processors":[34,6],"total_threads":2},{"id":8,"index":11,"logical_processors":[35,7],"total_threads":2},{"id":9,"index":12,"logical_processors":[36,8],"total_threads":2},{"id":10,"index":13,"logical_processors":[37,9],"total_threads":2}],"distances":[10,21],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":34359738368,"total_usable_bytes":33665085440}},{"caches":[{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":1,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":2,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":3,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":4,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":5,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":6,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":8,"index":7,"logical_processors":[21,49],"total_threads":2},{"id":9,"index":8,"logical_processors":[22,50],"total_threads":2},{"id":10,"index":9,"logical_processors":[23,51],"total_threads":2},{"id":11,"index":10,"logical_processors":[24,52],"total_threads":2},{"id":12,"index":11,"logical_processors":[25,53],"total_threads":2},{"id":13,"index":12,"logical_processors":[26,54],"total_threads":2},{"id":14,"index":13,"logical_processors":[27,55],"total_threads":2}],"distances":[21,10],"id":1,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":34359738368,"total_usable_bytes":33816535040}}]},"version":"70FR003JCN"},"update_version":656,"updated_at":"2023-10-20T01:29:13.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "91018292-660b-498f-8d8e-984afcd66625", filter: "host_memory", is_ok: false, reason: "no enough resource: node6-172-16-1-7, requested: 8192, total: 64355, free: 6144", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.11","access_mac":"6c:0b:84:11:01:4c","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":1959,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T21:25:14.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node7-172-16-1-11","id":"d2805334-011a-4ff8-876b-e4026d352a00","imported_at":"2023-10-19T21:25:14.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:35.000000Z","manager_uri":"https://172.16.1.11:8885","mem_cmtbound":1,"mem_reserved":18692,"mem_size":96516,"name":"node7-172-16-1-11","node_count":2,"ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC07HBGC","source":"local","status":"running","storage_driver":"Linux","storage_size":471791,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":76,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20230805.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC07HBGC","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":103079215104,"total_usable_bytes":101204516864}}]},"version":"70FR0017CN"},"update_version":247,"updated_at":"2023-10-20T01:28:35.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.5","access_mac":"6c:0b:84:81:ca:7d","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2389,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T21:01:04.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node4-172-16-1-5","id":"9e4c3c93-9605-4cc9-84ba-01c52157e78b","imported_at":"2023-10-19T21:01:04.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:28.000000Z","manager_uri":"https://172.16.1.5:8885","mem_cmtbound":1,"mem_reserved":13060,"mem_size":64260,"name":"node4-172-16-1-5","node_count":2,"ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC07L7XZ","source":"local","status":"running","storage_driver":"Linux","storage_size":443695,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":50,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20230805.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC07L7XZ","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]},"version":"70FR0017CN"},"update_version":271,"updated_at":"2023-10-20T01:28:28.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "9e4c3c93-9605-4cc9-84ba-01c52157e78b", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "d2805334-011a-4ff8-876b-e4026d352a00", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.10","access_mac":"6c:0b:84:bd:ec:cb","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2898,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T07:19:12.000000Z","deleted":false,"description":"主节点3","domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"master3-172-16-1-10","id":"9e438931-4ce8-4ed2-83e0-3a67f8b755d1","imported_at":"2023-10-19T07:19:12.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:50.000000Z","manager_uri":"https://172.16.1.10:8885","mem_cmtbound":1,"mem_reserved":10340,"mem_size":128772,"name":"master3-172-16-1-10","node_count":2,"ovn_mapped_ip_addr":"100.64.0.130","ovn_version":"2.12.4-2","page_size_kb":4,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC0AA8CL","source":"local","status":"running","storage_driver":"Linux","storage_size":471791,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_size_kb":0,"hugepages_option":"transparent","kernel_version":"5.4.130-1.yn20221208.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC0AA8CL","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":137438953472,"total_usable_bytes":135027351552}}]},"version":"70FR001GCN"},"update_version":1083,"updated_at":"2023-10-20T01:28:50.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "9e438931-4ce8-4ed2-83e0-3a67f8b755d1", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.6","access_mac":"6c:0b:84:81:d3:08","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2394,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T21:16:03.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node5-172-16-1-6","id":"0ec8e63a-10c1-4028-897c-1942435bcd65","imported_at":"2023-10-19T21:16:03.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:28.000000Z","manager_uri":"https://172.16.1.6:8885","mem_cmtbound":1,"mem_reserved":18692,"mem_size":96516,"name":"node5-172-16-1-6","node_count":2,"ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC0849AN","source":"local","status":"running","storage_driver":"Linux","storage_size":471791,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":76,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20230805.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC0849AN","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":103079215104,"total_usable_bytes":101204717568}}]},"version":"70FRR00DCN"},"update_version":256,"updated_at":"2023-10-20T01:28:28.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "0ec8e63a-10c1-4028-897c-1942435bcd65", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[info 2023-10-22 02:51:00 handler.timer.func1(handler.go:55)] Handler "/scheduler/forecast" cost: 56.267904ms
[info 2023-10-22 02:51:06 schedtag.(*dataManager).syncOnce(sync.go:118)] Schedtag data start sync
[info 2023-10-22 02:51:06 schedtag.(*dataManager).syncOnce(sync.go:135)] Schedtag finish sync, elapsed 5.949985ms
[info 2023-10-22 02:51:36 schedtag.(*dataManager).syncOnce(sync.go:118)] Schedtag data start sync
[info 2023-10-22 02:51:36 schedtag.(*dataManager).syncOnce(sync.go:135)] Schedtag finish sync, elapsed 6.369307ms
[info 2023-10-22 02:52:06 schedtag.(*dataManager).syncOnce(sync.go:118)] Schedtag data start sync
[info 2023-10-22 02:52:06 schedtag.(*dataManager).syncOnce(sync.go:135)] Schedtag finish sync, elapsed 5.699228ms
[info 2023-10-22 02:52:10 candidate.generalGetUpdateFunc.func1(default.go:99)] FullUpdateCounter: 1, update all hosts.
[info 2023-10-22 02:52:10 cache.(*schedulerCache).loadObjects(cache.go:155)] Start load CandidateCache - Hosts, period: 30s, ttl: 0s
[info 2023-10-22 02:52:10 cache.(*schedulerCache).loadObjects.func1(cache.go:159)] End load CandidateCache - Hosts, elapsed 36.242898ms

@zexi
Copy link
Member

zexi commented Oct 22, 2023

@chenjacken 这个存储有没有绑定到宿主机上？

@chenjacken
Copy link
Author

@chenjacken 这个存储有没有绑定到宿主机上？

都绑定到了所有的计算节点和三个控制节点的宿主机。

@chenjacken
Copy link
Author

chenjacken commented Oct 22, 2023 •
edited
Loading

发现一个问题：
1，创建虚拟机就报那个错误
2，直接创建一个硬盘，不指定块存储就报如上的那个错误调度失败，如果选择了指定的块存储，硬盘会创建成功。

image image image

调度失败日志：

[
    "No matching resources",
    "{\"error\":{\"code\":400,\"details\":\"Error #01: genericScheduler.Schedule: No resource are avaliable that match all of the following predicates: filter by host_status(-2), host_storage(-5), session_id=\\\"1697945758843\\\"\\n\",\"request\":{\"body\":\"{\\\"backup\\\":false,\\\"best_effort\\\":false,\\\"change_config\\\":false,\\\"count\\\":1,\\\"details\\\":false,\\\"disks\\\":[{\\\"backe...u_count\\\":0,\\\"vmem_size\\\":0}\",\"headers\":{\"Content-Length\":\"1100\",\"Content-Type\":\"application/json\",\"User-Agent\":\"yunioncloud-go/201708\",\"X-Auth-Token\":\"*\",\"X-Request-Id\":\"d0b8b5-1094bc\",\"X-Yunion-Lang\":\"zh-CN\",\"X-Yunion-Parent-Id\":\"0.0\",\"X-Yunion-Peer-Service-Name\":\"compute_v2\",\"X-Yunion-Remote-Addr\":\"default-scheduler:30887\",\"X-Yunion-Span-Id\":\"0.0.0\",\"X-Yunion-Span-Name\":\"\",\"X-Yunion-Strace-Debug\":\"true\",\"X-Yunion-Strace-Id\":\"4160b767\"},\"method\":\"POST\",\"url\":\"https://default-scheduler:30887/scheduler\"}}}"
]

@zexi
Copy link
Member

zexi commented Oct 22, 2023

[
"No matching resources",
"{"error":{"code":400,"details":"Error #1: genericScheduler.Schedule: No resource are avaliable that match all of the following predicates: filter by host_status(-2), host_storage(-5), session_id=\"1697945758843\"\n","request":
]

@chenjacken 使用下面的命令看下调度失败详细日志：

# 1697945758843 是上面日志里面的 session_id
climc scheduler-history-show --log --raw 1697945758843

@chenjacken
Copy link
Author

[
"No matching resources",
"{"error":{"code":400,"details":"Error #1: genericScheduler.Schedule: No resource are avaliable that match all of the following predicates: filter by host_status(-2), host_storage(-5), session_id="1697945758843"\n","request":
]

@chenjacken 使用下面的命令看下调度失败详细日志：

# 1697945758843 是上面日志里面的 session_id
climc scheduler-history-show --log --raw 1697945758843
[root@master1 ~]# climc scheduler-history-show --log --raw 1697945758843
consuming: 0s
error: 'genericScheduler.Schedule: No resource are avaliable that match all of the
  following predicates: filter by host_status(-2), host_storage(-5), session_id="1697945758843"'
input: '{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","index":0,"medium":"hybrid","size":10240}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","live_migrate":false,"name":"test-image","pending_usages":[{"brand":"OneCloud","cloud_env":"onpremise","count":0,"cpu":0,"deleted":false,"domain_id":"default","group":0,"hypervisor":"kvm","isolated_device":0,"memory":0,"provider":"OneCloud","region_id":"default","storage":10240,"tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","update_version":0,"zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},{"bucket":0,"cache":0,"deleted":false,"eip":0,"eport":0,"instance_snapshot":0,"loadbalancer":0,"mongodb":0,"object_cnt":0,"object_gb":0,"port":0,"rds":0,"snapshot":0,"update_version":0}],"prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"suggestion":false,"suggestion_all":false,"suggestion_limit":100,"vcpu_count":0,"vmem_size":0}'
output: ""
session_id: "1697945758843"
tasks:
- capacity_map:
    0ec8e63a-10c1-4028-897c-1942435bcd65:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    91018292-660b-498f-8d8e-984afcd66625:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    9e438931-4ce8-4ed2-83e0-3a67f8b755d1:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    9e4c3c93-9605-4cc9-84ba-01c52157e78b:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    9eb94b52-f91c-4082-8014-3709c1920fd7:
      MinValue: -1
      Values:
        host_status:
          Value: 0
    d2805334-011a-4ff8-876b-e4026d352a00:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    fe071d8d-46d3-4266-835d-93365960ec18:
      MinValue: -1
      Values:
        host_status:
          Value: 0
  consuming: 2.606056ms
  data:
    __meta__: null
    ChangeConfig: false
    HasIsolatedDevice: false
    InstanceGroupsDetail: null
    PendingUsages:
    - {}
    - {}
    Raw: '{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","index":0,"medium":"hybrid","size":10240}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","live_migrate":false,"name":"test-image","pending_usages":[{"brand":"OneCloud","cloud_env":"onpremise","count":0,"cpu":0,"deleted":false,"domain_id":"default","group":0,"hypervisor":"kvm","isolated_device":0,"memory":0,"provider":"OneCloud","region_id":"default","storage":10240,"tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","update_version":0,"zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},{"bucket":0,"cache":0,"deleted":false,"eip":0,"eport":0,"instance_snapshot":0,"loadbalancer":0,"mongodb":0,"object_cnt":0,"object_gb":0,"port":0,"rds":0,"snapshot":0,"update_version":0}],"prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"suggestion":false,"suggestion_all":false,"suggestion_limit":100,"vcpu_count":0,"vmem_size":0}'
    UserCred:
      TokenCredential:
        Context:
          ip: 112.94.186.250
          source: web
        Domain: Default
        DomainId: default
        Expires: "2023-10-22T17:01:51Z"
        ProjectDomain: Default
        ProjectDomainId: default
        RoleIds: 849fb962d16a4dc78e8c93458aa57a1a
        Roles: admin
        Token: 4e1fc4931efe9490deba0a4d07137d8079291bec25820a819f7da985dddc73ce
        User: admin
        UserId: 4876b8260fd9493e858f9efa6b3c45a6
        tenant: system
        tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
    backup: false
    baremetal_disk_configs:
    - conf: none
      count: 0
      driver: ""
      range: null
      size: null
      splits: ""
      type: hybrid
    best_effort: false
    candidates: []
    cdrom: ""
    count: 1
    cpu_desc: ""
    cpu_microcode: ""
    cpu_mode: ""
    details: false
    disks:
    - backend: rbd
      backup_id: ""
      boot_index: null
      cache: ""
      disk_id: ""
      disk_type: ""
      driver: ""
      existing_path: ""
      format: ""
      fs: ""
      image_encrypt_key_id: ""
      image_id: ""
      image_properties: null
      index: 0
      medium: hybrid
      mountpoint: ""
      nvme_device: null
      os_arch: ""
      schedtags: []
      size: 10240
      snapshot_id: ""
      storage_id: ""
    domain_id: default
    for_guests: null
    group_releations: null
    groups: null
    guest_status: ""
    host_id: ""
    host_mem_page_size: 0
    hypervisor: hypervisor
    id: ""
    ignore_filters: {}
    instance_backup_id: ""
    instance_snapshot_id: ""
    instance_type: ""
    is_container: false
    is_daemon: null
    isolated_devices: null
    live_migrate: false
    name: test-image
    nets: null
    os_arch: ""
    prefer_backup_host: ""
    prefer_host_id: ""
    prefer_manager_id: ""
    prefer_region_id: default
    prefer_wire_id: ""
    prefer_zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
    project_id: 43d375dfbfc1426d829bf8e28b8c3889
    record_to_history: false
    required_candidates: 1
    resource_type: shared
    reuse_network: false
    schedtags: []
    session_id: "1697945758843"
    skip_cpu_check: null
    skip_kernel_check: null
    sku: ""
    suggestion: false
    suggestion_all: false
    suggestion_details: false
    suggestion_limit: 100
    tag: ""
    target_host_kernel: ""
    type: ""
    vcpu_count: 0
    vmem_size: 0
  error: 'genericScheduler.Schedule: No resource are avaliable that match all of the
    following predicates: filter by host_status(-2), host_storage(-5), session_id="1697945758843"'
  logs:
  - 'Failed: master1-172-16-1-8:fe071d8d-46d3-4266-835d-93365960ec18 [host_status]
    host_status: enable_status is ''false'', expected ''true'''
  - 'Failed: master2-172-16-1-9:9eb94b52-f91c-4082-8014-3709c1920fd7 [host_status]
    host_status: enable_status is ''false'', expected ''true'''
  - 'Success: master3-172-16-1-10:9e438931-4ce8-4ed2-83e0-3a67f8b755d1 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: master3-172-16-1-10:9e438931-4ce8-4ed2-83e0-3a67f8b755d1 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node4-172-16-1-5:9e4c3c93-9605-4cc9-84ba-01c52157e78b [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node4-172-16-1-5:9e4c3c93-9605-4cc9-84ba-01c52157e78b [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node5-172-16-1-6:0ec8e63a-10c1-4028-897c-1942435bcd65 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node5-172-16-1-6:0ec8e63a-10c1-4028-897c-1942435bcd65 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node6-172-16-1-7:91018292-660b-498f-8d8e-984afcd66625 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node6-172-16-1-7:91018292-660b-498f-8d8e-984afcd66625 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node7-172-16-1-11:d2805334-011a-4ff8-876b-e4026d352a00 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node7-172-16-1-11:d2805334-011a-4ff8-876b-e4026d352a00 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  result: null
  status: failed
  time: "2023-10-22 03:35:58"
  type: hypervisor
time: "2023-10-22 03:35:58"

@zexi
Copy link
Member

zexi commented Oct 22, 2023

@chenjacken 看下存储那里的介质类型，看报错是没有找到 backend 为 rbd ，mediumType 为 hybrid 的存储

host_storage: no enough backend "rbd", mediumType "hybrid" storage

另外用 climc storage-list 看下每个存储的属性:

climc storage-list

@chenjacken
Copy link
Author

@chenjacken 看下存储那里的介质类型，看报错是没有找到 backend 为 rbd ，mediumType 为 hybrid 的存储

host_storage: no enough backend "rbd", mediumType "hybrid" storage

另外用 climc storage-list 看下每个存储的属性:

climc storage-list

是这个问题，所以创建硬盘时候指定存储块是成功的。

[root@master1 ~]# climc storage-list
+--------------------------------------+----------------------------------+----------+----------------------+--------+--------------+-------------+---------+--------------+
|                  ID                  |               Name               | Capacity | Actual_capacity_used | Status | Storage_type | Medium_type | Enabled | public_scope |
+--------------------------------------+----------------------------------+----------+----------------------+--------+--------------+-------------+---------+--------------+
| 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | 1054767  | 0                    | online | rbd          | hybrid      | true    | system       |
| 28a37de9-b491-4a16-8628-a282bccee8a8 | host_172.16.1.16_local_storage_0 | 455143   | 10128                | online | local        | ssd         | true    | system       |
| 1debb826-1d79-442e-8cf2-1dbd0cafdd60 | host_172.16.1.15_local_storage_0 | 459823   | 10120                | online | local        | ssd         | true    | system       |
| af307604-7179-4d51-8a9b-3dd68874e049 | host_172.16.1.11_local_storage_0 | 471791   | 24961                | online | local        | ssd         | true    | system       |
| 2d2cabfa-70d1-4229-8499-5fb1687e9f08 | host_172.16.1.6_local_storage_0  | 471791   | 10812                | online | local        | ssd         | true    | system       |
| c76bea60-b48b-4d49-89fa-72399efe1b06 | host_172.16.1.5_local_storage_0  | 443695   | 10650                | online | local        | ssd         | true    | system       |
| 7a36954f-44aa-47eb-88a3-4bb62cc398a9 | host_172.16.1.7_local_storage_0  | 443695   | 126198               | online | local        | ssd         | true    | system       |
| b11390f9-59db-4d9c-8608-d3f9fdb04886 | host_172.16.1.8_local_storage_0  | 443695   | 85723                | online | local        | ssd         | true    | system       |
| 6153222e-d59b-46ff-8f3a-cca7a804dc6b | host_172.16.1.9_local_storage_0  | 443695   | 228956               | online | local        | ssd         | true    | system       |
| 293ab9ee-952a-41cf-8a8c-a6d9098d33b9 | host_172.16.1.10_local_storage_0 | 471791   | 239840               | online | local        | ssd         | true    | system       |
+--------------------------------------+----------------------------------+----------+----------------------+--------+--------------+-------------+---------+--------------+
***  Total: 10 Pages: 1 Limit: 20 Offset: 0 Page: 1  ***

@chenjacken
Copy link
Author

@zexi 帮忙看看，会是哪里出问题了？

@zexi
Copy link
Member

zexi commented Oct 23, 2023

@chenjacken 执行 climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b ，把结果发下

@chenjacken
Copy link
Author

chenjacken commented Oct 23, 2023 •
edited
Loading 

[root@master1 ~]# climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b
access_ip: 172.16.1.5
access_mac: 6c:0b:84:81:ca:7d
auto_renew: false
billing_type: postpaid
cpu_architecture: x86_64
cpu_bound_count: 0
cpu_cache: 35840
cpu_cmtbound: 8
cpu_count: 56
cpu_desc: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
cpu_mhz: 2389
cpu_microcode: "0xb00001e"
cpu_reserved: 0
created_at: "2023-10-19T21:01:04.000000Z"
creating_cpu_count: 0
creating_guest_count: 0
creating_mem_size: 0
deleted: false
domain_id: default
domain_src: local
enable_health_check: true
enable_pxe_boot: true
enabled: true
fake_deleted_cpu_count: 0
fake_deleted_mem_size: 0
free_cpu_count: 440
free_mem_size: 43008
guest_count: 0
guest_reserved_resource:
  cpu_count: 8
  memory_size: 8192
  storage_size: 102400
guest_reserved_used:
  cpu_count: 0
  memory_size: 0
  storage_size: 0
host_status: online
host_type: hypervisor
hostname: node4-172-16-1-5
id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
imported_at: "2023-10-19T21:01:04.000000Z"
io_bound_count: 0
ipmi_info:
  present: false
is_baremetal: false
is_emulated: false
is_import: false
is_maintenance: false
is_public: true
isolated_devices:
- addr: "04:00.0"
  dev_type: GPU-HPC
  host_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
  id: fe5cf8b8-c42b-4d2d-8e06-b4d60d6f4e59
  model: GeForce RTX 3090
  vendor_device_id: 10de:2204
last_ping_at: "2023-10-20T01:28:28.000000Z"
manager_uri: https://172.16.1.5:8885
mem_cmtbound: 1
mem_reserved: 13060
mem_size: 64260
metadata:
  __auto_migrate_on_host_down: enable
  __auto_migrate_on_host_shutdown: enable
  __host_errors: '[{"content":"GPU 07:00.0 use kernel driver ast, skip it","name":"
    ","time":"2023-10-19T21:01:05.612871Z","type":"isolated_devices"}]'
  cpu_info: '{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56}'
  generate_name: node4-172-16-1-5
  hostname: node4-172-16-1-5
  hugepage_nr: "50"
  hugepage_size_kb: "1048576"
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  memory_used_mb: "54058"
  model: RD450X
  nest: enabled
  oem_name: lenovo
  on_kubernetes: "true"
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  root_partition_total_capacity_mb: "443695"
  root_partition_used_capacity_mb: "6957"
  sn: PC07L7XZ
  storage_type: ssd
  topology: '{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]}'
  version: 70FR0017CN
name: node4-172-16-1-5
net_interfaces:
  62d395c7-8a5c-4ea1-8741-1ccef5f521ca:
  - baremetal_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
    index: 0
    link_up: true
    mac: 6c:0b:84:81:ca:7d
    mtu: 0
    nic_type: admin
    rate: 0
    wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
networks:
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:16:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.6
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.6
  guest_ip6_mask: 0
  id: dab40ba9-01c7-48ad-8c16-a7ff708a36e8
  ifname_hint: adm023
  imported_at: "2023-10-19T21:16:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm023
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:12.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.10
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.10
  guest_ip6_mask: 0
  id: 3a632875-3c85-4056-8b10-8d2b9d521537
  ifname_hint: adm01
  imported_at: "2023-10-19T07:19:12.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm01
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:33:10.000000Z"
  deleted: false
  domain_id: default
  free_port: 51
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.200
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.150
  guest_ip6_mask: 0
  id: 51b8fbd8-9b90-48e6-8369-33fc86e58717
  ifname_hint: DHCP1
  imported_at: "2023-10-19T08:33:10.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: DHCP1
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T14:29:25.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.7
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.7
  guest_ip6_mask: 0
  id: ff1242fc-7b3f-42e8-878d-a4dfc562d6c0
  ifname_hint: adm021
  imported_at: "2023-10-19T14:29:25.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm021
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:00.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.8
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.8
  guest_ip6_mask: 0
  id: 0b27e637-cefa-4735-86fe-201116d34fa8
  ifname_hint: adm0
  imported_at: "2023-10-19T07:19:00.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm0
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: none
  public_src: local
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:01:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.5
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.5
  guest_ip6_mask: 0
  id: 0c973695-9738-403d-88ec-8620151137ce
  ifname_hint: adm022
  imported_at: "2023-10-19T21:01:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm022
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:30:40.000000Z"
  deleted: false
  description: 手动指定静态IP,自动分配请勿使用该子网。
  domain_id: default
  free_port: 15
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.90
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.70
  guest_ip6_mask: 0
  id: 7e49c4fd-0c08-49b3-8e2a-983bb8f8a0e2
  ifname_hint: HOST
  imported_at: "2023-10-19T08:30:40.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: Static
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:19.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.9
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.9
  guest_ip6_mask: 0
  id: 0a596e84-89ef-4f94-8c06-20b1f71449ed
  ifname_hint: adm02
  imported_at: "2023-10-19T07:19:19.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm02
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:25:14.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.11
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.11
  guest_ip6_mask: 0
  id: 8e82abdc-7e5f-45a2-8627-0b164ed3ed77
  ifname_hint: adm024
  imported_at: "2023-10-19T21:25:14.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm024
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
nics:
- link_up: true
  mac: 00:1b:21:bc:79:07
  masklen: 0
  mtu: 0
  rate: 0
- dns: 223.5.5.5,223.6.6.6
  gateway: 172.16.1.1
  ip_addr: 172.16.1.5
  link_up: true
  mac: 6c:0b:84:81:ca:7d
  masklen: 24
  mtu: 0
  net: adm022
  net_id: 0c973695-9738-403d-88ec-8620151137ce
  nic_type: admin
  rate: 0
  wire: bcast0
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- link_up: true
  mac: 6c:0b:84:81:ca:7e
  masklen: 0
  mtu: 0
  rate: 0
node_count: 2
ovn_version: 2.12.4-2
page_size_kb: 1048576
pending_usage:
  cpu: 0
  disk: {}
  instance_groups: {}
  isolated_device: 0
  memory: 0
  net: {}
progress: 100
public_scope: system
rack: rack0
region:
  created_at: "2023-10-19T06:12:20.000000Z"
  deleted: false
  description: Default Region
  enabled: true
  id: default
  imported_at: "2023-10-19T06:12:20.000000Z"
  is_emulated: false
  latitude: 0
  longitude: 0
  name: Default
  progress: 100
  provider: OneCloud
  source: local
  status: inservice
  update_version: 0
  updated_at: "2023-10-19T06:12:20.000000Z"
required_cpu_count: 0
required_mem_size: 0
resource_type: shared
running_cpu_count: 0
running_guest_count: 0
running_mem_size: 0
slots: slot0
sn: PC07L7XZ
source: local
status: running
storage_driver: Linux
storage_size: 443695
storage_type: ssd
storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
sys_info:
  cpu_info:
...//more
      id: 0
      memory:
        supported_page_sizes:
        - 1073741824
        total_physical_bytes: 68719476736
        total_usable_bytes: 67381719040
  version: 70FR0017CN
total_cpu_count: 448
total_mem_size: 64260
update_version: 271
updated_at: "2023-10-20T01:28:28.000000Z"
version: release/3.10(978c65dbc323101517)
zone:
  cloudregion_id: default
  created_at: "2023-10-19T06:12:29.000000Z"
  deleted: false
  id: d7754248-9f87-4e31-81a9-517c7b0ef852
  imported_at: "2023-10-19T06:12:29.000000Z"
  is_emulated: false
  name: zone0
  progress: 100
  source: local
  status: enable
  update_version: 0
  updated_at: "2023-10-19T06:12:29.000000Z"
zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852

@zexi
Copy link
Member

zexi commented Oct 23, 2023

@chenjacken 我看这台宿主机没有关联上这个存储，只有一个 local 的

storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852

执行下：climc host-storage-list --details 再看下这个 ceph 存储有没有和相关的宿主机挂载上

@zexi
Copy link
Member

zexi commented Oct 23, 2023

@chenjacken 我看了日志，发现调度服务没有查询到关联的 rbd 存储，用下面的命令能查询出来吗？

climc host-storage-list --host 9e4c3c93-9605-4cc9-84ba-01c52157e78b

@chenjacken
Copy link
Author

@chenjacken 我看了日志，发现调度服务没有查询到关联的 rbd 存储，用下面的命令能查询出来吗？

climc host-storage-list --host 9e4c3c93-9605-4cc9-84ba-01c52157e78b

命令是能查询到。

[root@master1 ~]# climc host-storage-list --host 9e4c3c93-9605-4cc9-84ba-01c52157e78b
+--------------------------------------+--------------------------------------+----------------------------+
|               Host_ID                |              Storage_ID              |        Mount_point         |
+--------------------------------------+--------------------------------------+----------------------------+
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | 77008084-6a7a-42c0-8687-7c8b9222160a | rbd:hwcloudpool            |
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | c76bea60-b48b-4d49-89fa-72399efe1b06 | /opt/cloud/workspace/disks |
+--------------------------------------+--------------------------------------+----------------------------+
***  Total: 2 Pages: 1 Limit: 20 Offset: 0 Page: 1  ***
[root@master1 ~]#

@chenjacken
Copy link
Author

是否可以重置或者替换POD镜像来解决问题？
或者是否数据库存在混乱的数据导致？

@zexi
Copy link
Member

zexi commented Oct 23, 2023

是否可以重置或者替换POD镜像来解决问题？ 或者是否数据库存在混乱的数据导致？

@chenjacken 这个问题看起来有点奇怪，有点像代码层面的问题，我又改了一个版本，用这个版本的调度器试试：registry.cn-beijing.aliyuncs.com/yunionio/scheduler:v3.10-lzx-debug-storages.0，然后再把日志发一份出来

@chenjacken
Copy link
Author

chenjacken commented Oct 23, 2023 •
edited
Loading

default-scheduler-3.log

谢谢~~
climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b的日志：

[root@master1 ~]# climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b
access_ip: 172.16.1.5
access_mac: 6c:0b:84:81:ca:7d
auto_renew: false
billing_type: postpaid
cpu_architecture: x86_64
cpu_bound_count: 0
cpu_cache: 35840
cpu_cmtbound: 8
cpu_count: 56
cpu_desc: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
cpu_mhz: 2389
cpu_microcode: "0xb00001e"
cpu_reserved: 0
created_at: "2023-10-19T21:01:04.000000Z"
creating_cpu_count: 0
creating_guest_count: 0
creating_mem_size: 0
deleted: false
domain_id: default
domain_src: local
enable_health_check: true
enable_pxe_boot: true
enabled: true
fake_deleted_cpu_count: 0
fake_deleted_mem_size: 0
free_cpu_count: 440
free_mem_size: 43008
guest_count: 0
guest_reserved_resource:
  cpu_count: 8
  memory_size: 8192
  storage_size: 102400
guest_reserved_used:
  cpu_count: 0
  memory_size: 0
  storage_size: 0
host_status: online
host_type: hypervisor
hostname: node4-172-16-1-5
id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
imported_at: "2023-10-19T21:01:04.000000Z"
io_bound_count: 0
ipmi_info:
  present: false
is_baremetal: false
is_emulated: false
is_import: false
is_maintenance: false
is_public: true
isolated_devices:
- addr: "04:00.0"
  dev_type: GPU-HPC
  host_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
  id: fe5cf8b8-c42b-4d2d-8e06-b4d60d6f4e59
  model: GeForce RTX 3090
  vendor_device_id: 10de:2204
last_ping_at: "2023-10-20T01:28:28.000000Z"
manager_uri: https://172.16.1.5:8885
mem_cmtbound: 1
mem_reserved: 13060
mem_size: 64260
metadata:
  __auto_migrate_on_host_down: enable
  __auto_migrate_on_host_shutdown: enable
  __host_errors: '[{"content":"GPU 07:00.0 use kernel driver ast, skip it","name":"
    ","time":"2023-10-19T21:01:05.612871Z","type":"isolated_devices"}]'
  cpu_info: '{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56}'
  generate_name: node4-172-16-1-5
  hostname: node4-172-16-1-5
  hugepage_nr: "50"
  hugepage_size_kb: "1048576"
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  memory_used_mb: "54058"
  model: RD450X
  nest: enabled
  oem_name: lenovo
  on_kubernetes: "true"
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  root_partition_total_capacity_mb: "443695"
  root_partition_used_capacity_mb: "6957"
  sn: PC07L7XZ
  storage_type: ssd
  topology: '{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]}'
  version: 70FR0017CN
name: node4-172-16-1-5
net_interfaces:
  62d395c7-8a5c-4ea1-8741-1ccef5f521ca:
  - baremetal_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
    index: 0
    link_up: true
    mac: 6c:0b:84:81:ca:7d
    mtu: 0
    nic_type: admin
    rate: 0
    wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
networks:
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:00.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.8
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.8
  guest_ip6_mask: 0
  id: 0b27e637-cefa-4735-86fe-201116d34fa8
  ifname_hint: adm0
  imported_at: "2023-10-19T07:19:00.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm0
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: none
  public_src: local
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:12.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.10
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.10
  guest_ip6_mask: 0
  id: 3a632875-3c85-4056-8b10-8d2b9d521537
  ifname_hint: adm01
  imported_at: "2023-10-19T07:19:12.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm01
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:25:14.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.11
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.11
  guest_ip6_mask: 0
  id: 8e82abdc-7e5f-45a2-8627-0b164ed3ed77
  ifname_hint: adm024
  imported_at: "2023-10-19T21:25:14.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm024
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:30:40.000000Z"
  deleted: false
  description: 手动指定静态IP,自动分配请勿使用该子网。
  domain_id: default
  free_port: 15
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.90
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.70
  guest_ip6_mask: 0
  id: 7e49c4fd-0c08-49b3-8e2a-983bb8f8a0e2
  ifname_hint: HOST
  imported_at: "2023-10-19T08:30:40.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: Static
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:16:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.6
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.6
  guest_ip6_mask: 0
  id: dab40ba9-01c7-48ad-8c16-a7ff708a36e8
  ifname_hint: adm023
  imported_at: "2023-10-19T21:16:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm023
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T14:29:25.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.7
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.7
  guest_ip6_mask: 0
  id: ff1242fc-7b3f-42e8-878d-a4dfc562d6c0
  ifname_hint: adm021
  imported_at: "2023-10-19T14:29:25.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm021
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:19.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.9
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.9
  guest_ip6_mask: 0
  id: 0a596e84-89ef-4f94-8c06-20b1f71449ed
  ifname_hint: adm02
  imported_at: "2023-10-19T07:19:19.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm02
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:01:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.5
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.5
  guest_ip6_mask: 0
  id: 0c973695-9738-403d-88ec-8620151137ce
  ifname_hint: adm022
  imported_at: "2023-10-19T21:01:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm022
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:33:10.000000Z"
  deleted: false
  domain_id: default
  free_port: 51
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.200
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.150
  guest_ip6_mask: 0
  id: 51b8fbd8-9b90-48e6-8369-33fc86e58717
  ifname_hint: DHCP1
  imported_at: "2023-10-19T08:33:10.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: DHCP1
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
nics:
- link_up: true
  mac: 00:1b:21:bc:79:07
  masklen: 0
  mtu: 0
  rate: 0
- dns: 223.5.5.5,223.6.6.6
  gateway: 172.16.1.1
  ip_addr: 172.16.1.5
  link_up: true
  mac: 6c:0b:84:81:ca:7d
  masklen: 24
  mtu: 0
  net: adm022
  net_id: 0c973695-9738-403d-88ec-8620151137ce
  nic_type: admin
  rate: 0
  wire: bcast0
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- link_up: true
  mac: 6c:0b:84:81:ca:7e
  masklen: 0
  mtu: 0
  rate: 0
node_count: 2
ovn_version: 2.12.4-2
page_size_kb: 1048576
pending_usage:
  cpu: 0
  disk: {}
  instance_groups: {}
  isolated_device: 0
  memory: 0
  net: {}
progress: 100
public_scope: system
rack: rack0
region:
  created_at: "2023-10-19T06:12:20.000000Z"
  deleted: false
  description: Default Region
  enabled: true
  id: default
  imported_at: "2023-10-19T06:12:20.000000Z"
  is_emulated: false
  latitude: 0
  longitude: 0
  name: Default
  progress: 100
  provider: OneCloud
  source: local
  status: inservice
  update_version: 0
  updated_at: "2023-10-19T06:12:20.000000Z"
required_cpu_count: 0
required_mem_size: 0
resource_type: shared
running_cpu_count: 0
running_guest_count: 0
running_mem_size: 0
slots: slot0
sn: PC07L7XZ
source: local
status: running
storage_driver: Linux
storage_size: 443695
storage_type: ssd
storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
sys_info:

...//more

      id: 1
      model: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
      total_cores: 14
      total_threads: 28
      vendor: GenuineIntel
    total_cores: 28
    total_threads: 56
  hugepage_nr: 50
  hugepage_size_kb: 1048576
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  model: RD450X
  nest: enabled
  oem_name: lenovo
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  sn: PC07L7XZ
  storage_type: ssd
  topology:
    architecture: smp
    nodes:

...//more

      memory:
        supported_page_sizes:
        - 1073741824
        total_physical_bytes: 68719476736
        total_usable_bytes: 67381719040
  version: 70FR0017CN
total_cpu_count: 448
total_mem_size: 64260
update_version: 271
updated_at: "2023-10-20T01:28:28.000000Z"
version: release/3.10(978c65dbc323101517)
zone:
  cloudregion_id: default
  created_at: "2023-10-19T06:12:29.000000Z"
  deleted: false
  id: d7754248-9f87-4e31-81a9-517c7b0ef852
  imported_at: "2023-10-19T06:12:29.000000Z"
  is_emulated: false
  name: zone0
  progress: 100
  source: local
  status: enable
  update_version: 0
  updated_at: "2023-10-19T06:12:29.000000Z"
zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852

[root@master1 ~]#

@chenjacken
Copy link
Author

image

@zexi
Copy link
Member

zexi commented Oct 23, 2023

@chenjacken 这个问题看起来是调度器scheduler查询到的数据和控制服务region不一致导致的，应该不是代码问题了。
你环境的数据库有没有做高可用同步？会不会是数据库两边数据不一致了？

@chenjacken
Copy link
Author

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@zexi
Copy link
Member

zexi commented Oct 23, 2023 •
edited
Loading

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@chenjacken 那应该是数据库的问题了，之前也遇到过数据不一致的问题，按照这个文档排查下：https://www.cloudpods.org/zh/docs/ops/databases/mysql_ha_recovery/

@chenjacken
Copy link
Author

chenjacken commented Oct 23, 2023 •
edited
Loading

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@chenjacken 那应该是数据库的问题了，之前也遇到过数据不一致的问题，按照这个文档排查下：https://www.cloudpods.org/zh/docs/ops/databases/mysql_ha_recovery/

谢谢。我排查下数据库的问题。

Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; 
Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; 
the event's master log mysql-bin.000003, end_log_pos 1031126226

1，数据库1节点：

MariaDB [(none)]> SHOW MASTER STATUS\G
*************************** 1. row ***************************
            File: mysql-bin.000008
        Position: 1004954600
    Binlog_Do_DB: 
Binlog_Ignore_DB: 
1 row in set (0.00 sec)

MariaDB [(none)]> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.16.1.10
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 2
              Master_Log_File: mysql-bin.000004
          Read_Master_Log_Pos: 84173703
               Relay_Log_File: mariadb-relay-bin.000008
                Relay_Log_Pos: 114629023
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: information_schema,performance_schema
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1062
                   Last_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1031126226
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1031124771
              Relay_Log_Space: 241383704
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 1062
               Last_SQL_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1031126226
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
1 row in set (0.00 sec)

MariaDB [(none)]>

1，数据库2节点：

MariaDB [(none)]> SHOW MASTER STATUS\G
*************************** 1. row ***************************
            File: mysql-bin.000004
        Position: 84162184
    Binlog_Do_DB: 
Binlog_Ignore_DB: 
1 row in set (0.00 sec)

MariaDB [(none)]> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.16.1.9
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 2
              Master_Log_File: mysql-bin.000008
          Read_Master_Log_Pos: 1004862639
               Relay_Log_File: mariadb-relay-bin.000008
                Relay_Log_Pos: 917050651
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: information_schema,performance_schema
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1062
                   Last_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1030599783
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1030598328
              Relay_Log_Space: 6260191426
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 1062
               Last_SQL_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1030599783
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
1 row in set (0.00 sec)

MariaDB [(none)]>

@chenjacken
Copy link
Author

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@chenjacken 那应该是数据库的问题了，之前也遇到过数据不一致的问题，按照这个文档排查下：https://www.cloudpods.org/zh/docs/ops/databases/mysql_ha_recovery/

!!!!问题是数据库双主高可用时候，主主数据同步出现问题，目前已经解决了。

非常感谢！！！！！！ @zexi

@zexi zexi closed this as completed Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chenjacken @zexi