Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Console with modified backend #11

Closed
hassanjameel1 opened this issue Oct 21, 2014 · 5 comments
Closed

Console with modified backend #11

hassanjameel1 opened this issue Oct 21, 2014 · 5 comments

Comments

@hassanjameel1
Copy link

Hi,

I am trying to try the ponger example using your modified netback. I cannot seem to make it work. The console output for the ClickOS VM says:

Xen Minimal OS!
start_info: 0x1be000(VA)
nr_pages: 0x800
shared_inf: 0x8ce6b000(MA)
pt_base: 0x1c1000(VA)
nr_pt_frames: 0x5
mfn_list: 0x1ba000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line:
stack: 0x177040-0x197040
MM: Init
_text: 0x0(VA)
_etext: 0xe1e1d(VA)
_erodata: 0x15d000(VA)
_edata: 0x15ebb8(VA)
stack start: 0x177040(VA)
_end: 0x1b9010(VA)
start_pfn: 1c9
max_pfn: 800
Mapping memory range 0x400000 - 0x800000
setting 0x0-0x15d000 readonly
skipped 0x1000
MM: Initialise page allocator for 1cb000(1cb000)-800000(800000)
MM: done
Demand map pfns at 801000-2000801000.
Heap resides at 2000802000-4000802000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x801000.
Initialising scheduler
Thread "Idle": pointer: 0x2000802050, stack: 0x1e0000
Thread "xenstore": pointer: 0x2000802800, stack: 0x1f0000
xenbus initialised on irq 1 mfn 0x2222b0
Thread "shutdown": pointer: 0x2000802fb0, stack: 0x200000
Dummy main: start_info=0x197040
Thread "main": pointer: 0x2000803760, stack: 0x210000
sparsing 0MB at 176000
"main"
[on_status:205] router id 0
[on_status:206] status change to Running
Thread "click": pointer: 0x2000810c50, stack: 0x230000
backend dom 0
Mapping TX rings
0 map errors

Am I doing it right? The example.config file that I am using is as follows:

name = 'clickos'
kernel = '/home/nfv1/click/clickos/minios/build/clickos_x86_64'
vcpus = '1'

pinning your VCPU helps performance

cpus = '3'
memory = '8'

uncoment this line if you wish to use our backend

vif = ['bridge=vale0,script=vif-vale']

vif = ['mac=68:05:ca:29:2e:be,bridge=vale0']

on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'preserve'
click = 'ponger.click'

uncomment this line to enable PVH mode for ClickOS

pvh=1

Best wishes,
Hassan

@jpemartins
Copy link
Contributor

Hi,

Does it get stuck there on:

"Mapping TX rings
0 map errors"

It doesn't show anything else?

Could you tell me your hardware specifications? and the output of these commands:

xl vcpu-list
xl info
xl dmesg
dmesg

I indeed have heard of cases where it gets stuck on mapping phase being unable to map the RX rings. But I was unable to reproduce it in my hardware.

Thanks,
Cheers,
Joao

@hassanjameel1
Copy link
Author

Hi Joao,

Thanks for your prompt reply. It does get stuck at:

"Mapping TX rings
0 map errors"

and shows nothing else. The hardware specs are as follows:

CPU: Intel(R) Xeon(R) CPU E5-1607 v2 @ 3.00 GHz (4-core) (3 assigned to Dom-0)
RAM: 16 GB (with 8 GB assigned to Dom-0)
Network Card: Intel Corporation 82599 10 Gigabit TN Network Connection (dual-port)

Let me know if you require any other hardware specification.

The output to "xl vcpu-list" is:

Name ID VCPU CPU State Time(s) CPU Affinity
Domain-0 0 0 0 -b- 511.7 0
Domain-0 0 1 1 r-- 397.9 1
Domain-0 0 2 2 -b- 410.2 2
clickos 7 0 3 r-- 367.0 3

The output to "xl info" is:

Name ID VCPU CPU State Time(s) CPU Affinity
Domain-0 0 0 0 -b- 511.7 0
Domain-0 0 1 1 r-- 397.9 1
Domain-0 0 2 2 -b- 410.2 2
clickos 7 0 3 r-- 367.0 3
nfv1@nfv1-Precision-T3610:~/Documents/nfv-project$ sudo xl info
host : nfv1-Precision-T3610
release : 3.13.0-37-generic
version : #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014
machine : x86_64
nr_cpus : 4
max_cpu_id : 3
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2992
hw_caps : bfebfbff:2c100800:00000000:00006f00:77bee3ff:00000000:00000001:00000281
virt_caps : hvm hvm_directio
total_memory : 16328
free_memory : 7908
sharing_freed_memory : 0
sharing_used_memory : 0
outstanding_claims : 0
free_cpus : 0
xen_major : 4
xen_minor : 4
xen_extra : .0
xen_version : 4.4.0
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset :
xen_commandline : placeholder dom0_mem=8192M,max:8192M dom0_max_vcpus=3 dom0_vcpus_pin
cc_compiler : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
cc_compile_by : stefan.bader
cc_compile_domain : canonical.com
cc_compile_date : Tue Sep 30 19:19:31 UTC 2014
xend_config_format : 4

The output of "xl dmesg" is:

(XEN) Xen version 4.4.0 (Ubuntu 4.4.0-0ubuntu5.2) (stefan.bader@canonical.com) (gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2) debug=n Tue Sep 30 19:19:31 UTC 2014
(XEN) Bootloader: GRUB 2.02~beta2-9ubuntu1
(XEN) Command line: placeholder dom0_mem=8192M,max:8192M dom0_max_vcpus=3 dom0_vcpus_pin
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 0000000000095400 (usable)
(XEN) 0000000000095400 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 0000000087ed6000 (usable)
(XEN) 0000000087ed6000 - 0000000088000000 (reserved)
(XEN) 0000000088000000 - 0000000088775000 (usable)
(XEN) 0000000088775000 - 0000000088800000 (reserved)
(XEN) 0000000088800000 - 0000000088ee9000 (usable)
(XEN) 0000000088ee9000 - 0000000089000000 (ACPI data)
(XEN) 0000000089000000 - 000000008a5a8000 (usable)
(XEN) 000000008a5a8000 - 000000008a800000 (ACPI NVS)
(XEN) 000000008a800000 - 000000008c594000 (usable)
(XEN) 000000008c594000 - 000000008c743000 (reserved)
(XEN) 000000008c743000 - 000000008c766000 (usable)
(XEN) 000000008c766000 - 000000008c7ec000 (ACPI NVS)
(XEN) 000000008c7ec000 - 000000008d000000 (usable)
(XEN) 0000000090000000 - 00000000a0000000 (reserved)
(XEN) 00000000fed1c000 - 00000000fed40000 (reserved)
(XEN) 00000000ff000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 0000000470000000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 DELL )
(XEN) ACPI: XSDT 88F29088, 008C (r1 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: FACP 88F32130, 010C (r5 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: DSDT 88F291A0, 8F8C (r2 DELL CBX3 23 INTL 20091112)
(XEN) ACPI: FACS 8A7F7080, 0040
(XEN) ACPI: APIC 88F32240, 0090 (r3 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: FPDT 88F322D0, 0044 (r1 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: MCFG 88F32318, 003C (r1 DELL OEMMCFG. 1072009 MSFT 97)
(XEN) ACPI: HPET 88F32358, 0038 (r1 DELL CBX3 1072009 AMI. 5)
(XEN) ACPI: ASF! 88F32390, 00A0 (r32 INTEL HCG 1 TFSM F4240)
(XEN) ACPI: SSDT 88F32430, CD380 (r2 INTEL CpuPm 4000 INTL 20051117)
(XEN) ACPI: EINJ 88FFF7B0, 0130 (r1 AMI AMI EINJ 0 0)
(XEN) ACPI: ERST 88FFF8E0, 0230 (r1 AMIER AMI ERST 0 0)
(XEN) ACPI: HEST 88FFFB10, 00A8 (r1 AMI AMI HEST 0 0)
(XEN) ACPI: BERT 88FFFBB8, 0030 (r1 AMI AMI BERT 0 0)
(XEN) ACPI: DMAR 88FFFBE8, 00B4 (r1 A M I OEMDMAR 1 INTL 1)
(XEN) ACPI: SLIC 88FFFCA0, 0176 (r3 DELL CBX3 1072009 MSFT 10013)
(XEN) System RAM: 16328MB (16720112kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 8a7f7080/0000000000000000, using 32
(XEN) Processor #0 7:14 APIC version 21
(XEN) Processor #2 7:14 APIC version 21
(XEN) Processor #4 7:14 APIC version 21
(XEN) Processor #6 7:14 APIC version 21
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2992.900 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) - APIC Register Virtualization
(XEN) - Virtual Interrupt Delivery
(XEN) - Posted Interrupt Processing
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 4 CPUs
(XEN) verify_tsc_reliability: TSC warp detected, disabling TSC_RELIABLE
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2403000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 000000045c000000->0000000460000000 (2076086 pages to be allocated)
(XEN) Init. ramdisk: 000000046edb6000->000000046ffff6ce
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff82403000
(XEN) Init. ramdisk: ffffffff82403000->ffffffff8364c6ce
(XEN) Phys-Mach map: ffffffff8364d000->ffffffff8464d000
(XEN) Start info: ffffffff8464d000->ffffffff8464d4b4
(XEN) Page tables: ffffffff8464e000->ffffffff84675000
(XEN) Boot stack: ffffffff84675000->ffffffff84676000
(XEN) TOTAL: ffffffff80000000->ffffffff84800000
(XEN) ENTRY ADDRESS: ffffffff81d361f0
(XEN) Dom0 has maximum 3 VCPUs
(XEN) Scrubbing Free RAM: ................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 280kB init memory.

The output of "dmesg" is:

[65497.038538] xen_netback/xen_netback:xenvif_netmap_unmap: vif6.0: revoking 1034 pages
[65497.038542] xen:grant_table: WARNING: g.e. 0x207 still in use!
[65497.038544] deferring g.e. 0x207 (pfn 0xffffffffffffffff)
[65497.038545] xen:grant_table: WARNING: g.e. 0x40e still in use!
[65497.038546] deferring g.e. 0x40e (pfn 0xffffffffffffffff)
[65497.038547] xen:grant_table: WARNING: g.e. 0x40f still in use!
[65497.038548] deferring g.e. 0x40f (pfn 0xffffffffffffffff)
[65497.038548] xen:grant_table: WARNING: g.e. 0x410 still in use!
[65497.038549] deferring g.e. 0x410 (pfn 0xffffffffffffffff)
[65497.038550] xen:grant_table: WARNING: g.e. 0x411 still in use!
[65497.038551] deferring g.e. 0x411 (pfn 0xffffffffffffffff)
[65497.109658] 085.609299 [ 719] nm_bdg_detach deleting priv
[65497.120414] xen:balloon: Cannot add additional memory (-17)
[65498.035669] xen:grant_table: freeing g.e. 0x207
[65498.035675] xen:grant_table: freeing g.e. 0x40e
[65498.035677] xen:grant_table: freeing g.e. 0x40f
[65498.035680] xen:grant_table: freeing g.e. 0x410
[65498.035682] xen:grant_table: freeing g.e. 0x411
[65497.038633] xen_netback/xen_netback:xenvif_netmap_detach: netmap_detach vif6.0
[65522.831735] 111.331376 [2259] netmap_attach success for vif7.0
[65522.844806] IPv6: ADDRCONF(NETDEV_UP): vif7.0: link is not ready
[65522.845175] IPv6: ADDRCONF(NETDEV_UP): vif7.0: link is not ready
[65522.898519] 111.398160 [1456] netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1
[65522.901717] xen_netback/xen_netback:xenvif_netmap_map: vif7.0: granted 10 1024 (ring/buffers)

Best wishes,
Hassan

@jpemartins
Copy link
Contributor

Hey Hassan,

Everything looks ok on the backend side and configurations.

Could you try booting the VM in one of the cores of Domain-0 ? For example in core 2 instead of core 3, then pinning it back to core 3.

Thanks,
Cheers

@hassanjameel1
Copy link
Author

HI Joao,

Great, it worked. I specified the cpu in the .cfg file and booted the VM. The console this time didn't get stuck. After that I pinned it back to core 3 by:

xl vcpu-pin clickos 0 3

Thanks a lot! Any reason why this worked?

@jpemartins
Copy link
Contributor

Not sure yet but will let you know when I fix the issue, in mini-os. Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants