Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More vpci cleanup: code reorg, enable VPCI 64-bit bar emulation for sos and pre-launched VMs #3241

Closed
donsheng opened this issue Jun 10, 2019 · 2 comments

Comments

@donsheng
Copy link
Contributor

donsheng commented Jun 10, 2019

The e820 may does not cover the pci hole that exceeds the mem_top (see above), and if sos reprograms the bar, it may hang as there is no ept existing for that pci hole. If with bar emulation for sos, this can be avoided.

Currently, we only do pci bar emulation for partition mode (prelaunched vm), but we also need to do pci bar emulation for sos to unify the code

@Mingyuan18
Copy link

[External_System_ID] ACRN-3453

@donsheng donsheng changed the title Enable VPCI 64-bit bar emulation for sos and pre-launched VMs More vpci cleanup: code reorg, enable VPCI 64-bit bar emulation for sos and pre-launched VMs Jun 27, 2019
wenlingz pushed a commit that referenced this issue Jun 27, 2019
find_pci_pdev is not used any more, remove it.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jun 27, 2019
Add get_offset_of_caplist() function to return capability offset based on header type:
For normal pci device and bridge, its capability offset is at offset 0x34
For cardbus, its capability offset is at offset 0x14

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jun 27, 2019
union pci_bar uses bit fields and follows the PCI bar spec definition to define the
bar flags portion and base address, this is to keep the same hardware format for vbar
register. The base/type of union pci_bar are still kept to minimize code changes
in one patch, they will be removed in subsequent patches.

define pci_pdev_get_bar_base() function to extract bar base address given a 32-bit raw
bar value

define a utility function pci_get_bar_type() to extract bar types
from raw bar value to simply code, as this function will be used in multiple
places later on: this function can be called on reg->value stored in struct
pci_bar to derive bar type.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jun 27, 2019
nr_bars in struct pci_pdev is used to store the actual # of bars (
6 for normal pci device and 2 for pci bridge), nr_bars will be used in subsequent
patches

Use uint32_t for bar related variables (bar index, etc) to unify the bar
related code (no casting between uint32_t and uint8_t)

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jun 27, 2019
And put the checking in vdev_pt_write_cfg instead to have less nesting in
vdev_pt_write_vbar to improve code readability.

Rename function:
vdev_pt_remap_generic_bar --> vdev_pt_remap_generic_mem_vbar

vdev_pt_read_cfg's function declaration is merged into one line instead of 2
lines

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jun 27, 2019
Use nr_bars instead of PCI_BAR_COUNT to check bar access offset.
As while normal pci device has max 6 bars, pci bridge only has 2 bars,
so for pci normal pci device, pci cfg offsets 0x10-0x24 are for bar access,
but for pci bridge, only 0x10-0x14 are for bar access (0x18-0x24 are
for other accesses).

Rename function:
 pci_bar_access --> is_bar_offset

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
There is no need to call get_bar_base(), as new_bar is set to val & mask,
where mask is the bar size mask, so new_base has already been set to be the
bar base address before get_bar_base() is called on it.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
pbar base can be built by using the base address fields stored in
struct pci_bar's reg member.

get_pbar_base: return pbar's base address in 64-bit. For 64-bit MMIO bar, its lower 32-bits
base address and upper 32-bits base are combined into one 64-bit base address

pci_bar_2_bar_base: helper function that is called by get_pbar_base

And changed related code to use get_pbar_base to get pbar base address in 64-bit

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
We added "union pci_bar_reg reg" to struct pci_bar in previous commit,
but only pci_pdev uses it and pci_vdev does not use it. Starting from
this commit, pci_vdev will use it:

In init_vdev_pt(), copy pbar's reg's flags portion to corresponding vbar's
reg.

When guest updates the vbar base address, the corresponding vbar reg's base
address will also be updated, so that in subsequent commits, we can eventually
remove the base member in struct pci_bar.

Rename local variable new_bar to base in vdev_pt_write_vbar

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
vbar base can be built by using the base address fields stored in
struct pci_bar's reg member.

get_vbar_base: return vbar's base address in 64-bit. For 64-bit MMIO bar, its lower 32-bits
base address and upper 32-bits base are combined into one 64-bit base address

And changed related code to use get_vbar_base to get vbar base address in
64-bit.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
To remember the previously mapped/registered vbar base

For the following reasons:
 register_mmio_emulation_handler() will throw an error if the the same addr_lo is
 alreayd registered before

 We are going to remove the base member from struct pci_bar, so we cannot use vdev->bar[idx].base
 in the code any more

 In subsequent commits, we will assume vdev_pt_remap_generic_mem_vbar() is called after a new
 vbar base is set, mainly because of 64-bit mmio bar handling, so we need a
 separate bar_base_mapped[] array to track the previously mapped vbar bases.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
…-bit

Replace new_base with vbar_base in vdev_pt_remap_generic_mem_vbar().
We will call vdev_pt_remap_generic_mem_vbar() after a new vbar base
is set, no need to pass new_base to vdev_pt_remap_generic_mem_vbar(),
as this new vbar base (vbar_base) can be obtained by calling get_vbar_base().

The reason we call vdev_pt_remap_generic_mem_vbar() after a new vbar base
is set is for 64-bit mmio handling: when the lower 32-bit of 64-bit mmio vbar is
set, we will defer calling vdev_pt_remap_generic_mem_vbar until its upper 32-bit
vbar base is set.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
Define/Use variable in place of code to improve readability:

Define new local variable struct pci_bar *vbar, and use vbar-> in place of vdev->bar[idx].

Define new local variable uint64_t vbar_base in init_vdev_pt

Rename uint64_t vbar[PCI_BAR_COUNT] of struct acrn_vm_pci_ptdev_config to uint64_t vbar_base[PCI_BAR_COUNT]

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
Only do the EPT mapping if vbar base (gpa) is not mapped to the same pbar (hpa) before.
Need to unmap the existing mapping for a vbar base (gpa), otherwise, hv will
throw an error.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
At this point, uint64_t base in struct pci_bar is not used by any code, so we
can remove it.

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 1, 2019
Create 2 functions from code:
 pci_base_from_size_mask
 vdev_pt_remap_mem_vbar

Use vbar in place of vdev->bar[idx] by setting vbar to &vdev->bar[idx]

Change base to uint64_t to accommodate 64-bit MMIO bar size masking in
subsequent commits

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jul 4, 2019
Enable 64-bit bar emulation, if pbar is of type PCIBAR_MEM64, vbar will also be
of type PCIBAR_MEM64 instead of PCIBAR_MEM32

With 64-bit bar emulation code in place, we can remove enum pci_bar_type type
from struct pci_bar as bar type can be derived from struct pci_bar's reg member
by using the pci_get_bar_type function

Rename functions:
  pci_base_from_size_mask --> git_size_masked_bar_base

Remove unused functions

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jul 4, 2019
For sos, its vbar base address is set to pbar base address (vbar gpa = pbar
hpa)

For pre-launched VMs, vbar base address is pre-assigned in vm_config

Rename vdev_pt_remap_msix_table_bar to vdev_pt_remap_msix_table_vbar and make it
a static function

Remove unused function prototye vdev_pt_remap_msix_table_bar() in vpci_priv.h

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jul 4, 2019
For PIO bar emulation, initially or when guest reprograms the PIO bar, allow guest
access for the specified bar base address (pio port) and sized by calling vev_pt_allow_io_vbar

For PIO bar emulation, just set the initial vbar base address equal to pbar
base address. For example, if the pbar base address is 0x2000, then set the
initial vbar base address also to 0x2000

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jul 4, 2019
Rename vbdf to bdf for the following reasons:
Use the same coding style as struct pci_pdev, as pci_pdev uses bdf instead of pbdf

pci_vdev implies the its bdf is virtual, no need to prefix bdf with the v
prefix (redundant)

Tracked-On: #3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
It will be used to define pci vdev own ops. And high level API
will call this ops intead of invoking device specific functions
directly.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
Change vbar/msi/msix access checking from private to public.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
And use the ops based operations instead of direct access vdev
specific API.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
And use vhostbridge for both SOS and pre-launched VM.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
It will be used to define pci vdev own ops. And high level API
will call this ops intead of invoking device specific functions
directly.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
Change vbar/msi/msix access checking from private to public.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
And use the ops based operations instead of direct access vdev
specific API.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 12, 2019
And use vhostbridge for both SOS and pre-launched VM.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 15, 2019
It will be used to define pci vdev own ops. And high level API
will call this ops intead of invoking device specific functions
directly.

Tracked-On: #3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 15, 2019
Change vbar/msi/msix access checking from private to public.

Tracked-On: #3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 15, 2019
And use the ops based operations instead of direct access vdev
specific API.

Tracked-On: #3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
acrnsi pushed a commit that referenced this issue Jul 15, 2019
And use vhostbridge for both SOS and pre-launched VM.

Tracked-On: #3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
fyin1-zz pushed a commit to fyin1-zz/acrn-hypervisor that referenced this issue Jul 18, 2019
This is a followup patch to fix the coding style issue introduced
in by commit "c2d25aafb889ade954af8795df2405a94024d860":
  The unmodified pointer should be defined as const

Also addressed one comments from Fei to use reversed function call
in vpci_init_pt_dev and vpci_deinit_pt_dev.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
wenlingz pushed a commit that referenced this issue Jul 22, 2019
This is a followup patch to fix the coding style issue introduced
in by commit "c2d25aafb889ade954af8795df2405a94024d860":
  The unmodified pointer should be defined as const

Also addressed one comments from Fei to use reversed function call
in vpci_init_pt_dev and vpci_deinit_pt_dev.

Tracked-On: #3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
Enable 64-bit bar emulation, if pbar is of type PCIBAR_MEM64, vbar will also be
of type PCIBAR_MEM64 instead of PCIBAR_MEM32

With 64-bit bar emulation code in place, we can remove enum pci_bar_type type
from struct pci_bar as bar type can be derived from struct pci_bar's reg member
by using the pci_get_bar_type function

Rename functions:
  pci_base_from_size_mask --> git_size_masked_bar_base

Remove unused functions

Tracked-On: projectacrn#3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
For sos, its vbar base address is set to pbar base address (vbar gpa = pbar
hpa)

For pre-launched VMs, vbar base address is pre-assigned in vm_config

Rename vdev_pt_remap_msix_table_bar to vdev_pt_remap_msix_table_vbar and make it
a static function

Remove unused function prototye vdev_pt_remap_msix_table_bar() in vpci_priv.h

Tracked-On: projectacrn#3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
For PIO bar emulation, initially or when guest reprograms the PIO bar, allow guest
access for the specified bar base address (pio port) and sized by calling vev_pt_allow_io_vbar

For PIO bar emulation, just set the initial vbar base address equal to pbar
base address. For example, if the pbar base address is 0x2000, then set the
initial vbar base address also to 0x2000

Tracked-On: projectacrn#3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
Rename vbdf to bdf for the following reasons:
Use the same coding style as struct pci_pdev, as pci_pdev uses bdf instead of pbdf

pci_vdev implies the its bdf is virtual, no need to prefix bdf with the v
prefix (redundant)

Tracked-On: projectacrn#3241
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
It will be used to define pci vdev own ops. And high level API
will call this ops intead of invoking device specific functions
directly.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
Change vbar/msi/msix access checking from private to public.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
And use the ops based operations instead of direct access vdev
specific API.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
And use vhostbridge for both SOS and pre-launched VM.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
junjiemao1 pushed a commit to junjiemao1/acrn-hypervisor that referenced this issue Jul 28, 2019
This is a followup patch to fix the coding style issue introduced
in by commit "c2d25aafb889ade954af8795df2405a94024d860":
  The unmodified pointer should be defined as const

Also addressed one comments from Fei to use reversed function call
in vpci_init_pt_dev and vpci_deinit_pt_dev.

Tracked-On: projectacrn#3241
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
@Mingyuan18
Copy link

The patch was merged and no regression found in below build:
acrn tag: acrn-2019w30.2-140000p
CL: 30450

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants