By default, all Legacy INTx interrupts are masked, so there is no need to
mask this interrupt during irq_map callback.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Callback for irq_mask_ack is the same as for irq_mask. As there is no
special handling for irq_ack, there is no need to define irq_mask_ack too.

Signed-off-by: Pali Rohár <pali@kernel.org>
Acked-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

… processing INTx IRQ

It is possible that we receive spurious INTx interrupt. Check for the
return value of generic_handle_domain_irq() when processing INTx IRQ.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Emulated root bridge currently provides only one Legacy INTA interrupt
which is used for reporting PCIe PME and ERR events and handled by kernel
PCIe PME and AER drivers.

Aardvark HW reports these PME and ERR events separately, so there is no
need to mix real INTA interrupt and emulated INTA interrupt for PCIe PME
and AER drivers.

Register a new advk-EMU irq chip and a new irq domain for emulated root
bridge and use this new separate irq domain for providing INTA interrupt
from emulated root bridge for PME and ERR events.

The real INTA interrupt from real devices is now separate.

A custom map_irq callback function on PCI host bridge structure is used to
allocate IRQ mapping for emulated root bridge from new irq domain. Original
callback of_irq_parse_and_map_pci() is used for all other devices as before.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Enable aardvark PME interrupt unconditionally by unmasking it and read PME
requester ID to emulated bridge config space immediately after receiving
interrupt.

PME requester ID is stored in the PCIE_MSG_LOG_REG register, which contains
the last inbound message. So when new inbound message is received by HW
(including non-PM), the content in PCIE_MSG_LOG_REG register is replaced by
a new value.

PCIe specification mandates that subsequent PMEs are kept pending until the
PME Status Register bit is cleared by software by writing a 1b.

Support for masking/unmasking PME interrupt on emulated bridge via
PCI_EXP_RTCTL_PMEIE bit is now implemented only in emulated bridge config
space, to ensure that we do not miss any aardvark PME interrupt.

Reading of PCI_EXP_RTCAP and PCI_EXP_RTSTA registers is simplified as final
value is now always stored into emulated bridge config space by the
interrupt handler, so there is no need to implement support for these
registers in read_pcie callback.

Clearing of W1C bit PCI_EXP_RTSTA_PME is now also simplified as it is done
by pci-bridge-emul.c code for emulated bridge config space. So there is no
need to implement support for clearing this bit in write_pcie callback.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Currently enabling PCI_EXP_RTSTA_PME bit in PCI_EXP_RTCTL register does
nothing. This is because PCIe PME driver expects to receive PCIe interrupt
defined in PCI_EXP_FLAGS_IRQ register, but aardvark hardware does not
trigger PCIe INTx/MSI interrupt for PME event, rather it triggers custom
aardvark interrupt which this driver is not processing yet.

Fix this issue by handling PME interrupt in advk_pcie_handle_int() and
chaining it to PCIe interrupt 0 with generic_handle_domain_irq() (since
aardvark sets PCI_EXP_FLAGS_IRQ to zero). With this change PCIe PME driver
finally starts receiving PME interrupt.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

…_PME on emulated bridge

To optimize advk_pci_bridge_emul_pcie_conf_write() code, touch
PCIE_ISR0_REG and PCIE_ISR0_MASK_REG registers only when it is really
needed, when processing PCI_EXP_RTCTL_PMEIE and PCI_EXP_RTSTA_PME bits.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

The emulated bridge returns incorrect value for PCI_EXP_RTSTA register
during readout in advk_pci_bridge_emul_pcie_conf_read() function: the
correct bit is BIT(16), but we are setting BIT(23), because the code
does
  *value = (isr0 & PCIE_MSG_PM_PME_MASK) << 16
where
  PCIE_MSG_PM_PME_MASK
is
  BIT(7).

The code should probably have been something like
  *value = (!!(isr0 & PCIE_MSG_PM_PME_MASK)) << 16,
but we are better of using an if() and using the proper macro for this
bit.

Fixes: 8a3ebd8 ("PCI: aardvark: Implement emulated root PCI bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

ERR interrupt is triggered when corresponding bit is unmasked in both ISR0
and PCI_EXP_DEVCTL registers. Unmasking ERR bits in PCI_EXP_DEVCTL register
is not enough. This means that currently the ERR interrupt is never
triggered.

Unmask ERR bits in ISR0 register at driver probe time. ERR interrupt is not
triggered until ERR bits are unmasked also in PCI_EXP_DEVCTL register,
which is done by AER driver. So it is safe to unconditionally unmask all
ERR bits in aardvark probe.

Aardvark HW sets PCI_ERR_ROOT_AER_IRQ to zero and when corresponding bits
in ISR0 and PCI_EXP_DEVCTL are enabled, the HW triggers a generic interrupt
on GIC. Chain this interrupt to PCIe interrupt 0 with
generic_handle_domain_irq() to allow processing of ERR interrupts.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

According to PCI 3.0 specification, sending both MSI and MSI-X interrupts
is done by DWORD memory write operation to doorbell message address. The
write operation for MSI has zero upper 16 bits and the MSI interrupt number
in the lower 16 bits, while the write operation for MSI-X contains a 32-bit
value from MSI-X table.

Since the driver only uses interrupt numbers from range 0..31, the upper
16 bits of the DWORD memory write operation to doorbell message address
are zero even for MSI-X interrupts. Thus we can enable MSI-X interrupts.

Testing proves that kernel can correctly receive MSI-X interrupts from PCIe
cards which supports both MSI and MSI-X interrupts.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

MSI address for receiving MSI interrupts needs to be correctly set before
enabling processing of MSI interrupts.

Move code for setting PCIE_MSI_ADDR_LOW_REG and PCIE_MSI_ADDR_HIGH_REG
from advk_pcie_init_msi_irq_domain() to advk_pcie_setup_hw(), before
enabling PCIE_CORE_CTRL2_MSI_ENABLE.

After this we can remove the now unused member msi_msg, which was used
only for MSI doorbell address. MSI address can be any address which cannot
be used to DMA to. So change it to the address of the main struct advk_pcie.

Fixes: 8c39d71 ("PCI: aardvark: Add Aardvark PCI host controller driver")
Signed-off-by: Pali Rohár <pali@kernel.org>
Acked-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: stable@vger.kernel.org # f21a8b1 ("PCI: aardvark: Move to MSI handling using generic MSI support")

We should not unmask MSIs at setup, but only when kernel asks for them
to be unmasked.

At setup, mask all MSIs, and implement IRQ chip callbacks for masking
and unmasking particular MSIs.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Refactor the masking of ISR0/1 Sources and unmasking of summary MSI interrupt
so that it corresponds to the comments:
- first mask all ISR0/1
- then unmask all MSIs
- then unmask summary MSI interrupt

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

In advk_pcie_handle_msi() the authors expect that when bit i in the W1C
register PCIE_MSI_STATUS_REG is cleared, the PCIE_MSI_PAYLOAD_REG is
updated to contain the MSI number corresponding to index i.

Experiments show that this is not so, and instead PCIE_MSI_PAYLOAD_REG
always contains the number of the last received MSI, overall.

Do not read PCIE_MSI_PAYLOAD_REG register for determining MSI interrupt
number. Since Aardvark already forbids more than 32 interrupts and uses
own allocated hwirq numbers, the msi_idx already corresponds to the
received MSI number.

Fixes: 8c39d71 ("PCI: aardvark: Add Aardvark PCI host controller driver")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Aardvark hardware supports Multi-MSI and MSI_FLAG_MULTI_PCI_MSI is already
set for the MSI chip. But when allocating MSI interrupt numbers for
Multi-MSI, the numbers need to be properly aligned, otherwise endpoint
devices send MSI interrupt with incorrect numbers.

Fix this issue by using function bitmap_find_free_region() instead of
bitmap_find_next_zero_area().

To ensure that aligned MSI interrupt numbers are used by endpoint devices,
we cannot use Linux virtual irq numbers (as they are random and not
properly aligned). Instead we need to use the aligned hwirq numbers.

This change fixes receiving MSI interrupts on Armada 3720 boards and
allows using NVMe disks which use Multi-MSI feature with 3 interrupts.

Without this NVMe disks freeze booting as linux nvme-core.c is waiting
60s for an interrupt.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

Rewrite the code to use irq_set_chained_handler_and_data() handler with
chained_irq_enter() and chained_irq_exit() processing instead of using
devm_request_irq().

advk_pcie_irq_handler() reads IRQ status bits and calls other functions
based on which bits are set. These functions then read its own IRQ status
bits and calls other aardvark functions based on these bits. Finally
generic_handle_irq() with translated linux IRQ numbers are called.

Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <kabel@kernel.org>

This function is now always used in driver remove method, drop the
__maybe_unused attribute.

Signed-off-by: Marek Behún <kabel@kernel.org>

- Add PCI_ERROR_RESPONSE and related definitions for signaling and checking
  for transaction errors on PCI (Naveen Naidu)

- Fabricate PCI_ERROR_RESPONSE data (~0) in config read wrappers, instead
  of in host controller drivers, when transactions fail on PCI (Naveen
  Naidu)

- Use PCI_POSSIBLE_ERROR() to check for possible failure of config reads
  (Naveen Naidu)

* pci/errors:
  PCI: xgene: Use PCI_ERROR_RESPONSE to identify config read errors
  PCI: hv: Use PCI_ERROR_RESPONSE to identify config read errors
  PCI: keystone: Use PCI_ERROR_RESPONSE to identify config read errors
  PCI: Use PCI_ERROR_RESPONSE to identify config read errors
  PCI: cpqphp: Use PCI_POSSIBLE_ERROR() to check config reads
  PCI/PME: Use PCI_POSSIBLE_ERROR() to check config reads
  PCI/DPC: Use PCI_POSSIBLE_ERROR() to check config reads
  PCI: pciehp: Use PCI_POSSIBLE_ERROR() to check config reads
  PCI: vmd: Use PCI_POSSIBLE_ERROR() to check config reads
  PCI/ERR: Use PCI_POSSIBLE_ERROR() to check config reads
  PCI: rockchip-host: Drop error data fabrication when config read fails
  PCI: rcar-host: Drop error data fabrication when config read fails
  PCI: altera: Drop error data fabrication when config read fails
  PCI: mvebu: Drop error data fabrication when config read fails
  PCI: aardvark: Drop error data fabrication when config read fails
  PCI: kirin: Drop error data fabrication when config read fails
  PCI: histb: Drop error data fabrication when config read fails
  PCI: exynos: Drop error data fabrication when config read fails
  PCI: mediatek: Drop error data fabrication when config read fails
  PCI: iproc: Drop error data fabrication when config read fails
  PCI: thunder: Drop error data fabrication when config read fails
  PCI: Drop error data fabrication when config read fails
  PCI: Use PCI_SET_ERROR_RESPONSE() for disconnected devices
  PCI: Set error response data when config read fails
  PCI: Add PCI_ERROR_RESPONSE and related definitions

- Make emulated ROM BAR read-only by default (Pali Rohár)

- Make some emulated legacy PCI bits read-only for PCIe devices (Pali
  Rohár)

- Update reserved bits in emulated PCIe Capability (Pali Rohár)

- Allow drivers to emulate different PCIe Capability versions (Pali Rohár)

- Set emulated Capabilities List bit for all PCIe devices, since they must
  have at least a PCIe Capability (Pali Rohár)

* remotes/lorenzo/pci/bridge-emul:
  PCI: pci-bridge-emul: Set PCI_STATUS_CAP_LIST for PCIe device
  PCI: pci-bridge-emul: Correctly set PCIe capabilities
  PCI: pci-bridge-emul: Fix definitions of reserved bits
  PCI: pci-bridge-emul: Properly mark reserved PCIe bits in PCI config space
  PCI: pci-bridge-emul: Make expansion ROM Base Address register read-only

- Declare bitmap correctly and as part of struct nwl_msi managed resource
  (Christophe JAILLET)

* remotes/lorenzo/pci/xilinx-nwl:
  PCI: xilinx-nwl: Simplify code and fix a memory leak

- Use bitmap ops for MSI allocator (Christophe JAILLET)

- Fix IB window setup, which was broken by the fact that IB resources are
  now sorted in address order instead of DT dma-ranges order (Rob Herring)

* remotes/lorenzo/pci/xgene:
  PCI: xgene: Fix IB window setup
  PCI: xgene-msi: Use bitmap_zalloc() when applicable

- Reset everything below VMD before enumerating to work around failure to
  enumerate NVMe devices when guest OS reboots (Nirmal Patel)

* remotes/lorenzo/pci/vmd:
  PCI: vmd: Clean up domain before enumeration

- Fix aarch32 abort handler so it doesn't check the wrong bus clock before
  accessing the host controller (Marek Vasut)

* remotes/lorenzo/pci/rcar:
  PCI: rcar: Check if device is runtime suspended instead of __clk_is_enabled()

- Undo PM setup in qcom_pcie_probe() error handling path (Christophe
  JAILLET)

- Use __be16 type to store return value from cpu_to_be16() (Manivannan
  Sadhasivam)

* remotes/lorenzo/pci/qcom:
  PCI: qcom: Use __be16 type to store return value from cpu_to_be16()
  PCI: qcom: Fix an error handling path in 'qcom_pcie_probe()'

- Implement pci_remap_iospace() for ARM so mvebu can use
  devm_pci_remap_iospace() instead of the previous ARM-specific
  pci_ioremap_io() interface (Pali Rohár)

- Use the standard pci_host_probe() instead of the device-specific
  mvebu_pci_host_probe() (Pali Rohár)

- Replace all uses of ARM-specific pci_ioremap_io() with the
  ARM implementation of the standard pci_remap_iospace() interface and
  remove pci_ioremap_io() (Pali Rohár)

* remotes/lorenzo/pci/mvebu:
  arm: ioremap: Remove unused ARM-specific function pci_ioremap_io()
  arm: ioremap: Replace pci_ioremap_io() usage by pci_remap_iospace()

- Declare mt7621_pci_ops static (Sergio Paracuellos)

* remotes/lorenzo/pci/mt7621:
  PCI: mt7621: Declare mt7621_pci_ops static

- Disable Mediatek DVFSRC voltage request since lack of DVFSRC to respond
  to the request causes failure to exit L1 PM Substate (Jianjun Wang)

* remotes/lorenzo/pci/mediatek-gen3:
  PCI: mediatek-gen3: Disable DVFSRC voltage request

- Assert PERST# for 100ms to allow power and clock to stabilize (qizhong
  cheng)

* remotes/lorenzo/pci/mediatek:
  PCI: mvebu: Remove custom mvebu_pci_host_probe() function
  PCI: mvebu: Replace pci_ioremap_io() usage by devm_pci_remap_iospace()
  arm: ioremap: Implement standard PCI function pci_remap_iospace()

- Don't ioremap NULL when DT lacks ATU resource (Tim Harvey)

- Drop redundant qcom-ep error message for platform_get_irq_byname()
  failure (Krzysztof Wilczyński)

* remotes/lorenzo/pci/dwc:
  PCI: qcom-ep: Remove surplus dev_err() when using platform_get_irq_byname()
  PCI: dwc: Do not remap invalid res

- Declare bitmap correctly for use by bitmap interfaces (Christophe
  JAILLET)

- Clean up computation of legacy and non-legacy MSI bitmasks (Florian
  Fainelli)

* remotes/lorenzo/pci/brcmstb:
  PCI: brcmstb: Do not use __GENMASK
  PCI: brcmstb: Declare a bitmap as a bitmap, not as a plain unsigned long

- Enable clock gating to save power (Hector Martin)

- Fix REFCLK1 enable/poll logic (Hector Martin)

* remotes/lorenzo/pci/apple:
  PCI: apple: Fix REFCLK1 enable/poll logic
  PCI: apple: Enable clock gating

- Add bridge emulation definitions for PCIe DEVCAP2, DEVCTL2, DEVSTA2,
  LNKCAP2, LNKCTL2, LNKSTA2, SLTCAP2, SLTCTL2, SLTSTA2 (Pali Rohár)

- Add aardvark support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2 registers
  (Pali Rohár)

- Clear all MSIs at setup to avoid spurious interrupts (Pali Rohár)

- Disable bus mastering when unbinding host controller driver (Pali Rohár)

- Mask all interrupts when unbinding host controller driver (Pali Rohár)

- Fix memory leak in host controller unbind (Pali Rohár)

- Assert PERST# when unbinding host controller driver (Pali Rohár)

- Disable link training when unbinding host controller driver (Pali Rohár)

- Disable common PHY when unbinding host controller driver (Pali Rohár)

- Fix resource type checking to check only IORESOURCE_MEM, not
  IORESOURCE_MEM_64, which is a flavor of IORESOURCE_MEM (Pali Rohár)

* remotes/lorenzo/pci/aardvark:
  PCI: aardvark: Fix checking for MEM resource type
  PCI: aardvark: Disable common PHY when unbinding driver
  PCI: aardvark: Disable link training when unbinding driver
  PCI: aardvark: Assert PERST# when unbinding driver
  PCI: aardvark: Fix memory leak in driver unbind
  PCI: aardvark: Mask all interrupts when unbinding driver
  PCI: aardvark: Disable bus mastering when unbinding driver
  PCI: aardvark: Comment actions in driver remove method
  PCI: aardvark: Clear all MSIs at setup
  PCI: aardvark: Add support for DEVCAP2, DEVCTL2, LNKCAP2 and LNKCTL2 registers on emulated bridge
  PCI: pci-bridge-emul: Add definitions for missing capabilities registers
  PCI: pci-bridge-emul: Add description for class_revision field

- Add Gen4 automotive device IDs (Kelvin Cao)

- Declare state_names[] as static so it's not allocated and initialized for
  every call (Kelvin Cao)

* pci/switchtec:
  PCI/switchtec: Declare local state_names[] as static
  PCI/switchtec: Add Gen4 automotive device IDs

- Fix infinite loop in pciehp IRQ handler on power fault (Lukas Wunner)

* pci/hotplug:
  PCI: pciehp: Fix infinite loop in IRQ handler upon power fault

* pci/enumeration:
  PCI: Use pci_find_vsec_capability() when looking for TBT devices