Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QETH performance Issue (global) #195

Closed
ivan-w opened this issue Feb 16, 2017 · 16 comments
Closed

QETH performance Issue (global) #195

ivan-w opened this issue Feb 16, 2017 · 16 comments
Assignees

Comments

@ivan-w
Copy link
Contributor

ivan-w commented Feb 16, 2017

Under linux....

Under z/VM 6.3 - when a QDIO/QETH triple subchannel set (tried in Layer 2 mode) is activated (using CREATE VSWITCH UPLINK ETHERNET RDEV ... This command ensures the OS opens the QETH interface in layer 2 mode.

The QETH is created as follows :

0A00.3 QETH ifname tapX

tapX is predefined and bound to a bridge using brctl.

The tapX interface indicated by the ifname is opened (so everything looks fine) as shown by devlist and brctl showstp brX shows the interface is open and forwarding frames.

However, all I/O on ALL devices in hercules take an inordinate amount of time after that (anything before the activation would take sub millisecond of time now takes about a second)... Operations for the system become very sluggish (Logging in a user which prior to the activation of the QDIO/QETH interface took less than a second can take up to a minute after activation)...

Even the hercules console becomes sluggish (even with panrate fast - it kind of jumps all over the place).

Looking at the host CPU activity doesn't show any loop or unusual CPU tie up.

I suspect a lock issue - maybe a lock held by the QETH module while waiting for a frame or a timeout.

I will try to get a PTT trace on that to see if anything odd shows up.

@mcisho
Copy link
Contributor

mcisho commented Feb 16, 2017

Afaik, QETH does not work, and never has worked, with a virtual machine. The magic combination of CHSC replies from Hercules to VM that results in the correct CHSC replies from VM to a virtual machine is still a mystery.

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 16, 2017 via email

@mcisho
Copy link
Contributor

mcisho commented Feb 16, 2017

There's a lot of qeth.c that is there, but it's definitely not all there.

QETH should, imho, always use a TAP, and if the guest wants to use L3, QETH should deal with it, just like a real OSA does. Always using a TAP means QETH needs to handle ARP and Neighbor Discovery. QETH also needs to handle multiple data devices, as z/OS requires two data device, one for IPv4 and the other for IPv6. I'm not sure QETH will ever work with virtual machines, due to the lack of documentation of CHSC requests/responses. In fact CHSC is the stumbling block to QETH working well with anything other than a Linux guest.

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 17, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 17, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 17, 2017 via email

@jphartmann
Copy link
Contributor

" the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."

Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.

Have you looked at the MAC addresses? I think Q NIC DET will show them.

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 17, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 17, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 17, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 20, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 20, 2017 via email

@ivan-w
Copy link
Contributor Author

ivan-w commented Feb 21, 2017

I think I fixed it ;)

The issue is that in channel.c, when the I/O interrupt is prepared to be presented (present_io_interrupt() doesn't actually perform the PSW switch) - it indicates the device now needs a TSCH to clear the interrupt (dev->tschpending). This apparently does not apply for QDIO Thin Interrupts (the actual point of QDIO Thin Interrupts). Therefore, the interrupt was never off the subchannel and it couldn't present any other interrupt to indicate the change in the QDIO Queue/Buffer status. Furthermore since there was already an interrupt pending clearing, the channel code wouldn't present any further interrupts.

With this change, I managed to have the z/VM TCPIP stack talk to a CP VSWITCH and to the outside world.

******** Sample output ***********
netstat devlink
VM TCP/IP Netstat Level 630 TCP/IP Server Name: TCPIP

Device OSAQDIO1 Type: OSD Status: Ready
Queue size: 0 CPU: 0 Address: 0A00 Port name: OSA1
Link ETH0 Type: QDIOETHERNET Port number: 0
Transport Type: Ethernet MAC: 02-00-01-00-00-05
Speed: 1000000000
BytesIn: 2404 BytesOut: 732
Forwarding: Enabled MTU: 8992 IPv6: Enabled
IPv4 Path MTU Discovery: Disabled
IPv4 VIPA ARP
IPv6 VIPA ND
Multicast Group Members
--------------- -------
224.0.0.1 1
FF02::1:FF00:5 1
FF02::1 1
Ready; T=0.05/0.07 12:18:21
ping 192.168.1.1
Ping Level 630: Pinging host 192.168.1.1. (My internet router at home)
Enter #CP EXT to interrupt.
PING: Ping #1 response took 0.010 seconds. Successes so far 1.
Ready; T=0.11/0.24 12:18:31
q vswitch det
VSWITCH SYSTEM VSW01 Type: QDIO Connected: 1 Maxconn: INFINITE
PERSISTENT RESTRICTED ETHERNET Accounting: OFF
USERBASED LOCAL
VLAN Unaware
MAC address: 02-00-01-00-00-01 MAC Protection: Unspecified
IPTimeout: 5 QueueStorage: 8
Isolation Status: OFF VEPA Status: OFF
Uplink Port:
State: Ready
PMTUD setting: EXTERNAL PMTUD value: 8992 Trace Pages: 8
RDEV: 0A00.P00 VDEV: 0600 Controller: DTCVSW2 ACTIVE
Uplink Port Connection:
RX Packets: 25 Discarded: 483 Errors: 0
TX Packets: 25 Discarded: 0 Errors: 0
RX Bytes: 1978 TX Bytes: 1850
Device: 0600 Unit: 000 Role: DATA Port: 2049
Partner Switch Capabilities: Unavailable
Adapter Connections: Connected: 1
Adapter Owner: TCPIP NIC: 0A00.P00 Name: OSA1 Type: QDIO
RX Packets: 46 Discarded: 0 Errors: 0
TX Packets: 11 Discarded: 0 Errors: 0
RX Bytes: 6582 TX Bytes: 850
Device: 0A00 Unit: 000 Role: DATA Port: 0004
Options: Ethernet Broadcast
Unicast MAC Addresses:
02-00-01-00-00-05 IP: 192.168.1.126
Multicast MAC Addresses:
01-00-5E-00-00-01
33-33-00-00-00-01
33-33-FF-00-00-05
Ready; T=0.01/0.01 12:19:32
netstat home
VM TCP/IP Netstat Level 630 TCP/IP Server Name: TCPIP

IPv4 Home address entries:

Address Subnet Mask Link VSWITCH


192.168.1.126 255.255.255.0 ETH0

IPv6 Home address entries:

Link: ETH0
Address: FE80::1FF:FE00:5
Flags: Autoconfigured
DAD State: Address Passed DAD
Origin: Link-local

Link: ETH0
Address: 2A01:CB19:678:2500::1FF:FE00:5
Flags: Autoconfigured
DAD State: Interface ID Passed DAD
Origin: Received Router Advertisement

Ready; T=0.05/0.08 12:21:58
ping ns3.nic.fr
Ping Level 630: Pinging host NS3.NIC.FR (2001:660:3006:1::1:1).
Enter #CP EXT to interrupt.
PING: Ping #1 response took 0.039 seconds. Successes so far 1.
Ready; T=0.12/0.20 12:22:34


So I have Layer 2 working under z/VM... It's both IPv4 and IPv6 capable....

I have a couple of adjustments to apply before I commit a patch.

--Ivan

@ivan-w ivan-w self-assigned this Feb 21, 2017
@ivan-w ivan-w closed this as completed Feb 21, 2017
@GKevinGodwin610
Copy link

Hi @ivan-w ,

Sorry to re-open a VERY old thread, but I am having a real issue around getting QETH on Z/VM 5.3 Eval working with Hercules on a Ubuntu 18.04 linux machine.

I can get the V/VM to ipl without errors etc and I have the hercules.conf setup right... I think...

The hercules (v 4.3) console shows this output when ipl is run:

HHC03800I 0:0A01 QETH: Adapter mode set to Layer 2
HHC00901I 0:0A01 QETH: Interface tap1, type TAP opened
HHC03997I 0:0A01 QETH: tap1: using MAC address 0E:D6:F5:DA:58:E8
HHC03997I 0:0A01 QETH: tap1: using MTU 1500

and on the linux console the ifconfig shows the following:

tap1: flags=579<UP,BROADCAST,RUNNING,ALLMULTI> mtu 1500
inet6 fe80::cd6:f5ff:feda:58e8 prefixlen 64 scopeid 0x20
ether 0e:d6:f5:da:58:e8 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 185 bytes 24512 (24.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

I then have both my physical ethernet port and the tap1 in a bridge as below:

bridge name bridge id STP enabled interfaces
bridge0 8000.0ed6f5da58e8 no enp5s0
tap1

But in Z/VM I cannot ping anything on the LAN and from the LAN I cannot ping the IP from within the same LAN that I have configured on the Z/VM.

I am NOT a Z/VM expert at all and this is the reason I am trying to get this working... so I can learn how to use Z/VM.

In Z/VM the Q VSWITCH DET output show this:

q vswitch det
DMSDCS1083E Saved segment CMSPIPES does not exist
DMSDCS1083E Saved segment CMSPIPES does not exist
DMSDCS1083E Saved segment CMSVMLIB does not exist
DMSACC724I 191 replaces A (191)
Ready; T=0.21/0.29 06:31:27
06:31:27 VSWITCH SYSTEM VSW1 Type: VSWITCH Connected: 1 Maxconn: INFINITE
06:31:27 PERSISTENT RESTRICTED ETHERNET Accounting: OFF
06:31:27 VLAN Unaware
06:31:27 MAC address: 02-00-00-00-00-01
06:31:27 State: Ready
06:31:27 IPTimeout: 5 QueueStorage: 8
06:31:27 RDEV: 0A00 VDEV: 0A00 Controller: DTCVSW2
06:31:27 VSWITCH Connection:
06:31:27 RX Packets: 0 Discarded: 388 Errors: 0
06:31:27 TX Packets: 0 Discarded: 0 Errors: 0
06:31:27 RX Bytes: 0 TX Bytes: 0
06:31:27 Device: 0A02 Unit: 002 Role: DATA Port: 0001 Index: 000
1
06:31:27 Adapter Connections:
06:31:27 Adapter Owner: TCPIP NIC: 0900 Name: UNASSIGNED
06:31:27 RX Packets: 0 Discarded: 0 Errors: 0
06:31:27 TX Packets: 0 Discarded: 0 Errors: 0
06:31:27 RX Bytes: 0 TX Bytes: 0
06:31:27 Device: 0902 Unit: 002 Role: DATA Port: 0065 Index: Non
e
Ready; T=0.01/0.01 06:31:27

and finally, the ifconfig -all output is:

ifconfig -all
LNK2NET inet addr: 192.168.1.120 mask: 255.255.255.0
DOWN MTU: 1500
vdev: 0900 type: QDIO ETHERNET portname: UNASSIGNED
ipv4 router type: NONROUTER ipv6: DISABLED
cpu: 0 forwarding: ENABLED
RX bytes: 0 TX bytes: 0
Ready; T=0.19/0.23 06:33:25

Any advise, guidance or suggestions would be really welcome.

Thanks.

Kevin

@Fish-Git
Copy link
Contributor

The hercules (v 4.3) console ...

Hi Kevin!

If you are using "Hercules 4.3", then you are very likely using SDL Hyperion, not regular Hercules Hyperion.

Please issue the version command on the Hercules panel. If it reports a version string similar to the following:

    HHC01413I Hercules version 4.3.9999.9975-SDL-gf99cae09-modified (4.3.9999.9975)

then you are definitely using SDL Hyperion 4.x Hercules and not regular Hercules 4.x Hyperion (which is a completely different Hercules).

If that is the case (and I highly suspect it is), then you have reported your problem IN THE WRONG PLACE!   :)

You need to report issues (problems) with SDL Hyperion 4.x Hercules here:

Please create a new GitHub Issue using the URL above explaining your problem and hopefully someone with Linux networking experience will be able to help you. (I'm a Windows person and know nothing about Linux networking.)

Hope that helps!

@mcisho
Copy link
Contributor

mcisho commented May 12, 2020

This is not a Hercules issue, it is a VM configuration issue and really should be opened on an appropriate VM list.

However, in the mean time the main problem is that LNK2NET is DOWN, when it should be UP. You should logon and start your TCPIP virtual machine, and look for any messages that might indicate what is wrong.

Shouldn't the VSWITCH have a PORTNAME?

Ian

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants