Add Support to use KSZ9893R in Iris 2.0 Hardware #3
Comments
|
Carlos Aguero wrote:
Yes, we are happy for you to develop/collaborate on a driver for the KSZ9893R that can be pushed upstream. |
|
The driver has been added to this branch: https://github.com/uvdl/yocto-ornl/tree/feature/ksz9893-support The driver is currently building but it has not been tested. As the driver is written for an older kernel some features are having conflicts and were disabled: KSZ_PTP KSZ_DSA KSZ_MRP KSZ_HSR If any of this is needed, the build errors need to be fixed. |
|
Also, some days ago Microchip started pushing the drivers for kernel 4.9. They can be found here: https://github.com/Microchip-Ethernet/EVB-KSZ9477/tree/master/KSZ/linux-drivers/ksz9897/linux-4.9 |
|
First impression is hopeful: but later, this: I can see led |
|
I am encouraged by the information presented in linux-drivers/ksz9897/linux-4.9/doc. However, in order to make use of these programs, they need to be compiled and added to the build. So we are going to need to add setup_sw.c, regs_bin at the least. |
|
I'll create a yocto build from Rocko which uses kernel 4.9, then we can add these latest drivers and the utilities. |
|
I already added the latest drivers in branch: |
|
Very encouraging: And upon login, got this: |
|
However while the interface can be given an IP, etc. no traffic flows and it does not correctly identify the RGMII to be 100Mb/s (its stuck at 10Mb/s and does not function). I get this: I do not believe that the RGMII interface is functioning. I was reading about this, and there are some references that may apply:
|
|
On today's telcon, Edison mentioned that he thought it might be incorrect to use the In both cases, when the boards boot, a line of the form: Is emitted. It is pretty clear that the FEC is the interface on the iMX6 that talks to the RGMII. |
|
On Friday, January 18, 2019 2:05 PM, Edison Fernandez wrote:
I reference the below items: From [3] there is a note:
Also [2, Sec 4.10, pg. 52] says:
This rather conflicts with [1, TABLE 4-30, pg. 58][2, TABLE 4-30, pg. 55] describing the pin direction unless one interprets the MAC to be the MAC inside the KSZ9893R, in which case it is consistent with the schematic note and the note on pg. 52. In either case, it seems the choice of RGMII TX-to-TX and RX-to-RX is correct. However, during my investigation on the RGMII interface, this discussion was observed:
What this appears to require is a 125MHz clock to be provided to the ENET_REF_CLK input on
I am going to try and make a wire on one of our boards from J8.49 to J9.52 (GPS TIMEPULSE is also an output but it is not enabled at power up; I’m hoping it will not cause problems). |
|
I didn't know about the naming convention in Micrel parts. If that's the case the connection should be correct. |
|
ok. i did the rework and cut the TIMEPULSE from the GPS. If you can figure out what the .dts should be to send the 125MHz clock out GPIO_16, that would be super! |
|
I just pushed some changes that aim to enable the 125MHz clock through GPIO_16, you can see the commit here: I took this thread as a reference. The only configuration I'm missing is IOMUXC_ENET_REF_CLK_SELECT_INPUT. I also included in the image the imx-test package which has memtool so you can read/write the iMX6 registers. To get the current value of IOMUXC_ENET_REF_CLK_SELECT_INPUT, you can run: /unit_tests/memtool -32 0x20E083C 1 E Reading 0x1 count starting at address 0x020E083C 0x020E083C: 00000000 We are currently getting a 0x0, but according to the thread, it should be a 0x01. You can change that by running: /unit_tests/memtool -32 0x20E083C=0x01 Finally, you can also check the CCM_ANALOG_PLL_ENETn registers by running: /unit_tests/memtool -32 0x20C80E0 4 E Reading 0x4 count starting at address 0x020C80E0 0x020C80E0: 80002003 80002003 80002003 80002003 You should get 80002003 which is the expected value. |
|
I confirm that I see a 125MHz signal (looks like a sine-wave) on GPIO_16 upon boot with imx6q-iris2.dtb. Here is what I get from the above: Looks good so far. I configured Looks like it is not able to recover the RXCLK. I do have a support ticket open with Microchip to review our schematic. In the meantime, I think the plan is to configure the KSZ9893 into Remote Loopback [2, Sec. 5.2.1.7, pg. 102] and begin to experiment with RGMII delay values to find the sweet spot (if it can exist...). |
|
I think we need to define a fixed link in fec. Fixed link Device Tree binding ------------------------------ Some Ethernet MACs have a "fixed link", and are not connected to a normal MDIO-managed PHY device. For those situations, a Device Tree binding allows to describe a "fixed link". |
|
So kind of like this? imx6q-iris2-b.dts |
|
Yes, like that. |
|
Progress... Then, I setup static IP and sent some ICMP traffic out. I made the default gateway a machine running tcpdump. I did NOT get any packets (no ARP table). I then composed some raw ethernet frames and set the right SRC/DST addresses. Still no traffic. So looks like were not getting anything on the RX side of the switch. I'll keep exploring the RGMII delay modes. Should we be concerned that its finding a "Generic PHY" and not the "KSZ9893"? |
|
Just added phy-handle to FEC node. Please give it a try and lets see how it goes. |
|
So I booted up with the updated image and first thing I observed is that the 125MHz clock went away. It also doesn't configure Oops. You said this had to be And I got the 125MHz clock back on the scope. Hmmm. Why did that go away? I did a diff between the previous and current .dts and it seems that almost every phandle got bumped by one: |
|
Here are the boot logs from both: Looks like the latter is missing the |
|
We need to study this system to understand how to setup the |
|
I got a response from Variscite on our schematic. There is an explanation for why RGMII does not work:
|
|
I got the shims to translate the voltage on the RGMII signals and to give 125MHz to J3.52 (ENET_REF_CLK). Not working. It is actually worse than on Jan 24 because all stats remain '0'. There is a regression from Jan 25 (the phy-handle addition) seems to cause it not configure an Anyway, I do not think we have the correct .dtsi expression yet. We do not need to loop back GPIO_16 as the shim has a 125MHz oscillator on it. I believe we have all the correct signalling now. I am concerned that we no longer observe TX statistics like we did on #3 (comment) Here are two boot log, one from a build of uvdl/yocto-ornl@develop and one from the build of uvdl/yocto-ornl@8af38e6 . boot-yocto-sd-2019-03-13-devel-R0-1-shim.log What do you think is the next step to diagnose this? |
|
Ok. I have integrated Edison's patch to instrument the ksz9893 driver. I had to make a branch of the kernel, because the Yocto build kept rejecting my efforts to integrate the patch or build it locally. The kernel branch is on commit uvdl/linux-imx@eaadda0. Here is the boot log: boot-yocto-sd-2019-03-15-debug_ksz9893-R0-1-shim.log So interpreting the trace, I see the following:
Ah, bummer, we didn't instrument this function. There are 4 sections. The first exits silently if there is no mii bus, the second exits with a dev_err() and we don't see that, so its not it. The third could exit silently if it could not allocate a bus. The fourth exits silently if it could not register an mdio bus. My money is on the fourth because: What is this hard-coded |
|
These two discussions seem relevant: Do we need to name some tag in our device tree to have it reference 'mdio'? Or is it something else for SPI? |
|
So upon instrumenting I continue to instrument as the investigation continues. |
|
So with uvdl/linux-imx@418e464, the issue appears to be that the driver is looking for a boot-yocto-sd-2019-03-16-debug_ksz9893-R0-1-shim.log Strangely, though later on in line 349 something does match. So I'm rather confused. Do we need to spinlock? |
|
Based on 'fsl-fec.txt', I made some mods to 'imx6q-iris2-R0.dts' in uvdl/linux-imx@d9ef2d1 I regenerated the kernel using https://github.com/uvdl/yocto-ornl/blob/project/debug_ksz9893/Makefile#L157, but it failed to update the .dtb. I will have to try it manually tomorrow. |
|
So over the weekend, I did two things:
Observations: The behavior remains the same in either branch: FEC comes up, attaches to PHY, appears to communicate with it, but no traffic is accounted for in the driver (and no traffic makes it to the network). Using 2 (commit uvdl/linux-imx@dbc1e6d), I collected the following logs: (the logs are "cleaned up" because some of the printk's overran program output on the terminal). What we see is that the shim (which translates RGMII voltage reference) had no appreciable effect on the operation. With the shim installed, I am able to probe the RGMII and SPI signals. I had previously confirmed that SPI transactions are occurring and that data is flowing (though the traces show alot of overshoot on the edges of all signals - probably need to adjust termination settings in the pinmux). In the below, I captured
From the above, what I can surmise is that:
I think the next step here is to disconnect the 125MHz CLK from the SHIM and do the |
|
That's a problem. I assumed the clock from the SHIM was working. I'll take a look at the kernel and see how is the support to generate the 125MHz clock. |
|
The kernel and devicetree seem to have the logic for generating the 125MHz clock. Could you try this support with this commit ? |
|
Another thing I would like to try is using |
|
On Monday, April 01, 2019 2:53 PM, Edison Fernandez wrote:
Ok, here is the log before the refclk adjustments: ksz_mii_read()/ksz_mii_write() was spamming the console, so I implemented the RLL reporting method to only show changes observed. |
|
As I was setting up for the refclk test, I noticed that the .dtsi file was still setup for switching the GPIO_16, so I made a branch debug/enet-ref-clk-gpio_16, and reverted the two lines from the commit you referenced. The log/scope observation does not show any difference (i.e. the TXCLK did not suddenly fire up...), but I post it here for completeness: |
|
Could you try using memtool to check the GPIO configuration as you did here? |
|
It was still running from last night, so I did the commands: For completeness, I attach the console log. There are periodic changes to the phy 2 reg 2 that are captured for whatever that is worth: Now, this was supposed to be the non-GPIO_16 run, but it behaves the same as the GPIO_16 one. I pulled up the The problem is that the So I am not expecting the I think I captured the steps you recommended from your Friday, March 15, 2019 12:25 PM email. Could you double check? |
|
The steps you followed to build the kernel are correct. Just keep in mind that the dtb will be under your Regarding the refclk I'm confused. was it running on your board? |
|
Ah, ok. So I use the So refclk as probed is passed in to J3.52 (GPIO1[23]) on the DART-MX6. So yes, it is running, but I do not think it has enough drive for some reason. I'm going to probe an unloaded shim to see what the clock generator chip is doing (its a DSC1001DL5-125.0000). |
|
Ok, first of all, let me report on what I have learned so far from the hardware debug. It turns out that my shim had the incorrect part number for the +3.3V regulator to the clock chip. It was only producing +1.5V, so that explains the low output level. For the purpose of the test, I bypassed the regulator and supplied the oscillator from +3.7V thru a 36ohm resistor. Here is the difference:
|
|
Ok, well with your commits, the kernel is panicking. I made a fix/enable-ksz9893 branch to try and resolve it, but I only got as far as the panic in port_get_link_speed(). Here is the log from uvdl/linux-imx@81c3dee on that branch: Basically delayed work is running with incomplete setup. Some clues are found in that the traps I put in to ksz_mii_read() are showing that I had to make two commits to get it past bootup: uvdl/linux-imx@03ed24a, uvdl/linux-imx@5a0ec85 |
|
Could you please give this commit a try? It does the phy_connect in the fec_enet_mii_probe function instead of the ksz sw initialization. |
|
Hmm. So I am having a hard time reproducing the state that I had when I posted #3 (comment). This would have been done against the kernel state in uvdl/linux-imx@ecc8cc3. When I run that same kernel build today, I get the following when I ping: Granted, I am using a dodgy circuit to power the oscillator, but could that cause TXCLK to STOP? And why is it tracking TXCTL during pings? (I expect TXCTL to trigger during ping as the iMX6 MAC is sending to the KSZ9893, but I expect TXCLK to be a stable 125Mhz synced to ENET_REF_CLK) Here is the same condition, but probing the REFCLK oscillator supply: Notice the very strong ripple on the power supply. That cannot be good, but can it explain the behavior of TXCLK? |
|
I posted this invitation on the NXP forum for collaboration. If you are coming here from that, WELCOME! |
|
From the recent emails, these are interesting discussions to think about: |
|
Activity during the last couple of days: On Thursday, April 04, 2019 3:59 PM, Edison Fernandez wrote:
This is from uvdl/linux-imx@ecc8cc3 which has become the reference for a lot of testing since it correctly triggers TXCTL during the ping test… On Fri, Apr 5, 2019 at 11:46 AM, Edison Fernandez wrote:
Results from test 1: It did not appear that the dbg_msg calls got emitted after writing 8 to printk. I remember I had to change a #define in the driver source code to get them (from before, I didn’t do it here). The register
|
|
I had setup another DART-MX6 on the DT6C board running Using test 2 (uvdl/linux-imx@b96df96), I observed a variety of unusual behavior looking at the scope for TXCTL/TXCLK and RXCTL/RXCLK. What I found was that by changing the value of register
When I used
So I think we are close to zero-ing in on a minimal set of configuration needed to make this work. |
|
Ok, so as best as I can tell there are two settings that make everything work:
These have to be set when boot-yocto-ecc8cc36c98e97a29dcbfc5ee550e0c04f14c8ed-0x5b-0x02.log So next steps are to determine what (if any) modifications are actually required to the |
|
With uvdl/linux-imx@40704e0, I am able to use the Generic PHY (in a suboptimal, no flow control way): boot-yocto-40704e0706645154c33fe67e18c4161714421664-0x5b-0x02.log |
|
Actually, I no longer think that it is necessary to set the sniffer port (reg |
|
Tuesday, April 09, 2019 11:47 AM, Edison Fernandez wrote:
On Tue, Apr 9, 2019 at 11:09 AM Vacaliuc, Bogdan wrote:
On Wednesday, April 10, 2019 1:06 PM Edison Fernandez wrote:
I got pretty close with uvdl/linux-imx@87f0822; it may be that we can modify the ksz driver in such a way as to accept the MDIO bus created by the fec. What do you think? |
|
The problem I see with that is that when the fec driver creates its bus it also registers its own read/write functions and we need the bus to use the ones implemented by the switch driver which are that ones that at the end communicate with the switch via SPI in our case. |
|
Ok, well then is there any other driver that does this and how is it implemented? What I'd like to do is find a way to modify the fec driver minimally and using just the DT. The plan is to have something that can be pushed upstream. |
|
Adding the 'go.sh' script that I used above for integration into other distro builds: |
|
A new datasheet is available for the KSZ9893, updated 2019-06-11. Table 3-3, pg. 15 updates the strapping resistor settings, which shows that for RGMII operation Now, when the system boots, it reads: (the setting needs to read back 5b) Well, so it does not appear that any further strapping can help here, as the bit in question above is the RGMII ingress delay for the clock. From pg. 122 of KSZ9893R, this defaults to 0, does not have a strapping setting, and (unfortunately) needs to be 1 in our application. So it appears that we still cannot achieve a driverless operation of the KSZ9893 as the delay can only be set by programming the 0x3301 register. |
Add SPI mode support in KSZ9893R using as reference the kernel driver available for KSZ9897 in ksz_spi.c
The text was updated successfully, but these errors were encountered: