[Balloon] White Rabbit Reliability

[alexazon]

Dusan and me noticed that plugging in and out the fibre cable has a fancy effect on our timing receivers. Every time when a node is synchronized again, all Outputs are “shifted” +-4ns (compared to a node which wasn't unplugged).

Test setup:

• Scope
• PMC Card (based on proposed_master) => Connected to the scope (red)
• EXPLODER5A (based on balloon, plugged in and out) => Connected to the scope (green)

Start condition:

We restarted both receivers and just used the clock generator and the ECA to output pulses. They are perfectly aligned. You can see that on the following picture:

Reproducing the error:

As already mentioned, just plug the fibre cable out and in again. The clock/output pulse will be shifted in a range of +-4ns. We also got a picture of the behaviour:

This happens on PMC and EXPLODER.

Exploder bitstream information:

[root@scuxl0095 ~]# eb-info dev/wbm0
Project : exploder5_csco_tr
Platform : exploder5 +db[12] +wrex2
FPGA model : Arria V (5AGXMA3D4F27I3)
Source info : balloon-1267
Build type : developer preview
Build date : Thu Sep 15 03:17:22 CEST 2016
Prepared by : Jenkins Nightly Build csco-tg@gsi.de
Prepared on : tsl002.acc.gsi.de
OS version : Debian GNU/Linux 8.5 (jessie), kernel 3.16.0-4-amd64
Quartus : Version 16.0.0 Build 211 04/27/2016 SJ Standard Edition

1ff7194 ftm: fixed EBM/PQ...

[alexazon]

Info:
Unable to reproduce with the latest version of balloon.

[miree]

After tests with SCU3, EXPLODER5A, VETAR2, SPECv4, this issue could be reproduced on EXPLODER5A only. It seems to appear on units with Arria V FPGA.

[alexazon]

It seems to appear on units with Arria V FPGA.

Did you test this with a pexaria5?

[bradomyn]

no yet, we don't any in the lab, we have to use one from the testing facility, and now they are busy testing more critical bugs, namely, missing events... We have assumed that the problem is related to ArriaV since already two formfactor are affected: PMC and Exploder. Since we are not going to fix this problem for Balloon before release, when we start debugging we'll check the Pexarria. 2016-11-24 10:04 GMT+01:00 alex-cscotg <notifications@github.com>:

…

It seems to appear on units with Arria V FPGA. Did you test this with a pexaria5? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#6 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABIZCqTtLJVQSLe-bHIf6LEngB-enjN4ks5rBVMxgaJpZM4J-9pQ> .

[miree]

The issue was reproduced on a pexaria5 by Alessandro while monitoring the WR parameters "bitslide" and "setpoint". These were unrelated to the observed shift (+- 4 ns) in the pps signal. After discussing with Alessandro, Alex, Cesar and me, it is suspected that the problem might be in the SERDES Transmitter/Receiver of the Arria V FPGA. Perhaps the tuning of the 125MHz clock phase has some feedback on the SERDES?

It was decided to do the following:

• Study SERDES (datasheet & VHDL).
• Check if there were changes in the SERDES configuration in the past.
• Check if it is possible to reset the SERDES after each WR synchronization.
• Slow down the WR synchronization and check if this avoids the problem.

In addition, it was also observed that, sometimes, while being in TRACK PHASE the pps signal was varying by 1ns (without unplugging the WR link).

[alexazon]

/closed