Series Termination Noise Emission

[pathfinder49]

Summary

I've investigated unexpected digital cross-talk in V1.1. This leads me to conclude that the series termination injects significant noise into adjacent traces.

Observations

For the majority of channels I looked at, the digital cross-talk is significantly improved. In particular, the new digital trace routing and channel layout seem to have resolved the cross-talk from digital traces passing near channels.

Channels with series termination in the vicinity of analogue traces give cross-talk

Channel 8 is close to the series termination of many channels. Each of these channels exhibits significant digital cross-talk to channel 8.

The table below lists channels and the observed 2.55 MHz digital cross-talk spur.

Channel	Description	Spur intensity /dBmV
17	Series termination near the P5V0_ref-via	-33
28	Series termination near the ch8 op-amp	-60
30	Series termination near ch8 op-amp and resistor network	-58
19	Series termination near resistor network	-61
8	Ch8 is near its own series termination	-61
22	Somewhat further away from the resistor network	-75
25	Somewhat further away from the resistor network	-77
10	Termination about 1 cm away from ch8, yet still gives (reduced) cross-talk	-80
16	Termination slightly further away than ch10	none
31	No nearby traces and series termination far away	none
11	Again traces and termination are reasonably far away	none

Overall, the spur intensity scales with the distance of the series termination to the analogue traces rather than the digital trace distance to vias.

Channel 8 seems to be the worst case scenario. Ch11 also has some terminations near the DAC. However, these only result much weaker spurs.
The spurs are:

• ch11 -> ch11: -80 dBmV
• ch17 -> ch11: -87 dBmV

Identical trace routing away from the series termination does not lead to cross-talk.

The routing of the ch28 digital traces through channels 8, 11, 14, 17 and 20 is identical. However, a spur is only seen in channel 8

Series termination radiated emissions

Connecting a sniffer coil to the spectum analyser, I can detect 2.55 MHz radiation that appears to be coming from the series termination rather than the FPGA.

Conclusion

The series termination appears to cause significant emissions over distances upto 1 cm. It is unclear to me what the mechanism for these emsisions is.

[pathfinder49]

@gkasprow Have you got any concrete ideas how the noise would be coupling over such large distances?

@hartytp The easiest fix would be to drop the series termination. However that would leave us unable to kill the parallel termination cross-talk. Otherwise, we might be able to eliminate this if we add some more layers and find a better place for the series termination.

[hartytp]

The table below lists channels and the observed 2.55 MHz digital cross-talk spur.

So this table is looking at the output of channel 8 while repeatedly setting one other channel to -9V5, right?

[hartytp]

@pathfinder49 good work on this. There are two relatively quick tests we should do to verify your hypothesis before we start thinking about a fix. To save time, let's pick just one channel that has significant cross-talk to channel 8 and work on that.

1. Record the voltage waveforms at the DAC with a low-capacitance (active) probe and the cross-talk spur size. My guess is that the voltage levels will be marginal anyway with both series and parallel termination, so that needs to be fixed in any case.
2. Remove the parallel termination (e.g. DNP the caps). Record the voltage waveforms at the DAC and the cross-talk spur size
3. Repopulate the parallel termination and DNP the series termination (replace with a solder blob to short). Record the voltage waveforms and spur size

I'd guess that the answer is that there is a voltage drop across the resistor (due to the current flow) which means there must be a sizable e-field inside, which can couple capacitively to nearby traces.

Once we verify with certainty that the termination resistors are the source of the issue we can look at potential fixes in the layout. I guess that ideally we'd have the FPGA inside a screening can but that likely won't fit (can we make Fastino a bit longer without breaking CPICs compatibility?).

What do you think @gkasprow ?

[hartytp]

@gkasprow I'm sure we had a discussion about series/parallel termination, but I can't find the issue right now. Can you remind me why we ended up with both populated in this revision? Was that an oversight?

@pathfinder49 can you also remind me how bad the parallel-termination spurs are? If there isn't an easy layout fix to the series termination spurs (without reducing the channel count) we should consider making the decision to just live with the parallel termination spurs if they are sufficiently small...

[gkasprow]

series termination does not create such return current as parallel does. Essentially series termination eats up half of the signal amplitude, then the signal propagates and bounces back from open-end, inverts its phase and this way recreates valid signal amplitude. It is fragile to the load - the higher the capacitance, the worse the signal inversion and the amplitude reconstruction. if the capacitance is too high and the line shorter than signal rise time, the RC filter starts dominating over the long line behavior.
The parallel termination involves both capacitive and inductive coupling (with adjacent tracks) since the current is passing through the line.
In series termination, the capacitive coupling dominates and inductive one is much lower because the current is much lower since it does not have a way to close the loop to GND.
We can't have both terminations enabled because they will simply eat half of the signal. The circuit may still somehow work, but the jitter may increase and transmission will not be reliable.
So the recommendation was to use only series termination. In v1 there was no physical space on PCB, but once we removed the CMCs, we gained more space, and series termination could be added. We left the parallel one just in case we want to revert back to it.
I'm not surprised that you observe such behavior with both terminations enabled - since we operate with lower logic levels, the DAC logic inputs work close to the noise zone (or even in it) and are very fragile to any couplings from nearby tracks.
So, the first thing to do - disable all parallel terminations and repeat the measurements.

[pathfinder49]

can you also remind me how bad the parallel-termination spurs are?

The only large spurs I've seen are related to the series termination. The remaining spurs are all around -80 dBmV.

[gkasprow]

what do you mean by related? You observe them in proximity of serial termination resistors?

[pathfinder49]

what do you mean by related? You observe them in proximity of serial termination resistors?

I should have expressed myself more clearly. All spurs that I have seen which are not approximately -80 dBmV are listed in the table at the top of this thread.

[pathfinder49]

• Remove the parallel termination (e.g. DNP the caps). Record the voltage waveforms at the DAC and the cross-talk spur size
• Repopulate the parallel termination and DNP the series termination (replace with a solder blob to short). Record the voltage waveforms and spur size

De-soldering the parallel termination capacitors makes no difference. I've measured the digital crosstalk with both the ch8 and ch28 parallel termination de-soldered an the cros-stalk is unchanged (-61 dBmV).

Re-soldering the parallel termination capacitors for ch28 and replacing the series termination with a short also doesn't change the cross-talk (-60 dBmV).

Replacing the series thermination fo ch28 with an open circuit, eliminates the cross-talk.

Edit: I have performed a complimentary measurement on channel 30. Channel 30 has the FPGA side series termination vias closer to channel 8. For the open circuit measurement, the ch30->ch8 digital to analogue crosstalk spur is reduced in amplitude, but still measurable.

[pathfinder49]

@gkasprow Is it plausible that the vias around the series termination produce the cross-talk by inductive coupling? Only eliminating the digital trace currents seems to give a significant reduction in cross-talk. I find it quite unlikely that the trace routing after the series termination is at fault. The routing is identical for channels not experiencing cross-talk.

[gkasprow]

You can route the signal above the board using a piece of thin wire and then see if that helps

[pathfinder49]

You can route the signal above the board using a piece of thin wire and then see if that helps

The series termination resistors for ch28 have vias immediately before and after them. I've desoldered the ch28 series termination resistor and poulated the ch28 parallel termination. I've also added an air wire from the FPGA side pad of the series termination to the DAC side of the parralel termination capacitors. This configuration eliminates the DAC side via from the current path. The FPGA side via remains within the path. This configuration significantly increases the digital cross-talk of ch28 to ch8. The spur amplitude is now -55 dBmV. Touching (but not moving) the wires with my finger increases the spur to -40 dBmV. Touching the wires with a plastic pen makes no difference.

I would speculate the following:

• Touching the wire increases the spur amplitude as my fingers increase the capacitance of the wire. This increases the current flowing through the via.
• The spur amplitude is increased with this configuration due to a combination of the following factors
  • The air wire capacitance is different to that of the trace. I wouldn't expect this to make a large difference.
  • The current flow within the FPGA and DAC side series termination vias is in opposite directions. As the separation of the two vias is smaller than the via-ch8 distance, it seems plausible for the inductive coupling of the two vias to partially cancel. (Edit: I must have looked at the wrong channel schematic. For ch28, most of the FPGA side vias are far away from ch8.)

[pathfinder49]

I'm fairly confident blaming this on the series termination vias. This will need fixing in a new revision. I can think of two ways forward. @gkasprow please chip in if you can think of other good options.

1. Remove the series termination.
2. Move the series termination away from all DACs. I doubt this is possible without: significantly reworking the layout, adding more layers and/or increasing the board length.

Points to Consider

On the whole, digital crosstalk on V1.1 looks much improved. I'm aware of two digital crosstlak patterns on V1.1.

1. The vias of the series termination cause crosstalk to nearby channels. In the worst case this gives a -30 dBmV spur. This is on-par with the worst spurs found in V1.0 This crosstalk is concentrated into very few channels, leaving the majority of channels "clean".
2. Crosstalk into an adjacent channel. Some channels exhibit digital crosstlak to the adjacent DAC channels. This results in -80 dBmV spurs. Some, but not all, of these spurs are eliminated by removing the parallel termination. This crosstalk is scattered around the DAC channels.

[hartytp]

I'm not surprised that you observe such behavior with both terminations enabled - since we operate with lower logic levels, the DAC logic inputs work close to the noise zone (or even in it) and are very fragile to any couplings from nearby tracks.
So, the first thing to do - disable all parallel terminations and repeat the measurements.

@gkasprow to be clear, the spurs in @pathfinder49's data are not due to jitter in the signals. He is updating one channel and looking at a different channel (which isn't updating). The cross-talk he sees is from the clock line on the channel that updates to the output of the channel that doesn't update.

This only happens for the couple of channels that are physically located close to the termination resistors, so it seems to be a layout issue (maybe more via couplings).

[hartytp]

what do you mean by related? You observe them in proximity of serial termination resistors?

To be clear: the issue is that there is quite a large cross-coupling between the various digital lines and the output of channel 8. This appears to be because of the proximity of that channel to the digital signal routing. I don't think that the termination resistors are directly responsible however, adding them forced us to place extra vias and traces and I suspect those are what causes the issue (e.g. via-trace coupling). See below

[hartytp]

@gkasprow so the conclusion seems to be this (@pathfinder49 correct me if I'm wrong):

• the new layout works really nicely except for channels 8 and 5 (are these the right ones @pathfinder49? are there any other badly effected ones). These channels are very close to lots of vias and digital traces which seem to induce cross-coupling
• to fix the layout we need to move the vias and traces further from these channels

@gkasprow can you see any way of doing this easily? AFAICT (but @gkasprow please correct me if I'm wrong!) there isn't any real way of improving the layout with either:

• remove the series termination (simplifies the routing quite a bit)
• or, lengthen the PCB and do some major re-routing to move the FPGA further from the DACs

what do you think @gkasprow ?

[hartytp]

Crosstalk into an adjacent channel. Some channels exhibit digital crosstlak to the adjacent DAC channels. This results in -80 dBmV spurs. Some, but not all, of these spurs are eliminated by removing the parallel termination. This crosstalk is scattered around the DAC channels.

In an ideal world we would remove the parallel termination. Even if it doesn't kill all the spurs, it does improve the spectral purity of the outputs. And, as always, it's hard to track down the causes of spurs when there are multiple sources. Removing the spurs we understand would make it easier to understand the remaining spurs in a future release if we wanted to.

Having said that, -80dBmV is acceptable so if comes down to a choice between having a couple of channels with really bad channels or living with the parallel-termination spurs then I'd vote to keep the parallel termination.

Either way, let's wait to see what @gkasprow thinks about the layout before making a decision. good work on this!

[pathfinder49]

the new layout works really nicely except for channels 8 and 5 (are these the right ones @pathfinder49? are there any other badly effected ones).

I haven't done a complete measurement of the impacted channels. I've measured:

• ch8 (measured upto -30 dBmV)
• ch11 (measured upto -80 dBmV)

I suspect channels 1, 3, 5 and 7 are also impacted. I'd expect all of these to be less bad than ch8. (The colsest series termination to these channels is on the bottom of the board.)

[hartytp]

okay, so it's the cluster of channels closest to the FPGA which have pretty serious digital cross-talk. That's not surprising...

I do wonder if removing the series termination resistors will allow us to buy ourselves enough space to fix this or whether we need to look at something more drastic like lengthening the board and pulling the FPGA back/adding another layer (@gkasprow )

[hartytp]

anyway, the good thing is that this design revision seems to be a strict improvement on the previous even if it doesn't fix all issues. (and, again, by removing a lot of spurs it makes it easier to drill down on the remaining ones)

[pathfinder49]

okay, so it's the cluster of channels closest to the FPGA which have pretty serious digital cross-talk. That's not surprising...

I do wonder if removing the series termination resistors will allow us to buy ourselves enough space to fix this or whether we need to look at something more drastic like lengthening the board and pulling the FPGA back/adding another layer (@gkasprow )

I believe the issue has very little to do with proximity to the FPGA.

1. The DACs are no closer to the FPGA than in V1.0. We did not see such crosstalk in V1.0
2. The cross-talk scales with the distance from the series termination. This is quite uncorrelated with the FPGA pin positions.
   1. Updating channels that come from a similar part of the FPGA, but do not have series termination near a channel, does not cause cross-talk.
   2. The channels that do cause crosstalk to ch8 are spread around the FPGA.

Removing the series termination (vias) should eliminate the machanism of this coupling. The distance to the FPGA should be fine.

[hartytp]

I believe th issue has very little to do with proximity to the FPGA.

Yes, you're right, sorry! What I meant was the cluster of vias around the series terminations, which are located close to the FPGA, (not the FPGA itself).

If we remove the series termination then we can presumably make this better. So our options are to do that and hope this all goes away; to move the FPGA to give ourselves more room; or something else along those lines.

[gkasprow]

Let's first exclude other possible reasons.
Does it happen on other boards in an identical way?
channel 4 and 6 have a similar amount of signal passing nearby.
If the issue appears also on other boards in a similar way, I would first check how the channels differ. Even if you copy the layout with the Altium tool, we have a lot of signals that are passing below. The pours are also connected differently.

[gkasprow]

@pathfinder49 are you toggling the digital lines of all DACs simultaneously or only of neighbouring DACs?

[gkasprow]

I have one crazy idea. Is there an option to run the design with a completely different clock and see if we still observe the issue with channel 8? It can be 1.8MHz or so.

[gkasprow]

I'm looking carefully at the differences between channels
Some channels have digital traces close to P5V4_DAC_VLOGIC_x via.
CH2 has NSS16 and not so close NSS13
CH4 has NSS22 and LDACn13
CH6 has MOSI19, strong coupling
CH8 has CLK28, but not so close; NSS25, strong coupling
CH11 has CLK25
CH24 has MOSI29
CH21 has MOSI29
CH18 has MOSI29
CH15 has MOSI29
CH12 has MOSI29
CH9 has MOSI29

What we are observing is 2.5MHz spurs. The signal that has this frequency is NSS.
And according to the via-trace coupling, the CH8 has let's say double coupling with NSS25
CH2 and CH4 have the via also coupled with NSS signals, but not that strong as CH8

This theory can be verified by switching of the NSS25. One can also remove C143_8 and see if the effect gets much worse.
Another idea is to add a ceramic 10u cap directly between pin 14 and pin 12 od DAC8. This would bypass the interference.

[pathfinder49]

Let's first exclude other possible reasons.
Does it happen on other boards in an identical way?
channel 4 and 6 have a similar amount of signal passing nearby.
If the issue appears also on other boards in a similar way, I would first check how the channels differ. Even if you copy the layout with the Altium tool, we have a lot of signals that are passing below. The pours are also connected differently.

How are you refering to the channels? I'm refering to them by the label printed on the board (0-31). The Altium model labels the channels (1-32). The problematic "channel 8" is the one with the long side directly facing the FPGA and series termination. This has deffinitly confused me quite a few times 😅

@pathfinder49 are you toggling the digital lines of all DACs simultaneously or only of neighbouring DACs?

I am toggling the digital lines of only a single channel. I then look which (if any) channels aquire a digital spur in the analogue output. This crosstalk is not between neighbouring DACs.

An example:
If I toggle the digital lines of only channel 28, I measure no large spurs in the analogue output of any of the nearby channels. Checking for crosstalk caused by the toggling of channel 28 digital lines in the analogue output of all 32 DAC channels, I find that the only channel that sees a large analogue spur is channel 8. Channel 28 and channel 8 are on opposite sides of the board.

[pathfinder49]

CH24 has MOSI29
CH21 has MOSI29
CH18 has MOSI29
CH15 has MOSI29
CH12 has MOSI29
CH9 has MOSI29

Assuming you are using the Altium numbering scheme, this is what I meant by this statement:

The routing of the ch28 digital traces through channels 8, 11, 14, 17 and 20 is identical.

When only the digital traces of channel 28 (channel 29 in Altium) are toggling, none of these channels except channel 8 (channel 9 in Altium) aquire a digital spur in their analogue output.

And according to the via-trace coupling

To me, the above point suggests this is not the digital traces coupling to analogue vias.

What we are observing is 2.5MHz spurs. The signal that has this frequency is NSS.

All toggling digital traces should have a ~2.5 MHz component. They repeat a fixed pattern at 2.5 MHz. (I've verified this by sticking the spectrum analyser onto the digital test-points)

[pathfinder49]

I suspect we've gotten our channel conventions mixed up. Happy to answer the rest of your questions if I'm missinterpreting 😄

[pathfinder49]

Oh I forgot to say earlier, the configuration is as follows:

• I have a single Fastino connected to an otherwise empty Kasli.
• Kasli is powered by an isolated 12V supply. This breaks DC ground loops.
• I've Improved the AC isolation of the supply using a large CMC and large capacitors to make an LC filter. This breaks AC ground loops.
• Fastino is connected to an BNC-IDC breakout board using a short ribbon cable. The breakout board is populated with CMCs.
• I use a BNC cable to connect the analogue output to a grounded, 50 Ohm spectum analyser. The input of the analyser is protected using a 100 kHz to 8GHz DC-block. The analyser noise floor at 2.5 MHz is -90 dBmV (with a bit of averaging).
• A ground strap directly connects the Fasino ground to the spectrum analyser ground. I have attached this to Fastino via the frontannel. (The front-pannel is well grounded.)
• The Kasli is mounted in a floated rack on a table. Fastino and the BNC-IDC board are taped down on the table and are not in contact with each other, Kasli or the rack.

[gkasprow]

I'm refering to Altium channel numbering

[gkasprow]

@pathfinder49 Did you observe such issues on more than one board?

[pathfinder49]

@pathfinder49 Did you observe such issues on more than one board?

So far, I've only checked one board.

[gkasprow]

@pathfinder49 Do you have another board to verify your finding?

[pathfinder49]

I have a total of 3 boards in the lab. I'll take a look at the others on Monday.

…

On Sat, 29 Aug 2020, 10:19 Greg Kasprowicz, ***@***.***> wrote: @pathfinder49 <https://github.com/pathfinder49> Do you have another board to verify your finding? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#76 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADNCDOO5WMEYHM72P2E3HE3SDDB3DANCNFSM4QLD2CKQ> .

[hartytp]

@pathfinder49 thanks! I'm willing to bet (beer?) that the issue will be the same on all boards, but worth checking.

@gkasprow given @pathfinder49's observations (and assuming the issue is the same on all boards) I don't see any way this can be anything other than coupling from the vias on the digital lines to the analog outputs. So, it does seem to me that we need to move those digital vias (associated with the termination resistors) away form the analog parts. Do you agree?

[gkasprow]

Yes, but before we do that, let's disconnect this particular trace segment by cutting the wire close to DAC and replacing it with a piece of magnet wire.

[hartytp]

Yes, but before we do that, let's disconnect this particular trace segment by cutting the wire close to DAC and replacing it with a piece of magnet wire.

So you'd like to cut one of the NSS lines in two places: as close to the FPGA as possible and as close to the DAC as possible and then replace the trace with a piece of magnet wire that's routed as far from channel 8 as possible?

That's going to be very tricky (impossible?) since traces are all routed on internal layers and only surface at the series termination pads. If we attach the air wire at the series termination pad then I don't think it will help: we're replacing a small antenna (the via) with a larger one (the air wire) anchored to the exact same spot in the PCB. Worse, we wouldn't even be disconnecting the original via (since we need that to bring the signal to the top layer), so AFAICT this can only make things worse.

Could you post an image of the layout with a sketch of exactly what change you suggest we make?

[pathfinder49]

I believe the closest thing I can easily do was this.

[gkasprow]

The termination resistor is on the top layer, the DAC is also on the top layer. If we cut the trace close to the DAC and solder wire directly to the DAC pad (and later to the vertically placed termination resistor), it would be possible and eliminate the internal trace segment with problematic via.
The thing @pathfinder49 did was close, but he did not (?) cut the trace close to DAC so the original trace segment was still there.
By adding the wire in the air, with the original trace still connected, a large stub was created which could cause additional side effects.
If we cut the internal trace segment, we would eliminate the problematic coupling between DAC via and digital signal.
Of course, the external wire may make things worse, but not necessarily. It depends on what dominates - radiated emission or capacitive coupling.

[pathfinder49]

Making further measurements with the ch28 air wires, I now believe the air wire crosstalk is dominated by wire radiation. (Suggested by @hartytp)

• Bending the air wires to be closer/further from ch8 significantly changes the crosstalk.
• Channels other than ch8 that are near the air wires experience similar crosstalk. These channels did not experience crosstalk without the air wires.
• I can easily measure 2.5 MHz radiation from the wires with an air coil. connected to the spectrum analyser.

@gkasprow we'd need to find another way to perform your measurement.

[pathfinder49]

Let's first exclude other possible reasons.
Does it happen on other boards in an identical way?

I have done digital to analogue crosstalk checks on the other two Fastino V1.1 boards I have in the lab. These all behave the same.

[pathfinder49]

• Remove the parallel termination (e.g. DNP the caps). Record the voltage waveforms at the DAC and the cross-talk spur size
• Repopulate the parallel termination and DNP the series termination (replace with a solder blob to short). Record the voltage waveforms and spur size

De-soldering the parallel termination capacitors makes no difference. I've measured the digital crosstalk with both the ch8 and ch28 parallel termination de-soldered an the cros-stalk is unchanged. (-61 dBmV)

Re-soldering the parallel termination capacitors for ch28 and replacing the series termination with a short also doesn't change the cross-talk. (-60 dBmV)

Replacing the series thermination fo ch28 with an open circuit, eliminates the cross-talk.

I have repeated this measurement with channel 30->ch8 crosstalk (31->9 in Altium). The behavior here is similar to that seen before.

• With the series termination resistor and no parallel termination, I observe a -58 dBmV spur.
• With the series termination shorted and parallel termination populated, the spur amplitude increases slightly to -56 dBmV. (I used bare wires to bridge the pads. These protrude slightly over the DAC side pads.)
• With an open circuit at the series termination, the the spur amplitude decreases to -70 dBmV.

One of the differences between this termination and the ch28 series termination is that the FPGA side vias of the ch30 series termination are also close to channel 8.

This measurement is consistent with capacitive coupling from the vias. In particular, the difference in the open circuit behavior between the ch28 and ch30 measurement is of interest.

For reference, here is the schematic for the region of interest.

[pathfinder49]

I think there is very little we can do except not have digital vias near analogue circuitry. Closing in favor of a discussion whether to remove the series termination or more significantly rework Fastino.

[gkasprow]

OK, all the time I was referring to CH8 according to Altium numbering and you were referring to ARTIQ numbering