running ARTIQ Sayma hardware at WUT

[jbqubit]

From #358

@hartytp said

@gkasprow If you give me remote access to one of your PCs that's connected to a Sayma then I'm happy to install ARTIQ and load it onto your board for tests.

@gkasprow said

@marmeladapk please help @hartytp to get access to our ARTIQ computer in our lab

@marmeladapk, @hartytp please do installation discussion in this Issue (vs eg email).

[hartytp]

Joe, we're on it. Don't worry.

This doesn't need an issue, I'll take care of it.

[jbqubit]

I know you'll get it done @hartytp. I'd like to make sure that the Q&A that goes along with bootstrapping @marmeladapk and @gkasprow understanding of how to install ARTIQ is visible. This will help M-Labs understand where the hangups are in installing the system, gaps in documentation.

[jbqubit]

@marmeladapk Were you able to build .bit for the Sayma?

[marmeladapk]

Yeah, although I thought that @hartytp offered to do this.

[hartytp]

Will do once i get remote access to a pc

[marmeladapk]

Soo, I may have jumped the gun after @jbqubit message and installed it myself. Steps I did:

1. Updated Conda _{(I think this point should be not recommended in manual but required for users like me - 1. I thought conda would be updated along with other system updates (TBH I'm not sure why I thought that) 2. Installation proceeded with older version of conda with message I treated as a warning, not error, but some dependencies weren't pulled).}

2. Clone artiq-dev repo: git clone --recursive https://github.com/m-labs/artiq ~/artiq-dev/artiq

3. cd ~/artiq-dev/artiq

4. Created conda environment: conda env create -f conda/artiq-dev.yaml

5. Activated environment source activate artiq-dev

6. pip install -e .

7. (Vivado and OpenOCD were installed earlier, but I had to change path to Vivado in miniconda3/envs/artiq-dev/lib/python3.5/site-packages/migen/build/xilinx/vivado.py line 185)

8. ls ./gateware/targets to see available targets

9. python3 -m artiq.gateware.targets.sayma_rtm
python3 -m artiq.gateware.targets.sayma_amc_drtio_master

I didn't flash the board yet, I may do it on Monday if @gkasprow won't use it.

@hartytp @jordens Is this, more or less, what I should do to run Artiq and to be able to synthesize latest gateware?

Notes about installation process - I think it's quite easy, however I would save some time if I were to update conda right away at the beginning - perhaps including source deactivate (if you're in some env) and conda update conda commands to installation manual could help some users and make manual more idiotproof.

Another note: in this part of the manual I think that points 6 to 9 shouldn't link to "Installing ARTIQ from source" but just relevant parts could be copy-pasted from there. I wasn't sure, for example, if I had to export path to clang or not.

[jordens]

@marmeladapk

• We have --gateware-toolchain-path to specify the xilinx toolchain path. The fact that that doesn't work for the RTM bitstream is a bug.
• openocd should also be pulled in by artiq-dev that will override your other openocd.
• Is a sentence like Install an up-to-date conda package be sufficient? Could you file an issue (or better a pull request with the change that would have helped you)?
• You are correct. The content of the items 6-9 should be referenced in such a way that it is absolutely cler to what section they refer. Could you file an issue?

[marmeladapk]

@jordens I didn't specify --gateware-toolchain-path, it's not mentioned in the manual so that's not a bug.

[jordens]

@marmeladapk not a bug, yes. just a hint. you can get the usual help on these command-line tools by running them with -h

[hartytp]

Soo, I may have jumped the gun after @jbqubit message and installed it myself. Steps I did:

@marmeladapk Great! Good for you. In the long run, it's much better if you can do this yourselves.

[marmeladapk]

Should I flash sayma_amc_drtio_master and sayma_rtm to work with Artiq?

[marmeladapk]

@jordens sayma_rtm target does not have --gateware-toolchain-path, should I file a bug?

[jordens]

@marmeladapk The drtio_master bitstream is for something else. Ignore it. The sayma_rtm bitstream needs to be loaded manually over JTAG. @sbourdeauducq can guide you through it.

You can file an issue for the missing --gateware-toolchain-path and other options. Those should be there (or sayma_rtm shoudl be better integrated).

[jordens]

@marmeladapk Or you could just give it a try and add an ArgumentParser using the code from the other targets as baseline. Do a pull request when you'd like us to look at it.
And if you come to our IRC channel (#m-labs @ freenode) we can help you with faster response times.

[hartytp]

@marmeladapk Could you try running Sayma RTM with an external 1.2GHz clock, please?

Configuration I'm using is:

• 1.2GHz clock attached to DAC_CLK SMP where clock mezzanine would go (attach clock source to one SMP, terminate the other with 50R to ground)
• Change this line to 0*DAC_CLK_SRC_SEL); and rebuild AMC firmware. There should now be a 1.2GHz clock fed to the HMC7043, and a 600MHz clock fed from the HMC7043 to the DACs. (Please verify that the clock looks okay using the UFL test points and a fast scope or spectrum analyzer)
• Now, look at the DAC's outputs on a scope. There should be a sawtooth ramp on each DAC port (see the photo I posted on another issue).

My finding is that the DACs all produce a sawtooth briefly after the firmware boots up, which then cuts out presumably due to a loss of the JSED link. If that happens for you, can you check the DAC power supplies etc for glitches and do some general sanity checking to see if there are any obvious hardware reasons for this error (otherwise, it is a M-Labs software/gateware issue).

Thanks!

[hartytp]

Also, @marmeladapk @gkasprow IIRC, you previously saw RF out of Sayma using a bitstream that @enjoy-digital sent you. Please can you remind me what your configuration was, and whether you saw any issues. e.g. did you get nice RF out and did the JSED links stay up? If so, this is less likely to be a hardware issue...

[gkasprow]

I used Florent binaries to test all Saymas working with 1GHz external clock applied to the SMP connectors.
The link was stable, but there is still unsolved issue with mux control line that caused some boards to loose the link. Florent attched this FPGA pin to GND and on some boards this particular pin was high, while on some of them was low.
I asked him to route it to the register and managed to control it from the software, this way I managed to make all boards working with Allaki. Jakub wrote little Python script that let me control the Allaki gain and RF switch.

[enjoy-digital]

@hartytp @gkasprow: from what i remember, i was able to also get the JESD link stable after we found that one calibration pin was floating on the AD9154 and fixed it. But this was with the designs from my sayma_test repository. I'm at M-Labs office tomorrow and will test if JESD is also stable with ARTIQ.

[gkasprow]

I applied this fix to all boards prior shipment

[jbqubit]

Updated Conda (I think this point should be not recommended in manual but required for users like me - 1. I thought conda would be updated along with other system updates (TBH I'm not sure why I thought that) 2. Installation proceeded with older version of conda with message I treated as a warning, not error, but some dependencies weren't pulled).

That's why the manual says to update conda (cf m-labs/artiq#785).

I didn't specify --gateware-toolchain-path, it's not mentioned in the manual so that's not a bug.

It wouldn't hurt if the manual indicated that this path is commonly needed. Just to grease the skids.

python3 -m artiq.gateware.targets.sayma_rtm
python3 -m artiq.gateware.targets.sayma_amc_drtio_master

That's great progress!

The sayma_rtm bitstream needs to be loaded manually over JTAG.

artiq_flash seems to handle flashing sayma_rtm when I checked last week.

Do a pull request when you'd like us to look at it.

@jordens Please let @marmeladapk focus on getting up to speed with ARTIQ on Sayma and test the hardware. Learning to edit ARTIQ source and push patches is desirable but maybe in a couple weeks. :)

I'm at M-Labs office tomorrow and will test if JESD is also stable with ARTIQ.

@enjoy-digital Glad that you'll be working next to @sbourdeauducq in HK. :) How long will your visit last?

[hartytp]

The link was stable, but there is still unsolved issue with mux control line that caused some boards to loose the link. Florent attched this FPGA pin to GND and on some boards this particular pin was high, while on some of them was low.

You mean the clock mux? I didn't see any issues with the clock, which was still present after the JESD link went down. While, it's possible that there was a brief glitch that I didn't catch, I don't think that was the issue in my case.

[gkasprow]

@hartytp some boards had this issue, some didn't

[hartytp]

Okay. Well, I'm not sure why my JESD links lost lock. Will be interesting to hear what @enjoy-digital finds in HK...

[jbqubit]

@marmeladapk How's this going?

[marmeladapk]

@sbourdeauducq Where is this script located?

ed: OK.

[sbourdeauducq]

python3 -c "import artiq; print(artiq.__path__[0])", then it's in frontend.

[jbqubit]

cf m-labs/artiq#854 (comment)

Oops, presumably Greg's test wasn't done with Artiq. That should be the next step, I guess...

@marmeladapk Would be best to just retrieve a RTM from Creotech/Techonosystem so you can duplicate the AMC+RTM that the rest of us are using.

[hartytp]

@marmeladapk I'll post you the second RTM that I have on Monday (the one with the 3V3 regulator issue).

[marmeladapk]

I don't know what's the scope of @wizath 's work, however when he finishes then I could work on Sayma in MTCA crate.

[marmeladapk]

@marmeladapk Would be best to just retrieve a RTM from Creotech/Techonosystem so you can duplicate the AMC+RTM that the rest of us are using.

@gkasprow ?

[gkasprow]

I will check what is easier to do. Guys at CTI also use RTM and AMC intensively...

[sbourdeauducq]

@marmeladapk Just do something like this (not tested so you may have to finish it yourself):

--- a/artiq/frontend/artiq_flash.py
+++ b/artiq/frontend/artiq_flash.py
@@ -232,19 +232,15 @@ class ProgrammerSayma(Programmer):
             "reset_config none",
             "adapter_khz 5000",
             "transport select jtag",
-            # tap 0, pld 0
-            "source {}".format(self._transfer_script("cpld/xilinx-xc7.cfg")),
-            # tap 1, pld 1
             "set CHIP XCKU040",
             "source {}".format(self._transfer_script("cpld/xilinx-xcu.cfg")))
         self.add_flash_bank("spi0", "xcu", index=0)
         self.add_flash_bank("spi1", "xcu", index=1)
 
-        add_commands(self._script, "echo \"RTM FPGA XADC:\"", "xadc_report xc7.tap")
         add_commands(self._script, "echo \"AMC FPGA XADC:\"", "xadc_report xcu.tap")
 
     def load_proxy(self):
-        self.load(find_proxy_bitfile("bscan_spi_xcku040-sayma.bit"), pld=1)
+        self.load(find_proxy_bitfile("bscan_spi_xcku040-sayma.bit"), pld=0)
 
     def start(self):
         add_commands(self._script, "xcu_program xcu.tap")
@@ -337,10 +333,8 @@ def main():
                 programmer.write_binary(*config["firmware"], firmware_fbi)
             elif action == "load":
                 if args.target == "sayma":
-                    rtm_gateware_bit = artifact_path("rtm_gateware", "rtm.bit")
-                    programmer.load(rtm_gateware_bit, 0)
                     gateware_bit = artifact_path("gateware", "top.bit")
-                    programmer.load(gateware_bit, 1)
+                    programmer.load(gateware_bit, 0)
                 else:
                     gateware_bit = artifact_path("gateware", "top.bit")
                     programmer.load(gateware_bit, 0)

If your JTAG breaks completely with the RTM disconnected, you need a newer MMC firmware with the bug fixed.

[marmeladapk]

@sbourdeauducq Thanks I missed pld # in load_proxy. Yesterday I tried all gatewares generated by your script, but it didn't help, still get preamble errors. Also seems that we have recreated 1,8 V bug overnight.

[sbourdeauducq]

Yesterday I tried all gatewares generated by your script, but it didn't help, still get preamble errors.

Did you edit the script so it changes the RX, not TX phase? As it is, it does nothing to the RX.

Also seems that we have recreated 1,8 V bug overnight.

Good. So at least we can make some progress with that bug despite inane EU customs bureaucrats (which are one of the reasons I moved to Hong Kong).

[marmeladapk]

@sbourdeauducq EU customs are nothing compared to our post service ;) That's where I expect the main delay will come from.

Did you edit the script so it changes the RX, not TX phase?

No, I missed your comment how to do it. I'll try that today.

[sbourdeauducq]

EU customs are nothing compared to our post service ;) That's where I expect the main delay will come from.

Didn't know EMS went through the regular post when sending to Poland. I'll avoid it next time. Is DHL working correctly?

Did you edit the script so it changes the RX, not TX phase?

No, I missed your comment how to do it. I'll try that today.

OK, then it is expected that you saw no change in the preamble errors.

[hartytp]

inane EU customs bureaucrats (which are one of the reasons I moved to Hong Kong).

You emigrated in search of laxer customs regulations?

[sbourdeauducq]

That's one of many reasons, yes; wasting 2+ hours per package (which is also typically delayed for days) containing something as basic as a few KF valves or a FPGA development kit isn't fun. In Hong Kong, this type of package is just waved through. Anyway, back to the point.

[marmeladapk]

I had some time to play with Kasli and Artiq. (notes for me to trace what I have done)

artiq_mkfs flash_storage.img -s mac 02:02:02:02:02:01 -s ip 192.168.95.174
artiq_flash -f flash_storage.img -t kasli storage start
artiq/examples/kasli_opticlock$ artiq_compile idle_kernel.py

Changed device address in artiq/examples/kasli_opticlock/device_db.py

artiq_coreconfig write -f idle_kernel idle_kernel.elf

artiq_flash -t kasli start to restart board.

Played a bit with artiq_run. Will continue tommorow, after I test new batch of Kaslis. Eventually I want to be able to test most functionalities with Artiq and use Vivado only for IBERT and platform cable. I'll start with writing and reading from DIOs.

[jordens]

Well done!

[hartytp]

@marmeladapk Nice! You should be able to use this to test Zotino v1.1 when it arrives. Might be quicker than the dev kit.

[marmeladapk]

https://github.com/marmeladapk/kasli-i2c

With multiple devices connected (in our case Sayma and Kasli):

lsusb -t to see to which bus and port is FTDI connected.

artiq_flash -I "ftdi_location 1:5" -t kasli start to restart Kasli. (ftdi_location bus:port)

With some simple code and kasli-i2c I tested RJ45 loopback, however I had some problems with I2C (registers in RJ45 DIO seem to set only after few tries). I didn't have those problems on my windows machine, I'll check it on Monday.

for i in range(0, 3):
     self.ttl8.off()
     delay(200 * us)
     self.ttl0.sample_input()

     self.ttl8.on()
     delay(200 * us)
     self.ttl0.sample_input()

r = []
for i in range(0, 6):
     r = r + [self.ttl0.sample_get()]
print(r)

[marmeladapk]

@jordens Did you test AD9910 on Kasli?

Currently with "refclk": 240e6 and "pll_n": 15 on all channels I get PLL lock timeout

[sbourdeauducq]

@marmeladapk I did and it worked fine. https://github.com/m-labs/artiq/tree/master/artiq/examples/kasli_sysu

[marmeladapk]

@sbourdeauducq python3 -m artiq.gateware.targets.kasli -V sysu

make: Entering '/home/pawel/artiq-dev/artiq/artiq_kasli/sysu/software/bootloader'
 CARGO    cargo/or1k-unknown-none/debug/libbootloader.a
env: "cargo": No such file or directory

edit: My bad, on clean repo it compiles.

[marmeladapk]

I added I2C device to coredevice/i2c.py (practically a copy-pasted version of PCA9548 driver):

class PCF8574:
    """Driver for the PCF8574 I2C bus extender.

    I2C transactions not real-time, and are performed by the CPU without
    involving RTIO.

    On the RJ45, this chip is used for selecting direction.
    """
    def __init__(self, dmgr, busno=0, address=0x7c, core_device="core"):
        self.core = dmgr.get(core_device)
        self.busno = busno
        self.address = address

    @kernel
    def set(self, directions):
        """Set direction.

        High - output, low - input

        :param directions: 2 bits of direction for all channels (0x00 - 0xFF)
        """
        i2c_start(self.busno)
        try:
            if not i2c_write(self.busno, self.address):
                raise I2CError("PCF8574 failed to ack address")
            if not i2c_write(self.busno, directions):
                raise I2CError("PCF8574 failed to ack control word")
        finally:
            i2c_stop(self.busno)

    @kernel
    def readback(self):
        i2c_start(self.busno)
        r = 0
        try:
            if not i2c_write(self.busno, self.address | 1):
                raise I2CError("PCF8574 failed to ack address")
            r = i2c_read(self.busno, False)
        finally:
            i2c_stop(self.busno)
        return r

Now with

self.i2c_switch0.set(7)
self.i2c_rj45_dir.set(0x00)
self.i2c_switch0.set(5)
self.i2c_rj45_dir.set(0x01)

I'm able to set direction on RJ45 DIO so I no longer need my hacky bitbang-I2C.

[jordens]

@marmeladapk Yes. Urukul-AD9910/v1.0 on Kasli/v1.0 has been tested at 100, 125, and 150 MHz refclk. Check Urukul clock mux setting, refclk power, vco range, i_cp. And maybe simulate the AD9910 PLL loop filter with the parameters you are using.

[jordens]

And check the DIP switch settings.

[jordens]

@marmeladapk The PCF8574 driver looks good. It would be great if you could add that to the API documentation and submit a pull request.

[hartytp]

Closing this as WUT have ARTIQ up and running (well done and thanks again @marmeladapk).

PCF8574 driver should have a separate issue if needed....