Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBBC3 Tsys support for non-continuous cal #194

Closed
9 of 10 tasks
wehimwich opened this issue Apr 7, 2023 · 11 comments
Closed
9 of 10 tasks

DBBC3 Tsys support for non-continuous cal #194

wehimwich opened this issue Apr 7, 2023 · 11 comments

Comments

@wehimwich
Copy link
Member

wehimwich commented Apr 7, 2023

@BeppeMc reported that DBBC3 Tsys with non-continuous cal (still!) works, but does not support formbbcand formif in tpi=.../tpical=.../tsys=.../etc.

Continuous cal is the preferred way to go if it is available. If someone can't use continuous cal, we can look into cleaning up the remaining loose ends. I think the exhaustive task list is:

  • Implement formbbc and formif mnemonics for DBBC3 in Tsys related commands
  • Display non-continuous cal Tsys in the DBBC3 Tsys monitor display window (Tsys updates will be few and far between)
  • Verify use of tpicd to log TPI values during scans (I think this probably already works)
    It did not because it was based on early v124 TPI on/off order, but fixed now.
  • Verify that fivpt/onoff still work for non-continuous cal (probably also okay)
  • Provide sample .prc (it should basically be the same as for DBBC2)
  • Possibly switch to using multicast for the count data (for speed)
    This should not be needed. The communication for the bbcNNN and iftpX commands seems plenty fast enough. It was tested using 64 BBCs and 8 IFs with DDC_E v126. Extrapolating from that, even 128 BBCs should be okay. This has improved since v121.
  • Increase size of command input buffers for DBBC3 non-continuous Tsys commands
  • Properly initialize decoding buffer for the DBBC3 non-continuous Tsys commands
  • Increase size of command input buffers for all other racks non-continuous Tsys commands
  • Properly initialize decoding buffer for all other racks non-continuous Tsys commands

For @BeppeMc's test, getting the count data via DBBC3 commands seemed fast enough for 8 BBCs and two IFs, but it is not clear yet if it will be for 64 BBCs (or more) and eight IFs:

2023.095.10:37:38.59;caltsys_man
2023.095.10:37:38.59&caltsys_man/ ifman
2023.095.10:37:38.59&caltsys_man/ bbc_gain=all,man
2023.095.10:37:38.59&caltsys_man/ !+2s
2023.095.10:37:38.59&caltsys_man/ tpi=1u,2u,3u,4u,9u,10u,11u,12u,ifa,ifb
2023.095.10:37:38.59&caltsys_man/ cal=on
2023.095.10:37:38.59&caltsys_man/ !+4s
2023.095.10:37:38.59&caltsys_man/ tpical=1u,2u,3u,4u,9u,10u,11u,12u,ifa,ifb
2023.095.10:37:38.59&caltsys_man/ cal=off
2023.095.10:37:38.59&caltsys_man/ !+1s
2023.095.10:37:38.59&caltsys_man/ tpdiff=1u,2u,3u,4u,9u,10u,11u,12u,ifa,ifb
2023.095.10:37:38.59&caltsys_man/ caltemp=1u,2u,3u,4u,9u,10u,11u,12u,ifa,ifb
2023.095.10:37:38.59&caltsys_man/ tsys=1u,2u,3u,4u,9u,10u,11u,12u,ifa,ifb
2023.095.10:37:42.07/tpi/001u,16185,002u,16227,003u,16024,004u,15463,ia,4354
2023.095.10:37:42.07/tpi/009u,16191,010u,15330,011u,15264,012u,16104,ib,5612
2023.095.10:37:42.13/cal/on
2023.095.10:37:43.31?ERROR dn  -30 DBBC3 multicast version: DDC_E,126,October 25th 2022
2023.095.10:37:43.31?ERROR dn  -35 DBBC3 multicast: DBBC3 has an unknown personality, see DN -30 error above
2023.095.10:37:46.14/tpical/001u,21102,002u,20580,003u,19834,004u,20395,ia,5682
2023.095.10:37:46.14/tpical/009u,19397,010u,18464,011u,18050,012u,18405,ib,6875
2023.095.10:37:46.21/cal/off
2023.095.10:37:47.22/tpdiff/001u,4917,002u,4353,003u,3810,004u,4932,ia,1328
2023.095.10:37:47.22/tpdiff/009u,3206,010u,3134,011u,2786,012u,2301,ib,1263
2023.095.10:37:47.22/caltemp/001u,10.977,002u,9.862,003u,8.948,004u,10.653,ia,19.030
2023.095.10:37:47.22/caltemp/009u,11.750,010u,15.064,011u,18.379,012u,13.387,ib,18.748
2023.095.10:37:47.22/tsys/001u,36.1,002u,36.8,003u,37.6,004u,33.4,ia,62.4
2023.095.10:37:47.22/tsys/009u,59.3,010u,73.7,011u,100.7,012u,93.7,ib,83.3
2023.095.10:38:03.31?ERROR dn  -30 DBBC3 multicast version: DDC_E,126,October 25th 2022
2023.095.10:38:03.31?ERROR dn  -35 DBBC3 multicast: DBBC3 has an unknown personality, see DN -30 error above
2023.095.10:38:23.31?ERROR dn  -30 DBBC3 multicast version: DDC_E,126,October 25th 2022
2023.095.10:38:23.31?ERROR dn  -35 DBBC3 multicast: DBBC3 has an unknown personality, see DN -30 error above
2023.095.10:38:43.31?ERROR dn  -30 DBBC3 multicast version: DDC_E,126,October 25th 2022
2023.095.10:38:43.31?ERROR dn  -35 DBBC3 multicast: DBBC3 has an unknown personality, see DN -30 error above
2023.095.10:38:47.15;preob
2023.095.10:38:47.21/cal/off
2023.095.10:38:54.70/tpi/001u,16209,002u,16230,003u,16039,004u,15457,ia,4354
2023.095.10:38:54.70/tpi/009u,15974,010u,15455,011u,15279,012u,15286,ib,5698
2023.095.10:38:54.76/cal/on
2023.095.10:38:58.77/tpical/001u,21136,002u,20598,003u,19882,004u,20390,ia,5683
2023.095.10:38:58.77/tpical/009u,19132,010u,18468,011u,18089,012u,18580,ib,6949
2023.095.10:38:58.84/cal/off
2023.095.10:38:59.85/tpdiff/001u,4927,002u,4368,003u,3843,004u,4933,ia,1329
2023.095.10:38:59.85/tpdiff/009u,3158,010u,3013,011u,2810,012u,3294,ib,1251
2023.095.10:38:59.85/caltemp/001u,10.977,002u,9.862,003u,8.948,004u,10.653,ia,19.030
2023.095.10:38:59.85/caltemp/009u,11.750,010u,15.064,011u,18.379,012u,13.387,ib,18.748
2023.095.10:38:59.85/tsys/001u,36.1,002u,36.6,003u,37.3,004u,33.4,ia,62.3
2023.095.10:38:59.85/tsys/009u,59.4,010u,77.3,011u,99.9,012u,62.1,ib,85.4
2023.095.10:39:04.31?ERROR dn  -30 DBBC3 multicast version: DDC_E,126,October 25th 2022
2023.095.10:39:04.31?ERROR dn  -35 DBBC3 multicast: DBBC3 has an unknown personality, see DN -30 error above
@vlbi-jun-yang
Copy link

At Onsala, we have to use non-continuous calibration at 22, 43 and 86 GHz. Because of the Chopper Wheel calibration, we have a local program to calcuate TCal and opacity. However, we still need these FS supports to get proper Tsys data in the log files.

@vlbi-jun-yang
Copy link

I have tried 32 BBCs to measure Tsys during the running 3-mm GMVA session. However, FS 10.2.0-alpha1 only outputed values for 12 BBCs. There seems to be an internal limit of the maximum BBC nubmers.

@vlbi-jun-yang
Copy link

Somehow, these total power monitoring data from DDC_U v126 are 0 in the FS log file. This is also strange.

See:

2023.128.08:10:35.51#dbtcn#tpi/ 001l,    0, 001u,    0, 002l,    0, 002u,    0, 003l,    0, 003u,    0, 004l,    0, 004u,    0
2023.128.08:10:35.51#dbtcn#tpi/ 005l,    0, 005u,    0, 006l,    0, 006u,    0, 007l,    0, 007u,    0, 008l,    0, 008u,    0
2023.128.08:10:35.51#dbtcn#tpi/ 065l,    0, 065u,    0, 066l,    0, 066u,    0, 067l,    0, 067u,    0, 068l,    0, 068u,    0
2023.128.08:10:35.51#dbtcn#tpi/ 069l,    0, 069u,    0, 070l,    0, 070u,    0, 071l,    0, 071u,    0, 072l,    0, 072u,    0
2023.128.08:10:35.51#dbtcn#tpi/ ia,        0
2023.128.08:10:35.51#dbtcn#tpi/ 009l,    0, 009u,    0, 010l,    0, 010u,    0, 011l,    0, 011u,    0, 012l,    0, 012u,    0
2023.128.08:10:35.51#dbtcn#tpi/ 013l,    0, 013u,    0, 014l,    0, 014u,    0, 015l,    0, 015u,    0, 016l,    0, 016u,    0
2023.128.08:10:35.51#dbtcn#tpi/ 073l,    0, 073u,    0, 074l,    0, 074u,    0, 075l,    0, 075u,    0, 076l,    0, 076u,    0
2023.128.08:10:35.51#dbtcn#tpi/ 077l,    0, 077u,    0, 078l,    0, 078u,    0, 079l,    0, 079u,    0, 080l,    0, 080u,    0

@vlbi-jun-yang
Copy link

I should also mention that these FRB observations using VLBI backends at L band require to non-continuous calibration. In the project Pinpointing REpeating ChIme Sources with EVN dishes, PRECISE, there are about 1000 hours ad-hoc VLBI obsevations per year.

@wehimwich
Copy link
Member Author

Thank you @vlbi-jun-yang.

I have tried 32 BBCs to measure Tsys during the running 3-mm GMVA session. However, FS 10.2.0-alpha1 only outputed values for 12 BBCs. There seems to be an internal limit of the maximum BBC nubmers.

There shouldn't be anything limiting how many of the detectors you can use per se. I believe the problem stems from the fact that the buffer for the DBBC3 commands has not been updated for the longer devices (and the increased number of them) for the DBBC3. Thank you for catching this. Could you please point me to (or send me) a log or post a snippet that shows this problem, so I can confirm the cause? If it is, I think you may be able put the detectors in multiple commands as a workaround.

Fixing this will improve the situation a lot, but it will still not be possible to handle naming every possible DBBC3 device individually in one command. The maximum input line, 512 characters, that can be processed is too short for that. It could be expanded to 1024 now to match the maximum class buffer length, but that won't handle all individual DBBC3 detectors either. When R2DBE support is added, the maximum class buffer length will be increased to 2048, which could handle this situation. OTOH, 512 already seems like a ridiculously long length.

Usually, for large numbers of detectors, more general mnemonics are used and are sufficient for most purposes. The most useful, formbbc and formif, were missing (thanks for reporting this @BeppeMc). Those have been added and will be available soon (see below). You can also use the more compact mnemonics for some devices (1u for 001u, ia for ifa, etc.) to save some space, but that won't help much.

Somehow, these total power monitoring data from DDC_U v126 are 0 in the FS log file. This is also strange.

This has already been fixed and will be available soon (see below). It stems from the fact that much of the original DBBC3 development used an early release of DBBC_V v124 that apparently had the order of the BBC cal-on/cal-off TPI values swapped in the multicast (this also affects non-continuous cal). The fix will be available soon (see below). For the early releases of v124, it will be necessary to use the tools for swapping TPI values in 10.2-alpha3 to handle this.

Once I have increased the command buffer size, I will put it and the other fixes onto main to make them available and close this issue. We still won't have onoff and fivpt verified, but that will take more time and coordination. They should (hopefully) work as is. I will move verifying them to the "Testing 10.2-alpha3" discussion, #196.

@wehimwich
Copy link
Member Author

We need a snappier name for "non-continuous cal". The best I have heard so far is "spot cal", which seems pretty good. If you have a suggest, please let me know.

@vlbi-jun-yang
Copy link

The text part below is the output from my local test of 32 BBCs. Because DBBC2 controled our hotload, these total power readout numbers from DBBC3 DDC_U v126 were not meaningful. Moreover, BBC 81 should be removed in the list.

2023.128.07:35:56.41;caltsys_man
2023.128.07:35:56.41&caltsys_man/ifman
2023.128.07:35:56.41&caltsys_man/if=ddc,bbc_gain=all\,man
2023.128.07:35:56.41&caltsys_man/!+1s
2023.128.07:35:56.41&caltsys_man/"tpi=formbbc,formif
2023.128.07:35:56.41&caltsys_man/tpi=1u,1l,2u,2l,3u,3l,4u,4l,5u,5l,6u,6l,7u,7l,8u,8l,9u,9l,10u,10l,11u,11l,12u,12l,13u,13l,14u,14l,15u,15l,16u,16l,65u,65l,66u,66l,67u,67l,68u,68l,69u.69l,70u,70l,71u,71l,72u,72l,73u,73l,74u,74l,75u,75l,76u,76l,77u,77l,78u,78l,79u,79l,80u,80l,81u,81l
2023.128.07:35:56.41&caltsys_man/calon
2023.128.07:35:56.41&caltsys_man/!+1s
2023.128.07:35:56.41&caltsys_man/"tpical=formbbc,formif
2023.128.07:35:56.41&caltsys_man/tpical=1u,1l,2u,2l,3u,3l,4u,4l,5u,5l,6u,6l,7u,7l,8u,8l,9u,9l,10u,10l,11u,11l,12u,12l,13u,13l,14u,14l,15u,15l,16u,16l,65u,65l,66u,66l,67u,67l,68u,68l,69u.69l,70u,70l,71u,71l,72u,72l,73u,73l,74u,74l,75u,75l,76u,76l,77u,77l,78u,78l,79u,79l,80u,80l,81u,81l
2023.128.07:35:56.41&caltsys_man/caloff
2023.128.07:35:56.41&caltsys_man/"tpdiff=formbbc,formif
2023.128.07:35:56.41&caltsys_man/tpidiff=1u,1l,2u,2l,3u,3l,4u,4l,5u,5l,6u,6l,7u,7l,8u,8l,9u,9l,10u,10l,11u,11l,12u,12l,13u,13l,14u,14l,15u,15l,16u,16l,65u,65l,66u,66l,67u,67l,68u,68l,69u.69l,70u,70l,71u,71l,72u,72l,73u,73l,74u,74l,75u,75l,76u,76l,77u,77l,78u,78l,79u,79l,80u,80l,81u,81l
2023.128.07:35:56.41&caltsys_man/"caltemp=formbbc,formif
2023.128.07:35:56.41&caltsys_man/caltemp=1u,1l,2u,2l,3u,3l,4u,4l,5u,5l,6u,6l,7u,7l,8u,8l,9u,9l,10u,10l,11u,11l,12u,12l,13u,13l,14u,14l,15u,15l,16u,16l,65u,65l,66u,66l,67u,67l,68u,68l,69u.69l,70u,70l,71u,71l,72u,72l,73u,73l,74u,74l,75u,75l,76u,76l,77u,77l,78u,78l,79u,79l,80u,80l,81u,81l
2023.128.07:35:56.41&caltsys_man/"tsys=formbbc,formif
2023.128.07:35:56.41&caltsys_man/tsys=1u,1l,2u,2l,3u,3l,4u,4l,5u,5l,6u,6l,7u,7l,8u,8l,9u,9l,10u,10l,11u,11l,12u,12l,13u,13l,14u,14l,15u,15l,16u,16l,65u,65l,66u,66l,67u,67l,68u,68l,69u.69l,70u,70l,71u,71l,72u,72l,73u,73l,74u,74l,75u,75l,76u,76l,77u,77l,78u,78l,79u,79l,80u,80l,81u,81l
2023.128.07:35:59.05/tpi/001l,15782,001u,15784,002l,16238,002u,16044,003l,16119,003u,15929,004l,15958
2023.128.07:35:59.05/tpi/004u,16308,005l,15859,005u,16183,006l,15963,006u,16036,007l,16064,007u,16288
2023.128.07:35:59.05/tpi/008l,16239,008u,16203
2023.128.07:35:59.05/tpi/009l,16074,009u,15849,010l,16115,010u,16118,011l,15915,011u,16103,012u,16246
2023.128.07:35:59.05&calon/sy=popen 'noise_force 1 2>&1' -n noise
2023.128.07:35:59.05#popen#noise/sh: 1: noise_force: not found
2023.128.07:35:59.05#popen#noise:exit status 7f00
2023.128.07:36:00.07/tpical/001l,15773,001u,15780,002l,16231,002u,16046,003l,16113,003u,15930
2023.128.07:36:00.07/tpical/004l,15955,004u,16313,005l,15857,005u,16171,006l,15969,006u,16036
2023.128.07:36:00.07/tpical/007l,16059,007u,16278,008l,16236,008u,16191
2023.128.07:36:00.07/tpical/009l,16075,009u,15844,010l,16121,010u,16116,011l,15927,011u,16099
2023.128.07:36:00.07/tpical/012u,16246

@vlbi-jun-yang
Copy link

I have also noticed that the SNAP command tpicd=? works for 32 BBCs. See its output:

2023.129.13:08:12.94;tpicd=?
2023.129.13:08:12.94/tpicd/no,500
2023.129.13:08:12.94/tpicd/a,001l,001u,002l,002u,003l,003u,004l,004u,005l,005u,006l,006u,007l,007u,008l,008u,065l,065u,066l,066u
2023.129.13:08:12.94/tpicd/a,067l,067u,068l,068u,069l,069u,070l,070u,071l,071u,072l,072u,ia
2023.129.13:08:12.94/tpicd/b,009l,009u,010l,010u,011l,011u,012l,012u,013l,013u,014l,014u,015l,015u,016l,016u,073l,073u,074l,074u
2023.129.13:08:12.94/tpicd/b,075l,075u,076l,076u,077l,077u,078l,078u,079l,079u,080l,080u,ib

@wehimwich
Copy link
Member Author

wehimwich commented May 9, 2023

Thanks. As an example, looking at:

2023.128.07:35:56.41&caltsys_man/tpi=1u,1l,2u,2l,3u,3l,4u,4l,5u,5l,6u,6l,7u,7l,8u,8l,9u,9l,10u,10l,11u,11l,12u,12l,13u,13l,14u,14l,15u,15l,16u,16l,65u,65l,66u,66l,67u,67l,68u,68l,69u.69l,70u,70l,71u,71l,72u,72l,73u,73l,74u,74l,75u,75l,76u,76l,77u,77l,78u,78l,79u,79l,80u,80l,81u,81l

The 80th character pf the tpi=... command is the 2 in the 12l, so the input is truncated at that point. The code was also not careful about the end of a secondary buffer, so it can pickup the last character of the previous parameter (in this case u) and that makes the incorrect 12 look like a correct 12u. In this case there would be no error message to warn you. 12u had already been selected anyway.

I have fixed both of these buffering problems. One or both of them also occur for the other rack types. I'll clean those up as well. In meantime, I think splitting the long lines into multiple ones, each with a total length of less than 80 characters (and making sure that each parameter is a complete specifier for a detector), is a work around.

@wehimwich
Copy link
Member Author

I have also noticed that the SNAP command tpicd=? works for 32 BBCs. See its output:

The tpicd command works differently. It automatically selects the detectors implied by the recording setup, effectively the same as formbbc,formif. There is no limit on how many of the detectors can be used. VGOS is routinely using 64.

I hadn't bother to implement those mnemonics for "spot" cal because I had thought (hoped?) we were only going to use continuous cal for the DBBC3. It wasn't possible to implement them until 10.1 anyway because we didn't have the core3h_mode command to select the channels for recording until that version.

@wehimwich
Copy link
Member Author

At Onsala, we have to use non-continuous calibration at 22, 43 and 86 GHz. Because of the Chopper Wheel calibration, we have a local program to calcuate TCal and opacity. However, we still need these FS supports to get proper Tsys data in the log files.

I have opened a discussion topic for Chopper Wheel support, #200

wehimwich added a commit that referenced this issue May 9, 2023
* Increase command buffer
  Now big enough to accomodate each detector being
  menitoned once its long form. BOSS can't currently
  pass a buffer that big.
* Pre-blank working buffer for parameter decode
  So we don't get some trailing left overs and mistake
  a partial parameter for a real one.

Closes #194
wehimwich added a commit that referenced this issue May 9, 2023
* Increase command buffer
  Now big enough to accomodate each detector being
  menitoned once in its long form. BOSS can't currently
  pass a buffer that big.
* Pre-blank working buffer for parameter decode
  So we don't get some trailing left-overs and mistake
  a partial parameter for a real one.

Closes #194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants