Skip to content

Xiaoya refactor block#43

Merged
dylanmcreynolds merged 17 commits into
als-computing:mainfrom
xiaoyachong:xiaoya-refactor-block
Apr 20, 2026
Merged

Xiaoya refactor block#43
dylanmcreynolds merged 17 commits into
als-computing:mainfrom
xiaoyachong:xiaoya-refactor-block

Conversation

@xiaoyachong
Copy link
Copy Markdown
Contributor

@xiaoyachong xiaoyachong commented Apr 16, 2026

The main changes of this PR

  1. Use the arroyopy block YAML format to replace the processor_cli_tpx.py.

  2. Define XPSResultStart, sent from XPSOperator to XPSWSResultPublisher, with the workflow:

ZMQ wire (splash_timepix / LabVIEW)
↓
(XPS operator) XPSTimepixZMQListener / XPSLabviewZMQListener
↓
(XPS operator) XPSStart / XPSRawEvent / XPSStop
↓
(XPS operator) XPSOperator
↓
(XPS operator) XPSResultStart / XPSResult / XPSResultStop
↓
(XPS operator) XPSWSResultPublisher
↓
(arroyo operator) XPSWebSocketListener
↓
LSE pipeline
  1. Calculate running_mean for flush n as follows, and I check with Johannes that the calculation is correct in this splash_timpix issue:
* Flush 1: Array Data = sum of cycles 1–1000
  cycles_in_flush = 1000, total_cycles = 1000
  cumulative_sum = flush1
  running_mean 1 = cumulative_sum / 1000

* Flush 2: Array Data = sum of cycles 1001–2000
  cycles_in_flush = 1000, total_cycles = 2000
  cumulative_sum = flush1 + flush2
  running_mean 2 = cumulative_sum / 2000

* Flush 3: Array Data = sum of cycles 2001–3000
  cycles_in_flush = 1000, total_cycles = 3000
  cumulative_sum = flush1 + flush2 + flush3
  running_mean 3 = cumulative_sum / 3000
  1. Test with splah_timepix simulator and LSE, and everything works well.
image

Issues with this PR

The splash_timepix simulator currently generates one flush per scan, which does not cause issues for LSE, since the model inference time is around 0.5 s per image on my MacBook. However, Johannes mentioned that in real experiments it is possible to generate hundreds or thousands of flushes. In that case, LSE may run into performance issues. We may need to add some accumulation logic in ArroyoXPS to limit the data frequency sent to LSE.
image

Companion PRs:
LSE: mlexchange/mlex_latent_explorer#76
arroyopy: als-computing/arroyopy#18

Comment thread src/tr_ap_xps/timepix.py Outdated
Comment thread block_configs/timepix_processor_block.yaml Outdated
Comment thread src/tr_ap_xps/pipeline/xps_operator.py Outdated
Comment thread src/tr_ap_xps/labview.py
Comment thread src/tr_ap_xps/schemas.py Outdated
Comment thread src/tr_ap_xps/schemas.py Outdated
Comment thread src/tr_ap_xps/websockets.py Outdated
Comment thread Dockerfile_labview_processor
Comment thread docker-compose.yml Outdated
@xiaoyachong xiaoyachong requested a review from taxe10 April 16, 2026 22:39
Copy link
Copy Markdown

@taxe10 taxe10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of very minor comments. Looks good.

Comment thread pyproject.toml Outdated
Comment thread src/tr_ap_xps/timepix.py Outdated
@dylanmcreynolds
Copy link
Copy Markdown
Contributor

I'm ok to merge this. Great work!

@dylanmcreynolds dylanmcreynolds merged commit 34f2e26 into als-computing:main Apr 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants