Skip to content

Make decorator that executes an agenda in parallel#1117

Merged
riclarsson merged 8 commits intoatmtools:mainfrom
riclarsson:par-agenda
Apr 17, 2026
Merged

Make decorator that executes an agenda in parallel#1117
riclarsson merged 8 commits intoatmtools:mainfrom
riclarsson:par-agenda

Conversation

@riclarsson
Copy link
Copy Markdown
Contributor

Now you can do

@pyarts.in_parallel(ws=ws)
def create_agenda_parallel(ws: pyarts.Workspace):
  ws.abs_speciesSet(species=["H2O", "CO2"])
  ws.atm_pointInit()

The two methods are executed in parallel. Note that this will come with overhead, and the example above is clearly faster to run serial.

@riclarsson riclarsson marked this pull request as draft April 16, 2026 12:40
@riclarsson
Copy link
Copy Markdown
Contributor Author

@olemke, there is a bug in the code here. I am not sure how to solve it, so I have disabled "Any" outside of agendas. I did not know how to find the actual cause here.

The issue: casting is nanobind to a py::object that is invalid for "Any" might cause segfault. To reproduce under current ARTS3: ws.ReadXML(ws.abs_cia_data, filename="cia/O2-CIA-O2.xml"). This will sometimes cause segfaults but cause bad-cast all the time. I thought I had tested this enough. The error is entirerly in nanobind looking code by the debug info, and I cannot figure it out...

@olemke
Copy link
Copy Markdown
Member

olemke commented Apr 16, 2026

So py::type_name(*_output) is causing the segfault when outputting the runtime error message in ReadXML. Taking your ReadXML example and passing it an xml file with the correct type (CIARecords instead of CIARecord), the data is read correctly into the WSV. Segfault only occurs when the runtime error is triggered. Does pyAny cause problems in other places? If it's just an issue in the runtime error message, we could maybe just work around it (like not printing the type of _output)? Maybe also something we can discuss in the meeting tomorrow.

@riclarsson riclarsson marked this pull request as ready for review April 17, 2026 05:03
@riclarsson riclarsson merged commit b17228e into atmtools:main Apr 17, 2026
9 checks passed
@riclarsson riclarsson deleted the par-agenda branch April 17, 2026 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants