Template API

A sirepo.template wraps a code. The wrapper normalizes the interface to the code with standard APIs.

Overview

A sirepo.template is a module that is accessible via sirepo.template.import_module, which returns a module object, but should be treated abstractly (to future proof purposes). Templates provide APIs for a code to a specific set of modules. Most APIs run inside the Job Agent process (see Job System Architecture Overview), but some APIs run directly in the server. (We are trying to eliminate that.)

The template APIs operate in a few general categories:

• simulation - generating scripts/inputs to run the code in a run_dir. A run_dir is bound to a computeJob, which persists state across calls.
• compute - running the code outside the context of a computeJob, but sometimes with a tmp_dir and parsing the result.
• export - exporting files for a simulation. Either individual files or a zip of everything needed for the simulation.
• import - upload and parse file native format and Sirepo archive files to create new simulations
• output - parsing the output from a computeJob for reports and graphs

Sirepo uses data arbitrarily but most times it is referring to a simulation data dict. For this document, sim_data means this specific dictionary.

sim_lib refers to a collection of files shared across all simulations for a specific user and template. Unrelated to sirepo.lib.

sirepo.lib API

sirepo.lib is standalone interface to templates to allow import and export of native file formats to canonical Sirepo (sirepo-data.json) format. There are two primary interfaces sirepo.lib.Importer.parse_file and sirepo.lib.Exporter.write_files. These interface with LibAdapter classes implemented in the templates. These class-based APIs are not used by the server. Import accepts a native file(s) and returns a sim_data. Export accepts a sim_data and generates the native files to run a simulation outside the context of Sirepo server.

Server APIs

• analysis_job_<method> - accepts application specific args, and run_dir. Returns an arbitrary JSON result or file related to output in run_dir.
• background_percent_complete - runs in the run_dir and returns an arbitrary JSON result.
• create_archive - produces a zip file containing all files associated with a specific simulation.
• fixup_old_data - accepts a sim_data object and updates it to match to the most recent version of the schema.
• get_data_file - accepts a run_dir and a frame number and return a raw file from the run_dir.
• import_file - accepts a Sirepo archive file as well as other native formats for the code. The output is a sim_data.
• new_simulation - fixup a newly created sim_data.
• post_execution_processing - executes in a run_dir usually reading logs to produce a UserAlert.
• prepare_for_client - accepts a sim_data and returns a sim_data, which some fixups before the data is returned to the server's client (usually the UI).
• prepare_for_save (srw only) - accepts a sim_data and returns a sim_data, which has been fixed up before it is to be written to disk. SRW updates the user model lists.
• prepare_sequential_output_file - post-processes the output from a sequential computeJob
• prepare_simulation - prepare the run_dir file that will be run as a computeJob.
• python_source_for_model - generate Python or native format file for a computeJob that can be run on the command line outside the server.
• remove_last_frame - clean up after a computeJob is canceled. This may remove the last output file in the run_dir.
• sim_frame_<report> or sim_frame - accepts frame_args which describes a specific output in a sequence from a computeJob. Returns JSON that describes a graph.
• stateful_compute_<method> - accepts application specific args and a specific simulation. Returns an arbitrary JSON result or file related to the simulation.
• stateless_compute_<method> - accepts application specific args. Returns an arbitrary JSON result or file.
• validate_file - accepts a sim_lib file_type and a path to an uploaded file. Returns an error string or None.

Server API Calls

As of this writing, all calls to templates are originated from these modules:

• job_cmd calls APIs:
  • analysis_job_* via template_common.analysis_job_dispatch via _do_analysis_job
  • background_percent_complete via _do_compute and _do_sbatch_status
  • get_data_file via _do_download_data_file
  • post_execution_processing via _on_do_compute_exit
  • prepare_sequential_output_file via _do_sequential_result
  • prepare_simulation via simulation_db.prepare_simulation via _do_compute
  • remove_last_frame via _do_cancel
  • sim_frame_* or sim_frame via template_common.sim_frame_dispatch via _do_get_simulation_frame
  • stateful_compute_* via template_common.stateful_compute_dispatch via _do_stateful_compute
  • stateless_compute_* via template_common.stateless_compute_dispatch via _do_stateless_compute
• server calls APIs:
  • create_archive via via exporter.create_archive via api_exportArchive
  • fixup_old_data via sim_data via simulation_db.save_simulation_json, simulation_db.validate_serial, and simulation_db.read_simulation_json via many API calls in server and via importer.read_json
  • import_file via api_importFile
  • new_simulation via api_newSimulation 6a7c9027cb23c26ced5c3/sirepo/server.py#L651)
  • prepare_for_client via api_copyNonSessionSimulation, api_copySimulation, api_importFile, api_newSimulation, api_saveSimulationData, and api_simulationData
  • prepare_for_save (srw only) via same calls as fixup_old_data
  • python_source_for_model via api_pythonSource and via exporter.create_archive via api_exportArchive
  • validate_file via api_uploadFile
• SRW unique case:
  • sirepo.template.srw.import_file creates a computeJob to import python. This was to isolate code before the introduction of statefullCompute.