Skip to content

proc: process info files should be locked during writing #53

@pmrowla

Description

@pmrowla

In certain situations it's possible that a reader tries to load proc info from file while it's being written to which causes a jsondecodeerror. ProcessManager should provide minimal amount of synchronization for these cases.

Alternatively, at a minimum we should just write to temporary paths (i.e. output_path.json.<random suffix>) and then fs.move to replace the existing file in:
https://github.com/iterative/dvc-task/blob/d132caa1ba677d1527c0ac3c67c849c0b506a639/src/dvc_task/proc/process.py#L133-L138

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions