Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuring CPU usage during SEG-Y import/export #392

Merged
merged 4 commits into from
May 21, 2024

Conversation

tasansal
Copy link
Collaborator

@tasansal tasansal commented May 21, 2024

Overview

This PR includes modifications to the blocked_io.py and mdio.py scripts to enhance CPU utilization reconfigurability through environment variables.

The MDIO__IMPORT__CPU_COUNT and MDIO__EXPORT__CPU_COUNT allow users to specify how many cores to use.

If the variables don't exist, it will use all the available cores of the machine.

Changes in blocked_io.py

  • Replaced the NUM_CORES definition with a more flexible approach:
default_cpus = cpu_count(logical=True)
NUM_CPUS = int(os.getenv("MDIO__IMPORT__CPU_COUNT", default_cpus))

Changes in mdio.py

  • Introduced environment variable for CPU count configuration:
default_cpus = cpu_count(logical=True)
NUM_CPUS = int(os.getenv("MDIO__EXPORT__CPU_COUNT", default_cpus))

Summary

These changes aim to provide better control over CPU usage through environment variables.

Changes have been made to improve the control of parallel computations in the application. The number of CPUs used by the mdio_to_segy and to_zarr methods can now be controlled by altering environmental variables MDIO__EXPORT__CPU_COUNT and MDIO__IMPORT__CPU_COUNT respectively. This allows users to optimize the program's performance based on their specific hardware setup.
@tasansal tasansal added the enhancement New feature or request label May 21, 2024
@tasansal tasansal self-assigned this May 21, 2024
@tasansal tasansal merged commit 0d9f993 into main May 21, 2024
20 checks passed
@tasansal tasansal deleted the feature/configure_cpu_counts branch May 21, 2024 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant