Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Aegean test error #203

Open
Deekshit-Vedula opened this issue Sep 21, 2023 · 6 comments
Open

Aegean test error #203

Deekshit-Vedula opened this issue Sep 21, 2023 · 6 comments
Labels
duplicate fixed in dev Issue has been fixed in the development (dev) branch.

Comments

@Deekshit-Vedula
Copy link

Hi,

I am trying to use Aegean for the first time and facing some issues. After running the setup.py I used the test command aegean tests/test_files.1904-66_SIN.fits to see if it works. I got the following error in the terminal

aegean:INFO This is Aegean 2.3.0-(2022-11-29)
aegean:INFO Found 8 cores
aegean:INFO Using 8 cores
aegean:INFO Finding sources.
source_finder:INFO Calculating background and rms data
Traceback (most recent call last):
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 370, in filter_mc_sharemem
    ibkg = SharedMemory(name=f'ibkg_{memory_id}', create=True, size=nbytes)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
OSError: [Errno 63] File name too long: '/ibkg_9397c36b-0353-42fa-a66e-dda9dec3e915'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ananddeekshit/opt/anaconda3/bin/aegean", line 4, in <module>
    __import__('pkg_resources').run_script('AegeanTools==2.3.0', 'aegean')
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1448, in run_script
    exec(code, namespace, namespace)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/EGG-INFO/scripts/aegean", line 12, in <module>
    sys.exit(aegean.main(sys.argv[1:]))
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/CLI/aegean.py", line 373, in main
    found = sf.find_sources_in_image(filename, outfile=options.outfile,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 2244, in find_sources_in_image
    self.load_globals(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 1321, in load_globals
    self._make_bkg_rms(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 1550, in _make_bkg_rms
    bkg, rms = filter_image(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 492, in filter_image
    bkg, rms = filter_mc_sharemem(im_name,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 398, in filter_mc_sharemem
    ibkg.close()
UnboundLocalError: local variable 'ibkg' referenced before assignment
(base) ananddeekshit@vedulas-air Aegean %
@nedarajabpour
Copy link

nedarajabpour commented Sep 26, 2023 •
edited
Loading

My problem was the same when I stopped AEGEAN, because it took a long time and after that Ive got an Error[17] and I fixed it. I think the previous version was faster. I changed these lines and it worked foe Error[17]. But unfortunately it took a long time again to find sources and I stopped it again, I solved my problem by changing these lines.

In this file AegeanTools/BANE.py I changed these two lines

ibkg = SharedMemory(name='ibkg', create=True, size=nbytes)

irms = SharedMemory(name='irms', create=True, size=nbytes)

I changed it to create=False for both and ran Aegean and then changed it back to True and it worked.

For Error[63] I changed te name of ibkg_{memory_id} and irms_{memory_id} in this file .

@Deekshit-Vedula
Copy link
Author

Thank you @nedarajabpour . I tried your suggestion and changed the name of ibkg_{memory_id} and irms_{memory_id} to just ibkg_ and irms_ but I got the following error.

aegean:INFO This is Aegean 2.3.0-(2022-11-29)
aegean:INFO Found 8 cores
aegean:INFO Using 8 cores
aegean:INFO Finding sources.
source_finder:INFO Calculating background and rms data
Traceback (most recent call last):
  File "/Users/ananddeekshit/opt/anaconda3/bin/aegean", line 4, in <module>
    __import__('pkg_resources').run_script('AegeanTools==2.3.0', 'aegean')
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1448, in run_script
    exec(code, namespace, namespace)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/EGG-INFO/scripts/aegean", line 12, in <module>
    sys.exit(aegean.main(sys.argv[1:]))
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/CLI/aegean.py", line 373, in main
    found = sf.find_sources_in_image(filename, outfile=options.outfile,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 2244, in find_sources_in_image
    self.load_globals(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 1321, in load_globals
    self._make_bkg_rms(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 1550, in _make_bkg_rms
    bkg, rms = filter_image(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 492, in filter_image
    bkg, rms = filter_mc_sharemem(im_name,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 380, in filter_mc_sharemem
    pool = ctx.Pool(processes=cores, maxtasksperchild=1,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/spawn.py", line 183, in get_preparation_data
    main_mod_name = getattr(main_module.__spec__, "name", None)
AttributeError: module '__main__' has no attribute '__spec__'
(base) ananddeekshit@vedulas-air Aegean %

@nedarajabpour
Copy link

Thank you @nedarajabpour . I tried your suggestion and changed the name of ibkg_{memory_id} and irms_{memory_id} to just ibkg_ and irms_ but I got the following error.

aegean:INFO This is Aegean 2.3.0-(2022-11-29)
aegean:INFO Found 8 cores
aegean:INFO Using 8 cores
aegean:INFO Finding sources.
source_finder:INFO Calculating background and rms data
Traceback (most recent call last):
  File "/Users/ananddeekshit/opt/anaconda3/bin/aegean", line 4, in <module>
    __import__('pkg_resources').run_script('AegeanTools==2.3.0', 'aegean')
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1448, in run_script
    exec(code, namespace, namespace)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/EGG-INFO/scripts/aegean", line 12, in <module>
    sys.exit(aegean.main(sys.argv[1:]))
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/CLI/aegean.py", line 373, in main
    found = sf.find_sources_in_image(filename, outfile=options.outfile,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 2244, in find_sources_in_image
    self.load_globals(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 1321, in load_globals
    self._make_bkg_rms(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/source_finder.py", line 1550, in _make_bkg_rms
    bkg, rms = filter_image(
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 492, in filter_image
    bkg, rms = filter_mc_sharemem(im_name,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/site-packages/AegeanTools-2.3.0-py3.9.egg/AegeanTools/BANE.py", line 380, in filter_mc_sharemem
    pool = ctx.Pool(processes=cores, maxtasksperchild=1,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/Users/ananddeekshit/opt/anaconda3/lib/python3.9/multiprocessing/spawn.py", line 183, in get_preparation_data
    main_mod_name = getattr(main_module.__spec__, "name", None)
AttributeError: module '__main__' has no attribute '__spec__'
(base) ananddeekshit@vedulas-air Aegean %

Dear Deekshit,

I dont know actully what did you do.but I have changed to ibkg_1 and irms_1 in all of this file.I think this name is repeated two or three times in this file. Have you changed all of them?

@PaulHancock
Copy link
Owner

PaulHancock commented Dec 15, 2023 •
edited
Loading

Hi Folks, I'm trying to diagnose the cause of this error. Can you let me know what OS you are using and the directory from which you are running Aegean? It would also be useful to see the output of aegean --version.

@PaulHancock
Copy link
Owner

Duplicate of #200

@PaulHancock PaulHancock marked this as a duplicate of #200 Dec 15, 2023
@nedarajabpour
Copy link

nedarajabpour commented Jan 2, 2024 via email

PaulHancock added a commit that referenced this issue Jan 22, 2024
@PaulHancock
use shorter sharemem filename to avoid #200 / #203
b6d5d12
@PaulHancock PaulHancock mentioned this issue Jan 22, 2024
Merged
PaulHancock added a commit that referenced this issue Jan 23, 2024
@PaulHancock
Merge pull request #207 from PaulHancock/200-bane-sharemem-filename
4adef44 
use shorter sharemem filename to avoid #200 / #203
@PaulHancock PaulHancock added the fixed in dev Issue has been fixed in the development (dev) branch. label Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate fixed in dev Issue has been fixed in the development (dev) branch.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@PaulHancock @nedarajabpour @Deekshit-Vedula