-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create version/option in quickquasars to generate Lyman alpha mocks for eBOSS #437
Comments
I vote for having a --eboss (all lowercase) option, rather than duplicating code. There's not reason for it to become messier just because of the extra option. What would require more thought is how to get the eboss+boss sky density, but I don't think we absolutely need this from the get-go as long as we have the right mean density. For the geometric part of the footprint we will need external data (e.g. the list of boss+eboss tiles), (would we add that to a desi repository?) @kdawson1000 might know where to get those input files. |
Nicolas, |
I would vote for
The reason is it would be problematic to force new version of desisim (which is the main DESI simulation repo) each time we make an improvement in the eboss sim. |
We worked quite a bit yesterday with @belaa it seems that the changes are quite minimal:
|
I agree it would be better to have a minimal change in desisim. However, I agree with Julien that we should also have a Lyman alpha repo where we can play with freely (for instance, for mini-test that Julian is working on). |
I started a branch to do that, but if someone else is already working on a different branch I can move the couple of lines there. |
After taking a closer look at the desisim code, I've encountered a few more blocking points to the point that I think we should actually decouple from desisim altogether and interface directly with specsim. @belaa could you point us to an example on how given a noiseless, high resolution eboss spectrum we can obtain a noisy version of it using specsim? @dkirkby mentioned that this could be even done as a function of plate/mjd, is that the case? |
@ngbusca Here is a notebook on my eboss_config branch that goes through how to simulate eBOSS spectra. As far as I know, I think you have to provide the fluxes for each source you're using either in the config file or as an argument to |
@dkirkby @belaa to zeroth order we would like to get the right s/n, sky, etc. on coadds (spPlates) on average (I think we already have that, right?). But a more realistic way to do it could be: I'm still not sure how a. would interplay with simulating quasar reobservations (ideas?). Would c. add a lot of overhead? would c. add correlated sky residuals? The zeroth order might be enough by the way. |
Do we really need this, Nico? It sounds quite complicated, specially when taking into account reobservations (and how these targest were selected). For now, I would work under the assumption that all spectra in DR14Q are random draws from the same SNR distribution, and that the only difference between eBOSS and BOSS_minus_eBOSS footprint is the density of high-z quasars. But it would be great to see these distributions of SNR in BOSS, in eBOSS, for the reobserved quasars... |
@andreufont I totally agree, that's my order zeroth above and we should definitely try it out and check the SNR, etc. before complicating things. I was just asking looking ahead, certainly not requesting anything new from specsim! |
Here are some notes as I try to understand the layers of quickquasars. Corrections welcome. The nested calling sequence is:
At the lowest level, in
However, the following top-level call, from
What output files do you need for the eboss mode? Are you only simulating quasars, or also contaminants? |
I tried pushing as far as I could, and eventually I got an error claiming that the requested X,Y position of the fiber was outside the FOV of the telescope. I think this is related to positioning the fibers in the focal plane according to Mayall but trying to simulate them with SDSS, which is why I suggested to give up using desisim altogether... |
Many of the steps being performed now are not relevant for quasar sims (e.g. setting moon conditions and galaxy size & shape). That's not necessarily a problem, unless it brings in a lot of hardcoded desi assumptions (like the field of view). |
I think it's fair to say that we (Julien?) Copy pasted code to get things running, but if you can identify code that is not relevant feel free to remove it :-) |
@kdawson1000 - Any progress on this? A part from the issues with the noise simulator, we also need to address the footprint discussion. Could you provide a function / file that defines the footprint for BOSS and for eBOSS? |
I thought Nicolas had a solution to this by simply using the DR16Q catalog. |
Yes, I forgot about this. @ngbusca , let me know if you would like to discuss this (on skype?) or whether you already have a good plan for how to proceed with the eBOSS mocks. |
@andreufont, Could I be of any help to have this moving? |
An update on my end: I got quickquasars to run on the eboss specsim config, although I'm not sure how to interpret the output. I get three FITS files: spectra-16-333.fits truth-16-333.fits zbest-16-333.fits I had to tweak some things in the specsim config file to get it to run which I'm trying to understand, but I was able to run it without any errors. |
@belaa, I be happy to have a look at these files. Could you tell me where you ran quickquasar and where are the output files? Could you give me reading permissions? |
@londumas I've been running it on my computer but have moved the output files here: /global/cscratch1/sd/belaa/desi/quickquasar_output I think you should have reading permissions but let me know if you can't access it for some reason. |
@belaa, I don't have permissions. could you run |
@londumas Ok I ran it and it should work now |
@belaa, Could you make sure you ran on the first level of the directories? i.e on |
@londumas Hi Helion, I'm sorry, does it work now? |
@belaa, Yes it does. I will have a look at it. |
@belaa, everything looks good for the moment. Could you run on way more (~200,000) so I can look at the footprint, |
Hi @londumas - there are two things that need to be done:
|
@andreufont, Perfect, thank you very much. Let me try this first today when I find the time. |
@andreufont, I managed to run quickquasar.
|
@julienguy and @andreufont, sory for the noise. I simply had to update SIMQSO to master. And it works now. |
Tag |
@andreufont, I have rerun quickquasar on the branch Here are things we could work on:
|
@londumas Sorry for the lag - my computer was out of commission for a couple days. I have looked into the code and think I may know why it is showing a limited spectral range, but need to investigate this further. As for running this on more realizations, I'm afraid I am still unfamiliar with the details of how quickquasars works - is each quasar specified by a transmission file? And if so, can you point me to where these are located? |
Hi @londumas - I'm glad it worked, and very curious to see how the correlation/BAO looks like! A couple of comments on your comments: H) downsampling in redshift according to the DR14Q n(z) H) Work better on the footprint, the high density of SEQUELS doesn't appear in the mocks H) The maximum lambda_obs is lower in mocks than in data. In eBOSS the pixels are binned in log lambda, in these mocks they are binned in lambda. |
About the footprint in DR14Q and in the mocks, I add here the footprint I get when using the cut in zmin=2.15. I also attach the python script in case you want to play with it.
|
@belaa, I did not realize the version I was testing only @andreufont version and not yours. |
@belaa, your mocks have a pretty good range of wavelength.
It would be nice to have the mocks o down to at least 3600. |
@londumas I don't think it should be difficult to fix. I will take a look later today. |
@londumas I just pushed my
Please let me know if you have any problems running this so I can make sure to fix them. |
@belaa, could you merge your branch with master? It is very far behind. |
@belaa, I merged the local version of your branch with master and got the following error.
|
@londumas Are you also pointing to the eboss_config branch in specsim? |
@belaa, yes. |
@belaa, I have tried to also merge |
@andreufont, I have updated the footprint file to remove DR7 and start at |
@londumas Yes I will look into this and get back to you. |
@andreufont, you can forget my remarks. I found the solutions. |
@andreufont, the latest version of
Is there other things you want to have a look to before we do a PR?
|
@londumas Can you please look at the output in |
@belaa, Thank you very much for the work. You have fixed the small wavelength which is very important, but not the large wavelength. If it is too much work to fix them, it is not important.
|
@belaa, do you think you would have time to work on this? |
@londumas yes, I can do this tomorrow. |
@belaa, perfect. Thank you. |
@belaa, I did a |
Now that the |
We would like to use quickquasars to generate Lyman alpha mocks for eBOSS. That would be useful for eBOSS, but it would also increase the number of eyes and hands on quickquasars and the London mocks.
@belaa already has a version of specsim that works on eBOSS specs, we just need to work on the rest of quickquasars (the easy part). Today I only started to think about it, but I plan to work on this tomorrow (wifi permitting).
The first thing we need to decided is: do we want --eBOSS to be an option of quickquasars, or do we want to have a separate script quickquasars-eBOSS.py ?
The advantage of the first would be that we don't have duplicated code, but it could potentially make our beloved DESI script a bit messier...
I believe the mess would be quite localized though:
Ok, it might be more work than I thought. We probably want to have a different script.
@julienguy @moustakas @sbailey @dkirkby @ngbusca
The text was updated successfully, but these errors were encountered: