Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multi-frequency support to pipeline #467

Closed
taramurphy opened this issue Feb 2, 2021 · 3 comments
Closed

Add multi-frequency support to pipeline #467

taramurphy opened this issue Feb 2, 2021 · 3 comments
Labels
enhancement New feature or request

Comments

@taramurphy
Copy link

At the moment the pipeline assumes all images are at the same frequency, which is true for the VAST Pilot Survey. However, it would be good to incorporate images at different frequencies (for example the latest RACS epoch is at a higher frequency. This requires some thought about how (and whether) to combine information from different frequencies.

Text from Adam's wishlist:

Multi-frequency support. This is something that TraP has that we don’t. Essentially the measurements and sources have a frequency band attached to them, but it only makes sense to calculate the metrics on a per band basis. Per band here usually means the central frequency +/- the bandwidth / 2. This is tricky but would be a nice science feature. This would mean multiple sets of metrics to be attached to one source so would likely mean database structure changes.

@taramurphy taramurphy added the enhancement New feature or request label Feb 2, 2021
@github-actions github-actions bot added this to To do in Pipeline Backlog Feb 2, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2021

Congrats on your first issue and Welcome to Askap pipeline development: please assign the issue to a milestone if not auto-assigned, and add relevant labels if possible

@ajstewart
Copy link
Contributor

ajstewart commented Feb 3, 2021

I thought about this a bit and I think you somewhat want to follow what the TraP has already done. It is an oversight on our part to not bake this in but I have a feeling it won't be too painful to achieve, just a bit fiddly to make sure everything flows right.

I think this would involve:

  • I think I'm right in that we don't need to worry about different association behaviour or separate average positions, so the whole association stage doesn't need to be changed.
  • Split off the source metrics to their own table - i.e. remove all the band dependant metrics from the source table to a new metrics table.
  • The relationship from sources to this new table is one to many - a source can have many metric entries but crucially the metric table entires for the same source have to be unique bands.
  • In the source metrics calculation in the code (finalise.py) you need to group by band - this may actually be the most tricky bit as you'll need to decide what is part of the same band, I forget how the TraP decides this but we could define our own band rules.
  • Two epoch metrics will also have to be grouped per band.
  • Change the UI to represent the multiple band metrics and light curves.
  • Change the parquet outputs to reflect the metric changes.

For all this I'm assuming that we're not trying to do anything special with the spectral index, but this would also open up the possibility of doing something that that as well.

@dlakaplan
Copy link

Tagging @marxide and @dlakaplan to get us in the discussion

@marxide marxide closed this as completed Apr 28, 2021
Pipeline Backlog automation moved this from To do to Done Apr 28, 2021
@askap-vast askap-vast locked and limited conversation to collaborators Apr 28, 2021

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

4 participants