Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions 02-sgd/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
data
runs
.snakemake
53 changes: 50 additions & 3 deletions 02-sgd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,64 @@

This is a complete example demonstrating how MPoL works using simulated data.

Before starting, you should have already run the scripts in the `generate-mock-baselines` folder to produce a mock sky image and interferometer baselines in a file called `mock_data.npz`. Then, you should copy that file to this repository under `data/mock_data.npz`.
# Prerequisites

This repository assumes that you will run all scripts from this `sgd` directory (the one containing `sgd/README.md`). Some aspects of the workflow are automated with Snakemake ([`Snakefile`](Snakefile)).
Before starting, you should have already run the scripts `00` and `01` folders to produce mock baselines. Then, you will need to copy the `mock_data.npz` into this directory in a new `data` folder. For example, from within this 02 folder, run

```shell
$ mkdir data
$ cp ../01-generate-mock-baselines/data/mock_data.npz data/
```

# Installation

You can install necessary Python packages into your environment by
```shell
$ pip install -r requirements.txt
```

and then you can run the code by

```shell
snakemake -c1 all
```

# Description of Contents

This repository assumes that you will run all scripts from this `02` directory (the one containing `02-sgd/README.md`). Some aspects of the workflow are automated with Snakemake ([`Snakefile`](Snakefile)).

First, we recommend looking at [`src/load_data.py`](src/load_data.py) to see how mock visibilities $\mathcal{V}(u,v)$ are generated from the mock image and baselines.

Then, we recommend looking at [`src/plot_baselines.py`](src/plot_baselines.py) and [`src/dirty_image.py`](src/dirty_image.py) to make diagnostic plots of the baseline and a dirty image of the data, to check that everything appears as you might expect.

You can run these simple scripts using

```
$ snakemake -c1 all
```

![baselines](analysis/baselines.png)

![Dirty Beam and Image](analysis/dirty_image.png)

# RML imaging workflow

The RML imaging workflow is demonstrated in [`src/sgd.py`](src/sgd.py). We recommend looking through that file before reading the rest of this document. If you are new to PyTorch idioms, we recommend familiarizing yourself with the [PyTorch basics](https://mpol-dev.github.io/MPoL/background.html#pytorch) first.

The RML imaging workflow is not part of the Snakemake workflow, instead, one runs the script like

```
$ python src/sgd.py --epochs=5
```

Note this will just result in a short test. Run `python src/sgd.py --help` to see all available command line arguments, and see below for configurations that will result in better images.

One can visualize the results using Tensorboard via

```shell
$ tensorboard --logdir runs
```

# Validation
Since this example uses mock data, we have the advantage of knowing the true sky image. This allows us to calculate a 'validation loss' between the synthesized image and the true sky.

Expand Down Expand Up @@ -61,5 +109,4 @@ Adding entropy regularization, and reducing learning rate slightly.
python src/sgd.py --tensorboard-log-dir=runs/ent0 --load-checkpoint=checkpoints/2.pt --save-checkpoint=checkpoints/ent0.pt --lr 1e-1 --FWHM 0.05 --epochs=50 --lam-ent=1e-5
```


Note that we could have started directly with the entropy regularization if we wished. The previous just demonstrates an exploratory workflow.
Binary file added 02-sgd/analysis/baselines.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added 02-sgd/analysis/dirty_image.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion 02-sgd/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
mpol
tensorboard
visread
matplotlib
matplotlib
snakemake
1 change: 1 addition & 0 deletions 02-sgd/src/sgd.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ def main():
vis_data.uu, vis_data.vv, vis_data.weight, vis_data.data
)

print("running on ", device)
print("total vis", len(train_dataset))

# set the batch sizes for the loaders
Expand Down
Loading