FeTS Challenge 2022: Task 1

Task_1/.gitattributes

@@ -0,0 +1 @@
+*.pth filter=lfs diff=lfs merge=lfs -text

Task_1/README.md

@@ -1,4 +1,4 @@
-# FeTS 2021 Challenge Task 1
+# FeTS 2022 Challenge Task 1
 Task 1 (**"Federated Training"**) aims at effective weight aggregation methods for the creation of a consensus model given a pre-defined segmentation algorithm for training, while also (optionally) accounting for network outages.
 
 Please ask any additional questions in our discussion pages on our github site and we will try to update this README.md as we identify confusions/gaps in our explanations and instructions.
@@ -13,7 +13,7 @@ Please ask any additional questions in our discussion pages on our github site a
 
 ### Instructions
 
-1. Register for the FeTS 2021 Challenge [here](https://www.med.upenn.edu/cbica/fets/miccai2021/) and submit a data request.
+1. Register for the FeTS 2022 Challenge [here](https://www.synapse.org/#!Synapse:syn28546456/wiki/617093) and submit a data request.
 2. ```git clone https://github.com/FETS-AI/Challenge.git```
 3. ```cd Challenge/Task_1```
 4. Create virtual environment (python 3.6-3.8): using Anaconda, a new environment can be created and activated using the following commands: 
@@ -23,9 +23,12 @@ conda create -p ./venv python=3.7 -y
 conda activate ./venv
 ```
 5. ```pip install --upgrade pip```
-6. ```pip install .```
-7. ```jupyter notebook```
-8. All lower-level details are in the [FeTS Challenge notebook](./FeTS_Challenge.ipynb)
+6. Install Pytorch LTS (1.8.2) for your system using [these instructions](https://pytorch.org/get-started/locally/)
+7. ```pip install .```
+8. ```python FeTS_Challenge.py```
+9. All lower-level details are in the [FeTS Challenge python file](./FeTS_Challenge.ipy)
+10. To view intermediate results with TensorBoard during training, you can run the following command:
+    ```tensorboard --logdir ~/.local/workspace/logs/tensorboard```
 
 ## Time to Convergence Metric (formerly "communication cost")
 Along with the typical DICE and Hausdorff metrics, we include a "time to convergence metric" intended to encourage solutions that converge to good scores quickly in terms of time. We simulate the time taken to run each round so that competitors don't need to be concerned with runtime optimizations such as compiled vs. interpreted code, and so that final scoring will not depend on the hardware used. This simulated time is computed in the experiment.py file and provided in the metrics output of the experiment execution.
@@ -61,18 +64,18 @@ We assign these network and compute distributions by drawing uniform-randomly fr
 For a given collaborator, these normal distributions are constant throughout the experiment. Again, each possible timing distribution is based on actual timing information from a subset of the hospitals in the FeTS intitiative. You can find these distributions in the experiment.py file (search for ## COLLABORATOR TIMING DISTRIBUTIONS), as well as the random seed used to ensure reproducibility.
 
 ## Data Partitioning and Sharding
-The FeTS 2021 data release consists of a training set and two CSV files - each providing information for how to partition the training data into non-IID institutional subsets. The release will contain subfolders for single patient records whose names have the format `FeTS21_Training_###`, and two CSV files: 
+The FeTS 2022 data release consists of a training set and two CSV files - each providing information for how to partition the training data into non-IID institutional subsets. The release will contain subfolders for single patient records whose names have the format `FeTS2022_###`, and two CSV files: 
 - **partitioning_1.csv**
 - **partitioning_2.csv**
 
 Each of the partitioning CSV files has two columns, `Partition_ID` and `Subject_ID`. The Subject_ID column exhausts of the patient records contained in the release. The InstitutionName column provides an integer identifier indicating to which institution the record should be assigned. The path to a partition CSV can be provided as the value of the parameter ```institution_split_csv_filename``` to the jupyter notebook function run_challenge_experiment to specify the institutional split used when running experimental federated training on your custom federation logic. A description of each of these split CSVs is provided in Table 1. We encourage participants to create and explore training performance for other non-IID splits of the training data to help in developing generalizable customizations to the federated logic that will perform well during the validation and testing phase. A third CSV is hidden from participants and defines a test partitioning to be used in the challenge testing phase. This hidden partitioning (also described in Table 1) is another refinement of the institution split, having similar difficulty level to the institution tumor size split in our own experiments using the default customization functions.
 
-Table 1: Information for partitionings provided in the FeTS 2021 data release as well as the hidden partitioning not provided in the release (to be used in the competition testing phase).
+Table 1: Information for partitionings provided in the FeTS 2022 data release as well as the hidden partitioning not provided in the release (to be used in the competition testing phase).
 
 |     Split name                      |     CSV filename                         |     Description                                                                                                                                                                                       |     Number of institutions      |
 |-------------------------------------|------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------|
-|     Institution Split               |     partitioning_1.csv                |     Split of FeTS 2021 training data by originating institution.                                                                                                                                    |     17                          |
-|     Institution Tumor Size Split    |     partitioning_2.csv      |     Refinement of the institution split by tumor size, further   splitting the larger institutions according to whether a record’s tumor size   fell above or below the mean size for that institution.    |     22                          |
+|     Institution Split               |     partitioning_1.csv                |     Split of FeTS 2022 training data by originating institution.                                                                                                                                    |     23                          |
+|     Institution Tumor Size Split    |     partitioning_2.csv      |     Refinement of the institution split by tumor size, further   splitting the larger institutions according to whether a record’s tumor size   fell above or below the mean size for that institution.    |     33                          |
 |     Test Split                      |          - not provided -       |     Undisclosed refinement of the institution split.                                                                                                                                                  |     Hidden from participants    |
 
 

Task_1/fets_challenge/__init__.py

@@ -1,4 +1,4 @@
-# Provided by the FeTS Initiative (www.fets.ai) as part of the FeTS Challenge 2021
+# Provided by the FeTS Initiative (www.fets.ai) as part of the FeTS Challenge 2022
 
 # Contributing Authors (alphabetical):
 # Micah Sheller (Intel)

Task_1/fets_challenge/custom_aggregation_wrapper.py

@@ -1,9 +1,10 @@
-from openfl.component.aggregation_functions import AggregationFunctionInterface
+from openfl.component.aggregation_functions.experimental import PrivilegedAggregationFunction
 
 
 # extends the openfl agg func interface to include challenge-relevant information
-class CustomAggregationWrapper(AggregationFunctionInterface):
+class CustomAggregationWrapper(PrivilegedAggregationFunction):
     def __init__(self, func):
+        super().__init__()
         self.func = func
         self.collaborators_chosen_each_round = None
         self.collaborator_times_per_round = None
@@ -17,12 +18,12 @@ def set_state_data_for_round(self,
     # pass-through that includes additional information from the challenge experiment wrapper
     def call(self,
              local_tensors,
-             db_iterator,
+             tensor_db,
              tensor_name,
              fl_round,
              *__):
         return self.func(local_tensors,
-                         db_iterator,
+                         tensor_db,
                          tensor_name,
                          fl_round,
                          self.collaborators_chosen_each_round,