Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
095ad25
refactor: update to python3.9 & otb 8.0.1
Jun 7, 2024
cc7011e
refactor: add missing configuration file
Jun 7, 2024
243d1fb
wip : start json file as parameter.
Jun 7, 2024
7cedd9b
refactor: json files as input parameters
Jun 12, 2024
13c58f1
style: remove unreachable main() and hardcoded path
Jun 12, 2024
36e1660
refactor: in_shps only used as list
Jun 12, 2024
7576db7
bug : cast values
Jun 12, 2024
6bb62f0
refactor: json as parameter
Jun 12, 2024
48d6954
refactor: paths as parameters
Aug 1, 2024
6e56232
add tests
Aug 1, 2024
bd47e80
test: add asserts to tests
Oct 1, 2024
07d5086
test: add tests docstrings
Oct 4, 2024
2ef9f91
doc: start adding doc
Aug 2, 2024
d37f41d
bug + doc: add Montreux tutorial
Oct 10, 2024
7601d6f
test: add missing data
Oct 23, 2024
7f4e0a4
test: add missing data
Oct 23, 2024
7483712
test: bug add missing directory and remove hard coded path
Oct 28, 2024
16fe605
style: add headers
Oct 28, 2024
3e57c54
feat: add docker and singularity capabilities.
Oct 28, 2024
7e66b14
doc: update install documentation
Oct 28, 2024
fc99f67
test: add asserts
Oct 30, 2024
fac2120
CI: add ci
Dec 5, 2024
306c1ba
Update build_and_test_app.yml
ArthurVincentCS Dec 5, 2024
78aab07
update CI
Dec 5, 2024
416357a
Merge branch 'add_doc' of https://github.com/ArthurVincentCS/ALCD int…
Dec 5, 2024
4f672c9
CI: add ci
Dec 5, 2024
4ae5430
CI: add ci
Dec 5, 2024
0fd17d1
doc: small edits and update the install section
Mar 7, 2025
9fef6fe
pulled doc_small_edits from upstream
cadauxe Apr 8, 2025
25ab4b2
docs : updated tutorials
cadauxe Apr 8, 2025
5ec891c
docs : updated tutorials
cadauxe Apr 8, 2025
f3e3400
docs : WIP tutorials
cadauxe Apr 8, 2025
199cac6
docs : corrected image display in tuto
cadauxe Apr 10, 2025
176a4ee
Update build_and_test_app.yml
cadauxe Apr 10, 2025
652acf1
docs : ci demo
cadauxe May 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build_and_test_app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: ALCD
on:
pull_request:
branches:
- "new_add_doc"
- "master"

permissions:
contents: read
Expand Down Expand Up @@ -58,4 +58,4 @@ jobs:
export OTB_INSTALL_DIR=/opt/otb
export LC_NUMERIC=C

pytest -s --cov-fail-under=65
pytest -s --cov-fail-under=65
24 changes: 22 additions & 2 deletions docs/source/how_to_install_alcd.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,26 @@
# ALCD Installation and Usage Guide

**Note**: ALCD is currently not distributed as a Python package. To use ALCD, you first need to clone the ALCD repository and then create either a Docker or Singularity image.
**Note**: ALCD is currently not distributed as a Python package. To use ALCD, you first need to clone the ALCD repository and then create either a Docker or Singularity image or use it in a conda environment.

## Solution 1 : Use a conda environment

If you don't have conda installed yet, then follow the [conda install guide](https://www.anaconda.com/docs/getting-started/miniconda/install)

```bash
git clone https://github.com/CNES/ALCD.git
cd ALCD
conda env create -f environment.yml
conda activate ALCD # then your are ready to use ALCD
```

## Solution 2 : Use the ALCD docker image

```bash
docker pull cnes/ALCD:latest
docker run -it cnes/ALCD:latest
```

## Solution 3: Build your own ALCD docker image

### Step 1: Clone the ALCD Repository
To begin, clone the ALCD repository to your local machine:
Expand Down Expand Up @@ -34,7 +54,7 @@ Explanation of the command options:
-v /path/to/local/DATA:/DATA Mounts your local / /path/to/local/DATA directory to /DATA within the Docker container, allowing data access and output within this directory.
alcd:latest: Refers to the Docker image tagged as alcd, built in step 2.

### Step 3: Create the Singularity Image
## Solution 4: Build your own ALCD singularity image

Navigate to the /singularity directory within the ALCD project:

Expand Down
Binary file added docs/source/images/colorized_classif_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/images/colorized_classif_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/images/quicklook_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/images/quicklook_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/source/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# ACLD documentation
# ALCD documentation

This is the current user manual for the Active Learning Cloud Detection (ALCD). This code produces classification ref-
erence masks from a Sentinel-2 L1C product, and is particularly designed to detect clouds and
Expand Down
183 changes: 62 additions & 121 deletions docs/source/notebooks/montreux.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "853e763d423e46e3a03b6dfc4ca39b8e",
"model_id": "cbf9a41ff98340d8a5da8f984a938a50",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -53,7 +53,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "ed53896a773f465da1f70801290550ac",
"model_id": "b1d7c666c63e47f3b7a99b30b99c5b7b",
"version_major": 2,
"version_minor": 0
},
Expand Down Expand Up @@ -264,21 +264,12 @@
"execution_count": 3,
"id": "ec8d0240-f49b-43ba-9b48-507b41a66ec7",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/osgeo/gdal.py:287: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.\n",
" warnings.warn(\n"
]
}
],
"outputs": [],
"source": [
"%%bash\n",
"# considering PATH_ALCD is the path to the script 'all_run_alcd.py'\n",
"export PATH_ALCD=\"../../..\"\n",
"python $PATH_ALCD/all_run_alcd.py -force True -s 0 -f True -l Montreux -d 20240213 -c 20240213 -dates False -kfold False -global_parameters global_parameters.json -paths_parameters paths_configuration.json -model_parameters model_parameters.json > tuto_log.txt"
"python $PATH_ALCD/all_run_alcd.py -force True -s 0 -f True -l Montreux -d 20240213 -c 20240213 -dates False -kfold False -global_parameters global_parameters.json -paths_parameters paths_configuration.json -model_parameters model_parameters.json > tuto_log.txt 2>/dev/null"
]
},
{
Expand All @@ -300,7 +291,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "54378f883c6e421d9a5a201a6dd60063",
"model_id": "6b9d43ae4fcb46a19055034f75988224",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -314,7 +305,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "0ca57bce63b946c3af59d67828bbeeec",
"model_id": "d51dee540def4bfa9ab6d6ac9124011e",
"version_major": 2,
"version_minor": 0
},
Expand Down Expand Up @@ -388,55 +379,16 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 5,
"id": "3d43025d-bde6-426c-bd59-7a49dc485e2e",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/osgeo/ogr.py:560: FutureWarning: Neither ogr.UseExceptions() nor ogr.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.\n",
" warnings.warn(\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x5908d47b4570): Needed elements (8000) will be clamped to total elements (165)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x5908d4bb7fc0): Needed elements (8000) will be clamped to total elements (164)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x5908d4c61110): Needed elements (8000) will be clamped to total elements (177)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x5908d47b2eb0): Needed elements (8000) will be clamped to total elements (147)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x5908d47b2070): Needed elements (8000) will be clamped to total elements (172)\n",
"\n",
"\n",
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.\n",
" return _methods._mean(a, axis=axis, dtype=dtype,\n",
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in divide\n",
" ret = ret.dtype.type(ret / rcount)\n",
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/numpy/core/_methods.py:206: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
" ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,\n",
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/numpy/core/_methods.py:163: RuntimeWarning: invalid value encountered in divide\n",
" arrmean = um.true_divide(arrmean, div, out=arrmean,\n",
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/numpy/core/_methods.py:198: RuntimeWarning: invalid value encountered in divide\n",
" ret = ret.dtype.type(ret / rcount)\n"
]
}
],
"outputs": [],
"source": [
"%%bash\n",
"export PATH_ALCD=\"../../..\"\n",
"python $PATH_ALCD/all_run_alcd.py -force True -s 1 -f True -l Montreux -d 20240213 -c 20240213 -dates False -kfold False -global_parameters global_parameters.json -paths_parameters paths_configuration.json -model_parameters model_parameters.json > tuto_log.txt"
"python $PATH_ALCD/all_run_alcd.py -force True -s 1 -f True -l Montreux -d 20240213 -c 20240213 -dates False -kfold False -global_parameters global_parameters.json -paths_parameters paths_configuration.json -model_parameters model_parameters.json > tuto_log.txt 2>/dev/null"
]
},
{
Expand All @@ -451,14 +403,14 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 6,
"id": "bdbb6818-5d4c-4461-8e73-09166ad8580f",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "0f34354a34b24fc59d1972d87aa9e8a2",
"model_id": "17f9e12f76bb41528fb02822b874890c",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -472,7 +424,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "487bfc0b6f8c49569262ff52c71d139d",
"model_id": "51455cfbf17f43ad9559e90d0188d563",
"version_major": 2,
"version_minor": 0
},
Expand Down Expand Up @@ -510,19 +462,28 @@
"\n",
"Here are the results after the first classification by ALCD.\n",
"\n",
"<div style=\"display: flex; justify-content: space-around;\">\n",
" <figure style=\"text-align: center;\">\n",
" <img src=\"montreux_outputs/Montreux_32TLS_20240213/Out/quicklook.png\" alt=\"Image 2\" style=\"width: 400px;\">\n",
" <figcaption>Sentinel-2 acquisition</figcaption>\n",
" </figure>\n",
" <figure style=\"text-align: center;\">\n",
" <img src=\"montreux_outputs/Montreux_32TLS_20240213/Out/colorized_classif.png\" alt=\"Image 1\" style=\"width: 400px;\">\n",
" <figcaption>Sentinel-2 classified</figcaption>\n",
" </figure>\n",
" <figure style=\"text-align: center;\">\n",
" <img src=\"../images/table.png\" alt=\"Image 2\" style=\"width: 400px;\">\n",
" <figcaption>legend</figcaption>\n",
" </figure>\n",
"<div style=\"text-align: center;\">\n",
"\n",
" <img src=\"../images/quicklook_1.png\" alt=\"Image 2\" width=\"500\">\n",
" \n",
" <p>Sentinel-2 acquisition</p>\n",
"\n",
"</div>\n",
"\n",
"<div style=\"text-align: center;\">\n",
" \n",
" <img src=\"../images/colorized_classif_1.png\" alt=\"Image 2\" width=\"500\">\n",
" \n",
" <p>Sentinel-2 classified</p>\n",
"\n",
"</div>\n",
"\n",
"<div style=\"text-align: center;\">\n",
" \n",
" <img src=\"../images/table.png\" alt=\"Image 2\" width=\"500\">\n",
" \n",
" <p>Classification legend</p>\n",
"\n",
"</div>"
]
},
Expand All @@ -538,7 +499,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 7,
"id": "bb41460f-0c58-4629-80ba-70c2b9b7f940",
"metadata": {},
"outputs": [],
Expand All @@ -563,44 +524,15 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 8,
"id": "50c5869e-6851-43ea-aae4-6ff7c0b5f624",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/ecadaux/miniconda3/envs/env_otb2/lib/python3.11/site-packages/osgeo/ogr.py:560: FutureWarning: Neither ogr.UseExceptions() nor ogr.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.\n",
" warnings.warn(\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x6007486be5f0): Needed elements (8000) will be clamped to total elements (161)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x600748c28930): Needed elements (8000) will be clamped to total elements (161)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x6007485d6b50): Needed elements (8000) will be clamped to total elements (174)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x6007485d80a0): Needed elements (8000) will be clamped to total elements (150)\n",
"\n",
"\n",
"WARNING: In /work/THEIA/CES/oso/CD/conda_env/iota2_build_env/conda-bld/otb_1736246894519/work/OTB/Modules/Filtering/Statistics/src/otbSamplerBase.cxx, line 42\n",
"RandomSampler (0x6007485d6800): Needed elements (8000) will be clamped to total elements (175)\n",
"\n",
"\n"
]
}
],
"outputs": [],
"source": [
"%%bash\n",
"# considering PATH_ALCD is the path to the script 'all_run_alcd.py'\n",
"export PATH_ALCD=\"../../..\"\n",
"python $PATH_ALCD/all_run_alcd.py -force False -s 1 -f False -l Montreux -d 20240213 -c 20240213 -dates False -kfold False -global_parameters global_parameters.json -paths_parameters paths_configuration.json -model_parameters model_parameters.json > log_tuto.txt"
"python $PATH_ALCD/all_run_alcd.py -force False -s 1 -f False -l Montreux -d 20240213 -c 20240213 -dates False -kfold False -global_parameters global_parameters.json -paths_parameters paths_configuration.json -model_parameters model_parameters.json > log_tuto.txt 2>/dev/null"
]
},
{
Expand All @@ -612,19 +544,28 @@
"source": [
"The results generated during this iteration overwrite the previous ones. With the addition of samples, we can see an evolution in the classification\n",
"\n",
"<div style=\"display: flex; justify-content: space-around;\">\n",
" <figure style=\"text-align: center;\">\n",
" <img src=\"montreux_outputs/Montreux_32TLS_20240213/Out/quicklook.png\" alt=\"Image 2\" style=\"width: 400px;\">\n",
" <figcaption>Sentinel-2 acquisition</figcaption>\n",
" </figure>\n",
" <figure style=\"text-align: center;\">\n",
" <img src=\"montreux_outputs/Montreux_32TLS_20240213/Out/colorized_classif.png\" alt=\"Image 1\" style=\"width: 400px;\">\n",
" <figcaption>Sentinel-2 classified</figcaption>\n",
" </figure>\n",
" <figure style=\"text-align: center;\">\n",
" <img src=\"../images/table.png\" alt=\"Image 2\" style=\"width: 400px;\">\n",
" <figcaption>legend</figcaption>\n",
" </figure>\n",
"<div style=\"text-align: center;\">\n",
"\n",
" <img src=\"../images/quicklook_2.png\" alt=\"Image 2\" width=\"500\">\n",
" \n",
" <p>Sentinel-2 acquisition</p>\n",
"\n",
"</div>\n",
"\n",
"<div style=\"text-align: center;\">\n",
" \n",
" <img src=\"../images/colorized_classif_2.png\" alt=\"Image 2\" width=\"500\">\n",
" \n",
" <p>Sentinel-2 classified</p>\n",
"\n",
"</div>\n",
"\n",
"<div style=\"text-align: center;\">\n",
" \n",
" <img src=\"../images/table.png\" alt=\"Image 2\" width=\"500\">\n",
" \n",
" <p>Classification legend</p>\n",
"\n",
"</div>\n",
"\n",
"## Advance usages\n",
Expand Down Expand Up @@ -720,7 +661,7 @@
"\t},\n",
"```\n",
"\n",
"### User features\n",
"### User features \n",
"\n",
"Users can integrate custom processes to modify the input data before classification by adding *\"user_function\"* and *\"user_module\"* fields in the \"user_choices\" section of the *global_parameters.json* file:\n",
"\n",
Expand Down Expand Up @@ -787,7 +728,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
"version": "3.11.11"
}
},
"nbformat": 4,
Expand Down
Loading