Skip to content
Merged

Dev #367

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
119 commits
Select commit Hold shift + click to select a range
794b141
added helper function for tests to create all project drafts of a cer…
Dec 29, 2019
6f70bba
added helper function for tests to create all project drafts of a cer…
Dec 29, 2019
6fa7ebf
move sample_data dir from root to tests
Dec 29, 2019
cfd27b2
test template for archive feature
Dec 29, 2019
70374b6
template for writting a test project to firebase
Jan 7, 2020
0933b1a
helper functions for project and project_draft creation from templates.
Jan 13, 2020
a9b29de
use helper functions to create a project.
Jan 13, 2020
94d436a
Create set_up module for loading and creating static test data.
Jan 13, 2020
420dd3d
helper functions for set up and tear down of tests
Jan 13, 2020
69ccb85
make use of set up and tear down helper functions
Jan 13, 2020
86e4de8
add test user
Jan 13, 2020
7766b9f
add test user
Jan 13, 2020
aaf960d
unit tests for firebase_to_postgres modul
Jan 13, 2020
1be3d2a
Merged with dev
Jan 15, 2020
eeb8521
Dedicated folder for unittests to sperate from manual tests scripts a…
Jan 15, 2020
1a2e35f
unittests
Jan 16, 2020
940c1e7
unittests
Jan 16, 2020
ae24976
test for argument project_id beeing empty string or None.
Jan 16, 2020
dccf442
simplify code, improve docstrings and comments, improve exception han…
Jan 18, 2020
fdabeec
tests for update_user_data function.
Jan 18, 2020
4b8b4e1
Template for testing create project.
Jan 22, 2020
3e3ec77
tests
Jan 27, 2020
f3aaf51
add project_draft creation and deletion.
Feb 3, 2020
c2e12ba
test template
Feb 3, 2020
cdd581a
Merge branch 'dev' into unittests
Feb 3, 2020
8a95cc7
Merge branch 'dev' into unittests
Feb 3, 2020
c1742fb
Merge branch 'dev' into unittests
Feb 6, 2020
c6d60c8
Correct test data path
Feb 6, 2020
b1e9dcb
Add if __name__ == __main__
Feb 6, 2020
4f92bdb
return key after pushing to firebase.
Feb 27, 2020
7e75373
Correcgt syntax error for string comparision.
Feb 27, 2020
81786e7
Use click test runner for testing cli.
Feb 27, 2020
595ef8f
Remove unused string formatters.
Feb 27, 2020
c33e2d9
Rename class and function.
Feb 27, 2020
88c7302
Cirrect syntax error in sql query.
Feb 27, 2020
7b33563
Merge branch 'dev' into unittests
Feb 27, 2020
fc1eaeb
Create postgres test project.
Feb 27, 2020
5b14875
Test template.
Feb 27, 2020
4c768dc
Merge branch 'dev' into unittests
Feb 27, 2020
3042ef7
Load and set data in one function
Mar 3, 2020
30acfd8
Merge branch 'travis' into unittests
Mar 4, 2020
37d3d09
Load and set fixtures in postgres and firebase.
Mar 4, 2020
4f5784e
Merge branch 'dev' into unittests
Mar 26, 2020
9340a1b
Remove check for empty argument. It is now been checked in CLI.
Mar 26, 2020
cd0f51e
Move to unittest dir.
Mar 26, 2020
4983708
Removed not needed statements (if and del)
Mar 26, 2020
ed86cdd
Rewrite.
Mar 30, 2020
32dc02c
Rewrite.
Mar 30, 2020
fae5085
Rewrite.
Mar 30, 2020
836936b
Remove integreation_tests dir infavor of unittests and later on tests…
Apr 6, 2020
e8bed94
Test gdal installation
Apr 6, 2020
e68f309
Configuration is now stored in environment vars.
Apr 6, 2020
5d469b6
Ignore imported module not used (F401) error from flake8 for this test.
Apr 6, 2020
7495ac3
Create configuration dict for logging due to path issues with logging…
Apr 6, 2020
8641280
Draft for completeness project.
Apr 10, 2020
b570db1
formatting convention added
TahiraU Apr 14, 2020
017df40
added urlB to build_area_tutorial and completeness tutorial generated
TahiraU Apr 14, 2020
500e8c4
test gitignore
TahiraU Apr 14, 2020
b015ace
test
TahiraU Apr 14, 2020
b38cee0
test2
TahiraU Apr 14, 2020
93ad32e
completeness tutorial added
TahiraU Apr 15, 2020
0b1a1e1
adjusted the tutorial for completeness project
TahiraU Apr 28, 2020
250f94b
adjusted create project for completeness project
TahiraU Apr 30, 2020
cee7169
fetch tileserverB if it's project type 4
TahiraU Apr 30, 2020
c2a14c7
build area modified for completeness
TahiraU May 7, 2020
875f2d1
renamed url to urlB
TahiraU May 7, 2020
453b2ee
simplify manager dashboard code #345
Hagellach37 May 7, 2020
6e3a3f8
set correct default view #345
Hagellach37 May 7, 2020
e0f2045
add .pre-commit config again
Hagellach37 May 7, 2020
e75bcd0
small fix
Hagellach37 May 7, 2020
0a9484f
changed default to TileServer and TileServerB for all project types
Hagellach37 May 7, 2020
3dabcb0
Merge pull request #346 from mapswipe/project_creation_form
Hagellach37 May 7, 2020
9b00129
move change detection projection creation into build_area project cre…
Hagellach37 May 8, 2020
bcf0a9a
add quadkey to aggregated results
Hagellach37 May 8, 2020
fa7d572
Merge pull request #347 from mapswipe/add_quadkey_to_agg_results
Hagellach37 May 8, 2020
c7c7c61
adjust group size based on project type #256
Hagellach37 May 8, 2020
5fac7c7
add basic user stats per project #335
Hagellach37 May 8, 2020
c9e4046
Merge pull request #353 from mapswipe/user_stats
Hagellach37 May 8, 2020
3f885ef
Add type hint
May 12, 2020
41cca1c
Run isort and black
May 12, 2020
566d744
Add information to logging message
May 12, 2020
e1702ac
Remove debugging print statements
May 13, 2020
93e07e7
Remove hardcoded API Keys
May 13, 2020
8ee0760
IMAGE URL modified local directory to input file removed, and moved t…
TahiraU May 14, 2020
fd27250
Define project type and its class constructor in definition.py
May 15, 2020
f5e4b0d
Define project type and its class constructor in definition.py
May 15, 2020
3f0c6aa
Define project type and its class constructor in definition.py
May 15, 2020
20efb7a
create completness tutorial based on xMax=17
TahiraU May 20, 2020
78dfa6c
Rename project types.
May 21, 2020
25ff666
incorporate project types names and add class for tutorial
Hagellach37 May 21, 2020
2b6b238
Merge pull request #359 from mapswipe/rename-project-types
Hagellach37 May 21, 2020
78329a4
Merge branch 'dev' into unittests
May 22, 2020
997b002
Add unittests.
May 22, 2020
ae167f7
move test_create_project.py to wip (work in progres)
May 22, 2020
bbc8ea3
Run unittests
May 22, 2020
1d10841
Use exception firebase_admin.db.TransactionAbortedError(message)
May 22, 2020
8817c6e
move user fixture to project type fixtures
May 22, 2020
788ef01
Removed unused and renamed template.
May 22, 2020
cc0f553
Initialize the Firebase SDK with either GOOGLE_APPLICATION_CREDENTIAL…
May 23, 2020
5d26b0b
Add check if firebase reference is valid.
May 23, 2020
93c0d8d
Remove comment
May 24, 2020
54e4164
Do not run tests due to unresolved authentication error from tarvis t…
May 24, 2020
8a2098f
Remove api cred.
May 24, 2020
3209204
Ran test for gdal installation.
May 24, 2020
36391bd
Ran test for gdal installation.
May 24, 2020
5626995
Merge pull request #361 from mapswipe/unittests
Matthias-Schaub May 24, 2020
bbc57a6
Merge pull request #357 from mapswipe/project_types
Matthias-Schaub May 24, 2020
c32514a
Revert "incorporate project types names and add class for tutorial"
May 24, 2020
dc677af
Merge branch 'dev' into completeness
May 24, 2020
a8811fe
add tutorial creation workflow
Hagellach37 May 26, 2020
b649ffd
fix imports for tests
Hagellach37 May 26, 2020
d8671c2
clean up and adjust tasks_to_geojson for multiple zoom levels
Hagellach37 May 26, 2020
3370513
delete file not used
Hagellach37 May 26, 2020
30d90ad
use TileServer class for project creation
Hagellach37 May 26, 2020
e466b87
Merge pull request #334 from mapswipe/completeness
Hagellach37 May 26, 2020
0e24796
catch error in update user data get last updated timestamp
Hagellach37 May 26, 2020
8a0a877
fix typo in tileServer attribute
Hagellach37 May 26, 2020
d74b76c
Merge pull request #366 from mapswipe/hotfix-add-usernames-in-firebase
Hagellach37 May 27, 2020
f6f30a0
filter only users with timestamp #364
Hagellach37 Jun 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -132,3 +132,4 @@ nginx.conf
*/.firebaserc

.DS_Store

5 changes: 2 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,8 @@ before_script:

script:
# - TODO: Run black and flake8
- python -m unittest -v tests.integration_tests.test_gdal
# TODO: Use line below once tests have been implemented
#- python -m unittest discover -v -s mapswipe_workers/tests/integration_tests/ -p 'test_*.py'
- python -m unittest -v tests.unittests.test_gdal
# - TODO: python -m unittest discover -v -s mapswipe_workers/tests/unittests/ -p 'test_*.py'

# before_deploy:
# # SSH setup to deploy to server after build.
Expand Down
241 changes: 77 additions & 164 deletions manager_dashboard/manager_dashboard/create.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions manager_dashboard/manager_dashboard/css/mapswipe.css
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@

#geometryMap { height: 300px; width: 100%}
#geometryChangeDetectionMap { height: 300px; width: 100%}
#geometryCompletenessMap { height: 300px; width: 100%}

div.project-data {
padding-bottom: 50px !important;
Expand Down
186 changes: 88 additions & 98 deletions manager_dashboard/manager_dashboard/js/forms.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,66 @@ function adjust_textarea(h) {
h.style.height = (h.scrollHeight)+"px";
}

function displayProjectTypeFormular(projectType) {
if (projectType == 1) {
document.getElementById("BuildAreaProjectFormular").style.display = "block";
document.getElementById("FootprintProjectFormular").style.display = "None",
document.getElementById("ChangeDetectionProjectFormular").style.display = "None";
document.getElementById("tileServerBuildArea").value = "bing";
displayTileServer ("bing", "BuildArea", "");
setTimeout(function(){ BuildAreaMap.invalidateSize()}, 400);
} else if (projectType == 2) {
document.getElementById("FootprintProjectFormular").style.display = "block";
document.getElementById("BuildAreaProjectFormular").style.display = "None";
document.getElementById("ChangeDetectionProjectFormular").style.display = "None";
document.getElementById("tileServerFootprint").value = "bing";
displayTileServer ("bing", "Footprint", "");
} else if (projectType == 3) {
document.getElementById("FootprintProjectFormular").style.display = "None"
document.getElementById("BuildAreaProjectFormular").style.display = "None";
document.getElementById("ChangeDetectionProjectFormular").style.display = "block";
document.getElementById("tileServerChangeDetectionA").value = "bing";
document.getElementById("tileServerChangeDetectionB").value = "bing";
displayTileServer ("bing", "ChangeDetectionA", "");
displayTileServer ("bing", "ChangeDetectionB", "");
setTimeout(function(){ ChangeDetectionMap.invalidateSize()}, 400);
function initForm() {
initMap();
displayProjectTypeForm("build_area")
}


function initMap() {
ProjectAoiMap = L.map('geometryMap').setView([0.0, 0.0], 4);
L.tileLayer( 'https://maps.wikimedia.org/osm-intl/{z}/{x}/{y}.png', {
attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a>',
subdomains: ['a','b','c']
}).addTo( ProjectAoiMap );
console.log('added map');
aoiLayer = L.geoJSON().addTo(ProjectAoiMap);
setTimeout(function(){ ProjectAoiMap.invalidateSize()}, 400);

}

function displayProjectTypeForm(projectType) {
document.getElementById("projectType").value = projectType;
switch (projectType) {
case "build_area":
displayTileServer("bing", "A");
document.getElementById("groupSize").value = 120;
document.getElementById("form_project_aoi_geometry").style.display = "block";
document.getElementById("form_project_task_geometry").style.display = "None";
document.getElementById("form_zoom_level").style.display = "block";
document.getElementById("form_tile_server_a").style.display = "block";
document.getElementById("form_tile_server_b").style.display = "None";
setTimeout(function(){ ProjectAoiMap.invalidateSize()}, 400);
break;
case "footprint":
displayTileServer("bing", "A");
document.getElementById("groupSize").value = 25;
document.getElementById("form_project_aoi_geometry").style.display = "None";
document.getElementById("form_project_task_geometry").style.display = "block";
document.getElementById("form_zoom_level").style.display = "None";
document.getElementById("form_tile_server_a").style.display = "block";
document.getElementById("form_tile_server_b").style.display = "None";
break;
case "change_detection":
case "completeness":
displayTileServer("bing", "A");
displayTileServer("bing", "B");
if (projectType == "change_detection") {
document.getElementById("groupSize").value = 25;
} else {
document.getElementById("groupSize").value = 80;
}
document.getElementById("form_project_aoi_geometry").style.display = "block";
document.getElementById("form_project_task_geometry").style.display = "None";
document.getElementById("form_zoom_level").style.display = "block";
document.getElementById("form_tile_server_a").style.display = "block";
document.getElementById("form_tile_server_b").style.display = "block";
setTimeout(function(){ ProjectAoiMap.invalidateSize()}, 400);
break;
}
}

function addTileServerCredits (tileServer, projectType, which) {
function addTileServerCredits (tileServerName, which) {
var credits = {
"bing": "© 2019 Microsoft Corporation, Earthstar Geographics SIO",
"maxar_premium": "© 2019 Maxar",
Expand All @@ -41,40 +74,35 @@ function addTileServerCredits (tileServer, projectType, which) {
"sinergise": "© 2019 Sinergise",
"custom": "Please add imagery credits here."
}

document.getElementById("tileServerCredits"+projectType+which).value = credits[tileServer]
document.getElementById("tileServer"+which+"Credits").value = credits[tileServerName]
}


function displayTileServer (t, projectType, which) {
tileServer = t.value
if (tileServer == "custom") {
document.getElementById("tileServerUrlField"+projectType+which).style.display = "block"
document.getElementById("tileServerLayerNameField"+projectType+which).style.display = "block";
} else if (tileServer == "sinergise") {
document.getElementById("tileServerUrlField"+projectType+which).style.display = "None";
document.getElementById("tileServerLayerNameField"+projectType+which).style.display = "block";
} else {
document.getElementById("tileServerUrlField"+projectType+which).style.display = "None";
document.getElementById("tileServerLayerNameField"+projectType+which).style.display = "None";
function displayTileServer (tileServerName, which) {
switch (tileServerName) {
case "custom":
document.getElementById("tileServer"+which+"UrlField").style.display = "block";
document.getElementById("tileServer"+which+"LayerNameField").style.display = "block";
break;
case "sinergise":
document.getElementById("tileServer"+which+"UrlField").style.display = "None";
document.getElementById("tileServer"+which+"LayerNameField").style.display = "block";
break;
default:
document.getElementById("tileServer"+which+"UrlField").style.display = "None";
document.getElementById("tileServer"+which+"LayerNameField").style.display = "None";
}
addTileServerCredits(tileServer, projectType, which)
addTileServerCredits(tileServerName, which)
}

function clear_fields() {
console.log('clear fields.')
document.getElementById('projectNumber').value = 1
document.getElementById('geometry').value = null
document.getElementById('inputAoi').value = null
document.getElementById('geometryInfo').innerHTML = ''
document.getElementById('geometryContent').innerHTML = ''
BuildAreaLayer.clearLayers()
document.getElementById('geometryChangeDetection').value = null
document.getElementById('geometryChangeDetectionInfo').innerHTML = ''
document.getElementById('geometryChangeDetectionContent').innerHTML = ''
ChangeDetectionLayer.clearLayers()
//document.getElementById('imageText').innerHTML = ''
//document.getElementById('imageFile').src = ''
displayProjectTypeFormular(1)
aoiLayer.clearLayers()
displayProjectTypeForm("build_area")
}

function displaySuccessMessage() {
Expand All @@ -89,30 +117,18 @@ function displayImportForm() {
function openFile(event) {
var input = event.target;

var info_element_id = event.target.id + 'Info'
var content_element_id = event.target.id + 'Content'
var map_element_id = event.target.id + 'Map'

var info_output = document.getElementById(info_element_id);
// clear info field
var info_output = document.getElementById("geometryInfo");
info_output.innerHTML = '';
info_output.style.display = 'block'

var content_output = document.getElementById(content_element_id);

if (event.target.id === 'geometry') {
var map = BuildAreaMap
var layer = BuildAreaLayer
var zoomLevel = parseInt(document.getElementById('zoomLevel').value)
} else {
var map = ChangeDetectionMap
var layer = ChangeDetectionLayer
var zoomLevel = parseInt(document.getElementById('zoomLevelChangeDetection').value)
}
// clear map layers
aoiLayer.clearLayers()

// Check file size before loading
var filesize = input.files[0].size;
if (filesize > 2.5 * 1024 * 1024) {
var err='filesize is too big (max 2.5MB): ' + filesize/(1000*1000)
if (filesize > 1 * 1024 * 1024) {
var err='filesize is too big (max 1MB): ' + filesize/(1000*1000)
info_output.innerHTML = '<b>Error reading GeoJSON file</b><br>' + err;
info_output.style.display = 'block'
} else {
Expand Down Expand Up @@ -145,8 +161,8 @@ function openFile(event) {
info_output.innerHTML += 'Feature Type: ' + type + '<br>';
info_output.style.display = 'block'

// check project size

// check project size, based on zoom level
var zoomLevel = parseInt(document.getElementById('zoomLevel').value);
area = turf.area(feature)/1000000 // area in square kilometers
maxArea = (23 - zoomLevel) * (23 - zoomLevel) * 200
console.log('project size: ' + area + ' sqkm')
Expand All @@ -157,21 +173,16 @@ function openFile(event) {
info_output.style.display = 'block'

// add feature to map
layer.clearLayers()
layer.addData(geojsonData);
map.fitBounds(layer.getBounds());
aoiLayer.addData(geojsonData);
ProjectAoiMap.fitBounds(aoiLayer.getBounds());
console.log('added input geojson feature')

// add text to html object
info_output.innerHTML += 'Project seems to be valid :)';
info_output.style.display = 'block'

if (event.target.id === 'geometry') {
BuildAreaGeometry = text
} else {
ChangeDetectionGeometry = text
}

// set project aoi geometry
projectAoiGeometry = text
}
catch(err) {
info_output.innerHTML = '<b>Error reading GeoJSON file</b><br>' + err;
Expand Down Expand Up @@ -202,30 +213,9 @@ function openImageFile(event) {
reader.readAsDataURL(input.files[0]);
};

function initMap() {
BuildAreaMap = L.map('geometryMap').setView([0.0, 0.0], 4);
L.tileLayer( 'https://maps.wikimedia.org/osm-intl/{z}/{x}/{y}.png', {
attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a>',
subdomains: ['a','b','c']
}).addTo( BuildAreaMap );
console.log('added map');
BuildAreaLayer = L.geoJSON().addTo(BuildAreaMap);
setTimeout(function(){ BuildAreaMap.invalidateSize()}, 400);

ChangeDetectionMap = L.map('geometryChangeDetectionMap').setView([0.0, 0.0], 4);
L.tileLayer( 'https://maps.wikimedia.org/osm-intl/{z}/{x}/{y}.png', {
attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a>',
subdomains: ['a','b','c']
}).addTo( ChangeDetectionMap );
console.log('added map');
ChangeDetectionLayer = L.geoJSON().addTo(ChangeDetectionMap);
setTimeout(function(){ ChangeDetectionMap.invalidateSize()}, 400);
}


function closeModal() {
var modal = document.getElementById("uploadModal");
modal.style.display = "none";
var modalSuccess = document.getElementById("modalSuccess");
modalSuccess.style.display = "none";
}
}
Loading