Calculate response Demo crashes #44

trotor · 2017-07-05T09:42:19Z

Steps:

Download ArcSDM and ArcSDM_DemoData.
Unzip ArcSDM-Master.zip, this creates ArcSDM-master folder.
Unzip ArcSDM_DemoData.zip inside ArcSDM-master folder, this creates Data folder.
Execute file InitWorkDir.bat inside ArcSDM-master, this creates work folder.
Open ArcMAP 10.5 and open ArcSDM_Desktop_work.mxd inside work folder
Create folder connection for ArcSDM-master folder.
In Arc Toolbox, select add toolbox and select ArcSDM-master/Toolbox/ ArcSDM.pyt
In CalculateResponse demo.tbx, Step 1 and Step 2 appear with a red x. As well as LogisticRegressionDemotbx
In Step 1, select edit and appears the model where calculate weights appears with a red x.
Selected Calculate weights directly with
a. Evidence raster layer : lithoname200k, rc_dnsstru, rc_magn_res4k, rc_till_cu, rc_em_real
b. Training points orogenic gold thin
c. Changed Output to end in WD
d. Everting else as default
Run calculate response with the obtained tables, orogenic_gold_thin and defaults
ArcMAP closes. Output obtained before crash:

Running script CalculateResponse...
Reloaded ArcSDM modules
========== arcsdm values  ==========
ArcSdm 5.00.12 for ArcGis (Pro and Desktop)
Workspace:           C:\Users\irca\Downloads\ArcSDM-master\work\database.gdb (LocalDatabase)
Scratch workspace:   C:\Users\irca\Downloads\ArcSDM-master\work\scratch.gdb (LocalDatabase)
Mask:                "study_area" and it is FeatureClass
Mask size:           98856.95500162625
Cell Size:           500.0
# Training Sites:    40
Unit Cell Area:      1.0km^2, Cells in area: 98856.95500162625
Prior Probability:   0.000405
Training Set:        C:\Users\irca\Downloads\ArcSDM-master\work\database.gdb\Study\orogenic_gold_thin
Study Area Raster:   C:\Users\irca\Downloads\ArcSDM-master\work\database.gdb\study_area
Study Area Area:     98856.95500162625km^2
Map Units:           meter
========================================= Starting calculate response =========================================
Study Area:          98856.95500162625
# training points:   40
Prior_prob:          0.00040462504635452285
Input rasters: ['lithoname200k', "'Reclassified rasters\\rc_dnsstru'", "'Reclassified rasters\\rc_em_real'", "'Reclassified rasters\\rc_magn_res4k'", "'Reclassified rasters\\rc_till_cu'"]
Input rasters: ['lithoname200k', 'rc_dnsstru', 'rc_em_real', 'rc_magn_res4k', 'rc_till_cu']
Creating weight rasters
=========================================
Processing lithoname200k

The text was updated successfully, but these errors were encountered:

trotor · 2017-07-06T12:31:32Z

Could someone test this with new demo dataset?

T

trotor · 2017-07-11T07:31:16Z

Waiting for feedback, can't replicate

trotor · 2017-07-19T07:20:32Z

Can't replicate this with user description of the problem with new demodata and toolbox (10.4.1 arcgis).

Completed script CalculateResponse...   Succeeded at Wed Jul 19 10:07:13 2017 (Elapsed Time: 1 minutes 17 seconds) | Completed script CalculateResponse...   Succeeded at Wed Jul 19 10:07:13 2017 (Elapsed Time: 1 minutes 17 seconds) | Completed script CalculateResponse... |   | Succeeded at Wed Jul 19 10:07:13 2017 (Elapsed Time: 1 minutes 17 seconds)
-- | -- | -- | -- | --
Completed script CalculateResponse...   Succeeded at Wed Jul 19 10:07:13 2017 (Elapsed Time: 1 minutes 17 seconds) | Completed script CalculateResponse... |   | Succeeded at Wed Jul 19 10:07:13 2017 (Elapsed Time: 1 minutes 17 seconds)
Completed script CalculateResponse...
  | Succeeded at Wed Jul 19 10:07:13 2017 (Elapsed Time: 1 minutes 17 seconds)

trotor · 2017-07-19T08:59:55Z

Confirmed crash in ArcGis pro 2.0 investigating

trotor · 2017-07-19T10:13:40Z

arcpy.ArcSDM.CalculateResponse("lithoname200k", r"C:\ArcSDM\work\database.gdb\lithoname200k_WC", "orogenic_gold_thin", None, -99, 1, r"C:\ArcSDM\work\database.gdb\W_pprb", r"C:\ArcSDM\work\database.gdb\W_std", r"C:\ArcSDM\work\database.gdb\W_MDvar", r"C:\ArcSDM\work\database.gdb\W_Tstd", r"C:\ArcSDM\work\database.gdb\W_conf")

This crashes the tool - investigating

trotor · 2017-07-19T10:31:27Z

Command line :

arcpy.ImportToolbox(r"C:\arcsdm\toolbox\arcsdm.pyt")

arcpy.env.workspace = r"C:\ArcSDM\work\database.gdb"
arcpy.env.scratchWorkspace = r"C:\ArcSDM\work\database.gdb"
arcpy.env.mask = r"C:\ArcSDM\work\database.gdb\study_area"

arcpy.ArcSDM.CalculateResponse(r"C:\ArcSDM\work\database.gdb\lithoname200k", r"C:\ArcSDM\work\database.gdb\lithoname200k_WC", r"C:\ArcSDM\work\database.gdb\orogenic_gold_thin", None, -99, 1, r"C:\ArcSDM\work\database.gdb\W_pprb", r"C:\ArcSDM\work\database.gdb\W_std", r"C:\ArcSDM\work\database.gdb\W_MDvar", r"C:\ArcSDM\work\database.gdb\W_Tstd", r"C:\ArcSDM\work\database.gdb\W_conf")

trotor · 2017-07-19T11:02:18Z

Crash happens on this line (186):
gp.AddJoin_management(RasterLayer,"Value",Wts_Table,"CLASS")

irvcaza · 2017-07-19T15:28:39Z

Managed to execute Calculate response successfully using only

rc_till_cu
rc_magn_res4k
rc_em_real
But any combination that contains
rc_dnsstru or
lithoname_200k
produces the error. Therefore is more related to the layers.

Studding deeper I found out that the Coordinate Systems differ

orogenic_gold_thin -> EUREF_FIN_TM35FIN
rc_dnsstru -> ETRS_1989_Transverse_Mercator
rc_em_real -> EUREF_FIN_TM35FIN
rc_till_cu -> EUREF_FIN_TM35FIN
rc_magn_res4k -> EUREF_FIN_TM35FIN
lithoname200k -> ETRS_1989_Transverse_Mercator

That is possibly the origin of our error

trotor · 2017-07-20T11:31:07Z

It is problem with output spatial reference - needs confirmation

trotor · 2017-07-21T05:52:57Z

https://www.dropbox.com/s/nxmzxex0jhpbskq/issue44.rar?dl=0

Waiting for confirmation of bug.

irvcaza · 2017-07-21T10:04:43Z

Issue sent to Esri DE, waiting for response.

trotor · 2017-10-08T05:09:12Z

Closing this - demodata fixed and problem pinpointed. Thanks!

trotor · 2018-05-21T19:31:14Z

Reopening this because of #98

Eliasmgprado · 2018-07-10T14:30:19Z

I'm with the same problem, using the data in ArcSDM_Demodata/DemoData_version_files (dowloaded from github) using the following layers for Wofe Demo testing:

orogenic_Au
lithoname200k
rc_till_cu
rc_till_co
rc_dst_worms
rc_appa_res
rc_magn_res4k

ArcMap crashes when Calculate Response is executed. I have already checked the spatial reference and its ok.

Running on ArcMap 10.5.1

trotor · 2018-07-10T15:24:10Z

Okay, thank you for your feedback! Which version of ArcGis are you running this against?

…

On Tue, Jul 10, 2018 at 5:30 PM Elias Martins Guerra Prado < ***@***.***> wrote: I'm with the same problem, using the data in ArcSDM_Demodata/DemoData_version_files (dowloaded from github) using the following layers for Wofe Demo testing: - orogenic_Au - rc_dnsstru - rc_em_real - rc_till_cu - rc_magn_res4k - lithoname200k ArcMap crashes when Calculate Response is executed. I have already checked the spatial reference and its ok. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#44 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKV5eXgPfii7YlzkWk0jg49TLXhFw4iVks5uFLp8gaJpZM4OOGL0> .

Eliasmgprado · 2018-07-11T12:29:48Z

Running on ArcMap 10.5.1

Eliasmgprado · 2018-07-11T13:52:38Z

I got it!

The problem is in line 211 of calculateresponse.py

arcpy.AddJoin_management(RasterLayer,"VALUE",Wts_Table,"CLASS")

When i run the code below

my_raster_layer = arcpy.Raster(RasterLayer)
if my_raster_layer.hasRAT:
     arcpy.AddMessage("Has a VAT!")
else:
     arcpy.AddMessage("No VAT!")

before the line 205 - arcpy.MakeRasterLayer_management(Input_Raster,RasterLayer) -
the algorithm output "No VAT", that is, the RasterLayer created do not have Attribute Table, probably this is causing the crash.

This way, i have make the following modifications to the calculateresponse.py code for circunvent this issue.

Comment line 205
arcpy.MakeRasterLayer_management(Input_Raster,RasterLayer)
Modified line 211, replacing RasterLayer by Input_Raster
arcpy.AddJoin_management(Input_Raster,"VALUE",Wts_Table,"CLASS")
Modified line 236, replacing RasterLayer by Input_Raster
arcpy.CopyRaster_management(Input_Raster,Temp_Raster,'#','#',NoDataArg2)
Comment line 381
gp.makerasterlayer(Input_Raster,RasterLayer)
Comment line 383, No need for AddJoin again, as the Input_Raster already has the join
gp.AddJoin_management(Input_Raster,"Value",Wts_Table,"CLASS")
Modified line 397, replacing RasterLayer by Input_Raster
arcpy.CopyRaster_management(Input_Raster, Temp_Raster,"#","#",NoDataArg2)

After these modifications the code is running well. However for AddJoin_management() function work with the Input_Raster, i have to export all the used rasters to a gdb before running the tool.

Thanks for the attention, and for the help.

trotor · 2018-07-11T14:03:49Z

Thanks for the work you have put on this, very good! Does your comment in the end mean that this doesn't work with file system inputs, just with geodatabase? It is one serious showstopper for some tools (we would like them to work on both environments)

…

On Wed, Jul 11, 2018 at 4:52 PM Elias Martins Guerra Prado < ***@***.***> wrote: *I got it!* The problem is in line 211 of calculateresponse.py arcpy.AddJoin_management(RasterLayer,"VALUE",Wts_Table,"CLASS") When i run the code below my_raster_layer = arcpy.Raster(RasterLayer) if my_raster_layer.hasRAT: arcpy.AddMessage("Has a VAT!") else: arcpy.AddMessage("No VAT!") before the line 205 - arcpy.MakeRasterLayer_management(Input_Raster,RasterLayer) - the algorithm output "No VAT", that is, the RasterLayer created do not have Attribute Table, probably this is causing the crash. This way, i have make the following modifications to the calculateresponse.py code for circunvent this issue. - Comment line 205 arcpy.MakeRasterLayer_management(Input_Raster,RasterLayer) - Modified line 211, replacing RasterLayer by Input_Raster arcpy.AddJoin_management(Input_Raster,"VALUE",Wts_Table,"CLASS") - Modified line 236, replacing RasterLayer by Input_Raster arcpy.CopyRaster_management(Input_Raster,Temp_Raster,'#','#',NoDataArg2) - Comment line 381 gp.makerasterlayer(Input_Raster,RasterLayer) - Comment line 383, No need for AddJoin again, as the Input_Raster already has the join gp.AddJoin_management(Input_Raster,"Value",Wts_Table,"CLASS") - Modified line 397, replacing RasterLayer by Input_Raster arcpy.CopyRaster_management(Input_Raster, Temp_Raster,"#","#",NoDataArg2) After these modifications the code is running well. However for AddJoin_management() function work with the Input_Raster, i have to export all the used rasters to a gdb before running the tool. Thanks for the attention, and for the help. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#44 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKV5eTxb0B_oD-1tToWbWbsg9SsqjVwdks5uFgMngaJpZM4OOGL0> .

Eliasmgprado · 2018-07-11T14:10:55Z

Yes, for AddJoin_management to work without use the makerasterlayer function, the input must be with geodatabase. But if you want the tool to accept file system inputs, you can add some lines of code to transform the input files to geodatabase before start join the tables.

trotor · 2018-07-12T04:29:15Z

Okay, when I have time I must study this. Perhaps there is way to make it work on both with alternative data flows.

…

On Wed, Jul 11, 2018 at 5:10 PM Elias Martins Guerra Prado < ***@***.***> wrote: Yes, for AddJoin_management to work without use the makerasterlayer function, the input must be with geodatabase. But if you want the tool to accept file system inputs, you can add some lines of code to transform the input files to geodatabase before start join the tables. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#44 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKV5eTu34QhmKND55Q_OxwFsuv7X6K1Dks5uFgdwgaJpZM4OOGL0> .

artolaiho · 2020-06-17T11:35:18Z

I haven't same test data than above but I have tested with current data. My experience is that Calculate Response (from arto-dev branch) works with ArcMap and ArcGIS Pro if workspace is either File Geodatabase (which I recommend) or File System.

Here is short test using python on ArcMap 10.6.1:

import arcpy
print (arcpy.env.mask)
C:\ArcSDM\Work\DemoData_version_files\derivatives\study_area_g\study_area_g
import arcgisscripting
gp = arcgisscripting.create()
print(gp.workspace)
C:\ArcSDM\work
print(gp.scratchworkspace)
C:\ArcSDM\work\Scratch
Input_Raster = r"C:\ArcSDM\Work\DemoData_version_files\geology\lithoname200k\lithoname200k"
!!! Calculate Response may crash if Input Raster is RasterDataset but don't crash if it is RasterBand
Wts_Table = r"C:\ArcSDM\Work\DemoData_version_files\lithoname200k_CT.dbf"
!!! This table is output from Calculate Weights.
Training_Points = r"C:\ArcSDM\Work\DemoData_version_files\shape_files\Orogenic_Au.shp"
outputrastername = Input_Raster.replace(".","_")
outputrastername = outputrastername[:10] + "W"
Output_Raster = gp.CreateScratchName(outputrastername, '', 'rst', gp.scratchworkspace)
RasterLayer = "OutRas_lyr"
arcpy.MakeRasterLayer_management(Input_Raster,RasterLayer)
<Result 'OutRas_lyr'>
!!! MakeRasterLayer writes RasterLayer to TEMP folder.
arcpy.AddJoin_management(RasterLayer,"VALUE",Wts_Table,"CLASS")
<Result 'OutRas_lyr'>
Temp_Raster = gp.CreateScratchName('tmp_rst', '', 'rst', gp.scratchworkspace)
arcpy.CopyRaster_management(RasterLayer,Temp_Raster,'#','#','#')
<Result 'C:\ArcSDM\work\Scratch\tmp_rst17'>
outras = arcpy.sa.Lookup(Temp_Raster,"WEIGHT")

and so on... (without crash)

artolaiho · 2020-06-17T12:24:51Z

This works also with ArcMap 10.6.1 and ArcGIS Pro 2.5:
import arcpy
arcpy.ImportToolbox(r"C:\ArcSDM\Toolbox\ArcSDM.pyt")
arcpy.env.workspace = r"C:\ArcSDM\Work"
arcpy.env.scratchWorkspace = r"C:\ArcSDM\Work\Scratch"
arcpy.env.mask = r"C:\ArcSDM\Work\DemoData_version_files\derivatives\study_area_g\study_area_g"
!!! Note: Mask must be RasterBand, not RasterDataset (study_area_g twice above)
!!! Also Input raster must be RasterBand, not RasterDataset (lithoname200k twice below)
arcpy.ArcSDM.CalculateResponse(r"C:\ArcSDM\work\DemoData_version_files\geology\lithoname200k\lithoname200k", r"C:\ArcSDM\Work\DemoData_version_files\lithoname200k_CT.dbf", r"C:\ArcSDM\work\DemoData_version_files\shape_files\Orogenic_Au.shp", "#", -99, 1, r"C:\ArcSDM\work\W_pprb", r"C:\ArcSDM\work\W_std", r"C:\ArcSDM\work\W_MDvar", r"C:\ArcSDM\work\W_Tstd", r"C:\ArcSDM\work\W_conf")

artolaiho · 2020-08-17T08:15:28Z

Load the newest version of ArcSDM (V5.02) and test again. If the problem persists, reopen this issue.

trotor added the bug label Jul 5, 2017

trotor added this to the Toolbox demo milestone Jul 5, 2017

trotor self-assigned this Jul 5, 2017

trotor added Waiting input Demo labels Jul 13, 2017

trotor assigned irvcaza Jul 14, 2017

trotor mentioned this issue Jul 20, 2017

Wrong spatial reference system on demodatalayers #52

Closed

trotor removed the Demo label Jul 21, 2017

trotor closed this as completed Oct 8, 2017

trotor mentioned this issue May 21, 2018

Logistic regression tool crashes on certain data #98

Closed

trotor reopened this May 21, 2018

artolaiho closed this as completed Aug 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Calculate response Demo crashes #44

Calculate response Demo crashes #44

trotor commented Jul 5, 2017 •

edited

Loading

trotor commented Jul 6, 2017

trotor commented Jul 11, 2017

trotor commented Jul 19, 2017

trotor commented Jul 19, 2017

trotor commented Jul 19, 2017

trotor commented Jul 19, 2017 •

edited

Loading

trotor commented Jul 19, 2017

irvcaza commented Jul 19, 2017

trotor commented Jul 20, 2017

trotor commented Jul 21, 2017

irvcaza commented Jul 21, 2017

trotor commented Oct 8, 2017

trotor commented May 21, 2018

Eliasmgprado commented Jul 10, 2018 •

edited

Loading

trotor commented Jul 10, 2018 via email

Eliasmgprado commented Jul 11, 2018

Eliasmgprado commented Jul 11, 2018

trotor commented Jul 11, 2018 via email

Eliasmgprado commented Jul 11, 2018

trotor commented Jul 12, 2018 via email

artolaiho commented Jun 17, 2020

artolaiho commented Jun 17, 2020

artolaiho commented Aug 17, 2020

Calculate response Demo crashes #44

Calculate response Demo crashes #44

Comments

trotor commented Jul 5, 2017 • edited Loading

trotor commented Jul 6, 2017

trotor commented Jul 11, 2017

trotor commented Jul 19, 2017

trotor commented Jul 19, 2017

trotor commented Jul 19, 2017

trotor commented Jul 19, 2017 • edited Loading

trotor commented Jul 19, 2017

irvcaza commented Jul 19, 2017

trotor commented Jul 20, 2017

trotor commented Jul 21, 2017

irvcaza commented Jul 21, 2017

trotor commented Oct 8, 2017

trotor commented May 21, 2018

Eliasmgprado commented Jul 10, 2018 • edited Loading

trotor commented Jul 10, 2018 via email

Eliasmgprado commented Jul 11, 2018

Eliasmgprado commented Jul 11, 2018

trotor commented Jul 11, 2018 via email

Eliasmgprado commented Jul 11, 2018

trotor commented Jul 12, 2018 via email

artolaiho commented Jun 17, 2020

artolaiho commented Jun 17, 2020

artolaiho commented Aug 17, 2020

trotor commented Jul 5, 2017 •

edited

Loading

trotor commented Jul 19, 2017 •

edited

Loading

Eliasmgprado commented Jul 10, 2018 •

edited

Loading