02 Slice Census

[GeoWang18]

Executing: 02SliceCensus "D:\Project\OD time matrix\Draft\JournalofTransportGeography\Tool\TestData\US_PriSec_2018.gdb\PriSec_routing_LA\PriSec_routing_LA_ND_Junctions" "D:\Project\OD time matrix\Draft\JournalofTransportGeography\Tool\TestData\temp2.gdb" OBJECTID 3
Start Time: Sun Jul 5 11:33:59 2020
Running script 02SliceCensus...
Failed script 02SliceCensus...

Traceback (most recent call last):
File "D:\Project\OD time matrix\Draft\JournalofTransportGeography\Tool\LargeODcost-master\02Slice_census.py", line 43, in
arcpy.FeatureClassToFeatureClass_conversion(input, outputgdb, os.path.splitext(os.path.split(input)[1])[0]+"_"+str(i), "{0} IN ({1})".format(ID, str(criteria[1:len(criteria)-1])))
File "c:\program files (x86)\arcgis\desktop10.6\arcpy\arcpy\conversion.py", line 1918, in FeatureClassToFeatureClass
raise e
ExecuteError: ERROR 999999: Error executing function.
An invalid SQL statement was used.
An invalid SQL statement was used. [PriSec_routing_LA_ND_Junctions]
The table was not found. [PriSec_routing_LA_ND_Junctions_0]
Failed to execute (FeatureClassToFeatureClass).

Failed to execute (02SliceCensus).
Failed at Sun Jul 5 11:34:00 2020 (Elapsed Time: 1.38 seconds)

[everknight]

This error shows because the ID field of the input slice data is not a text field.
I propose to add an error message to alert user to convert ID field to text field.

[GeoWang18]

That is good, I just check the code and found the ID = arcpy.GetParameterAsText(2)

[geowcz]

This error shows because the ID field of the input slice data is not a text field.
I propose to add an error message to alert user to convert ID field to text field.

I added a new field in TEXT type and set the value = ID. It was deleted in both the input and output feature classes after finishing the process.

The revised codes are:

import os, arcpy
import timer_class as tc

'''
input = r"D:\Code\HSABook\Chapter2\FL_HSA.gdb\ZIP_Points"
outputgdb = r"D:\Code\HSABook\Chapter2\Large_ZIP-to-ZIP_OD\test.gdb"
ID = 'OBJECTID'
step = 200
'''

input = arcpy.GetParameterAsText(0)
outputgdb = arcpy.GetParameterAsText(1)
ID = arcpy.GetParameterAsText(2)
step = arcpy.GetParameterAsText(3)

totaln = int(arcpy.GetCount_management(input).getOutput(0))

if step == '#' or step == '':
if totaln > 200:
step = 200
else:
step = totaln
else:
step = int(step)

sys.path.append(os.path.split(os.path.realpath(file))[0])

arcpy.SetProgressor("step", "Reading files...")

newfieldname = str(ID) + "COPY"
fieldList = arcpy.ListFields(input)
for field in fieldList:
if field.name == newfieldname:
arcpy.DeleteField_management(input, [newfieldname])
arcpy.AddField_management(input, newfieldname, "TEXT")
arcpy.CalculateField_management(input, newfieldname, "!" + ID + "!", "PYTHON3")

i = 1
sc = arcpy.SearchCursor(input)
select_list = []
row = sc.next()
while row:
#select_list.append(str(row.getValue(ID)))
select_list.append(row.getValue(newfieldname))
if i % 10000 == 0:
arcpy.SetProgressorPosition(int(i/totaln*100))
i += 1
row = sc.next()

select_list.sort()
arcpy.ResetProgressor()

i = 0
watch = tc.timer()
arcpy.SetProgressor("step", "Splitting files in to smaller trunks...")
while i < totaln:
criteria = str(select_list[i:i+step])
outfeatname = os.path.splitext(os.path.split(input)[1])[0]+""+str(i)
#arcpy.FeatureClassToFeatureClass_conversion(input, outputgdb, os.path.splitext(os.path.split(input)[1])[0]+""+str(i), "{0} IN ({1})".format(str(ID), str(criteria[1:len(criteria)-1])))
arcpy.FeatureClassToFeatureClass_conversion(input, outputgdb, outfeatname, "{0} IN ({1})".format(newfieldname, str(criteria[1:len(criteria)-1])))
arcpy.DeleteField_management(outputgdb +"\" + outfeatname, [newfieldname])

progress = int(i/totaln*100)
watch.lap()
arcpy.SetProgressorPosition (progress)

if i % (step*5) == 0 and i > 0:
	togo = watch.format_time(watch.avg_sec*(int((totaln-i)/step)+1))
	arcpy.AddMessage('Each batch take about {0}secs. {1} hours  {2} mins {3} secs to go...'.format(str(watch.avg_sec)[0:5], togo[0], togo[1], togo[2]))

i += step

arcpy.DeleteField_management(input, [newfieldname])