Converting from GeoJson to ESRI Feature Set causes malformed ring data

[jozsef-kepes]

Using Python API 1.7.1 I have attempted to use the feature layer method edit_features() with a set of features that have been converted from GeoJson to an ESRI Feature Set. In doing so this error message was returned:

The specified geometry is not in the correct format. Object reference not set to an instance of an object.

While looking at the converted ESRI Feature Set, the ring data had nulls inserted into the data. Upon closer inspection of the from_geojson() FeatureSet method in /arcgis/featues/featury.py a few lines caught my attention.

# Lines 873-896
elif geo_type == "MultiPolygon":
    rings = []
    if HASARCPY:
        geom = arcpy.AsShape(geom)
        geometry = Geometry(json.loads(geom))
    else:
        coordkey = ([d for d in geom if d.lower() == 'coordinates']
                        or ['coordinates']).pop()
        coordinates = geom[coordkey]
        typekey = ([d for d in geom if d.lower() == 'type']
                        or ['type']).pop()
        if geom[typekey].lower() == "polygon":
            coordinates = [coordinates]
        part_list = []
        for part in coordinates:
            part_item = []
            for idx, ring in enumerate(part):
                if idx:
                    part_item.append(None)
                for coord in ring:
                    part_item.append(coord)
            if part_item:
                part_list.append([part_item])
        geometry["rings"] = part_list[0]

This seems to append a None value to the beginning of every new ring set. I’m not sure whether the reasoning for this is for internal api usage. However, when using this method within my context the rings with null entries will not be accepted by the edit_features() method. Instead, I would be expecting it to be placed within an array for the relevant ring geometries, and for the set of rings to also be placed in an array

Below I have provided code I used to test this issue.

import json
from arcgis.gis import GIS, User, ContentManager
from arcgis.features import FeatureSet

gis = GIS("https://www.arcgis.com", username="Username", password="******")
user = User(gis, 'Folder')
content = ContentManager(gis)

poly_service = content.search('title:poly_title AND type:Feature Service')
feature_item = poly_service[0]   
feature_layers = feature_item.layers
feature_set = feature_layers[0].query()
vem_flayer = feature_layers[0]

# Corrected ESRI-json rings without nulls
with open('corrected_esri_test.json') as infile:
    test = json.load(infile)
    esri_json = FeatureSet(test['features'])
    edit = vem_flayer.edit_features(adds=esri_json)
    print(edit)

# ESRI-json generated with from_geojson() method
with open('poly_test.json') as infile:
    test = json.load(infile)
    esri_json = FeatureSet.from_geojson(test)
    with open('esri_test.json','w') as outfile:
        outfile.write(str(esri_json))
        outfile.close()
        edit = vem_flayer.edit_features(adds=esri_json)
        print(edit)

[achapkowski]

Please post your conda environment installs settings by doing:

conda env export > environment.yml

Also please post your JSON files.

Are you using an environment with ArcPy or Not? If so, what version of Pro are you using?

[jozsef-kepes]

ArcPy is not being used and the python API is being run external of Pro. We are trying to run this on a AWS Lambda function. Looking at /arcgis/featues/feature.py provides some information as to why this might be a bug. Separate logic is being executed when ArcPy is not being used. We also noted the logic when its not using ArcPy only seems to take the first ring from a multi-polygon and leaves the rest, as well as adding nulls to separate inner rings.

The arcgis library was installed by running: pip install arcgis --no-deps by following the instructions found here

The json files used have been included below.
json-files.zip

[dionliddell]

I've discovered a possibly related issue within the feature.py module. I've posted a geonet question about it here: https://community.esri.com/thread/257817-featuresetfromgeojson-bug

Specifically, within the feature.py code snippet posted by the OP:

    geom = arcpy.AsShape(geom)
    geometry = Geometry(json.loads(geom))

It appears that geom is being set as an arcpy geometry object, and then geom is passed within the Geometry parameter as json.loads(geom). It appears that under certain circumstances this code is run and it causes json.loads to crash because json.loads is expecting 'str, bytes or bytearray' and not an arcpy geometry object.

[achapkowski]

A fix will be in the next release of the ArcGIS API for Python. Thanks for the detailed post on how to reproduce your issue.

[nfink-nlt]

I'm still seeing this issue in releases 1.8 and 1.9 where the conversion of a multipolygon to feature set will cause malformed data because a null value is being add for each ring in features\feature.py at lines 1058-1060:
for idx, ring in enumerate(part): if idx: part_item.append(None)

This will always append a null value to the list because the index will always be True with enumerate. Commenting out the code that add the null value to the list fixes the error. This bug currently keeps me from uploading the data to ArcGIS Online with the error "The specified geometry is not in the correct format. Object reference not set to an instance of an object."