Error Encountered while using pybna python library "InvalidTextRepresentation: malformed array literal: "{6'0"}" DETAIL: Unexpected array element.'

[mintu07ruet]

Hi spencerrecneps@ I was using the python pybna 1.0.2 library to calculate the stress for Portland, Oregon. I have overwritten the boundary shape file, the library successfully read the boundary shape file, imported census blocks, and received census jobs data using the below line of codes.

City Boundary is available here: https://gis-pdx.opendata.arcgis.com/datasets/1559e31273654eb9858397861f1fdefa/explore?location=45.629289%2C-122.551444%2C10.44

'''
i = pybna.Importer(config="config.yaml")
i.import_boundary('city_of_portland_boundary.shp', overwrite=True)
i.import_census_blocks(fips=41, overwrite=True)
i.import_census_jobs("received.neighborhood_census_block_jobs",state="OR", overwrite=True)
'''

C:\Users\mxm5116.conda\envs\geo_env\lib\site-packages\pybna\importer.py:58: ResourceWarning: unclosed file <_io.TextIOWrapper name='config.yaml' mode='r' encoding='cp1252'>
self.config = self.parse_config(yaml.safe_load(open(config)))
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Connecting to database
Copying boundary to database
Loading boundary

C:\Users\mxm5116.conda\envs\geo_env\lib\site-packages\pandas\io\sql.py:761: UserWarning: pandas only support SQLAlchemy connectable(engine/connection) ordatabase string URI or sqlite3 DBAPI2 connectionother DBAPI2 objects are not tested, please consider using SQLAlchemy
warnings.warn(

Loading data from http://www2.census.gov/geo/tiger/TIGER2010BLKPOPHU/tabblock2010_41_pophu.zip
Filtering blocks to boundary
Filtering out water
Copying blocks to database
Loading data for state OR
No data for state OR for year 2020. Checking previous year.

C:\Users\mxm5116.conda\envs\geo_env\lib\site-packages\pybna\importer.py:351: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
jobs = jobs_main.append(jobs_aux,ignore_index=True)

Copying jobs to database

After the above lines of code, when I was trying to import osm network using the below line of code, I encountered the error-
'''
i.import_osm_network()
i.import_osm_destinations()
'''

Downloading OSM data

Copying OSM ways to database

Copying OSM intersections to database

Processing OSM data in database

'''
InvalidTextRepresentation Traceback (most recent call last)
Input In [13], in <cell line: 1>()
----> 1 i.import_osm_network()
2 i.import_osm_destinations()

File ~.conda\envs\geo_env\lib\site-packages\pybna\importer.py:473, in Importer.import_osm_network(self, roads_table, ints_table, boundary_file, boundary_buffer, osm_file, keep_holding_tables, srid, km, overwrite)
462 self.gdf_to_postgis(
463 nodes,
464 osm_nodes_table,
(...)
468 conn=conn
469 )
471 conn.commit()
--> 473 self._process_osm(
474 roads_table,roads_schema,ints_table,ints_schema,osm_ways_table,
475 osm_ways_schema,osm_nodes_table,osm_nodes_schema,srid,km,overwrite,conn
476 )
478 conn.commit()
479 conn.close()

File ~.conda\envs\geo_env\lib\site-packages\pybna\importer.py:566, in Importer._process_osm(self, roads_table, roads_schema, ints_table, ints_schema, osm_ways_table, osm_ways_schema, osm_nodes_table, osm_nodes_schema, srid, km, overwrite, conn)
564 query = self.read_sql_from_file(fquery)
565 q = sql.SQL(query).format(**subs)
--> 566 cur.execute(q)
567 cur.close()
569 if commit:

InvalidTextRepresentation: malformed array literal: "{6'0"}"
DETAIL: Unexpected array element.
'''

I am using the first time of this library. Any help to solve this issue is greatly appreciated!

[mintu07ruet]

@spencerrecneps , I have attached my config file if you need it to check the issue. Thanks
config.zip
!

[theja]

Hi @mintu07ruet, sorry this took a while to respond. This may been fixed from updates in #83. Can you verify and let me know if it still persists.