RGeo::Geographic::SphericalPointImpl serialization not happening #307
Comments
|
I am experiencing the same behavior ... I have multiple geometry columns, and in some models it works fine, in others it does not. >> MatchingConfiguration.columns.find { |c| c.name == 'area' }
=> #<ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn:0x00007fe4cb94fe00 @sql_type_metadata=#<ActiveRecord::ConnectionAdapters::SqlTypeMetadata:0x00007fe4cb944398 @sql_type="geometry(MultiPolygon,4326)", @type=:geometry, @limit=nil, @precision=nil, @scale=nil>, @geographic=false, @name="area", @null=true, @default=nil, @default_function=nil, @collation=nil, @comment=nil, @serial=nil, @srid=4326, @has_z=false, @has_m=false, @geometric_type=RGeo::Feature::MultiPolygon, @limit={:srid=>4326, :type=>"multi_polygon"}>
>> Product.columns.find { |c| c.name == 'area' }
=> #<ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn:0x00007fe4cbe78350 @sql_type_metadata=#<ActiveRecord::ConnectionAdapters::SqlTypeMetadata:0x00007fe4cbe78508 @sql_type="geometry(MultiPolygon,4326)", @type=:geometry, @limit=nil, @precision=nil, @scale=nil>, @geographic=false, @name="area", @null=true, @default=nil, @default_function=nil, @collation=nil, @comment=nil, @serial=nil, @srid=4326, @has_z=false, @has_m=false, @geometric_type=RGeo::Feature::MultiPolygon, @limit={:srid=>4326, :type=>"multi_polygon"}>
>> MatchingConfiguration.first.area
MatchingConfiguration Load (1.1ms) SELECT "matching_configurations".* FROM "matching_configurations" ORDER BY "matching_configurations"."id" ASC LIMIT $1 [["LIMIT", 1]]
=> #<RGeo::Geographic::SphericalMultiPolygonImpl:0x3ff2668bde84 "MULTIPOLYGON (((8.6 50.1, ...)))">
>> Product.first.area
Product Load (0.5ms) SELECT "products".* FROM "products" ORDER BY "products"."created_at" ASC LIMIT $1 [["LIMIT", 1]]
=> "0106..."same code is working fine with adapter version |
|
Closest related issue i could find is rgeo/rgeo#113. I think there is an issue around those line : |
|
Hmm, this code wasn't touched in a while |
|
I also don’t see that issue in a production setup. My guess is that the dev hot reloading in rails is not reloading the adapter postgis adapter and so the model is reading raw values of the columns. |
|
I do not only see it in dev mode. When running my whole spsec suite it works fine, vut when running a single test with a model that uses geo columns, it just reads the raw string |
|
Still make me think to a load/reload adapter kind of issue. |
|
Would you mind posting what ruby version, server, and any gems used specifically in development/test? I think I've seen this issue before and started looking into it, but I haven't had any luck replicating it. I'm running Ubuntu 18, but other than that our setups look the same. |
|
Sure thing
Gemfile.lock |
|
Hi, we are also seeing some random issue around typecasting and so far our debug journey brought us to this patch electric-feel@0dea53f, which allows PostGIS types to be registered before AR loads the additional types, which should ensure that the AR connection Please feel free to give a try to this patch to see if it solves your problem, we will certainly create a proper PR with more details around the issue next week. AR will most likely also need a separate patch to fix the lazy loading of the type. |
|
Ok i will test this branch and report back |
|
@thibaudgg Looks like your patch works for me ! yay |
# Gemfile
gem 'activerecord-postgis-adapter', '~> 6.0.0', github: 'electric-feel/activerecord-postgis-adapter', branch: '0dea53f7763a21da2159710edf4bc011a9f4ccb8'Result is, all RSpec tests are green ✅ Works for us! Great work @thibaudgg. Can we get this in a PR and get it merged here? |
|
@klaustopher @kwent thanks for trying the patch out, I will for sure submit a PR soon. |
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS type issues on our staging env, while we don't know yet why our production env doesn't seem to be affected, debugging the issue on staging shows that the ActiveRecord connection `type_map` isn't initialized properly which leave the custom `ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn` without a defined type, while usually, the type is lazy loaded correctly later on by `PostgreSQLAdapter#get_oid_type` it could happen that the `ModelSchema#load_schema!` method is called first which will define and cache the attribute type (`@attribute_types`) with the wrong default value type (`ActiveModel::Type::Value`) `type_cast` using the `lookup_cast_type_from_column` method that reads straight from the `type_map` cache without lazy loading the type. The wrong type is then always returned for that column which brakes all the casting logic. This patch fixes the `SchemaStatements#initialize_type_map` method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the custom types. That way the custom column will be properly initialized and be present in the `type_map` cache. Some code references: - The `initialize_type_map` activerecord-postgis-adapter method overwritten https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105 - the `load_additional_types` method in Rails: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L593 - the `get_oid_type` method in Rails that lazy load the type: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L514-L516 - the `lookup_cast_type_from_column` method in Rails that does not lazy load the type: https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78 This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS type issues on our staging env, while we don't know yet why our production env doesn't seem to be affected, debugging the issue on staging shows that the ActiveRecord connection `type_map` isn't initialized properly which leave the custom `ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn` without a defined type, while usually, the type is lazy loaded correctly later on by `PostgreSQLAdapter#get_oid_type` it could happen that the `ModelSchema#load_schema!` method is called first which will define and cache the attribute type (`@attribute_types`) with the wrong default value type (`ActiveModel::Type::Value`) `type_cast` using the `lookup_cast_type_from_column` method that reads straight from the `type_map` cache without lazy loading the type. The wrong type is then always returned for that column which brakes all the casting logic. This patch fixes the `SchemaStatements#initialize_type_map` method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the custom types. That way the custom column will be properly initialized and be present in the `type_map` cache. Some code references: - The `initialize_type_map` activerecord-postgis-adapter method overwritten https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105 - the `load_additional_types` method in Rails: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L593 - the `get_oid_type` method in Rails that lazy load the type: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L514-L516 - the `lookup_cast_type_from_column` method in Rails that does not lazy load the type: https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78 This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS type issues on our staging env, while we don't know yet why our production env doesn't seem to be affected, debugging the issue on staging shows that the ActiveRecord connection `type_map` isn't initialized properly which leave the custom `ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn` without a defined type, while usually, the type is lazy loaded correctly later on by `PostgreSQLAdapter#get_oid_type` it could happen that the `ModelSchema#load_schema!` method is called first which will define and cache the attribute type (`@attribute_types`) with the wrong default value type (`ActiveModel::Type::Value`) `type_cast` using the `lookup_cast_type_from_column` method that reads straight from the `type_map` cache without lazy loading the type. The wrong type is then always returned for that column which brakes all the casting logic. This patch fixes the `SchemaStatements#initialize_type_map` method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the custom types. That way the custom column will be properly initialized and be present in the `type_map` cache. Some code references: - The `initialize_type_map` activerecord-postgis-adapter method overwritten https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105 - the `load_additional_types` method in Rails: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L593 - the `get_oid_type` method in Rails that lazy load the type: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L514-L516 - the `lookup_cast_type_from_column` method in Rails that does not lazy load the type: https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78 This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS type issues , debugging the issue shows that the ActiveRecord connection `type_map` isn't initialized properly which leave the custom `ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn` without a defined type, while usually, the type is lazy loaded correctly later on by `PostgreSQLAdapter#get_oid_type` it could happen that the `ModelSchema#load_schema!` method is called first which will define and cache the attribute type (`@attribute_types`) with the wrong default value type (`ActiveModel::Type::Value`) `type_cast` using the `lookup_cast_type_from_column` method that reads straight from the `type_map` cache without lazy loading the type. The wrong type is then always returned for that column which brakes all the casting logic. This patch fixes the `SchemaStatements#initialize_type_map` method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the custom types. That way the custom column will be properly initialized and be present in the `type_map` cache. Some code references: - The `initialize_type_map` activerecord-postgis-adapter method overwritten https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105 - the `load_additional_types` method in Rails: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L593 - the `get_oid_type` method in Rails that lazy load the type: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L514-L516 - the `lookup_cast_type_from_column` method in Rails that does not lazy load the type: https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78 This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS serialization/deserialization issues, debugging the issues shows that the ActiveRecord connection `type_map` isn't initialized properly which leave the custom `ActiveRecord::ConnectionAdapters::PostGIS::SpatialColumn` with no mapping for its PostGIS OID. Usually, Rails would add missing OID mappings when encountered, however, in that particular code path, it does not happen and because of the way `super` is done, that happens too late. Another issue in Rails must also be fixed, but that's another story :) This patch fixes the `SchemaStatements#initialize_type_map` method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the other PostGIS OIDs. That way the custom column will be properly initialized and be present in the `type_map` cache. Some code references: - The `initialize_type_map` activerecord-postgis-adapter method overwritten https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105 - the `load_additional_types` method in Rails: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L593 - the `get_oid_type` method in Rails that lazy load the type: https://github.com/rails/rails/blob/8d57cb39a88787bb6cfb7e1c481556ef6d8ede7a/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L514-L516 - the `lookup_cast_type_from_column` method in Rails that does not lazy load the type: https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78 This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS serialization/deserialization issues, debugging the issues shows that the ActiveRecord connection `type_map` isn't initialized properly. It does not contain the PostGIS OID mappings that are retrieved in [`load_additional_types`](https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105). Usually, Rails would add missing OID mappings when encountered, however, there are code paths that go around this (See [`lookup_cast_type_from_column`](https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78)). This is most likely a bug in Rails internals. This patch fixes the [`initialize_type_map`](https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105) method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the other PostGIS OIDs. That way the custom column will be properly initialized and be present in the `type_map` cache. This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS serialization/deserialization issues, debugging the issues shows that the ActiveRecord connection `type_map` isn't initialized properly. It does not contain the PostGIS OID mappings that are retrieved in [`load_additional_types`](https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105). Usually, Rails would add missing OID mappings when encountered, however, there are code paths that go around this (See [`lookup_cast_type_from_column`](https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78)). This is most likely a bug in Rails internals. This patch fixes the [`initialize_type_map`](https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105) method by ensuring that the specific PostGIS types are registered before calling `super` which will then call `load_additional_types` and registers the other PostGIS OIDs. This patch should solve rgeo#307 and rgeo#308.
thibaudgg
added a commit
to electric-feel/activerecord-postgis-adapter
that referenced
this issue
Feb 24, 2020
Since the Rails 6 upgrade, we started seeing some random PostGIS serialization/deserialization issues, debugging the issues shows that the ActiveRecord connection `type_map` isn't initialized properly. It does not contain the PostGIS OID type mappings after `initialize_type_map` has ended. Usually, Rails would add missing OID mappings when encountered, however, there are code paths that go around this (See [`lookup_cast_type_from_column`](https://github.com/rails/rails/blob/15748f6a052d4df68b6acf66456c181b42d9fe25/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L78)). This is most likely a bug in Rails internals. This patch fixes the [`initialize_type_map`](https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105) method by ensuring that the specific PostGIS types are registered before calling `super` which will then call [`load_additional_types`](https://github.com/rgeo/activerecord-postgis-adapter/blob/cc0e3f53520feed0b43800d8ee86561ee03de413/lib/active_record/connection_adapters/postgis/schema_statements.rb#L87-L105) and registers the other PostGIS OIDs. This patch should solve rgeo#307 and rgeo#308.
|
PR #309 submitted! |
|
@keithdoggett I think we can close this issue now? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Environment
Platform affected (untested on other platforms)
OSX
Rails
6.0.2.1
Gemfile.lock
PostgreSQL
Postgis
Given
Issue
Seeing on some when doing some http requests (it's not a consistent error, sometime behave well, sometime behave badly):
Looks like the adapter is not serializing
latlonto aRGeo::Geographic::SphericalPointImplobject.An idea of what could go wrong ?
Regards
The text was updated successfully, but these errors were encountered: