Map disappearing after creation + redirect to dashboard #5264

iriberri · 2015-08-27T12:35:38Z

I've only been able to reproduce this issue in my own team account (iriberri), but these are the steps:

Go to your dashboard
Click on new map
Create it from an empty layer
Skip the dialog of adding a feature in your empty map (it doesn't mind, if I add the data it happens too)
Click on visualize, get the viz.json, open the URL in another tab of the browser
In the map, click on the 🔙 arrow at the left of the map name to go to the dashboard -> this makes the item "ghost tables" appear in Rollbar. The table is dropped and registered by that order.
Refresh the tab with the viz.json: Visualization does not exist

The text was updated successfully, but these errors were encountered:

iriberri · 2015-09-02T17:58:51Z

I just noticed that this morning I created this example: http://bl.ocks.org/iriberri/a19efabe08ff2408f2f0 and the visualization does no longer exist. I haven't deleted it. 😢

iriberri · 2015-09-03T07:45:40Z

Hi team, this is seems to be happening outside my account. SB/6496255

Kartones · 2015-09-03T07:46:46Z

If is important, assign somebody from BE as santana is on vacations this days, but I'm overloaded with tickets and the "big feature" of this sprint so I cannot help you right now :(

juanignaciosl · 2015-09-03T08:03:10Z

It looks like we're creating the table without registering. Ghost tables shouldn't be triggered at all.

iriberri · 2015-09-03T08:05:38Z

This only happens with new empty tables, with new maps created from imports everything is 👍

By the way, the table lives after the ghost table drops and registers (and the contents are there)

rafatower · 2015-09-03T08:07:52Z

Related with oid changes: #3717

iriberri · 2015-09-03T08:14:09Z

First creation of the table:

-[ RECORD 1 ]----+-------------------------------------
name             | untitled_table_432
privacy          | 0
updated_at       | 2015-09-03 08:12:25.658405+00
tags             | 
geometry_columns | 
rows_counted     | 0
rows_estimated   | 0
indexes          | 
database_name    | 
description      | 
table_id         | 170250083
data_import_id   | 
map_id           | 25e25be2-c2fe-46e6-9d2e-70fde0b96abf
id               | 2b29d304-1057-4e0d-9a7e-5d966e9395b4
user_id          | mine

Going to dashboard, I see how it gets removed from the user_tables table.

After some time, I see the new record for it:

-[ RECORD 1 ]----+-------------------------------------
name             | untitled_table_432
privacy          | 0
updated_at       | 2015-09-03 08:13:19.920543+00
tags             | 
geometry_columns | 
rows_counted     | 0
rows_estimated   | 0
indexes          | 
database_name    | 
description      | 
table_id         | 170250094
data_import_id   | 
map_id           | 3cb9c4b5-ac63-4355-b0a4-6f61c06a068b
id               | 64f692ef-3c62-4cdd-bdb0-b53b5f13e10e
user_id          | mine

rafatower · 2015-09-03T08:30:21Z

The scenario is roughly as follows:

an empty table is created
it is cartodbfy'ed
during cartodbfycation it gets rewritten and its OID changes (stored in Table#table_id)
Because of the OID change, ghost tables drops the metadata associated to the new table (including the new viz) and then registers a new table with the new OID.

Things to check:

why the new table is rewritten
order of calls to register/cartodbfy

Comment out call to ghost tables #5264

rafatower · 2015-09-03T09:09:43Z

Got a trace of the full scenario by adding some traces to the code:

Calling cartodbfy:
table.id = b371dd3d-35f2-4462-98b2-870f8e48053b
table.qualified_table_name = "public"."untitled_table_20"
oid = 8664405
backtrace:
/home/developer/src/cartodb/app/models/table.rb:1176:in `open'
/home/developer/src/cartodb/app/models/table.rb:1176:in `cartodbfy'
/home/developer/src/cartodb/app/models/table.rb:444:in `after_create'
/home/developer/src/cartodb/app/models/table/user_table.rb:189:in `after_create'
/home/developer/src/cartodb/app/models/table.rb:86:in `save'
/home/developer/src/cartodb/app/controllers/api/json/tables_controller.rb:37:in `block (2 levels) in create'
/home/developer/src/cartodb/lib/cartodb/stats/aggregator.rb:97:in `method_missing'
/home/developer/src/cartodb/app/controllers/api/json/tables_controller.rb:36:in `block in create'
/home/developer/src/cartodb/lib/cartodb/stats/aggregator.rb:97:in `method_missing'
/home/developer/src/cartodb/app/controllers/api/json/tables_controller.rb:17:in `create'
/home/developer/src/cartodb/app/controllers/application_controller.rb:76:in `wrap_in_profiler'

After execution:
oid = 8664416
--------------------------------------------------------------------------------

**
** link_deleted_tables droping table untitled_table_20
**

Calling cartodbfy:
table.id = 98652961-dedf-4d4c-aeb4-20a67837152e
table.qualified_table_name = "public"."untitled_table_20"
oid = 8664416
backtrace:
/home/developer/src/cartodb/app/models/table.rb:1176:in `open'
/home/developer/src/cartodb/app/models/table.rb:1176:in `cartodbfy'
/home/developer/src/cartodb/app/models/table.rb:444:in `after_create'
/home/developer/src/cartodb/app/models/table/user_table.rb:189:in `after_create'
/home/developer/src/cartodb/app/models/table.rb:86:in `save'
/home/developer/src/cartodb/app/models/user.rb:1270:in `block in link_created_tables'
/home/developer/src/cartodb/app/models/user.rb:1258:in `each'
/home/developer/src/cartodb/app/models/user.rb:1258:in `link_created_tables'
/home/developer/src/cartodb/app/models/user.rb:1183:in `link_ghost_tables'
/home/developer/src/cartodb/lib/resque/user_jobs.rb:31:in `perform'

After execution:
oid = 8664416
--------------------------------------------------------------------------------

This reverts commit 413cf4c.

A bit of explanation about this fix: - Table.create_table_in_database! is a private method - It is invoked from Table.before_create when it is not a migration nor a register of the table, that is, only when creating empty tables. E.g: from TablesController.create - cartodbfy will be called a bit later, but it will be idempotent (no further changes). - If you look into Table.before_create, it will invoke Table.set_table_id which is the right place to edit such a thing (the OID of the just created use rtable). So this is not the perfect solution, but is the one that solves the problem at hand minimizing side effects.

- schema(:reload) is no longer responsiblity of cartodbfy - do not call set_the_geom_column! when creating a new table (it will set it to nil anyway) - call explicitly schema(:reload) where needed

rafatower · 2015-09-04T09:06:41Z

This is what we got right now:

digraph cartodbfy {
        Table_before_create -> Table_import_cleanup [label="if migrate_existing_table.present?"];
        Table_import_cleanup -> Table_cartodbfy;
        Table_cartodbfy -> CDB_CartodbfyTable;
        Table_after_create -> Table_cartodbfy;
        Synchronization_Adapter_cartodbfy -> Table_import_cleanup;
        Synchronization_Adapter_cartodbfy -> Table_cartodbfy;
        Relocator_TableDumper_migrate -> CDB_CartodbfyTable;
        SchemaDumper_migrate -> CDB_CartodbfyTable;
}

or graphically:

That explains the mess with double calls to cartodbfy.

This reverts commit 413cf4c.

What's in this commit: - import_cleanup is no longer responsible for cartodbfy nor schema reload. Also added a comment about overlap with cartodbfy and future deletion. - before_create executes always 2 steps at the end of the process: reload schema and set_table_id. - cartodbfy is no longer responsible for reloading schema. - after_create does no longer call cartodbfy.

import_cleanup used to call cartodbfy as the last step. Not anymore (there are good reasons for that). Instead just call cartodbfy explicitly once right after the import_cleanup and delete the other call. Take into accout successive calls are meant to be "idempotent". So, no effect other than wasting time.

by adding to the list of stubs cartodbfy (now explicit and in the before_create) and schema calls.

juanignaciosl self-assigned this Sep 3, 2015

juanignaciosl assigned rafatower and unassigned juanignaciosl Sep 3, 2015

rafatower pushed a commit that referenced this issue Sep 3, 2015

Comment out call to ghost tables #5264

413cf4c

rafatower pushed a commit that referenced this issue Sep 3, 2015

Merge pull request #5360 from CartoDB/5264-fix-ghost-tables-deletion

6d67557

Comment out call to ghost tables #5264

rafatower pushed a commit that referenced this issue Sep 3, 2015

Revert "Comment out call to ghost tables #5264"

c987713

This reverts commit 413cf4c.

rafatower pushed a commit that referenced this issue Sep 3, 2015

Fix for table deleted right after creation #5264

b3919ff

rafatower pushed a commit that referenced this issue Sep 3, 2015

Add feature flag ghost_tables back #5264

9dffa2a

rafatower pushed a commit that referenced this issue Sep 3, 2015

Set the geom to default for new tables #5264

4ae12b9

rafatower pushed a commit that referenced this issue Sep 4, 2015

Revert "Comment out call to ghost tables #5264"

f3007ee

This reverts commit 413cf4c.

rafatower pushed a commit that referenced this issue Sep 4, 2015

Add feature flag ghost_tables back #5264

66ce4c0

rafatower pushed a commit that referenced this issue Sep 4, 2015

Test for failing scenario #5264

b72a5d6

rafatower pushed a commit that referenced this issue Sep 4, 2015

Fix test #5264

2f1f77d

by adding to the list of stubs cartodbfy (now explicit and in the before_create) and schema calls.

rafatower pushed a commit that referenced this issue Sep 4, 2015

Fix test by mocking Table.schema #5264

411e4d0

rafatower mentioned this issue Sep 4, 2015

5264 fix ghost tables deletion take2 #5383

Merged

rafatower added this to the Torrelodones milestone Sep 4, 2015

rafatower closed this as completed Sep 4, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Map disappearing after creation + redirect to dashboard #5264

Map disappearing after creation + redirect to dashboard #5264

iriberri commented Aug 27, 2015

iriberri commented Sep 2, 2015

iriberri commented Sep 3, 2015

Kartones commented Sep 3, 2015

juanignaciosl commented Sep 3, 2015

iriberri commented Sep 3, 2015

rafatower commented Sep 3, 2015

iriberri commented Sep 3, 2015

rafatower commented Sep 3, 2015

rafatower commented Sep 3, 2015

rafatower commented Sep 4, 2015

Map disappearing after creation + redirect to dashboard #5264

Map disappearing after creation + redirect to dashboard #5264

Comments

iriberri commented Aug 27, 2015

iriberri commented Sep 2, 2015

iriberri commented Sep 3, 2015

Kartones commented Sep 3, 2015

juanignaciosl commented Sep 3, 2015

iriberri commented Sep 3, 2015

rafatower commented Sep 3, 2015

iriberri commented Sep 3, 2015

rafatower commented Sep 3, 2015

rafatower commented Sep 3, 2015

rafatower commented Sep 4, 2015