firebase deploy firebase:indexes fails when any index was already created

[marcin-ptaszynski]

[REQUIRED] Environment info

firebase-tools: 7.0.2

Platform: Ubuntu

[REQUIRED] Test case

v1: indexes.json

{
  "indexes": [
    {
      "collectionGroup": "authors",
      "queryScope": "COLLECTION",
      "fields": [
        {
          "fieldPath": "name",
          "order": "ASCENDING"
        },
        {
          "fieldPath": "born",
          "order": "ASCENDING"
        }
      ]
    }
  ]
}

v2: indexes.json: added new index

{
  "indexes": [
    {
      "collectionGroup": "authors",
      "queryScope": "COLLECTION",
      "fields": [
        {
          "fieldPath": "name",
          "order": "ASCENDING"
        },
        {
          "fieldPath": "born",
          "order": "ASCENDING"
        }
      ]
    },
    {
      "collectionGroup": "authors",
      "queryScope": "COLLECTION",
      "fields": [
        {
          "fieldPath": "name",
          "order": "ASCENDING"
        },
        {
          "fieldPath": "born",
          "order": "DESCENDING"
        }
      ]
    }
  ]
}

[REQUIRED] Steps to reproduce

1. run firebase deploy --only firestore:indexes with first version of indexes.json
2. update file with additional index and re-run

[REQUIRED] Expected behavior

Both commands are successful and target firestore database contains both indices

[REQUIRED] Actual behavior

First run succeeds, but latter fails with error: Error: HTTP Error: 409, index already exists. Only first index is present.

Workaround

1. Delete all indexes before deploy
2. Place in indexes.json only indexes not present in database.

It would seems this has been caused by: #1120

[samtstern]

@marcin-ptaszynski thanks! This seems like an obvious bug with a simple fix.

[samtstern]

@marcin-ptaszynski I am not able to reproduce this. I am able to repeatedly deploy your v2 indexes file without issue. When I run deploy with --debug I see that it is skipping existing indexes:

$ npx firebase deploy --debug --only firestore:indexes
[2019-07-10T15:53:21.592Z] ----------------------------------------------------------------------
[2019-07-10T15:53:21.594Z] Command:       /Users/samstern/.nvm/versions/node/v8.12.0/bin/node /private/var/folders/xl/6lkrzp7j07581mw8_4dlt3b000643s/T/tmp.KEsvRfvZ/node_modules/.bin/firebase deploy --debug --only firestore:indexes
[2019-07-10T15:53:21.594Z] CLI Version:   7.0.2
[2019-07-10T15:53:21.594Z] Platform:      darwin
[2019-07-10T15:53:21.594Z] Node Version:  v8.12.0
[2019-07-10T15:53:21.595Z] Time:          Wed Jul 10 2019 08:53:21 GMT-0700 (PDT)
[2019-07-10T15:53:21.596Z] ----------------------------------------------------------------------

[2019-07-10T15:53:21.605Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2019-07-10T15:53:21.606Z] > authorizing via signed-in user
[2019-07-10T15:53:21.606Z] [iam] checking project fir-dumpster for permissions ["datastore.indexes.create","datastore.indexes.delete","datastore.indexes.list","datastore.indexes.update","firebase.projects.get"]
[2019-07-10T15:53:21.608Z] >>> HTTP REQUEST POST https://cloudresourcemanager.googleapis.com/v1/projects/fir-dumpster:testIamPermissions  
 permissions=[datastore.indexes.create, datastore.indexes.delete, datastore.indexes.list, datastore.indexes.update, firebase.projects.get]
[2019-07-10T15:53:21.743Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Wed, 10 Jul 2019 15:53:21 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, server-timing=gfet4t7; dur=64, alt-svc=quic=":443"; ma=2592000; v="46,43,39", accept-ranges=none, transfer-encoding=chunked

=== Deploying to 'fir-dumpster'...

i  deploying firestore
i  firestore: reading indexes from firestore.indexes.json...
[2019-07-10T15:53:21.748Z] >>> HTTP REQUEST GET https://firestore.googleapis.com/v1beta2/projects/fir-dumpster/databases/(default)/collectionGroups/-/indexes  
 
[2019-07-10T15:53:22.125Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Wed, 10 Jul 2019 15:53:22 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43,39", accept-ranges=none, transfer-encoding=chunked
[2019-07-10T15:53:22.125Z] >>> HTTP REQUEST GET https://firestore.googleapis.com/v1beta2/projects/fir-dumpster/databases/(default)/collectionGroups/-/fields?filter=indexConfig.usesAncestorConfig=false  
 
[2019-07-10T15:53:22.450Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Wed, 10 Jul 2019 15:53:22 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43,39", accept-ranges=none, transfer-encoding=chunked
i  firestore: there are some indexes defined in your project that are not present in your firestore indexes file. Run firebase firestore:indexes and save the result to correct the discrepancy.
[2019-07-10T15:53:22.451Z] Skipping existing index: {"collectionGroup":"authors","queryScope":"COLLECTION","fields":[{"fieldPath":"name","order":"ASCENDING"},{"fieldPath":"born","order":"ASCENDING"}]}
[2019-07-10T15:53:22.451Z] Skipping existing index: {"collectionGroup":"authors","queryScope":"COLLECTION","fields":[{"fieldPath":"name","order":"ASCENDING"},{"fieldPath":"born","order":"DESCENDING"}]}
i  firestore: there are some field overrides defined in your project that are not present in your firestore indexes file. Run firebase firestore:indexes and save the result to correct the discrepancy.
✔  firestore: deployed indexes in firestore.indexes.json successfully

✔  Deploy complete!

Can you try doing the same and pasting the output here?

[marcin-ptaszynski]

@samtstern , apologies. It was a mistake on my side - I've simplified example too much.

Duplicate is not skipped when __name__ field is explicit in index, e.g.:

{
  "indexes": [
    {
      "collectionGroup": "authors",
      "queryScope": "COLLECTION",
      "fields": [
        {
          "fieldPath": "name",
          "order": "ASCENDING"
        },
        {
          "fieldPath": "born",
          "order": "DESCENDING"
        },
        {
          "fieldPath": "__name__",
          "order": "DESCENDING"
        }
      ]
    }
  ]
}

According to Firestore documentation, __name__ is always added implicitly to index and inherits last field path order, so above would be equivalent if 3rd field was left-out.

Still, sth like:

{
  "indexes": [
    {
      "collectionGroup": "authors",
      "queryScope": "COLLECTION",
      "fields": [
        {
          "fieldPath": "name",
          "order": "ASCENDING"
        },
        {
          "fieldPath": "born",
          "order": "DESCENDING"
        },
        {
          "fieldPath": "__name__",
          "order": "ASCENDING"
        }
      ]
    }
  ]
}

is not possible. Probably most use-cases don't need such combination, anyway.

We can fix it on our side by removing explicit __name__.

Thanks a lot!

[samtstern]

@marcin-ptaszynski that makes sense, thanks for following up! We have some logic to ignore __name__ since the backend always reports it but we don't want users to have to specify that in their indexes. And yes as you said, the index is the same without that.

[Beamanator]

I had the same error occurring - Error: HTTP Error: 409, index already exists on firebase-tools v7.0.0 without having "fieldPath": "__name__", explicitly set anywhere in my firestore.indexes.json file.

Once I updated from v7.0.0 to v7.1.0, the issue was fixed. I'm only noting this here because I didn't see any recent release notes about fixing this issue, so hopefully this can help someone like me :)

[actuallymentor]

I'm running into the same error. Is there debug info I can provide that is useful?

[samtstern]

@actuallymentor could you file a new issue if you're able to reproduce this?