output: Add support for WEIGHT_FLOAT32 with text vector

[HashidaTKS]

Groonga regarded weight as uint32_t even if its actual type is float32 in grn_output_vector.

So even if we specified WEIGHT_FLOAT32 like the query below, the output weight was not float32 but uint32_t.

How to reproduce:

table_create Memos TABLE_HASH_KEY ShortText

column_create Memos tags COLUMN_VECTOR|WITH_WEIGHT|WEIGHT_FLOAT32 ShortText

load --table Memos
[
{
  "_key": "Groonga is fast",
  "tags": {
    "groonga": 2,
    "full text search": 1
  }
}
]

select Memos

In the query above,

        {
          "groonga": 2,
          "full text search": 1
        }

is not expected, the expected result is like below.

        {
          "groonga": 2.8,
          "full text search": 1.2
        }

[HashidaTKS]

@kou @komainu8

Would you review this when you have time?

[kou]

Could you move tests to test/command/suite/select/columns/flags/vector/ because they are tests for WITH_WEIGHT and WEIGHT_FLOAT32 flags with COLUMN_VECTOR flag?

[kou]

Suggested change

	&_value, &weight_float, &domain);
	&_value, &weight_float, &domain);

[HashidaTKS]

@kou

Thanks, I have addressed your comments.

[kou]

Why do you want to include cast in test path? Is cast important for this? Can we reproduce this with --columns[text_tags].type Tags? If so, we should use it. We can add a test for --columns[text_tags].type ShortText in a separated pull request.

How about

• test/command/suite/select/columns/flags/vector/with_weight_weight_float32.test
• test/command/suite/select/columns/flags/vector/with_weight.test
  ?

[HashidaTKS]

Can we reproduce this with --columns[text_tags].type Tags?

I tried it but didn't reproduce this.
It is because Groonga passes grn_output_uvector in that case (although I don't understand why...) and grn_output_uvector already supports WEIGHT_FLOAT32.

table_create Tags TABLE_PAT_KEY ShortText
[[0,0.0,0.0],true]
table_create Memos TABLE_HASH_KEY ShortText
[[0,0.0,0.0],true]
column_create Memos tags COLUMN_VECTOR|WITH_WEIGHT|WEIGHT_FLOAT32 Tags
[[0,0.0,0.0],true]
load --table Memos
[
{
  "_key": "Groonga is fast",
  "tags": {
    "groonga": 2.8,
    "full text search": 1.2
  }
}
]
[[0,0.0,0.0],1]
select Memos --columns[text_tags].stage output              --columns[text_tags].type Tags              --columns[text_tags].flags COLUMN_VECTOR|WITH_WEIGHT|WEIGHT_FLOAT32              --columns[text_tags].value 'tags'              --output_columns _key,tags,text_tags
[
  [
    0,
    0.0,
    0.0
  ],
  [
    [
      [
        1
      ],
      [
        [
          "_key",
          "ShortText"
        ],
        [
          "tags",
          "Tags"
        ],
        [
          "text_tags",
          "Tags"
        ]
      ],
      [
        "Groonga is fast",
        {
          "groonga": 2.8,
          "full text search": 1.2
        },
        {
          "groonga": 2.8,
          "full text search": 1.2
        }
      ]
    ]
  ]
]

Do you know why Groonga passes grn_output_uvector in that case?

[kou]

Because reference vector uses grn_uvector not grn_vector.

[kou]

Then, how about the following?

table_create Memos TABLE_HASH_KEY ShortText
column_create Memos tags COLUMN_VECTOR|WITH_WEIGHT|WEIGHT_FLOAT32 ShortText
load --table Memos
[
{
  "_key": "Groonga is fast",
  "tags": {
    "groonga": 2.8,
    "full text search": 1.2
  }
}
]
select Memos \
  --columns[copied_tags].stage output \
  --columns[copied_tags].type ShortText \
  --columns[copied_tags].flags COLUMN_VECTOR|WITH_WEIGHT|WEIGHT_FLOAT32 \
  --columns[copied_tags].value 'tags' \
  --output_columns _key,tags,copied_tags

[HashidaTKS]

Thanks.
I took some trial and error, and the query to reproduce ended up like below.

table_create Memos TABLE_HASH_KEY ShortText

column_create Memos tags COLUMN_VECTOR|WITH_WEIGHT|WEIGHT_FLOAT32 ShortText

load --table Memos
[
{
  "_key": "Groonga is fast",
  "tags": {
    "groonga": 2.8,
    "full text search": 1.2
  }
}
]

select Memos

[HashidaTKS]

@kou

Please re-review this.

[kou]

Could you update the description before we merge this?

[HashidaTKS]

Could you update the description before we merge this?

Oh, sorry, updated.