Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there support of built-in UUID type? #90

Closed
pashinin opened this issue Jun 18, 2020 · 12 comments · Fixed by #104
Closed

Is there support of built-in UUID type? #90

pashinin opened this issue Jun 18, 2020 · 12 comments · Fixed by #104
Assignees

Comments

@pashinin
Copy link

How can I insert uuid field from Golang?

@funny-falcon
Copy link

Do you mean tarantool/tarantool@d68fc29 ?

Certainly one could introduce such type by hands with such ext encoding.

I'll think about soon.

@pashinin
Copy link
Author

pashinin commented Jun 19, 2020

Yes

and it is now possible to insert UUIDs into spaces, but only into unindexed fields

and I wanted to insert into an indexed UUID field.

Edit: can you give an example of how to do it?

@Totktonada
Copy link
Member

2.4.0-182-gd68fc2924 allows to insert an UUID value into an unindexed field, but 2.4.0-183-gb238def80 enables support of indexing of such values.

@pashinin
Copy link
Author

@Totktonada fine. I do not understand what to pass to "insert".

Example:

// insert new tuple { 10, 1 }
resp, err = client.Insert(spaceNo, []interface{}{uint(10), 1})
// or
resp, err = client.Insert("test", []interface{}{uint(10), 1})

How to insert UUID:

resp, err = client.Insert("test", []interface{}{?????-????-????-??????, 1})

I tried:

"github.com/satori/go.uuid"

uuid.NewV4().String(),
// uuid.NewV4().Bytes(),
// uuid.NewV4(),

@funny-falcon
Copy link

Type should be registered in messagepack library.
See "Old way to to register types": https://github.com/tarantool/go-tarantool#custom-unpacking-and-typed-selects-and-function-calls

@pashinin
Copy link
Author

You better know in what format it should be. Look at this please:

First I wirte MP_EXT, then MP_UUID and then bytes of UUID but something is wrong.

func encodeUUID(e *msgpack.Encoder, v reflect.Value) error {
	m := v.Interface().(uuid.UUID)
	// MP_EXT = 10
	if err := e.EncodeUint8(10); err != nil {
		return err
	}

	// MP_UUID = 2
	if err := e.EncodeUint8(2); err != nil {
		return err
	}

	if err := e.EncodeBytes(m.Bytes()); err != nil {
		return err
	}

	return nil
}

Have this error now:

Invalid MsgPack - packet body (0x14)

@eaglemoor
Copy link

Why u can't use uuid as string ?

@funny-falcon
Copy link

Sorry, but I don't understand your code. It doesn't coherent with messagepack specification

It seems vmihailenko's msgpack defines special way to register EXT types:
https://github.com/vmihailenco/msgpack/blob/v2.9.1/ext.go#L26
And then define encoder with format like https://github.com/tarantool/tarantool/blob/master/src/lib/uuid/mp_uuid.c#L42-L74

@funny-falcon
Copy link

But looks like it still will be decoded in a strange way in a generic result:
https://github.com/vmihailenco/msgpack/blob/v2.9.1/ext_test.go#L112

@AmsTaFFix
Copy link

AmsTaFFix commented Jun 9, 2021

hi all, we found a way to send uuid to tarantool with vmihailenco/msgpack.v2

Library: https://github.com/citilinkru/uuid-msgpack

@pashinin
Your example will not work because of method EncodeBytes will always add len of array first, so you get additional 2 bytes + 16 bytes and it's invalid msgpack.

Update msgpack and use code below

app.lua

box.cfg({
    listen = 3301,
    memtx_memory=128 * 1024 * 1024,
    read_only=false,
    log_level=2,
    log_format='json',
})

box.once('bootstrap', function()
    box.schema.user.create('test_user', { password = 'test_password' })
    box.schema.user.grant('test_user', 'read,write,execute', 'universe')
end)

local uuid = require("uuid")
local log = require("log")
api = {}
api.test = function(id)
    log.error("is uuid " .. tostring(uuid.is_uuid(id)))
    log.error("type is " .. type(id))
    log.error("id is " .. id:str())

    return id
end

main.go prototype

package main

import (
	"github.com/google/uuid"
	"github.com/tarantool/go-tarantool"
	"gopkg.in/vmihailenco/msgpack.v2"
	"log"
	"reflect"
)

const extId = 2

func init() {
	msgpack.Register(reflect.TypeOf((*uuid.UUID)(nil)).Elem(),
		func(e *msgpack.Encoder, v reflect.Value,) error {
			id := v.Interface().(uuid.UUID)
			bytes, err := id.MarshalBinary()
			if err != nil {
				return err
			}
			_, err = e.Writer().Write(bytes)
			if err != nil {
				return err
			}

			return nil
		},
		func(d *msgpack.Decoder, v reflect.Value) error {
			bytes := make([]byte, 18)
			_, err := d.Buffered().Read(bytes)
			if err != nil {
				return fmt.Errorf("can't read bytes: %w", err)
			}

			if bytes[0] != codes.FixExt16 {
				return fmt.Errorf("wrong ext len '%d'", bytes[0])
			}

			if bytes[1] != extId {
				return fmt.Errorf("wrong ext id '%d'", bytes[1])
			}

			id, err := uuid.FromBytes(bytes[2:])
			if err != nil {
				return fmt.Errorf("can't create uuid from bytes: %w", err)
			}

			v.Set(reflect.ValueOf(id))
			return nil
		},
	)
	msgpack.RegisterExt(extId, (*uuid.UUID)(nil))
}

func main() {
	tntClient, err := tarantool.Connect("localhost:3301", tarantool.Opts{User: "test_user", Pass: "test_password"})
	if err != nil {
		log.Fatalf("can't connect to tarantool: %s", err)
	}

	id := uuid.New()
	log.Printf("uuid as string is '%s'", id.String())
	resp, err := tntClient.Call17("api.test", []interface{}{id})
	if err != nil {
		log.Fatalf("can't call api.test: %s", err)
	}
	idFromTnt := (resp.Data[0]).(uuid.UUID)
	log.Printf("id returned from tnt as string is '%s'", idFromTnt)
}

@Maximilan4
Copy link

@AmsTaFFix I have a problem with your decoder example, when trying to decode map to struct. Then I have two additional fixed bytes too, as you described above. I add this before uuid.FromBytes(bytes):

if bytes.Equal(buffer[:2], []byte{0xd8, 0x2}) {
    twoMore := make([]byte, 2)
    _, err = d.Buffered().Read(twoMore)

    if err != nil {
	return err
    }
    bytes = append(bytes, twoMore...)
    bytes = bytes[2:]
}

@AmsTaFFix
Copy link

AmsTaFFix commented Jul 5, 2021

@Maximilan4 thank you, i fixed my example. And we created lib for that - https://github.com/citilinkru/uuid-msgpack

@DifferentialOrange DifferentialOrange self-assigned this Oct 29, 2021
DifferentialOrange added a commit that referenced this issue Nov 1, 2021
UUID type was introduced in Tarantool 2.4.1.
See more in commit messages:
tarantool/tarantool@d68fc29

Closes #90
DifferentialOrange added a commit that referenced this issue Nov 1, 2021
UUID type was introduced in Tarantool 2.4.1.
See more in commit messages:
tarantool/tarantool@d68fc29

This patch provides UUID support for all space operations and as
function return result.

Closes #90
DifferentialOrange added a commit that referenced this issue Nov 1, 2021
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages:
tarantool/tarantool@d68fc29


Closes #90
DifferentialOrange added a commit that referenced this issue Nov 15, 2021
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages:
tarantool/tarantool@d68fc29


Closes #90
DifferentialOrange added a commit that referenced this issue Nov 24, 2021
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages [1].

To use UUID with google/uuid in msgpack, import
tarantool/go-tarantool/uuid submodule.

1. tarantool/tarantool@d68fc29

Closes #90
DifferentialOrange added a commit that referenced this issue Nov 25, 2021
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages [1].

To use UUID with google/uuid in msgpack, import
tarantool/go-tarantool/uuid submodule.

1. tarantool/tarantool@d68fc29

Closes #90
ligurio pushed a commit that referenced this issue Mar 7, 2022
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages [1].

To use UUID with google/uuid in msgpack, import
tarantool/go-tarantool/uuid submodule.

1. tarantool/tarantool@d68fc29

Closes #90
ligurio pushed a commit that referenced this issue Mar 30, 2022
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages [1].

To use UUID with google/uuid in msgpack, import
tarantool/go-tarantool/uuid submodule.

1. tarantool/tarantool@d68fc29

Closes #90
ligurio pushed a commit that referenced this issue Apr 7, 2022
This patch provides UUID support for all space operations and as
function return result. UUID type was introduced in Tarantool 2.4.1.
See more in commit messages [1].

To use UUID with google/uuid in msgpack, import
tarantool/go-tarantool/uuid submodule.

1. tarantool/tarantool@d68fc29

Closes #90
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

    Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

    Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

	Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#112).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

	Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 2, 2022
Overview

This release adds a number of features. Also it significantly improves
testing, CI and documentation.

Breaking changes

    There are no breaking changes in the release.

New features

    Support UUID type in msgpack (#90).

    queue-utube handling (#85).

    Master discovery (#113).

    SQL support (#62).

Bugfixes

    Reset buffer if its average use size smaller than quater of
    capacity (#95).

Testing

    Coveralls support (#149).

    Reusable testing workflow (integration testing with latest
    Tarantool) (#112).

    Simple CI based on GitHub actions (#114).

    Handle everything with `go test` (#115).

    Fix queue tests (#107).

    Make test case consistent with comments (#105).

Other

    Go modules support (#91).

    Update API documentation: comments and examples (#123).
oleg-jukovec added a commit that referenced this issue Jun 2, 2022
Overview

This release adds a number of features. Also it significantly improves
testing, CI and documentation.

Breaking changes

    There are no breaking changes in the release.

New features

    Support UUID type in msgpack (#90).

    queue-utube handling (#85).

    Master discovery (#113).

    SQL support (#62).

Bugfixes

    Reset buffer if its average use size smaller than quater of
    capacity (#95).

Testing

    Coveralls support (#149).

    Reusable testing workflow (integration testing with latest
    Tarantool) (#112).

    Simple CI based on GitHub actions (#114).

    Handle everything with `go test` (#115).

    Fix queue tests (#107).

    Make test case consistent with comments (#105).

Other

    Go modules support (#91).

    Update API documentation: comments and examples (#123).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants