Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move to latest gocb version #2379

Closed
tleyden opened this issue Mar 13, 2017 · 2 comments
Closed

Move to latest gocb version #2379

tleyden opened this issue Mar 13, 2017 · 2 comments
Assignees
@tleyden

Comments

@tleyden
Copy link
Contributor

tleyden commented Mar 13, 2017

Factored out of #2377

Need to add gocbcore repo to the manifest
@tleyden tleyden self-assigned this Mar 13, 2017
tleyden pushed a commit that referenced this issue Mar 13, 2017
Fixes #2379 Move to latest gocb version
8c3db43
@tleyden tleyden mentioned this issue Mar 13, 2017
Closed
@tleyden tleyden mentioned this issue Mar 14, 2017
Closed
@tleyden tleyden added the review label Mar 14, 2017
@tleyden tleyden mentioned this issue Mar 17, 2017
Closed
@tleyden
Copy link
Contributor Author

tleyden commented Mar 20, 2017 •
edited

  • Are there other places where it's important to set JSON flag correctly on docs (currently setting everything with no flags -- what about n1ql / views?)
  • If we switch gocb and set binary flag ..
  • If we change to Update(interface{}), we'll need to pass in another param interface{} which will be a pointer to struct
  • With attachment bodies, might be making explicit calls to GetRaw/SetRaw
  • Currently using SetRaw where we want to write JSON (like auth situation), those cases we want to unmarshal into json

@tleyden
Copy link
Contributor Author

tleyden commented Mar 20, 2017 •
edited

Custom transcoder:

  • Encoding: no setting of flags, just write raw bytes
  • Decoding: ignore flags, read raw bytes

Note: only change SG usage to install custom transcoder

Leave our code the way it is
Still need to research flags

tleyden pushed a commit that referenced this issue Mar 22, 2017
Fixes #2379 Move to latest gocb version and fixes #2376 add WriteUpdate
f311e5d 
Pull from couchbase repo, use gopkg.in in layout path

Point to commit instead of tag

Initial pass at #2376 Add WriteUpdate (or equivalent) to gocb

#2376

PR notes

Add hybrid bucket that defaults to gocb, falls back to go-couchbase for unsupported ops

PR feedback

Point to gocb commit w/ workaround

#2376 (comment)

Switch back to couchbase/gocb

Add sgtranscoder

Use sgtranscoder for gocb buckets in sync gateway context.

Fix conversion error by passing []byte instead if empty interface{}

Define unit test URL in single place

Only SetTranscoder if no error opening bucket

Centralize UnitTestUrl() to single place.  Add default drivers for bucket types

Return error if unexpected error, similar to WriteUpdate

Fix failing unit test

./test.sh -run TestGetMissingUser was failing with:

2017-03-20 19:06:30.782937 I | IsDocNotFoundError called with error: Key not found..  type: *gocbcore.memdError
2017-03-20 19:06:30.782947 I | default, return false

	auth_test.go:242
	assert.Equals(t, err, nil)

	expected: <nil>
	     got: &gocbcore.memdError{code:0x1}

Default to GoCBGoCouchbaseHybrid

Default unit tests to walrus

Default to go-couchbase

http://uberjenkins.sc.couchbase.com/job/cen7-sync-gateway-functional-tests-base-cc/308/console

Debug func test failure

#2387 (comment)

Revert "Default to go-couchbase"

This reverts commit d8b2af5.

Use gocbcore errors and remove homebrewed GoCBError type

Debug IsDocNotFoundError

Dereference pointer to fix issue where comparison not working

2017-03-21 14:59:22.362921 I | IsDocNotFoundError called with err: Key not found. with type: *gocbcore.memdError

That didn’t work either .. compare Error()

Fix all gocb error usage

Remove case CouchbaseBucketGoCB check when checking errors

Still seeing 500 errors when running tests

pytest  -s --mode=cc --skip-provisioning -k "test_rolling_ttl_expires[sync_gateway_default_functional_tests]" testsuites/syncgateway/functional/tests/

shows in logs

2017-03-21T15:27:54.568-04:00 HTTP:  #16: GET /db/exp_3?conflicts=true&revs=true  (as sg_user)
2017-03-21T15:27:54.568-04:00 HTTP: #16:     --> 500 Internal error: Key not found.  (1.4 ms)

Switch back to go-couchbase to compare

pytest  -s --mode=cc --skip-provisioning -k "test_db_offline_tap_loss_sanity[bucket_online_offline/bucket_online_offline_default_dcp-100]" testsuites/syncgateway/functional/tests/ fails with GoCBGoCouchbaseHybrid

Debugging failing bucket offline test

Default gocb

Debug Incr issue after call w/ Adam

Outer fix: Don't bother trying to lookup LastSequence() if offline

Temp disable outer fix

Add debug.PrintStack()

Re-enable outer fix

Revert "Add debug.PrintStack()"

This reverts commit 3701ca5.

Revert "Debug Incr issue after call w/ Adam"

This reverts commit 71c3caa.

Remove unneeded debug logging
tleyden pushed a commit that referenced this issue Mar 22, 2017
Fixes #2379 Move to latest gocb version and fixes #2376 add WriteUpdate
7dd5458 
Pull from couchbase repo, use gopkg.in in layout path

Point to commit instead of tag

Initial pass at #2376 Add WriteUpdate (or equivalent) to gocb

#2376

PR notes

Add hybrid bucket that defaults to gocb, falls back to go-couchbase for unsupported ops

PR feedback

Point to gocb commit w/ workaround

#2376 (comment)

Switch back to couchbase/gocb

Add sgtranscoder

Use sgtranscoder for gocb buckets in sync gateway context.

Fix conversion error by passing []byte instead if empty interface{}

Define unit test URL in single place

Only SetTranscoder if no error opening bucket

Centralize UnitTestUrl() to single place.  Add default drivers for bucket types

Return error if unexpected error, similar to WriteUpdate

Fix failing unit test

./test.sh -run TestGetMissingUser was failing with:

2017-03-20 19:06:30.782937 I | IsDocNotFoundError called with error: Key not found..  type: *gocbcore.memdError
2017-03-20 19:06:30.782947 I | default, return false

	auth_test.go:242
	assert.Equals(t, err, nil)

	expected: <nil>
	     got: &gocbcore.memdError{code:0x1}

Default to GoCBGoCouchbaseHybrid

Default unit tests to walrus

Default to go-couchbase

http://uberjenkins.sc.couchbase.com/job/cen7-sync-gateway-functional-tests-base-cc/308/console

Debug func test failure

#2387 (comment)

Revert "Default to go-couchbase"

This reverts commit d8b2af5.

Use gocbcore errors and remove homebrewed GoCBError type

Debug IsDocNotFoundError

Dereference pointer to fix issue where comparison not working

2017-03-21 14:59:22.362921 I | IsDocNotFoundError called with err: Key not found. with type: *gocbcore.memdError

That didn’t work either .. compare Error()

Fix all gocb error usage

Remove case CouchbaseBucketGoCB check when checking errors

Still seeing 500 errors when running tests

pytest  -s --mode=cc --skip-provisioning -k "test_rolling_ttl_expires[sync_gateway_default_functional_tests]" testsuites/syncgateway/functional/tests/

shows in logs

2017-03-21T15:27:54.568-04:00 HTTP:  #16: GET /db/exp_3?conflicts=true&revs=true  (as sg_user)
2017-03-21T15:27:54.568-04:00 HTTP: #16:     --> 500 Internal error: Key not found.  (1.4 ms)

Switch back to go-couchbase to compare

pytest  -s --mode=cc --skip-provisioning -k "test_db_offline_tap_loss_sanity[bucket_online_offline/bucket_online_offline_default_dcp-100]" testsuites/syncgateway/functional/tests/ fails with GoCBGoCouchbaseHybrid

Debugging failing bucket offline test

Default gocb

Debug Incr issue after call w/ Adam

Outer fix: Don't bother trying to lookup LastSequence() if offline

Temp disable outer fix

Add debug.PrintStack()

Re-enable outer fix

Revert "Add debug.PrintStack()"

This reverts commit 3701ca5.

Revert "Debug Incr issue after call w/ Adam"

This reverts commit 71c3caa.

Remove unneeded debug logging
tleyden pushed a commit that referenced this issue Mar 22, 2017
Fixes #2379 Move to latest gocb version and fixes #2376 add WriteUpdate
96b3bb0 
Pull from couchbase repo, use gopkg.in in layout path

Point to commit instead of tag

Initial pass at #2376 Add WriteUpdate (or equivalent) to gocb

#2376

PR notes

Add hybrid bucket that defaults to gocb, falls back to go-couchbase for unsupported ops

PR feedback

Point to gocb commit w/ workaround

#2376 (comment)

Switch back to couchbase/gocb

Add sgtranscoder

Use sgtranscoder for gocb buckets in sync gateway context.

Fix conversion error by passing []byte instead if empty interface{}

Define unit test URL in single place

Only SetTranscoder if no error opening bucket

Centralize UnitTestUrl() to single place.  Add default drivers for bucket types

Return error if unexpected error, similar to WriteUpdate

Fix failing unit test

./test.sh -run TestGetMissingUser was failing with:

2017-03-20 19:06:30.782937 I | IsDocNotFoundError called with error: Key not found..  type: *gocbcore.memdError
2017-03-20 19:06:30.782947 I | default, return false

	auth_test.go:242
	assert.Equals(t, err, nil)

	expected: <nil>
	     got: &gocbcore.memdError{code:0x1}

Default to GoCBGoCouchbaseHybrid

Default unit tests to walrus

Default to go-couchbase

http://uberjenkins.sc.couchbase.com/job/cen7-sync-gateway-functional-tests-base-cc/308/console

Debug func test failure

#2387 (comment)

Revert "Default to go-couchbase"

This reverts commit d8b2af5.

Use gocbcore errors and remove homebrewed GoCBError type

Debug IsDocNotFoundError

Dereference pointer to fix issue where comparison not working

2017-03-21 14:59:22.362921 I | IsDocNotFoundError called with err: Key not found. with type: *gocbcore.memdError

That didn’t work either .. compare Error()

Fix all gocb error usage

Remove case CouchbaseBucketGoCB check when checking errors

Still seeing 500 errors when running tests

pytest  -s --mode=cc --skip-provisioning -k "test_rolling_ttl_expires[sync_gateway_default_functional_tests]" testsuites/syncgateway/functional/tests/

shows in logs

2017-03-21T15:27:54.568-04:00 HTTP:  #16: GET /db/exp_3?conflicts=true&revs=true  (as sg_user)
2017-03-21T15:27:54.568-04:00 HTTP: #16:     --> 500 Internal error: Key not found.  (1.4 ms)

Switch back to go-couchbase to compare

pytest  -s --mode=cc --skip-provisioning -k "test_db_offline_tap_loss_sanity[bucket_online_offline/bucket_online_offline_default_dcp-100]" testsuites/syncgateway/functional/tests/ fails with GoCBGoCouchbaseHybrid

Debugging failing bucket offline test

Default gocb

Debug Incr issue after call w/ Adam

Outer fix: Don't bother trying to lookup LastSequence() if offline

Temp disable outer fix

Add debug.PrintStack()

Re-enable outer fix

Revert "Add debug.PrintStack()"

This reverts commit 3701ca5.

Revert "Debug Incr issue after call w/ Adam"

This reverts commit 71c3caa.

Remove unneeded debug logging
tleyden pushed a commit that referenced this issue Mar 22, 2017
Fixes #2379 Move to latest gocb version and fixes #2376 add WriteUpdate
d6d4a39
@tleyden tleyden mentioned this issue Mar 24, 2017
Merged
adamcfraser pushed a commit that referenced this issue Mar 28, 2017
@tleyden @adamcfraser
Fixes #2379 Move to latest gocb version and fixes #2376 add WriteUpda…
5bb66ef 
…te (#2416)

* Fixes #2379 Move to latest gocb version and fixes #2376 add WriteUpdate

* Deal with errors related to type assertions

Trying to get around couchbaselabs/mobile-testkit#1072

Still seeing errors on startup:

https://gist.github.com/tleyden/c0b57263234efada16724d424cbcd1ac

* Point to SG Accel branch

* Remove type assertions and call base.GetGoCBBucketFromBaseBucket

* Point to latest SG accel

* Only set SGTranscoder for GoCB hybrid buckets

Due to #2416 (comment)

* Point to sg accel master

* PR Feedback IsCasMismatch

* PR Feedback: Call GetRaw instead of Get

Particularly with the (temporary) fix that's mentioned in GetRaw?

* PR feedback: add comments re numNodesReplicateTo / numNodesPersistTo

* PR Feedback: Change to plain for {} until a cleaner approach can be found

* PR Feedback: found an issue with error comparison

I just noticed an issue while looking into this.

Currently we're calling:

if err != nil && err != gocb.ErrKeyNotFound {
but in my testing I found that this comparison doesn't work, since gocb.ErrKeyNotFound is actually a pointer and it doesn't seem to provide any guarantees that it will be returning a pointer to the same object. I didn't actually look at the code, but from my testing that comparison approach wasn't reliable, which is why I switched to the string comparison approach.

I will push up a fix that uses the string comparison.

* PR Feedback: make HTTP errors more accurate

* PR Feedback: remove logging / todos

* PR Feedback: explicitly set cas = 0 if key not found err
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tleyden tleyden

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tleyden @adamcfraser