Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi get, pluggable hashing and some code cleanup (#1) #145

Closed
wants to merge 91 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
8b67064
Multi get, pluggable hashing and some code cleanup (#1)
blackmad Mar 26, 2021
ba33762
remove comment to self
blackmad Mar 26, 2021
c57620e
Merge remote-tracking branch 'upstream/master'
blackmad Mar 26, 2021
7653985
.d.ts
justjake Mar 27, 2021
9e2520e
confirmed these were actually Buffers
justjake Mar 27, 2021
9d925d2
fix file name
justjake Mar 29, 2021
7b57c8b
add version command (#5)
blackmad Mar 29, 2021
2fc73ae
fix comment to accurately describe as buffer
justjake Mar 29, 2021
65e399f
Merge pull request #4 from makenotion/jake--types
justjake Mar 29, 2021
9a88a2b
version and versionAll (#6)
blackmad Mar 29, 2021
d6d527c
no-op lint fix commit
blackmad Mar 30, 2021
ef2b165
move sources into src/
justjake Apr 9, 2021
ceaf4e7
tsconfig
justjake Apr 9, 2021
f1cfc0e
build with tsc
justjake Apr 9, 2021
1119839
rename .js to .ts
justjake Apr 9, 2021
35a4789
constants: convert to module
justjake Apr 9, 2021
fc911c6
tsify header.ts
justjake Apr 9, 2021
998ae68
protocol.ts
justjake Apr 9, 2021
e553a66
utils.ts
justjake Apr 9, 2021
efc52d6
serializer.ts
justjake Apr 9, 2021
67db715
server.ts
justjake Apr 9, 2021
415b35a
WIP
justjake Apr 10, 2021
9f58c61
WIP2
justjake Apr 10, 2021
f056af4
memjs.ts
justjake Apr 10, 2021
e15050a
fix exports
justjake Apr 10, 2021
b6a66fa
revert changes to tests
justjake Apr 10, 2021
a38b281
fix tests
justjake Apr 10, 2021
889ee89
WIP TSify more
blackmad Apr 11, 2021
42ad49d
TSify tests (poorly?)
blackmad Apr 12, 2021
0f7a3b4
all tests pass!
blackmad Apr 12, 2021
f0051bd
better tap test type
blackmad Apr 12, 2021
95ebb58
Update src/memjs/header.ts
justjake Apr 12, 2021
631e573
Update src/memjs/memjs.ts
justjake Apr 12, 2021
5246465
Update src/memjs/memjs.ts
justjake Apr 12, 2021
128be58
Update src/memjs/memjs.ts
justjake Apr 12, 2021
c54fec6
Update src/memjs/memjs.ts
justjake Apr 12, 2021
07cb9a8
ci
justjake Apr 12, 2021
0cd16db
Merge pull request #8 from makenotion/jake--typescript
justjake Apr 12, 2021
ed0b4e1
return cas from get + getMulti
justjake Apr 12, 2021
db2deee
move test into src
justjake Apr 12, 2021
b4459f4
mess with imports
justjake Apr 12, 2021
9456ef9
bench seems busted...
justjake Apr 12, 2021
d55fe86
ci on every branch pr?
justjake Apr 12, 2021
cdd75d9
test includes tsc now
justjake Apr 12, 2021
e1cef37
pull request target??
justjake Apr 12, 2021
24164aa
fix getmulti result
justjake Apr 12, 2021
f6ae576
newer versions of makeRequestBuffer
justjake Apr 12, 2021
9a1cd1e
set: pass through CAS token
justjake Apr 12, 2021
0f88a3a
remove no-op fill(0), since Buffer.alloc returns zeroed buffer these …
justjake Apr 12, 2021
871fddd
verify cas is passed through to server
justjake Apr 12, 2021
6856969
refactor to remove repeated error handling boilerplate
justjake Apr 12, 2021
524856b
correctly handle CAS failure
justjake Apr 13, 2021
471d42d
cli tester thingy
justjake Apr 13, 2021
b7dbe88
make cli self-executing
justjake Apr 13, 2021
477a74e
remove ambiguous getmulti case
justjake Apr 13, 2021
1fc7181
remove unused logger
justjake Apr 13, 2021
258c3a9
more commento
justjake Apr 13, 2021
ba50aeb
test set w/ cas
justjake Apr 13, 2021
efce2eb
Merge pull request #9 from makenotion/jake--cas
justjake Apr 13, 2021
87cb9c7
WIP: remove callbacks from public methods (#10)
justjake Apr 13, 2021
e7a523d
Check in and productionize typescript build (#11)
justjake Apr 13, 2021
c3432a4
Make getMulti return type non-null (#12)
justjake Apr 13, 2021
2f3670a
error on some weird edge cases we are having trouble debugging (#13)
blackmad May 18, 2021
799b44b
Fix incr seq timing in get milti (#14)
blackmad May 20, 2021
fdb2efe
added callbacks to versionAll for better errors
arturoolvera Feb 6, 2023
9a781ea
readme, gitignore, linting
arturoolvera Feb 6, 2023
59493e2
naming and notation
arturoolvera Feb 6, 2023
4a0e8e0
naming
arturoolvera Feb 6, 2023
bbf0e84
comments
arturoolvera Feb 9, 2023
2a63687
didnt compile
arturoolvera Feb 9, 2023
39cf942
Merge pull request #15 from makenotion/arturo--add-versionAll-callbacks
arturoolvera Feb 10, 2023
2a19d80
Hanlde server restart
leon-root Apr 7, 2023
5341559
nit, remove white space
leon-root Apr 7, 2023
2ede258
Merge pull request #16 from makenotion/leon/socket.close
leon-root May 17, 2023
9d21572
Add getMultiWithErrors
schleyfox Sep 27, 2023
c334faa
Test getMultiWithErrors
schleyfox Sep 29, 2023
7010678
Merge pull request #17 from makenotion/benjamin-getmultiwitherrors
schleyfox Sep 29, 2023
bade443
Error out waiting clients on close, handle too many connections
schleyfox Oct 3, 2023
cb1693c
Comment error string
schleyfox Oct 3, 2023
5da51e6
Merge pull request #18 from makenotion/benjamin--handle-close-and-too…
schleyfox Oct 3, 2023
99b24c4
Reset responseBuffer on error, close, and new connections
schleyfox Oct 3, 2023
dff5481
Merge pull request #19 from makenotion/benjamin--reset-buffers-on-rec…
schleyfox Oct 3, 2023
19c9bb3
bump version to clear circleCI cache
leon-root Oct 3, 2023
c84319e
Merge pull request #20 from makenotion/bump.version
leon-root Oct 3, 2023
a8a1eca
Add versionAllWithErrors
schleyfox Nov 15, 2023
4f93728
add types
schleyfox Nov 15, 2023
aab7778
Add tests
schleyfox Nov 16, 2023
02dabaa
bump version
schleyfox Nov 16, 2023
9137a13
Merge pull request #21 from makenotion/benjamin--versionallwitherrors
schleyfox Nov 16, 2023
7a521a2
Use sock.destroy() instead of end() on timeout, 1.2.4-NOTION.1
schleyfox Feb 14, 2024
d7ce226
Merge pull request #22 from makenotion/benjamin--destroy-socket-on-ti…
schleyfox Feb 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 7 additions & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
"semi": [
2,
"always"
]
],
"no-unused-vars": 0
},
"globals": {
"Promise": "readonly"
Expand All @@ -24,5 +25,10 @@
"node": true,
"browser": true
},
"parserOptions": {
"es6": true,
"ecmaVersion": 6,
"sourceType": "module"
},
"extends": "eslint:recommended"
}
25 changes: 25 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Node.js CI

on:
push:
branches: [ $default-branch ]
pull_request:
branches: [ $default-branch, '*' ]

jobs:
build:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [12.x]

steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm test
- run: bin/ensure-clean.sh
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ pids
logs
results
build/
lib/test

node_modules
npm-debug.log
.nyc_output/

.vscode
.DS_Store*
41 changes: 39 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,43 @@ MemJS
[![Build Status](https://secure.travis-ci.org/alevy/memjs.png)](http://travis-ci.org/alevy/memjs?branch=master)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/alevy/memjs?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)

---

Notion fork of MemJS

## Making Changes

see [this](https://dev.notion.so/notion/Developing-dependencies-locally-6705f66af9a24944ac875c44234e68f1) doc

**Linking**

Clone `memjs`, run `npm install`, link `memjs` to local Notion:

```
cd ~/memjs
npm link

cd ~/notion-next
npm link memjs
```

**Developing**

Run `npx tsc --watch` in the `memjs` directory.

**Unlinking**

```
cd ~/notion-next
npm unlink --no-save package-js

cd ~/package-js/
npm unlink
```

---


MemJS is a pure Node.js client library for using memcache, in particular, the
[MemCachier](http://memcachier.com/) service. It
uses the binary protocol and support SASL authentication.
Expand Down Expand Up @@ -65,8 +102,8 @@ Environment variables are only used as a fallback for explicit parameters.

You can start using MemJS immediately from the node console:

$ var memjs = require('memjs')
$ var client = memjs.Client.create()
$ const memjs = require('memjs')
$ const client = memjs.Client.create()
$ client.get('hello', function(err, val) { console.log(val); })

If callbacks are not specified, the command calls return promises.
Expand Down
74 changes: 37 additions & 37 deletions bench/memjs.js
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
var memjs = require("memjs");
var header = require('header');
var b = require("benchmark");
const memjs = require('memjs');
const header = require('header');
const b = require('benchmark');

function makeString(n) {
var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
var text = "";
var i;
const possible = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
let text = '';
let i;

for(i=0; i < n; i++ ) {
text += possible.charAt(Math.floor(Math.random() * possible.length));
Expand All @@ -14,29 +14,29 @@ function makeString(n) {
return text;
}

var x = (function() {
var suite = new b.Suite();
let x = (function() {
const suite = new b.Suite();

var headerBuf = Buffer.from([0x81, 1, 7, 0, 4, 0, 0, 1, 0, 0, 0, 9, 0, 0, 0, 0, 0x0a, 0, 0, 0, 0, 0, 0, 0]);
var parsedHeader = header.fromBuffer(headerBuf);
const headerBuf = Buffer.from([0x81, 1, 7, 0, 4, 0, 0, 1, 0, 0, 0, 9, 0, 0, 0, 0, 0x0a, 0, 0, 0, 0, 0, 0, 0]);
const parsedHeader = header.fromBuffer(headerBuf);

suite.add('Header#fromBuffer', function() {
header.fromBuffer(headerBuf);
})
.add('Header#toBuffer', function() {
header.toBuffer(parsedHeader);
})
.add('Header#toBuffer', function() {
header.toBuffer(parsedHeader);
})
// add listeners
.on('cycle', function(event) {
console.log(String(event.target));
})
.on('cycle', function(event) {
console.log(String(event.target));
})
// run async
.run({ 'async': true });
.run({ 'async': true });
}());

x = (function() {
var suite = new b.Suite();
var responseHeader = {
const suite = new b.Suite();
const responseHeader = {
magic: 0x81,
opcode: 1,
keyLength: 15,
Expand All @@ -47,51 +47,51 @@ x = (function() {
opaque: 0,
cas: Buffer.from([0x0a, 0, 0, 0, 0, 0, 0, 0])
};
var buf = Buffer.alloc(24 + 15 + 1024 * 10);
const buf = Buffer.alloc(24 + 15 + 1024 * 10);
header.toBuffer(responseHeader).copy(buf);
buf.write(makeString(55));

var server = new memjs.Server();
const server = new memjs.Server();

var dummyFunc = function(arg) {
const dummyFunc = function(arg) {
server.onResponse(dummyFunc);
return arg;
};

var i;
let i;
for (i = 0; i < 10; i++) {
server.onResponse(dummyFunc);
}

suite.add('Server#respond', function() {
server.respond('helloworldthisisatesttohowmuchyouareaversetocopyingoverthestargatecommisionandromanticizingaboutmylifelongpassionforlearningandyieldingtoyourdesires');
})
.add('Server#responseHandler', function() {
server.responseHandler(buf);
})
.add('Server#responseHandler', function() {
server.responseHandler(buf);
})
// add listeners
.on('cycle', function(event) {
console.log(String(event.target));
})
.on('cycle', function(event) {
console.log(String(event.target));
})
// run async
.run({ 'async': true });
.run({ 'async': true });
}());

x = (function() {
var suite = new b.Suite();
var client = memjs.Client.create();
const suite = new b.Suite();
const client = memjs.Client.create();

suite.cycles = 0;
suite.add('Client#get', function() {
client.get("hello", function(/* err, val */) {
client.get('hello', function(/* err, val */) {
suite.cycles++;
});
})
// add listeners
.on('cycle', function(event) {
console.log(String(event.target) + " " + suite.cycles);
});
client.set("hello", makeString(10240), function(/* err, val */) {
.on('cycle', function(event) {
console.log(String(event.target) + ' ' + suite.cycles);
});
client.set('hello', makeString(10240), function(/* err, val */) {
// run async
suite.run({ 'async': true });
});
Expand Down
48 changes: 24 additions & 24 deletions bench/timers.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,40 +6,40 @@
* Check how fast various timers are in node.
*/

var Benchmark = require('benchmark');
var Microtime = require('microtime');
const Benchmark = require('benchmark');
const Microtime = require('microtime');

var suite = new Benchmark.Suite();
const suite = new Benchmark.Suite();

// add tests
suite.add('Date.now()', function() {
// system time, not-monotonic, ms
Date.now();
})
.add('Microtime.now()', function() {
.add('Microtime.now()', function() {
// system time, not-monotonic, us (POSIX: gettimeofday)
Microtime.now();
})
.add('process.hrtime()', function() {
Microtime.now();
})
.add('process.hrtime()', function() {
// monotonic, ns (returns: [seconds, nanoseconds])
process.hrtime();
})
.add('process.hrtime() ms-round', function() {
process.hrtime();
})
.add('process.hrtime() ms-round', function() {
// monotonic, ns (returns: [seconds, nanoseconds])
var time = process.hrtime();
return (time[0] * 1000) + Math.round(time[1] / 1000000);
})
.add('process.hrtime() ms-floor', function() {
const time = process.hrtime();
return (time[0] * 1000) + Math.round(time[1] / 1000000);
})
.add('process.hrtime() ms-floor', function() {
// monotonic, ns (returns: [seconds, nanoseconds])
var time = process.hrtime();
return (time[0] * 1000) + Math.floor(time[1] / 1000000);
})
const time = process.hrtime();
return (time[0] * 1000) + Math.floor(time[1] / 1000000);
})
// add listeners
.on('cycle', function(event) {
console.log(String(event.target));
})
.on('complete', function() {
console.log('Fastest is ' + this.filter('fastest').map('name'));
})
.on('cycle', function(event) {
console.log(String(event.target));
})
.on('complete', function() {
console.log('Fastest is ' + this.filter('fastest').map('name'));
})
// run async
.run({ 'async': true });
.run({ 'async': true });
10 changes: 10 additions & 0 deletions bin/ensure-clean.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash

if [[ -z "$(git status --porcelain)" ]]; then
echo "Git looks clean, you're good to go" > /dev/stderr
else
git status
git diff
echo "ERROR: Git looks dirty, see above. Check in all changes." > /dev/stderr
exit 2
fi
53 changes: 53 additions & 0 deletions cli.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#! node --inspect
const MemJS = require(".");
const client = MemJS.Client.create();

let i = 0;

function next(promise) {
const id = i++;
const varname = `r${id}`;

const assign = (r) => {
global[varname] = r;
global["r"] = r;
};

if (promise.then) {
promise.then(
(r) => {
console.log(`r${id} = resolved`, r);
assign(r);
},
(r) => {
console.log(`r${id} = rejected`, r);
assign(r);
}
);
} else {
console.log(`r${id} =`, promise);
assign(promise);
}
}

Object.assign(global, {
MemJS,
client,
next,
p: next,
});

console.log(`
Run as node --inspect cli.js,
then open Chrome inspector up and click the little node icon.

MemJS: memjs library export
client: a client connected to localhost:11211
p: a function to resolve promises

Example:
p(client.get('foo'))
r1.cas.toString()
`);

setInterval(() => {}, 5000);