document using memdown

[eklem]

Test first

[eklem]

Added stuff from tests, but need to make a working example to make it accurate.

[eklem]

Make an extra version of the demo that uses memdown. Got it working over at designing-with-search-data.

[eklem]

Have something working over at designing-with-search-data

[fergiemcdowall]

This is now (briefly) documented here -> https://github.com/fergiemcdowall/search-index/blob/master/docs/FAQ.md#can-i-use-another-backend-like-mysql-or-redis

Does it make sense or is it too cryptic?

[eklem]

I think it makes sense. Will try

[tonprince]

Is there any working sample how to initialize a redis store with search-index? I am trying to connect to a Redis server on Upstash.

[tonprince]

I could connect to the Redis instance:

let redis = new Redis("redis....:35034");
let db = redisdown("tripstorekrabi")
db.open({ redis }, async (err) => {
  if (err) {
    rej(err);
  } else {
    let { PUT } = await si({ db });
    await PUT([{ "_id": "111122", "test": "hello world" }]);
  }
});

but always gets the following error:

TypeError: this._end is not a function
    at Iterator.AbstractIterator.end (/Users/tonprincepro/Dropbox/Tripstore/Website/workspace/com.tripstorekrabi/hero/api/booking/node_modules/abstract-leveldown/abstract-iterator.js:67:8)
    at Iterator._end (/Users/tonprincepro/Dropbox/Tripstore/Website/workspace/com.tripstorekrabi/hero/api/booking/node_modules/encoding-down/index.js:150:11)
    at Iterator.AbstractIterator.end (/Users/tonprincepro/Dropbox/Tripstore/Website/workspace/com.tripstorekrabi/hero/api/booking/node_modules/abstract-leveldown/abstract-iterator.js:67:8)
    at ReadStream._destroy (/Users/tonprincepro/Dropbox/Tripstore/Website/workspace/com.tripstorekrabi/hero/api/booking/node_modules/level-iterator-stream/index.js:40:18)
    at ReadStream.destroy (/Users/tonprincepro/Dropbox/Tripstore/Website/workspace/com.tripstorekrabi/hero/api/booking/node_modules/readable-stream/lib/internal/streams/destroy.js:35:8)
    at ReadStream.emit (node:events:406:35)
    at endReadableNT (/Users/tonprincepro/Dropbox/Tripstore/Website/workspace/com.tripstorekrabi/hero/api/booking/node_modules/readable-stream/lib/_stream_readable.js:1094:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

[fergiemcdowall]

Can you get any reads and writes to work with levelup + redisdown? (cutting out search-index and using a redis backed level instance as described here -> https://github.com/Level/levelup)

[tonprince]

I did not try that yet. I would like to run search-index as a serverless function. Currently I have deployed it as a Google Cloud function. It works fine, except that in case the function instance is killed the data in tmp dir is lost.

Do you have any suggestions how to persist the data in search-index efficiently in a serverless function? Would Redis suitable?

[tonprince]

Can you get any reads and writes to work with levelup + redisdown? (cutting out search-index and using a redis backed level instance as described here -> https://github.com/Level/levelup)

It works with levelup + redisdown.

import Redis from "ioredis";
import levelup from "levelup";
import redisdown from "redisdown";

let redis = new Redis(":PASSWORD@SERVERNAME:35034");
let db = redisdown("tripstorekrabi")

db.open({ redis }, async (err) => {
  if (err) {
    rej(err);
  } else {
    let leveldb = levelup(db)
    leveldb.put('name', 'levelup', (err) => {
      if (err) {
        return console.log('Ooops!', err);
      }

      leveldb.get('name', (err, value) => {
        if (err) {
          return console.log('Ooops!', err);
        }

        console.log('name=' + value);
      })
    })
  }
});

Console log: name=levelup

[fergiemcdowall]

Hmm- interesting. I will keep this open as a bug for just now, and look into it when I have time.

@tonprince Please keep us posted if you find a solution!

[tonprince]

Finally I moved the search-index to compute engine instance, with backup and restore logic from/to Cloud storage. It works like a charm. It uses the standard write logic to file system.

[fergiemcdowall]

Great to hear @tonprince ! 🎉🚀