Persistent data in nodeJS

[Zignixx]

Hey there,

i'm playing with the matrix-js-sdk in nodeJS. Works great! But i wonder about the store data. Somehow i'm not sure how to save and load it with nodeJS (persistent). Tried the obe with window.indexedDB thing but it looks like i can't use that in nodeJS.

Anyone got a clue how i can save and load the store in nodeJS?

Thanks!

[hughns]

I've used node-localstorage to provide a compatible storage instance for Node.js environments.

Something like this:

import { LocalStorage } from 'node-localstorage';
import { createClient } from 'matrix-js-sdk/lib';
import { LocalStorageCryptoStore } from 'matrix-js-sdk/lib/crypto/store/localStorage-crypto-store';
import { WebStorageSessionStore } from 'matrix-js-sdk/lib/store/session/webstorage';

const localStorage = new LocalStorage('.storage');
const sessionStore = new WebStorageSessionStore(localStorage);
const cryptoStore = new LocalStorageCryptoStore(localStorage);

const client = createClient({
    sessionStore,
    cryptoStore,
});

[hughns]

Please re-open if more is needed.

[terion-name]

@hughns client options doesn't have sessionStore option. Only cryptoStore and store. And store doesn't store chats data

/**
 * Construct a Matrix Client. Similar to {@link MatrixClient}
 * except that the 'request', 'store' and 'scheduler' dependencies are satisfied.
 * @param opts - The configuration options for this client. These configuration
 * options will be passed directly to {@link MatrixClient}.
 *
 * @returns A new matrix client.
 * @see {@link MatrixClient} for the full list of options for
 * `opts`.
 */
export function createClient(opts: ICreateClientOpts): MatrixClient {
    return new MatrixClient(amendClientOpts(opts));
}

export interface ICreateClientOpts {
    baseUrl: string;

    idBaseUrl?: string;

    /**
     * The data store used for sync data from the homeserver. If not specified,
     * this client will not store any HTTP responses. The `createClient` helper
     * will create a default store if needed.
     */
    store?: Store;

    /**
     * A store to be used for end-to-end crypto session data. If not specified,
     * end-to-end crypto will be disabled. The `createClient` helper will create
     * a default store if needed. Calls the factory supplied to
     * {@link setCryptoStoreFactory} if unspecified; or if no factory has been
     * specified, uses a default implementation (indexeddb in the browser,
     * in-memory otherwise).
     */
    cryptoStore?: CryptoStore;

    /**
     * The scheduler to use. If not
     * specified, this client will not retry requests on failure. This client
     * will supply its own processing function to
     * {@link MatrixScheduler#setProcessFunction}.
     */
    scheduler?: MatrixScheduler;

    /**
     * The function to invoke for HTTP requests.
     * Most supported environments have a global `fetch` registered to which this will fall back.
     */
    fetchFn?: typeof global.fetch;

    userId?: string;

    /**
     * A unique identifier for this device; used for tracking things like crypto
     * keys and access tokens. If not specified, end-to-end encryption will be
     * disabled.
     */
    deviceId?: string;

    accessToken?: string;

    /**
     * Identity server provider to retrieve the user's access token when accessing
     * the identity server. See also https://github.com/vector-im/element-web/issues/10615
     * which seeks to replace the previous approach of manual access tokens params
     * with this callback throughout the SDK.
     */
    identityServer?: IIdentityServerProvider;

    /**
     * The default maximum amount of
     * time to wait before timing out HTTP requests. If not specified, there is no timeout.
     */
    localTimeoutMs?: number;

    /**
     * Set to true to use
     * Authorization header instead of query param to send the access token to the server.
     *
     * Default false.
     */
    useAuthorizationHeader?: boolean;

    /**
     * Set to true to enable
     * improved timeline support, see {@link MatrixClient#getEventTimeline}.
     * It is disabled by default for compatibility with older clients - in particular to
     * maintain support for back-paginating the live timeline after a '/sync'
     * result with a gap.
     */
    timelineSupport?: boolean;

    /**
     * Extra query parameters to append
     * to all requests with this client. Useful for application services which require
     * `?user_id=`.
     */
    queryParams?: Record<string, string>;

    /**
     * Device data exported with
     * "exportDevice" method that must be imported to recreate this device.
     * Should only be useful for devices with end-to-end crypto enabled.
     * If provided, deviceId and userId should **NOT** be provided at the top
     * level (they are present in the exported data).
     */
    deviceToImport?: IExportedDevice;

    /**
     * Key used to pickle olm objects or other sensitive data.
     */
    pickleKey?: string;

    /**
     * Verification methods we should offer to the other side when performing an interactive verification.
     * If unset, we will offer all known methods. Currently these are: showing a QR code, scanning a QR code, and SAS
     * (aka "emojis").
     */
    verificationMethods?: Array<VerificationMethod>;

    /**
     * Whether relaying calls through a TURN server should be forced. Default false.
     */
    forceTURN?: boolean;

    /**
     * Up to this many ICE candidates will be gathered when an incoming call arrives.
     * Gathering does not send data to the caller, but will communicate with the configured TURN
     * server. Default 0.
     */
    iceCandidatePoolSize?: number;

    /**
     * True to advertise support for call transfers to other parties on Matrix calls. Default false.
     */
    supportsCallTransfer?: boolean;

    /**
     * Whether to allow a fallback ICE server should be used for negotiating a
     * WebRTC connection if the homeserver doesn't provide any servers. Defaults to false.
     */
    fallbackICEServerAllowed?: boolean;

    /**
     * If true, to-device signalling for group calls will be encrypted
     * with Olm. Default: true.
     */
    useE2eForGroupCall?: boolean;

    livekitServiceURL?: string;

    /**
     * Crypto callbacks provided by the application
     */
    cryptoCallbacks?: ICryptoCallbacks;

    /**
     * Method to generate room names for empty rooms and rooms names based on membership.
     * Defaults to a built-in English handler with basic pluralisation.
     */
    roomNameGenerator?: (roomId: string, state: RoomNameState) => string | null;

    /**
     * If true, participant can join group call without video and audio this has to be allowed. By default, a local
     * media stream is needed to establish a group call.
     * Default: false.
     */
    isVoipWithNoMediaAllowed?: boolean;

    /**
     * If true, group calls will not establish media connectivity and only create the signaling events,
     * so that livekit media can be used in the application layert (js-sdk contains no livekit code).
     */
    useLivekitForGroupCalls?: boolean;
}

[t3chguy]

client options doesn't have sessionStore option.

It did when he answered.

[terion-name]

And matrix-js-sdk/lib/store has no session/webstorage

There is no way to store data in node environment now as I see and this is ridiculous

[terion-name]

client options doesn't have sessionStore option.

It did when he answered.

so what to do now, if that was removed?

[t3chguy]

The sessionStore stored like 1 value, it was moved into one of the other two stores. The suggested approach is to either write your own store or use the IDB store plus an IDB adapter like https://github.com/bigeasy/indexeddb

[terion-name]

The sessionStore stored like 1 value, it was moved into one of the other two stores. The suggested approach is to either write your own store or use the IDB store plus an IDB adapter like https://github.com/bigeasy/indexeddb

@t3chguy [bigeasy/indexeddb](https://github.com/bigeasy/indexeddb) is not finished, abandoned and npm module is literally empty. There is no proper working persistent implementation of IDB for node.

Why not include any store for node when sdk supports node environment? Absence of persistence layer makes it almost unusable in node

[t3chguy]

Why not include any store for node when sdk supports node environment? Absence of persistence layer makes it almost unusable in node

Contributions welcome, we only have the time to maintain efforts towards the ways we use the SDK, which is for Element Web, which does not run in Node.