Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions database/arango/arango.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
require("dotenv").config();
const { Database } = require("arangojs");
const { sendFetch } = require("../../lib/sendFetch");
const logger = require("./../../lib/log")(__filename);

const arangoModule = {};
let db;

arangoModule.checkIfDatabaseExists = async (name) => {
// returns array of objects containing a '_name' key.
// Ex [{ _name: 'lol' }, { _name: 'cakes' }]
const names = await db.databases();
return names.map((db) => db._name).includes(name);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that this function requests all of the data from the database in JSON form and then search for the name. I think this can cause some inefficiency. Just like postgres and elasticsearch, I assume that there is a way to query the databases with specific name using arango.

Copy link
Collaborator Author

@anthonykhoa anthonykhoa Oct 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes there is, I considered doing it that way, but eventually concluded that it is better to stick to using their arangojs API in order to keep the code more consistent.

Another way I could do this is by making a request to a http REST api that arango provides. This is how deleting an account is done -- I use their http REST API. Here are the pros and cons of using each method to check if a user exists:

Using http REST api:

  • Pros
    1. Queries for only one user, so when the promise is received in const names = await db.databases(), I don't have to use map and includes functions.
  • Cons
    1. Have to make 2 requests to an arango database -- the first time to get a jwt token, the second time to check if a user exists. This is a type of additional logic that arangojs api would not need.

Using arangojs api:

  • Pros
    1. Only 1 request is made to an arango database -- getting a jwt is not necessary because we authenticated ourselves in the function to connect to the arango database.
    2. Keeps the code consistent. The functions to stop and start a connection to an arango database, as well as the methods to create a new user, create a database, and delete a database, are done using their arangojs api. Therefore using arangojs api to check if a user exists is appropriate because it keeps the code more consistent.
  • Cons
    1. Queries for more than one user, so additional logic must be performed to check if a user exists, after receiving the promise (using .map and .includes). Using http REST api, I would not need this type of additional logic.

Efficiency

I believe there is no noticeable difference in efficiency between the two methods, unless learndatabases.dev gets a looot of users. You can see here that on average, it took arango only 1.07 seconds to aggregate a collection of 1,632,803 documents. Since efficiency is not a concern, I think using arangojs is better for now. In the future, if efficiency becomes a concern, then we can switch to using the http REST api that arango provides.

Complexity

I think both methods are equally complex. With http REST api you have to first send a request to get a jwt token, and then use it in the next request to check if a user exists. With arangojs api, you have to filter through the array you receive after using db.databases().

Copy link
Collaborator

@hoiekim hoiekim Oct 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you said it won't be a big problem for now since we don't have a lot of requests but I want our app to be open to a chance to be so.

I have done some research and found something that can be a clue.

  1. https://arangodb.github.io/arangojs/latest/classes/_database_.database.html#query
    This function let us use AQL
  2. https://www.arangodb.com/docs/stable/aql/tutorial-filter.html
    This is how to get document that matches specific condition

Copy link
Collaborator Author

@anthonykhoa anthonykhoa Oct 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have already tried looking into using AQL commands to do user management, but I wasn't able to do so. This is why I used http rest API to delete arango users instead -- because I couldn't use AQL commands to do so. It is possible for us to make a collection of users to query from, but that would make our code more complex than it has to be, and so we should not do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

};

arangoModule.startArangoDB = async () => {
try {
db = new Database({
url: process.env.ARANGO_URL,
databaseName: "_system",
auth: {
username: process.env.ARANGO_USER,
password: process.env.ARANGO_PW,
},
});
logger.info("Connected to Arango Database");
} catch (err) {
logger.error(err);
throw new Error("Error in connecting to Arango Database", err);
}
};

arangoModule.closeArangoDB = () => {
try {
db.close();
logger.info("Successful closing of connection to Arango database");
} catch (err) {
logger.error(err);
throw new Error("Error closing Arango database", err);
}
};

arangoModule.createAccount = async (account) => {
if (!account) return;
const { username, dbPassword } = account;
if (!username || !dbPassword) return;
if (await arangoModule.checkIfDatabaseExists(username)) return;

try {
await db.createDatabase(username, {
users: [{ username, password: dbPassword }],
});
logger.info(`Successfully created Arango user and database ${username}`);
} catch (err) {
logger.error(err);
throw new Error("Error in creating arango database :(", err);
}
};

arangoModule.deleteAccount = async (username) => {
if (!username) return;
if (!(await arangoModule.checkIfDatabaseExists(username))) return;

try {
// grabs JWT token for use in second fetch
const {
jwt,
} = await sendFetch(
`${process.env.ARANGO_URL}_db/_system/_open/auth`,
"post",
{ username: process.env.ARANGO_USER, password: process.env.ARANGO_PW }
);

// uses jwt token to authenticate request to delete user from arango database
await sendFetch(
`${process.env.ARANGO_URL}_db/_system/_api/user/${username}`,
"delete",
null,
`bearer ${jwt}`
);

logger.info(`Successfully deleted Arango user ${username}`);
// deletes database(username and database names are the same for each user)
await db.dropDatabase(username);
logger.info(`Successfully deleted Arango database ${username}`);
} catch (err) {
logger.error(err);
throw new Error(
"Sum ting wong... deletion of arango user has failed.",
err
);
}
};

module.exports = arangoModule;
129 changes: 129 additions & 0 deletions database/arango/arango.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
jest.mock("arangojs");
jest.mock("../../lib/sendFetch");
const { sendFetch } = require("../../lib/sendFetch");
const Arango = require("arangojs");
jest.mock("../../lib/log");
const logGen = require("../../lib/log");
const logger = {
info: jest.fn(),
error: jest.fn(),
};
logGen.mockReturnValue(logger);
jest.mock("dotenv");
Arango.Database = jest.fn().mockReturnValue({
close: jest.fn(),
createDatabase: jest.fn(),
});

const {
startArangoDB,
closeArangoDB,
createAccount,
deleteAccount,
checkIfDatabaseExists,
} = require("./arango");

describe("ArangoDB functions", () => {
beforeEach(() => {
jest.clearAllMocks();
});

test("should call Database function", () => {
startArangoDB();
expect(Arango.Database).toHaveBeenCalledTimes(1);
});

test("should retun false", async () => {
const db = new Arango.Database();
db.databases = jest.fn().mockReturnValue([{ _name: "lol" }]);
const res = await checkIfDatabaseExists("hi");
expect(res).toEqual(false);
});

test("should retun true", async () => {
const db = new Arango.Database();
db.databases = jest.fn().mockReturnValue([{ _name: "lol" }]);
const res = await checkIfDatabaseExists("lol");
expect(res).toEqual(true);
});

test("should call db.close function", () => {
const db = new Arango.Database();
closeArangoDB();
expect(db.close).toHaveBeenCalledTimes(1);
});

test("should call db.close function and log error", () => {
const db = new Arango.Database();
db.close = jest.fn().mockImplementation(() => {
throw new Error();
});
try {
closeArangoDB();
} catch (err) {}
expect(db.close).toHaveBeenCalledTimes(1);
expect(logger.error).toHaveBeenCalledTimes(1);
});

test("should call db.createDatabase function, logger.error when theres an \
error, and do nothing if argument is invalid", async () => {
const db = new Arango.Database();
await createAccount();
expect(db.createDatabase).toHaveBeenCalledTimes(0);
expect(logger.error).toHaveBeenCalledTimes(0);

db.databases = jest.fn().mockReturnValue([{ _name: "lol" }]);
await createAccount({ username: "lol", dbPassword: "hi" });
expect(db.createDatabase).toHaveBeenCalledTimes(0);

db.databases = jest.fn().mockReturnValue([{ _name: "lol" }]);
await createAccount({ username: "", dbPassword: "" });
expect(db.createDatabase).toHaveBeenCalledTimes(0);

try {
db.databases = jest.fn().mockReturnValue([{ _name: "lol" }]);
await createAccount({ username: "hi", dbPassword: "hi" });
expect(db.createDatabase).toHaveBeenCalledTimes(1);

db.createDatabase = jest.fn().mockImplementation(() => {
throw new Error();
});
await createAccount({ username: "hi", dbPassword: "hi" });
} catch (err) {}
expect(logger.error).toHaveBeenCalledTimes(1);
});

test("should send two fetch requests if successful, logger.error when theres \
an error, and do nothing if argument is invalid", async () => {
const db = new Arango.Database();
await deleteAccount();
expect(sendFetch).toHaveBeenCalledTimes(0);
expect(logger.error).toHaveBeenCalledTimes(0);

db.databases = jest.fn().mockReturnValue([{ _name: "lol" }]);
sendFetch.mockReturnValue({ jwt: "lol" });
await deleteAccount("hi");
expect(sendFetch).toHaveBeenCalledTimes(0);

db.dropDatabase = jest.fn().mockReturnValue("lol");
await deleteAccount("lol");
expect(sendFetch).toHaveBeenCalledTimes(2);
expect(db.dropDatabase).toHaveBeenCalledTimes(1);

try {
sendFetch.mockImplementation(() => {
throw new Error();
});
await deleteAccount("lol");
} catch (err) {}
expect(logger.error).toHaveBeenCalledTimes(1);
});

test("should call Database function and FAIL", () => {
Arango.Database.mockImplementation(() => {
throw new Error();
});
startArangoDB();
expect(logger.error).toHaveBeenCalledTimes(1);
});
});
65 changes: 31 additions & 34 deletions database/elasticsearch/elastic.js
Original file line number Diff line number Diff line change
@@ -1,35 +1,20 @@
const fetch = require("node-fetch");
const logger = require("./../../lib/log")(__filename);
require("dotenv").config();

const { sendFetch } = require("../../lib/sendFetch");
const es = {};

const ES_HOST = process.env.ES_HOST || "http://127.0.0.1:9200";
const authorization =
"Basic " +
Buffer.from(`elastic:${process.env.ES_PASSWORD}`).toString("base64");

const sendESRequest = (path, method, body) => {
const options = {
method,
headers: {
Authorization: authorization,
"content-type": "application/json",
},
};
if (body) {
options.body = JSON.stringify(body);
}
return fetch(`${ES_HOST}${path}`, options).then((r) => r.json());
};

es.createAccount = async (account) => {
if (!account.username || !account.dbPassword) {
logger.error("Account data is invalid");
throw new Error("Account data is invalid");
}
const r1 = await sendESRequest(
`/_security/role/${account.username}`,
const r1 = await sendFetch(
`${ES_HOST}/_security/role/${account.username}`,
"POST",
{
indices: [
Expand All @@ -38,21 +23,28 @@ es.createAccount = async (account) => {
privileges: ["all"],
},
],
}
},
authorization
);
const r2 = await sendESRequest(
`/_security/user/${account.username}`,
const r2 = await sendFetch(
`${ES_HOST}/_security/user/${account.username}`,
"POST",
{
email: account.email,
password: account.dbPassword,
roles: [account.username],
}
},
authorization
);
const r3 = await sendFetch(
`${ES_HOST}/${account.username}-example/_doc`,
"POST",
{
message:
"Congratulations! You have created your first index at Elasticsearch!",
},
authorization
);
const r3 = await sendESRequest(`/${account.username}-example/_doc`, "POST", {
message:
"Congratulations! You have created your first index at Elasticsearch!",
});
const err = r1.error || r2.error || r3.error;
if (err) {
logger.error(err);
Expand All @@ -75,15 +67,19 @@ es.deleteAccount = async (account) => {
logger.error("Account data is invalid");
throw new Error("Account data is invalid");
}
const r1 = await sendESRequest(
`/_security/user/${account.username}`,
"DELETE"
const r1 = await sendFetch(
`${ES_HOST}/_security/user/${account.username}`,
"DELETE",
null,
authorization
);
const r2 = await sendESRequest(
`/_security/role/${account.username}`,
"DELETE"
const r2 = await sendFetch(
`${ES_HOST}/_security/role/${account.username}`,
"DELETE",
null,
authorization
);
const r3 = await sendESRequest(`/${account.username}-*`, "DELETE");
const r3 = await sendFetch(`/${account.username}-*`, "DELETE", null, authorization);
const err = r1.error || r2.error;
if (err || !r1.found || !r2.found) {
logger.error("Deleting Elasticsearch user failed");
Expand All @@ -100,7 +96,8 @@ es.checkAccount = async (account) => {
logger.error("Account data is invalid");
throw new Error("Account data is invalid");
}
const r1 = await sendESRequest(`/_security/user/${username}`, "GET");

const r1 = await sendFetch(`${ES_HOST}/_security/user/${username}`, "GET", null, authorization);
logger.info(
`Checking Elasticsearch account for ${username} result:`,
!!r1[username]
Expand Down
20 changes: 0 additions & 20 deletions database/neo4j/neo4j.js

This file was deleted.

24 changes: 0 additions & 24 deletions database/neo4j/neo4j.test.js

This file was deleted.

Loading