Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding TypeDB #125350

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
58af248
typedb: init at 2.1.1
haskie-lambda May 30, 2021
f4bb177
maintainers: add haskie
haskie-lambda May 30, 2021
1ce0419
typedb: add typedb to all-packages and completed install/patch procedure
haskie-lambda May 30, 2021
68151ee
Merge branch 'NixOS:master' into master
haskie-lambda May 30, 2021
985789b
typedb: added test
haskie-lambda May 30, 2021
5f86c9e
typedb: moved tests to installCheckPhase as discussed in https://www.…
haskie-lambda May 31, 2021
4d786b4
Merge branch 'NixOS:master' into master
haskie-lambda May 31, 2021
05b06cf
typedb: using fetchurl instead of fetchTarball now
haskie-lambda Jun 2, 2021
a93b019
typedb: merge remote-tracking branch 'upstream/master'
haskie-lambda Jun 2, 2021
4cab489
typedb: changed implementation to suggestions by roberth
haskie-lambda Jun 3, 2021
46058a2
typedb: Merge remote-tracking branch 'upstream/master'
haskie-lambda Jun 3, 2021
4e28da1
implemented fix suggested by roberth
haskie-lambda Jun 7, 2021
ddd56f8
typedb: merge remote-tracking branch 'upstream/master'
haskie-lambda Jun 7, 2021
b4d3ebb
typedb: merge remote-tracking branch 'upstream/master'
haskie-lambda Jun 10, 2021
9e9146a
typedb: added wrapper script again to supply options as variables
haskie-lambda Jun 10, 2021
16a0318
typedb: fixed install- and installCheck-phase
haskie-lambda Jun 10, 2021
6de79fa
typedb: merge remote-tracking branch 'upstream/master'
haskie-lambda Jun 10, 2021
092a461
typedb: removed unneccessary file
haskie-lambda Jun 10, 2021
c25bc44
typedb: reformatted using `nixpkgs-fmt`
haskie-lambda Jun 10, 2021
80a7680
typedb: removed trailing whitespace
haskie-lambda Jun 10, 2021
6a0830e
typedb: added passthrough tests
haskie-lambda Jun 10, 2021
a774de5
typedb: updated hash of darwin sources
haskie-lambda Jun 11, 2021
6953e98
typedb: merge remote-tracking branch 'upstream/master'
haskie-lambda Jun 11, 2021
4fa8793
typedb: Merge branch 'NixOS:master' into master
haskie-lambda Jul 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions maintainers/maintainer-list.nix
Expand Up @@ -3991,6 +3991,12 @@
githubId = 54728477;
name = "Happy River";
};
haskie = {
email = "faebl.taylor@pm.me";
github = "haskie-lambda";
githubId = 23012490;
name = "Fabian Schneider";
};
haslersn = {
email = "haslersn@fius.informatik.uni-stuttgart.de";
github = "haslersn";
Expand Down
130 changes: 130 additions & 0 deletions pkgs/servers/typedb/default.nix
@@ -0,0 +1,130 @@
{ stdenv, lib, openjdk, which, system, fetchzip, fetchurl, server-data ? "~/.typedb_home", server-port ? "1729", server-logs ? "~/.typedb_home/logs" }:
let
typedbVersion = "2.1.1";
systems = {
x86_64-linux = {
src = fetchurl {
url = "https://github.com/vaticle/typedb/releases/download/2.1.1/typedb-all-linux-2.1.1.tar.gz";
sha256 = "00wvkiahmrys32ckp6r59bbrzfd7jzsphniyjdj4p9bh7k339gn2";
};
dir = "typedb-all-linux-${typedbVersion}";
};
x86_64-darwin = {
src = fetchzip {
url = "https://github.com/vaticle/typedb/releases/download/2.1.1/typedb-all-mac-2.1.1.zip";
sha256 = "16hlfy6kh2rnvcralz206q13mghb0rv8wazpg6q3h324p5rdys54";
};
dir = "typedb-all-mac-${typedbVersion}";
};
x86_64-windows = {
src = fetchzip {
url = "https://github.com/vaticle/typedb/releases/download/2.1.1/typedb-all-windows-2.1.1.zip";
sha256 = "0vd66gfshkg697z07nhy957mwqzlli4r4pmn67hx58n9mkg024kq";
};
dir = "typedb-all-windows-${typedbVersion}";
};
};

currentSystem = systems.${system} or (throw "unsupported system ${system}");
typedbDir = currentSystem.dir;
srcFolder = currentSystem.src;
javaPatch = ''
20c20
< JAVA_BIN=java
---
> JAVA_BIN=${openjdk}/bin/java
32c32
< which "$'' + "{JAVA_BIN}\" > /dev/null\n" +
"---\n" +
"> ${which}/bin/which \"$" + "{JAVA_BIN}\" > /dev/null";

typedb-properties-patch = ''
19c19
< server.data=server/data/
---
> server.data=${server-data}
22c22
< server.logs=server/logs/
---
> server.logs=${server-logs}
24c24
< server.port=1729
---
> server.port=${server-port}
Comment on lines +43 to +55
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it's already much better, this is still a bit awkward. Ideally changes in configuration would not require a rebuild.
If we're lucky, we might still be able to use an environment variable instead of hardcoding it into the store path, although that's not the default property file behavior.

Somewhat relevant discussion https://stackoverflow.com/questions/2263929/regarding-application-properties-file-and-environment-variable. If it's built on apache commons, we're lucky.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to specify properties or property files at runtime?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to the typedb team not;
only at startup as a cli option or in the config file;

I decided to add a wrapper script again that makes it possible to change the three configuration options via environment variables.
at least until I am able to change the implementation of the property file reader in the typedb.

I am still debugging it, but at least it builds and the tests run fine now;
there are still some issues though...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only at startup as a cli option or in the config file;

The config file is not really a config file if it's packaged immutably with the software, so cli options are probably the easiest option. Do you plan to write a NixOS service as well?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't think about this before;
but I could actually do that, have to read up on that though ^^

'';
typedb-wrapper-patch = ''
3c3
< TYPEDB_SERVER_DATA="__SERVER_DATA__"
---
> TYPEDB_SERVER_DATA="${server-data}"
8c8
< TYPEDB_SERVER_LOGS="__SERVER_LOGS__"
---
> TYPEDB_SERVER_LOGS="${server-logs}"
13c13
< TYPEDB_SERVER_PORT="__SERVER_PORT__"
---
> TYPEDB_SERVER_PORT="${server-port}"
'';
typedbWrapper = ./typedbWrapper.sh;

in
stdenv.mkDerivation rec {
pname = "typedb";
version = typedbVersion;

src = srcFolder;

dontBuild = true;

/*
in preparation for the coming change of properties file parsing
allowing for environment variable reading :
> echo "${typedb-properties-patch}" > typedb-properties.patch
> patch ./server/conf/typedb.properties typedb-properties.patch
*/
installPhase = ''
#patch before install
echo '${javaPatch}' > typedb_java.patch
echo '${typedb-wrapper-patch}' > typedb_wrapper.patch
patch ./typedb typedb_java.patch

mkdir $out
cp -r ./* $out
mkdir $out/bin

cat ${typedbWrapper} > $out/bin/typedb
echo "$out/typedb server --data=\$TYPEDB_SERVER_DATA --logs=\$TYPEDB_SERVER_LOGS --port=\$TYPEDB_SERVER_PORT \"\$@\"" >> $out/bin/typedb
patch $out/bin/typedb typedb_wrapper.patch

chmod u+x $out/bin/typedb
rm typedb_java.patch
rm typedb_wrapper.patch
'';

doInstallCheck = true;
installCheckPhase = ''
mkdir "$out/server_data"
mkdir "$out/server_log"
haskie-lambda marked this conversation as resolved.
Show resolved Hide resolved
echo "testing server"
TYPEDB_SERVER_DATA=$out/server_data TYPEDB_SERVER_LOGS=$out/server_log $out/bin/typedb --help > /dev/null
echo "testing client"
TYPEDB_SERVER_DATA=$out/server_data TYPEDB_SERVER_LOGS=$out/server_log $out/bin/typedb client --help > /dev/null
rm -r $out/server_data
rm -r $out/server_log
'';
doCheck = true;

meta = with lib; {
description = "A distributed knowledge graph, a logical database to organise large and complex networks of data as one body of knowledge";
longDescription = ''
TypeDB is a distributed knowledge graph: a logical database to organise large and complex networks of data as one body of knowledge.
TypeDB provides the knowledge engineering tools for developers to easily leverage the power of Knowledge Representation and Automated
Reasoning when building complex systems. Ultimately, TypeDB serves as the knowledge-base foundation for intelligent systems.
'';
homepage = "https://www.grakn.ai/";
license = licenses.gpl3Plus;
platforms = lib.attrNames systems;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
platforms = lib.attrNames systems;
platforms = attrNames systems;

maintainers = [ maintainers.haskie ];
};
}
12 changes: 12 additions & 0 deletions pkgs/servers/typedb/typedbWrapper.sh
@@ -0,0 +1,12 @@
if [ -z "$TYPEDB_SERVER_DATA" ]; then
TYPEDB_SERVER_DATA="__SERVER_DATA__"
mkdir $TYPEDB_SERVER_DATA
fi;
if [ -z "$TYPEDB_SERVER_LOGS" ]; then
TYPEDB_SERVER_LOGS="__SERVER_LOGS__"
mkdir $TYPEDB_SERVER_LOGS
fi;
if [ -z "$TYPEDB_SERVER_PORT" ]; then
TYPEDB_SERVER_PORT="__SERVER_PORT__"
fi;

2 changes: 2 additions & 0 deletions pkgs/top-level/all-packages.nix
Expand Up @@ -29004,6 +29004,8 @@ in

trigger = callPackage ../games/trigger { };

typedb = callPackage ../servers/typedb { };

typespeed = callPackage ../games/typespeed { };

uchess = callPackage ../games/uchess {
Expand Down