Set up database connection and create tables #247

rvl · 2019-05-09T07:05:46Z

Relates to issue #143 and #154

Overview

Added partial newDBLayer function which sets up the db connection and runs migrations.
Added a small test case to check that the connection works and tables exist.
Add database queries for wallet metadata.

paweljakubas · 2019-05-10T14:07:36Z

lib/core/cardano-wallet-core.cabal

@@ -46,6 +46,7 @@ library
    , generic-lens
    , http-api-data
    , http-media
+    , lens


we use generic-lens already in codebase. If I remember well, it was decided to use this lens library. probably it is good idea in the last commit to comply with the rest of the code (if possible)

paweljakubas · 2019-05-10T14:12:08Z

lib/core/cardano-wallet-core.cabal

+    , persistent-sqlite
+    , conduit
+    , monad-logger
+    , mtl


we use transformers already. probably it is good idea in the last commit to comply with the rest of the code

paweljakubas · 2019-05-11T22:17:32Z

@rvl Please notice that the branch was rebased against rvl/147/sql-schema (which was rebased before against master) and forced pushed, so if #227 is merged then you can put it against master and it should compile and be green (made needed fixes and also cleaned imports). I started addressing PrivateKey table and stumbled into problem how to really represent (Key 'RootK XPrv, W.Hash "encryption") . I tried to represent them as (Text, Text) via ByteString but then have problem to return from db to Key 'RootK XPrv

rvl · 2019-05-11T22:31:25Z

@paweljakubas Thanks 😄

paweljakubas · 2019-05-13T20:20:27Z

@rvl Please read it before you continue working :

rvl/147/sql-conn is rebased against master, compiles and should be green
I ported all tests from MVarSpec adding SQL flavor (especially to wallet state which is represented as (UTxO, PendingTx))
A number of tests does not pass - I denoted them as xit rather than it (at this moment 16 tests do not pass)
I moved shared parts between MVar and SQL to DBSpec - still properties can also be moved to DBspec and shared as they are almost the same - maybe we can do it at the end together with lens refactoring

paweljakubas · 2019-05-17T07:30:20Z

lib/core/src/Cardano/Wallet/DB/Sqlite.hs

+       -- ^ Database file location, or Nothing for in-memory database
+    -> IO (DBLayer IO s t)
+newDBLayer fp = do
+    conn <- createSqliteBackend fp logStderr


wil this logStderr enable all logs (CREATE_TABLE, INSERT, ...) as a default or errors only? I would give my votes for the second option

indeed, it introduces logs like :

Wallet table Migrating: CREATE TABLE "wallet"("wallet_id" VARCHAR NOT NULL,"name" VARCHAR NOT NULL,"passphrase_last_updated_at" TIMESTAMP NULL,"status" INTEGER NOT NULL,"delegation" VARCHAR NULL, PRIMARY KEY ("wallet_id")) CREATE TABLE "wallet"("wallet_id" VARCHAR NOT NULL,"name" VARCHAR NOT NULL,"passphrase_last_updated_at" TIMESTAMP NULL,"status" INTEGER NOT NULL,"delegation" VARCHAR NULL, PRIMARY KEY ("wallet_id")); []

added more customizable logging function (one can choose LogLevel)

paweljakubas · 2019-05-17T07:31:26Z

lib/core/src/Cardano/Wallet/DB/Sqlite.hs

+        W.lastUpdatedAt <$> meta ^. #passphraseInfo
+    , walTableStatus = meta ^. #status
+    , walTableDelegation = delegationToText $ meta ^. #delegation
+    , walTableAddressScheme = Sequential


This probably has to be addressed in another PR. Either removed or detected whether we are Random or Sequential

Yes there is not yet reading/writing of the wallet state.
However we may even remove this field because the address scheme is fixed by a type parameter of DBLayer.

exactly. This is in line what @KtorZ wrote on slack yesterday:

Then, we'll have one - external seq wallet layer + seq db layer - fully owned seq wallet + seq db layer - fully owned rnd wallet + rnd db layer

Address scheme is set for each db instance. Hence, walTableAddressScheme is probably superflouous

removed address scheme

paweljakubas

LGTM

KtorZ · 2019-05-17T08:14:55Z

bors r+

iohk-bors · 2019-05-17T08:39:51Z

Canceled

It had an infinite loop.

This will show whether cascading deletes work under sqlite -- once the functions to write other tables are implemented.

rvl added this to the SQLite implementation for the DB Layer milestone May 9, 2019

rvl force-pushed the rvl/147/sql-conn branch from 5000c06 to c93c43f Compare May 9, 2019 07:21

KtorZ assigned rvl May 9, 2019

paweljakubas force-pushed the rvl/147/sql-schema branch 2 times, most recently from 58834a6 to b424657 Compare May 9, 2019 16:40

rvl force-pushed the rvl/147/sql-schema branch from 5a812c3 to 0635746 Compare May 9, 2019 21:35

rvl force-pushed the rvl/147/sql-conn branch from c93c43f to e3aa33d Compare May 9, 2019 21:44

rvl force-pushed the rvl/147/sql-schema branch from 0635746 to 1c6a908 Compare May 10, 2019 05:19

rvl force-pushed the rvl/147/sql-conn branch 2 times, most recently from 7eebd26 to 5dfe82e Compare May 10, 2019 09:14

paweljakubas force-pushed the rvl/147/sql-schema branch 3 times, most recently from d219095 to f2c65ad Compare May 10, 2019 13:33

paweljakubas reviewed May 10, 2019

View reviewed changes

paweljakubas force-pushed the rvl/147/sql-schema branch 2 times, most recently from a1efedb to ca6be91 Compare May 11, 2019 18:00

paweljakubas force-pushed the rvl/147/sql-conn branch 2 times, most recently from 9522984 to 53739ab Compare May 11, 2019 19:13

paweljakubas mentioned this pull request May 13, 2019

Define SQL Schema representing relevant Haskell types #147

Closed

5 tasks

paweljakubas force-pushed the rvl/147/sql-schema branch from ca6be91 to a3ec3a6 Compare May 13, 2019 08:54

paweljakubas changed the base branch from rvl/147/sql-schema to master May 13, 2019 10:20

paweljakubas force-pushed the rvl/147/sql-conn branch 4 times, most recently from 4acfcf9 to aba61e3 Compare May 13, 2019 20:17

paweljakubas self-assigned this May 13, 2019

paweljakubas reviewed May 17, 2019

View reviewed changes

paweljakubas approved these changes May 17, 2019

View reviewed changes

rvl mentioned this pull request May 17, 2019

Sqlite: add a test of cascading delete #276

Merged

paweljakubas force-pushed the rvl/147/sql-conn branch from fec073b to e448eac Compare May 17, 2019 09:41

rvl and others added 19 commits May 17, 2019 11:42

Add getPending :: Wallet s t -> Set Tx

31e9d8e

DB.Sqlite: Fix type of WalletState column

67d5557

Add PersistField instance for Address

3ed3c1a

Put back haddocks which got lost in a rebase.

7dcb324

Sqlite: Add on delete cascade to schema

091e9a9

Sqlite: Fix instance PersistField TxStatus

370e6bb

It had an infinite loop.

Sqlite: fix the schema

625d9ce

tests: Add simple Sqlite DB layer test

ef1a2cb

Sqlite: Add database connection and wallet metadata queries

4872966

fixup! Sqlite: Add database connection and wallet metadata queries

95e2bac

fixup! tests: Add simple Sqlite DB layer test

1e114f0

fixup! Sqlite: fix the schema

e990d30

Sqlite: use exceptions for handling ErrWalletAlreadyExists

cbfc0c8

Sqlite: Some testing of exception handling

8c863a2

Sqlite: add a couple source comments

700187e

Sqlite: add a test of cascading delete

0743119

This will show whether cascading deletes work under sqlite -- once the functions to write other tables are implemented.

Remove superfluous address scheme from db

5a8ffa1

use beforeAll and beforeWith combination in DBSpec

6cc1e5f

Control more logging from db

e448eac

paweljakubas merged commit e03bcc8 into master May 17, 2019

iohk-bors bot deleted the rvl/147/sql-conn branch May 17, 2019 10:26

akegalj mentioned this pull request May 21, 2019

Extend the current DB layer with an SQLite interface #154

Closed

12 tasks

KtorZ removed this from the SQLite implementation for the DB Layer milestone Jun 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set up database connection and create tables #247

Set up database connection and create tables #247

rvl commented May 9, 2019 •

edited

Loading

paweljakubas May 10, 2019 •

edited

Loading

paweljakubas May 14, 2019

paweljakubas May 10, 2019 •

edited

Loading

paweljakubas May 13, 2019

paweljakubas commented May 11, 2019

rvl commented May 11, 2019

paweljakubas commented May 13, 2019

paweljakubas May 17, 2019

paweljakubas May 17, 2019

paweljakubas May 17, 2019

paweljakubas May 17, 2019 •

edited

Loading

rvl May 17, 2019

paweljakubas May 17, 2019

paweljakubas May 17, 2019

paweljakubas left a comment

KtorZ commented May 17, 2019

iohk-bors bot commented May 17, 2019

Set up database connection and create tables #247

Set up database connection and create tables #247

Conversation

rvl commented May 9, 2019 • edited Loading

Overview

paweljakubas May 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paweljakubas May 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paweljakubas commented May 11, 2019

rvl commented May 11, 2019

paweljakubas commented May 13, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paweljakubas May 17, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paweljakubas left a comment

Choose a reason for hiding this comment

KtorZ commented May 17, 2019

iohk-bors bot commented May 17, 2019

Canceled

rvl commented May 9, 2019 •

edited

Loading

paweljakubas May 10, 2019 •

edited

Loading

paweljakubas May 10, 2019 •

edited

Loading

paweljakubas May 17, 2019 •

edited

Loading