Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persistent storage for append /caching performance #251

Merged
merged 1 commit into from Aug 30, 2019

Conversation

@aphelionz
Copy link
Contributor

aphelionz commented May 17, 2019

This PR:

  • Implements the persistent store work encapsulated in orbit-db-storage-adapter and its dependents.
  • Removes the test/utils folder, favoring the updated orbit-db-test-utils package.
    • by proxy drops these dependencies: go-ipfs-dep, http-server, ipfs, is-node, ipfs-repo, ipfsd-ctl, is-node, istanbul
  • Restores the loading visualizer tool in the examples folder
  • Fixes browser tests by removing the --disable-dev-shm-usage flag and forking mocha-headless-chrome
@aphelionz aphelionz changed the title FIFO nexts for append performance FIFO nexts for append performance - 10x boost May 29, 2019
@aphelionz aphelionz changed the title FIFO nexts for append performance - 10x boost FIFO nexts and persistent storage for append performance - 10x boost May 29, 2019
@aphelionz aphelionz mentioned this pull request May 31, 2019
10 of 10 tasks complete
@aphelionz aphelionz force-pushed the fix/append-performance branch 2 times, most recently from 52e67cd to 392e875 Jul 10, 2019
src/entry-index.js Outdated Show resolved Hide resolved
@aphelionz aphelionz force-pushed the fix/append-performance branch from 392e875 to ffc70e6 Jul 15, 2019
@aphelionz aphelionz force-pushed the fix/append-performance branch from 96ceef6 to c90c0be Aug 2, 2019
@aphelionz aphelionz changed the title FIFO nexts and persistent storage for append performance - 10x boost Persistent storage for append /caching performance Aug 12, 2019
@aphelionz aphelionz force-pushed the fix/append-performance branch from 0d0c3c9 to b807d6a Aug 22, 2019
Ipfs,
MemStore,
IdentityProvider
}

This comment has been minimized.

Copy link
@haadcode

haadcode Aug 28, 2019

Member

What's the purpose of this example? There's not a lot to it, so perhaps add a comment describing what to look for, or if this is not intended as an example, move the file to a more appropriate place.

This comment has been minimized.

Copy link
@aphelionz

aphelionz Aug 28, 2019

Author Contributor

Oh, I think I was just making sure that everything worked in the browser, or I had started on something and didn't finish it. Either way I'll simply remove for now.

This comment has been minimized.

Copy link
@haadcode

haadcode Aug 30, 2019

Member

This should still be removed, no?

This comment has been minimized.

Copy link
@aphelionz

aphelionz Aug 30, 2019

Author Contributor

Negative. This is the entry point for the webpack example build, so we need that to build the bundle which runs the examples

.circleci/config.yml Outdated Show resolved Hide resolved
test/signed-log.spec.js Outdated Show resolved Hide resolved
test/signed-log.spec.js Outdated Show resolved Hide resolved
test/entry-io.spec.js Outdated Show resolved Hide resolved
test/replicate.spec.js Outdated Show resolved Hide resolved
@aphelionz aphelionz force-pushed the fix/append-performance branch from 1baaefa to 03bf08c Aug 28, 2019
fix: all tests passing locally

chore: package-lock.json

Loading visualizer working

test: updating entry.spec

test: entry-io.spec

test: log-append.spec

test: append and crdt

test: heads-tails.spec

test: log-iterator

test: log-join.spec

test: log-load.spec

test: log.spec

test: replicate.spec

random fixes

chore: linting

test: passing with new storage paradigm

chore: linting

ci: working to fix the build

successful make rebuild

chore:dependencies

test: fixing keystore paradigm

chore: updating keystore syntax

chore: updating dependencies

fix: traversal bailing earlier

Adding pointerCount for more reaslistic benchmark

chore: build and linting

Giving up the ghost on the append performance, just gonna default store to 32

Update package.json

Update package.json

add entry-index module

New build files, master merged in

chore: package-lock.json

Loading visualizer working

test: updating entry.spec

test: entry-io.spec

test: log-iterator

test: log.spec

random fixes

chore: linting

test: passing with new storage paradigm

chore: linting

ci: working to fix the build

successful make rebuild

chore:dependencies

test: fixing keystore paradigm

chore: updating keystore syntax

chore: updating dependencies

chore: build and linting

Giving up the ghost on the append performance, just gonna default store to 32

Update package.json

Update package.json

add entry-index module

New build files, master merged in

CI test

removing pify

Fixing go-ipfs tests

build files

linting

Proper package.json

package lock

Adding log-creator back

Fixing test:node in preparation for test:browser

linting

Ready...?

make rebuild after rebase

feat: FIFO nexts

fix: all tests passing locally

chore: package-lock.json

Loading visualizer working

test: updating entry.spec

test: entry-io.spec

test: log-append.spec

test: append and crdt

test: heads-tails.spec

test: log-iterator

test: log-join.spec

test: log-load.spec

test: log.spec

test: replicate.spec

random fixes

chore: linting

test: passing with new storage paradigm

chore: linting

ci: working to fix the build

successful make rebuild

chore:dependencies

test: fixing keystore paradigm

chore: updating keystore syntax

chore: updating dependencies

fix: traversal bailing earlier

Adding pointerCount for more reaslistic benchmark

chore: build and linting

Giving up the ghost on the append performance, just gonna default store to 32

Update package.json

Update package.json

add entry-index module

New build files, master merged in

chore: package-lock.json

Loading visualizer working

test: updating entry.spec

test: entry-io.spec

test: log-iterator

test: log.spec

random fixes

chore: linting

test: passing with new storage paradigm

chore: linting

ci: working to fix the build

successful make rebuild

chore:dependencies

test: fixing keystore paradigm

chore: updating keystore syntax

chore: updating dependencies

chore: build and linting

Giving up the ghost on the append performance, just gonna default store to 32

Update package.json

Update package.json

add entry-index module

New build files, master merged in

CI test

removing pify

Fixing go-ipfs tests

build files

linting

Proper package.json

package lock

Adding log-creator back

Fixing test:node in preparation for test:browser

linting

build files

final make rebuild

Build files

CircleCI

CircleCI stuff

Build files

WIP PR comments

moving defaultStorage to orbit-db-test-utils

more PR updates

Linting and connectPeers

Better keystore instantiation and closing

Linting

git ignore stuff
@aphelionz aphelionz force-pushed the fix/append-performance branch from 6fa1566 to e5f23fe Aug 30, 2019
@haadcode

This comment has been minimized.

Copy link
Member

haadcode commented Aug 30, 2019

LGTM 👍

@aphelionz aphelionz merged commit 79594dc into master Aug 30, 2019
1 check passed
1 check passed
ci/circleci Your tests passed on CircleCI!
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.