-
-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
de-emphasizing large cabal support & features #130
Comments
cc @cblgh @nikolaiwarner @Karissa @substack @cinnamon-bun @telamon |
Hi All!
@noffle -- thanks for this. The public cabal has been great as an easy
go-to for bringing people in so they could test cabal out quickly and get
excited about it; but perhaps it's currently sometimes undermining that
purpose, if a user's first experience of it is that it's frozen, or very
slow. I myself was unable to participate in the public cabal with my
(beloved) underpowered netbook until I borrowed a higher-powered machine.
Aside: I wonder if this is an issue for p2p social networking in general:
that it's quite a challenge to run a p2p social network in 'monolothic'
mode. I saw a post on Scuttlebutt recently discussing a similar issue:
trying to on-board a friend to Scuttlebutt was going to result in many,
many gigabytes of peers-of-peers' data flowing into the inductee's machine.
I also want to add a fully p2p tool for small-group collaboration is
already a huge accomplishment! Cabal is currently extremely useful in many
important contexts. In fact, most chat users in the world are probably more
familiar with e.g. GChat or Slack conversations with only a few people;
and I suspect that the majority of the chat use-cases out in the world
would be handled very well by Cabal. So, I think Cabal is already a huge
"success" as a technology, since it works brilliantly for what are likely
the bulk of use-cases; and that there's no shame in broadcasting that it's
intended (at least currently) for small-ish chats.
The idea of shifting the bulk of the 'general, all-comers public
discussion' to IRC seems like a good idea to me. And maybe there could be a
corresponding shift in emphasis on the cabal landing site: suggesting that
it's ideal for 'private, small-group collaboration' -- that you should "try
spinning one up with your friends". I could imagine an illustration
graphic that shows a few nodes connected in a small-world network manner,
emphasizing the suggested manner of connection.
For individuals who want to test out cabal immediately, I wonder if there'd
be some way to automatically have a 'bot' spin up as a peer on a remote
server? -- Maybe it could even give a little tutorial, and then suggest
inviting (a few) friends.
I guess this is all just a overly-wordy +1 to @noffle's suggestion. Yay,
cabal-32! :)
…On Fri, Sep 13, 2019 at 12:17 PM noffle ***@***.***> wrote:
cc @cblgh <https://github.com/cblgh> @nikolaiwarner
<https://github.com/nikolaiwarner> @Karissa <https://github.com/karissa>
@substack <https://github.com/substack> @cinnamon-bun
<https://github.com/cinnamon-bun> @telamon <https://github.com/telamon>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#130?email_source=notifications&email_token=AAE4GEE2VU4KOJGDAKI3YQLQJO4IZA5CNFSM4IWRYOQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6VQDLQ#issuecomment-531300782>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAE4GEFZTNROW6PNIXVK3ODQJO4IZANCNFSM4IWRYOQA>
.
|
I say +1 yes. Also most slack/gchat/signal group texts are relatively small. Instead of having a public cabal link on the front page, it could be interesting to have a small set of instructions on how to start your own cabal and share it? Cheers! |
@noffle what's the main bottleneck? Is it still caused by doing 500 feed.replicate() calls simultaneously? I know i've suggested this in the past but with hypercore-protocolv7 upgrade I get a gut feeling that it's even better suited for feed-hotswapping. We could ensure that no more than 32 simultenous hypercores are replicated at any one time and as soon as one feed hit's the end, the next one in queue will be activated. Not unlike the live-feed-fowarding mechanism we'd need to re-que a feed for replication whenever a new feed entry is detected either from local peer or from other peer. Would that solve the timeouts and freezes? |
tl;dr HELL YEAHlet us refocus on stability & core features & usability i really like karissa's suggestion of highlighting how to start a cabal for my main reasons for having the public cabal, apart from getting to know new
i'm +1 on cabal-32 or: 32 people (and their misc devices). so yeah, in so yes! in my work i will personally be down-prioritizing the bigger use-cases which is great because i'm, like, *so* excited for iterating and improving on thank you noffle and everyone else, yr the best 🖤 |
Yay!!
Random thought that occurred to me (apologies if this is already mentioned,
I'm scatter-brained) would it be hard to include the ability to set a
"user # limit" feature into cabal -- refusing connections beyond a certain
size? It might be a nice feature for folks starting cabals to have -- they
can be assured that the cabal size stays small -- and it might be an
elegant way (?) to ensure a 'private' cabal by only allowing a certain
number of members in (would be even cooler if you only bless certain peers,
but that seems harder to manage as peer IDs could change across computers
or something).
Another thought: could still offer "public" cabals where small groups are
willing to open theirs to new users (to help with onboarding) -- such
groups could list their cabals on a main site as "currently open to new
members" and then just un-list when they've gotten as many as they want to
handle -- similar to how (I think) Scuttlebutt lists open 'pubs' ...
</ random thought>
Cheerio!
…On Sat, Sep 14, 2019 at 5:10 AM Alexander Cobleigh ***@***.***> wrote:
tl;dr HELL YEAH
let us refocus on stability & core features & usability
i really like karissa's suggestion of highlighting how to start a cabal for
your friends, as well as dwblair's realization of:
*"well, maybe that first potentially laggy af load isn't actually a good
experience"*
my main reasons for having the public cabal, apart from getting to know new
people (surprisingly enough, like 99.999% of the random internet people
visiting have been really cool),
- it's been a good stress test and helped us find a lot of edge-cases
faster
- it's an easy way to get going and experience the magic p2p backlog
i'm +1 on cabal-32 or: 32 people *(and their misc devices)*. so yeah, in
reality i see cabal-32 more like cabal-32 *±16* 🙃
so yes! in my work i will personally be down-prioritizing the bigger
use-cases
like 200-2000+ community slacks in favour of focusing on
*a cabal that works good for friend circles, small groups, and small coops*.
at least for the next
10 months or so, whereupon we can maybe look at what we have and what
progress has been made around the hypercore ecosystem and such
which is great because i'm, like, **so** excited for iterating and
improving on
what we already have as well as writing cool new features :33
thank you noffle and everyone else, yr the best 🖤
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#130?email_source=notifications&email_token=AAE4GEDGP2JUUZNLP2NYLHLQJSTANA5CNFSM4IWRYOQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6WX5OI#issuecomment-531463865>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAE4GEFDI5P7SXAIJA5FBG3QJSTANANCNFSM4IWRYOQA>
.
|
Thank you everyone for writing back (here & in IRC). I really appreciate the great perf-improving ideas that @telamon and @substack have put forward, and yes: I'm totally down for those existing. And, my experience so far has been that there are a lot of challenges around large cabals that any single fix can't fully address: handling many connections for a very active large cabal, dealing with not having too many "hot" hypercores replicating at a time, preventing a cabal /w massive history from taking forever to do initial sync, separating p2p-db and ui into different threads or processes to keep the UI responsive, and even things around "bad actors" that happen in public chats, like nick impersonation. There are a lot of challenges to solve to make large cabals work well & feel good & feel safe, and I think every patch y'all make will be a positive step in that direction. Really, I could just avoid the public cabal & personally ignore perf stuff, but I also really want folx who try cabal to quickly grok what makes cabal such a powerful & useful tool. That's not something I can fully control, but if we know cabal works great already for small semi-private or private groups, it might help to focus the website / intro path toward that use case, as @okdistribute suggested. This all cuts into the core Q of "what is cabal?". Who is cabal for? What problem is cabal trying to solve? This is a bigger Q /w loads of surface area, but I'm keen to think more deeply on this & see where we can all align our efforts, especially since cabal has such a massive set of cool things it could be. |
I know its really hard to prioritize these questions, but I'd love to see more brainspace put to this (myself included). When @noffle first posted this question, these questions were also the ones I was thinking about. I was curious, so went to the existing cabal.chat site and its mostly about the technology. I'm sure there is some copy about Cabal that is more user-focused but something to take note of. We've struggled with the same things with Dat and I'm not sure I have any helpful approaches but I know that it does take some time trying to answer her questions and then trying to work on how to share that with others. |
i just removed the public cabal from the github pages site and the cli instructions, so that's a first step. the site probably needs to be redesigned a tad to better support and feature how to get going with yr friends |
@cblgh I still haven't managed to replicate the public cabal, could someone zip it up or dat it? I'd like to use the collection of feeds for future benchmarking and testing. 😃 |
@telamon sent in a pm |
A while ago I had the idea of using plaintext "invite codes", where the cabal key is just |
Thanks y'all for participating in this convo! Closing this issue. ❤️ |
Hi friends. 👋
On my entry level 2013 laptop, the public cabal is unusably slow for me. I've had to resort to running it with
--seed
and letting it sync for a while, then running it with--offline
to read/write, then once more with--seed
to push my changes. The networking -- specifically, the massive # of hypercores (517 so far) -- doesn't scale well. Loading the cores from disk and doing the expensive hypercore-protocol handshake takes a lot of CPU. And, since node is single-threaded, this locks up the UI as well.Yesterday I realized that there's an incongruity in the way we're developing: the public cabal is the only big cabal out there (as far as we know). The rest seem to generally be small groups. In this context, cabal works really well. I find it super fast and responsive even on my laptop. So despite most folx using cabal with smaller groups right now, I've noticed most of my development time has been going into trying to make the public cabal fast enough to be usable (for me & whoever else isn't on very fast/modern/expensive hardware).
Working on making cabal scale well for use cases like Freenode-style IRC (cabal with many many channels and many many members) or massive slacks (like https://lgbtq.technology) is interesting and valid and useful, but maybe not a great focus while we're also trying to just get cabal reliable and functional.
This might be more of a me-thing than how the rest of y'all feel, but focusing on making cabal scale while we're so tiny & underresourced & unstable has been sapping a lot of my energy toward the project. I'd much rather focus on making cabal-32 (e.g. cabal tech optimized for <= 32 members) work well, as I tend to enjoy small groups of friends much much more than giant chatrooms anyways.
Curious to hear what y'all think, and also whether it makes sense to keep having a giant "public cabal" vs keep using IRC as a place for randos to hop in. I like that we have a small, generally closed dev cabal right now, and I've found that to be working really well.
tl;dr: I'm keen to de-emphasize big public cabals for now & scaling problems, and refocus on stability & core features & usability.
The text was updated successfully, but these errors were encountered: