Enable to see the 'firehose'

[eschnou]

With a small community and to help bootstrap things, having access to the full firehose of messages in the UI would be awesome. Since I haven't looked at the technical details yet, I wonder if it is at all possible (i.e. does my client see all messages or only the one routed to me?).

If someone think is possible and has some pointers to get me started, I'm happy to dive in the code and try a first contribution.

[toyg]

I think Miguel is the only one who can answer. From what I understand from the bug report about "too many users crashing twisterd", there are architectural limits that are hard to deal with -- basically, if I understood correctly, every time you follow a user, you're actually joining a sort of torrent with all his posts. Joining hundreds and hundreds of torrents at the same time is hard work, and it seems to have problems when you follow more than 250 users or so.

[gubatron]

has there an acceptable limit of incoming messages per second been measured (to avoid crashes or freezing)?
my guess is that you'll have to add some sort of throttling and queuing (might be there already, still getting my macosx to build the project, can't wait to be of help)

[miguelfreitas]

Giacomo,

I have been think about this. You understood correctly the part about
joining torrents, but it is not difficult to fix that limit, not at all.

The way I modeled torrent swarms was removing all direct access to files
(within a given torrent) and replacing them with a leveldb instance. I
don't know much about leveldb, but i'd guess it probably keeps a few file
descriptors open, like 3 per torrent or something (i don't know how many).

It is not difficult to replace individual torrent's leveldb instances for a
single global leveldb. That is, just one database for all swarms. We just
need to concatenate the hash or username when computing the key in leveldb.

libtorrent already seems to do a good job about round-robing the torrents,
so file descriptors used for connections shouldn't be an architectural
limitation.

regards,

Miguel

On Sun, Jan 5, 2014 at 1:10 AM, Giacomo Lacava notifications@github.comwrote:

I think Miguel is the only one who can answer. From what I understand from
the bug report about "too many users crashing twisterd", there are
architectural limits that are hard to deal with -- basically, if I
understood correctly, every time you follow a user, you're actually joining
a sort of torrent with all his posts. Joining hundreds and hundreds of
torrents at the same time is hard work, and it seems to have problems when
you follow more than 250 users or so.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-31595114
.

[miguelfreitas]

On Sun, Jan 5, 2014 at 2:50 PM, Angel Leon notifications@github.com wrote:

has there an acceptable limit of incoming messages per second been
measured (to avoid crashes or freezing)?

Not that i can think of. But what you mean by incoming messages per second,
the ones from users you follow?

I'd guess the ability of libtorrent to download pieces/second =
posts/second is pretty high.

[iShift]

Old issue i think we can close that