Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fediverse roadmap - what is going to change? #33

Open
9 of 24 tasks
mariha opened this issue Jul 12, 2021 · 9 comments
Open
9 of 24 tasks

Fediverse roadmap - what is going to change? #33

mariha opened this issue Jul 12, 2021 · 9 comments
Labels
dev:federation Cross platform development, federation features

Comments

@mariha
Copy link
Member

mariha commented Jul 12, 2021

This is a container issue which aggregates all federated features in one place. It reflects my best effort (very imperfect!) to match single actionable steps with the outcome they are expected to contribute to most.

Here are the changes that we anticipate and how they map to specific GH issues:

Each item has a rough effort/complexity estimate: (XS) - S - M - L - (XL). The estimate represents my perception of them not an objective measure. With bigger complexity comes greater uncertainty.

  1. from communities locked in silos platforms with self-selected boards not representing communities values any more
    to freedom to be on a platform of their choice, with variety of platforms existing, variety of governance, variety of communities to be part of all participating in wider network of hosts and travelers

  2. from few platforms without clear identity each, competing with each other for users
    to the bigger diversity of platforms with their own specific identity developed. Shared access to wide base of hosts and travelers would relieve the burden on platforms to fulfill all needs, and they would not fear of missing out since other needs would be covered by other platforms. One could then imagine platforms in other languages than English or dedicated to certain life choices (think of ecology or veganism)

  3. from users jumping between multiple accounts, one for each platform
    to single account and one search to query all platforms and find the closest hosts among all of them

  4. from users from underrepresented groups (non-english speakers, women, lgbtq, …) hesitating to participate because of feeling of exclusiveness or fear of unsafe interactions
    to the community, where everyone feels welcome, can find their place and group where they belong to, feel safe and can fully express themselves. Full diversity of members from all sort of backgrounds and groups equally represented in the community.

    • incubate new communities (guidelines, best practices/example for governance, supporting tools) (S)
    • reach out to other communities, onboard and embrace their diverse feedback (M-L)
    • modular architecture with addons/plugins for an instance setup (L)
    • abstract protocol, welcome new platforms in the network (M)
  5. from devs-maintainers competing with time to keep up to date with ever changing technologies, tired with maintaining outdated platforms,
    to new platforms emerging every now and then with fresh ideas, energy and tech stacks. This can be achieved by making it easy for new platforms to start based on already existing ones and evolve them freely.

    • contributors' community with inclusive dev process - C4 (ongoing process, but let's say M for instigation)
    • strong devops practices, smooth flow with good ci/cd pipeline (S), executable specification (S), etc.
    • modular/micro-service architecture - good decomposition and encapsulation to decrease complexity inside a module, separate processes (services) to be inclusive to contributors with skills from different tech stacks, yet deployed on one machine for operations simplicity (M)
  6. from platforms keeping hosts forever despite them being inactive and unresponsive
    to only committed hosts participating in the network on their own rules and preferences.

    • user has a choice which instances to federate with (to be visible to? discoverability of instances?, whitelisted and blacklisted? instances) (S-L)

  1. from users exchanging direct messages
    to indirect hosting requests

    I travel and publish: Who can host me?
    Everyone in the network can see it and someone answers: come over to my home, we’ll be waiting this evening for you.

    • the request includes everything needed to determine for who it is relevant: (a) expected location and radius (b) current location, direction and speed (c) …
    • single platform Indirect hosting requests fedi-trustroots#30 (L)
    • federated (L-XL)
@mariha mariha pinned this issue Aug 1, 2021
@mariha mariha added the dev:federation Cross platform development, federation features label Aug 2, 2021
@lennartvanlaake
Copy link

lennartvanlaake commented Sep 24, 2021

Comments for tweaking the roadmap for a Memorandum of Understanding for a funding partner. Not all comments are relevant for the roadmap as such :)

Most of the comments are about filtering/tweaking/prioritising the steps above to make it easier to transalte them to concrete deliverables.

This is a container issue which aggregates all federated features in one place. It reflects my best effort (very imperfect!) to match single actionable steps with the outcome they are expected to contribute to most.

Here are the changes that we anticipate and how they map to specific GH issues:

Each item has a rough effort/complexity estimate: (XS) - S - M - L - (XL). The estimate represents my perception of them not an objective measure. With bigger complexity comes greater uncertainty.

  1. from communities locked in silos platforms with self-selected boards not representing communities values any more
    to freedom to be on a platform of their choice, with variety of platforms existing, variety of governance, variety of communities to be part of all participating in wider network of hosts and travelers

The issue #16 linked here also mentions discoverability and sso. I guess that should be out of scope of the 'M' estimated here. I would suggest that the definition of done here should be just having the bike touring one running as a copy of Trustroods and rebranded to look like a new platform.

Would it make sense to cut this one up into some smaller pieces? I can imagine that in we would always need some sort of protocol to abstract what a Hospex profile looks like. A MVP could be to just get the bike touring one to create a profile based on a JSON file in that format, which I'd estimate as an 'M' by itself.

Perhaps it would make sense for all of these uncertain/big issues to lift the creation of the protocol, creating an MVP with the most basic implemantion of the protocol and anything else we'd like to do into three separate milestones. That way we can communicate progress more easily.

Is the measurable outcome of this ticket that we just get platform prefixes, or that we actually fix all potential issues related to user-uniqueness across our federated instances? I'd say that depending on which one of these the outcome is, the time estimate varies greatly.

  • Admins choose which instances to federate with, individual users can opt-out (whitelisted instances) (XS)

Maybe I'm not understading something, but this seems like quite a pain to build. Intuitively I'd say somewhere between M and L. Perhaps there is some overlap - to make admins select which instances to federate with you need to create some way of securely managing trust between the federating instances. You will also need that for a large part of the other tickets in the roadmap. That is probably one of the bigger headaches of this project. Once you get that sorted, selecting which instances to federate with becomes a lot less hard.

And do we want to commit to opt-out rather than opt-in at this stage? We could also just say that this ticket is done once an admin can select instances to federate with.

  1. from few platforms without clear identity each, competing with each other for users
    to the bigger diversity of platforms with their own specific identity developed. Shared access to wide base of hosts and travelers would relieve the burden on platforms to fulfill all needs, and they would not fear of missing out since other needs would be covered by other platforms. One could then imagine platforms in other languages than English or dedicated to certain life choices (think of ecology or veganism)

Would it make sense to link this to #16? If we first make it easy to deploy new instances of trustroots, perhaps by making all the branding stuff more dynamic/adjustable, we can also more easily launch the bike touring one. We could even say that this could be step 1, after which we use the adjustments to rebrand the bike touring instance.

  • Easy operations and maintenance of a running instance (good observability, logging, reliability and upgrades) (M)

I find it hard to image when we can tell NLNet when this is done. Perhaps we can specify a couple of things we would like concretely (perhaps that all these instances expose well-labeled prometheus-formatted metrics).

I would suggest scoping this ticket out of the NLNet MoU, as I'd say that this would only makes sense once we've completed the base project itself.

Is this a feature? I get that it's work that needs to be done, but I'm struggling a bit on where to draw the line between making stuff MoU-milestones or not.

Perhaps it would make sense to group a couple of more administrative things together (i.e. find out a licence that works for us, promote the organisation, set up a meeting schedule with the community) and present that as a milestone together?

  1. from users jumping between multiple accounts, one for each platform
    to single account and one search to query all platforms and find the closest hosts among all of them

I guess that a large part of this would be managing trust between different instances, so once you get that, I guess SSO won't be that hard :)

If we'd be able to pull off some sort of SSO/trust management between platforms, we could also skip aggregated data and just immediately show the data from the users themselves. I guess that the user story here would be just being able to see dots on a map representing users from other platforms, so maybe just list that as a milestone?

If you can see and message someone from platform A to platform B, why would you need a guest account?

See above on #26

In my experience messaging is a massive pain to set up, I guess doing that in a federated way with added external software from Matrix/ActivityPub/whatever would be even harder. My gut feeling says this would be an L to XL.

  1. from users from underrepresented groups (non-english speakers, women, lgbtq, …) hesitating to participate because of feeling of exclusiveness or fear of unsafe interactions
    to the community, where everyone feels welcome, can find their place and group where they belong to, feel safe and can fully express themselves. Full diversity of members from all sort of backgrounds and groups equally represented in the community.

    • incubate new communities (guidelines, best practices/example for governance, supporting tools) (S)
    • reach out to other communities, onboard and embrace their diverse feedback (M-L)
    • modular architecture with addons/plugins for an instance setup (L)
    • abstract protocol, welcome new platforms in the network (M)

All these things are not really software-deliverables that we can use as milestones. Perhaps leave them out of the MoU?

  1. from devs-maintainers competing with time to keep up to date with ever changing technologies, tired with maintaining outdated platforms,
    to new platforms emerging every now and then with fresh ideas, energy and tech stacks. This can be achieved by making it easy for new platforms to start based on already existing ones and evolve them freely.

    • contributors' community with inclusive dev process - C4 (ongoing process, but let's say M for instigation)

Don't really know what C4 means, but is there something we can turn into a deliverable for this?

  • strong devops practices, smooth flow with good ci/cd pipeline (S), executable specification (S), etc.

I think we could use the CI/CD pipeline as an eary deliverable :)

  • modular/micro-service architecture - good decomposition and encapsulation to decrease complexity inside a module, separate processes (services) to be inclusive to contributors with skills from different tech stacks, yet deployed on one machine for operations simplicity (M)

To me this sounds more like a way we want to make things, rather than a deliverable. I'd try to keep non-deliverables out of the MoU.

  1. from platforms keeping hosts forever despite them being inactive and unresponsive
    to only committed hosts participating in the network on their own rules and preferences.

    • user has a choice which instances to federate with (to be visible to? discoverability of instances?, whitelisted and blacklisted? instances) (S-L)

Isn't this already included in:

  • Admins choose which instances to federate with, individual users can opt-out (whitelisted instances) (XS)
  1. from users exchanging direct messages
    to indirect hosting requests

    I travel and publish: Who can host me?
    Everyone in the network can see it and someone answers: come over to my home, we’ll be waiting this evening for you.

    • the request includes everything needed to determine for who it is relevant: (a) expected location and radius (b) current location, direction and speed (c) …
    • single platform Indirect hosting requests fedi-trustroots#30 (L)

Super excited about this feature for the bicycle touring instance. It does feel like a bit of an orphan to me, since it does not appear to be directly related to federation. So inuitively I'd put it somewhere on the end of the backlog?

  • federated (L-XL)

@lennartvanlaake
Copy link

lennartvanlaake commented Sep 24, 2021

From a practical perspective I would imagine the following sequential roadmap, if I translate the issues into super-high-over user stories:

  • Turn Trustroots into a piece of software that is easily adjusted to the needs/branding of any community
  • Make sure two instances of this software run, with one of them designated for the bicycle touring community
  • Allow and admin from instance A to "trust" instance B
  • Allow users from one instance to import their profile from instance A to instance B after trust is established
  • Allow a user that has a profile on both instance A and B to switch between platforms without logging in again (SSO)
  • Allow a user logged into instance A to see (a subset of) profiles from instance B
  • Allow a user logged into instance A to message (a subset of) users from instance B

@mariha
Copy link
Member Author

mariha commented Oct 1, 2021

[WIP]

I took the high level roadmap that you created and added issues under each point. Our roadmap doesn't need to be sequential, it's ok for a few people to work on different features in parallel.

  1. Turn Trustroots into a piece of software that is easily adjusted to the needs/branding of any community

  2. Make sure two instances of this software run, with one of them designated for the bicycle touring community

  3. Allow an admin from instance A to "trust" instance B

    • Admins choose which instances to federate with, individual users can opt-out (whitelisted instances) (XS)
  4. (?) Allow users from one instance to import their profile from instance A to instance B after trust is established

  5. Allow a user logged into instance A to see (a subset of) profiles from instance B

  6. Allow a user logged into instance A to message (a subset of) users from instance B

Other features good for federation:

  1. Allow a user that has a profile on both instance A and B to switch between platforms without logging in again (SSO)

  • currently agreed on:

  • nice to have included:

  • other:

    • incubate new communities (guidelines, best practices/example for governance, supporting tools) (S)

    • reach out to other communities, onboard and embrace their diverse feedback (M-L)

    • modular architecture with addons/plugins for an instance setup (L)

    • abstract protocol, welcome new platforms in the network (M)

    • contributors' community with inclusive dev process - C4 (ongoing process, but let's say M for instigation)

    • strong devops practices, smooth flow with good ci/cd pipeline (S), executable specification (S), etc.

    • modular/micro-service architecture - good decomposition and encapsulation to decrease complexity inside a module, separate processes (services) to be inclusive to contributors with skills from different tech stacks, yet deployed on one machine for operations simplicity (M)

    • user has a choice which instances to federate with (to be visible to? discoverability of instances?, whitelisted and blacklisted? instances) (S-L)

@chmac
Copy link

chmac commented Dec 23, 2021

From reading this point and its consequences:

Admins choose which instances to federate with, individual users can opt-out (whitelisted instances) (XS)

It seems like the goal here is to have a few communities which choose which other communities to interoperate with. Do you have any thoughts on the risk that you end up with a few large networks who don't want to interoperate with other networks for political reasons? Effectively a small number of people (the server admins) get to make decisions which worsen the quality of experience of the majority. Isn't that the exact issue you have with the current situation (multiple current sites, none interoperating and seemingly not so motivated to collaborate)?

@mariha
Copy link
Member Author

mariha commented Dec 26, 2021

I haven't thought about it but it's a valid point. Seems like it might be better for everyone to be able to add instances they want to federate with, to allow freedom of creating your own instance.

What about black-listing some instances? It's also powerful, but seems not that easy to abuse (especially if had to be done in the open) and it could be useful mechanism to cut-off harmful space while 'good' individuals remain free to move. By abandoning 'evil' instance, those 'good' individuals make at the same time pressure on its admins to not-tolerate harmful behaviors inside that instance. It's like putting an embargo on a country... Although individuals who decide to move loose old connections, they can built new ones in the 'free' space, and those old friends could keep observing them. If 'evil' admins decide to cut themselves off the 'free' space by blacklisting 'good' instances a new ones can always be created...

@chmac
Copy link

chmac commented Dec 30, 2021

@mariha Black listing is definitely an interesting idea. I take from mastodon that some kind of spam control is really essential in these types of systems. I think personally, I definitely prefer blacklisting to whitelisting in general. I think it promotes much more openness in the system.

It's all somewhat moot without "profile portability" though. If I can't take my profile (complete with all my connections, history, reputation, etc) to a new network, then whether the networks talk to each other or not seems a lot less meaningful to me. The idea of portable profiles is pretty challenging. :-)

Happy to have a catch up nerd talk on the tech stuff in the new year if you like.

@mariha
Copy link
Member Author

mariha commented Dec 31, 2021

@chmac yes! I’ll be happy to have a chat on technical topics and was meaning to propose it in coming days, just got pulled by other things going on.

Answering quickly, per my understanding, portability could be solved by storing data in a Solid Pod in some common among platforms format. This prevents from platforms locking-in their users. Less secure but also an option that works for some scenarios is an export and import of data… also easy self-hosting if someone chose too.

I think (haven’t read and don't understand enough for sure) I could also imagine references and connections in ScuttleButt but not sure how communities would work then?

More importantly though, if we prioritized enabling diversity and evolution of hospitality exchange platforms over fighting with power and preventing community lock-in, what I personally lean towards, communication between platforms takes precedence over data portability. Not saying that it’s not important but could be done as a second step.

I’ll be more then happy to discuss all of this! Let’s plan something in our OHN channel.

@chmac
Copy link

chmac commented Jan 5, 2022

As I understand it, the key challenge is how to identify a user. If the user's identity is like in matrix @username:host then they're locked to that host. All their "reputation" is linked to the "id". But if their identity is a uuid or a hash of a public key (something portable), then their "reputation" is portable, but the question becomes how do you find them? Then there's a need to have some kind of "address resolution" system where I can figure out where a user's profile currently lives.

Will message you now on matrix to see about catching up on the topic.

@mariha
Copy link
Member Author

mariha commented Jan 5, 2022

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev:federation Cross platform development, federation features
Projects
None yet
Development

No branches or pull requests

3 participants