[WIP] Public stream archives

[adnrs96]

This PR will track our progress on getting #4817 ready.
The first commit introduces changes so that we don't supply user emails in the rendered content of messages having user mentions. Second commit introduces DB changes in the Stream table so that we can call some of the streams global-public i.e. they are visible to the world without having to login.
Discussion on this is being tracked by https://chat.zulip.org/#narrow/stream/backend/topic/global-public.20streams

[adnrs96]

Steps for testing this out after pulling locally:

• Run provisioning using tools/provision
• Open up dbshell in a terminal by using the command ./manage dbshell
• Query the DB for stream names and id's using select id, name from zerver_stream; query.
• Note down the id for streams you wanna make public. Lets say we have a stream Scotland with id 3
• Open up another terminal and use it to open shell using the command ./manage shell
• In this shell execute the following lines

x = [comma separated list of id's which you wanna make web public goes here]
for id in x:
    a = Stream.objects.get(id=id)
    do_change_stream_global_public(a, True)

• Visit the link of following nature
  http://localhost:9991/archive/streams/<stream_id>/topic/<topic_name>
  Example:
  http://localhost:9991/archive/streams/3/topic/Scotland2

[adnrs96]

Steps for testing this out after pulling locally:

• Run provisioning using tools/provision
• Open up dbshell in a terminal by using the command ./manage dbshell
• Query the DB for stream names and id's using select id, name from zerver_stream; query.
• Note down the id for streams you wanna make public. Lets say we have a stream Scotland with id 3
• Open up another terminal and use it to open shell using the command ./manage shell
• In this shell execute the following lines

x = [comma separated list of id's which you wanna make web public goes here]
for id in x:
    a = Stream.objects.get(id=id)
    do_change_stream_global_public(a, True)

• Visit the link of following nature
  http://localhost:9991/archive/streams/<stream_id>/topic/<topic_name>
  Example:
  http://localhost:9991/archive/streams/3/topic/Scotland2

[adnrs96]

Note: I am temporarily using the global_public DB changes here which are essentially useful for testing this out.
Ideally we will have another PR introducing DB changes for this feature and get that PR merged before this.

[adnrs96]

Note: I am temporarily using the global_public DB changes here which are essentially useful for testing this out.
Ideally we will have another PR introducing DB changes for this feature and get that PR merged before this.

[timabbott]

@adnrs96 I took a quick look at this; I'm going to merge the first commit after fixing the tests. I also opened #8588 for a related issue that this reminded me of.

[timabbott]

@adnrs96 I took a quick look at this; I'm going to merge the first commit after fixing the tests. I also opened #8588 for a related issue that this reminded me of.

[timabbott]

I merged the first 5 commits, since those look great.

[timabbott]

I merged the first 5 commits, since those look great.

[timabbott]

I think we should make this error message clear about the possibilities: "This stream either does not exist or is not publicly archived to the web.".

[timabbott]

(And we should tag these strings for translation).

[timabbott]

I think "No messages have been sent to this topic within the publicly archived stream #%(stream_name)s".

[timabbott]

Once we merge the main infra commits, I think we can split off a project to make a better error page for this case; I think we'll want to show a list of the public streams in the organization.

[timabbott]

I went ahead and merged this, since it doesn't break anything, and it'll be easier to iterate from here, I think. Some notes on immediate follow-ups:

• Let's fix the error message items I noticed above.
• We're going to want to make the "left sidebar" equivalent for this feature (which I would consider building as its own page, rather than actually placing it as a sidebar). I would probably start with just building an endpoint to get the metadata and list of topics in a web-public stream (you can borrow some logic from the existing /topics endpoint), sorted by time. This feels pretty independent of whatever we're doing with message rendering.
• Similarly, we probably will also want to make a /archive homepage for a realm, that just has the realm's banner, and a list of the web-public streams in the organization that you can click on, to get to the per-stream pages.
• I think we can refactor test_web_public_stream_topic to use the Rome stream; as we add more tests, I think it'll be nicer to be building on top of the existing stream with this setting.

[timabbott]

I went ahead and merged this, since it doesn't break anything, and it'll be easier to iterate from here, I think. Some notes on immediate follow-ups:

• Let's fix the error message items I noticed above.
• We're going to want to make the "left sidebar" equivalent for this feature (which I would consider building as its own page, rather than actually placing it as a sidebar). I would probably start with just building an endpoint to get the metadata and list of topics in a web-public stream (you can borrow some logic from the existing /topics endpoint), sorted by time. This feels pretty independent of whatever we're doing with message rendering.
• Similarly, we probably will also want to make a /archive homepage for a realm, that just has the realm's banner, and a list of the web-public streams in the organization that you can click on, to get to the per-stream pages.
• I think we can refactor test_web_public_stream_topic to use the Rome stream; as we add more tests, I think it'll be nicer to be building on top of the existing stream with this setting.