Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Systematically survey message content for unimplemented features #190

Open
gnprice opened this issue Jun 14, 2023 · 2 comments
Open

Systematically survey message content for unimplemented features #190

gnprice opened this issue Jun 14, 2023 · 2 comments
Labels
a-api Implementing specific parts of the Zulip server API a-content Parsing and rendering Zulip HTML content, notably message contents

Comments

@gnprice
Copy link
Member

gnprice commented Jun 14, 2023

Our parsing of Zulip message content HTML is designed to be precise about what it expects, and explicit about anything it doesn't understand. This means that when we encounter some content that does something we don't have support for, our code generally knows that, rather than silently plow ahead with a wrong interpretation.

This is helpful because, among other things, it means that if we take a corpus of Zulip message content, we can run our parser on it to learn about constructs that exist in the wild that we haven't yet implemented. (This includes constructs that a current Zulip server will generate, and constructs that older servers would generate and consequently still exist in old messages.)

So we should do that, as an iterative process:

  • Run on some set of messages; find things that are unimplemented; file issues; fix them, or at least those that are most common, until the number of unsupported messages is small.
  • Then run on a larger set of messages, or one with more very old messages, to find more unimplemented things.
  • Repeat until it's difficult to find a message with something we don't know about, and we're comfortable with whatever set of known unimplemented features remain.

Some specific likely steps in that process:

  • Write a script that can fetch a bunch of messages in a loop, feed them through parseContent, and report any UnimplementedNode results (as well as any crashes, which should be fixed immediately).
  • Run that script on some recent public messages (i.e., messages to public streams) on chat.zulip.org.
  • Run it on all public messages on chat.zulip.org.
  • Run it on all public messages in realms that publicly list themselves as open communities. For example, have the script sign up a test user in each realm and log in as that test user.
@gnprice gnprice added the a-content Parsing and rendering Zulip HTML content, notably message contents label Jun 14, 2023
@gnprice gnprice added this to the Launch milestone Jun 14, 2023
gnprice added a commit that referenced this issue Jun 14, 2023
Filed #187, #188, and #190.

Other items already existed as #80, #82, #95, #122, and #123,
or were already complete (a compose box, and three ways of
attaching something to a message.)
@gnprice
Copy link
Member Author

gnprice commented Jul 6, 2023

We'll also want to keep these scripts in reusable form. It'll be useful to rerun them periodically, both as we make changes to the parser and to validate that there aren't new patterns we haven't implemented and are unaware of.

In particular when there's an invariant we believe applies and want to validate that empirically, we can add asserts for that invariant to the parser and then rerun the survey script, and see if it finds failures.

(We'll basically always want to run the script with assertions enabled, since the point of it is to find situations we didn't expect.)

@gnprice
Copy link
Member Author

gnprice commented Aug 14, 2023

(Separately, when we run into an unimplemented feature in a message we're trying to show in the message list, our current UI is pretty loudly explicit about that. That's helpful for development but we'll naturally want to handle it differently for normal use beyond the beta: #194.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a-api Implementing specific parts of the Zulip server API a-content Parsing and rendering Zulip HTML content, notably message contents
Projects
Status: No status
Development

No branches or pull requests

1 participant