Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Carry out research about pip documentation #8517

ei8fdb opened this issue Jul 1, 2020 · 4 comments

Carry out research about pip documentation #8517

ei8fdb opened this issue Jul 1, 2020 · 4 comments
type: docs Documentation related UX User experience related


Copy link

ei8fdb commented Jul 1, 2020

Research questions:

i.e. what do we want to know?

  • When pip users have a problem or experience an error, what do they do? (note: crossover with Carry out research to understand who uses pip #8518)
  • When pip users want to accomplish a specific task, what do they do? (note: crossover with Carry out research to understand who uses pip #8518)
  • How do users use pip's current documentation?
    • Does it provide good orientation / is the structure clear?
    • Does the documentation look "official"?
      • Does it look trustworthy?
      • Does it look like an "official" Python property?
    • Do users understand the terminology?
  • How do users distinguish between pip documentation and other documentation in the packaging ecosystem (e.g., PyPI help page, etc.)?
  • How do pip/packaging maintainers think about documentation and write documentation?
    • Is it obvious where documentation should go?
    • What ideas do they have about improving the docs?
    • Are there (written or unwritten) documentation standards?

Preliminary research plan

i.e. how are we going to find all that out?

Note: We should expect some foundational insights from @ei8fdb's work on Carry out research to understand who uses pip

Some initial ideas:

  • Investigate if we can use analytics to draw conclusions about pip's docs (e.g. time on page, etc.)
  • Gather feedback while users are on pip's documentation (e.g. allow users to send feedback in real time)
  • Conduct user interviews with a range of pip users
  • Ask users to participate in journal studies
  • Collect feedback from the community via surveys
  • Conduct interviews with packaging maintainers


i.e. where can we go for help?


i.e. who can help us?

  • @evildmp

    • Review research plan
    • Review outputs / recommendations (see below)
    • Periodic involvement as docs are rebuilt
  • @jdittrich

    • Assist in collecting and analysing data
    • Note: Jan has an academic interest in how people understand documentation (PHD subject)


i.e. what will we have at the end of all this?

  1. Documentation about pip's documentation - how it is used, it's shortcomings, etc.
  2. Strategy for how to restructure / rewrite pip's documentation
  3. Guide for how to contribute to pip's documentation, outlining:
    • documentation best practices / principles
    • copywriting styleguide / glossary of terms

Questions for the team

To be discussed with the pip maintainers

  • Which research questions (from the list above) are most important?
  • What is the best way to involve the team when working out the strategy for restructuring / rewriting the docs?
    • Should we run a workshop on this?
    • Should UX periodically present our ideas for feedback?
      • How often?
      • What channel? (team meeting, HackMD review, GitHub, other?)
  • How can we put in place the resources / people to execute the strategy?
    • Is it within the scope of the currently funded work?
    • If not, how can we move forward on this?
@ei8fdb ei8fdb added the UX User experience related label Jul 1, 2020
@pypa pypa locked and limited conversation to collaborators Jul 1, 2020
@nlhkabu nlhkabu added the type: docs Documentation related label Jul 28, 2020
Copy link
Contributor Author

ei8fdb commented Nov 19, 2020

Adding this here after @pradyunsg tagged me and others in PR #9146.

My first question is what questions do we want to answer with analytics? Gathering ideas on what we want to find out would help us decide.

Here are my thoughts -

To understand how users are using the documentation

The following data is commonly gathered in order to understand how documentation is used :

  • what pages are most visited? (e.g. top 10/100)
  • how long do users spend on each page?
  • inbound referrers
  • outbound links
  • what paths users take through the documentation

With the above data we would be able to understand how the documentation is used, decide on what pages need to be updated frequently, what pages need to be tested with users, what pages can be removed/rationalised.

Helpful, but needs further thought

The following information would be helpful to have, tho' would need to be discussed to make sure it was implemented in the right way.

  • the language the user speaks or the coarse geographic region the user has visited from

This would give us rough understanding of the language the user may speak - useful when deciding on translation.

  • the coarse device type information - mobile/tablet/laptop/desktop/other device

This would be useful to understand about information layout

Consent and transparency

My opinion would be analytics would be opt-in for the documentation visitor. Explaining why we requst permission to gather the data should be communicated.

It would also be important to be transparent with the community about what type of data we are gathering, and what it is being used for.

I'd like to suggest discussion about making the above data public. It would be important to make sure we were not gathering IP address, granular geographic information. This would show the community what data is generated, and (I would think) be interesting for the community to see.

Copy link

pfmoore commented Nov 19, 2020

For some reason I can't apply a 👍 emoji to the above comment, so I'm putting it here.

@pypa pypa unlocked this conversation Nov 19, 2020
Copy link

pradyunsg commented Nov 19, 2020

It was a locked issue, which, doesn't allow reactions.

Copy link
Contributor Author

ei8fdb commented Nov 19, 2020

It was a locked issue, which, doesn't allow reactions.

I've unlocked it now. Don't remember why... 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
type: docs Documentation related UX User experience related
None yet

No branches or pull requests

4 participants