Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs(new-audits): revise issue template #6233

Merged
merged 3 commits into from
Oct 13, 2018
Merged

docs(new-audits): revise issue template #6233

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2fbe7cf
docs(new-audits): revise issue template
paulirish Oct 12, 2018
6dc2ab5
Update new-audits.md
paulirish Oct 12, 2018
402800e
feedback
paulirish Oct 12, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions docs/new-audits.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,24 @@ Lighthouse audits that surface in the report should be:
- Measurable (especially for performance audits) or have clear pass/fail states.
- Not use 3rd party APIs for completing the audit check.

*Note: it's possible to submit a new audit proposal for something that you as a contributor feel very passionate about, but the Lighthouse team feels that it wouldn't add value to a significant portion of the Lighthouse users. In this case, we'd be open to having your audit be a part of the report when run with the full config on the Command Line.*

## Process for creating a new audit
1. Briefly scan the criteria we’ve laid out above. If you think the principles match with your proposed new audit, then proceed!
2. Next step is to create an issue on GitHub with answers to the following questions:
- Description of audit and audit category (please include pass/fail states, and how it might be measured)
- Explanation of how it’s different from other audits
- What % of developers/pages will this impact (estimates OK, data points preferred)
- How is the new audit making a better web for end users? (data points preferred)
- What is the resourcing situation (who will create the audits, maintain the audits, and write/maintain the documentation)
- Do you envision this audit in the Lighthouse report or the full config in the CLI? If in the report, which section?
- How much support is needed from the Lighthouse team?
- Any other links or documentation that we should check out?
```
#### Provide a basic description of the audit
#### How would the audit appear in the report?
- What detailed info is shown in all cases, or just failure cases?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a weird way of phrasing this, but I don't have a great alternative that's also short. What's the goal here?

  • what are all the possible states (pass and fail) of the test results
  • what information should be shown in each case to help the user improve
    ?
    (but phrased better :)

- How would passing/failing be defined?
#### How is this audit different from existing ones?
#### What % of developers/pages will this impact?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I've always thought this one is a weaker version of what we really want to know here which is "What is the global total impact of this audit?" An audit that affects 100% of developers so insignificantly that no one will notice is not as good as an audit that affects half and has a huge impact to each.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the pro/con weighting for it is also different than it once was because now we have notApplicable and can just hide it, so visual noise is less of an issue (though still a concern in the limit). Now it's more maintenance/run time cost as the potential negative.

_(Estimates OK, data points preferred)_
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put <!--comments--> around this instead of italics?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_? Trying to format or something else?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the gh issue they would paste this into this would show up as _italics_

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I see. Yeah that is confusing if you aren't copy/pasting :)

#### How is the new audit making a better web for end users?
_(Data points preferred)_
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put <!--comments--> around this instead of italics?

#### What is the resourcing situation?
- Who will create the audits, write the documentation, and maintain both?
#### Any other links or documentation that we should check out?
```
3. Once the proposal is submitted, then Lighthouse team will take a look and followup. We will discuss possible implementation approaches, and associated runtime overhead.
With this new information we can better understand the impl cost and effort required and prioritize the audit into our sprint/roadmap.
4. Depending on the prioritization, we'll then work with you to figure out the necessary engineering/UX/product details.