OWD BCD project: increase mdn-bcd-collector coverage to 90%

[queengooborg]

Problem statement

Projects such as #85 and #160 that work to synchronize collector results with BCD are great. However, if the collector only covers a small set of the features BCD tracks, we can never guarantee complete accuracy.

And if we run the collector for new browser releases (#168), then ideally we should be able to update as many BCD data points as possible.

Command to get coverage stats: npm run feature-coverage -- -p api -p css -p html -p javascript -p mathml -p svg -c

Proposed solutions

This project is all about writing new tests to increase the collector's coverage of features tracked in BCD. The goal should be to get above 90% of overall coverage for testable features.

Task list

• Increase test coverage for APIs
• Increase test coverage for CSS
• Increase test coverage for HTML
• Increase test coverage for javascript
• If possible, add tests for MathML
• Increase test coverage for SVG

Progress Report

v10.10.2

api, css, html, javascript, mathml, svg: collector covers 12236 (90.02%) of 13593 entries tracked in bcd (1357 (9.98%) missing)

api: collector covers 8201 (95.15%) of 8619 entries tracked in bcd (418 (4.85%) missing)
css: collector covers 1841 (81.64%) of 2255 entries tracked in bcd (414 (18.36%) missing)
html: collector covers 549 (76.36%) of 719 entries tracked in bcd (170 (23.64%) missing)
javascript: collector covers 1202 (87.42%) of 1375 entries tracked in bcd (173 (12.58%) missing)
mathml: collector covers 29 (21.97%) of 132 entries tracked in bcd (103 (78.03%) missing)
svg: collector covers 414 (83.98%) of 493 entries tracked in bcd (79 (16.02%) missing)

v10.10.0 / v10.10.1

api, css, html, javascript, mathml, svg: collector covers 12202 (89.48%) of 13637 entries tracked in bcd (1435 (10.52%) missing)

api: collector covers 8167 (94.29%) of 8662 entries tracked in bcd (495 (5.71%) missing)
css: collector covers 1841 (81.64%) of 2255 entries tracked in bcd (414 (18.36%) missing)
html: collector covers 549 (76.25%) of 720 entries tracked in bcd (171 (23.75%) missing)
javascript: collector covers 1202 (87.42%) of 1375 entries tracked in bcd (173 (12.58%) missing)
mathml: collector covers 29 (21.97%) of 132 entries tracked in bcd (103 (78.03%) missing)
svg: collector covers 414 (83.98%) of 493 entries tracked in bcd (79 (16.02%) missing)

v10.9.0

api, css, html, javascript, mathml, svg: collector covers 11642 (88.30%) of 13184 entries tracked in bcd (1542 (11.70%) missing)

api: collector covers 8133 (94.34%) of 8621 entries tracked in bcd (488 (5.66%) missing)
css: collector covers 1369 (76.95%) of 1779 entries tracked in bcd (410 (23.05%) missing)
html: collector covers 549 (76.57%) of 717 entries tracked in bcd (168 (23.43%) missing)
javascript: collector covers 1173 (85.43%) of 1373 entries tracked in bcd (200 (14.57%) missing)
mathml: collector covers 29 (21.97%) of 132 entries tracked in bcd (103 (78.03%) missing)
svg: collector covers 389 (69.22%) of 562 entries tracked in bcd (173 (30.78%) missing)

v10.8.0

api, css, html, javascript, mathml, svg: collector covers 11613 (88.08%) of 13184 entries tracked in bcd (1571 (11.92%) missing)

api: collector covers 8128 (94.28%) of 8621 entries tracked in bcd (493 (5.72%) missing)
css: collector covers 1369 (76.95%) of 1779 entries tracked in bcd (410 (23.05%) missing)
html: collector covers 525 (73.22%) of 717 entries tracked in bcd (192 (26.78%) missing)
javascript: collector covers 1173 (85.43%) of 1373 entries tracked in bcd (200 (14.57%) missing)
mathml: collector covers 29 (21.97%) of 132 entries tracked in bcd (103 (78.03%) missing)
svg: collector covers 389 (69.22%) of 562 entries tracked in bcd (173 (30.78%) missing)

v10.7.2

api, css, html, javascript, mathml, svg: collector covers 11381 (84.83%) of 13416 entries tracked in bcd (2035 (15.17%) missing)

api: collector covers 8110 (92.06%) of 8809 entries tracked in bcd (699 (7.94%) missing)
css: collector covers 1174 (64.65%) of 1816 entries tracked in bcd (642 (35.35%) missing)
html: collector covers 524 (73.39%) of 714 entries tracked in bcd (190 (26.61%) missing)
javascript: collector covers 1155 (83.51%) of 1383 entries tracked in bcd (228 (16.49%) missing)
mathml: collector covers 29 (21.97%) of 132 entries tracked in bcd (103 (78.03%) missing)
svg: collector covers 389 (69.22%) of 562 entries tracked in bcd (173 (30.78%) missing)

v10.7.1

2227/13378 (16.65%) entries missing from collector that are in bcd for api, css, html, javascript, mathml, svg

727/8777 (8.28%) entries missing from collector that are in bcd for api
660/1810 (36.46%) entries missing from collector that are in bcd for css
190/715 (26.57%) entries missing from collector that are in bcd for html
347/1384 (25.07%) entries missing from collector that are in bcd for javascript
132/132 (100.00%) entries missing from collector that are in bcd for mathml
171/560 (30.54%) entries missing from collector that are in bcd for svg

v10.7.0

all: 2324/13356 (17.40%) entries missing from collector that are in bcd for api, css, html, javascript, mathml, svg

api: 724/8758 (8.27%) entries missing from collector that are in bcd
css: 670/1811 (37.00%) entries missing from collector that are in bcd
html: 257/711 (36.15%) entries missing from collector that are in bcd
javascript: 370/1384 (26.73%) entries missing from collector that are in bcd
mathml: 132/132 (100.00%) entries missing from collector that are in bcd
svg: 171/560 (30.54%) entries missing from collector that are in bcd

[queengooborg]

As of mdn-bcd-collector v10.10.2, we have now reached over 90% feature coverage! 🎉

[Elchi3]

@queengooborg can we update the feature coverage script to list untestable features along with the other stats?

So something like:

api, css, html, javascript, mathml, svg: collector covers 12236 (90.02%) of 13593 entries, 1357 (9.98%) missing, and XXX (x.xx%) are untestable.

[queengooborg]

I can update the script, though I'd have to change the output format (untestable features are omitted from the stats)!