Suggestion for Inclusion on Privacytests.org

[iAnonymous3000]

I'm writing to suggest that the Vanadium browser should be included on privacytests.org, a website that conducts rigorous tests on various browsers to evaluate their privacy and security features.

Given the increasing interest in privacy-centric browsers and the unique hardening features that Vanadium provides as a part of the GrapheneOS project, Vanadium's performance in these tests would provide valuable information to users seeking secure and private browsing experiences.

Privacytests.org conducts various tests, including cookie handling, tracker blocking, fingerprinting resistance, etc. Seeing Vanadium's performance in these tests would give a clear picture of how it stands against other popular browsers regarding privacy and security.

Additionally, having Vanadium included in these tests would provide a benchmark for future development and a way to track improvements in the browser's privacy and security capabilities over time.

I encourage you to consider this suggestion and reach out to the team at privacytests.org to explore the possibility of including Vanadium in their testing suite.

Thank you for your time and consideration.

[thestinger]

Vanadium isn't intended for use outside of GrapheneOS and we don't make it available for use outside GrapheneOS. Vanadium also isn't yet a major focus of GrapheneOS. It will be in the future, but it's fairly unlikely we'll make it available for use elsewhere even once it has more of the planned features.

State Partitioning tests, Navigation tests, HTTPS tests and Fingerprinting resistance tests are fine as concepts, although there are mistakes in how the testing is done which give false positives and false negatives. Fingerprinting resistance section is extremely incomplete and attempting to portray it as a list of features diverging from mainstream browsers is fundamentally misleading. It's not the reality of how fingerprinting works in practice or what's possible.

We don't agree with the DNT and GPC approach, and therefore the Misc tests section is problematic for that reason. We also disagree with portraying Tor as strictly a positive feature and implying superiority to another approach such as Apple's nested VPN. There are substantial advantages to a user choosing several parties for nested hops over a random route through an untrustworthy network with many actively malicious relays and especially exit nodes. Many exit nodes have intermittently blocked access to GrapheneOS services due to their admins being malicious towards our project. It is not strictly positive.

We fundamentally disagree with the approach of enumerating badness. Tracking query parameter tests and Tracker content blocking tests fundamentally go against achieving real privacy instead of hard-wiring specific special cases until websites simply adjust to work around it as Facebook has started doing by encrypting parameters together. It only helps in certain special cases, and it's fine to do it but we don't think it should be presented as if it's on the same level as real fundamental privacy improvements.

We disagree with enough of this test suite that we do not want Vanadium included and aren't going to promote it. We will likely add some enumerating badness features to Vanadium eventually, but we see that as fundamentally different from serious privacy features. It's an opportunistic, trivial to bypass approach.

[arthuredelstein]

Hi @thestinger -- thank you for your detailed comments here. Thanks also to @iAnonymous3000 for opening this thread.

To try to find some better mutual understanding on this subject, I respond to some of your comments below. I'd like to mention at the start that PrivacyTests is not intended to provide a "score" for web browsers. Indeed no browser passes all tests, and that's to be expected -- different browsers are targeted to different user segments and thus make different design tradeoffs. Rather the tests are meant to provide transparency to different aspects of browser privacy and make it possible to see which browsers are putting in a strong privacy effort, and which aren't.

State Partitioning tests, Navigation tests, HTTPS tests and Fingerprinting resistance tests are fine as concepts, although there are mistakes in how the testing is done which give false positives and false negatives.

Please could you share details of false positive or negatives to help me track this down? I make every effort to give accurate results and would be very grateful for any information that leads to fixing a bug in the tests. Someone from GrapheneOS previously made similar comments, but unfortunately there isn't enough information in either thread to track down any issues.

Fingerprinting resistance section is extremely incomplete and attempting to portray it as a list of features diverging from mainstream browsers is fundamentally misleading. It's not the reality of how fingerprinting works in practice or what's possible.

I agree that the fingerprinting resistance testing section is very incomplete: I intend to expand it in the future. But I'm not sure what the rest of your comment means. These tests are testing exactly how screen and font fingerprinting resistance works in Tor Browser, Brave, Mull, etc (as reflected in the test results). But I am open to expanding the testing approach to cover any other sort of effective protection against screen and font fingerprinting -- let me know if you have suggestions.

We fundamentally disagree with the approach of enumerating badness. Tracking query parameter tests and Tracker content blocking tests fundamentally go against achieving real privacy instead of hard-wiring specific special cases until websites simply adjust to work around it as Facebook has started doing by encrypting parameters together. It only helps in certain special cases, and it's fine to do it but we don't think it should be presented as if it's on the same level as real fundamental privacy improvements.

In practice, I'm not looking only for "fundamental" privacy improvements. I'm also interested in pragmatic approaches to privacy that work. Blocking trackers is very effective and provides very real privacy in practice: a tracker can't track you if it's never loaded. Tracker blocking also provides "defense in depth" by complementing the more universal protections such as fingerprinting resistance, which are never perfect. Likewise, because tracking query parameters are a widely-used tracking method, it is currently very beneficial to privacy to block them. Both kinds of blocking work well today despite the risk that the situation might evolve in the future.

Thanks again for hearing me out, @thestinger. I hear great things about GrapheneOS and I hope one day to learn more about Vanadium's privacy approach.