Two weeks ago, I made this post. The goal was simple: I wanted to dig into the details of Chromium and Firefox to see if the claims that Chromium is more secure than Firefox are true or not. You'll notice I also started turning that post into an update log, but only one update got released. There is a reason for that. Life suddenly got extremely busy for me, I could barely make time to continue researching. However, during that time, I spent a lot of time thinking about the issue. I tried breaking down the problem in a million different ways to find a way to simplify it and start from the ground up.

I came to a conclusion today, a realization. I have no way to put this gently: I cannot conclusively determine which one is more secure. This will upset many of you, and it upsets me too considering I maintain my own list of software that relies on only providing the most secure and private versions of some software. I need to explain why there cannot be a solid conclusion.

I managed to collect many sources to be used for the research. A lot of the information is parroting this article which, despite having many sources, fails to provide sources for some of the most crucial claims made there ("Fission in its current state is not as mature as Chromium's site isolation" has no source, for example). My favorite source is this Stanford paper which I think does a great job at tackling the problem. The problem I noticed is that a lot of privacy advice is given from an echo chamber.

Think about what privacy advice you like to give, and think about where you heard that. A YouTube video? Reddit? Lemmy? Naomi Brockwell gives a lot of advice that stems directly from Michael Bazzell's Extreme Privacy book, as I found out after reading it. Her videos about convincing people to use Signal are paraphrased passages from the book itself, which has a whole section about it. People touting Chromium as more secure than Firefox, or that the Play Store is a more secure option than F-Droid or Aurora Store, often get their information from GrapheneOS. I've never seen anyone research those in depth.

The point I'm trying to make is that a lot of privacy advice is circular reporting. I'm certain that if Michael Bazzell and GrapheneOS were to provide sources as to where they got their information (they rarely do, I checked) it would come to light that it boils down to a few real sources. GrapheneOS, no doubt, likely has inspected at least some part of the Firefox codebase, but Firefox is rapidly changing, so any sources that used to be true may not be true today.

FUTO Keyboard and GrayJay get recommended often because of Louis Rossmann, but HeliBoard and FreeTube (or NewPipe) were options long before those pieces of software. The reason the former became so recommended over the latter is simply because people used a popular figure, Louis Rossmann, as a primary source. It then became an echo chamber of recommendations and best practices.

That doesn't mean the claims of Chromium being more secure are false, but as a researcher it is very hard to credit something that doesn't provide any primary sources. In the eyes of a researcher, GrapheneOS's word holds just as much weight as a random internet user, without any proof. I see it play out like this: A source like GrapheneOS or Extreme Privacy makes a claim, secondary sources such as GrapheneOS users or Naomi Brockwell present this information without providing the sources, the general privacy community sees both, and begin giving the same recommendations on Reddit or Lemmy (sometimes with sources), and eventually the privacy community as a whole starts presenting that information, without any primary sources. Even if GrapheneOS, Extreme Privacy, or Louis Rossmann provided no research or direct comparisons, their word is taken without question and becomes the overarching recommendations in the privacy community. They each gained credibility in their own ways, but there should always be scrutiny when making a claim, no matter how credible.

The main reason why I cannot give a concrete conclusion is this: the focus on the article was to compare Chromium's Site Isolation to Firefox's implementation, however there are too many variables at play. Chromium may be more secure on one Linux distro than another. Debian is an example. Firefox supposedly has worse site isolation on Linux, but then how does Tails deal with that? It's based on Debian, so does that make it insecure for both browsers? Tor is based on Firefox ESR, which is an extended support release with less security, but Tor is also deemed a better option than Chromium browsers for anonymity. Isolating iframes doesn't really affect daily use, so is it really necessary to shame Firefox for that? Some variants of Firefox harden the browser for security, but some variants of Chromium (such as Brave Browser) try to enhance privacy. No matter what limits I set, how many operating systems or browser variants I set, there is no way to quantify which one is more secure.

"Is Chromium more secure? Yes, under XYZ conditions, with ABC variants, on IJK operating systems. Chromium variants XYZ are good for privacy, but ABC Firefox variants are better at privacy..." The article would be a mess. The idea for the article came because I was truly sick of the lack of true in-depth sources about the matter, and so I wanted to create that. I now realize it was a goal that is far too ambitious for me, or even a small group of people. Tor and Brave give different approaches to fingerprinting protection (blending in vs. randomizing), and there's no way to directly compare the two. The same goes for the security of each. There is no "Tails" for Chromium, but there is no "Vanadium" for Firefox. There's no one to one comparison for the code, because some of it is outside of the browser itself.

I regret making that initial post, because it set unrealistic expectations. It focused on a problem that can't tell the whole picture, and then promised to tell that whole picture. At a point, it comes down to threat model. Do you really need to squeeze out that extra privacy or security? Is someone going to go through that much effort? You know how to spot dark patterns, you know not to use privacy invasive platforms. Take a reality check. Both Chromium and Firefox are better than any proprietary alternatives, that's a fact. Don't bother trying to find the "perfect" Linux distro or browser for privacy and security, because you already don't use Windows. Privacy is a spectrum, and as long as you at least take some steps towards that, you've already done plenty.

Be careful next time you hear a software recommendation or a best practice. Be careful next time you recommend software or a best practice. Always think about where you heard that, and do your own research. There are some problems that are impossible or infeasible to solve, so just pick what you feel is best. I really am sorry that I wasn't able to provide what I promised, so instead I will leave a few of the sources I found helpful, just in case another ambitious person or group decides to research the matter. Not all of these sources are good, but it's a place to start:

GrapheneOS responded to my requests for a comment after this post was made, here: https://lemmy.ml/post/22142738

https://www.cvedetails.com/version-list/0/3264/1/

https://en.wikipedia.org/wiki/Site_isolation

https://madaidans-insecurities.github.io/firefox-chromium.html

https://news.ycombinator.com/item?id=38588557

https://seclab.stanford.edu/websec/chromium/chromium-security-architecture.pdf

https://grapheneos.org/usage#web-browsing

https://www.reddit.com/r/browsers/comments/17vy1v5/reasons_firefox_is_more_secure_than_chrome/

https://www.wilderssecurity.com/threads/security-chromium-versus-firefox.450867/

https://forums.freebsd.org/threads/why-im-switching-from-firefox-to-ungoogled-chromium.87878/