Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

My domain is considered disposable #13230

Open
Torsteinws opened this issue Jul 17, 2023 · 21 comments
Open

My domain is considered disposable #13230

Torsteinws opened this issue Jul 17, 2023 · 21 comments

Comments

@Torsteinws
Copy link

Changing the email address to an email with the domain torsteinws.no gives the following error:

Cannot use disposable email addresses (Blocklist).

I would not call Norwegian top level domains disposable because they are actually difficult to get. In fact, I would argue that email addresses with Norwegian TLDs are more linked to your identity than other email addresses. Here is why:

In order to get a Norwegian TLD you have to:

  1. Be a Norwegian citizen that is nationally registered with a valid Norwegian birth ID and a valid Norwegian post address.
  2. Be over 18 years old.
  3. Own no more than 5 Norwegian TLDs.
  4. Verify all of the above with your national electronic ID called BankID

Furthermore, in order to get a BankID you have to physically meet up at a Norwegian bank to verify your identity. There you will have to show your passport and so forth.

@jxu
Copy link

jxu commented Jul 18, 2023

It may be just using a domain that isn't a known large email provider (assuming this is your personal domain)

@Torsteinws
Copy link
Author

Yes, it is my personal domain. I even use it as my username on lichess and github :-)

@Torsteinws
Copy link
Author

Sorry, I see that this is probably too much information. But I want to provide you a good context for the issue:

Email setup

  • I use proton mail as my main email provider. There I have a set of email addresses which I consider to be secret.
  • SimpleLogin is included in Proton Mail, so I use that service to create aliases for my secret email addresses on proton.

About SimpleLogin

I know that SimpleLogin has a bad reputation for creating burner addresses, but that is not the use case for me at all.
I use the service so that I can:

  1. Easily route my email to different inboxes. (I have different inboxes for different purposes such as shopping, subscriptions, business etc..)
  2. Easily do complex filtering, categorization and labeling of all incoming emails.
  3. Have an effective protection layer against spam.
  4. Use aliases to make it harder to aggregate leaked online data about me.

Q&A

Q: Why don't you use one of your secret proton addresses on Lichess? Lichess will after all never abuse this information!

A: If Lichess has a security breach my secret email address would be exposed all out in the open, and I would then permanently lose all protection against spam. I also consider this a potential threat to my privacy.

Q: Why don't you create an email address in proton that is only used for Lichess?

A: Proton only allows 15 email addresses in their paid plan, and I have unfortunately used them all.

Q: But you are using SimpleLogin, doesn't that mean that your email is disposable?

A: No, torsteinws.no is deeply tied to my identity. I always use torsteinws as my username on online services (just look at my github and lichess profile), and the domain was very hard to get due to all the restrictions that are imposed on Norwegian TLDs.

Q: Why are you so adamant on keeping your proton addresses secret?

A: Using aliases to keep the addresses secret is the best protection I have against spam and online privacy abuse. I intend to have these addresses for the rest of my life, and I must therefore follow a zero trust policy.

@notDavid
Copy link

I have the exact same issue, i'm using my own personal domain which i purchased myself and is only used by myself. Yet i get the error: Cannot use disposable email addresses (Blocklist).

Please fix, and block only the Simplelogin disposable email domains, they are:

  • simplelogin.com
  • aleeas.com
  • 8alias.com
  • slmails.com
  • silomails.com
  • 8shield.net
  • dralias.com
  • slmail.me
  • simplelogin.fr

@HexPandaa
Copy link

I'm facing the same registration issue using my own domain, I'm also using SimpleLogin to redirect my mail to my Proton Mail account.

@Adamatt
Copy link

Adamatt commented Feb 4, 2024

Exact same issue here.
I want to leave chess.com and use lichess instead, but playing against anonymous players is frustrating and i can't create an account.

Lichess should not blacklist personal domains even if they're tied to Simplelogin which can do a lot more than disposable emails (for which a known list exists).

@BourbonCrow
Copy link

Exact same issue here. I want to leave chess.com and use lichess instead, but playing against anonymous players is frustrating and i can't create an account.

Lichess should not blacklist personal domains even if they're tied to Simplelogin which can do a lot more than disposable emails (for which a known list exists).

100% agree my domain tied to protonmail works but the domain i use for simple login does not both are my own domains

@wincent
Copy link

wincent commented Feb 24, 2024

Not sure when the changes were made, but I registered almost three years ago with a custom domain. Tonight, trying to update my email address to a different custom domain, I ran into the dreaded "This email domain has a poor reputation and cannot be used" message, which seems a bit overzealous...

@ornicar
Copy link
Collaborator

ornicar commented Feb 25, 2024

Is hurrell.net the domain? Because lichess tells me it's fine.

I found your email address on your github profile.

@wincent
Copy link

wincent commented Feb 25, 2024

Nope, it's another domain that I use for signups. Pretty much never used for outbound email, only for receiving, which might be why it has "no" reputation.

@pmz
Copy link

pmz commented Mar 11, 2024

hey! I'm experiencing the same problem. My domains are hosted at Fastmail.
Using Fastmail provided addresses (ex: airpost.net) has the same problem:

Cannot use disposable email addresses (Blocklist).

@ornicar
Copy link
Collaborator

ornicar commented Mar 11, 2024

I just checked and lichess is telling me that airpost.net is ok?

@peszko1
Copy link

peszko1 commented Mar 19, 2024

I have also similar problem with rnrbros.com domain. It's my personal domain hosted at fastmail.com. Now I have Cannot use disposable email addresses (Blocklist). and at first try I had some message with email poor quality. So as a result I cannot register.

@ornicar
Copy link
Collaborator

ornicar commented Mar 19, 2024

verifymail.io, our disposable email detection provider, recommends to block that domain: https://verifymail.io/domain/rnrbros.com
I reckon you should take it to them.

@wincent
Copy link

wincent commented Mar 19, 2024

I reckon you should take it to them.

All the people with domains of their own could indeed take it to them, on a case-by-case basis, but it seems that their "privacy/alias" detection seems like a very blunt instrument. I'd go so far as to say it's broken. I checked a handful of domains I own that are not "privacy/alias" domains — they're just Fastmail-hosted domains — and this site misclassifies all of them as "privacy/alias". It seems like they have their "detection" dialed all the way up so as to drop false negatives to basically zero, at the cost of absurdly frequent false positives.

@ornicar
Copy link
Collaborator

ornicar commented Mar 19, 2024

I agree that's a problem... Yet we need some sort of throwaway email detection for 2 reasons:

  1. Add a third-party verification step that makes massive multi-accounting harder
  2. Make sure people can receive their "forgotten password" email.

That's literally all we use emails for, but it's quite important. We get dozens of support requests every day from people who used a throwaway email address and have forgotten their password.

I'll look into accepting email domains that verifymail.io classifies as "privacy/alias".

@maertsen
Copy link

I reckon you should take it to them.

All the people with domains of their own could indeed take it to them, on a case-by-case basis

Which is what I did, earlier today. I tried to sign-up, as a first time user, with a domain I'm 100% certain has never been used on this service before. They misclassified me.
I understand the value of a service like this; just speaking up to make false-positives visible too. I hope they get back to me.

@ornicar
Copy link
Collaborator

ornicar commented Mar 20, 2024

I deployed the changed above this morning.

@wincent
Copy link

wincent commented Mar 20, 2024

I deployed the changed above this morning.

Thanks for doing that.

I suspect this isn't going to make any difference to the people who've been active in this thread. Take, for example, https://verifymail.io/domain/rnrbros.com that you linked to above; that one is returning "Block: true", "Disposable: false", "Privacy: true". I checked all of my Fastmail-hosted domains, and they are all like that. I gather the "Block" recommendation is coming because they are "Privacy: true"? I am reasonably sure that in my case, the domains aren't being used for outbound abuse (Fastmail has SPF + DKIM set-up by default, lacking only DMARC).

As far as I can tell, "Privacy: true" is going to lead all Fastmail hosted domains to have a "Block: true" recommendation, based on what I see in the verifymail.io FAQ:

What are privacy email addresses?
Many email providers are privacy-focused, which can include awesome features such as email forwarding or unlimited email alias. These features are great for their users, however can be a nightmare for administrators or moderators who can't distingish between a legitimate email address or a single-use email address. The "privacy" classification allows us to notify our clients when an email address is using an email provider with privacy features that could potentially be misused by malicious individuals. This classification does not mean that the email provider is "privacy-focused". Use this classification at your own discretion, since these types of emails can be used by both privacy-conscious visitors and bad actors.

@maertsen
Copy link

I suspect this isn't going to make any difference to the people who've been active in this thread. Take, for example, https://verifymail.io/domain/rnrbros.com that you linked to above; that one is returning "Block: true", "Disposable: false", "Privacy: true". I checked all of my Fastmail-hosted domains, and they are all like that.

I think your observation is correct, but your conclusion does not follow from the code for me.

I looked at 38419c0, which gives me the impression only Disposable is relevant after the change, Block and Privacy are logged too (two lines down), but no longer affect $ok.

To add to your observation: verifymail got back to me and confirmed that "Block: true", "Disposable: false", "Privacy: true" is the correct response for my personal domain in their view. The remainder of their e-mail is along the lines of the FAQ item you quoted.

Now, the website as currently deployed still rejects my domain in the registration flow, with error Cannot use disposable email addresses (Blocklist).. So either I misunderstand how this works (which would be unsurprising) or perhaps there's caching involved?

@maertsen
Copy link

Looking a bit further (oh FOSS ❤️‍🔥), I think the relevant bits of code are:

Possible error messages:

object EmailAddressValidator:
enum Result(val error: Option[String]):
def valid = error.isEmpty
case Passlist extends Result(none)
case Alright extends Result(none)
case DomainMissing extends Result("The email address domain is missing.".some) // no translation needed
case Blocklist extends Result("Cannot use disposable email addresses (Blocklist).".some)
case DnsMissing extends Result("This email domain doesn't seem to work (missing MX DNS)".some)
case DnsTimeout extends Result("This email domain doesn't seem to work (timeout MX DNS)".some)
case DnsBlocklist
extends Result(
"Cannot use disposable email addresses (DNS blocklist).".some
)
case Reputation extends Result("This email domain has a poor reputation and cannot be used.".some)

Blacklist checked first, then DNS, then verifymail:

else if disposable(domain.into(Domain)) then fuccess(Result.Blocklist)
else
dnsApi
.mx(domain)
.flatMap: domains =>
if domains.isEmpty then fuccess(Result.DnsMissing)
else if domains.exists(disposable.asMxRecord) then fuccess(Result.DnsBlocklist)
else
verifyMail(domain).map: ok =>
if ok then Result.Alright else Result.Reputation

verifymail API calls are cached

private val cache = mongoCache[Domain.Lower, Boolean](512, prefix, 100 days, _.toString): loader =>
_.maximumSize(512).buildAsyncFuture(loader(fetch))

that blocklist is constructed at startup and refreshed every so often

if config.disposableEmail.enabled then
scheduler.scheduleOnce(33 seconds)(disposableEmailDomain.refresh())
scheduler.scheduleWithFixedDelay(
config.disposableEmail.refreshDelay,
config.disposableEmail.refreshDelay
): () =>
disposableEmailDomain.refresh()

like this

private[security] def refresh(): Unit =
for
blacklist <- ws.url(providerUrl).get().map(_.body[String].linesIterator).recover { case e: Exception =>
logger.warn("DisposableEmailDomain.refresh", e)
Iterator.empty
}
checked <- verifyMailBlocked()
do
val regexStr = s"${toRegexStr(blacklist)}|${toRegexStr(checked.iterator)}"

defined as

private lazy val disposableEmailDomain = DisposableEmailDomain(
ws = ws,
providerUrl = config.disposableEmail.providerUrl,
verifyMailBlocked = () => verifyMail.fetchAllBlocked

using all cached domains that failed the verifymail API

// expensive
private[security] def fetchAllBlocked: Fu[List[String]] =
cache.coll
.distinctEasy[String, List](
"_id",
$doc(
"_id".$regex(s"^$prefix:"),
"v" -> false
),
_.sec
)
.map: ids =>
val dropSize = prefix.length + 1
ids.map(_.drop(dropSize))

So the good news is: worst case this bug solves itself in 100 days, going by the default expiration logic 😉

I did not play (li)chess tonight, but I still had some fun. Thank you for your work on this project, the passion shows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests