Skip to content

Commit

Permalink
use neutral language for naming and describing assets in this repo
Browse files Browse the repository at this point in the history
  • Loading branch information
martenson committed Oct 8, 2018
1 parent 4b8a6bb commit 7448bb6
Show file tree
Hide file tree
Showing 4 changed files with 40 additions and 40 deletions.
52 changes: 26 additions & 26 deletions README.md
Expand Up @@ -3,21 +3,21 @@ List of disposable email domains

[![Licensed under CC0](https://licensebuttons.net/p/zero/1.0/88x31.png)](https://creativecommons.org/publicdomain/zero/1.0/)

This repo contains a [list of disposable and temporary email address domains](disposable_email_blacklist.conf) often used to register dummy users in order to spam/abuse some services.
This repo contains a [list of disposable and temporary email address domains](disposable_email_blocklist.conf) often used to register dummy users in order to spam/abuse some services.

Originally collected to filter new user registration at https://usegalaxy.org and later merged with other lists found online. I cannot guarantee all of these can still be considered disposable but they probably were at one point in time.

Whitelist
Allowlist
=========
The file [whitelist.conf](whitelist.conf) gathers email domains that are often identified as disposable but in fact are not.
The file [allowlist.conf](allowlist.conf) gathers email domains that are often identified as disposable but in fact are not.

Example Usage
=============
**Python**
```Python
blacklist = ('disposable_email_blacklist.conf')
blacklist_content = [line.rstrip() for line in blacklist.readlines()]
if email.split('@')[1] in blacklist_content:
blocklist = ('disposable_email_blocklist.conf')
blocklist_content = [line.rstrip() for line in blocklist.readlines()]
if email.split('@')[1] in blocklist_content:
message = "Please enter your permanent email address."
return (False, message)
else:
Expand All @@ -36,9 +36,9 @@ True
1. Make sure the passed email is valid. You can check that with [filter_var](https://secure.php.net/manual/en/function.filter-var.php)
2. Make sure you have the mbstring extension installed on your server
```php
function isDisposableEmail($email, $blacklist_path = null) {
if (!$blacklist_path) $blacklist_path = __DIR__ . '/disposable_email_blacklist.conf';
$disposable_domains = file($blacklist_path, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
function isDisposableEmail($email, $blocklist_path = null) {
if (!$blocklist_path) $blocklist_path = __DIR__ . '/disposable_email_blocklist.conf';
$disposable_domains = file($blocklist_path, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$domain = mb_strtolower(explode('@', trim($email))[1]);
return in_array($domain, $disposable_domains);
}
Expand All @@ -47,12 +47,12 @@ function isDisposableEmail($email, $blacklist_path = null) {

In resource model, usually it is `user.rb`
```Ruby
before_validation :reject_email_blacklist
before_validation :reject_email_blocklist

def reject_email_blacklist
blacklist = File.read('config/disposable_email_blacklist.conf').split("\n")
def reject_email_blocklist
blocklist = File.read('config/disposable_email_blocklist.conf').split("\n")

if blacklist.include?(email.split('@')[1])
if blocklist.include?(email.split('@')[1])
   errors[:email] << 'invalid email'
return false
else
Expand All @@ -68,7 +68,7 @@ end
const readline = require('readline'),
fs = require('fs');

const input = fs.createReadStream('./disposable_email_blacklist.conf'),
const input = fs.createReadStream('./disposable_email_blocklist.conf'),
output = [],
rl = readline.createInterface({input});

Expand All @@ -82,7 +82,7 @@ rl.on('line', (line) => {
rl.on('close', () => {
try {
const json = JSON.stringify(output);
fs.writeFile('disposable_email_blacklist.json', json, () => console.log('--- FINISHED ---'));
fs.writeFile('disposable_email_blocklist.json', json, () => console.log('--- FINISHED ---'));
} catch (e) {
console.log(e);
}
Expand All @@ -91,33 +91,33 @@ rl.on('close', () => {

**C#**
```C#
private static readonly Lazy<HashSet<string>> _emailBlackList = new Lazy<HashSet<string>>(() =>
{
var lines = File.ReadLines("disposable_email_blacklist.conf")
private static readonly Lazy<HashSet<string>> _emailBlockList = new Lazy<HashSet<string>>(() =>
{
var lines = File.ReadLines("disposable_email_blocklist.conf")
.Where(line => !string.IsNullOrWhiteSpace(line) && !line.TrimStart().StartsWith("//"));
return new HashSet<string>(lines, StringComparer.OrdinalIgnoreCase);
});

private static bool IsBlacklisted(string domain) => _emailBlackList.Value.Contains(domain);
private static bool IsBlocklisted(string domain) => _emailBlockList.Value.Contains(domain);

...

var addr = new MailAddress(email);
if (IsBlacklisted(addr.Host)))
throw new ApplicationException("Email is blacklisted.");
if (IsBlocklisted(addr.Host)))
throw new ApplicationException("Email is blocklisted.");
```

Contributing
============
Feel free to create PR with additions or request removal of some domain (with reasons).

Use
Use

`$ cat disposable_email_blacklist.conf your_file | tr '[:upper:]' '[:lower:]' | sort -f | uniq -i > new_file.conf`
`$ cat disposable_email_blocklist.conf your_file | tr '[:upper:]' '[:lower:]' | sort -f | uniq -i > new_file.conf`

`$ comm -23 new_file.conf whitelist.conf > disposable_email_blacklist.conf`
`$ comm -23 new_file.conf allowlist.conf > disposable_email_blocklist.conf`

to add contents of another file in the same format (only second level domains on new line without @). It also converts uppercase to lowercase, sorts, removes duplicates and removes whitelisted domains.
to add contents of another file in the same format (only second level domains on new line without @). It also converts uppercase to lowercase, sorts, removes duplicates and removes allowlisted domains.

Changelog
============
Expand All @@ -126,4 +126,4 @@ Changelog

* 12/6/16 - Available as [PyPI module](https://pypi.org/project/disposable-email-domains) thanks to [@di](https://github.com/di)

* 7/27/16 - Converted all domains to the second level. This means that starting from [this commit](https://github.com/martenson/disposable-email-domains/commit/61ae67aacdab0b19098de2e13069d7c35b74017a) the implementers should take care of matching the second level domain names properly i.e. `@xxx.yyy.zzz` should match `yyy.zzz` in blacklist more info in [#46](https://github.com/martenson/disposable-email-domains/issues/46)
* 7/27/16 - Converted all domains to the second level. This means that starting from [this commit](https://github.com/martenson/disposable-email-domains/commit/61ae67aacdab0b19098de2e13069d7c35b74017a) the implementers should take care of matching the second level domain names properly i.e. `@xxx.yyy.zzz` should match `yyy.zzz` in blocklist more info in [#46](https://github.com/martenson/disposable-email-domains/issues/46)
File renamed without changes.
File renamed without changes.
28 changes: 14 additions & 14 deletions verify.py
@@ -1,6 +1,6 @@
#!/usr/bin/env python

"""Verify the integrity of the domain blacklist
"""Verify the integrity of the domain blocklist
"""

import io
Expand All @@ -11,11 +11,11 @@
from requests import get


blacklist = "disposable_email_blacklist.conf"
whitelist = "whitelist.conf"
blocklist = "disposable_email_blocklist.conf"
allowlist = "allowlist.conf"

files = {
filename: open(filename).read().splitlines() for filename in [whitelist, blacklist]
filename: open(filename).read().splitlines() for filename in [allowlist, blocklist]
}


Expand Down Expand Up @@ -44,7 +44,7 @@ def check_for_public_suffixes(filename):
print(
"At least one valid public suffix found in {!r}, please "
"remove it. See https://publicsuffix.org for details on why this "
"shouldn't be blacklisted.".format(filename)
"shouldn't be blocklisted.".format(filename)
)
sys.exit(1)

Expand Down Expand Up @@ -91,19 +91,19 @@ def check_for_intersection(filename_a, filename_b):
if __name__ == "__main__":

# Check if any domains have a public suffix
check_for_public_suffixes(blacklist)
check_for_public_suffixes(blocklist)

# Check if any domains are not lowercase
check_for_non_lowercase(whitelist)
check_for_non_lowercase(blacklist)
check_for_non_lowercase(allowlist)
check_for_non_lowercase(blocklist)

# Check if any domains are duplicated in the same list
check_for_duplicates(whitelist)
check_for_duplicates(blacklist)
check_for_duplicates(allowlist)
check_for_duplicates(blocklist)

# Check if any lists are not sorted
check_sort_order(whitelist)
check_sort_order(blacklist)
check_sort_order(allowlist)
check_sort_order(blocklist)

# Check if any domains are in both the whitelist and blacklist
check_for_intersection(whitelist, blacklist)
# Check if any domains are in both the allowlist and blocklist
check_for_intersection(allowlist, blocklist)

0 comments on commit 7448bb6

Please sign in to comment.