Skip to content

Commit

Permalink
Add back Cula and DHSH crawlers
Browse files Browse the repository at this point in the history
  • Loading branch information
MaxGiting committed Aug 25, 2020
1 parent b524ab1 commit a934d7f
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 1 deletion.
2 changes: 1 addition & 1 deletion raw/Crawlers.json

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions raw/Crawlers.txt
Expand Up @@ -5,6 +5,7 @@
^COMODO DCV
^DangDang
^DavClnt
^DHSH
^FDM
^git\/
^Goose\/
Expand Down Expand Up @@ -230,6 +231,7 @@ Crowsnest
Criteo
CSHttp
CSSCheck
Cula\/
curb
Curious George
curl
Expand Down
2 changes: 2 additions & 0 deletions src/Fixtures/Crawlers.php
Expand Up @@ -26,6 +26,7 @@ class Crawlers extends AbstractProvider
'^COMODO DCV',
'^DangDang',
'^DavClnt',
'^DHSH',
'^FDM ',
'^git\/',
'^Goose\/',
Expand Down Expand Up @@ -251,6 +252,7 @@ class Crawlers extends AbstractProvider
'Criteo',
'CSHttp',
'CSSCheck',
'Cula\/',
'curb',
'Curious George',
'curl',
Expand Down
7 changes: 7 additions & 0 deletions tests/crawlers.txt
Expand Up @@ -3527,3 +3527,10 @@ Amazon CloudFront
swcd (unknown version) CFNetwork/1125.2 Darwin/19.4.0
YahooMailProxy; https://help.yahoo.com/kb/yahoo-mail-proxy-SLN28749.html
Mozilla/5.0 (Java) outbrain
DHSH
DHSH Virus Check
DHSH Ads Check
DHSH Statistik Chec
DHSH Law Check
DHSH User Check
Mozilla/5.0+(compatible; Cula/2.0; https://cula.io/)

0 comments on commit a934d7f

Please sign in to comment.