pastebin.com Content Monitoring Tool
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
README
pastemon.conf.sample
pastemon.pl
proxies.conf
regex.conf.sample
user-agents.conf

README

Introduction
------------

pastemon.pl is a script which runs in the background as a daemon and monitors pastebin.com for
interesting content (based on regular expressions). Found information is sent to syslog

The script can also generate (CEF events).

More information is available here: 
http://blog.rootshell.be/2012/01/17/monitoring-pastebin-com-within-your-siem/

v1.14 - 2012/10/31
------------------
- [FEATURE] Added SQLite DB support to store pasties details
  (some fields must still be implemented)

v1.13 - 2012/10/24
------------------
- [CONTRIBUTION] Added support for multiple SMTP recipients (email addresses separared by commas)
  Contribution from coreyroach@hotmail.com
- [CONTRIBUTION] Added a new macro-% to specify the site name in the dump function.
  '%S' will be replaced by the site name. Example: '%S/%Y/%M' => 'pastebin.com/2012/10'. 

v1.12 - 2012/09/20
------------------
- [BUGFIX] Fixed FuzzyMatch() which was broken with gziped pasties.
- [FEATURE] Email notification: The subject is now appended with the <description> field(s)
  corresponding to the matched regex(es). This allows a better view of received emails as well
  as filtering them.
- [BUGFIX] Fixed FuzzyMatch() to detect properly duplicate pasties (fixed regex).

v1.11 - 2012/09/14
------------------
- [FEATURE] Added support for nopaste.me.
- [FEATURE] Added support for pastesite.com.
- [FEATURE] Added configurable sleep delays per pastie website.

v1.10 - 2012/08/01
------------------
- [FEATURE] If not configuration file is specified, pastemon.pl tries to load /etc/pastemon.conf
  by default.

- [FEATURE] pastemon.pl uses specific Perl modules like WordPress::XMLRPC or Text::JaroWinkler.
  The script now handles properly environment without those modules. It's not required to comment
  them in the code. If a module is missing, related configuration is automatically disabled.

- [FEATURE] Added optional compression (via IO::Compress::Gzip) of dumped pasties.
  In configuration file:

	<core>
		<compress-pasties>yes</compress-pasties>
	</core>

v1.9 - 2012/07/23
-----------------
- [FEATURE] pastemon can now follow (search for regex) URLs detected in pasties. This is
  configured via the main configuration file:

	<urls>
		<follow>yes</follow>
		<matching>(bit\.ly)</matching>
	</urls>

- [FEATURE] The regex.conf format changed to an XML format.
  Examples:
	<regex>
		<search>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}</search>
		<count>10</count>
		<description>IP Address</description>
	</regex>

- [FEATURE] A minimum number of regex occurences can be defined to notify
  (<count> tag in the XML file)

- [FEATURE] HTTP requests are now using now a random User-Agent. 

- [BUGFIX] Optimized the detection of already processed pasties. This reduces the amount of HTTP 
  requests send to the website.

v1.8 - 2012/06/25
-----------------
- [FEATURE] Adder for pastie.org! 
- [FEATURE] Added multi-thread support (1 thread per website monitored)
- [FEATURE] Added substitution macro in the dump directory. Support macros are:
	%Y - replace with the current year
	%M - replace with the current month
	%D - replace with the current day
  Directory is automatically created.
  Example: /home/user/pastemon/%Y/%M/%D

- [FEATURE] Added a new configuration directive:
	<dump-all>yes|1</dump-all>
  This feature enables a dump of *ALL* pastie wheter they match a regex or not.
  This is similar to a mirror mode
  WARNING: Huge disk space might be required by this feature!
- [BUGFIX] Test if the provided SMTP server (for mail notifications) is available
  (Thanks to @manuelsubredu for the patch)
- [BUGFIX] Fixed an issue in createBlogPost() which caused an unexpected process exit.

v1.7 - 2012/05/11
-----------------
- Added support for "included" regular expressions
- Fixed in bug in getRegexDesc()
- Added support for comments ('#') in the regex configuration file
- Moved configuration parameters from command line switches to an XML file
- Added matching regex description in dump files 
- Added SMTP notifications
- Added distance check to detect duplicate pasties (using Jaro-Winkler algorithm)

v1.6 - 2012/02/21
-----------------
- Added a detection of "slow down" messages returned by Pastebin (add a small pause)
- Added support for Wordpress XMLRPC 
- Added support for random proxies
- Some bug fixes

v1.5 - 2012/02/19
-----------------
- Fixed the regex to grab pasties from the archive page. (HTML code changed)

v1.4 - 2012/02/15
-----------------
- Fixed a bug with CEF events: custome fields start at 1 not 0! (Thanks to Heiko Hansen for the report)
- Notify the presence of a proxy variable (HTTP_PROXY) 

v1.3 - 2012/01/26
-----------------
- Added a '--pidfile=file' configuration switch to specify an alternative location for the PID file.
  This allows the script to be executed with a non-root account.
- Added a '--sample=x' configuration to display a sample a data matching a regular expression. 'x' is 
  the number of bytes displayed before and after the matching string. This is useful to estimate the
  value of the pastie.  Example:
  Found in http://pastebin.com/raw.php?i=Q8pQRHKW : belgium (2 times) | Sample: g(0) ""\n  [32] => string(11) "Belgium(32)"\n  [31] => string(14) "Ne

v1.2 - 2012/01/21
-----------------
- Fixed a bug affecting the case sensitivity search
- New feature: an exception can be associated to a regular expression in the configuration file.
  The syntax is: "regex1 _EXCLUDE_ regex2". This could prevent some false positive matches.

v1.1 - 2012/01/20
-----------------
- Added a '--dump' configuration switch to save matching pasties in a directory.
  This is to keep the pasties posted with an expiration date (example: for later review)

v1.0 - 2012/01/18
-----------------
Initial release