New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix multiple bugs with the API calls and queue system #40

Merged
merged 48 commits into from Jul 25, 2017

Conversation

Projects
None yet
2 participants
@cixtor
Contributor

cixtor commented Jul 3, 2017

This pull-request includes all the bug fixes applied to the code before version 1.8.7 was released.

This also includes a fix for the display of the newsletter invitation which was appearing in places that we didn't intend it to appear like the result of the plugin updates and other parts of the admin dashboard. From now on, the plugin will only display this invitation inside the pages associated with the plugin; notice that this modification also excludes the Ajax requests.

This pull-request also changes the status of the three hardening options that suggest the user to block the execution of PHP files in the core WordPress directories. If the plugin detects that the website is behind the Sucuri Firewall it will mark these three options as applied automatically, this is because the Firewall has built-in features to prevent the execution of vulnerable files (not only PHP but also other extensions) in places not intended by the developer.

This also includes a fix for a rare case where the plugin was deleting the entire content and uploads directories. This was happening because an admin can change the location of the storage folder for the plugin' settings, and if the directory was the same as the content or uploads directories the plugin would delete them if the admin requested the deactivation of the plugin, which eventually triggered the deletion of the settings, security logs and the parent directory where these files are contained.

Some modifications to the code that powers the audit logs panel. This includes the error messages at the bottom of the table and a fix for the order of the logs when the data from the local queue system is merged with the data coming from the API service, they will be now ordered correctly by the date when the event was triggered.

This pull-request also adds an option to allow admins to force the plugin to stop sending email alerts for events associated to a post status transition, for example, when a post is changed from being a draft to published, or from being private to trash, etc. This will help some users to reduce the amount of alerts that they get in their inbox due to the amount of data in their websites.

A fix for an infinite loop when a custom SMTP plugin is installed along the Sucuri plugin. In this scenario, the Sucuri plugin will send an email alert to the admins when a successful or failed login is detected. The message is intercepted by the custom SMTP plugin and then (for some technicalities in their code) a temporary object is created before the message is sent. The Sucuri plugin detects the creation of this temporary object and sends an alert to notify the admins, and here they both fall into an infinite loop.

This pull-request also modifies the execution of the tools available in the "Firewall (WAF)" page, they will be now retrieve the data for the settings and audit logs via Ajax, and the execution of the cache flush will also run via Ajax to make the interface more responsive.

This pull-request also adds a new option in the "Firewall (WAF)" page to allow users to blacklist IP addresses when the Firewall API key is available. Notice that this option is only available to Sucuri customers, and more specifically those who have purchased the Firewall. Additionally, the plugin will automatically blacklist any IP address who fails log into the website more than five times during the same day. Users will have to ask one of the admins to log into the Firewall dashboard to unblacklist them.

Add an option to configure the automatic flush of the Sucuri Firewall cache (disabled by default). Every time a page or post is modified and saved into the database the plugin will send a HTTP request to the firewall API service and except that, if the API key is valid, the cache is reset. Notice that the cache of certain files is going to stay as it is due to the configuration on the edge of the servers.

Add smart limit to send logs from the queue to the API. We will use the maximum execution time setting to limit the number of logs that the plugin will try to send to the API service before the server times out. In a regular installation, the limit is set to 30 seconds, since the timeout for the HTTP request is 5 seconds we will instruct the plugin to wait (30 secs - 5 secs) and an additional one second to spare processing, so in a regular installation the plugin will try to send as much logs as possible to the API service in less than 25 seconds.

cixtor added some commits Jun 30, 2017

Add option to ignore events for post transitions
When the type of a post is changed, for example, from new to publish or
from draft to private, or from private to trash, etc the plugin will
send an email alert to the admins of the website and also report the
change to the API. However, in some cases these transitions happen too
often, specially with the draft status, and people may prefer to ignore
such alerts. This commit adds an option to the settings page to allow
people to disable the email alerts for post status transitions.
Fix infinite loop with email alerts and SMTP plugin
The plugin detects changes in the posts, there are some other plugins
that intercept PHPMailer and create a post object that is later used to
send the real message to the users. This object is also detected by our
plugin and is considered an additional event that must be reported, so
after the first execution the operation falls into an infinite loop.
Add option to blacklist IP addresses with the Firewall API
This tool allows you to whitleist and blacklist one or more IP addresses
from accessing your website. You can also configure the plugin to
automatically blacklist any IP address involved in a password guessing
brute-force attack. If a legitimate user fails to submit the correct
credentials of their account they will have to log into the Firewall
dashboard in order to delete their IP address from the blacklist, or try
to login once again through a VPN.
Add mechanism to automatically blacklist IP addresses
For every failed login attempt the plugin will log the event into a flat
file, once five failed logins have beem recorded the plugin will send
the IP address to the Firewall IP to be blacklisted. If the operation
succeeds the plugin will delete the records from the flat file to free
space. If the operation fails, it will keep counting the failed logins
into the flat file until the Firewall API is responsive. Both the
blacklist and unblacklist actions will be logged into the event monitor
as well for visibility.
Fix order of the audit logs when the queue is merged
Considering that the logs from the API service will be merged with the
logs from the local queue system to complement the information until the
queue is emptied, we will have to sort the entries in the list to keep
the dates in sync.
Remove unnecessary automatic blacklisting of IP addresses
The Sucuri Firewall already blacklists IP addresses automatically when
these are detected to be involved in a password guessing attack, not
only per customer but across the entire stack. Having this option in the
plugin is redundant.
Add support for other English and Spanish based languages
When the plugin is installed in a WordPress website set to use a
language other than the supported ones it will try to find a POT file
with a similar translation and copy the original file into the new
locale, if no suitable similar translation file is found it will default
to use the English POT file, and if the languages directory is not
writable it will default the global `$locale` variable to use English
which may result in unwanted behaviour with other plugins. Here is a
list of supported languages:

- en_NZ uses en_US; English (New Zealand)
- en_CA uses en_US; English (Canada)
- en_ZA uses en_US; English (South Africa)
- en_GB uses en_US; English (UK)
- en_AU uses en_US; English (Australia)
- es_AR uses es_ES; Español de Argentina
- es_MX uses es_ES; Español de México
- es_CO uses es_ES; Español de Colombia
- es_GT uses es_ES; Español de Guatemala
- es_VE uses es_ES; Español de Venezuela
- es_CL uses es_ES; Español de Chile
- es_PE uses es_ES; Español de Perú
Add pre-checks for every plugin page for simplicity
This method verifies if the visibility of the requested page is allowed
for the current user in session which usually needs to be granted admin
privileges to access the plugin's tools. It also checks if the required
SPL library is available and if the settings file is writable.
Add option to stop sending the failed login passwords
By default, the plugin will report which password was used during a
password guessing brute force attack, both to the API service and to the
admin users via an email alert. This may pose a security breach if one
of the admin users missed a character in their password and mistakingly
sent the text to the API and to the other administrators. For this, we
will allow the admins to disable this behavior.
Add option to configure the malware scanner target URL
The remote malware scanner provided by the plugin is powered by Sucuri
SiteCheck, a service that takes a publicly accessible URL and scans it
for malicious code. If your website is not visible to the Internet, for
example, if it is hosted in a local development environment or a
restricted network, the scanner will not be able to work on it. Also,
if the website was installed in a non-standard directory the scanner
will report a "404 Not Found" error. You can use this option to change
the URL that will be scanned.
Add option to enable the auto clear cache firewall function
Every time a page or post is modified and saved into the database the
plugin will send a HTTP request to the firewall API service and except
that, if the API key is valid, the cache is reset. Notice that the cache
of certain files is going to stay as it is due to the configuration on
the edge of the servers.
Add smart limit to send logs from the queue to the API
We will use the maximum execution time setting to limit the number of
logs that the plugin will try to send to the API service before the
server times out. In a regular installation, the limit is set to 30
seconds, since the timeout for the HTTP request is 5 seconds we will
instruct the plugin to wait (30 secs - 5 secs) and an additional one
second to spare processing, so in a regular installation the plugin will
try to send as much logs as possible to the API service in less than 25
seconds.
Add action name in POST requests for visibility
Every time a POST request is sent to the Sucuri WordPress API the plugin
will include the name of the action in the URL, this will allow us to
generate some statistics in the server about which actions are used the
most across the entire code base.
Add fallback mechanism for the API event log collector
If the user have not disabled the communication with the API service,
the plugin will send a message with information about every triggered
event in the website in realtime with a maximum connection time of two
seconds. If the API service does not responds on time the plugin will
insert the event into the local queue system and it will try to send the
message again with a scheduled task every 24 hours, once the operation
succeeds the event will be deleted from the queue.
Modify output for the malware results to simplify links
Instead of sending the entire HTML code with the table containing all
the links, iframes and JavaScript files found by the malware scanner,
the plugin will now send the list as a JSON slice, and the HTML code
will be written using the same JavaScript code that executes the Ajax
request. This reduces the size of the object and simplifies the code.
Add option to override the timezone for the datetime
This option defines the timezone that will be used through out the
entire plugin to print the dates and times whenever is necessary. This
option also affects the date and time of the logs visible in the audit
logs panel which is data that comes from a remote server configured to
use Eastern Daylight Time (EDT). WordPress offers an option in the
general settings page to allow you to configure the timezone for the
entire website, however, if you are experiencing problems with the time
in the audit logs, this option will help you fix them.

@dcid dcid merged commit 13de2f4 into Sucuri:master Jul 25, 2017

cixtor added some commits Jul 26, 2017

Add option to configure the WordPress checksums API
The webmaster can change this URL using an option form the settings page
This allows them to control which repository will be used to check the
integrity of the installation.

For example, projectnami.org offers an option to use Microsoft SQL Server
instead of MySQL has a different set of files and even with the same
filenames many of them have been modified to support the new database
engine, since the checksums are different than the official ones the
number of false positives will increase. This option allows the webmaster
to point the plugin to a different URL where the new checksums for this
project will be retrieved.
Add maximum execution time avoidance in the integrity tool
In some environments, the WordPress integrity tool may report a
significant amount of corrupt files, the user will promptly select them
all and execute the "restore" action but due to the way the web server
was configured the operation will fail because the PHP interpreter is
not able to process so many files before the maximum execution time is
reached. This commit fixes this by monitoring the elapsed time and
breaking the loop before the timeout; if this happen the plugin will
display an error message warning the user that the operation reached its
maximum execution time and only a handful of files were processed.
Add option to set WordPress GitHub as the integrity repository
The webmaster can change this URL using an option form the settings page.
This allows them to control which repository will be used to check the
integrity of the installation.

For example, projectnami.org offers an option to use Microsoft SQL Server
instead of MySQL has a different set of files and even with the same
filenames many of them have been modified to support the new database
engine, since the checksums are different than the official ones the
number of false positives will increase. This option allows the webmaster
to point the plugin to a different URL where the new checksums for this
project will be retrieved.

If the custom API is part of GitHub infrastructure, the plugin will try
to build the expected JSON object from the output, if it fails it will
pass the unmodified response to the rest of the code and try to analyze
the integrity of the installation with that information.
Add support to run diff on deleted WordPress files
Right now, the diff utility cannot be used with deleted core files
because the tool needs to compare two existing files, otherwise it
returns an empty result. This commit changes this by creating an empty
temporary file that will act as the modified version of the code, this
way the diff utility will be able to highlight the deleted content.
Modify event report to prefer the custom queue system
When an event is triggered by WordPress the plugin will send the report
to a local queue system which will be dequeued every 24 hours when it
sends the data to the remote API service. The number of logs that will
be send to the API per batch depends on the maximum execution time
allowed by the web server, in a normal configuration, when the maximum
execution time is 30 seconds the plugin will send approximately 15,000
HTTP requests, of course, only if there are +15,000 logs in the queue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment