List of goals, suggestions and ideas for future releases. #34
Comments
|
One question, no GUI? |
|
A GUI for managing CIDRAM exists (the "front-end"), but it's disabled by default for security (in case people using CIDRAM didn't realise it was there and an attacker tried to hijack it and things like that). You'll need to manually change the value of " |
|
Just installed it on my wordpress setup, pretty far it's super awesome.. really man, it's awesome.. I was actually looking for this kind of thing.. thank you for letting me know about this awesome script. You are my new Batman. 👍 Question:
|
|
Glad to hear that it's working well, and you're welcome. 😄 👍
It varies, depending on the signature files and modules used. Described below are the current estimated false positive rates (or "risk") for the various "components" (signature files and modules) available:
(Of course, take in mind that the false positive risk for each component listed here is based on the intentions of each component; If it blocks something that we don't want blocked, if it's a mistake, or something that it shouldn't be blocking, it would be regarded as a false positive, but if it's not a mistake, or if it's something that the component is actually meant to block, that wouldn't be regarded as a false positive. An example here would be with the "Baidu blocker module" or "Yandex blocker module": Using either of those two modules would most likely completely destroy any SEO work and PR for Baidu and Yandex search engines specifically, due to that it would be blocking them, and so, anybody that cares about those two search engines shouldn't use those two specific modules, but at the same time, because blocking those two search engines is what those two modules are designed to do, this would not be regarded as a false positive. On the other hand though, if they suddenly started blocking other search engines, or other unrelated traffic, then that would definitely be considered a false positive). "Low", "Medium", and "High" are used as descriptors for the false positive risk as a means of keeping it simple, due to that the different demographics and different needs of different websites would likely result in significant variances in the actual specific numbers of false positives ultimately seen by any particular website. "High" is almost an absolute guarantee that there'll be significant false positives seen, any I would recommend only using components marked as "high" if they're absolutely required for meeting the needs of a specific website. Any components marked as "Low" are considered reasonably accurate at even the worst of times, and false positives should be either zero, or extremely rare. "Medium" covers basically everything between the two extremes (although generally favouring more towards "low", rather than "high"). It's quite possible that you already know this, but due to that discussion is public, for the benefit of anyone reading this that might not already know this, the definition of "false positive" in relation to and in the context of CIDRAM, according to the CIDRAM documentation, is as follows: What is a "false positive"?The term "false positive" (alternatively: "false positive error"; "false alarm"), described very simply, and in a generalised context, is used when testing for a condition, to refer to the results of that test, when the results are positive (i.e., the condition is determined to be "positive", or "true"), but are expected to be (or should have been) negative (i.e., the condition, in reality, is "negative", or "false"). A "false positive" could be considered analogous to "crying wolf" (wherein the condition being tested is whether there's a wolf near the herd, the condition is "false" in that there's no wolf near the herd, and the condition is reported as "positive" by the shepherd by way of calling "wolf, wolf"), or analogous to situations in medical testing wherein a patient is diagnosed as having some illness or disease, when in reality, they have no such illness or disease. Related outcomes when testing for a condition can be described using the terms "true positive", "true negative" and "false negative". A "true positive" refers to when the results of the test and the actual state of the condition are both true (or "positive"), and a "true negative" refers to when the results of the test and the actual state of the condition are both false (or "negative"); A "true positive" or a "true negative" is considered to be a "correct inference". The antithesis of a "false positive" is a "false negative"; A "false negative" refers to when the results of the test are negative (i.e., the condition is determined to be "negative", or "false"), but are expected to be (or should have been) positive (i.e., the condition, in reality, is "positive", or "true"). In the context of CIDRAM, these terms refer to the signatures of CIDRAM and what/whom they block. When CIDRAM blocks an IP address due to bad, outdated or incorrect signatures, but shouldn't have done so, or when it does so for the wrong reasons, we refer to this event as a "false positive". When CIDRAM fails to block an IP address that should have been blocked, due to unforeseen threats, missing signatures or shortfalls in its signatures, we refer to this event as a "missed detection" (which is analogous to a "false negative"). This can be summarised by the table below:
Anyhow. In regards to any of the other available components, that haven't been lised above (e.g., Macmathan's country blocklists): Due to the frequency of updates, I assume that they're quite accurate, and that false positives would be quite rare, but that said, I haven't described any specific risk level for them, due to that I haven't extensively tested any of them in regards to false positives (those that I've listed above though, I've tested).
Different components (e.g., different parts of the signatures database) have different update frequencies. Most of Macmathan's country blocklists are updated once per month, but some of them are occasionally updated more frequently. The components that I maintain myself, I tend to update either whenever I've got enough spare time to go through them and on an "as needed" basis (e.g., when it becomes apparent that updates are required, or when false positive are reported); This tends to result in updates occurring, on average, about ~4 times per month.
To keep up-to-date, yep. 😄 If you've enabled the front-end, you can update everything automatically via the "updates" page. On the front-end, there should be an updates page linked, and all components (installed, and available for installation) should be listed on that page. Whenever visiting that page, if anything is outdated, an "Update all" button should appear on the top of the page. Clicking that button will automatically update everything. Alternatively, if the front-end is disabled, it's also possible to update everything manually, but it'll probably take a little more time to do, and doing it manually is a little less convenient than doing it automatically: Because CIDRAM uses "flatfiles" (i.e., no SQL database, no external drivers, etc), if you're able to identify which components are outdated, you can update them by simply replacing the files corresponding to those components with their up-to-date counterparts, and then everything should be up-to-date again and working correctly as per normal. The trick, however, is identifying which components are outdated, and which files correspond to them (this is something that the updates page will handle automatically, which makes it an easier solution for updating everything). The best bet for doing it manually, would be to monitor the repository here, and Macmathan's blocklist page (if using any of his blocklists), to be able to see when updates are announced/listed/etc. There's also Cronable, which is a separate package I maintain, that can be used to truly automatically update everything (with Cronable, everything is done via Cron, and you don't need to even worry about visiting the updates page at all), but I would only recommend Cronable to people that are very familiar with how to create and schedule cronjobs on their server (due to using Cron, the technical debt is slightly higher, and I've had some users in the past mention that they couldn't get it working properly, but it could be a preferable solution for anyone that's short on spare time and has experience working with cronjobs).
In a manner, it's already supported: CIDRAM is capable of blocking hostnames, and there are some modules available already for the purpose of blocking bad/dangerous hostnames. It doesn't currently have any GUI or front-end page comparable to that seen in your screenshot for adding custom hostnames to be blocked though. It's a good idea though. I can definitely add that to the to-do list and try to implement something like this in the future. 😄 👍
My understanding of it is that Macmathan's website has come under attack multiple times in the past from spammers and hackers using Indian IPs, and as a result, he's decided to block all Indian IPs from accessing any of the pages on his website as a precaution against future attacks. I think there are some other countries blocked there too, but I'm not entirely sure exactly which countries are and aren't blocked (I know that India, China, and North Korea are blocked there; I know that my own country, Australia, is not blocked there; as to exactly which countries are and aren't blocked though, he would probably need to answer that question himself, I think). In any case though, it's definitely nothing personal against specific places or people; his decisions about what he blocks are based on the kind of web traffic he sees to his websites (friendly guy that cares about security, and a great ally in the fight against bots and spammers). And, of course, we both definitely welcome users from everywhere around the world. 😄 AFAIK though, the CIDRAM front-end is not subject to any of the blocklists at his website, and it should be possible to download+install any of the blocklists that he maintains via the CIDRAM front-end, regardless of which country the requests route through (although I haven't tested it extensively for all countries; I've only done some cursory testing for some specific ISPs and providers). I don't think he uses this platform at all (GitHub), but if you needed to contact him for support specific to the country blocklists, or about his website, if you're able to access it via a VPN or something, he has a bug tracker page available here that you could use, or alternatively, he does frequently visit one of our other support endpoints, hosted by Spambot Security (that said though, Spambot Security has its own set of signatures and security policies, most of which haven't been updated in a few years and are now very outdated, so I have no idea whether you'll be able to access it at the moment; a different person, "Zaphod", maintains that, and he hasn't updated it in a while now). (GitHub is the primary and recommended support endpoint for CIDRAM, but there are a few other support endpoints around, Spambot Security being one of them; I'm not sure how long into the future Spambot Security will continue to exist though, so, although it could be good to register there if you're able to, GitHub definitely remains the preferred support endpoint). |
|
Consider adding Google's new powered by AI Invisible reCaptcha ? |
|
reCaptcha should be already available. |
|
You should be able to configure this to use the invisible mode. |
|
Ok, then please open a new issue for this. |
|
It still makes sense to track it separately in another issue or this thread here would get very long over the time. |
|
@Maikuolan How about adding Browser Integrity Check? is it possible? so, people cant retrieve data using some basic get contents functions, like
what do you think about this? This might help you, https://github.com/ItayRosen/Browser-Integrity-Check I also worried about how Google, Bing, Yandex, Facebook will handle it. |
|
(Above comment seen. Definitely sounds like a cool idea. Was planning on replying earlier, but had a busy night and didn't get to it; Also sick with the flu at the moment and feeling like I need a few hours downtime. I'll take a closer look at this either later today or tomorrow and reply properly afterward). |
|
OK folks, a bit of necro-reply but I joined GitHub just a bunch of days ago and thought the issue could do with an "official" statement regarding the question about my domain.
Correct assumptions. It is my personal domain after all, and I have had to deal with my share of some countries wealth of spammers, exploit searching hackers and script kiddies. To make one thing clear, it is never a personal thing against any country or group of people or similar (and so says my disclaimer as well). But with the internet being what it is the blocks are a sad necessity for the time being to keep the site safe. The actual list can and will not be disclosed because of the nature of it. I'm sorry for the inconvenience it might cause.
I can confirm that CIDRAM is supposed to be able to download and install any of my lists and modules from within the front-end, even from blocked countries and hosts. If it doesn't work, let me know.
I didn't use this platform until a few days ago. I'll keep an eye on it from now on, but for the time being the main support for my lists is on my site. |
|
Well, a versioned variant of the lists eg using Git may be an idea. But it is not recommended. |
Welcome on GitHub. 🎉 |
|
For Wordpress users Would be great if CIDRAM was accessible in the admin dashboard. Unless I am doing something wrong, every time I run an update of CIDRAM from the WordPress plugin update section the update wipes out all my settings. After updating I have setup everything from scratch - country blocks, ignore files, etc. A lot of work when you have multiple sites. My dream is to open the WordPress admin panel, select CIDRAM, push the update button. As usual, thanks for all the hard work! |
|
In general settings should still be stored and not wiped on update https://wordpress.org/plugins/cidram/ @Maikuolan where do we have the repo for the WordPress plugin? |
|
It definitely wipes them all out for me. The config.ini is wiped clean. The entire CIDRAM/Vault is reset to a clean install and all loaded modules are gone. Admin password to front end reset to default. I am referring to running the CIDRAM plugin update from within WordPress/Plugins Section–not from the CIDRAM front end. |
|
In the documentation currently:
Unfortunately, this is due to the way that WordPress updates are performed: WordPress doesn't compare actual individual changed files between different package (plugin, theme, etc) versions, nor provides any way to implement more controlled update mechanisms specific to individual packages. Instead, it simply downloads a ZIP of the entire latest version of whatever's being updated, decompresses its contents, and replaces all the files of the old package with the files of new package (essentially like a fresh install, minus the database changes). For plugins that hook into the WordPress database itself, this isn't usually problem, due to that their settings, configuration, customisations and so on will usually be stored in the database, which isn't affected by these updates. For plugins/packages such as CIDRAM however, which don't touch the WordPress database at all, and instead store their own data internally (in the case of CIDRAM, in the form of the data in the vault directory, such as the To fix this problem, I think I'd probably need some way to temporarily override the WordPress update system when updating CIDRAM via the plugins dashboard. Not quite sure currently how viable that is though.
The WordPress plugins database has its own internal version control (SVN, rather than Git; a bit unfortunate IMO, as I prefer Git over SVN, but it is what it is). SVN repository for the CIDRAM WordPress plugin can be accessed via: https://plugins.svn.wordpress.org/cidram/ Content there should be the same as here though. Usually what I'll do is just copy/paste the latest CIDRAM codebase as is available on GitHub (but with |
|
Hi Maikuolan Yes. I am aware of all this already. 😃 I always RFTM. I thought this was a sorta wish list so I put it out there. I think that one issue that will stop the WordPress community from adopting CIDRAM fully is the lack of integration into the database, as you said, it does not "hook into the WordPress database itself." If I didn't like the plugin so much I would not go through the hassle of setting it up again whenever there is a WordPress plugin update. I would love to see the usage really take off in the WP community. You guys fight the good fight. CIDRAM is a fantastic complement to other security plugins that do not have the ability to block by country. |
Much appreciated, and no problem. 👍 I figure worth replying with that anyhow, in case anyone reads these messages which hadn't yet seen that part of the documetation (a few less whats and whys down the track to deal with). In any case, I agree; Definitely something worth pursuing. |
|
"Implement mechanisms for platform contingencies/redundancies." Working on this currently, and so far, everything looks good at my end. I just need to update the documentation and L10N data for it now, before I commit. Next commit to the repo should have that box ticked off, hopefully. Edit: Done and pushed. |
|
"Possible expansion of the format allowed for some configuration directives (tracktime, etc)." This has been on the list for long time now, since way back when IP tracking was first introduced, I think.. if I'm remembering correctly. Kind of a small item, and should technically be easy to implement.. but I'm a little stuck on ideas for exactly what it should look like, or.. whether we really still need it, for that matter. Anyone have any thoughts about this? Strike it from the list? Keep it on the list and try to come up with something? If the latter.. any ideas? Would prefer to stick to standards, if at all possible, because coming up with something totally different, and totally original, could just add further confusion, because that would mean that users would then need to learn something totally different and totally original to work with it. I haven't really found too many "standard" ways of doing this though. If I implemented something like I could go the SI approach, implementing something Worth noting that following through with this would just be to provide an extra convenience for those that don't want to type in actually whole integers to represent a unit of time (as is the case for the format currently), and isn't necessarily adding a new "feature" to CIDRAM or anything like that. Likewise, it isn't something particularly needed, and wouldn't really impact anything if it was to be struck from the list. |
|
"Possible expansion of the format allowed for some configuration directives (tracktime, etc)." (i.e., the to-do mentioned by my previous reply). No feedback received yet, I also don't have any new ideas for how to progress further with it yet, and it has always been regarded as very low priority, so I'm thinking I might just scratch it from the list (i.e., not something really wanted/needed anymore anyway). "Stable/unstable updates selector option (maybe?)." The codebase has changed quite a lot since that to-do was first added to the list (the channel switcher, for example, hadn't been added to the codebase yet). Between the channel switcher, the ability to just push any future unstable or questionable commits into their own branch (until further testing, further improvements, or whatever else that might be needed can be done), and better, improved testing practices from herein and into the future, I'm not sure we necessarily need to explicitly code in the ability to distinguish and choose between stable/unstable updates from the front-end. Scratching that from the list too, would mean that there wouldn't be any remaining to-do items on that list (and we could then close this issue and move onto other things). There are some other things mentioned in the discussion attached to this issue, such as the suggestion to be able to manage CIDRAM via the WordPress admin dashboard. I'll create new, separate issues for any such things mentioned in the attached discussion (more maintainable that way, rather than leaving open really old issues that deal with multiple, unrelated to-do tasks, problems, etc). Will mark this issue as "implemented" as close accordingly shortly. |
List of goals, suggestions and ideas for future releases.
This list isn't static; Entries may be added and/or removed without notice. Listed order does not represent priority. List is not comprehensive (issues may exist that aren't listed).
The text was updated successfully, but these errors were encountered: