New CVE-ID Fathead Plugin #45

Merged
merged 18 commits into from Mar 30, 2013

Conversation

Projects
None yet
2 participants
Contributor

soh-cah-toa commented Feb 22, 2013

Greetings! Here's all my work on a Fathead plugin for displaying the description of a given CVE-ID. I've followed the instructions in the README file and it feels pretty much complete.

Unfortunately, since duckpan doesn't work with Fathead plugins, I have no way of testing it except for manually viewing the output.txt file. It looks good to me though.

My only concern is that there's no place to specify when the plugin is actually triggered. There's nothing like a Spice triggers() function where you give the appropriate regex. Perhaps Fathead plugins are handled differently; I just didn't see anything in the tutorial.

Hopefully, I've done everything right. I'll be glad to fix anything that needs improvement.

Contributor

rpicard commented Feb 22, 2013

Looks great! It deployed without issue, which is always nice. You can see it here: https://robert.duckduckgo.com/?q=CVE-1999-0009.

I think it might be a good idea to preface the description with something like, "Vulnerability description: description" so it is clear what exactly we're showing. That's something we've done before, with things like the pip plugin (example).

Contributor

rpicard commented Feb 22, 2013

Oh, and triggers are something that we add internally. If you have suggestions, please let me know! The plugin will trigger if somebody searches the exact title, or the title + a trigger. For this one, I'm thinking that cve and cve id will be sufficient, but let me know if there are others.

Appended "Vulnerability description" to each CVE-ID entry.
Suggested by rpicard, doing so adds a little context and clarity.
Contributor

soh-cah-toa commented Feb 25, 2013

Good idea. Already added it in 991ae1b.

I just realized something else though: some CVE-ID's are merely reserved for future use and have no associated page. They're just archived as:

** RESERVED ** This candidate has been reserved by an organization or individual that will use it when announcing a new security problem. When the candidate has been publicized, the details for this candidate will be provided.

Is this worth handling as a special case? Should I simply exclude it from output.txt? I mean, it is still a valid identifier; it just has no associated details page yet.

There's also the case of rejected identifiers. Some are rejected as being duplicates while others turned out to just be configuration errors. Just like with reserved identifiers, they're archived but have no associated details page. I can do a few things with these special cases.

  1. Include rejected duplicates with the standard "this is a duplicate" description but modify the URL to point to the identifier it's a duplicate of.
  2. Include rejected duplicates but with the description and URL it's a duplicate of (e.g. title is CVE-1999-1056 but description and URL are that of CVE-1999-1359.)
  3. Exclude rejected duplicates from output.txt completely.

It might be easier to just exclude them all together since the resulting parser logic would be a bit convoluted. The duplicated CVE-ID's are ones that haven't been parsed yet. They're still further into the archive. Also, sometimes they're duplicates of multiple identifiers.

As for configuration error rejects, I see no other option than to exclude it from output.txt since it simply has no details page.

I just want to make sure I do it the DDG way before going ahead with it myself. I don't want to impose.

Contributor

rpicard commented Feb 26, 2013

You know more about this than me, so I'm inclined to defer to you for what would be more useful to the user.

  • From what it looks like, I think the reserved ones should be included.
  • For rejected identifiers, you would need to make it very clear that the linked page is for the original, not the duplicate, if you wanted to include them. With that in mind, I'll let you decided what to do there (leaving them out is fine).

soh-cah-toa added some commits Mar 4, 2013

Fixed issue of rejected and reserved CVE-ID's in output.txt.
New parse_entry() subroutine now excludes rejected identifiers from
output.txt and includes reversed identifiers but with a clear
message about their reserved status.
Contributor

soh-cah-toa commented Mar 4, 2013

Alright, so I think this is finally finished. ;)

After banging my head against the wall for days, I decided to just exclude rejected identifiers from output.txt. It was getting overly complicated and there was just no way to handle entries that were duplicates of two or three other identifiers. I'm fine with it though.

Reserved identifiers are included with a description of their reserved status. Since they have no associated page, I made their URL an empty string in output.txt, just like the other fields that aren't relevant. I hope that's how you prevent the "More at Blah Blah Blah" link from appearing at the bottom. A dead link like this isn't exactly useful.

Granted everything is in good standing, how long does it usually take to fully integrate plugins? I'm not familiar with how DDG updates all its servers.

Contributor

rpicard commented Mar 5, 2013

It might turn out that we need to remove reserved identifiers since they don't have a link, but I'll give it a try and see how it looks. I'm pretty busy today, so I should be able to do that later this week.

Once everything looks good, it should only take a few minutes to bring the plugin to all of the servers.

Contributor

soh-cah-toa commented Mar 5, 2013

Yeah, that was my dilemma with reserved identifiers. It just seemed counter-intuitive to what ZCI is meant for. I'm not averse to removing them if that's what it comes down to.

Take your time though. I'm in no rush.

Modified parser to ignore reserved CVE-ID's.
If it is decided that reserved identifiers should be excluded from
the results, then this branch can be merged with master. Otherwise,
it can be deleted.
Contributor

soh-cah-toa commented Mar 14, 2013

By the way, if you decide that reserved identifiers should be excluded, I made a reserved-ids branch that takes care of it. That way it can quickly be merged if you give the word or removed if not.

Contributor

rpicard commented Mar 17, 2013

Hey @soh-cah-toa, I've been pretty busy with school. Is it possible at all to find a page that explains what the reserved IDs are? We'd like to keep them and link them all to an informative page about it. I couldn't find one myself, but I figured you're probably more knowledgeable about that than I am.

Contributor

soh-cah-toa commented Mar 18, 2013

The only place that archives reserved identifiers is the Mitre CVE database. For example, this is what one looks like. However, the problem is that I've set the source value to "CVE Details" in the plugin meta file. In it's current state, the ZCI result would read "More at CVE Details" but the link wouldn't actually point to a CVE Details page; it'd be a page from the Mitre database. I don't know if it's possible to change that based on the given URL but I doubt it.

The easy solution would be to set source to "Mitre CVE Database" and change every URL to point to the Mitre database instead of the CVE Details database. The reason I initially went with CVE Details though is because it's a much more extensive and helpful resource. I'd be sad to see that be changed but there may not be another way around the issue.

Contributor

rpicard commented Mar 18, 2013

I was thinking of a page on CVE Details, such as an FAQ, that explains what reserved identifiers are. It doesn't necessarily have to be the information for that specific identifier.

Contributor

soh-cah-toa commented Mar 18, 2013

You're right. The FAQ has a "Why are some CVE entries are missing?" (their typo, not mine) section that explains why rejected and reserved identifiers don't appear in their database. I could use http://www.cvedetails.com/faq.php#missingcves as the URL for them. The page is so small though that using the anchor hardly moves the page. It may be good enough though if that's what you're looking for.

Contributor

rpicard commented Mar 18, 2013

I'll see what some other guys think, but it'll probably be better to go with leaving them out altogether.

Contributor

rpicard commented Mar 22, 2013

https://duckduckgo.com/?q=CVE-2004-0408

The Fathead is live now! I did just notice, however, that in your meta file you specify cve 2009-1234 and 2009-1234 cve as example queries. For those to work, you'd want to include a line for 2009-1234 and I'd set cve as a trigger. It wouldn't be a big deal. Do you want to do that?

Included xxxx-yyyy component of each CVE-ID in output.txt.
By adding a keyword trigger for "cve", this will allow users to search
for "CVE-1999-1234" as well as "cve 1999-1234".
Contributor

soh-cah-toa commented Mar 30, 2013

Ok, I see what you mean. For an identifier like CVE-1999-1234, output.txt should have an entry for "CVE-1999-1234" as well as "1999-1234". Already committed the change just now.

If that's everything, then I guess this is ready to be merged into master.

(I have another Fathead plugin in progress for a Ruby reference so expect another pull request soon.)

Contributor

rpicard commented Mar 30, 2013

Cool. I just deployed the updates.

It's worth noting that I did edit it so that the added entries were redirects to the originals. This preserves consistency in the title of the ZCI box.

https://duckduckgo.com/?q=cve+1999+0001

Thanks for working with us on this. I'm looking forward to the next one! 👀

@rpicard rpicard merged commit 8e5c807 into duckduckgo:master Mar 30, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment