Permalink
Browse files

Initial import of sources from Yahoo!

  • Loading branch information...
1 parent 641e8f5 commit 27ecac9fc88baafcaa183f0c74282202e56d7aac @jschauma jschauma committed Sep 30, 2010
Showing with 1,955 additions and 6 deletions.
  1. +25 −0 BUGS
  2. +34 −0 LICENSE
  3. +59 −0 Makefile
  4. +4 −6 README
  5. +9 −0 TODO
  6. +205 −0 bin/fetch-vlist.sh
  7. +18 −0 bin/run-yvc.py
  8. +19 −0 conf/yvc.conf
  9. BIN doc/ContributionLicenseAgreementYahoo.pdf
  10. +18 −0 doc/html/Makefile
  11. +77 −0 doc/html/index.html
  12. +80 −0 doc/man/fetch-vlist.1
  13. +204 −0 doc/man/yvc.1
  14. +67 −0 doc/man/yvc.conf.5
  15. +565 −0 lib/yvc.py
  16. +171 −0 misc/harvest_freebsd_yvc.pl
  17. +93 −0 misc/redhat_oval_to_yvc.py
  18. +12 −0 test/Makefile
  19. +295 −0 test/test.py
View
@@ -0,0 +1,25 @@
+Known bugs:
+-----------
+
+Patchlevels are not dealt with correctly. That is, if, for example, the
+package listed in the vulnerabilities file is marked as "foo-1.2pl3" and a
+package with a tiny version such as "foo-1.2.1" is installed, it may falsely
+match. That is, comparison of "foo-1.2pl3" and "foo-1.2.1" claims that the
+patchlevel version is higher. (The converse scenario also holds.)
+
+This is a restriction of the used distutils.versions.LooseVersion
+implementation. Presumably, the assumption is that a piece of software
+wouldn't mix patchlevels with tiny versions (?). Note that the expensive
+shell-out to parse_version(1) wouldn't solve this problem either: that program
+operates on the same assumption.
+
+----
+
+Deeply nested brace expansions are not correctly dealt with. The
+braceExpansion function is able to handle simply nested expansions such as
+"foo-{,bar{-baz,-bla}}", but deeper levels of nesting may not yield the
+expected results.
+
+For the purposes of the vulnerability list, this seems acceptable for the time
+being, as deeply nested version strings are not found. An alternative (albeit
+very expensive) would be to shell out to zsh to do brace expansion.
View
@@ -0,0 +1,34 @@
+Software Copyright License Agreement (BSD License)
+
+Copyright (c) 2010, Yahoo! Inc.
+All rights reserved.
+
+Redistribution and use of this software in source and binary forms, with
+or without modification, are permitted provided that the following
+conditions are met:
+
+* Redistributions of source code must retain the above
+ copyright notice, this list of conditions and the
+ following disclaimer.
+
+* Redistributions in binary form must reproduce the above
+ copyright notice, this list of conditions and the
+ following disclaimer in the documentation and/or other
+ materials provided with the distribution.
+
+* Neither the name of Yahoo! Inc. nor the names of its
+ contributors may be used to endorse or promote products
+ derived from this software without specific prior
+ written permission of Yahoo! Inc.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
View
@@ -0,0 +1,59 @@
+# Copyright (c) 2008,2010 Yahoo! Inc.
+#
+# This example Makefile can be used to maintain vulnerability list.
+# See 'make help' for more information.
+
+# Location to which to upload the vlists.
+LOCATION="<hostname>:~/public_html/yvc/"
+FBVLIST=fbvlist
+RH4VLIST=rh4vlist
+RH5VLIST=rh5vlist
+LISTS= ${RH5VLIST} ${RH4VLIST} ${FBVLIST}
+
+GONERS= ${RH5VLIST}.in ${RH4VLIST}.in ${FBVLIST}.in \
+ com.redhat.rhsa-all.xml.bz2
+
+date!=date
+
+all: sign upload
+
+help:
+ @echo "The following targets are available:"
+ @echo "all sign + upload"
+ @echo "clean remove any interim files"
+ @echo "help print this help"
+ @echo "sign sign the vulnerability list"
+ @echo "upload upload the vulnerability list"
+
+sign: ${LISTS}
+
+${FBVLIST}: ${FBVLIST}.in
+ gpg -o ${FBVLIST} --clearsign ${FBVLIST}.in
+ chmod a+r ${FBVLIST}
+
+${FBVLIST}.in:
+ @echo "# Generated on ${date}" > ${FBVLIST}.in
+ perl ./misc/harvest_freebsd_yvc.pl >> ${FBVLIST}.in
+
+
+${RH4VLIST}: ${RH4VLIST}.in
+ gpg -o ${RH4VLIST} --clearsign ${RH4VLIST}.in
+ chmod a+r ${RH4VLIST}
+
+${RH4VLIST}.in:
+ python ./misc/redhat_oval_to_yvc.py 4 > ${RH4VLIST}.in
+
+
+${RH5VLIST}: ${RH5VLIST}.in
+ gpg -o ${RH5VLIST} --clearsign ${RH5VLIST}.in
+ chmod a+r ${RH5VLIST}
+
+${RH5VLIST}.in:
+ python ./misc/redhat_oval_to_yvc.py 5 > ${RH5VLIST}.in
+
+
+upload: sign
+ scp ${LISTS} ${LOCATION}
+
+clean:
+ rm -f ${LISTS} ${GONERS}
View
@@ -1,5 +1,3 @@
-This is a placeholder README for the 'yvc' utility.
-
'yvc' is a software package vulnerability checker.
'yvc' compares the given package name against the list of known
@@ -9,9 +7,9 @@ further information for each vulnerable package.
'yvc' was conceptually based on NetBSD's audit-packages program (see
http://www.netbsd.org/support/security/#check-pkgsrc) and was written by
-Jan Schaumann <jschauma@netmeister.org> at Yahoo! Inc.
+Jan Schaumann <jschauma@yahoo-inc.com> in 2008 while working at Yahoo!
+Inc. Yahoo! open sourced this tool in the hopes that it will be useful to
+other people -- unless otherwise noted, all files are released under the
+terms of a 3-clause BSD license as noted in the file LICENSE.
The 'y' in yvc can stand for a number of things. Make up your own.
-
-'yvc' will be made available to the world at large in the very exciting
-and very near future.
View
@@ -0,0 +1,9 @@
+
+package for public:
+ - identify required packages
+ - write configure script to handle fetch-vlist:
+ - determine appropriate vlists to use
+ - provide option for place to upload/download
+ - write python magic to install correctly
+
+review helper scripts to ensure they work (efficiently) on all platforms
View
@@ -0,0 +1,205 @@
+#! /bin/sh
+#
+# Copyright (c) 2008,2009,2010 Yahoo! Inc.
+#
+# Originally written by Jan Schaumann <jschauma@yahoo-inc.com> in July 2008.
+#
+# The fetch-vlist tool is used to download the vulnerability lists to be
+# used by the 'yvc' tool. After downloading them, it will verify the PGP
+# signature and, if it checks out, install the files in the final
+# destination.
+
+# Only used during development:
+# set -eu
+
+###
+### Globals
+###
+
+DONT=""
+EXIT_VALUE=0
+GPG="gpg"
+GPG_FLAGS="--verify -q"
+GPG_REDIR="2>/dev/null"
+IGNORE_PGP_ERRS=0
+PROGNAME="${0##*/}"
+TMPFILES=""
+
+##
+## Modify this section to specify where to fetch your vlists from.
+##
+NLISTS=4
+VLIST1="http://ftp.netbsd.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities"
+VLIST1_LOCATION="/usr/local/var/var/yvc/nbvlist"
+VLIST2="http://<somewhere>/yvc/fbvlist"
+VLIST2_LOCATION="/usr/local/var/yvc/fbvlist"
+VLIST3="http://<somewhere>/yvc/rh4vlist"
+VLIST3_LOCATION="/usr/local/var/yvc/rh4vlist"
+VLIST4="http://<somewhere>/yvc/rh5vlist"
+VLIST4_LOCATION="/usr/local/var/yvc/rh5vlist"
+
+WGET="wget"
+WGET_FLAGS="-t 1 -T 10 -q"
+
+###
+### Functions
+###
+
+# function : cleanup
+# purose : exit handler to remove any temporarily created files
+
+cleanup() {
+ rm -f ${TMPFILES}
+}
+
+# function : error
+# purpose : print message to stderr and exit 1
+# input : any string
+# output : input is echo'd to stderr, program aborted
+
+error() {
+ warn ${1}
+ exit 1
+}
+
+# function : warn
+# purpose : print message to stderr
+# input : any string
+# output : input is echo'd to stderr
+# sets EXIT_VALUE to 1 to indicate failure
+
+warn() {
+ echo "${PROGNAME}: ${1}" >&2
+ EXIT_VALUE=1
+}
+
+# function : fetchVerifyInstall
+# purpose : fetch, verify and install all vlists
+# input : none
+# result : all files are fetched, verified and installed into their
+# final location; any errors encountered are caught and an
+# appropriate error message printed
+
+fetchVerifyInstall() {
+ local n
+
+ n=1
+ while [ $n -le ${NLISTS} ]; do
+ local tmpfile=$(mktemp /tmp/${PROGNAME}.XXXXXX)
+ local list=$(eval echo \$VLIST${n})
+ local target=$(eval echo \$VLIST${n}_LOCATION)
+
+ TMPFILES="${TMPFILES} ${tmpfile}"
+ n=$(( $n + 1 ))
+
+ fetchList ${tmpfile} ${list} || {
+ warn "Unable to fetch ${list}."
+ continue
+ }
+
+ verifySignature ${tmpfile} || {
+ if [ ${IGNORE_PGP_ERRS} -ne 1 ]; then
+ warn "Unable to verify signature of ${list}."
+ continue
+ fi
+ }
+
+ installFile ${tmpfile} ${target} || {
+ warn "Unable to install ${tmpfile} as ${target}."
+ continue
+ }
+ done
+}
+
+# function : fetchList
+# purpose : download the list from the given URL into a temporary
+# location
+# input : temporary file, list URL
+# returns : exit value of wget command
+
+fetchList() {
+ local tmpfile=${1}
+ local url=${2}
+
+ ${DONT} ${WGET} -O ${tmpfile} ${WGET_FLAGS} ${url}
+}
+
+# function : installFile
+# purpose : install the temporary file into the final destination if
+# needed
+# input : temporary file, final location
+
+installFile() {
+ local tmpfile=${1}
+ local final=${2}
+
+ ${DONT} cmp -s ${tmpfile} ${final} || {
+ ${DONT} mv ${tmpfile} ${final} && \
+ ${DONT} chmod 444 ${final}
+ }
+}
+
+# function : usage
+# purpose : print a usage summary
+# returns : nothing, usage printed to stdout
+
+usage() {
+ echo "Usage: ${PROGNAME} [-dhiv]"
+ echo " -d don't do anything, just report what would be done"
+ echo " -h print this help and exit"
+ echo " -i ignore any pgp errors"
+ echo " -v be verbose"
+}
+
+# function : verifySignature
+# purpose : verify the pgp signature on the given file
+# input : filename
+# returns : retval of gpg command
+
+verifySignature() {
+ local file=${1}
+ ${DONT} eval ${GPG} ${GPG_FLAGS} ${file} ${GPG_REDIR}
+}
+
+###
+### Main
+###
+
+trap cleanup 0
+
+while getopts 'dhiv' opt; do
+ case ${opt} in
+ d)
+ DONT="echo"
+ ;;
+ h|\?)
+ usage
+ exit 0
+ # NOTREACHED
+ ;;
+ i)
+ IGNORE_PGP_ERRS=1
+ ;;
+ v)
+ WGET_FLAGS="-v"
+ GPG_FLAGS="${GPG_FLAGS} -v"
+ GPG_REDIR=""
+ ;;
+ *)
+ usage
+ exit 1
+ # NOTREACHED
+ ;;
+ esac
+done
+shift $(( ${OPTIND} - 1 ))
+
+if [ $# -ne 0 ]; then
+ usage
+ exit 1
+ # NOTREACHED
+fi
+
+fetchVerifyInstall
+
+exit ${EXIT_VALUE}
View
@@ -0,0 +1,18 @@
+#! /usr/local/bin/python2.5
+#
+# Copyright (c) 2008,2010 Yahoo! Inc.
+#
+# Originally written by Jan Schaumann <jschauma@yahoo-inc.com> in July 2008.
+#
+# The entire functionality of the yvc(1) tool is found in the
+# yahoo.yvc.Checker class. This script just invokes the 'main' function
+# provided by yahoo.yvc.
+
+###
+### Main
+###
+
+if __name__ == "__main__":
+ import sys
+ from yahoo.yvc import main
+ main(sys.argv[1:])
View
@@ -0,0 +1,19 @@
+# This is the default configuration file for yvc(1). See yvc.conf(5) for
+# details.
+
+# This section is required, don't remove it.
+[YVC]
+
+# A list of vulnerability types that should be ignored.
+# See yvc(1) for the exhaustive list of possible vulnerability types.
+# For example:
+# IGNORE_TYPES = denial-of-service, permissions-race
+
+# A list of URLs that should be ignored. For example:
+# IGNORE_URLS = http://online.securityfocus.com/archive/1/272180
+
+# The files in which the list of vulnerabilities are found.
+VLISTS = /usr/local/var/yvc/fbvlist
+
+# Level of verbosity.
+#VERBOSITY = 1
Binary file not shown.
Oops, something went wrong.

0 comments on commit 27ecac9

Please sign in to comment.