README

NAME
    UMLS::Interface README

  SYNOPSIS
    This package provides a Perl interface to the Unified Medical Language
    System (UMLS). The UMLS is a knowledge representation framework encoded
    designed to support broad scope biomedical research queries. There
    exists three major sources in the UMLS. The Metathesaurus which is a
    taxonomy of medical concepts, the Semantic Network which categorizes
    concepts in the Metathesaurus, and the SPECIALIST Lexicon which contains
    a list of biomedical and general English terms used in the biomedical
    domain. The UMLS-Interface package is set up to access the Metathesaurus
    and the Semantic Network present in a MySQL database.

  INSTALL
    To install the module, run the following magic commands:

      perl Makefile.PL
      make
      make test
      make install

    This will install the module in the standard location. You will, most
    probably, require root privileges to install in standard system
    directories. To install in a non-standard directory, specify a prefix
    during the 'perl Makefile.PL' stage as:

      perl Makefile.PL PREFIX=/home/programs

    It is possible to modify other parameters during installation. The
    details of these can be found in the ExtUtils::MakeMaker documentation.
    However, it is highly recommended not messing around with other
    parameters, unless you know what you're doing.

  DATABASE SETUP
    The interface assumes that the UMLS is present as a mysql database. The
    names of these databases can be passed as configuration options at
    initialization. However, if the names of the database is not provided at
    initialization, then default values are used -- the database for the
    UMLS is called 'umls'. The UMLS database must contain six tables: 1.
    MRREL 2. MRCONSO 3. MRSAB 4. MRDOC 5. MRDEF 6. SRDEF 7. MRSTY

    All other tables in the databases will be ignored, and any of these
    tables missing would raise an error.

    The mysql server can be on the same machine as the module or could be on
    a remotely accessible machine. The location of the server can be
    provided during initialization of the module.

  INITIALIZING THE MODULE
    To create an instance of the interface object, using default values for
    all configuration options:

      use UMLS::Interface;
      my $interface = UMLS::Interface->new();

    The following configuration options are also provided though:

        'driver'       -> Default value 'mysql'. This option specifies the 
                          Perl DBD driver that should be used to access the
                          database. This implies that the some other DBMS
                          system (such as PostgresSQL) could also be used,
                          as long as there exist Perl DBD drivers to
                          access the database.
        'umls'         -> Default value 'umls'. This option specifies the name
                          of the UMLS database.
        'hostname'     -> Default value 'localhost'. The name or the IP 
                          address of the machine on which the database 
                          server is running.
        'socket'       -> Default value '/tmp/mysql.sock'. The socket on 
                          which the database server is using.
        'port'         -> The port number on which the database server 
                          accepts connections.
        'username'     -> Username to use to connect to the database server. 
                          If not provided, the module attempts to connect as 
                          an anonymous user.
        'password'     -> Password for access to the database server. If not
                          provided, the module attempts to access the server
                          without a password.

        'forcerun'     -> This parameter will bypass any command prompts such 
                          as asking if you would like to continue with the index 
                          creation. 

        'realtime'     -> This parameter will not create a database of path 
                          information (what we refer to as the index) but obtain
                          the path information about a concept on the fly

        'cuilist'      -> This parameter contains a file containing a list 
                          of CUIs in which the path information should be 
                          store for - if the CUI isn't on the list the path 
                          information for that CUI will not be stored

        'verbose'      -> This parameter will print out the table information 
                          to a config file in the UMLSINTERFACECONFIG directory

        'config'       -> This parameter contains the location of the config
                          file

    These are passed through a hash. For example:

        my %options = ();
        $options{'config'}   = $config;
        $options{'realtime'} = 1;

        my $interface = UMLS::Interface->new(\%options);

    Keep in mind that the database configuration options can be included in
    the MySQL my.cnf file. This is preferable. The directions for this are
    in the INSTALL file. It is Stage 5 Step D.

    These options can be reconfigured during run time using the reConfig()
    method.

        $options{'config'} = $newconfig;
        $interface->reConfig(\%options);

  USING THE MODULE
    Once the object of module is successfully created after following the
    steps described in the previous section, a number of methods can be
    called upon this object. The output of methods varies:

     @array refers to an array
     $array refers to a reference to an array
     $hash  refers to a reference to a hash

    The methods are as follows:

         my $root = $interface->root();                
             Returns the concept ID of the root of the tree.

         my $depth = $interface->depth();
             Returns the depth of the tree.

         my $version = $interface->version();
             Return the version of UMLS.

         my $bool = $interface->exists($cui);
             Determines if a CUI exists

         my $bool = $interface->validCui($cui);
             Checks if CUI is a valid concept

         my $array = $interface->getSab($cui);
             Returns the list of sources the concept exists in

         my $array = $interface->getConceptList($term);    
             Returns the list of all CUIs of a given term
             from the SAB parameter specified in the config 
             file or the default 

         my $array = $interface->getDefConceptList($term);
             Returns the list of all CUIs of a given term
             from the SABDEF parameter specified in the 
             config file or the default 

         my $array = $interface->getAllConcepts($term);
             Returns the list of all CUIs of a given term
             in the entire UMLS. 

         my $hash = $interface->getCuiList();              
             Returns a list of CUIs from the source(s) specified 
             in the configuration file

         my $array = $interface->getCuisFromSource($sab);
             Returns an list of CUIs in a specified source

         my $array = $interface->getCuisFromSource($sab);       
             Returns a list of CUIs from a specific source

         my $array = $interface->getTermList($cui);            
             Returns the list of terms and their sources using 
             the SAB parameter in the configuration file or the 
             default

         my $array = $interface->getDefTermList($cui);            
             Returns the list of terms and their sources using 
             the SABDEF parameter in the configuration file or 
             the default     

         my $array = $interface->getAllTerms($cui);             
             Returns the list of terms corresponding to a CUI 
             for all sources 

         my $hash = $interface->getCompounds();
             Returns all the compound terms in the sources specified in
             the configuration file.

         my $term = $interface->getPreferredTerms($cui);             
             Returns the preferred term of a CUI if that term 
             exists in the sources specified by the SAB parameter 
             in the configuration file or the default

         my $term = $interface->getAllPreferredTerms($cui);             
             Returns the preferred term of a CUI regardless
             of the source information in the configuration
             file

         my $array = $interface->getParents($cui);              
             Returns the parent of a given CUI

         my $array = $interface->getChildren($cui);             
             Returns the children of a given CUI

         my $array = $interface->getRelated($cui, $rel);              
             Returns the CUI relations of a given CUI and relation

         my $array = $interface->getRelationsBetweenCuis($cui1, $cui2);
             Returns the relations between two CUIs.

         my $array$interface->getRelations($cui);           
             Returns all of the relations associated with a CUI in 
             the sources specified in the configuration file 

         my $array = $interface->getCuiDef($cui);               
             Returns the definition(s) of a given CUI

         my $array = $interface->getExtendedDefinition($cui);
             Returns the extended definition of a given CUI


         my $array = $interface->getSt($cui);                  
             Returns the TUI(s) of the semantic type(s) associated 
             with a CUI

         my $abr = $interface->getStAbr($tui);                
             Returns the abbreviation of a semantic type of a TUI

         my $tui = $interface->getStTui($abr);
             Returns the TUI of an abbreviation of a semantic type

         my $string = $interface->getStString($abr);             
             Returns the name of the semantic type given its 

         my $def = $interface->getStDef($abr);                
             Returns the definition of a semantic type given its 
             abbreviation

         my $array = $interface->getSemanticRelation($st1, $st2);
             Returns a list of semantic relation between the two 
             semantic types. 

         my $array = $interface->getSemanticGroup($cui1);
             Returns a list of semantic groups of a given CUI

         my $array = $interface->getSemanticGroupOfSt($st);
             Returns a list of semantic groups of a given semantic type

         my $array = $interface->pathsToRoot($cui);             
             Returns a list of concept IDs that denote the path from 
             the input CUI to the root using the source and relation 
             information in the configuration file

         my $array = $interface->findShortestPath($cui1, $cui2);        
             Returns the shortest path between two CUIs

         my $array = $interface->findLeastCommonSubsumer($cui1, $cui2); 
             Returns the least common subsumer between two CUIs 

         my $min = $interface->findMinimumDepth($cui);        
             Returns the minimum depth of a CUI given the sources
             and relations specified in the configuration file

         my $max = $interface->findMaximumDepth($cui);        
             Returns the maximum depth of a CUI given the sources
             and relations specified in the configuration file

         my $int = $interface->findNumberOfCloserConcepts($cui1, $cui2);
             Returns the number of concepts closer to cui1 than cui2

         my $double = $interface->getIC($cui);                   
             Returns the information content of a CUI 
    
         my $double = $interface->getProbability($cui);
             Returns the probability of a concept

         my $int = $interface->getFrequency($cui);
             Returns the frequency of a CUI that was used to calculate
             its information content and probability

         my $N = $interface->getN();
             Returns the total number of CUIs the probabilities were 
             calculated with

         my $hash = $interface->getPropagationCuis();
             Returns a list of CUIs that the counts were propagated over

         my $hash = $interface->propagateCounts(\%hash);
             Returns the propagation counts of the input CUIs

          my $array = $interface->stPathsToRoot($tui);
             Returns all the path to the root information of the given
             semantic type (TUI)

         my $array = $interface->stFindShortestPath($tui1, $tui2);
             Returns the shortest paths between the two semantic types
             (TUIs)

         my $double = $interface->getStIC($tui);
             Returns the information content of a given semantic type (TUI)
 
         my $double = $interface->getStProbability($tui);
             Returns the probability of a given semantic type (TUI)

         my $stN = $interface->getStN();
             Returns the total number of semantic types used to 
             obtain the probability of a semantic type

         $interface->setPropagationParameters(\%parameters);
             Sets the propagation parameters 

         $interface->setStSmoothing();
             Sets the smoothing parameter to smooth the input counts

         my $hash = $interface->propagateStCounts(\%hash);
             Returns the propagation counts of the input semantic types

         $interface->loadStPropagationHash(\%hash);
             Load the propagation hash with probability counts
 

         my $hash = $interface->returnTableNames();        
             Returns the mysql database table names in human and hex form 
             created by the package for a given configuration

         $interface->dropConfigTable();               
             Drops the temporary table created by the UMLS-Interface 
             module of path information for a specified set of sources 

         $interface->removeConfigFiles();
             Removes the configuration files created by the 
             verbose option

    These methods essentially expose an interface as required by the
    UMLS::Similarity modules. The UMLS::Similarity modules require that any
    interface to a taxonomy provide the above methods.

  CONFIGURATION
    UMLS-Interface allows information to be extracted from the UMLS given a
    specified set of sources and relations through the use of a
    configuration file.

    There are six configuration options: SAB, REL, RELA, SABDEF, RELDEF, and
    RELADEF.

    The SAB and REL options are used to determine which sources and
    relations the path information is to be obtained from. The RELA option
    narrows down the relation even further. The RELA will only be applied to
    the PAR/CHD and RB/RN relations.

    The SABDEF and RELDEF options are used to determine which sources and
    relations to use when creating the EXTENDED DEFINITION. The RELA option
    narrows down the relation even further. The RELADEF will only be applied
    to the PAR/CHD and RB/RN relations.

    You can specify a single source, multiple sources or the entire UMLS
    (using the UMLS_ALL option). Keep in mind that the greater the number of
    sources the larger the search space so if you obtaining path information
    about two concepts this will take longer. The names of the sources in
    the configuration file are expected to be in the SAB (source
    abbreviation) form. A listing of the sources and their SABs can be
    found:

    <http://www.nlm.nih.gov/research/umls/knowledge_sources/metathesaurus/re
    lease/source_vocabularies.html>

    You can specify any relations that exist in the specified set of sources
    that you defined. The directional (hierarchical) relations though are
    PAR/CHD and RB/RN. The other relations (such as RO and SIB) are not
    directional which means when obtaining path information when using these
    relations may take much longer than obtaining path information using the
    directional relations. A listing of the different relations can be found
    here (scroll down to the REL table):

    <http://www.nlm.nih.gov/research/umls/knowledge_sources/metathesaurus/re
    lease/abbreviations.html>

    If you do plan on using a multiple sources or the entire UMLS, we would
    advise you to use the --realtime option which is explained below, in the
    Interface.pm documentation and the path programs in the utils/
    directory. We also have a am UMLS_ALL option for this so you do not have
    to specify each and every source and relation.

    The format of the configuration file is as follows:

    SAB :: <include|exclude> <source1, source2, ... sourceN>

    REL :: <include|exclude> <relation1, relation2, ... relationN>

    RELA :: <include|exclude> <rela1, rela2, ... relaN>

    For example, if we wanted to use the MSH vocabulary with only the RB/RN
    relations, the configuration file would be:

     SAB :: include MSH
     REL :: include RB, RN

    or

     SAB :: include MSH
     REL :: exclude PAR, CHD

    If we wanted to use the SNOMEDCT vocabulary with only the PAR/CHD
    relations that are is-a relations, the configuration file would be:

     SAB :: include SNOMEDCT
     REL :: include PAR, CHD 
     RELA :: include isa, inverse_isa

    The format for SABDEF and RELDEF is similar.

    The SABDEF and RELDEF options are used to determine the sources and
    relations the extended definition is to be obtained from.

    The format of the configuration file is as follows:

    SABDEF :: <include|exclude> <source1, source2, ... sourceN>

    RELDEF :: <include|exclude> <relation1, relation2, ... relationN>

    RELADEF :: <include|exclude> <rela1, rela2, ... relaN>

    Note: RELDEF takes any of MRREL relations and two special 'relations':

          1. CUI which refers to the CUIs definition

          2. TERM which refers to the terms associated with the CUI

    For example, if we wanted to use the definitions from MSH vocabulary and
    we only wanted the definition of the CUI and the definitions of the CUIs
    SIB relation, the configuration file would be:

     SABDEF :: include MSH
     RELDEF :: include CUI, SIB

    If you wanted only the PAR/CHD definitions which are is-a relations.

     SABDEF :: include MSH
     RELDEF :: include PAR, CHD
     RELADEF :: include isa, inverse_isa

    For all of these options, there is an UMLS_ALL tag. If used with SAB or
    SABDEF, it would include all of the UMLS sources. If used with the REL
    or RELDEF, it would include all of the possible relations (as well as
    CUI and TERM for RELDEF). If used with the RELA or RELADEF, it would
    include all of the RELA relations including those with no RELA relation.
    Note that this is also the default for this option which is why it is
    optional. An example of using the UMLS_ALL option is as follows:

     SAB :: include UMLS_ALL
     REL :: include UMLS_ALL

    and another is:

     SABDEF :: include UMLS_ALL
     RELDEF :: include UMLS_ALL

    If you go to the configuration file directory, there will be example
    configuration files for the different runs that you have performed.

    For more information about the configuration options please see the
    README.

  PROPAGATION
    The Information Content (IC) is defined as the negative log of the
    probability of a concept. The probability of a concept, c, is determine
    by summing the probability of the concept occurring in some text plus
    the probability its descendants occurring in some text.

    The following is an example of the method UMLS-Interface uses to
    propagation counts to determine the probability of a concept in the
    sources/relations specified in the configuration file. In this method,
    we percolate the counts up the hierarchy, and in the case of multiple
    inheritance, we send a full count up all the paths to the parent.

    The icfrequency file contains the frequency of the following concepts
    existing in some corpus. For example, our corpus consists of three
    concepts, A B & C, each occurring five times:

     SAB :: <sources>
     N:15
     A<>5
     B<>5
     C<>5

    In this case, our sources and relations consist of the following
    'graph': Notation....A->D means A is a child of D....

     A->D
     B->D
     B->E
     D->F
     C->E
     E->F

    So A B and C are "leaf" nodes and F is the root.

    Step 1: determine the descendants of each nodes

     Descendants(A) = {}
     Descendants(B) = {}
     Descendants(C) = {}
     Descendants(D) = {A, B}
     Descendants(E) = {B, C}
     Descendants(F) = {A, B, C, D, E, F}

    Step 2: determine the probability of a concept, P(c), occurring by
    summing the probability of each of descendants plus its probability.

     P(A) = freq(A) / N = .33
     P(B) = freq(B) / N = .33
     P(C) = freq(C) / N = .33
     P(D) = (freq(A)+freq(B)+freq(D)) / N = .66
     P(E) = (freq(B)+freq(C)+freq(E)) / N = .66
     P(F) = (freq(A)+freq(B)+freq(C)+freq(D)+freq(E)+freq(F)) / N = .99

    Step 3: print the probability of the concept occurring, P(c), for each
    node in the sources/relations defined in the configuration table.

     SMOOTH :: 0 <- or 1 if smoothing was used
     SAB :: <sources>
     REL :: <relations>
     RELA :: <relas>  <- if any are specified in the config
     A<>.33
     B<>.33
     C<>.33
     D<>.66
     E<>.66
     F<>.99

    The information content for the nodes is then calculated by taking -log
    of this probability.

    We have an option that incorporates Laplace smoothing. Laplace smoothing
    is where the frequency count of each of the concepts in the taxonomy is
    incremented by one. The advantage of doing this is that it avoids having
    a concept that has a probability of zero. The disadvantage is that it
    can shift the overall probability mass of the concepts from what is
    actually seen in the corpus.

  REFERENCING
        If you write a paper that has used UMLS-Interface in some way, we'd 
        certainly be grateful if you sent us a copy and referenced UMLS-Interface. 
        We have a published paper that provides a suitable reference:

        @inproceedings{McInnesPP09,
           title={{UMLS-Interface and UMLS-Similarity : Open Source 
                   Software for Measuring Paths and Semantic Similarity}}, 
           author={McInnes, B.T. and Pedersen, T. and Pakhomov, S.V.}, 
           booktitle={Proceedings of the American Medical Informatics 
                      Association (AMIA) Symposium},
           year={2009}, 
           month={November}, 
           address={San Fransisco, CA}
        }

        This paper is also found in
        <http://www-users.cs.umn.edu/~bthomson/publications/pubs.html>
        or
        <http://www.d.umn.edu/~tpederse/Pubs/amia09.pdf>

  CONTACT US
    If you have any trouble installing and using UMLS-Interface, please
    contact us via the users mailing list :

    umls-similarity@yahoogroups.com

    You can join this group by going to:

    <http://tech.groups.yahoo.com/group/umls-similarity/>

    You may also contact us directly if you prefer :

        Bridget T. McInnes: bthomson at cs.umn.edu

        Ted Pedersen      : tpederse at d.umn.edu

  SOFTWARE COPYRIGHT AND LICENSE
    Copyright (C) 2004-2010 Bridget T McInnes, Siddharth Patwardhan, Serguei
    Pakhomov, Ying Liu and Ted Pedersen

    This suite of programs is free software; you can redistribute it and/or
    modify it under the terms of the GNU General Public License as published
    by the Free Software Foundation; either version 2 of the License, or (at
    your option) any later version.

    This program is distributed in the hope that it will be useful, but
    WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
    Public License for more details.

    You should have received a copy of the GNU General Public License along
    with this program; if not, write to the Free Software Foundation, Inc.,
    59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

    Note: The text of the GNU General Public License is provided in the file
    'GPL.txt' that you should have received with this distribution.