eml-faq.xml

<?xml version="1.0"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
<!--
       '$RCSfile: eml-faq.xml,v $'
       Copyright: 1997-2002 Regents of the University of California,
                            University of New Mexico, and
                            Arizona State University
        Sponsors: National Center for Ecological Analysis and Synthesis and
                  Partnership for Interdisciplinary Studies of Coastal Oceans,
                     University of California Santa Barbara
                  Long-Term Ecological Research Network Office,
                     University of New Mexico
                  Center for Environmental Studies, Arizona State University
   Other funding: National Science Foundation (see README for details)
                  The David and Lucile Packard Foundation
     For Details: http://knb.ecoinformatics.org/

        '$Author: obrien $'
          '$Date: 2009-03-05 22:39:37 $'
      '$Revision: 1.29 $'

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
-->
<article class="faq">
  <title>EML Frequently Asked Questions</title>
  <para>
    <ulink url="./index.html">Back to EML Contents</ulink>
  </para>
  <qandaset>
    <qandaentry id="id.1">
      <question>
        <para>What is EML?</para>
      </question>
      <answer>
        <para>EML stands for Ecological Metadata Language. It exists as a set
        of XML Schema documents that allow for the structural expression of
        metadata necessary to document a typical data set in the ecological
        sciences.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.2">
      <question>
        <para>Who is responsible for EML?</para>
      </question>
      <answer>
        <para>The first two released versions of EML, EML 1.0 and EML 1.4.1
        were developed at the National Center for Ecological Analysis and
        Synthesis (NCEAS), University of California at Santa Barbara, in Santa
        Barbara, California USA. The effort to produce EML 2.x (and all of
        the beta releases preceding it) is organized through the EML Project,
        an open source, community oriented project
        dedicated to providing a high-quality metadata specification
        for describing data relevant to the ecological discipline.
        The project is completely comprised of
        <ulink url="https://raw.github.com/NCEAS/eml/master/members.md">voluntary project members</ulink> 
        who donate their time and experience in order to advance
        information management for ecology. Project decisions are made by
        consensus according to the voting procedures described in the <ulink url="http://www.ecoinformatics.org/charter.html">ecoinformatics.org
        Charter</ulink>.  Significant contributions for these recent releases
        have come from individuals at NCEAS, the Long Term Ecological
        Research Program (CAP, NET, KBS, JRN), and the Joseph W. Jones
        Ecological Research Center in Newton, GA.
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.3">
      <question>
        <para>Why would I want to use EML when FGDC now supports biological
        data through the CSDGM?</para>
      </question>
      <answer>
        <para>EML is modular and extensible.</para>
      </answer>
      <answer>
        <para>The Content Standard for Digital Geospatial Metadata (CSDGM)
        developed by the Federal Geographic Data Committee (FGDC) is a
        monolithic standard, and so it is difficult to mix and match parts of
        it with other standards -- mainly because of all of the spatial
        requirements. So, we built EML as a series of modules that can be
        linked together and can be linked to other metadata standards. This
        gives us the most flexibility, and given that we can easily translate
        into CSDGM compliant documents, there is little cost. Second, we're
        building advanced data processing tools that can automatically parse
        data sets and analyze them based on the EML metadata descriptions. Due
        to various shortcomings in the FGDC standard, mostly oriented around
        its tight focus on spatial data, we have found that the CSDGM isn't
        adequate for these needs, e.g., how can one add machine parsable,
        semantically oriented attribute tags to CSDGM? Answer, you can't,
        because it is monolithic and doesn't permit dynamic ties to other
        metadata specs -- the only extension method is via the administrative
        challenge of creating a superset of the CSDGM -- not very maintainable.
        In addition, the level of granularity for metadata in FGDC is very
        patchy -- it goes into great detail for spatial projections, etc., but
        is incredibly terse with respect to describing methods and non-standard
        data formats. This is appropriate in the spatial world where there are
        few data formats (&lt; 100, many sensor derived streams), but not so
        good in ecology where there is no standardization of data formats
        (&gt;&gt;&gt;5000, very few sensor derived).</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.4">
      <question>
        <para>Is there documentation for EML in English?</para>
      </question>
      <answer>
        <para>Yes, there is a <ulink url="index.html">formal
        specification</ulink> of EML describing its development history,
        architecture, and modules.  The intent of each module is described
        in narrative and there is a technical description of each module
        in XML notation. Included as part of the technical description is
        an element-by-element description of the module. We will eventually
        provide examples on usage.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.5">
      <question>
        <para>Why is EML such an important development?</para>
      </question>
      <answer>
        <para>The last decade has witnessed a tremendous explosion of
        ecological and environmental data, catalyzed by societal concerns and
        facilitated by advancing technologies. These data have the potential to
        greatly enhance understanding of the complexity of the biosphere.
        However, broad-scale or synthetic research is stymied because data are
        largely unorganized and inaccessible as a consequence of their
        tremendous heterogeneity, complexity, and spatial dispersion in many
        separate repositories. EML is the first content standard designed
        specifically to address these issues for ecological data. Wide adoption
        and use of EML will create exciting new opportunities for data
        discovery, access, integration and synthesis.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.6">
      <question>
        <para>How do I get EML?</para>
      </question>
      <answer>
        <para>All the documents associated with the EML development effort are
        available via the project web server at
        <ulink url="http://knb.ecoinformatics.org/software/eml/">
        http://knb.ecoinformatics.org/software/eml/</ulink>.

        These projects are licensed under the GPL (Gnu Public License)
        agreement and can be freely distributed and modified.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.7">
      <question>
        <para>The EML Schema documents are quite complex. An average ecologist
        probably cannot and more likely does not want to mark up content in an
        XML editor. How then do you get content into EML?</para>
      </question>
      <answer>
        <para>The Knowledge Network for Biocomplexity project has developed a
        software client specifically to address this need.
        <ulink url="http://knb.ecoinformatics.org/software/">Morpho</ulink>

        (after the butterfly genus) is written in java (making portable across
        computer platforms) combines an easy to use interface to EML with a
        number of tools to make it easier for ecologists to document data.
        These include a reverse-engineering wizard. Morpho is available from
        <ulink url="http://knb.ecoinformatics.org/software">
        http://knb.ecoinformatics.org/software</ulink>.  Morpho currently
        supports the EML 2.1.0 release.</para>

      </answer>
    </qandaentry>
    <qandaentry id="id.8">
      <question>
        <para>EML contains provisions for communication. Is it possible to
        document in EML dynamic online data resources?</para>
      </question>
      <answer>
        <para>Yes, there are provisions in the eml-physical module for
        descriptions of online data resources. The eml-physical module
        describes the structural characteristics of data formats as delivered
        over the wire or as found in a file system. One physical object (which
        can be a bytestream or an object in a file system) might contain
        multiple entities (for example, this would be typical in a MS Access
        file that contained multiple tables of data). However, it is typically
        used to describe a file or stream that is in some text-based format
        such as ASCII or UTF-8, and includes the information needed to parse
        the data stream to extract the entity and its attributes from the
        stream. There are 3 distribution types, online, offline, and inline. To
        describe an online dataset in EML you would populate the online element
        with the distribution information.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.9">
      <question>
        <para>Do I need to download special software to use EML?</para>
      </question>
      <answer>
        <para>No, but there is software available to work with EML. See
        <link linkend="id.7">FAQ 7</link>

        .</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.10">
      <question>
        <para>How can I get my existing metadata into EML?</para>
      </question>
      <answer>
        <para>There are several approaches that can be used to convert existing
        metadata into EML depending on what form your existing metadata
        take.</para>
        <itemizedlist>
          <listitem>
            <para>Case 1: If your metadata is currently in a text format
            (not stored in a database) use the following conversion methods.

            <orderedlist>
                <listitem>
                  <para>Write a script (PERL, PHP, JAVA, etc.) to convert the
                text into EML compliant XML.</para>
                </listitem>
                <listitem>
                  <para>Convert the text metadata into XHTML (HTML that is XML
                compliant). Write an XSLT script to transform the XHTML file
                into EML compliant XML.</para>
                </listitem>
                <listitem>
                  <para>Use an special purpose XML editor that generates EML (
                <ulink url="http://knb.ecoinformatics.org/software">
                Morpho</ulink>

                or
                <ulink url="http://ces.asu.edu/bdi/Subjects/xylographa">
                Xylographa</ulink>

                ) and manually retype the metadata.</para>
                </listitem>
                <listitem>
                  <para>Use a general purpose XML development tool such as XML
                Spy that can create a sample document from an XML Schema and
                retype the metadata manually.</para>
                </listitem>
                <listitem>
                  <para>Use a simple text editor and do everything from
                scratch.</para>
                </listitem>
                <listitem>
                  <para>Use specialized data transformation software such as the
                Data Junction suite to extract text data and then map it into
                an EML structure.</para>
                </listitem>
              </orderedlist>
            </para>
          </listitem>
          <listitem>
            <para>Case 2: If your metadata is stored in a relational database
            use the following conversion methods.
            <orderedlist>
                <listitem>
                  <para>Both Microsoft SQL Server and Oracle have utilities to
                generate XML from their database. If you use a tool like that,
                then you will have to write an XSLT script to transform the
                generated XML into EML.</para>
                </listitem>
                <listitem>
                  <para>Use a vendor neutral Database-to-XML generator such as
                <ulink url="http://cocoon.apache.org/">Cocoon</ulink>

                (an Apache open source free tool). Cocoon can query the
                database, generate XML, and has a tool for creating the XSL
                Transformation scripts to convert the first stage XML output
                into EML format.</para>
                </listitem>
                <listitem>
                  <para>Use a specialized tool such as
                <ulink url="http://ces.asu.edu/bdi/Subjects/Xanthoria/">
                Xanthoria</ulink>

                (like Cocoon in may respects, but is easier to use) to generate
                XML from the database. Then use a tool such as XML Spy or
                Stylus Studio to develop the XSLT script to convert the
                generated XML into EML compliant XML.</para>
                </listitem>
                <listitem>
                  <para>Use specialized data transformation software such as the
                Data Junction query the database and map it into an EML
                structure.</para>
                </listitem>
              </orderedlist>
            </para>
          </listitem>
          <listitem>
            <para>Case 3: If your metadata is already in XML but in some other
            form such as NBII or FGDC use the following conversion method.
            <orderedlist>
                <listitem>
                  <para>Write an XSLT script to convert from the current format
                to EML (e.g. FGDC to EML).</para>
                </listitem>
              </orderedlist>
            </para>
          </listitem>
        </itemizedlist>
        <para>NOTE: In each of the cases it may be necessary to add some
        additional metadata in order to produce EML compliant documents.
        <ulink url="http://knb.ecoinformatics.org/software">Morpho</ulink>

        will automatically create EML compliant metadata either by adding it
        for you or indicating that certain fields are mandatory.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.11">
      <question>
        <para>Once I convert my metadata into EML, what do I do with it? If I
        am storing all my metadata in text-based EML files, how am I supposed
        to query them or use them for data management?</para>
      </question>
      <answer>
        <para>EML is an exchange standard for communication of metadata but it
        can be used as the framework for a data management system.
        <ulink url="http://knb.ecoinformatics.org/software">Metacat</ulink>

        is a multipurpose XML metadata and data repository that is optimized
        for use with EML. If you store your metadata in a relational database
        management system or plan to then there are also solutions.
        <ulink url="http://xml.apache.org/cocoon/">Cocoon</ulink>

        and
        <ulink url="http://ces.asu.edu/bdi/Subjects/Xanthoria/">
        Xanthoria</ulink>

        are examples of programs that can get EML out of an RDBMS. Cocoon and
        Xanthoria are both java applications that use java database connection
        hooks and style sheets to retrieve and format data. Xanthoria is a
        light-weight solution and the XSLT stylesheets for EML 2.0 have already
        been written. This solution lets a site stick with the RDBMS system
        that they probably have integrated with their site management
        activities, yet also have their metadata exposed via EML.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.12">
      <question>
        <para>Does the modularity of EML mean that one description can be
        shared by many documents?</para>
      </question>
      <answer>
        <para>In a previous version, EML packages (via RDF like triples)
        supported linking across packages, so you could re-use the same
        document in multiple packages. In EML 2.0.0 Release Candidate 1 we
        redesigned the packaging structure to only allow linking within a
        single package. Thus, one could re-use a party description or attribute
        list within a package, but not across several. This is a compromise
        that keeps some reusability but has fewer management problems. Along
        with this change is an ability to put all metadata and data in a single
        document for transport -- while still not limiting ourselves to a
        monolithic structure. This has benefits (akin to db normalization) and
        costs (access control, ownership, and multiple update problems).</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.13">
      <question>
        <para>How are EML modules linked together?</para>
      </question>
      <answer>
        <para>With "id" attributes and "references" elements
        in each module. Certain modules within EML allow you to
        identify specific sub-trees with a unique identifier (id). This 
        identifier can then be used in place of content in other parts 
        of the EML document by placing it in a "references" element.</para>
      </answer>
      <answer>
        <para>Our general approach in EML has been to create ComplexTypes (CT)
        when we wanted a particular block to be reusable. This concept was
        extended for linking modules together by adding an optional attribute
        named "id" of type "xs:string" for each ComplexType. This allows us to
        uniquely address each block defined by a CT.
        For the "ResourceBase" CT, this id element replaces the "identifier"
        element and acts as the overall identifier for the package. The content
        model for each CT is a choice between the existing content model and a
        new element named "references" of type "xs:string". This element is
        used to hold a reference to an existing sub-tree identified by its id.
        This relationship between the "references" element and the "id"
        identifiers is enforced by defining a "key" for the "id"
        elements and a "keyref" for the "references" elements. This use of
        a key and keyref differs slightly from the XML Schema case because
        in XML Schema, keys can not be null, whereas we want people to be
        able to optionally omit the "id" attribute.  Consequently, we have
        incorporated the rules about the correspondence between keys and
        keyrefs into the EML specification, but not into the schemas directly.
        Thus, in order to validate that an EML document is valid EML, you must
        use a parser that understands the referencing system in EML and can
        check that it is used correctly.  An example system that handles this
        key validation is shipped with the EML distribution (see the
        "EML Parser").
        
        Here's a fragment of an example xml doc to illustrate:
        <programlisting><![CDATA[
            ...
            <creator id="id.p1">
              <individualName><surName>Jones</surName></individualName>
            </creator>
            <associatedParty>
              <references>id.p1</references>
              <role>lackey</role>
            </associatedParty>
            <contact>
              <references>id.p1</references>
            </contact>
            ...
        ]]></programlisting>
        </para>
        <para>This even works for types that extend other types as long as the
        subclass is the one that does the referencing (e.g., associatedParty
        can reference creator, but not vice versa). 
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.14">
      <question>
        <para>Can I put data into EML as well as metadata?</para>
      </question>
      <answer>
        <para>Yes, there are provisions in the eml-physical module for
        inclusion of data. The module describes the structural characteristics
        of data formats as delivered over the wire or as found in a file
        system. One physical object (which can be a bytestream or an object in
        a file system) might contain multiple entities (for example, this would
        be typical in a MS Access file that contained multiple tables of data).
        However, it is typically used to describe a file or stream that is in
        some text-based format such as ASCII or UTF-8, and includes the
        information needed to parse the data stream to extract the entity and
        its attributes from the stream. There are 3 distribution types, online,
        offline, and inline. To include data in EML you would populate the
        inline element with the data file described in the data format element.
        The data that is in the inline element should conform to the
        description provided by the eml-physical module.
        Binary data files can be included using Base64 encoding.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.15">
      <question>
        <para>What can I do with my EML structured metadata?</para>
      </question>
      <answer>
        <para>Tools are currently being developed to allow automated
        heterogeneous data integration, analytical processing and quality
        testing based on EML metadata. In general, using a metadata standard
        such as EML will lessen your data entropy and make it more useful to
        you and others in the future.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.16">
      <question>
        <para>Can I validate my EML documents against the DTD?</para>
      </question>
      <answer>
        <para>No. As of EML 2.0.0 we are no longer creating DTDs as part of
        the EML release. Only
        <ulink url="http://www.w3.org/XML/Schema">XML Schemas</ulink>

        will be released. Even then, there are some EML rules which are not
        expressible in XML Schema and for which you must use a specialized
        validator, such as the "EML Parser" that ships with the distribution.
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.17">
      <question>
        <para>Are there required elements in EML?</para>
      </question>
      <answer>
        <para>Yes, although we've made every attempt to limit required elements
        in the cause of flexibility there are a number of pieces of information
        required to make sense of the metadata document. To make the metadata
        more useful we do have recommended usages on the modules. See
        specification for details about required fields and recommended usage.
        In the future we may provide usage compliance information such that if
        you want your data and metadata to be useful in a particular analytical
        context you will be provided with those elements of EML that are
        required for that purpose.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.18">
      <question>
        <para>There appear to be multiple places to put some types of metadata
        in EML. How do I know which of these places is the right place for my
        information?</para>
      </question>
      <answer>
        <para>The <ulink url="index.html">EML Specification</ulink>
        describes each element in a detailed normative manner. EML is
        hierarchical so where you use different elements is very important. For
        instance, if you use a TemporalCoverage element and reference it to a
        dataset element, you are saying that that entire dataset took place
        during that time. If, instead, you reference it to a dataTable, you
        are saying that only that table was covered by that time period. You
        must gauge exactly what you are trying to describe in the structure
        that you are using. Questions about possible bugs in the definitions
        of elements can be posed via email to the
        <ulink url="mailto:eml-dev@ecoinformatics.org">eml-dev mailing
        list</ulink>
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.19">
      <question>
        <para>The differences between "method" and "protocol" seem
        to be very subtle in EML. How do I distinguish between
        the two?</para>
      </question>
      <answer>
        <para> The eml-methods module describes the methods followed
        in the creation of the dataset being described, including
        description of field, laboratory and processing steps,
        sampling methods and units, quality control procedures. The
        eml-methods module is used to describe the "actual" procedures
        that are used in the creation or the subsequent processing of
        a dataset. Likewise, eml-methods is used to describe
        processes that have been used to define / improve the quality
        of a data file, or to identify potential problems with the
        data file. The eml-protocol module is intended to be used to
        document a "prescribed" procedure, whereas the eml-method
        module is used to describe procedures that were actually
        performed. The distinction is that the use of the term
        "protocol" is used in the "prescriptive" sense, and the term
        "method" is used in the "descriptive" sense. This distinction
        allows managers to build a protocol library of well-known,
        established protocols (procedures), but also document what
        procedure was truly performed in relation to the established
        protocol. The method may have diverged from the protocol
        purposefully, or perhaps incidentally, but the procedural
        lineage is still preserved and understandable.
        </para>
        <para>The eml-methods module, like other modules, may be
        referenced via the &lt;references&gt; tag. This allows a method
        to be described once, and then used as a reference in other
        locations within the EML document via it's ID.
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.20">
      <question>
        <para>How can 'references' be treated in XSLT transformations of EML?</para>
      </question>
      <answer>
        <para>XSLT can be used to transform EML to other formats, but the treatment
        of 'references' elements is somewhat complicated. A text file describing the
        details of one method for handling the 'references' elements is available at
        <ulink url="http://knb.ecoinformatics.org/software/eml/eml-2.0.1/references_XSLT.txt">
        http://knb.ecoinformatics.org/software/eml/eml-2.0.1/references_XSLT.txt</ulink>
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.21">
      <question>
        <para>How do the elements of the Dublin Core Metadata Initiative map onto EML?</para>
      </question>
      <answer>
        <para>EML attempts to capture the metadata elements of the Dublin Core in its
        specification, while still maintaining the flexibility to be able to document the full range
        of ecological resources.  More detailed information on the representation of the Dublin
        Core elements in EML can be found at 
        <ulink url="http://knb.ecoinformatics.org/software/eml/eml-2.0.1/eml-dublinCore.html">
        http://knb.ecoinformatics.org/software/eml/eml-2.0.1/eml-dublinCore.html</ulink>
        </para>
      </answer>
    </qandaentry>
    <qandaentry id="id.22">
      <question>
        <para>I'm interested in contributing to EML. Can I?</para>
      </question>
      <answer>
        <para>We welcome contributions to this work in any form.
        Individuals who invest substantial amounts of time and make valuable
        contributions to the development and maintenance of EML (in the
        opinion of current project members) will be invited to become EML
        project members according to the rules set forth in the <ulink url="http://www.ecoinformatics.org/charter.html">ecoinformatics.org
        Charter</ulink>. Contributions can take many forms, including the
        development of the EML schemas, writing documentation, and helping
        with maintenance, among others.
        </para>
        <para>You can contact the
        <ulink url="mailto:eml-dev@ecoinformatics.org">eml-dev mailing
        list</ulink>
        if you would like to make a contribution in person-hours to this
        project and would like to discuss how that might occur.  In general,
        we want all of the help we can get!</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.23">
      <question>
        <para>Where can I get EML?</para>
      </question>
      <answer>
        <para>You can download archived releases from
        <ulink url="http://knb.ecoinformatics.org/software/eml/">
        http://knb.ecoinformatics.org/software/eml/</ulink>

        or you can check out the latest development version from 
        <ulink url="https://github.com/NCEAS/eml/">GitHub</ulink>

        server.</para>
      </answer>
    </qandaentry>
    <qandaentry id="id.24">
      <question>
        <para>How is EML 2.1.0 different from EML 2.0.0/1, and how do I upgrade my documents?</para>
      </question>
      <answer>
      <para>There are several improvements to EML in version 2.1.0 -- too many to put here. For more 
      information,  see the EML 2.1.0 Information page: <ulink url="./eml-210info.html">
      <citetitle>Information for EML 2.1.0 Document Authors</citetitle>.  </ulink>
      That page also contains instructions for using the 
      transformation stylesheet that comes with EML2.1.0 to convert v2.0.0/1 documents 
      to v2.1.0.</para>
      </answer>

    </qandaentry>
  </qandaset>
</article>