Skip to content

How to Publish Resources

Jump to bottom
Marten edited this page Jun 9, 2019 · 4 revisions

The success of a geoportal depends on the quality resources it hosts. '''IMPORTANT: A resource must be published to the geoportal and approved before it can be discovered through the geoportal search.''' There are several ways to publish a resource. These are discussed in the sections below.

Note: Publishing privileges are only available to users who are registered in the geoportal organization's LDAP. To become a publisher, first Create a user account, and then send a message requesting to be made a publisher to the geoportal administrator (you can do this through the geoportal's Feedback link). Once the geoportal administrator has worked with his/her security administrator to grant your login Publisher status, you will be able to login to the geoportal as a Publisher.

Register a resource on the network

By registering a resource on the network, you provide connection information to a resource that can be revisited by the geoportal catalog for updates. This process is called ''synchronization''. For example, if you had a web-accessible folder (WAF) containing metadata records, you could register that folder and the geoportal would copy the records from the WAF to the geoportal catalog. Further, it would revisit the WAF frequently to check if there were new records, delete records in the geoportal catalog that were no longer in the WAF, and update existing records that had changed. There are several types of resources that can be registered, as discussed below. To register a resource, follow these steps:

##Step 1: Access the Resource Registration Page & Choose a Protocol

  • Login to the geoportal and click on the '''Administration''' tab.
  • Select the '''Add''' link at the top.
  • Choose '''Register a resource on the network'''.
  • On the next page, you will see a form with several sections. The first is Protocol Type. Select the protocol radio that corresponds to the resource which you want to synchronize, as follows.

Note - it is possible for an organization to support additional protocols through the Extending the Web Harvester customization.

URL

  • GetCapabilities URL to an OGC service (OWS, WMS, WCS, WFS, WPS, SOS, SPS, CSW, or KML)
  • REST URL formatted in GEORSS
  • OpenSearch description XML
  • URL to a metadata XML (such as FGDC, ISO-based profiles, Dublin Core, etc.)

ArcGIS

  • URL to an ArcGIS Server which contains services
  • URL to one specific service, e.g., MapServer, GlobeServer, ImageServer, GeoDataServer, MobileServer, GPServer, GeocodeServer, GeometryServer,NAServer, FeatureServer, SearchServer, WMSServer, WFSServer, and WCSServer

ESRI MS

  • ESRI Metadata Server service / ArcIMS metadata service

OAI

  • Open Archive Initiative service

WAF

  • URL to a web-accessible folder (HTTP/FTP)

CSW

THREDDS

  • [http://www.unidata.ucar.edu/publications/factsheets/2010sheets/thredds_factsheet.pdf THREDDS] data server catalog .xml endpoint. Geoportal can iterate through subfolders for harvest.

ATOM

ATOM Feeds compliant with ATOM 1.0 can be provided here. Each 'entry' will be harvested as one record. For example, to harvest public ArcGIS Online items into Geoportal Server (for example if you want to provide simple CSW discovery access to your ArcGIS Online subscription), register the RSS feed resulting from a search: http://www.arcgis.com/sharing/rest/search?q=park&f=rss. The search can be any ArcGIS Online search, including for a specific group or content owned by a specific user.

AGP-TO-AGP

Harvest items from one Portal for ArcGIS instance to another Portal for ArcGIS instance, click AGP-TO-AGP Harvesting with the Geoportal for details.

DCAT

  • URL to a DCAT-formatted JSON file.

Note: ArcIMS Image services, ArcIMS Feature services, and Z39.50 connections are not supported through the Register a network service feature.

Step 2: Fill out info that corresponds to the chosen protocol

After you have chosen the protocol, fields that correspond to that protocol will display. Enter information in the fields. The table below describes what each field means - note, only applies to protocols with additional fields.

Field Description
URL URL by which the geoportal should access the resource. The geoportal will use the information available from the URL to generate the metadata document describing the resource. You can test that the service is running by clicking the Test button next to the URL input box.
Title The title of the resource. Some resources, such as CS-W catalogs, have titles by default. Entering a title for a resource that already has a title will result in the new title being displayed on search results.

Additional fields for ArcGIS protocol

Field Description
REST Url The REST URL to the ArcGIS Server service.
SOAP Url Because the REST URL for an ArcGIS Server service is not explicitly associated with its SOAP URL, this allows the SOAP URL to be associated with the resource in the geoportal catalog. If the SOAP URL usually contains "?wsdl", leave this off of the URL, e.g., http://servername/arcgis/services instead of http://servername/arcgis/services?wsdl.

Additional fields for ESRI MS protocol

Field Description
Port Number Port upon which the metadata service is running.
Service Name Name of the metadata service to be accessed - e.g. Browse_Metadata.
User Name User name if service is secure.
User Password User password if service is secure.
Root Folder If you want to harvest only from a certain publisher's folder, indicate which folder here.

Additional fields for OAI protocol

Field Description
Prefix Prefix of the metadata records stored in the OAI database.
Set Name of the set or database from which you want to retrieve resources.

Additional fields for WAF protocol

Field Description
User Name If the folder is secure, then this is the username to access to the folder.
User Password If the folder is secure, then this is the password to access to the folder. Note: Subfolders will be automatically included.

Additional fields for CSW protocol

Field Description
Profile CSW profile used to communicate with the resource. If you are registering an Esri Geoportal Server, the CSW profile is usually ''ArcGIS Server Geoportal Extension (GPT)''. For a list of supported CSW profiles and details on how to add support for additional profiles to your geoportal implementation, see Supported CSW Profiles for Synchronization.

Additional fields for DCAT protocol

Field Description
Format Currently 'JSON'

Step 3: Choose additional options for Synchronization

  • After you have filled out the additional fields related to the Protocol Type, specify who will own the resources retrieved from the registered resources by choosing a user from the Resource ownership dropdown.
  • After defining the resource ownership, check the appropriate options in the '''Purpose of the registration''' section. Each option is described below:
    • Allow this resource to be found when searching for metadata: Checking this option means that the resource you are registering on the form will be discoverable when a user searches for resources on the search page.
    • Include this endpoint on the federated search list: This option is only available when the Protocol Type is CSW. Checking this option allows the resource to be listed as a remote catalog in the federated search on the Search page. See https://github.com/Esri/geoportal-server/wiki/How-to-Publish-Resources#wiki-Register_an_endpoint_for_federated_search for more details.
    • Allow this resource to be synchronized at an interval''': Checking this option indicates that this resource should be synchronized at a specified time interval, which is defined further down in the form. If you uncheck this option, then the next set of options are automatically disabled and you can proceed to the bottom of the form.
  • After checking the Purpose options, move to the '''Choose actions to be performed during synchronization''' section and check the appropriate options. Each is described below:
    • Update this resource description: This option means that when the resource is synchronized, the metadata describing the resource itself will be updated if the source endpoint has been updated. For example, if you register a CS-W catalog GetCapabilities endpoint, and between synchronization intervals the GetCapabilities information is updated, the resource's metadata will be also updated the next time the resource is synchronized.
    • Copy the entire contents of this remote catalog into the local catalog: Checking this option means that when synchronization occurs, all child resources within the registered resource will be harvested into the local geoportal catalog. For example, a CS-W endpoint containing 1000 records would harvest all 1000 records into the geoportal catalog each time synchronization occurs, if this option is checked. Automatically approve newly acquired resources: Checking this option means that all harvested child resources from the registered resource will be automatically approved after they are synchronized and published to the geoportal.
  • After checking the synchronization options, indicate '''How often do you want this resource so be synchronized''' by selecting one of the radios.
    • Periodically: This option let you define how frequent the resource is synchronized, select one of the options to define the frequency or select Manual synchronization only if you only want manual synchronization.
    • At the specified time: This option let you schedule synchronization to start at a specific time of a day, you can schedule it to start at a specific time of the day (e.g. 1:00 am everyday), a specific time of the week (e.g. 3:00 am of every Friday), or a specific time of the month (e.g. 4:00 am of every 3rd day of the month). Click the "Add time" button to add a specific time to the schedule.
  • Click the Create or Create and Close button at the bottom of the form. This publishes a metadata document that describes the resource. The resource will be added to the list of resources in the geoportal on the Administration > Manage page.

Step 4: Get Approval for your registered endpoint to be harvested

Even for registered endpoints, a geoportal Administrator must approve your endpoint before it can be scheduled for synchronization. See How to Manage and Edit Resources for more information on managing registered resources.

Step 5: View harvesting statistics

An administrator can check status of harvesting jobs in geoportal through the harvesting statistics page, this feature is very useful for an administrator to know if a harvesting job has run successfully, for example, if the harvesting statistics indicate too many pending jobs in last 24 hours and not much completed could be an issue an administrator need to investigate.

To show harvesting summary:

To show detailed harvesting statistics:

#Register an endpoint for federated search Federated search allows users to search a remote endpoint - such as a CSW endpoint, ArcGIS.com, YouTube, etc. - from the Geoportal search interface. The records returned from a federated search are not stored in the geoportal database, but at their respective endpoints. Out of the box, your geoportal is configured to search its own catalog and ArcGIS.com. You can see this when you go to the geoportal Search page, and expand the "Records shown from:" box to reveal a list of supported search endpoints (see 'federated search' in the table for the Geoportal Search topic - How-to-Search-for-Resources.

There are two ways to register an endpoint for federated search, as described below.

CSW endpoints

For CSW endpoints, it is possible to register the endpoint as follows:

  • Login to the geoportal as a publisher, and click the Administration tab. Then, click the "register a resource on the network" link.
  • In the Register a Network Resource form, enter the URL of the endpoint you wish to support in the federated search, and also the protocol that matches that endpoint. See Supported CSW Profiles for Synchronization if you need to know what CSW profiles are supported out of the box, or to configure the geoportal to support additional profiles.
  • Fill out the rest of the form, making sure to deselect the 'Allow this resource to be synchronized at an interval' option (which disables harvesting) and select the 'Include this endpoint on the federated search list' option (which includes the endpoint in the federated search list).
  • Save the form, and then have a geoportal Administrator user Approve the registered endpoint from the Administration page.
  • After approving the registered resource, you should see the endpoint listed in the federated search list on the Search page.

##OpenSearch endpoints For OpenSearch endpoints, transformation template information will need to be added into the geoportal's configuration file and is not possible through the geoportal interface at this time. For steps on how to update the geoportal's configuration file to support your OpenSearch endpoint, see Add an OpenSearch endpoint for Federated Search).

Once the resource has been either registered or configured, it will be available from the "Search In" option on the geoportal's search page.

#Upload or validate metadata file from the local disk Uploading a metadata XML file from your local machine or mapped network drive is a quick way to get one local metadata document published to the geoportal, or to check that a metadata record is valid. A valid document will have all the mandatory metadata elements populated, and will publish to the geoportal. An invalid document will fail to publish. To validate or upload a metadata file, follow these steps:

  • Login to the geoportal.
  • Click the Administration tab.
  • Click the Add link.
  • On the Add page, select Upload metadata file from the local disk from the choices listed.
  • On the Upload page, click Choose File and browse to the location of your metadata XML file.
  • Click OK to load the file into the interface.
  • If you want to validate the document, click Validate. If your document is invalid, the geoportal will report which elements in the document need to be addressed. If the metadata schema to which the document conforms is not supported by the geoportal, the error will say that the schema is invalid. If the document is valid, you will see a successful message.
  • When you are ready to upload the file, click Upload button. After a few moments, you will receive a message saying one of the following:
    • Your document was published successfully. This means that it is posted in the geoportal, and awaits administrator approval before it can be discovered through search.
    • Your document was replaced successfully. This means that it has replaced another existing document in the geoportal with this uploaded version. This occurs when you are republishing a document to update it. The geoportal checks the following items below in the order listed to identify if a document is replacing another. If the geoportal does not find a match using this criteria, the document will be duplicated:
      • Checks for an internal file identifier. Examples are an ISO fileIdentifier metadata element or an Esri/PublishedDocID element.
      • Checks the file name, looking for the following pattern: {uuid}.xml.
      • Checks where the document is coming from by looking at the source URI. Source URI is assigned at the time of publishing, and is stored in the geoportal database.
      • An error message. There are many different errors. Examples are failure to publish because the document is invalid, failure because the metadata schema is unrecognized, or that the document itself cannot be parsed (the document may not be xml, or a valid schema).
  • You can see your uploaded document by clicking on the '''Manage''' link at the top of the page. The document will be listed in the table of resources.

#Use dedicated editor to create metadata manually The geoportal supports creating standards-based metadata to describe your resource using the geoportal interface. Using the dedicated editor to create metadata manually is a good way to publish resource information when you don't have existing metadata to upload or the resource can't be registered as a resource on the network. The geoportal Create Metadata page supports several metadata schemas out of the box, and can be configured to support additional standards or profiles. For more information about metadata schemas supported by the Geoportal Server, see Customizations#Customize Metadata. To create metadata using the geoportal Create Metadata page, follow these steps:

  • Login to the geoportal as a publisher or an administrator.
  • Click the Administration tab.
  • Click the Add link.
  • On the Add page, select Use dedicated editor to create metadata manually from the choices listed.
  • On the next screen, a list of the available metadata schemas will display. Click the radial next to the schema to which you want your metadata document to conform.
  • On the next page, a form with many fields loads. Fill out as much information as possible about your data resource. Fields shown in bold-italic are mandatory; the form will not publish if these are not filled out. Other fields are optional. Some elements that are not mandatory become mandatory when its section is activated. Sections are portions of the form that define a group of information elements. If a section has a blank box next to it, then it is not mandatory but optional. To activate that portion of the form and enter in information, put a check in the blank box and the section will expand, revealing the now-mandatory and optional elements.
  • When you have finished filling out the form, there are options at the bottom of the screen. Choose any of them, according to their functionality.
  • Save as Draft: saves the metadata document with a "Draft" status. If you want to finish the record later, you will need to login, go to the Administration tab, select your draft document, and click the pencil icon to open the document in this Create Metadata form again.
  • Save: saves the information you have entered, and submits the form to the geoportal for publishing.
  • Validate: submits the metadata form to the geoportal and will report any errors or missing values necessary for the document to publish. Note that this option is not always available - if the online form uses the tabbed editing interface - for example, the ISO 19115, Gemini, or FGDC editors as of version 1.2 - the validation button will not appear because validation prompts will appear when you click Save or Save as Draft. #When you are ready to publish the metadata to the geoportal, click Save and your metadata will be published. Remember that the published metadata will not be retrieved in search results until it has been approved by the geoportal administrator.
  • You can see your created metadata by clicking the Administration tab. It will be listed in the table of resources.

Publish through the Geoportal Publish Client

If you use ArcGIS Desktop to author and update information associated with your resource, the easiest way to publish the resource to the geoportal is through the Geoportal Publish Client. The Publish Client allows you to publish many resources at one time. See the Geoportal Publish Client topic for more information on how to install and use the Publish Client.

Publish using the CSW INSERT transaction

It is possible to interact with the geoportal resource catalog directly using Catalog Service for the Web (CS-W) protocol. One of the supported transactions in the geoportal's CS-W service is 'insert'. Using a CS-W insert request, it is possible to insert a record into the geoportal catalog. For more information, see the topic on the geoportal's Catalog Service.

Publish through the REST API

It is possible to publish resource metadata using the HTTP PUT operation. In this scenario, you would need to have access to an XML Editor that can open and save XML documents using HTTP GET and HTTP PUT protocols. This publishing would work similar to the process of editing a metadata document using an XML editor (see How to Manage and Edit Resources).

As of version 1.2.6, it is possible for administrator users to POST a record and have it automatically approved using the 'approve=true' parameter. E.g., the POST URL would look like this: http://localhost:8080/geoportal/rest/manage/document?approve=true

This posts the record to the geoportal and if it passes validation, automatically approves it. This functionality is reserved for users in the administrator group alone.

Clone this wiki locally