Skip to content
The official Go client for Elasticsearch
Go Makefile Shell JavaScript CSS HTML Dockerfile
Branch: master
Clone or download

Latest commit

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci Remove the executable bit from file permissions Oct 14, 2019
.doc/examples Examples: Update the generated documentation examples May 13, 2020
.github/workflows CI: Add workflows for Github Actions Mar 12, 2020
.jenkins CI: Add the 7.8 configuration for Jenkins May 21, 2020
_examples Examples: Add the Kafka example for the BulkIndexer helper Mar 11, 2020
esapi API: Update the APIs for Elasticsearch 8.x (8a086ba05de) May 21, 2020
estransport Transport: Add support for global request headers Apr 28, 2020
esutil Util: Fix for writeMeta in BulkIndexer helper Apr 28, 2020
internal Generator: Docs: Fix incorrect skipping of examples based on digest May 13, 2020
.codecov.yml Remove the executable bit from file permissions Oct 14, 2019
.dockerignore Remove the executable bit from file permissions Oct 14, 2019
.gitignore Remove the executable bit from file permissions Oct 14, 2019
Dockerfile Dockerfile: Update "gotestsum" to 0.4.1 Feb 26, 2020
LICENSE Remove the executable bit from file permissions Oct 14, 2019
Makefile CI: Update the default certificates Apr 29, 2020
README.md README: Add note about persistent HTTP connections May 15, 2020
doc.go Ignore the ELASTICSEARCH_URL variable when address is passed in confi… Dec 10, 2019
elasticsearch.go Transport: Add support for global request headers Apr 28, 2020
elasticsearch_benchmark_test.go Add the short Apache 2 license header to code files Oct 14, 2019
elasticsearch_example_test.go Add the short Apache 2 license header to code files Oct 14, 2019
elasticsearch_integration_test.go Transport: Update the estransport.New() method to allow returning an … Feb 4, 2020
elasticsearch_internal_test.go Ignore the ELASTICSEARCH_URL variable when address is passed in confi… Dec 10, 2019
go.mod Update the package import path to use a "v8" suffix Apr 8, 2019
go.sum Remove the executable bit from file permissions Oct 14, 2019

README.md

go-elasticsearch

The official Go client for Elasticsearch.

GoDoc Go Report Card codecov.io Build Unit Integration API

Compatibility

The client major versions correspond to the compatible Elasticsearch major versions: to connect to Elasticsearch 7.x, use a 7.x version of the client, to connect to Elasticsearch 6.x, use a 6.x version of the client.

When using Go modules, include the version in the import path, and specify either an explicit version or a branch:

require github.com/elastic/go-elasticsearch/v7 7.x
require github.com/elastic/go-elasticsearch/v7 7.0.0

It's possible to use multiple versions of the client in a single project:

// go.mod
github.com/elastic/go-elasticsearch/v6 6.x
github.com/elastic/go-elasticsearch/v7 7.x

// main.go
import (
  elasticsearch6 "github.com/elastic/go-elasticsearch/v6"
  elasticsearch7 "github.com/elastic/go-elasticsearch/v7"
)
// ...
es6, _ := elasticsearch6.NewDefaultClient()
es7, _ := elasticsearch7.NewDefaultClient()

The master branch of the client is compatible with the current master branch of Elasticsearch.

Installation

Add the package to your go.mod file:

require github.com/elastic/go-elasticsearch/v8 master

Or, clone the repository:

git clone --branch master https://github.com/elastic/go-elasticsearch.git $GOPATH/src/github.com/elastic/go-elasticsearch

A complete example:

mkdir my-elasticsearch-app && cd my-elasticsearch-app

cat > go.mod <<-END
  module my-elasticsearch-app

  require github.com/elastic/go-elasticsearch/v8 master
END

cat > main.go <<-END
  package main

  import (
    "log"

    "github.com/elastic/go-elasticsearch/v8"
  )

  func main() {
    es, _ := elasticsearch.NewDefaultClient()
    log.Println(elasticsearch.Version)
    log.Println(es.Info())
  }
END

go run main.go

Usage

The elasticsearch package ties together two separate packages for calling the Elasticsearch APIs and transferring data over HTTP: esapi and estransport, respectively.

Use the elasticsearch.NewDefaultClient() function to create the client with the default settings.

es, err := elasticsearch.NewDefaultClient()
if err != nil {
  log.Fatalf("Error creating the client: %s", err)
}

res, err := es.Info()
if err != nil {
  log.Fatalf("Error getting response: %s", err)
}

defer res.Body.Close()
log.Println(res)

// [200 OK] {
//   "name" : "node-1",
//   "cluster_name" : "go-elasticsearch"
// ...

NOTE: It is critical to both close the response body and to consume it, in order to re-use persistent TCP connections in the default HTTP transport. If you're not interested in the response body, call io.Copy(ioutil.Discard, res.Body).

When you export the ELASTICSEARCH_URL environment variable, it will be used to set the cluster endpoint(s). Separate multiple adresses by a comma.

To set the cluster endpoint(s) programatically, pass a configuration object to the elasticsearch.NewClient() function.

cfg := elasticsearch.Config{
  Addresses: []string{
    "http://localhost:9200",
    "http://localhost:9201",
  },
  // ...
}
es, err := elasticsearch.NewClient(cfg)

To set the username and password, include them in the endpoint URL, or use the corresponding configuration options.

cfg := elasticsearch.Config{
  // ...
  Username: "foo",
  Password: "bar",
}

To set a custom certificate authority used to sign the certificates of cluster nodes, use the CACert configuration option.

cert, _ := ioutil.ReadFile(*cacert)

cfg := elasticsearch.Config{
  // ...
  CACert: cert,
}

To configure other HTTP settings, pass an http.Transport object in the configuration object.

cfg := elasticsearch.Config{
  Transport: &http.Transport{
    MaxIdleConnsPerHost:   10,
    ResponseHeaderTimeout: time.Second,
    TLSClientConfig: &tls.Config{
      MinVersion: tls.VersionTLS11,
      // ...
    },
    // ...
  },
}

See the _examples/configuration.go and _examples/customization.go files for more examples of configuration and customization of the client. See the _examples/security for an example of a security configuration.

The following example demonstrates a more complex usage. It fetches the Elasticsearch version from the cluster, indexes a couple of documents concurrently, and prints the search results, using a lightweight wrapper around the response body.

// $ go run _examples/main.go

package main

import (
  "bytes"
  "context"
  "encoding/json"
  "log"
  "strconv"
  "strings"
  "sync"

  "github.com/elastic/go-elasticsearch/v8"
  "github.com/elastic/go-elasticsearch/v8/esapi"
)

func main() {
  log.SetFlags(0)

  var (
    r  map[string]interface{}
    wg sync.WaitGroup
  )

  // Initialize a client with the default settings.
  //
  // An `ELASTICSEARCH_URL` environment variable will be used when exported.
  //
  es, err := elasticsearch.NewDefaultClient()
  if err != nil {
    log.Fatalf("Error creating the client: %s", err)
  }

  // 1. Get cluster info
  //
  res, err := es.Info()
  if err != nil {
    log.Fatalf("Error getting response: %s", err)
  }
  defer res.Body.Close()
  // Check response status
  if res.IsError() {
    log.Fatalf("Error: %s", res.String())
  }
  // Deserialize the response into a map.
  if err := json.NewDecoder(res.Body).Decode(&r); err != nil {
    log.Fatalf("Error parsing the response body: %s", err)
  }
  // Print client and server version numbers.
  log.Printf("Client: %s", elasticsearch.Version)
  log.Printf("Server: %s", r["version"].(map[string]interface{})["number"])
  log.Println(strings.Repeat("~", 37))

  // 2. Index documents concurrently
  //
  for i, title := range []string{"Test One", "Test Two"} {
    wg.Add(1)

    go func(i int, title string) {
      defer wg.Done()

      // Build the request body.
      var b strings.Builder
      b.WriteString(`{"title" : "`)
      b.WriteString(title)
      b.WriteString(`"}`)

      // Set up the request object.
      req := esapi.IndexRequest{
        Index:      "test",
        DocumentID: strconv.Itoa(i + 1),
        Body:       strings.NewReader(b.String()),
        Refresh:    "true",
      }

      // Perform the request with the client.
      res, err := req.Do(context.Background(), es)
      if err != nil {
        log.Fatalf("Error getting response: %s", err)
      }
      defer res.Body.Close()

      if res.IsError() {
        log.Printf("[%s] Error indexing document ID=%d", res.Status(), i+1)
      } else {
        // Deserialize the response into a map.
        var r map[string]interface{}
        if err := json.NewDecoder(res.Body).Decode(&r); err != nil {
          log.Printf("Error parsing the response body: %s", err)
        } else {
          // Print the response status and indexed document version.
          log.Printf("[%s] %s; version=%d", res.Status(), r["result"], int(r["_version"].(float64)))
        }
      }
    }(i, title)
  }
  wg.Wait()

  log.Println(strings.Repeat("-", 37))

  // 3. Search for the indexed documents
  //
  // Build the request body.
  var buf bytes.Buffer
  query := map[string]interface{}{
    "query": map[string]interface{}{
      "match": map[string]interface{}{
        "title": "test",
      },
    },
  }
  if err := json.NewEncoder(&buf).Encode(query); err != nil {
    log.Fatalf("Error encoding query: %s", err)
  }

  // Perform the search request.
  res, err = es.Search(
    es.Search.WithContext(context.Background()),
    es.Search.WithIndex("test"),
    es.Search.WithBody(&buf),
    es.Search.WithTrackTotalHits(true),
    es.Search.WithPretty(),
  )
  if err != nil {
    log.Fatalf("Error getting response: %s", err)
  }
  defer res.Body.Close()

  if res.IsError() {
    var e map[string]interface{}
    if err := json.NewDecoder(res.Body).Decode(&e); err != nil {
      log.Fatalf("Error parsing the response body: %s", err)
    } else {
      // Print the response status and error information.
      log.Fatalf("[%s] %s: %s",
        res.Status(),
        e["error"].(map[string]interface{})["type"],
        e["error"].(map[string]interface{})["reason"],
      )
    }
  }

  if err := json.NewDecoder(res.Body).Decode(&r); err != nil {
    log.Fatalf("Error parsing the response body: %s", err)
  }
  // Print the response status, number of results, and request duration.
  log.Printf(
    "[%s] %d hits; took: %dms",
    res.Status(),
    int(r["hits"].(map[string]interface{})["total"].(map[string]interface{})["value"].(float64)),
    int(r["took"].(float64)),
  )
  // Print the ID and document source for each hit.
  for _, hit := range r["hits"].(map[string]interface{})["hits"].([]interface{}) {
    log.Printf(" * ID=%s, %s", hit.(map[string]interface{})["_id"], hit.(map[string]interface{})["_source"])
  }

  log.Println(strings.Repeat("=", 37))
}

// Client: 8.0.0-SNAPSHOT
// Server: 8.0.0-SNAPSHOT
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// [201 Created] updated; version=1
// [201 Created] updated; version=1
// -------------------------------------
// [200 OK] 2 hits; took: 5ms
//  * ID=1, map[title:Test One]
//  * ID=2, map[title:Test Two]
// =====================================

As you see in the example above, the esapi package allows to call the Elasticsearch APIs in two distinct ways: either by creating a struct, such as IndexRequest, and calling its Do() method by passing it a context and the client, or by calling the Search() function on the client directly, using the option functions such as WithIndex(). See more information and examples in the package documentation.

The estransport package handles the transfer of data to and from Elasticsearch. At the moment, the implementation is really minimal: it only round-robins across the configured cluster endpoints. In future, more features — retrying failed requests, ignoring certain status codes, auto-discovering nodes in the cluster, and so on — will be added.

Helpers

The esutil package provides convenience helpers for working with the client. At the moment, it provides the esutil.JSONReader() and the esutil.BulkIndexer helpers.

Examples

The _examples folder contains a number of recipes and comprehensive examples to get you started with the client, including configuration and customization of the client, using a custom certificate authority (CA) for security (TLS), mocking the transport for unit tests, embedding the client in a custom type, building queries, performing requests individually and in bulk, and parsing the responses.

License

(c) 2019 Elasticsearch. Licensed under the Apache License, Version 2.0.

You can’t perform that action at this time.