🍺 Add support for elasticsearch storage driver

This commit is contained in:
hacpai 2015-09-07 17:03:29 +08:00
parent cefada41b8
commit 3386114814
328 changed files with 43156 additions and 0 deletions

View File

@ -0,0 +1,29 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
/generator
/cluster-test/cluster-test
/cluster-test/*.log
/cluster-test/es-chaos-monkey
/spec
/tmp

View File

@ -0,0 +1,19 @@
sudo: false
language: go
go:
- 1.5
- tip
env:
matrix:
- ES_VERSION=1.6.2
- ES_VERSION=1.7.1
before_script:
- mkdir ${HOME}/elasticsearch
- wget http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-${ES_VERSION}.tar.gz
- tar -xzf elasticsearch-${ES_VERSION}.tar.gz -C ${HOME}/elasticsearch
- ${HOME}/elasticsearch/elasticsearch-${ES_VERSION}/bin/elasticsearch >& /dev/null &
- sleep 15

View File

@ -0,0 +1,27 @@
# How to contribute
Elastic is an open-source project and we are looking forward to each
contribution.
## Your Pull Request
To make it easy to review and understand your changes, please keep the
following things in mind before submitting your pull request:
* Work on the latest possible state of `olivere/elastic`.
* Create a branch dedicated to your change.
* If possible, write a test case which confirms your change.
* Make sure your changes and your tests work with all recent versions of
Elasticsearch. At the moment, we're targeting the current and the previous
release, e.g. the 1.4 and the 1.3 branch.
* Test your changes before creating a pull request (`go test ./...`).
* Don't mix several features or bug fixes in one pull request.
* Create a meaningful commit message.
* Explain your change, e.g. provide a link to the issue you are fixing and
probably a link to the Elasticsearch documentation and/or source code.
* Format your source with `go fmt`.
## Additional Resources
* [GitHub documentation](http://help.github.com/)
* [GitHub pull request documentation](http://help.github.com/send-pull-requests/)

View File

@ -0,0 +1,22 @@
# This is a list of people who have contributed code
# to the Elastic repository.
#
# It is just my small "thank you" to all those that helped
# making Elastic what it is.
#
# Please keep this list sorted.
Alexey Sharov [@nizsheanez](https://github.com/nizsheanez)
Conrad Pankoff [@deoxxa](https://github.com/deoxxa)
Corey Scott [@corsc](https://github.com/corsc)
Gerhard Häring [@ghaering](https://github.com/ghaering)
Guilherme Silveira [@guilherme-santos](https://github.com/guilherme-santos)
Jack Lindamood [@cep21](https://github.com/cep21)
Junpei Tsuji [@jun06t](https://github.com/jun06t)
Maciej Lisiewski [@c2h5oh](https://github.com/c2h5oh)
Mara Kim [@autochthe](https://github.com/autochthe)
Medhi Bechina [@mdzor](https://github.com/mdzor)
Nicholas Wolff [@nwolff](https://github.com/nwolff)
Sacheendra talluri [@sacheendra](https://github.com/sacheendra)
Sean DuBois [@Sean-Der](https://github.com/Sean-Der)
zakthomas [@zakthomas](https://github.com/zakthomas)

View File

@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright © 2012-2015 Oliver Eilhard
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.

View File

@ -0,0 +1,419 @@
# Elastic
Elastic is an [Elasticsearch](http://www.elasticsearch.org/) client for the
[Go](http://www.golang.org/) programming language.
[![Build Status](https://travis-ci.org/olivere/elastic.svg?branch=master)](https://travis-ci.org/olivere/elastic)
[![Godoc](http://img.shields.io/badge/godoc-reference-blue.svg?style=flat)](https://godoc.org/github.com/olivere/elastic)
[![license](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://raw.githubusercontent.com/olivere/elastic/master/LICENSE)
See the [wiki](https://github.com/olivere/elastic/wiki) for additional information about Elastic.
## Releases
**Notice that the master branch always refers to the latest version of Elastic. If you want to use stable versions of Elastic, you should use the packages released via [gopkg.in](https://gopkg.in).**
Here's the version matrix:
Elasticsearch version | Elastic version -| Package URL
----------------------|------------------|------------
2.x | 3.0 **beta** | [`gopkg.in/olivere/elastic.v3-unstable`](https://gopkg.in/olivere/elastic.v3-unstable) ([source](https://github.com/olivere/elastic/tree/release-branch.v3) [doc](http://godoc.org/gopkg.in/olivere/elastic.v3-unstable))
1.x | 2.0 | [`gopkg.in/olivere/elastic.v2`](https://gopkg.in/olivere/elastic.v2) ([source](https://github.com/olivere/elastic/tree/release-branch.v2) [doc](http://godoc.org/gopkg.in/olivere/elastic.v2))
0.9-1.3 | 1.0 | [`gopkg.in/olivere/elastic.v1`](https://gopkg.in/olivere/elastic.v1) ([source](https://github.com/olivere/elastic/tree/release-branch.v1) [doc](http://godoc.org/gopkg.in/olivere/elastic.v1))
**Example:**
You have Elasticsearch 1.6.0 installed and want to use Elastic. As listed above, you should use Elastic 2.0. So you first install Elastic 2.0.
```sh
$ go get gopkg.in/olivere/elastic.v2
```
Then you use it via the following import path:
```go
import "gopkg.in/olivere/elastic.v2"
```
### Elastic 3.0
Elastic 3.0 targets Elasticsearch 2.x and is currently under [active development](https://github.com/olivere/elastic/tree/release-branch.v3). It is not published to gokpg yet.
There are a lot of [breaking changes in Elasticsearch 2.0](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-2.0.html) and we will use this as an opportunity to [clean up and refactor Elastic as well](https://github.com/olivere/elastic/blob/release-branch.v3/CHANGELOG-3.0.md).
### Elastic 2.0
Elastic 2.0 targets Elasticsearch 1.x and published via [`gopkg.in/olivere/elastic.v2`](https://gopkg.in/olivere/elastic.v2).
### Elastic 1.0
Elastic 1.0 is deprecated. You should really update Elasticsearch and Elastic
to a recent version.
However, if you cannot update for some reason, don't worry. Version 1.0 is
still available. All you need to do is go-get it and change your import path
as described above.
## Status
We use Elastic in production since 2012. Although Elastic is quite stable
from our experience, we don't have a stable API yet. The reason for this
is that Elasticsearch changes quite often and at a fast pace.
At this moment we focus on features, not on a stable API.
Having said that, there have been no big API changes that required you
to rewrite your application big time.
More often than not it's renaming APIs and adding/removing features
so that we are in sync with the Elasticsearch API.
Elastic has been used in production with the following Elasticsearch versions:
0.90, 1.0, 1.1, 1.2, 1.3, 1.4, and 1.5.
Furthermore, we use [Travis CI](https://travis-ci.org/)
to test Elastic with the most recent versions of Elasticsearch and Go.
See the [.travis.yml](https://github.com/olivere/elastic/blob/master/.travis.yml)
file for the exact matrix and [Travis](https://travis-ci.org/olivere/elastic)
for the results.
Elasticsearch has quite a few features. A lot of them are
not yet implemented in Elastic (see below for details).
I add features and APIs as required. It's straightforward
to implement missing pieces. I'm accepting pull requests :-)
Having said that, I hope you find the project useful.
## Usage
The first thing you do is to create a Client. The client connects to
Elasticsearch on http://127.0.0.1:9200 by default.
You typically create one client for your app. Here's a complete example.
```go
// Create a client
client, err := elastic.NewClient()
if err != nil {
// Handle error
}
// Create an index
_, err = client.CreateIndex("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
// Add a document to the index
tweet := Tweet{User: "olivere", Message: "Take Five"}
_, err = client.Index().
Index("twitter").
Type("tweet").
Id("1").
BodyJson(tweet).
Do()
if err != nil {
// Handle error
panic(err)
}
// Search with a term query
termQuery := elastic.NewTermQuery("user", "olivere")
searchResult, err := client.Search().
Index("twitter"). // search in index "twitter"
Query(&termQuery). // specify the query
Sort("user", true). // sort by "user" field, ascending
From(0).Size(10). // take documents 0-9
Pretty(true). // pretty print request and response JSON
Do() // execute
if err != nil {
// Handle error
panic(err)
}
// searchResult is of type SearchResult and returns hits, suggestions,
// and all kinds of other information from Elasticsearch.
fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)
// Each is a convenience function that iterates over hits in a search result.
// It makes sure you don't need to check for nil values in the response.
// However, it ignores errors in serialization. If you want full control
// over iterating the hits, see below.
var ttyp Tweet
for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) {
if t, ok := item.(Tweet); ok {
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
}
// TotalHits is another convenience function that works even when something goes wrong.
fmt.Printf("Found a total of %d tweets\n", searchResult.TotalHits())
// Here's how you iterate through results with full control over each step.
if searchResult.Hits != nil {
fmt.Printf("Found a total of %d tweets\n", searchResult.Hits.TotalHits)
// Iterate through results
for _, hit := range searchResult.Hits.Hits {
// hit.Index contains the name of the index
// Deserialize hit.Source into a Tweet (could also be just a map[string]interface{}).
var t Tweet
err := json.Unmarshal(*hit.Source, &t)
if err != nil {
// Deserialization failed
}
// Work with tweet
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
} else {
// No hits
fmt.Print("Found no tweets\n")
}
// Delete the index again
_, err = client.DeleteIndex("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
```
See the [wiki](https://github.com/olivere/elastic/wiki) for more details.
## API Status
Here's the current API status.
### APIs
- [x] Search (most queries, filters, facets, aggregations etc. are implemented: see below)
- [x] Index
- [x] Get
- [x] Delete
- [x] Delete By Query
- [x] Update
- [x] Multi Get
- [x] Bulk
- [ ] Bulk UDP
- [ ] Term vectors
- [ ] Multi term vectors
- [x] Count
- [ ] Validate
- [x] Explain
- [x] Search
- [ ] Search shards
- [x] Search template
- [x] Facets (most are implemented, see below)
- [x] Aggregates (most are implemented, see below)
- [x] Multi Search
- [x] Percolate
- [ ] More like this
- [ ] Benchmark
### Indices
- [x] Create index
- [x] Delete index
- [x] Get index
- [x] Indices exists
- [x] Open/close index
- [x] Put mapping
- [x] Get mapping
- [ ] Get field mapping
- [x] Types exist
- [x] Delete mapping
- [x] Index aliases
- [ ] Update indices settings
- [x] Get settings
- [ ] Analyze
- [x] Index templates
- [ ] Warmers
- [ ] Status
- [x] Indices stats
- [ ] Indices segments
- [ ] Indices recovery
- [ ] Clear cache
- [x] Flush
- [x] Refresh
- [x] Optimize
- [ ] Upgrade
### Snapshot and Restore
- [ ] Snapshot
- [ ] Restore
- [ ] Snapshot status
- [ ] Monitoring snapshot/restore progress
- [ ] Partial restore
### Cat APIs
Not implemented. Those are better suited for operating with Elasticsearch
on the command line.
### Cluster
- [x] Health
- [x] State
- [x] Stats
- [ ] Pending cluster tasks
- [ ] Cluster reroute
- [ ] Cluster update settings
- [ ] Nodes stats
- [x] Nodes info
- [ ] Nodes hot_threads
- [ ] Nodes shutdown
### Search
- [x] Inner hits (for ES >= 1.5.0; see [docs](www.elastic.co/guide/en/elasticsearch/reference/1.5/search-request-inner-hits.html))
### Query DSL
#### Queries
- [x] `match`
- [x] `multi_match`
- [x] `bool`
- [x] `boosting`
- [ ] `common_terms`
- [ ] `constant_score`
- [x] `dis_max`
- [x] `filtered`
- [x] `fuzzy_like_this_query` (`flt`)
- [x] `fuzzy_like_this_field_query` (`flt_field`)
- [x] `function_score`
- [x] `fuzzy`
- [ ] `geo_shape`
- [x] `has_child`
- [x] `has_parent`
- [x] `ids`
- [ ] `indices`
- [x] `match_all`
- [x] `mlt`
- [x] `mlt_field`
- [x] `nested`
- [x] `prefix`
- [x] `query_string`
- [x] `simple_query_string`
- [x] `range`
- [x] `regexp`
- [ ] `span_first`
- [ ] `span_multi_term`
- [ ] `span_near`
- [ ] `span_not`
- [ ] `span_or`
- [ ] `span_term`
- [x] `term`
- [x] `terms`
- [ ] `top_children`
- [x] `wildcard`
- [ ] `minimum_should_match`
- [ ] `multi_term_query_rewrite`
- [x] `template_query`
#### Filters
- [x] `and`
- [x] `bool`
- [x] `exists`
- [ ] `geo_bounding_box`
- [x] `geo_distance`
- [ ] `geo_distance_range`
- [x] `geo_polygon`
- [ ] `geoshape`
- [ ] `geohash`
- [x] `has_child`
- [x] `has_parent`
- [x] `ids`
- [ ] `indices`
- [x] `limit`
- [x] `match_all`
- [x] `missing`
- [x] `nested`
- [x] `not`
- [x] `or`
- [x] `prefix`
- [x] `query`
- [x] `range`
- [x] `regexp`
- [ ] `script`
- [x] `term`
- [x] `terms`
- [x] `type`
### Facets
- [x] Terms
- [x] Range
- [x] Histogram
- [x] Date Histogram
- [x] Filter
- [x] Query
- [x] Statistical
- [x] Terms Stats
- [x] Geo Distance
### Aggregations
- [x] min
- [x] max
- [x] sum
- [x] avg
- [x] stats
- [x] extended stats
- [x] value count
- [x] percentiles
- [x] percentile ranks
- [x] cardinality
- [x] geo bounds
- [x] top hits
- [ ] scripted metric
- [x] global
- [x] filter
- [x] filters
- [x] missing
- [x] nested
- [x] reverse nested
- [x] children
- [x] terms
- [x] significant terms
- [x] range
- [x] date range
- [x] ipv4 range
- [x] histogram
- [x] date histogram
- [x] geo distance
- [x] geohash grid
### Sorting
- [x] Sort by score
- [x] Sort by field
- [x] Sort by geo distance
- [x] Sort by script
### Scan
Scrolling through documents (e.g. `search_type=scan`) are implemented via
the `Scroll` and `Scan` services. The `ClearScroll` API is implemented as well.
## How to contribute
Read [the contribution guidelines](https://github.com/olivere/elastic/blob/master/CONTRIBUTING.md).
## Credits
Thanks a lot for the great folks working hard on
[Elasticsearch](http://www.elasticsearch.org/)
and
[Go](http://www.golang.org/).
## LICENSE
MIT-LICENSE. See [LICENSE](http://olivere.mit-license.org/)
or the LICENSE file provided in the repository for details.

View File

@ -0,0 +1,107 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
)
type AliasService struct {
client *Client
actions []aliasAction
pretty bool
}
type aliasAction struct {
// "add" or "remove"
Type string
// Index name
Index string
// Alias name
Alias string
// Filter
Filter *Filter
}
func NewAliasService(client *Client) *AliasService {
builder := &AliasService{
client: client,
actions: make([]aliasAction, 0),
}
return builder
}
func (s *AliasService) Pretty(pretty bool) *AliasService {
s.pretty = pretty
return s
}
func (s *AliasService) Add(indexName string, aliasName string) *AliasService {
action := aliasAction{Type: "add", Index: indexName, Alias: aliasName}
s.actions = append(s.actions, action)
return s
}
func (s *AliasService) AddWithFilter(indexName string, aliasName string, filter *Filter) *AliasService {
action := aliasAction{Type: "add", Index: indexName, Alias: aliasName, Filter: filter}
s.actions = append(s.actions, action)
return s
}
func (s *AliasService) Remove(indexName string, aliasName string) *AliasService {
action := aliasAction{Type: "remove", Index: indexName, Alias: aliasName}
s.actions = append(s.actions, action)
return s
}
func (s *AliasService) Do() (*AliasResult, error) {
// Build url
path := "/_aliases"
// Parameters
params := make(url.Values)
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
// Actions
body := make(map[string]interface{})
actionsJson := make([]interface{}, 0)
for _, action := range s.actions {
actionJson := make(map[string]interface{})
detailsJson := make(map[string]interface{})
detailsJson["index"] = action.Index
detailsJson["alias"] = action.Alias
if action.Filter != nil {
detailsJson["filter"] = (*action.Filter).Source()
}
actionJson[action.Type] = detailsJson
actionsJson = append(actionsJson, actionJson)
}
body["actions"] = actionsJson
// Get response
res, err := s.client.PerformRequest("POST", path, params, body)
if err != nil {
return nil, err
}
// Return results
ret := new(AliasResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of an alias request.
type AliasResult struct {
Acknowledged bool `json:"acknowledged"`
}

View File

@ -0,0 +1,123 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
const (
testAliasName = "elastic-test-alias"
)
func TestAliasLifecycle(t *testing.T) {
var err error
client := setupTestClientAndCreateIndex(t)
// Some tweets
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "sandrae", Message: "Cycling is fun."}
tweet3 := tweet{User: "olivere", Message: "Another unrelated topic."}
// Add tweets to first index
_, err = client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
// Add tweets to second index
_, err = client.Index().Index(testIndexName2).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
// Flush
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName2).Do()
if err != nil {
t.Fatal(err)
}
/*
// Alias should not yet exist
aliasesResult1, err := client.Aliases().Do()
if err != nil {
t.Fatal(err)
}
if len(aliasesResult1.Indices) != 0 {
t.Errorf("expected len(AliasesResult.Indices) = %d; got %d", 0, len(aliasesResult1.Indices))
}
*/
// Add both indices to a new alias
aliasCreate, err := client.Alias().
Add(testIndexName, testAliasName).
Add(testIndexName2, testAliasName).
//Pretty(true).
Do()
if err != nil {
t.Fatal(err)
}
if !aliasCreate.Acknowledged {
t.Errorf("expected AliasResult.Acknowledged %v; got %v", true, aliasCreate.Acknowledged)
}
// Search should return all 3 tweets
matchAll := NewMatchAllQuery()
searchResult1, err := client.Search().Index(testAliasName).Query(&matchAll).Do()
if err != nil {
t.Fatal(err)
}
if searchResult1.Hits == nil {
t.Errorf("expected SearchResult.Hits != nil; got nil")
}
if searchResult1.Hits.TotalHits != 3 {
t.Errorf("expected SearchResult.Hits.TotalHits = %d; got %d", 3, searchResult1.Hits.TotalHits)
}
/*
// Alias should return both indices
aliasesResult2, err := client.Aliases().Do()
if err != nil {
t.Fatal(err)
}
if len(aliasesResult2.Indices) != 2 {
t.Errorf("expected len(AliasesResult.Indices) = %d; got %d", 2, len(aliasesResult2.Indices))
}
*/
// Remove first index should remove two tweets, so should only yield 1
aliasRemove1, err := client.Alias().
Remove(testIndexName, testAliasName).
//Pretty(true).
Do()
if err != nil {
t.Fatal(err)
}
if !aliasRemove1.Acknowledged {
t.Errorf("expected AliasResult.Acknowledged %v; got %v", true, aliasRemove1.Acknowledged)
}
searchResult2, err := client.Search().Index(testAliasName).Query(&matchAll).Do()
if err != nil {
t.Fatal(err)
}
if searchResult2.Hits == nil {
t.Errorf("expected SearchResult.Hits != nil; got nil")
}
if searchResult2.Hits.TotalHits != 1 {
t.Errorf("expected SearchResult.Hits.TotalHits = %d; got %d", 1, searchResult2.Hits.TotalHits)
}
}

View File

@ -0,0 +1,160 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type AliasesService struct {
client *Client
indices []string
pretty bool
}
func NewAliasesService(client *Client) *AliasesService {
builder := &AliasesService{
client: client,
indices: make([]string, 0),
}
return builder
}
func (s *AliasesService) Pretty(pretty bool) *AliasesService {
s.pretty = pretty
return s
}
func (s *AliasesService) Index(indexName string) *AliasesService {
s.indices = append(s.indices, indexName)
return s
}
func (s *AliasesService) Indices(indexNames ...string) *AliasesService {
s.indices = append(s.indices, indexNames...)
return s
}
func (s *AliasesService) Do() (*AliasesResult, error) {
var err error
// Build url
path := "/"
// Indices part
indexPart := make([]string, 0)
for _, index := range s.indices {
index, err = uritemplates.Expand("{index}", map[string]string{
"index": index,
})
if err != nil {
return nil, err
}
indexPart = append(indexPart, index)
}
path += strings.Join(indexPart, ",")
// TODO Add types here
// Search
path += "/_aliases"
// Parameters
params := make(url.Values)
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
// Get response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// {
// "indexName" : {
// "aliases" : {
// "alias1" : { },
// "alias2" : { }
// }
// },
// "indexName2" : {
// ...
// },
// }
indexMap := make(map[string]interface{})
if err := json.Unmarshal(res.Body, &indexMap); err != nil {
return nil, err
}
// Each (indexName, _)
ret := &AliasesResult{
Indices: make(map[string]indexResult),
}
for indexName, indexData := range indexMap {
indexOut, found := ret.Indices[indexName]
if !found {
indexOut = indexResult{Aliases: make([]aliasResult, 0)}
}
// { "aliases" : { ... } }
indexDataMap, ok := indexData.(map[string]interface{})
if ok {
aliasesData, ok := indexDataMap["aliases"].(map[string]interface{})
if ok {
for aliasName, _ := range aliasesData {
aliasRes := aliasResult{AliasName: aliasName}
indexOut.Aliases = append(indexOut.Aliases, aliasRes)
}
}
}
ret.Indices[indexName] = indexOut
}
return ret, nil
}
// -- Result of an alias request.
type AliasesResult struct {
Indices map[string]indexResult
}
type indexResult struct {
Aliases []aliasResult
}
type aliasResult struct {
AliasName string
}
func (ar AliasesResult) IndicesByAlias(aliasName string) []string {
indices := make([]string, 0)
for indexName, indexInfo := range ar.Indices {
for _, aliasInfo := range indexInfo.Aliases {
if aliasInfo.AliasName == aliasName {
indices = append(indices, indexName)
}
}
}
return indices
}
func (ir indexResult) HasAlias(aliasName string) bool {
for _, alias := range ir.Aliases {
if alias.AliasName == aliasName {
return true
}
}
return false
}

View File

@ -0,0 +1,146 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestAliases(t *testing.T) {
var err error
client := setupTestClientAndCreateIndex(t)
// Some tweets
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "sandrae", Message: "Cycling is fun."}
tweet3 := tweet{User: "olivere", Message: "Another unrelated topic."}
// Add tweets to first index
_, err = client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
// Add tweets to second index
_, err = client.Index().Index(testIndexName2).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
// Flush
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName2).Do()
if err != nil {
t.Fatal(err)
}
// Alias should not yet exist
aliasesResult1, err := client.Aliases().
Indices(testIndexName, testIndexName2).
//Pretty(true).
Do()
if err != nil {
t.Fatal(err)
}
if len(aliasesResult1.Indices) != 2 {
t.Errorf("expected len(AliasesResult.Indices) = %d; got %d", 2, len(aliasesResult1.Indices))
}
for indexName, indexDetails := range aliasesResult1.Indices {
if len(indexDetails.Aliases) != 0 {
t.Errorf("expected len(AliasesResult.Indices[%s].Aliases) = %d; got %d", indexName, 0, len(indexDetails.Aliases))
}
}
// Add both indices to a new alias
aliasCreate, err := client.Alias().
Add(testIndexName, testAliasName).
Add(testIndexName2, testAliasName).
//Pretty(true).
Do()
if err != nil {
t.Fatal(err)
}
if !aliasCreate.Acknowledged {
t.Errorf("expected AliasResult.Acknowledged %v; got %v", true, aliasCreate.Acknowledged)
}
// Alias should now exist
aliasesResult2, err := client.Aliases().
Indices(testIndexName, testIndexName2).
//Pretty(true).
Do()
if err != nil {
t.Fatal(err)
}
if len(aliasesResult2.Indices) != 2 {
t.Errorf("expected len(AliasesResult.Indices) = %d; got %d", 2, len(aliasesResult2.Indices))
}
for indexName, indexDetails := range aliasesResult2.Indices {
if len(indexDetails.Aliases) != 1 {
t.Errorf("expected len(AliasesResult.Indices[%s].Aliases) = %d; got %d", indexName, 1, len(indexDetails.Aliases))
}
}
// Check the reverse function:
indexInfo1, found := aliasesResult2.Indices[testIndexName]
if !found {
t.Errorf("expected info about index %s = %v; got %v", testIndexName, true, found)
}
aliasFound := indexInfo1.HasAlias(testAliasName)
if !aliasFound {
t.Errorf("expected alias %s to include index %s; got %v", testAliasName, testIndexName, aliasFound)
}
// Check the reverse function:
indexInfo2, found := aliasesResult2.Indices[testIndexName2]
if !found {
t.Errorf("expected info about index %s = %v; got %v", testIndexName, true, found)
}
aliasFound = indexInfo2.HasAlias(testAliasName)
if !aliasFound {
t.Errorf("expected alias %s to include index %s; got %v", testAliasName, testIndexName2, aliasFound)
}
// Remove first index should remove two tweets, so should only yield 1
aliasRemove1, err := client.Alias().
Remove(testIndexName, testAliasName).
//Pretty(true).
Do()
if err != nil {
t.Fatal(err)
}
if !aliasRemove1.Acknowledged {
t.Errorf("expected AliasResult.Acknowledged %v; got %v", true, aliasRemove1.Acknowledged)
}
// Alias should now exist only for index 2
aliasesResult3, err := client.Aliases().Indices(testIndexName, testIndexName2).Do()
if err != nil {
t.Fatal(err)
}
if len(aliasesResult3.Indices) != 2 {
t.Errorf("expected len(AliasesResult.Indices) = %d; got %d", 2, len(aliasesResult3.Indices))
}
for indexName, indexDetails := range aliasesResult3.Indices {
if indexName == testIndexName {
if len(indexDetails.Aliases) != 0 {
t.Errorf("expected len(AliasesResult.Indices[%s].Aliases) = %d; got %d", indexName, 0, len(indexDetails.Aliases))
}
} else if indexName == testIndexName2 {
if len(indexDetails.Aliases) != 1 {
t.Errorf("expected len(AliasesResult.Indices[%s].Aliases) = %d; got %d", indexName, 1, len(indexDetails.Aliases))
}
} else {
t.Errorf("got index %s", indexName)
}
}
}

View File

@ -0,0 +1,301 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type BulkService struct {
client *Client
index string
_type string
requests []BulkableRequest
//replicationType string
//consistencyLevel string
timeout string
refresh *bool
pretty bool
}
func NewBulkService(client *Client) *BulkService {
builder := &BulkService{
client: client,
requests: make([]BulkableRequest, 0),
}
return builder
}
func (s *BulkService) reset() {
s.requests = make([]BulkableRequest, 0)
}
func (s *BulkService) Index(index string) *BulkService {
s.index = index
return s
}
func (s *BulkService) Type(_type string) *BulkService {
s._type = _type
return s
}
func (s *BulkService) Timeout(timeout string) *BulkService {
s.timeout = timeout
return s
}
func (s *BulkService) Refresh(refresh bool) *BulkService {
s.refresh = &refresh
return s
}
func (s *BulkService) Pretty(pretty bool) *BulkService {
s.pretty = pretty
return s
}
func (s *BulkService) Add(r BulkableRequest) *BulkService {
s.requests = append(s.requests, r)
return s
}
func (s *BulkService) NumberOfActions() int {
return len(s.requests)
}
func (s *BulkService) bodyAsString() (string, error) {
buf := bytes.NewBufferString("")
for _, req := range s.requests {
source, err := req.Source()
if err != nil {
return "", err
}
for _, line := range source {
_, err := buf.WriteString(fmt.Sprintf("%s\n", line))
if err != nil {
return "", nil
}
}
}
return buf.String(), nil
}
func (s *BulkService) Do() (*BulkResponse, error) {
// No actions?
if s.NumberOfActions() == 0 {
return nil, errors.New("elastic: No bulk actions to commit")
}
// Get body
body, err := s.bodyAsString()
if err != nil {
return nil, err
}
// Build url
path := "/"
if s.index != "" {
index, err := uritemplates.Expand("{index}", map[string]string{
"index": s.index,
})
if err != nil {
return nil, err
}
path += index + "/"
}
if s._type != "" {
typ, err := uritemplates.Expand("{type}", map[string]string{
"type": s._type,
})
if err != nil {
return nil, err
}
path += typ + "/"
}
path += "_bulk"
// Parameters
params := make(url.Values)
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
if s.refresh != nil {
params.Set("refresh", fmt.Sprintf("%v", *s.refresh))
}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
// Get response
res, err := s.client.PerformRequest("POST", path, params, body)
if err != nil {
return nil, err
}
// Return results
ret := new(BulkResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
// Reset so the request can be reused
s.reset()
return ret, nil
}
// BulkResponse is a response to a bulk execution.
//
// Example:
// {
// "took":3,
// "errors":false,
// "items":[{
// "index":{
// "_index":"index1",
// "_type":"tweet",
// "_id":"1",
// "_version":3,
// "status":201
// }
// },{
// "index":{
// "_index":"index2",
// "_type":"tweet",
// "_id":"2",
// "_version":3,
// "status":200
// }
// },{
// "delete":{
// "_index":"index1",
// "_type":"tweet",
// "_id":"1",
// "_version":4,
// "status":200,
// "found":true
// }
// },{
// "update":{
// "_index":"index2",
// "_type":"tweet",
// "_id":"2",
// "_version":4,
// "status":200
// }
// }]
// }
type BulkResponse struct {
Took int `json:"took,omitempty"`
Errors bool `json:"errors,omitempty"`
Items []map[string]*BulkResponseItem `json:"items,omitempty"`
}
// BulkResponseItem is the result of a single bulk request.
type BulkResponseItem struct {
Index string `json:"_index,omitempty"`
Type string `json:"_type,omitempty"`
Id string `json:"_id,omitempty"`
Version int `json:"_version,omitempty"`
Status int `json:"status,omitempty"`
Found bool `json:"found,omitempty"`
Error string `json:"error,omitempty"`
}
// Indexed returns all bulk request results of "index" actions.
func (r *BulkResponse) Indexed() []*BulkResponseItem {
return r.ByAction("index")
}
// Created returns all bulk request results of "create" actions.
func (r *BulkResponse) Created() []*BulkResponseItem {
return r.ByAction("create")
}
// Updated returns all bulk request results of "update" actions.
func (r *BulkResponse) Updated() []*BulkResponseItem {
return r.ByAction("update")
}
// Deleted returns all bulk request results of "delete" actions.
func (r *BulkResponse) Deleted() []*BulkResponseItem {
return r.ByAction("delete")
}
// ByAction returns all bulk request results of a certain action,
// e.g. "index" or "delete".
func (r *BulkResponse) ByAction(action string) []*BulkResponseItem {
if r.Items == nil {
return nil
}
items := make([]*BulkResponseItem, 0)
for _, item := range r.Items {
if result, found := item[action]; found {
items = append(items, result)
}
}
return items
}
// ById returns all bulk request results of a given document id,
// regardless of the action ("index", "delete" etc.).
func (r *BulkResponse) ById(id string) []*BulkResponseItem {
if r.Items == nil {
return nil
}
items := make([]*BulkResponseItem, 0)
for _, item := range r.Items {
for _, result := range item {
if result.Id == id {
items = append(items, result)
}
}
}
return items
}
// Failed returns those items of a bulk response that have errors,
// i.e. those that don't have a status code between 200 and 299.
func (r *BulkResponse) Failed() []*BulkResponseItem {
if r.Items == nil {
return nil
}
errors := make([]*BulkResponseItem, 0)
for _, item := range r.Items {
for _, result := range item {
if !(result.Status >= 200 && result.Status <= 299) {
errors = append(errors, result)
}
}
}
return errors
}
// Succeeded returns those items of a bulk response that have no errors,
// i.e. those have a status code between 200 and 299.
func (r *BulkResponse) Succeeded() []*BulkResponseItem {
if r.Items == nil {
return nil
}
succeeded := make([]*BulkResponseItem, 0)
for _, item := range r.Items {
for _, result := range item {
if result.Status >= 200 && result.Status <= 299 {
succeeded = append(succeeded, result)
}
}
}
return succeeded
}

View File

@ -0,0 +1,112 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"strings"
)
// -- Bulk delete request --
// Bulk request to remove document from Elasticsearch.
type BulkDeleteRequest struct {
BulkableRequest
index string
typ string
id string
routing string
refresh *bool
version int64 // default is MATCH_ANY
versionType string // default is "internal"
}
func NewBulkDeleteRequest() *BulkDeleteRequest {
return &BulkDeleteRequest{}
}
func (r *BulkDeleteRequest) Index(index string) *BulkDeleteRequest {
r.index = index
return r
}
func (r *BulkDeleteRequest) Type(typ string) *BulkDeleteRequest {
r.typ = typ
return r
}
func (r *BulkDeleteRequest) Id(id string) *BulkDeleteRequest {
r.id = id
return r
}
func (r *BulkDeleteRequest) Routing(routing string) *BulkDeleteRequest {
r.routing = routing
return r
}
func (r *BulkDeleteRequest) Refresh(refresh bool) *BulkDeleteRequest {
r.refresh = &refresh
return r
}
func (r *BulkDeleteRequest) Version(version int64) *BulkDeleteRequest {
r.version = version
return r
}
// VersionType can be "internal" (default), "external", "external_gte",
// "external_gt", or "force".
func (r *BulkDeleteRequest) VersionType(versionType string) *BulkDeleteRequest {
r.versionType = versionType
return r
}
func (r *BulkDeleteRequest) String() string {
lines, err := r.Source()
if err == nil {
return strings.Join(lines, "\n")
}
return fmt.Sprintf("error: %v", err)
}
func (r *BulkDeleteRequest) Source() ([]string, error) {
lines := make([]string, 1)
source := make(map[string]interface{})
deleteCommand := make(map[string]interface{})
if r.index != "" {
deleteCommand["_index"] = r.index
}
if r.typ != "" {
deleteCommand["_type"] = r.typ
}
if r.id != "" {
deleteCommand["_id"] = r.id
}
if r.routing != "" {
deleteCommand["_routing"] = r.routing
}
if r.version > 0 {
deleteCommand["_version"] = r.version
}
if r.versionType != "" {
deleteCommand["_version_type"] = r.versionType
}
if r.refresh != nil {
deleteCommand["refresh"] = *r.refresh
}
source["delete"] = deleteCommand
body, err := json.Marshal(source)
if err != nil {
return nil, err
}
lines[0] = string(body)
return lines, nil
}

View File

@ -0,0 +1,42 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestBulkDeleteRequestSerialization(t *testing.T) {
tests := []struct {
Request BulkableRequest
Expected []string
}{
// #0
{
Request: NewBulkDeleteRequest().Index("index1").Type("tweet").Id("1"),
Expected: []string{
`{"delete":{"_id":"1","_index":"index1","_type":"tweet"}}`,
},
},
}
for i, test := range tests {
lines, err := test.Request.Source()
if err != nil {
t.Fatalf("case #%d: expected no error, got: %v", i, err)
}
if lines == nil {
t.Fatalf("case #%d: expected lines, got nil", i)
}
if len(lines) != len(test.Expected) {
t.Fatalf("case #%d: expected %d lines, got %d", i, len(test.Expected), len(lines))
}
for j, line := range lines {
if line != test.Expected[j] {
t.Errorf("case #%d: expected line #%d to be %s, got: %s", i, j, test.Expected[j], line)
}
}
}
}

View File

@ -0,0 +1,173 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"strings"
)
// Bulk request to add document to Elasticsearch.
type BulkIndexRequest struct {
BulkableRequest
index string
typ string
id string
opType string
routing string
parent string
timestamp string
ttl int64
refresh *bool
version int64 // default is MATCH_ANY
versionType string // default is "internal"
doc interface{}
}
func NewBulkIndexRequest() *BulkIndexRequest {
return &BulkIndexRequest{
opType: "index",
}
}
func (r *BulkIndexRequest) Index(index string) *BulkIndexRequest {
r.index = index
return r
}
func (r *BulkIndexRequest) Type(typ string) *BulkIndexRequest {
r.typ = typ
return r
}
func (r *BulkIndexRequest) Id(id string) *BulkIndexRequest {
r.id = id
return r
}
func (r *BulkIndexRequest) OpType(opType string) *BulkIndexRequest {
r.opType = opType
return r
}
func (r *BulkIndexRequest) Routing(routing string) *BulkIndexRequest {
r.routing = routing
return r
}
func (r *BulkIndexRequest) Parent(parent string) *BulkIndexRequest {
r.parent = parent
return r
}
func (r *BulkIndexRequest) Timestamp(timestamp string) *BulkIndexRequest {
r.timestamp = timestamp
return r
}
func (r *BulkIndexRequest) Ttl(ttl int64) *BulkIndexRequest {
r.ttl = ttl
return r
}
func (r *BulkIndexRequest) Refresh(refresh bool) *BulkIndexRequest {
r.refresh = &refresh
return r
}
func (r *BulkIndexRequest) Version(version int64) *BulkIndexRequest {
r.version = version
return r
}
func (r *BulkIndexRequest) VersionType(versionType string) *BulkIndexRequest {
r.versionType = versionType
return r
}
func (r *BulkIndexRequest) Doc(doc interface{}) *BulkIndexRequest {
r.doc = doc
return r
}
func (r *BulkIndexRequest) String() string {
lines, err := r.Source()
if err == nil {
return strings.Join(lines, "\n")
}
return fmt.Sprintf("error: %v", err)
}
func (r *BulkIndexRequest) Source() ([]string, error) {
// { "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
// { "field1" : "value1" }
lines := make([]string, 2)
// "index" ...
command := make(map[string]interface{})
indexCommand := make(map[string]interface{})
if r.index != "" {
indexCommand["_index"] = r.index
}
if r.typ != "" {
indexCommand["_type"] = r.typ
}
if r.id != "" {
indexCommand["_id"] = r.id
}
if r.routing != "" {
indexCommand["_routing"] = r.routing
}
if r.parent != "" {
indexCommand["_parent"] = r.parent
}
if r.timestamp != "" {
indexCommand["_timestamp"] = r.timestamp
}
if r.ttl > 0 {
indexCommand["_ttl"] = r.ttl
}
if r.version > 0 {
indexCommand["_version"] = r.version
}
if r.versionType != "" {
indexCommand["_version_type"] = r.versionType
}
if r.refresh != nil {
indexCommand["refresh"] = *r.refresh
}
command[r.opType] = indexCommand
line, err := json.Marshal(command)
if err != nil {
return nil, err
}
lines[0] = string(line)
// "field1" ...
if r.doc != nil {
switch t := r.doc.(type) {
default:
body, err := json.Marshal(r.doc)
if err != nil {
return nil, err
}
lines[1] = string(body)
case json.RawMessage:
lines[1] = string(t)
case *json.RawMessage:
lines[1] = string(*t)
case string:
lines[1] = t
case *string:
lines[1] = *t
}
} else {
lines[1] = "{}"
}
return lines, nil
}

View File

@ -0,0 +1,63 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
"time"
)
func TestBulkIndexRequestSerialization(t *testing.T) {
tests := []struct {
Request BulkableRequest
Expected []string
}{
// #0
{
Request: NewBulkIndexRequest().Index("index1").Type("tweet").Id("1").
Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
Expected: []string{
`{"index":{"_id":"1","_index":"index1","_type":"tweet"}}`,
`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
},
},
// #1
{
Request: NewBulkIndexRequest().OpType("create").Index("index1").Type("tweet").Id("1").
Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
Expected: []string{
`{"create":{"_id":"1","_index":"index1","_type":"tweet"}}`,
`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
},
},
// #2
{
Request: NewBulkIndexRequest().OpType("index").Index("index1").Type("tweet").Id("1").
Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
Expected: []string{
`{"index":{"_id":"1","_index":"index1","_type":"tweet"}}`,
`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
},
},
}
for i, test := range tests {
lines, err := test.Request.Source()
if err != nil {
t.Fatalf("case #%d: expected no error, got: %v", i, err)
}
if lines == nil {
t.Fatalf("case #%d: expected lines, got nil", i)
}
if len(lines) != len(test.Expected) {
t.Fatalf("case #%d: expected %d lines, got %d", i, len(test.Expected), len(lines))
}
for j, line := range lines {
if line != test.Expected[j] {
t.Errorf("case #%d: expected line #%d to be %s, got: %s", i, j, test.Expected[j], line)
}
}
}
}

View File

@ -0,0 +1,17 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
)
// -- Bulkable request (index/update/delete) --
// Generic interface to bulkable requests.
type BulkableRequest interface {
fmt.Stringer
Source() ([]string, error)
}

View File

@ -0,0 +1,370 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"testing"
)
func TestBulk(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
index1Req := NewBulkIndexRequest().Index(testIndexName).Type("tweet").Id("1").Doc(tweet1)
index2Req := NewBulkIndexRequest().Index(testIndexName).Type("tweet").Id("2").Doc(tweet2)
delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("tweet").Id("1")
bulkRequest := client.Bulk()
bulkRequest = bulkRequest.Add(index1Req)
bulkRequest = bulkRequest.Add(index2Req)
bulkRequest = bulkRequest.Add(delete1Req)
if bulkRequest.NumberOfActions() != 3 {
t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 3, bulkRequest.NumberOfActions())
}
bulkResponse, err := bulkRequest.Do()
if err != nil {
t.Fatal(err)
}
if bulkResponse == nil {
t.Errorf("expected bulkResponse to be != nil; got nil")
}
if bulkRequest.NumberOfActions() != 0 {
t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 0, bulkRequest.NumberOfActions())
}
// Document with Id="1" should not exist
exists, err := client.Exists().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if exists {
t.Errorf("expected exists %v; got %v", false, exists)
}
// Document with Id="2" should exist
exists, err = client.Exists().Index(testIndexName).Type("tweet").Id("2").Do()
if err != nil {
t.Fatal(err)
}
if !exists {
t.Errorf("expected exists %v; got %v", true, exists)
}
// Update
updateDoc := struct {
Retweets int `json:"retweets"`
}{
42,
}
update1Req := NewBulkUpdateRequest().Index(testIndexName).Type("tweet").Id("2").Doc(&updateDoc)
bulkRequest = client.Bulk()
bulkRequest = bulkRequest.Add(update1Req)
if bulkRequest.NumberOfActions() != 1 {
t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 1, bulkRequest.NumberOfActions())
}
bulkResponse, err = bulkRequest.Do()
if err != nil {
t.Fatal(err)
}
if bulkResponse == nil {
t.Errorf("expected bulkResponse to be != nil; got nil")
}
if bulkRequest.NumberOfActions() != 0 {
t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 0, bulkRequest.NumberOfActions())
}
// Document with Id="1" should have a retweets count of 42
doc, err := client.Get().Index(testIndexName).Type("tweet").Id("2").Do()
if err != nil {
t.Fatal(err)
}
if doc == nil {
t.Fatal("expected doc to be != nil; got nil")
}
if !doc.Found {
t.Fatalf("expected doc to be found; got found = %v", doc.Found)
}
if doc.Source == nil {
t.Fatal("expected doc source to be != nil; got nil")
}
var updatedTweet tweet
err = json.Unmarshal(*doc.Source, &updatedTweet)
if err != nil {
t.Fatal(err)
}
if updatedTweet.Retweets != 42 {
t.Errorf("expected updated tweet retweets = %v; got %v", 42, updatedTweet.Retweets)
}
}
func TestBulkWithIndexSetOnClient(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
index1Req := NewBulkIndexRequest().Index(testIndexName).Type("tweet").Id("1").Doc(tweet1)
index2Req := NewBulkIndexRequest().Index(testIndexName).Type("tweet").Id("2").Doc(tweet2)
delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("tweet").Id("1")
bulkRequest := client.Bulk().Index(testIndexName).Type("tweet")
bulkRequest = bulkRequest.Add(index1Req)
bulkRequest = bulkRequest.Add(index2Req)
bulkRequest = bulkRequest.Add(delete1Req)
if bulkRequest.NumberOfActions() != 3 {
t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 3, bulkRequest.NumberOfActions())
}
bulkResponse, err := bulkRequest.Do()
if err != nil {
t.Fatal(err)
}
if bulkResponse == nil {
t.Errorf("expected bulkResponse to be != nil; got nil")
}
// Document with Id="1" should not exist
exists, err := client.Exists().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if exists {
t.Errorf("expected exists %v; got %v", false, exists)
}
// Document with Id="2" should exist
exists, err = client.Exists().Index(testIndexName).Type("tweet").Id("2").Do()
if err != nil {
t.Fatal(err)
}
if !exists {
t.Errorf("expected exists %v; got %v", true, exists)
}
}
func TestBulkRequestsSerialization(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
index1Req := NewBulkIndexRequest().Index(testIndexName).Type("tweet").Id("1").Doc(tweet1)
index2Req := NewBulkIndexRequest().OpType("create").Index(testIndexName).Type("tweet").Id("2").Doc(tweet2)
delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("tweet").Id("1")
update2Req := NewBulkUpdateRequest().Index(testIndexName).Type("tweet").Id("2").
Doc(struct {
Retweets int `json:"retweets"`
}{
Retweets: 42,
})
bulkRequest := client.Bulk()
bulkRequest = bulkRequest.Add(index1Req)
bulkRequest = bulkRequest.Add(index2Req)
bulkRequest = bulkRequest.Add(delete1Req)
bulkRequest = bulkRequest.Add(update2Req)
if bulkRequest.NumberOfActions() != 4 {
t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 4, bulkRequest.NumberOfActions())
}
expected := `{"index":{"_id":"1","_index":"` + testIndexName + `","_type":"tweet"}}
{"user":"olivere","message":"Welcome to Golang and Elasticsearch.","retweets":0,"created":"0001-01-01T00:00:00Z"}
{"create":{"_id":"2","_index":"` + testIndexName + `","_type":"tweet"}}
{"user":"sandrae","message":"Dancing all night long. Yeah.","retweets":0,"created":"0001-01-01T00:00:00Z"}
{"delete":{"_id":"1","_index":"` + testIndexName + `","_type":"tweet"}}
{"update":{"_id":"2","_index":"` + testIndexName + `","_type":"tweet"}}
{"doc":{"retweets":42}}
`
got, err := bulkRequest.bodyAsString()
if err != nil {
t.Fatalf("expected no error, got: %v", err)
}
if got != expected {
t.Errorf("expected\n%s\ngot:\n%s", expected, got)
}
// Run the bulk request
bulkResponse, err := bulkRequest.Do()
if err != nil {
t.Fatal(err)
}
if bulkResponse == nil {
t.Errorf("expected bulkResponse to be != nil; got nil")
}
if bulkResponse.Took == 0 {
t.Errorf("expected took to be > 0; got %d", bulkResponse.Took)
}
if bulkResponse.Errors {
t.Errorf("expected errors to be %v; got %v", false, bulkResponse.Errors)
}
if len(bulkResponse.Items) != 4 {
t.Fatalf("expected 4 result items; got %d", len(bulkResponse.Items))
}
// Indexed actions
indexed := bulkResponse.Indexed()
if indexed == nil {
t.Fatal("expected indexed to be != nil; got nil")
}
if len(indexed) != 1 {
t.Fatalf("expected len(indexed) == %d; got %d", 1, len(indexed))
}
if indexed[0].Id != "1" {
t.Errorf("expected indexed[0].Id == %s; got %s", "1", indexed[0].Id)
}
if indexed[0].Status != 201 {
t.Errorf("expected indexed[0].Status == %d; got %d", 201, indexed[0].Status)
}
// Created actions
created := bulkResponse.Created()
if created == nil {
t.Fatal("expected created to be != nil; got nil")
}
if len(created) != 1 {
t.Fatalf("expected len(created) == %d; got %d", 1, len(created))
}
if created[0].Id != "2" {
t.Errorf("expected created[0].Id == %s; got %s", "2", created[0].Id)
}
if created[0].Status != 201 {
t.Errorf("expected created[0].Status == %d; got %d", 201, created[0].Status)
}
// Deleted actions
deleted := bulkResponse.Deleted()
if deleted == nil {
t.Fatal("expected deleted to be != nil; got nil")
}
if len(deleted) != 1 {
t.Fatalf("expected len(deleted) == %d; got %d", 1, len(deleted))
}
if deleted[0].Id != "1" {
t.Errorf("expected deleted[0].Id == %s; got %s", "1", deleted[0].Id)
}
if deleted[0].Status != 200 {
t.Errorf("expected deleted[0].Status == %d; got %d", 200, deleted[0].Status)
}
if !deleted[0].Found {
t.Errorf("expected deleted[0].Found == %v; got %v", true, deleted[0].Found)
}
// Updated actions
updated := bulkResponse.Updated()
if updated == nil {
t.Fatal("expected updated to be != nil; got nil")
}
if len(updated) != 1 {
t.Fatalf("expected len(updated) == %d; got %d", 1, len(updated))
}
if updated[0].Id != "2" {
t.Errorf("expected updated[0].Id == %s; got %s", "2", updated[0].Id)
}
if updated[0].Status != 200 {
t.Errorf("expected updated[0].Status == %d; got %d", 200, updated[0].Status)
}
if updated[0].Version != 2 {
t.Errorf("expected updated[0].Version == %d; got %d", 2, updated[0].Version)
}
// Succeeded actions
succeeded := bulkResponse.Succeeded()
if succeeded == nil {
t.Fatal("expected succeeded to be != nil; got nil")
}
if len(succeeded) != 4 {
t.Fatalf("expected len(succeeded) == %d; got %d", 4, len(succeeded))
}
// ById
id1Results := bulkResponse.ById("1")
if id1Results == nil {
t.Fatal("expected id1Results to be != nil; got nil")
}
if len(id1Results) != 2 {
t.Fatalf("expected len(id1Results) == %d; got %d", 2, len(id1Results))
}
if id1Results[0].Id != "1" {
t.Errorf("expected id1Results[0].Id == %s; got %s", "1", id1Results[0].Id)
}
if id1Results[0].Status != 201 {
t.Errorf("expected id1Results[0].Status == %d; got %d", 201, id1Results[0].Status)
}
if id1Results[0].Version != 1 {
t.Errorf("expected id1Results[0].Version == %d; got %d", 1, id1Results[0].Version)
}
if id1Results[1].Id != "1" {
t.Errorf("expected id1Results[1].Id == %s; got %s", "1", id1Results[1].Id)
}
if id1Results[1].Status != 200 {
t.Errorf("expected id1Results[1].Status == %d; got %d", 200, id1Results[1].Status)
}
if id1Results[1].Version != 2 {
t.Errorf("expected id1Results[1].Version == %d; got %d", 2, id1Results[1].Version)
}
}
func TestFailedBulkRequests(t *testing.T) {
js := `{
"took" : 2,
"errors" : true,
"items" : [ {
"index" : {
"_index" : "elastic-test",
"_type" : "tweet",
"_id" : "1",
"_version" : 1,
"status" : 201
}
}, {
"create" : {
"_index" : "elastic-test",
"_type" : "tweet",
"_id" : "2",
"_version" : 1,
"status" : 423,
"error" : "Locked"
}
}, {
"delete" : {
"_index" : "elastic-test",
"_type" : "tweet",
"_id" : "1",
"_version" : 2,
"status" : 404,
"found" : false
}
}, {
"update" : {
"_index" : "elastic-test",
"_type" : "tweet",
"_id" : "2",
"_version" : 2,
"status" : 200
}
} ]
}`
var resp BulkResponse
err := json.Unmarshal([]byte(js), &resp)
if err != nil {
t.Fatal(err)
}
failed := resp.Failed()
if len(failed) != 2 {
t.Errorf("expected %d failed items; got: %d", 2, len(failed))
}
}

View File

@ -0,0 +1,244 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"strings"
)
// Bulk request to update document in Elasticsearch.
type BulkUpdateRequest struct {
BulkableRequest
index string
typ string
id string
routing string
parent string
script string
scriptType string
scriptLang string
scriptParams map[string]interface{}
version int64 // default is MATCH_ANY
versionType string // default is "internal"
retryOnConflict *int
refresh *bool
upsert interface{}
docAsUpsert *bool
doc interface{}
ttl int64
timestamp string
}
func NewBulkUpdateRequest() *BulkUpdateRequest {
return &BulkUpdateRequest{}
}
func (r *BulkUpdateRequest) Index(index string) *BulkUpdateRequest {
r.index = index
return r
}
func (r *BulkUpdateRequest) Type(typ string) *BulkUpdateRequest {
r.typ = typ
return r
}
func (r *BulkUpdateRequest) Id(id string) *BulkUpdateRequest {
r.id = id
return r
}
func (r *BulkUpdateRequest) Routing(routing string) *BulkUpdateRequest {
r.routing = routing
return r
}
func (r *BulkUpdateRequest) Parent(parent string) *BulkUpdateRequest {
r.parent = parent
return r
}
func (r *BulkUpdateRequest) Script(script string) *BulkUpdateRequest {
r.script = script
return r
}
func (r *BulkUpdateRequest) ScriptType(scriptType string) *BulkUpdateRequest {
r.scriptType = scriptType
return r
}
func (r *BulkUpdateRequest) ScriptLang(scriptLang string) *BulkUpdateRequest {
r.scriptLang = scriptLang
return r
}
func (r *BulkUpdateRequest) ScriptParams(params map[string]interface{}) *BulkUpdateRequest {
r.scriptParams = params
return r
}
func (r *BulkUpdateRequest) RetryOnConflict(retryOnConflict int) *BulkUpdateRequest {
r.retryOnConflict = &retryOnConflict
return r
}
func (r *BulkUpdateRequest) Version(version int64) *BulkUpdateRequest {
r.version = version
return r
}
// VersionType can be "internal" (default), "external", "external_gte",
// "external_gt", or "force".
func (r *BulkUpdateRequest) VersionType(versionType string) *BulkUpdateRequest {
r.versionType = versionType
return r
}
func (r *BulkUpdateRequest) Refresh(refresh bool) *BulkUpdateRequest {
r.refresh = &refresh
return r
}
func (r *BulkUpdateRequest) Doc(doc interface{}) *BulkUpdateRequest {
r.doc = doc
return r
}
func (r *BulkUpdateRequest) DocAsUpsert(docAsUpsert bool) *BulkUpdateRequest {
r.docAsUpsert = &docAsUpsert
return r
}
func (r *BulkUpdateRequest) Upsert(doc interface{}) *BulkUpdateRequest {
r.upsert = doc
return r
}
func (r *BulkUpdateRequest) Ttl(ttl int64) *BulkUpdateRequest {
r.ttl = ttl
return r
}
func (r *BulkUpdateRequest) Timestamp(timestamp string) *BulkUpdateRequest {
r.timestamp = timestamp
return r
}
func (r *BulkUpdateRequest) String() string {
lines, err := r.Source()
if err == nil {
return strings.Join(lines, "\n")
}
return fmt.Sprintf("error: %v", err)
}
func (r *BulkUpdateRequest) getSourceAsString(data interface{}) (string, error) {
switch t := data.(type) {
default:
body, err := json.Marshal(data)
if err != nil {
return "", err
}
return string(body), nil
case json.RawMessage:
return string(t), nil
case *json.RawMessage:
return string(*t), nil
case string:
return t, nil
case *string:
return *t, nil
}
}
func (r BulkUpdateRequest) Source() ([]string, error) {
// { "update" : { "_index" : "test", "_type" : "type1", "_id" : "1", ... } }
// { "doc" : { "field1" : "value1", ... } }
// or
// { "update" : { "_index" : "test", "_type" : "type1", "_id" : "1", ... } }
// { "script" : { ... } }
lines := make([]string, 2)
// "update" ...
command := make(map[string]interface{})
updateCommand := make(map[string]interface{})
if r.index != "" {
updateCommand["_index"] = r.index
}
if r.typ != "" {
updateCommand["_type"] = r.typ
}
if r.id != "" {
updateCommand["_id"] = r.id
}
if r.routing != "" {
updateCommand["_routing"] = r.routing
}
if r.parent != "" {
updateCommand["_parent"] = r.parent
}
if r.timestamp != "" {
updateCommand["_timestamp"] = r.timestamp
}
if r.ttl > 0 {
updateCommand["_ttl"] = r.ttl
}
if r.version > 0 {
updateCommand["_version"] = r.version
}
if r.versionType != "" {
updateCommand["_version_type"] = r.versionType
}
if r.refresh != nil {
updateCommand["refresh"] = *r.refresh
}
if r.retryOnConflict != nil {
updateCommand["_retry_on_conflict"] = *r.retryOnConflict
}
if r.upsert != nil {
updateCommand["upsert"] = r.upsert
}
command["update"] = updateCommand
line, err := json.Marshal(command)
if err != nil {
return nil, err
}
lines[0] = string(line)
// 2nd line: {"doc" : { ... }} or {"script": {...}}
source := make(map[string]interface{})
if r.docAsUpsert != nil {
source["doc_as_upsert"] = *r.docAsUpsert
}
if r.doc != nil {
// {"doc":{...}}
source["doc"] = r.doc
} else if r.script != "" {
// {"script":...}
source["script"] = r.script
if r.scriptLang != "" {
source["lang"] = r.scriptLang
}
/*
if r.scriptType != "" {
source["script_type"] = r.scriptType
}
*/
if r.scriptParams != nil && len(r.scriptParams) > 0 {
source["params"] = r.scriptParams
}
}
lines[1], err = r.getSourceAsString(source)
if err != nil {
return nil, err
}
return lines, nil
}

View File

@ -0,0 +1,79 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestBulkUpdateRequestSerialization(t *testing.T) {
tests := []struct {
Request BulkableRequest
Expected []string
}{
// #0
{
Request: NewBulkUpdateRequest().Index("index1").Type("tweet").Id("1").Doc(struct {
Counter int64 `json:"counter"`
}{
Counter: 42,
}),
Expected: []string{
`{"update":{"_id":"1","_index":"index1","_type":"tweet"}}`,
`{"doc":{"counter":42}}`,
},
},
// #1
{
Request: NewBulkUpdateRequest().Index("index1").Type("tweet").Id("1").
RetryOnConflict(3).
DocAsUpsert(true).
Doc(struct {
Counter int64 `json:"counter"`
}{
Counter: 42,
}),
Expected: []string{
`{"update":{"_id":"1","_index":"index1","_retry_on_conflict":3,"_type":"tweet"}}`,
`{"doc":{"counter":42},"doc_as_upsert":true}`,
},
},
// #2
{
Request: NewBulkUpdateRequest().Index("index1").Type("tweet").Id("1").
RetryOnConflict(3).
Script(`ctx._source.retweets += param1`).
ScriptLang("js").
ScriptParams(map[string]interface{}{"param1": 42}).
Upsert(struct {
Counter int64 `json:"counter"`
}{
Counter: 42,
}),
Expected: []string{
`{"update":{"_id":"1","_index":"index1","_retry_on_conflict":3,"_type":"tweet","upsert":{"counter":42}}}`,
`{"lang":"js","params":{"param1":42},"script":"ctx._source.retweets += param1"}`,
},
},
}
for i, test := range tests {
lines, err := test.Request.Source()
if err != nil {
t.Fatalf("case #%d: expected no error, got: %v", i, err)
}
if lines == nil {
t.Fatalf("case #%d: expected lines, got nil", i)
}
if len(lines) != len(test.Expected) {
t.Fatalf("case #%d: expected %d lines, got %d", i, len(test.Expected), len(lines))
}
for j, line := range lines {
if line != test.Expected[j] {
t.Errorf("case #%d: expected line #%d to be %s, got: %s", i, j, test.Expected[j], line)
}
}
}
}

View File

@ -0,0 +1,28 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import "net/url"
// canonicalize takes a list of URLs and returns its canonicalized form, i.e.
// remove anything but scheme, userinfo, host, and port. It also removes the
// slash at the end. It also skips invalid URLs or URLs that do not use
// protocol http or https.
//
// Example:
// http://127.0.0.1:9200/path?query=1 -> http://127.0.0.1:9200
func canonicalize(rawurls ...string) []string {
canonicalized := make([]string, 0)
for _, rawurl := range rawurls {
u, err := url.Parse(rawurl)
if err == nil && (u.Scheme == "http" || u.Scheme == "https") {
u.Fragment = ""
u.Path = ""
u.RawQuery = ""
canonicalized = append(canonicalized, u.String())
}
}
return canonicalized
}

View File

@ -0,0 +1,41 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"reflect"
"testing"
)
func TestCanonicalize(t *testing.T) {
tests := []struct {
Input []string
Output []string
}{
{
Input: []string{"http://127.0.0.1/"},
Output: []string{"http://127.0.0.1"},
},
{
Input: []string{"http://127.0.0.1:9200/", "gopher://golang.org/", "http://127.0.0.1:9201"},
Output: []string{"http://127.0.0.1:9200", "http://127.0.0.1:9201"},
},
{
Input: []string{"http://user:secret@127.0.0.1/path?query=1#fragment"},
Output: []string{"http://user:secret@127.0.0.1"},
},
{
Input: []string{"https://somewhere.on.mars:9999/path?query=1#fragment"},
Output: []string{"https://somewhere.on.mars:9999"},
},
}
for _, test := range tests {
got := canonicalize(test.Input...)
if !reflect.DeepEqual(got, test.Output) {
t.Errorf("expected %v; got: %v", test.Output, got)
}
}
}

View File

@ -0,0 +1,96 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// ClearScrollService is documented at http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/search-request-scroll.html.
type ClearScrollService struct {
client *Client
pretty bool
scrollId []string
bodyJson interface{}
bodyString string
}
// NewClearScrollService creates a new ClearScrollService.
func NewClearScrollService(client *Client) *ClearScrollService {
return &ClearScrollService{
client: client,
scrollId: make([]string, 0),
}
}
// ScrollId is a list of scroll IDs to clear.
// Use _all to clear all search contexts.
func (s *ClearScrollService) ScrollId(scrollId ...string) *ClearScrollService {
s.scrollId = make([]string, 0)
s.scrollId = append(s.scrollId, scrollId...)
return s
}
// buildURL builds the URL for the operation.
func (s *ClearScrollService) buildURL() (string, url.Values, error) {
path, err := uritemplates.Expand("/_search/scroll", map[string]string{})
if err != nil {
return "", url.Values{}, err
}
return path, url.Values{}, nil
}
// Validate checks if the operation is valid.
func (s *ClearScrollService) Validate() error {
return nil
}
// Do executes the operation.
func (s *ClearScrollService) Do() (*ClearScrollResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Setup HTTP request body
body := strings.Join(s.scrollId, ",")
// Get HTTP response
res, err := s.client.PerformRequest("DELETE", path, params, body)
if err != nil {
return nil, err
}
// Return operation response
ret := new(ClearScrollResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// ClearScrollResponse is the response of ClearScrollService.Do.
type ClearScrollResponse struct {
}

View File

@ -0,0 +1,72 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
_ "net/http"
"testing"
)
func TestClearScroll(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Match all should return all documents
res, err := client.Scroll(testIndexName).Size(1).Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Errorf("expected results != nil; got nil")
}
if res.ScrollId == "" {
t.Errorf("expected scrollId in results; got %q", res.ScrollId)
}
// Search should succeed
_, err = client.Scroll(testIndexName).Size(1).ScrollId(res.ScrollId).Do()
if err != nil {
t.Fatal(err)
}
// Clear scroll id
clearScrollRes, err := client.ClearScroll().ScrollId(res.ScrollId).Do()
if err != nil {
t.Fatal(err)
}
if clearScrollRes == nil {
t.Error("expected results != nil; got nil")
}
// Search result should fail
_, err = client.Scroll(testIndexName).Size(1).ScrollId(res.ScrollId).Do()
if err == nil {
t.Fatalf("expected scroll to fail")
}
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,620 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"bytes"
"encoding/json"
"log"
"net/http"
"regexp"
"strings"
"testing"
"time"
)
func findConn(s string, slice ...*conn) (int, bool) {
for i, t := range slice {
if s == t.URL() {
return i, true
}
}
return -1, false
}
// -- NewClient --
func TestClientDefaults(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
if client.healthcheckEnabled != true {
t.Errorf("expected health checks to be enabled, got: %v", client.healthcheckEnabled)
}
if client.healthcheckTimeoutStartup != DefaultHealthcheckTimeoutStartup {
t.Errorf("expected health checks timeout on startup = %v, got: %v", DefaultHealthcheckTimeoutStartup, client.healthcheckTimeoutStartup)
}
if client.healthcheckTimeout != DefaultHealthcheckTimeout {
t.Errorf("expected health checks timeout = %v, got: %v", DefaultHealthcheckTimeout, client.healthcheckTimeout)
}
if client.healthcheckInterval != DefaultHealthcheckInterval {
t.Errorf("expected health checks interval = %v, got: %v", DefaultHealthcheckInterval, client.healthcheckInterval)
}
if client.snifferEnabled != true {
t.Errorf("expected sniffing to be enabled, got: %v", client.snifferEnabled)
}
if client.snifferTimeoutStartup != DefaultSnifferTimeoutStartup {
t.Errorf("expected sniffer timeout on startup = %v, got: %v", DefaultSnifferTimeoutStartup, client.snifferTimeoutStartup)
}
if client.snifferTimeout != DefaultSnifferTimeout {
t.Errorf("expected sniffer timeout = %v, got: %v", DefaultSnifferTimeout, client.snifferTimeout)
}
if client.snifferInterval != DefaultSnifferInterval {
t.Errorf("expected sniffer interval = %v, got: %v", DefaultSnifferInterval, client.snifferInterval)
}
}
func TestClientWithoutURL(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
// Two things should happen here:
// 1. The client starts sniffing the cluster on DefaultURL
// 2. The sniffing process should find (at least) one node in the cluster, i.e. the DefaultURL
if len(client.conns) == 0 {
t.Fatalf("expected at least 1 node in the cluster, got: %d (%v)", len(client.conns), client.conns)
}
if !isTravis() {
if _, found := findConn(DefaultURL, client.conns...); !found {
t.Errorf("expected to find node with default URL of %s in %v", DefaultURL, client.conns)
}
}
}
func TestClientWithSingleURL(t *testing.T) {
client, err := NewClient(SetURL("http://localhost:9200"))
if err != nil {
t.Fatal(err)
}
// Two things should happen here:
// 1. The client starts sniffing the cluster on DefaultURL
// 2. The sniffing process should find (at least) one node in the cluster, i.e. the DefaultURL
if len(client.conns) == 0 {
t.Fatalf("expected at least 1 node in the cluster, got: %d (%v)", len(client.conns), client.conns)
}
if !isTravis() {
if _, found := findConn(DefaultURL, client.conns...); !found {
t.Errorf("expected to find node with default URL of %s in %v", DefaultURL, client.conns)
}
}
}
func TestClientWithMultipleURLs(t *testing.T) {
client, err := NewClient(SetURL("http://localhost:9200", "http://localhost:9201"))
if err != nil {
t.Fatal(err)
}
// The client should sniff both URLs, but only localhost:9200 should return nodes.
if len(client.conns) != 1 {
t.Fatalf("expected exactly 1 node in the local cluster, got: %d (%v)", len(client.conns), client.conns)
}
if !isTravis() {
if client.conns[0].URL() != DefaultURL {
t.Errorf("expected to find node with default URL of %s in %v", DefaultURL, client.conns)
}
}
}
func TestClientSniffSuccess(t *testing.T) {
client, err := NewClient(SetURL("http://localhost:19200", "http://localhost:9200"))
if err != nil {
t.Fatal(err)
}
// The client should sniff both URLs, but only localhost:9200 should return nodes.
if len(client.conns) != 1 {
t.Fatalf("expected exactly 1 node in the local cluster, got: %d (%v)", len(client.conns), client.conns)
}
}
func TestClientSniffFailure(t *testing.T) {
_, err := NewClient(SetURL("http://localhost:19200", "http://localhost:19201"))
if err == nil {
t.Fatalf("expected cluster to fail with no nodes found")
}
}
func TestClientSniffDisabled(t *testing.T) {
client, err := NewClient(SetSniff(false), SetURL("http://localhost:9200", "http://localhost:9201"))
if err != nil {
t.Fatal(err)
}
// The client should not sniff, so it should have two connections.
if len(client.conns) != 2 {
t.Fatalf("expected 2 nodes, got: %d (%v)", len(client.conns), client.conns)
}
// Make two requests, so that both connections are being used
for i := 0; i < len(client.conns); i++ {
client.Flush().Do()
}
// The first connection (localhost:9200) should now be okay.
if i, found := findConn("http://localhost:9200", client.conns...); !found {
t.Fatalf("expected connection to %q to be found", "http://localhost:9200")
} else {
if conn := client.conns[i]; conn.IsDead() {
t.Fatal("expected connection to be alive, but it is dead")
}
}
// The second connection (localhost:9201) should now be marked as dead.
if i, found := findConn("http://localhost:9201", client.conns...); !found {
t.Fatalf("expected connection to %q to be found", "http://localhost:9201")
} else {
if conn := client.conns[i]; !conn.IsDead() {
t.Fatal("expected connection to be dead, but it is alive")
}
}
}
func TestClientHealthcheckStartupTimeout(t *testing.T) {
start := time.Now()
_, err := NewClient(SetURL("http://localhost:9299"), SetHealthcheckTimeoutStartup(5*time.Second))
duration := time.Now().Sub(start)
if err != ErrNoClient {
t.Fatal(err)
}
if duration < 5*time.Second {
t.Fatalf("expected a timeout in more than 5 seconds; got: %v", duration)
}
}
// -- Start and stop --
func TestClientStartAndStop(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
running := client.IsRunning()
if !running {
t.Fatalf("expected background processes to run; got: %v", running)
}
// Stop
client.Stop()
running = client.IsRunning()
if running {
t.Fatalf("expected background processes to be stopped; got: %v", running)
}
// Stop again => no-op
client.Stop()
running = client.IsRunning()
if running {
t.Fatalf("expected background processes to be stopped; got: %v", running)
}
// Start
client.Start()
running = client.IsRunning()
if !running {
t.Fatalf("expected background processes to run; got: %v", running)
}
// Start again => no-op
client.Start()
running = client.IsRunning()
if !running {
t.Fatalf("expected background processes to run; got: %v", running)
}
}
// -- Sniffing --
func TestClientSniffNode(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
ch := make(chan []*conn)
go func() { ch <- client.sniffNode(DefaultURL) }()
select {
case nodes := <-ch:
if len(nodes) != 1 {
t.Fatalf("expected %d nodes; got: %d", 1, len(nodes))
}
pattern := `http:\/\/[\d\.]+:9200`
matched, err := regexp.MatchString(pattern, nodes[0].URL())
if err != nil {
t.Fatal(err)
}
if !matched {
t.Fatalf("expected node URL pattern %q; got: %q", pattern, nodes[0].URL())
}
case <-time.After(2 * time.Second):
t.Fatal("expected no timeout in sniff node")
break
}
}
func TestClientSniffOnDefaultURL(t *testing.T) {
client, _ := NewClient()
if client == nil {
t.Fatal("no client returned")
}
ch := make(chan error, 1)
go func() {
ch <- client.sniff(DefaultSnifferTimeoutStartup)
}()
select {
case err := <-ch:
if err != nil {
t.Fatalf("expected sniff to succeed; got: %v", err)
}
if len(client.conns) != 1 {
t.Fatalf("expected %d nodes; got: %d", 1, len(client.conns))
}
pattern := `http:\/\/[\d\.]+:9200`
matched, err := regexp.MatchString(pattern, client.conns[0].URL())
if err != nil {
t.Fatal(err)
}
if !matched {
t.Fatalf("expected node URL pattern %q; got: %q", pattern, client.conns[0].URL())
}
case <-time.After(2 * time.Second):
t.Fatal("expected no timeout in sniff")
break
}
}
// -- Selector --
func TestClientSelectConnHealthy(t *testing.T) {
client, err := NewClient(
SetSniff(false),
SetHealthcheck(false),
SetURL("http://127.0.0.1:9200", "http://127.0.0.1:9201"))
if err != nil {
t.Fatal(err)
}
// Both are healthy, so we should get both URLs in round-robin
client.conns[0].MarkAsHealthy()
client.conns[1].MarkAsHealthy()
// #1: Return 1st
c, err := client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[0].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[0].URL())
}
// #2: Return 2nd
c, err = client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[1].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[1].URL())
}
// #3: Return 1st
c, err = client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[0].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[0].URL())
}
}
func TestClientSelectConnHealthyAndDead(t *testing.T) {
client, err := NewClient(
SetSniff(false),
SetHealthcheck(false),
SetURL("http://127.0.0.1:9200", "http://127.0.0.1:9201"))
if err != nil {
t.Fatal(err)
}
// 1st is healthy, second is dead
client.conns[0].MarkAsHealthy()
client.conns[1].MarkAsDead()
// #1: Return 1st
c, err := client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[0].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[0].URL())
}
// #2: Return 1st again
c, err = client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[0].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[0].URL())
}
// #3: Return 1st again and again
c, err = client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[0].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[0].URL())
}
}
func TestClientSelectConnDeadAndHealthy(t *testing.T) {
client, err := NewClient(
SetSniff(false),
SetHealthcheck(false),
SetURL("http://127.0.0.1:9200", "http://127.0.0.1:9201"))
if err != nil {
t.Fatal(err)
}
// 1st is dead, 2nd is healthy
client.conns[0].MarkAsDead()
client.conns[1].MarkAsHealthy()
// #1: Return 2nd
c, err := client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[1].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[1].URL())
}
// #2: Return 2nd again
c, err = client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[1].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[1].URL())
}
// #3: Return 2nd again and again
c, err = client.next()
if err != nil {
t.Fatal(err)
}
if c.URL() != client.conns[1].URL() {
t.Fatalf("expected %s; got: %s", c.URL(), client.conns[1].URL())
}
}
func TestClientSelectConnAllDead(t *testing.T) {
client, err := NewClient(
SetSniff(false),
SetHealthcheck(false),
SetURL("http://127.0.0.1:9200", "http://127.0.0.1:9201"))
if err != nil {
t.Fatal(err)
}
// Both are dead
client.conns[0].MarkAsDead()
client.conns[1].MarkAsDead()
// #1: Return ErrNoClient
c, err := client.next()
if err != ErrNoClient {
t.Fatal(err)
}
if c != nil {
t.Fatalf("expected no connection; got: %v", c)
}
// #2: Return ErrNoClient again
c, err = client.next()
if err != ErrNoClient {
t.Fatal(err)
}
if c != nil {
t.Fatalf("expected no connection; got: %v", c)
}
// #3: Return ErrNoClient again and again
c, err = client.next()
if err != ErrNoClient {
t.Fatal(err)
}
if c != nil {
t.Fatalf("expected no connection; got: %v", c)
}
}
// -- ElasticsearchVersion --
func TestElasticsearchVersion(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
version, err := client.ElasticsearchVersion(DefaultURL)
if err != nil {
t.Fatal(err)
}
if version == "" {
t.Errorf("expected a version number, got: %q", version)
}
}
// -- IndexNames --
func TestIndexNames(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
names, err := client.IndexNames()
if err != nil {
t.Fatal(err)
}
if len(names) == 0 {
t.Fatalf("expected some index names, got: %d", len(names))
}
var found bool
for _, name := range names {
if name == testIndexName {
found = true
break
}
}
if !found {
t.Fatalf("expected to find index %q; got: %v", testIndexName, found)
}
}
// -- PerformRequest --
func TestPerformRequest(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
res, err := client.PerformRequest("GET", "/", nil, nil)
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatal("expected response to be != nil")
}
ret := new(PingResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
t.Fatalf("expected no error on decode; got: %v", err)
}
if ret.Status != 200 {
t.Errorf("expected HTTP status 200; got: %d", ret.Status)
}
}
func TestPerformRequestWithLogger(t *testing.T) {
var w bytes.Buffer
out := log.New(&w, "LOGGER ", log.LstdFlags)
client, err := NewClient(SetInfoLog(out))
if err != nil {
t.Fatal(err)
}
res, err := client.PerformRequest("GET", "/", nil, nil)
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatal("expected response to be != nil")
}
ret := new(PingResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
t.Fatalf("expected no error on decode; got: %v", err)
}
if ret.Status != 200 {
t.Errorf("expected HTTP status 200; got: %d", ret.Status)
}
got := w.String()
pattern := `^LOGGER \d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} GET http://.*/ \[status:200, request:\d+\.\d{3}s\]\n`
matched, err := regexp.MatchString(pattern, got)
if err != nil {
t.Fatalf("expected log line to match %q; got: %v", pattern, err)
}
if !matched {
t.Errorf("expected log line to match %q; got: %v", pattern, got)
}
}
func TestPerformRequestWithLoggerAndTracer(t *testing.T) {
var lw bytes.Buffer
lout := log.New(&lw, "LOGGER ", log.LstdFlags)
var tw bytes.Buffer
tout := log.New(&tw, "TRACER ", log.LstdFlags)
client, err := NewClient(SetInfoLog(lout), SetTraceLog(tout))
if err != nil {
t.Fatal(err)
}
res, err := client.PerformRequest("GET", "/", nil, nil)
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatal("expected response to be != nil")
}
ret := new(PingResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
t.Fatalf("expected no error on decode; got: %v", err)
}
if ret.Status != 200 {
t.Errorf("expected HTTP status 200; got: %d", ret.Status)
}
lgot := lw.String()
if lgot == "" {
t.Errorf("expected logger output; got: %q", lgot)
}
tgot := tw.String()
if tgot == "" {
t.Errorf("expected tracer output; got: %q", tgot)
}
}
// failingTransport will run a fail callback if it sees a given URL path prefix.
type failingTransport struct {
path string // path prefix to look for
fail func(*http.Request) (*http.Response, error) // call when path prefix is found
next http.RoundTripper // next round-tripper (use http.DefaultTransport if nil)
}
// RoundTrip implements a failing transport.
func (tr *failingTransport) RoundTrip(r *http.Request) (*http.Response, error) {
if strings.HasPrefix(r.URL.Path, tr.path) && tr.fail != nil {
return tr.fail(r)
}
if tr.next != nil {
return tr.next.RoundTrip(r)
}
return http.DefaultTransport.RoundTrip(r)
}
func TestPerformRequestWithMaxRetries(t *testing.T) {
var numFailedReqs int
fail := func(r *http.Request) (*http.Response, error) {
numFailedReqs += 1
return &http.Response{Request: r, StatusCode: 400}, nil
}
// Run against a failing endpoint and see if PerformRequest
// retries correctly.
tr := &failingTransport{path: "/fail", fail: fail}
httpClient := &http.Client{Transport: tr}
client, err := NewClient(SetHttpClient(httpClient), SetMaxRetries(5))
if err != nil {
t.Fatal(err)
}
res, err := client.PerformRequest("GET", "/fail", nil, nil)
if err == nil {
t.Fatal("expected error")
}
if res != nil {
t.Fatal("expected no response")
}
// Connection should be marked as dead after it failed
if numFailedReqs != 5 {
t.Errorf("expected %d failed requests; got: %d", 5, numFailedReqs)
}
}

View File

@ -0,0 +1,16 @@
.PHONY: build run-omega-cluster-test
default: build
build:
go build cluster-test.go
run-omega-cluster-test:
go run -race cluster-test.go \
-nodes=http://192.168.2.65:8200,http://192.168.2.64:8200 \
-n=5 \
-retries=5 \
-sniff=true -sniffer=10s \
-healthcheck=true -healthchecker=5s \
-errorlog=errors.log

View File

@ -0,0 +1,63 @@
# Cluster Test
This directory contains a program you can use to test a cluster.
Here's how:
First, install a cluster of Elasticsearch nodes. You can install them on
different computers, or start several nodes on a single machine.
Build cluster-test by `go build cluster-test.go` (or build with `make`).
Run `./cluster-test -h` to get a list of flags:
```sh
$ ./cluster-test -h
Usage of ./cluster-test:
-errorlog="": error log file
-healthcheck=true: enable or disable healthchecks
-healthchecker=1m0s: healthcheck interval
-index="twitter": name of ES index to use
-infolog="": info log file
-n=5: number of goroutines that run searches
-nodes="": comma-separated list of ES URLs (e.g. 'http://192.168.2.10:9200,http://192.168.2.11:9200')
-retries=0: number of retries
-sniff=true: enable or disable sniffer
-sniffer=15m0s: sniffer interval
-tracelog="": trace log file
```
Example:
```sh
$ ./cluster-test -nodes=http://127.0.0.1:9200,http://127.0.0.1:9201,http://127.0.0.1:9202 -n=5 -index=twitter -retries=5 -sniff=true -sniffer=10s -healthcheck=true -healthchecker=5s -errorlog=error.log
```
The above example will create an index and start some search jobs on the
cluster defined by http://127.0.0.1:9200, http://127.0.0.1:9201,
and http://127.0.0.1:9202.
* It will create an index called `twitter` on the cluster (`-index=twitter`)
* It will run 5 search jobs in parallel (`-n=5`).
* It will retry failed requests 5 times (`-retries=5`).
* It will sniff the cluster periodically (`-sniff=true`).
* It will sniff the cluster every 10 seconds (`-sniffer=10s`).
* It will perform health checks periodically (`-healthcheck=true`).
* It will perform health checks on the nodes every 5 seconds (`-healthchecker=5s`).
* It will write an error log file (`-errorlog=error.log`).
If you want to test Elastic with nodes going up and down, you can use a
chaos monkey script like this and run it on the nodes of your cluster:
```sh
#!/bin/bash
while true
do
echo "Starting ES node"
elasticsearch -d -Xmx4g -Xms1g -Des.config=elasticsearch.yml -p es.pid
sleep `jot -r 1 10 300` # wait for 10-300s
echo "Stopping ES node"
kill -TERM `cat es.pid`
sleep `jot -r 1 10 60` # wait for 10-60s
done
```

View File

@ -0,0 +1,357 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"log"
"math/rand"
"os"
"runtime"
"strings"
"sync/atomic"
"time"
elastic "gopkg.in/olivere/elastic.v2"
)
type Tweet struct {
User string `json:"user"`
Message string `json:"message"`
Retweets int `json:"retweets"`
Image string `json:"image,omitempty"`
Created time.Time `json:"created,omitempty"`
Tags []string `json:"tags,omitempty"`
Location string `json:"location,omitempty"`
Suggest *elastic.SuggestField `json:"suggest_field,omitempty"`
}
var (
nodes = flag.String("nodes", "", "comma-separated list of ES URLs (e.g. 'http://192.168.2.10:9200,http://192.168.2.11:9200')")
n = flag.Int("n", 5, "number of goroutines that run searches")
index = flag.String("index", "twitter", "name of ES index to use")
errorlogfile = flag.String("errorlog", "", "error log file")
infologfile = flag.String("infolog", "", "info log file")
tracelogfile = flag.String("tracelog", "", "trace log file")
retries = flag.Int("retries", elastic.DefaultMaxRetries, "number of retries")
sniff = flag.Bool("sniff", elastic.DefaultSnifferEnabled, "enable or disable sniffer")
sniffer = flag.Duration("sniffer", elastic.DefaultSnifferInterval, "sniffer interval")
healthcheck = flag.Bool("healthcheck", elastic.DefaultHealthcheckEnabled, "enable or disable healthchecks")
healthchecker = flag.Duration("healthchecker", elastic.DefaultHealthcheckInterval, "healthcheck interval")
)
func main() {
flag.Parse()
runtime.GOMAXPROCS(runtime.NumCPU())
if *nodes == "" {
log.Fatal("no nodes specified")
}
urls := strings.SplitN(*nodes, ",", -1)
testcase, err := NewTestCase(*index, urls)
if err != nil {
log.Fatal(err)
}
testcase.SetErrorLogFile(*errorlogfile)
testcase.SetInfoLogFile(*infologfile)
testcase.SetTraceLogFile(*tracelogfile)
testcase.SetMaxRetries(*retries)
testcase.SetHealthcheck(*healthcheck)
testcase.SetHealthcheckInterval(*healthchecker)
testcase.SetSniff(*sniff)
testcase.SetSnifferInterval(*sniffer)
if err := testcase.Run(*n); err != nil {
log.Fatal(err)
}
select {}
}
type RunInfo struct {
Success bool
}
type TestCase struct {
nodes []string
client *elastic.Client
runs int64
failures int64
runCh chan RunInfo
index string
errorlogfile string
infologfile string
tracelogfile string
maxRetries int
healthcheck bool
healthcheckInterval time.Duration
sniff bool
snifferInterval time.Duration
}
func NewTestCase(index string, nodes []string) (*TestCase, error) {
if index == "" {
return nil, errors.New("no index name specified")
}
return &TestCase{
index: index,
nodes: nodes,
runCh: make(chan RunInfo),
}, nil
}
func (t *TestCase) SetIndex(name string) {
t.index = name
}
func (t *TestCase) SetErrorLogFile(name string) {
t.errorlogfile = name
}
func (t *TestCase) SetInfoLogFile(name string) {
t.infologfile = name
}
func (t *TestCase) SetTraceLogFile(name string) {
t.tracelogfile = name
}
func (t *TestCase) SetMaxRetries(n int) {
t.maxRetries = n
}
func (t *TestCase) SetSniff(enabled bool) {
t.sniff = enabled
}
func (t *TestCase) SetSnifferInterval(d time.Duration) {
t.snifferInterval = d
}
func (t *TestCase) SetHealthcheck(enabled bool) {
t.healthcheck = enabled
}
func (t *TestCase) SetHealthcheckInterval(d time.Duration) {
t.healthcheckInterval = d
}
func (t *TestCase) Run(n int) error {
if err := t.setup(); err != nil {
return err
}
for i := 1; i < n; i++ {
go t.search()
}
go t.monitor()
return nil
}
func (t *TestCase) monitor() {
print := func() {
fmt.Printf("\033[32m%5d\033[0m; \033[31m%5d\033[0m: %s%s\r", t.runs, t.failures, t.client.String(), " ")
}
for {
select {
case run := <-t.runCh:
atomic.AddInt64(&t.runs, 1)
if !run.Success {
atomic.AddInt64(&t.failures, 1)
fmt.Println()
}
print()
case <-time.After(5 * time.Second):
// Print stats after some inactivity
print()
break
}
}
}
func (t *TestCase) setup() error {
var errorlogger *log.Logger
if t.errorlogfile != "" {
f, err := os.OpenFile(t.errorlogfile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0664)
if err != nil {
return err
}
errorlogger = log.New(f, "", log.Ltime|log.Lmicroseconds|log.Lshortfile)
}
var infologger *log.Logger
if t.infologfile != "" {
f, err := os.OpenFile(t.infologfile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0664)
if err != nil {
return err
}
infologger = log.New(f, "", log.LstdFlags)
}
// Trace request and response details like this
var tracelogger *log.Logger
if t.tracelogfile != "" {
f, err := os.OpenFile(t.tracelogfile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0664)
if err != nil {
return err
}
tracelogger = log.New(f, "", log.LstdFlags)
}
client, err := elastic.NewClient(
elastic.SetURL(t.nodes...),
elastic.SetErrorLog(errorlogger),
elastic.SetInfoLog(infologger),
elastic.SetTraceLog(tracelogger),
elastic.SetMaxRetries(t.maxRetries),
elastic.SetSniff(t.sniff),
elastic.SetSnifferInterval(t.snifferInterval),
elastic.SetHealthcheck(t.healthcheck),
elastic.SetHealthcheckInterval(t.healthcheckInterval))
if err != nil {
// Handle error
return err
}
t.client = client
// Use the IndexExists service to check if a specified index exists.
exists, err := t.client.IndexExists(t.index).Do()
if err != nil {
return err
}
if exists {
deleteIndex, err := t.client.DeleteIndex(t.index).Do()
if err != nil {
return err
}
if !deleteIndex.Acknowledged {
return errors.New("delete index not acknowledged")
}
}
// Create a new index.
createIndex, err := t.client.CreateIndex(t.index).Do()
if err != nil {
return err
}
if !createIndex.Acknowledged {
return errors.New("create index not acknowledged")
}
// Index a tweet (using JSON serialization)
tweet1 := Tweet{User: "olivere", Message: "Take Five", Retweets: 0}
_, err = t.client.Index().
Index(t.index).
Type("tweet").
Id("1").
BodyJson(tweet1).
Do()
if err != nil {
return err
}
// Index a second tweet (by string)
tweet2 := `{"user" : "olivere", "message" : "It's a Raggy Waltz"}`
_, err = t.client.Index().
Index(t.index).
Type("tweet").
Id("2").
BodyString(tweet2).
Do()
if err != nil {
return err
}
// Flush to make sure the documents got written.
_, err = t.client.Flush().Index(t.index).Do()
if err != nil {
return err
}
return nil
}
func (t *TestCase) search() {
// Loop forever to check for connection issues
for {
// Get tweet with specified ID
get1, err := t.client.Get().
Index(t.index).
Type("tweet").
Id("1").
Do()
if err != nil {
//failf("Get failed: %v", err)
t.runCh <- RunInfo{Success: false}
continue
}
if !get1.Found {
//log.Printf("Document %s not found\n", "1")
//fmt.Printf("Got document %s in version %d from index %s, type %s\n", get1.Id, get1.Version, get1.Index, get1.Type)
t.runCh <- RunInfo{Success: false}
continue
}
// Search with a term query
termQuery := elastic.NewTermQuery("user", "olivere")
searchResult, err := t.client.Search().
Index(t.index). // search in index t.index
Query(&termQuery). // specify the query
Sort("user", true). // sort by "user" field, ascending
From(0).Size(10). // take documents 0-9
Pretty(true). // pretty print request and response JSON
Do() // execute
if err != nil {
//failf("Search failed: %v\n", err)
t.runCh <- RunInfo{Success: false}
continue
}
// searchResult is of type SearchResult and returns hits, suggestions,
// and all kinds of other information from Elasticsearch.
//fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)
// Number of hits
if searchResult.Hits != nil {
//fmt.Printf("Found a total of %d tweets\n", searchResult.Hits.TotalHits)
// Iterate through results
for _, hit := range searchResult.Hits.Hits {
// hit.Index contains the name of the index
// Deserialize hit.Source into a Tweet (could also be just a map[string]interface{}).
var tweet Tweet
err := json.Unmarshal(*hit.Source, &tweet)
if err != nil {
// Deserialization failed
//failf("Deserialize failed: %v\n", err)
t.runCh <- RunInfo{Success: false}
continue
}
// Work with tweet
//fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
} else {
// No hits
//fmt.Print("Found no tweets\n")
}
t.runCh <- RunInfo{Success: true}
// Sleep some time
time.Sleep(time.Duration(rand.Intn(500)) * time.Millisecond)
}
}

View File

@ -0,0 +1,186 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// ClusterHealthService allows to get the status of the cluster.
// It is documented at http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/cluster-health.html.
type ClusterHealthService struct {
client *Client
pretty bool
indices []string
waitForStatus string
level string
local *bool
masterTimeout string
timeout string
waitForActiveShards *int
waitForNodes string
waitForRelocatingShards *int
}
// NewClusterHealthService creates a new ClusterHealthService.
func NewClusterHealthService(client *Client) *ClusterHealthService {
return &ClusterHealthService{client: client, indices: make([]string, 0)}
}
// Index limits the information returned to a specific index.
func (s *ClusterHealthService) Index(index string) *ClusterHealthService {
s.indices = make([]string, 0)
s.indices = append(s.indices, index)
return s
}
// Indices limits the information returned to specific indices.
func (s *ClusterHealthService) Indices(indices ...string) *ClusterHealthService {
s.indices = make([]string, 0)
s.indices = append(s.indices, indices...)
return s
}
// MasterTimeout specifies an explicit operation timeout for connection to master node.
func (s *ClusterHealthService) MasterTimeout(masterTimeout string) *ClusterHealthService {
s.masterTimeout = masterTimeout
return s
}
// Timeout specifies an explicit operation timeout.
func (s *ClusterHealthService) Timeout(timeout string) *ClusterHealthService {
s.timeout = timeout
return s
}
// WaitForActiveShards can be used to wait until the specified number of shards are active.
func (s *ClusterHealthService) WaitForActiveShards(waitForActiveShards int) *ClusterHealthService {
s.waitForActiveShards = &waitForActiveShards
return s
}
// WaitForNodes can be used to wait until the specified number of nodes are available.
func (s *ClusterHealthService) WaitForNodes(waitForNodes string) *ClusterHealthService {
s.waitForNodes = waitForNodes
return s
}
// WaitForRelocatingShards can be used to wait until the specified number of relocating shards is finished.
func (s *ClusterHealthService) WaitForRelocatingShards(waitForRelocatingShards int) *ClusterHealthService {
s.waitForRelocatingShards = &waitForRelocatingShards
return s
}
// WaitForStatus can be used to wait until the cluster is in a specific state.
// Valid values are: green, yellow, or red.
func (s *ClusterHealthService) WaitForStatus(waitForStatus string) *ClusterHealthService {
s.waitForStatus = waitForStatus
return s
}
// Level specifies the level of detail for returned information.
func (s *ClusterHealthService) Level(level string) *ClusterHealthService {
s.level = level
return s
}
// Local indicates whether to return local information. If it is true,
// we do not retrieve the state from master node (default: false).
func (s *ClusterHealthService) Local(local bool) *ClusterHealthService {
s.local = &local
return s
}
// buildURL builds the URL for the operation.
func (s *ClusterHealthService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_cluster/health/{index}", map[string]string{
"index": strings.Join(s.indices, ","),
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.waitForRelocatingShards != nil {
params.Set("wait_for_relocating_shards", fmt.Sprintf("%d", *s.waitForRelocatingShards))
}
if s.waitForStatus != "" {
params.Set("wait_for_status", s.waitForStatus)
}
if s.level != "" {
params.Set("level", s.level)
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
if s.waitForActiveShards != nil {
params.Set("wait_for_active_shards", fmt.Sprintf("%d", *s.waitForActiveShards))
}
if s.waitForNodes != "" {
params.Set("wait_for_nodes", s.waitForNodes)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *ClusterHealthService) Validate() error {
return nil
}
// Do executes the operation.
func (s *ClusterHealthService) Do() (*ClusterHealthResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
resp := new(ClusterHealthResponse)
if err := json.Unmarshal(res.Body, resp); err != nil {
return nil, err
}
return resp, nil
}
// ClusterHealthResponse is the response of ClusterHealthService.Do.
type ClusterHealthResponse struct {
ClusterName string `json:"cluster_name"`
Status string `json:"status"`
TimedOut bool `json:"timed_out"`
NumberOfNodes int `json:"number_of_nodes"`
NumberOfDataNodes int `json:"number_of_data_nodes"`
ActivePrimaryShards int `json:"active_primary_shards"`
ActiveShards int `json:"active_shards"`
RelocatingShards int `json:"relocating_shards"`
InitializingShards int `json:"initializing_shards"`
UnassignedShards int `json:"unassigned_shards"`
NumberOfPendingTasks int `json:"number_of_pending_tasks"`
}

View File

@ -0,0 +1,109 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"net/url"
"testing"
)
func TestClusterHealth(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
// Get cluster health
res, err := client.ClusterHealth().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatalf("expected res to be != nil; got: %v", res)
}
if res.Status != "green" && res.Status != "red" && res.Status != "yellow" {
t.Fatalf("expected status \"green\", \"red\", or \"yellow\"; got: %q", res.Status)
}
}
func TestClusterHealthURLs(t *testing.T) {
tests := []struct {
Service *ClusterHealthService
ExpectedPath string
ExpectedParams url.Values
}{
{
Service: &ClusterHealthService{
indices: []string{},
},
ExpectedPath: "/_cluster/health/",
},
{
Service: &ClusterHealthService{
indices: []string{"twitter"},
},
ExpectedPath: "/_cluster/health/twitter",
},
{
Service: &ClusterHealthService{
indices: []string{"twitter", "gplus"},
},
ExpectedPath: "/_cluster/health/twitter%2Cgplus",
},
{
Service: &ClusterHealthService{
indices: []string{"twitter"},
waitForStatus: "yellow",
},
ExpectedPath: "/_cluster/health/twitter",
ExpectedParams: url.Values{"wait_for_status": []string{"yellow"}},
},
}
for _, test := range tests {
gotPath, gotParams, err := test.Service.buildURL()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if gotPath != test.ExpectedPath {
t.Errorf("expected URL path = %q; got: %q", test.ExpectedPath, gotPath)
}
if gotParams.Encode() != test.ExpectedParams.Encode() {
t.Errorf("expected URL params = %v; got: %v", test.ExpectedParams, gotParams)
}
}
}
func TestClusterHealthWaitForStatus(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
// Cluster health on an index that does not exist should never get to yellow
health, err := client.ClusterHealth().Index("no-such-index").WaitForStatus("yellow").Timeout("1s").Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if health.TimedOut != true {
t.Fatalf("expected to timeout; got: %v", health.TimedOut)
}
if health.Status != "red" {
t.Fatalf("expected health = %q; got: %q", "red", health.Status)
}
// Cluster wide health
health, err = client.ClusterHealth().WaitForStatus("green").Timeout("10s").Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if health.TimedOut != false {
t.Fatalf("expected no timeout; got: %v "+
"(does your local cluster contain unassigned shards?)", health.TimedOut)
}
if health.Status != "green" {
t.Fatalf("expected health = %q; got: %q", "green", health.Status)
}
// Cluster wide health via shortcut on client
err = client.WaitForGreenStatus("10s")
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
}

View File

@ -0,0 +1,197 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// ClusterStateService returns the state of the cluster.
// It is documented at http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/cluster-state.html.
type ClusterStateService struct {
client *Client
pretty bool
indices []string
metrics []string
local *bool
masterTimeout string
flatSettings *bool
}
// NewClusterStateService creates a new ClusterStateService.
func NewClusterStateService(client *Client) *ClusterStateService {
return &ClusterStateService{
client: client,
indices: make([]string, 0),
metrics: make([]string, 0),
}
}
// Index the name of the index. Use _all or an empty string to perform
// the operation on all indices.
func (s *ClusterStateService) Index(index string) *ClusterStateService {
s.indices = make([]string, 0)
s.indices = append(s.indices, index)
return s
}
// Indices is a list of index names. Use _all or an empty string to
// perform the operation on all indices.
func (s *ClusterStateService) Indices(indices ...string) *ClusterStateService {
s.indices = make([]string, 0)
s.indices = append(s.indices, indices...)
return s
}
// Metric limits the information returned to the specified metric.
// It can be one of: version, master_node, nodes, routing_table, metadata,
// blocks, or customs.
func (s *ClusterStateService) Metric(metric string) *ClusterStateService {
s.metrics = make([]string, 0)
s.metrics = append(s.metrics, metric)
return s
}
// Metrics limits the information returned to the specified metrics.
// It can be any of: version, master_node, nodes, routing_table, metadata,
// blocks, or customs.
func (s *ClusterStateService) Metrics(metrics ...string) *ClusterStateService {
s.metrics = make([]string, 0)
s.metrics = append(s.metrics, metrics...)
return s
}
// Local indicates whether to return local information. If it is true,
// we do not retrieve the state from master node (default: false).
func (s *ClusterStateService) Local(local bool) *ClusterStateService {
s.local = &local
return s
}
// MasterTimeout specifies the timeout for connection to master.
func (s *ClusterStateService) MasterTimeout(masterTimeout string) *ClusterStateService {
s.masterTimeout = masterTimeout
return s
}
// FlatSettings indicates whether to return settings in flat format (default: false).
func (s *ClusterStateService) FlatSettings(flatSettings bool) *ClusterStateService {
s.flatSettings = &flatSettings
return s
}
// buildURL builds the URL for the operation.
func (s *ClusterStateService) buildURL() (string, url.Values, error) {
// Build URL
metrics := strings.Join(s.metrics, ",")
if metrics == "" {
metrics = "_all"
}
indices := strings.Join(s.indices, ",")
if indices == "" {
indices = "_all"
}
path, err := uritemplates.Expand("/_cluster/state/{metrics}/{indices}", map[string]string{
"metrics": metrics,
"indices": indices,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
if s.flatSettings != nil {
params.Set("flat_settings", fmt.Sprintf("%v", *s.flatSettings))
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *ClusterStateService) Validate() error {
return nil
}
// Do executes the operation.
func (s *ClusterStateService) Do() (*ClusterStateResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(ClusterStateResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// ClusterStateResponse is the response of ClusterStateService.Do.
type ClusterStateResponse struct {
ClusterName string `json:"cluster_name"`
Version int `json:"version"`
MasterNode string `json:"master_node"`
Blocks map[string]interface{} `json:"blocks"`
Nodes map[string]*ClusterStateNode `json:"nodes"`
Metadata *ClusterStateMetadata `json:"metadata"`
RoutingTable map[string]*ClusterStateRoutingTable `json:"routing_table"`
RoutingNodes *ClusterStateRoutingNode `json:"routing_nodes"`
Allocations []interface{} `json:"allocations"`
Customs map[string]interface{} `json:"customs"`
}
type ClusterStateMetadata struct {
Templates map[string]interface{} `json:"templates"`
Indices map[string]interface{} `json:"indices"`
Repositories map[string]interface{} `json:"repositories"`
}
type ClusterStateNode struct {
Name string `json:"name"`
TransportAddress string `json:"transport_address"`
Attributes map[string]interface{} `json:"attributes"`
// TODO(oe) are these still valid?
State string `json:"state"`
Primary bool `json:"primary"`
Node string `json:"node"`
RelocatingNode *string `json:"relocating_node"`
Shard int `json:"shard"`
Index string `json:"index"`
}
type ClusterStateRoutingTable struct {
Indices map[string]interface{} `json:"indices"`
}
type ClusterStateRoutingNode struct {
Unassigned []interface{} `json:"unassigned"`
Nodes map[string]interface{} `json:"nodes"`
}

View File

@ -0,0 +1,92 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"net/url"
"testing"
)
func TestClusterState(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
// Get cluster state
res, err := client.ClusterState().Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatalf("expected res to be != nil; got: %v", res)
}
if res.ClusterName == "" {
t.Fatalf("expected a cluster name; got: %q", res.ClusterName)
}
}
func TestClusterStateURLs(t *testing.T) {
tests := []struct {
Service *ClusterStateService
ExpectedPath string
ExpectedParams url.Values
}{
{
Service: &ClusterStateService{
indices: []string{},
metrics: []string{},
},
ExpectedPath: "/_cluster/state/_all/_all",
},
{
Service: &ClusterStateService{
indices: []string{"twitter"},
metrics: []string{},
},
ExpectedPath: "/_cluster/state/_all/twitter",
},
{
Service: &ClusterStateService{
indices: []string{"twitter", "gplus"},
metrics: []string{},
},
ExpectedPath: "/_cluster/state/_all/twitter%2Cgplus",
},
{
Service: &ClusterStateService{
indices: []string{},
metrics: []string{"nodes"},
},
ExpectedPath: "/_cluster/state/nodes/_all",
},
{
Service: &ClusterStateService{
indices: []string{"twitter"},
metrics: []string{"nodes"},
},
ExpectedPath: "/_cluster/state/nodes/twitter",
},
{
Service: &ClusterStateService{
indices: []string{"twitter"},
metrics: []string{"nodes"},
masterTimeout: "1s",
},
ExpectedPath: "/_cluster/state/nodes/twitter",
ExpectedParams: url.Values{"master_timeout": []string{"1s"}},
},
}
for _, test := range tests {
gotPath, gotParams, err := test.Service.buildURL()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if gotPath != test.ExpectedPath {
t.Errorf("expected URL path = %q; got: %q", test.ExpectedPath, gotPath)
}
if gotParams.Encode() != test.ExpectedParams.Encode() {
t.Errorf("expected URL params = %v; got: %v", test.ExpectedParams, gotParams)
}
}
}

View File

@ -0,0 +1,349 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// ClusterStatsService is documented at http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/cluster-stats.html.
type ClusterStatsService struct {
client *Client
pretty bool
nodeId []string
flatSettings *bool
human *bool
}
// NewClusterStatsService creates a new ClusterStatsService.
func NewClusterStatsService(client *Client) *ClusterStatsService {
return &ClusterStatsService{
client: client,
nodeId: make([]string, 0),
}
}
// NodeId is documented as: A comma-separated list of node IDs or names to limit the returned information; use `_local` to return information from the node you're connecting to, leave empty to get information from all nodes.
func (s *ClusterStatsService) NodeId(nodeId []string) *ClusterStatsService {
s.nodeId = nodeId
return s
}
// FlatSettings is documented as: Return settings in flat format (default: false).
func (s *ClusterStatsService) FlatSettings(flatSettings bool) *ClusterStatsService {
s.flatSettings = &flatSettings
return s
}
// Human is documented as: Whether to return time and byte values in human-readable format..
func (s *ClusterStatsService) Human(human bool) *ClusterStatsService {
s.human = &human
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *ClusterStatsService) Pretty(pretty bool) *ClusterStatsService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *ClusterStatsService) buildURL() (string, url.Values, error) {
// Build URL
var err error
var path string
if len(s.nodeId) > 0 {
path, err = uritemplates.Expand("/_cluster/stats/nodes/{node_id}", map[string]string{
"node_id": strings.Join(s.nodeId, ","),
})
if err != nil {
return "", url.Values{}, err
}
} else {
path, err = uritemplates.Expand("/_cluster/stats", map[string]string{})
if err != nil {
return "", url.Values{}, err
}
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.flatSettings != nil {
params.Set("flat_settings", fmt.Sprintf("%v", *s.flatSettings))
}
if s.human != nil {
params.Set("human", fmt.Sprintf("%v", *s.human))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *ClusterStatsService) Validate() error {
return nil
}
// Do executes the operation.
func (s *ClusterStatsService) Do() (*ClusterStatsResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(ClusterStatsResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// ClusterStatsResponse is the response of ClusterStatsService.Do.
type ClusterStatsResponse struct {
Timestamp int64 `json:"timestamp"`
ClusterName string `json:"cluster_name"`
ClusterUUID string `json:"uuid"`
Status string `json:"status"`
Indices *ClusterStatsIndices `json:"indices"`
Nodes *ClusterStatsNodes `json:"nodes"`
}
type ClusterStatsIndices struct {
Count int `json:"count"`
Shards *ClusterStatsIndicesShards `json:"shards"`
Docs *ClusterStatsIndicesDocs `json:"docs"`
Store *ClusterStatsIndicesStore `json:"store"`
FieldData *ClusterStatsIndicesFieldData `json:"fielddata"`
FilterCache *ClusterStatsIndicesFilterCache `json:"filter_cache"`
IdCache *ClusterStatsIndicesIdCache `json:"id_cache"`
Completion *ClusterStatsIndicesCompletion `json:"completion"`
Segments *ClusterStatsIndicesSegments `json:"segments"`
Percolate *ClusterStatsIndicesPercolate `json:"percolate"`
}
type ClusterStatsIndicesShards struct {
Total int `json:"total"`
Primaries int `json:"primaries"`
Replication float64 `json:"replication"`
Index *ClusterStatsIndicesShardsIndex `json:"index"`
}
type ClusterStatsIndicesShardsIndex struct {
Shards *ClusterStatsIndicesShardsIndexIntMinMax `json:"shards"`
Primaries *ClusterStatsIndicesShardsIndexIntMinMax `json:"primaries"`
Replication *ClusterStatsIndicesShardsIndexFloat64MinMax `json:"replication"`
}
type ClusterStatsIndicesShardsIndexIntMinMax struct {
Min int `json:"min"`
Max int `json:"max"`
Avg float64 `json:"avg"`
}
type ClusterStatsIndicesShardsIndexFloat64MinMax struct {
Min float64 `json:"min"`
Max float64 `json:"max"`
Avg float64 `json:"avg"`
}
type ClusterStatsIndicesDocs struct {
Count int `json:"count"`
Deleted int `json:"deleted"`
}
type ClusterStatsIndicesStore struct {
Size string `json:"size"` // e.g. "5.3gb"
SizeInBytes int64 `json:"size_in_bytes"`
ThrottleTime string `json:"throttle_time"` // e.g. "0s"
ThrottleTimeInMillis int64 `json:"throttle_time_in_millis"`
}
type ClusterStatsIndicesFieldData struct {
MemorySize string `json:"memory_size"` // e.g. "61.3kb"
MemorySizeInBytes int64 `json:"memory_size_in_bytes"`
Evictions int64 `json:"evictions"`
Fields map[string]struct {
MemorySize string `json:"memory_size"` // e.g. "61.3kb"
MemorySizeInBytes int64 `json:"memory_size_in_bytes"`
} `json:"fields"`
}
type ClusterStatsIndicesFilterCache struct {
MemorySize string `json:"memory_size"` // e.g. "61.3kb"
MemorySizeInBytes int64 `json:"memory_size_in_bytes"`
Evictions int64 `json:"evictions"`
}
type ClusterStatsIndicesIdCache struct {
MemorySize string `json:"memory_size"` // e.g. "61.3kb"
MemorySizeInBytes int64 `json:"memory_size_in_bytes"`
}
type ClusterStatsIndicesCompletion struct {
Size string `json:"size"` // e.g. "61.3kb"
SizeInBytes int64 `json:"size_in_bytes"`
Fields map[string]struct {
Size string `json:"size"` // e.g. "61.3kb"
SizeInBytes int64 `json:"size_in_bytes"`
} `json:"fields"`
}
type ClusterStatsIndicesSegments struct {
Count int64 `json:"count"`
Memory string `json:"memory"` // e.g. "61.3kb"
MemoryInBytes int64 `json:"memory_in_bytes"`
IndexWriterMemory string `json:"index_writer_memory"` // e.g. "61.3kb"
IndexWriterMemoryInBytes int64 `json:"index_writer_memory_in_bytes"`
IndexWriterMaxMemory string `json:"index_writer_max_memory"` // e.g. "61.3kb"
IndexWriterMaxMemoryInBytes int64 `json:"index_writer_max_memory_in_bytes"`
VersionMapMemory string `json:"version_map_memory"` // e.g. "61.3kb"
VersionMapMemoryInBytes int64 `json:"version_map_memory_in_bytes"`
FixedBitSet string `json:"fixed_bit_set"` // e.g. "61.3kb"
FixedBitSetInBytes int64 `json:"fixed_bit_set_memory_in_bytes"`
}
type ClusterStatsIndicesPercolate struct {
Total int64 `json:"total"`
// TODO(oe) The JSON tag here is wrong as of ES 1.5.2 it seems
Time string `json:"get_time"` // e.g. "1s"
TimeInBytes int64 `json:"time_in_millis"`
Current int64 `json:"current"`
MemorySize string `json:"memory_size"` // e.g. "61.3kb"
MemorySizeInBytes int64 `json:"memory_sitze_in_bytes"`
Queries int64 `json:"queries"`
}
// ---
type ClusterStatsNodes struct {
Count *ClusterStatsNodesCounts `json:"counts"`
Versions []string `json:"versions"`
OS *ClusterStatsNodesOsStats `json:"os"`
Process *ClusterStatsNodesProcessStats `json:"process"`
JVM *ClusterStatsNodesJvmStats `json:"jvm"`
FS *ClusterStatsNodesFsStats `json:"fs"`
Plugins []*ClusterStatsNodesPlugin `json:"plugins"`
}
type ClusterStatsNodesCounts struct {
Total int `json:"total"`
MasterOnly int `json:"master_only"`
DataOnly int `json:"data_only"`
MasterData int `json:"master_data"`
Client int `json:"client"`
}
type ClusterStatsNodesOsStats struct {
AvailableProcessors int `json:"available_processors"`
Mem *ClusterStatsNodesOsStatsMem `json:"mem"`
CPU []*ClusterStatsNodesOsStatsCPU `json:"cpu"`
}
type ClusterStatsNodesOsStatsMem struct {
Total string `json:"total"` // e.g. "16gb"
TotalInBytes int64 `json:"total_in_bytes"`
}
type ClusterStatsNodesOsStatsCPU struct {
Vendor string `json:"vendor"`
Model string `json:"model"`
MHz int `json:"mhz"`
TotalCores int `json:"total_cores"`
TotalSockets int `json:"total_sockets"`
CoresPerSocket int `json:"cores_per_socket"`
CacheSize string `json:"cache_size"` // e.g. "256b"
CacheSizeInBytes int64 `json:"cache_size_in_bytes"`
Count int `json:"count"`
}
type ClusterStatsNodesProcessStats struct {
CPU *ClusterStatsNodesProcessStatsCPU `json:"cpu"`
OpenFileDescriptors *ClusterStatsNodesProcessStatsOpenFileDescriptors `json:"open_file_descriptors"`
}
type ClusterStatsNodesProcessStatsCPU struct {
Percent float64 `json:"percent"`
}
type ClusterStatsNodesProcessStatsOpenFileDescriptors struct {
Min int64 `json:"min"`
Max int64 `json:"max"`
Avg int64 `json:"avg"`
}
type ClusterStatsNodesJvmStats struct {
MaxUptime string `json:"max_uptime"` // e.g. "5h"
MaxUptimeInMillis int64 `json:"max_uptime_in_millis"`
Versions []*ClusterStatsNodesJvmStatsVersion `json:"versions"`
Mem *ClusterStatsNodesJvmStatsMem `json:"mem"`
Threads int64 `json:"threads"`
}
type ClusterStatsNodesJvmStatsVersion struct {
Version string `json:"version"` // e.g. "1.8.0_45"
VMName string `json:"vm_name"` // e.g. "Java HotSpot(TM) 64-Bit Server VM"
VMVersion string `json:"vm_version"` // e.g. "25.45-b02"
VMVendor string `json:"vm_vendor"` // e.g. "Oracle Corporation"
Count int `json:"count"`
}
type ClusterStatsNodesJvmStatsMem struct {
HeapUsed string `json:"heap_used"`
HeapUsedInBytes int64 `json:"heap_used_in_bytes"`
HeapMax string `json:"heap_max"`
HeapMaxInBytes int64 `json:"heap_max_in_bytes"`
}
type ClusterStatsNodesFsStats struct {
Path string `json:"path"`
Mount string `json:"mount"`
Dev string `json:"dev"`
Total string `json:"total"` // e.g. "930.7gb"`
TotalInBytes int64 `json:"total_in_bytes"`
Free string `json:"free"` // e.g. "930.7gb"`
FreeInBytes int64 `json:"free_in_bytes"`
Available string `json:"available"` // e.g. "930.7gb"`
AvailableInBytes int64 `json:"available_in_bytes"`
DiskReads int64 `json:"disk_reads"`
DiskWrites int64 `json:"disk_writes"`
DiskIOOp int64 `json:"disk_io_op"`
DiskReadSize string `json:"disk_read_size"` // e.g. "0b"`
DiskReadSizeInBytes int64 `json:"disk_read_size_in_bytes"`
DiskWriteSize string `json:"disk_write_size"` // e.g. "0b"`
DiskWriteSizeInBytes int64 `json:"disk_write_size_in_bytes"`
DiskIOSize string `json:"disk_io_size"` // e.g. "0b"`
DiskIOSizeInBytes int64 `json:"disk_io_size_in_bytes"`
DiskQueue string `json:"disk_queue"`
DiskServiceTime string `json:"disk_service_time"`
}
type ClusterStatsNodesPlugin struct {
Name string `json:"name"`
Version string `json:"version"`
Description string `json:"description"`
URL string `json:"url"`
JVM bool `json:"jvm"`
Site bool `json:"site"`
}

View File

@ -0,0 +1,85 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"net/url"
"testing"
)
func TestClusterStats(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
// Get cluster stats
res, err := client.ClusterStats().Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatalf("expected res to be != nil; got: %v", res)
}
if res.ClusterName == "" {
t.Fatalf("expected a cluster name; got: %q", res.ClusterName)
}
}
func TestClusterStatsURLs(t *testing.T) {
fFlag := false
tFlag := true
tests := []struct {
Service *ClusterStatsService
ExpectedPath string
ExpectedParams url.Values
}{
{
Service: &ClusterStatsService{
nodeId: []string{},
},
ExpectedPath: "/_cluster/stats",
},
{
Service: &ClusterStatsService{
nodeId: []string{"node1"},
},
ExpectedPath: "/_cluster/stats/nodes/node1",
},
{
Service: &ClusterStatsService{
nodeId: []string{"node1", "node2"},
},
ExpectedPath: "/_cluster/stats/nodes/node1%2Cnode2",
},
{
Service: &ClusterStatsService{
nodeId: []string{},
flatSettings: &tFlag,
},
ExpectedPath: "/_cluster/stats",
ExpectedParams: url.Values{"flat_settings": []string{"true"}},
},
{
Service: &ClusterStatsService{
nodeId: []string{"node1"},
flatSettings: &fFlag,
},
ExpectedPath: "/_cluster/stats/nodes/node1",
ExpectedParams: url.Values{"flat_settings": []string{"false"}},
},
}
for _, test := range tests {
gotPath, gotParams, err := test.Service.buildURL()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if gotPath != test.ExpectedPath {
t.Errorf("expected URL path = %q; got: %q", test.ExpectedPath, gotPath)
}
if gotParams.Encode() != test.ExpectedParams.Encode() {
t.Errorf("expected URL params = %v; got: %v", test.ExpectedParams, gotParams)
}
}
}

View File

@ -0,0 +1,90 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
"sync"
"time"
)
// conn represents a single connection to a node in a cluster.
type conn struct {
sync.RWMutex
nodeID string // node ID
url string
failures int
dead bool
deadSince *time.Time
}
// newConn creates a new connection to the given URL.
func newConn(nodeID, url string) *conn {
c := &conn{
nodeID: nodeID,
url: url,
}
return c
}
// String returns a representation of the connection status.
func (c *conn) String() string {
c.RLock()
defer c.RUnlock()
return fmt.Sprintf("%s [dead=%v,failures=%d,deadSince=%v]", c.url, c.dead, c.failures, c.deadSince)
}
// NodeID returns the ID of the node of this connection.
func (c *conn) NodeID() string {
c.RLock()
defer c.RUnlock()
return c.nodeID
}
// URL returns the URL of this connection.
func (c *conn) URL() string {
c.RLock()
defer c.RUnlock()
return c.url
}
// IsDead returns true if this connection is marked as dead, i.e. a previous
// request to the URL has been unsuccessful.
func (c *conn) IsDead() bool {
c.RLock()
defer c.RUnlock()
return c.dead
}
// MarkAsDead marks this connection as dead, increments the failures
// counter and stores the current time in dead since.
func (c *conn) MarkAsDead() {
c.Lock()
c.dead = true
if c.deadSince == nil {
utcNow := time.Now().UTC()
c.deadSince = &utcNow
}
c.failures += 1
c.Unlock()
}
// MarkAsAlive marks this connection as eligible to be returned from the
// pool of connections by the selector.
func (c *conn) MarkAsAlive() {
c.Lock()
c.dead = false
c.Unlock()
}
// MarkAsHealthy marks this connection as healthy, i.e. a request has been
// successfully performed with it.
func (c *conn) MarkAsHealthy() {
c.Lock()
c.dead = false
c.deadSince = nil
c.failures = 0
c.Unlock()
}

View File

@ -0,0 +1,152 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// CountService is a convenient service for determining the
// number of documents in an index. Use SearchService with
// a SearchType of count for counting with queries etc.
type CountService struct {
client *Client
indices []string
types []string
query Query
pretty bool
}
// CountResult is the result returned from using the Count API
// (http://www.elasticsearch.org/guide/reference/api/count/)
type CountResult struct {
Count int64 `json:"count"`
Shards shardsInfo `json:"_shards,omitempty"`
}
func NewCountService(client *Client) *CountService {
builder := &CountService{
client: client,
}
return builder
}
func (s *CountService) Index(index string) *CountService {
if s.indices == nil {
s.indices = make([]string, 0)
}
s.indices = append(s.indices, index)
return s
}
func (s *CountService) Indices(indices ...string) *CountService {
if s.indices == nil {
s.indices = make([]string, 0)
}
s.indices = append(s.indices, indices...)
return s
}
func (s *CountService) Type(typ string) *CountService {
if s.types == nil {
s.types = make([]string, 0)
}
s.types = append(s.types, typ)
return s
}
func (s *CountService) Types(types ...string) *CountService {
if s.types == nil {
s.types = make([]string, 0)
}
s.types = append(s.types, types...)
return s
}
func (s *CountService) Query(query Query) *CountService {
s.query = query
return s
}
func (s *CountService) Pretty(pretty bool) *CountService {
s.pretty = pretty
return s
}
func (s *CountService) Do() (int64, error) {
var err error
// Build url
path := "/"
// Indices part
indexPart := make([]string, 0)
for _, index := range s.indices {
index, err = uritemplates.Expand("{index}", map[string]string{
"index": index,
})
if err != nil {
return 0, err
}
indexPart = append(indexPart, index)
}
if len(indexPart) > 0 {
path += strings.Join(indexPart, ",")
}
// Types part
typesPart := make([]string, 0)
for _, typ := range s.types {
typ, err = uritemplates.Expand("{type}", map[string]string{
"type": typ,
})
if err != nil {
return 0, err
}
typesPart = append(typesPart, typ)
}
if len(typesPart) > 0 {
path += "/" + strings.Join(typesPart, ",")
}
// Search
path += "/_count"
// Parameters
params := make(url.Values)
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
// Set body if there is a query specified
var body interface{}
if s.query != nil {
query := make(map[string]interface{})
query["query"] = s.query.Source()
body = query
}
// Get response
res, err := s.client.PerformRequest("POST", path, params, body)
if err != nil {
return 0, err
}
// Return result
ret := new(CountResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return 0, err
}
if ret != nil {
return ret.Count, nil
}
return int64(0), nil
}

View File

@ -0,0 +1,83 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import "testing"
func TestCount(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Count documents
count, err := client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 3 {
t.Errorf("expected Count = %d; got %d", 3, count)
}
// Count documents
count, err = client.Count(testIndexName).Type("tweet").Do()
if err != nil {
t.Fatal(err)
}
if count != 3 {
t.Errorf("expected Count = %d; got %d", 3, count)
}
// Count documents
count, err = client.Count(testIndexName).Type("gezwitscher").Do()
if err != nil {
t.Fatal(err)
}
if count != 0 {
t.Errorf("expected Count = %d; got %d", 0, count)
}
// Count with query
query := NewTermQuery("user", "olivere")
count, err = client.Count(testIndexName).Query(query).Do()
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected Count = %d; got %d", 2, count)
}
// Count with query and type
query = NewTermQuery("user", "olivere")
count, err = client.Count(testIndexName).Type("tweet").Query(query).Do()
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected Count = %d; got %d", 2, count)
}
}

View File

@ -0,0 +1,126 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"errors"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// CreateIndexService creates a new index.
type CreateIndexService struct {
client *Client
pretty bool
index string
timeout string
masterTimeout string
bodyJson interface{}
bodyString string
}
// NewCreateIndexService returns a new CreateIndexService.
func NewCreateIndexService(client *Client) *CreateIndexService {
return &CreateIndexService{client: client}
}
// Index is the name of the index to create.
func (b *CreateIndexService) Index(index string) *CreateIndexService {
b.index = index
return b
}
// Timeout the explicit operation timeout, e.g. "5s".
func (s *CreateIndexService) Timeout(timeout string) *CreateIndexService {
s.timeout = timeout
return s
}
// MasterTimeout specifies the timeout for connection to master.
func (s *CreateIndexService) MasterTimeout(masterTimeout string) *CreateIndexService {
s.masterTimeout = masterTimeout
return s
}
// Body specifies the configuration of the index as a string.
// It is an alias for BodyString.
func (b *CreateIndexService) Body(body string) *CreateIndexService {
b.bodyString = body
return b
}
// BodyString specifies the configuration of the index as a string.
func (b *CreateIndexService) BodyString(body string) *CreateIndexService {
b.bodyString = body
return b
}
// BodyJson specifies the configuration of the index. The interface{} will
// be serializes as a JSON document, so use a map[string]interface{}.
func (b *CreateIndexService) BodyJson(body interface{}) *CreateIndexService {
b.bodyJson = body
return b
}
// Pretty indicates that the JSON response be indented and human readable.
func (b *CreateIndexService) Pretty(pretty bool) *CreateIndexService {
b.pretty = pretty
return b
}
// Do executes the operation.
func (b *CreateIndexService) Do() (*CreateIndexResult, error) {
if b.index == "" {
return nil, errors.New("missing index name")
}
// Build url
path, err := uritemplates.Expand("/{index}", map[string]string{
"index": b.index,
})
if err != nil {
return nil, err
}
params := make(url.Values)
if b.pretty {
params.Set("pretty", "1")
}
if b.masterTimeout != "" {
params.Set("master_timeout", b.masterTimeout)
}
if b.timeout != "" {
params.Set("timeout", b.timeout)
}
// Setup HTTP request body
var body interface{}
if b.bodyJson != nil {
body = b.bodyJson
} else {
body = b.bodyString
}
// Get response
res, err := b.client.PerformRequest("PUT", path, params, body)
if err != nil {
return nil, err
}
ret := new(CreateIndexResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of a create index request.
// CreateIndexResult is the outcome of creating a new index.
type CreateIndexResult struct {
Acknowledged bool `json:"acknowledged"`
}

View File

@ -0,0 +1,26 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
)
// Decoder is used to decode responses from Elasticsearch.
// Users of elastic can implement their own marshaler for advanced purposes
// and set them per Client (see SetDecoder). If none is specified,
// DefaultDecoder is used.
type Decoder interface {
Decode(data []byte, v interface{}) error
}
// DefaultDecoder uses json.Unmarshal from the Go standard library
// to decode JSON data.
type DefaultDecoder struct{}
// Decode decodes with json.Unmarshal from the Go standard library.
func (u *DefaultDecoder) Decode(data []byte, v interface{}) error {
return json.Unmarshal(data, v)
}

View File

@ -0,0 +1,49 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"bytes"
"encoding/json"
"sync/atomic"
"testing"
)
type decoder struct {
dec json.Decoder
N int64
}
func (d *decoder) Decode(data []byte, v interface{}) error {
atomic.AddInt64(&d.N, 1)
dec := json.NewDecoder(bytes.NewReader(data))
dec.UseNumber()
return dec.Decode(v)
}
func TestDecoder(t *testing.T) {
dec := &decoder{}
client := setupTestClientAndCreateIndex(t, SetDecoder(dec), SetMaxRetries(0))
tweet := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
// Add a document
indexResult, err := client.Index().
Index(testIndexName).
Type("tweet").
Id("1").
BodyJson(&tweet).
Do()
if err != nil {
t.Fatal(err)
}
if indexResult == nil {
t.Errorf("expected result to be != nil; got: %v", indexResult)
}
if dec.N <= 0 {
t.Errorf("expected at least 1 call of decoder; got: %d", dec.N)
}
}

View File

@ -0,0 +1,130 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type DeleteService struct {
client *Client
index string
_type string
id string
routing string
refresh *bool
version *int
pretty bool
}
func NewDeleteService(client *Client) *DeleteService {
builder := &DeleteService{
client: client,
}
return builder
}
func (s *DeleteService) Index(index string) *DeleteService {
s.index = index
return s
}
func (s *DeleteService) Type(_type string) *DeleteService {
s._type = _type
return s
}
func (s *DeleteService) Id(id string) *DeleteService {
s.id = id
return s
}
func (s *DeleteService) Parent(parent string) *DeleteService {
if s.routing == "" {
s.routing = parent
}
return s
}
func (s *DeleteService) Refresh(refresh bool) *DeleteService {
s.refresh = &refresh
return s
}
func (s *DeleteService) Version(version int) *DeleteService {
s.version = &version
return s
}
func (s *DeleteService) Pretty(pretty bool) *DeleteService {
s.pretty = pretty
return s
}
// Do deletes the document. It fails if any of index, type, and identifier
// are missing.
func (s *DeleteService) Do() (*DeleteResult, error) {
if s.index == "" {
return nil, ErrMissingIndex
}
if s._type == "" {
return nil, ErrMissingType
}
if s.id == "" {
return nil, ErrMissingId
}
// Build url
path, err := uritemplates.Expand("/{index}/{type}/{id}", map[string]string{
"index": s.index,
"type": s._type,
"id": s.id,
})
if err != nil {
return nil, err
}
// Parameters
params := make(url.Values)
if s.refresh != nil {
params.Set("refresh", fmt.Sprintf("%v", *s.refresh))
}
if s.version != nil {
params.Set("version", fmt.Sprintf("%d", *s.version))
}
if s.routing != "" {
params.Set("routing", fmt.Sprintf("%s", s.routing))
}
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
// Get response
res, err := s.client.PerformRequest("DELETE", path, params, nil)
if err != nil {
return nil, err
}
// Return response
ret := new(DeleteResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of a delete request.
type DeleteResult struct {
Found bool `json:"found"`
Index string `json:"_index"`
Type string `json:"_type"`
Id string `json:"_id"`
Version int64 `json:"_version"`
}

View File

@ -0,0 +1,292 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// DeleteByQueryService deletes documents that match a query.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/docs-delete-by-query.html.
type DeleteByQueryService struct {
client *Client
indices []string
types []string
analyzer string
consistency string
defaultOper string
df string
ignoreUnavailable *bool
allowNoIndices *bool
expandWildcards string
replication string
routing string
timeout string
pretty bool
q string
query Query
}
// NewDeleteByQueryService creates a new DeleteByQueryService.
// You typically use the client's DeleteByQuery to get a reference to
// the service.
func NewDeleteByQueryService(client *Client) *DeleteByQueryService {
builder := &DeleteByQueryService{
client: client,
}
return builder
}
// Index limits the delete-by-query to a single index.
// You can use _all to perform the operation on all indices.
func (s *DeleteByQueryService) Index(index string) *DeleteByQueryService {
if s.indices == nil {
s.indices = make([]string, 0)
}
s.indices = append(s.indices, index)
return s
}
// Indices sets the indices on which to perform the delete operation.
func (s *DeleteByQueryService) Indices(indices ...string) *DeleteByQueryService {
if s.indices == nil {
s.indices = make([]string, 0)
}
s.indices = append(s.indices, indices...)
return s
}
// Type limits the delete operation to the given type.
func (s *DeleteByQueryService) Type(typ string) *DeleteByQueryService {
if s.types == nil {
s.types = make([]string, 0)
}
s.types = append(s.types, typ)
return s
}
// Types limits the delete operation to the given types.
func (s *DeleteByQueryService) Types(types ...string) *DeleteByQueryService {
if s.types == nil {
s.types = make([]string, 0)
}
s.types = append(s.types, types...)
return s
}
// Analyzer to use for the query string.
func (s *DeleteByQueryService) Analyzer(analyzer string) *DeleteByQueryService {
s.analyzer = analyzer
return s
}
// Consistency represents the specific write consistency setting for the operation.
// It can be one, quorum, or all.
func (s *DeleteByQueryService) Consistency(consistency string) *DeleteByQueryService {
s.consistency = consistency
return s
}
// DefaultOperator for query string query (AND or OR).
func (s *DeleteByQueryService) DefaultOperator(defaultOperator string) *DeleteByQueryService {
s.defaultOper = defaultOperator
return s
}
// DF is the field to use as default where no field prefix is given in the query string.
func (s *DeleteByQueryService) DF(defaultField string) *DeleteByQueryService {
s.df = defaultField
return s
}
// DefaultField is the field to use as default where no field prefix is given in the query string.
// It is an alias to the DF func.
func (s *DeleteByQueryService) DefaultField(defaultField string) *DeleteByQueryService {
s.df = defaultField
return s
}
// IgnoreUnavailable indicates whether specified concrete indices should be
// ignored when unavailable (missing or closed).
func (s *DeleteByQueryService) IgnoreUnavailable(ignore bool) *DeleteByQueryService {
s.ignoreUnavailable = &ignore
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard indices
// expression resolves into no concrete indices (including the _all string
// or when no indices have been specified).
func (s *DeleteByQueryService) AllowNoIndices(allow bool) *DeleteByQueryService {
s.allowNoIndices = &allow
return s
}
// ExpandWildcards indicates whether to expand wildcard expression to
// concrete indices that are open, closed or both. It can be "open" or "closed".
func (s *DeleteByQueryService) ExpandWildcards(expand string) *DeleteByQueryService {
s.expandWildcards = expand
return s
}
// Replication sets a specific replication type (sync or async).
func (s *DeleteByQueryService) Replication(replication string) *DeleteByQueryService {
s.replication = replication
return s
}
// Q specifies the query in Lucene query string syntax. You can also use
// Query to programmatically specify the query.
func (s *DeleteByQueryService) Q(query string) *DeleteByQueryService {
s.q = query
return s
}
// QueryString is an alias to Q. Notice that you can also use Query to
// programmatically set the query.
func (s *DeleteByQueryService) QueryString(query string) *DeleteByQueryService {
s.q = query
return s
}
// Routing sets a specific routing value.
func (s *DeleteByQueryService) Routing(routing string) *DeleteByQueryService {
s.routing = routing
return s
}
// Timeout sets an explicit operation timeout, e.g. "1s" or "10000ms".
func (s *DeleteByQueryService) Timeout(timeout string) *DeleteByQueryService {
s.timeout = timeout
return s
}
// Pretty indents the JSON output from Elasticsearch.
func (s *DeleteByQueryService) Pretty(pretty bool) *DeleteByQueryService {
s.pretty = pretty
return s
}
// Query sets the query programmatically.
func (s *DeleteByQueryService) Query(query Query) *DeleteByQueryService {
s.query = query
return s
}
// Do executes the delete-by-query operation.
func (s *DeleteByQueryService) Do() (*DeleteByQueryResult, error) {
var err error
// Build url
path := "/"
// Indices part
indexPart := make([]string, 0)
for _, index := range s.indices {
index, err = uritemplates.Expand("{index}", map[string]string{
"index": index,
})
if err != nil {
return nil, err
}
indexPart = append(indexPart, index)
}
if len(indexPart) > 0 {
path += strings.Join(indexPart, ",")
}
// Types part
typesPart := make([]string, 0)
for _, typ := range s.types {
typ, err = uritemplates.Expand("{type}", map[string]string{
"type": typ,
})
if err != nil {
return nil, err
}
typesPart = append(typesPart, typ)
}
if len(typesPart) > 0 {
path += "/" + strings.Join(typesPart, ",")
}
// Search
path += "/_query"
// Parameters
params := make(url.Values)
if s.analyzer != "" {
params.Set("analyzer", s.analyzer)
}
if s.consistency != "" {
params.Set("consistency", s.consistency)
}
if s.defaultOper != "" {
params.Set("default_operator", s.defaultOper)
}
if s.df != "" {
params.Set("df", s.df)
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
if s.replication != "" {
params.Set("replication", s.replication)
}
if s.routing != "" {
params.Set("routing", s.routing)
}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
if s.q != "" {
params.Set("q", s.q)
}
// Set body if there is a query set
var body interface{}
if s.query != nil {
query := make(map[string]interface{})
query["query"] = s.query.Source()
body = query
}
// Get response
res, err := s.client.PerformRequest("DELETE", path, params, body)
if err != nil {
return nil, err
}
// Return result
ret := new(DeleteByQueryResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// DeleteByQueryResult is the outcome of executing Do with DeleteByQueryService.
type DeleteByQueryResult struct {
Indices map[string]IndexDeleteByQueryResult `json:"_indices"`
}
// IndexDeleteByQueryResult is the result of a delete-by-query for a specific
// index.
type IndexDeleteByQueryResult struct {
Shards shardsInfo `json:"_shards"`
}

View File

@ -0,0 +1,76 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestDeleteByQuery(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Count documents
count, err := client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 3 {
t.Fatalf("expected count = %d; got: %d", 3, count)
}
// Delete all documents by sandrae
q := NewTermQuery("user", "sandrae")
res, err := client.DeleteByQuery().Index(testIndexName).Type("tweet").Query(q).Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatalf("expected response != nil; got: %v", res)
}
idx, found := res.Indices[testIndexName]
if !found {
t.Errorf("expected Found = true; got: %v", found)
}
if idx.Shards.Failed > 0 {
t.Errorf("expected no failed shards; got: %d", idx.Shards.Failed)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
count, err = client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Fatalf("expected Count = %d; got: %d", 2, count)
}
}

View File

@ -0,0 +1,57 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type DeleteIndexService struct {
client *Client
index string
}
func NewDeleteIndexService(client *Client) *DeleteIndexService {
builder := &DeleteIndexService{
client: client,
}
return builder
}
func (b *DeleteIndexService) Index(index string) *DeleteIndexService {
b.index = index
return b
}
func (b *DeleteIndexService) Do() (*DeleteIndexResult, error) {
// Build url
path, err := uritemplates.Expand("/{index}/", map[string]string{
"index": b.index,
})
if err != nil {
return nil, err
}
// Get response
res, err := b.client.PerformRequest("DELETE", path, nil, nil)
if err != nil {
return nil, err
}
// Return result
ret := new(DeleteIndexResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of a delete index request.
type DeleteIndexResult struct {
Acknowledged bool `json:"acknowledged"`
}

View File

@ -0,0 +1,136 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// DeleteMappingService allows to delete a mapping along with its data.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-delete-mapping.html.
type DeleteMappingService struct {
client *Client
pretty bool
index []string
typ []string
masterTimeout string
}
// NewDeleteMappingService creates a new DeleteMappingService.
func NewDeleteMappingService(client *Client) *DeleteMappingService {
return &DeleteMappingService{
client: client,
index: make([]string, 0),
typ: make([]string, 0),
}
}
// Index is a list of index names (supports wildcards). Use `_all` for all indices.
func (s *DeleteMappingService) Index(index ...string) *DeleteMappingService {
s.index = append(s.index, index...)
return s
}
// Type is a list of document types to delete (supports wildcards).
// Use `_all` to delete all document types in the specified indices..
func (s *DeleteMappingService) Type(typ ...string) *DeleteMappingService {
s.typ = append(s.typ, typ...)
return s
}
// MasterTimeout specifies the timeout for connecting to master.
func (s *DeleteMappingService) MasterTimeout(masterTimeout string) *DeleteMappingService {
s.masterTimeout = masterTimeout
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *DeleteMappingService) Pretty(pretty bool) *DeleteMappingService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *DeleteMappingService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/{index}/_mapping/{type}", map[string]string{
"index": strings.Join(s.index, ","),
"type": strings.Join(s.typ, ","),
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *DeleteMappingService) Validate() error {
var invalid []string
if len(s.index) == 0 {
invalid = append(invalid, "Index")
}
if len(s.typ) == 0 {
invalid = append(invalid, "Type")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *DeleteMappingService) Do() (*DeleteMappingResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("DELETE", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(DeleteMappingResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// DeleteMappingResponse is the response of DeleteMappingService.Do.
type DeleteMappingResponse struct {
Acknowledged bool `json:"acknowledged"`
}

View File

@ -0,0 +1,40 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestDeleteMappingURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Indices []string
Types []string
Expected string
}{
{
[]string{"twitter"},
[]string{"tweet"},
"/twitter/_mapping/tweet",
},
{
[]string{"store-1", "store-2"},
[]string{"tweet", "user"},
"/store-1%2Cstore-2/_mapping/tweet%2Cuser",
},
}
for _, test := range tests {
path, _, err := client.DeleteMapping().Index(test.Indices...).Type(test.Types...).buildURL()
if err != nil {
t.Fatal(err)
}
if path != test.Expected {
t.Errorf("expected %q; got: %q", test.Expected, path)
}
}
}

View File

@ -0,0 +1,118 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// DeleteTemplateService deletes a search template. More information can
// be found at http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-template.html.
type DeleteTemplateService struct {
client *Client
pretty bool
id string
version *int
versionType string
}
// NewDeleteTemplateService creates a new DeleteTemplateService.
func NewDeleteTemplateService(client *Client) *DeleteTemplateService {
return &DeleteTemplateService{
client: client,
}
}
// Id is the template ID.
func (s *DeleteTemplateService) Id(id string) *DeleteTemplateService {
s.id = id
return s
}
// Version an explicit version number for concurrency control.
func (s *DeleteTemplateService) Version(version int) *DeleteTemplateService {
s.version = &version
return s
}
// VersionType specifies a version type.
func (s *DeleteTemplateService) VersionType(versionType string) *DeleteTemplateService {
s.versionType = versionType
return s
}
// buildURL builds the URL for the operation.
func (s *DeleteTemplateService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_search/template/{id}", map[string]string{
"id": s.id,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.version != nil {
params.Set("version", fmt.Sprintf("%d", *s.version))
}
if s.versionType != "" {
params.Set("version_type", s.versionType)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *DeleteTemplateService) Validate() error {
var invalid []string
if s.id == "" {
invalid = append(invalid, "Id")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *DeleteTemplateService) Do() (*DeleteTemplateResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("DELETE", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(DeleteTemplateResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// DeleteTemplateResponse is the response of DeleteTemplateService.Do.
type DeleteTemplateResponse struct {
Found bool `json:"found"`
Index string `json:"_index"`
Type string `json:"_type"`
Id string `json:"_id"`
Version int `json:"_version"`
}

View File

@ -0,0 +1,115 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestDelete(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Count documents
count, err := client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 3 {
t.Errorf("expected Count = %d; got %d", 3, count)
}
// Delete document 1
res, err := client.Delete().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if res.Found != true {
t.Errorf("expected Found = true; got %v", res.Found)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
count, err = client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected Count = %d; got %d", 2, count)
}
// Delete non existent document 99
res, err = client.Delete().Index(testIndexName).Type("tweet").Id("99").Refresh(true).Do()
if err != nil {
t.Fatal(err)
}
if res.Found != false {
t.Errorf("expected Found = false; got %v", res.Found)
}
count, err = client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected Count = %d; got %d", 2, count)
}
}
func TestDeleteWithEmptyIDFails(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Delete document with blank ID
_, err = client.Delete().Index(testIndexName).Type("tweet").Id("").Do()
if err != ErrMissingId {
t.Fatalf("expected to not accept delete without identifier, got: %v", err)
}
// Delete document with blank type
_, err = client.Delete().Index(testIndexName).Type("").Id("1").Do()
if err != ErrMissingType {
t.Fatalf("expected to not accept delete without type, got: %v", err)
}
// Delete document with blank index
_, err = client.Delete().Index("").Type("tweet").Id("1").Do()
if err != ErrMissingIndex {
t.Fatalf("expected to not accept delete without index, got: %v", err)
}
}

View File

@ -0,0 +1,51 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
/*
Package elastic provides an interface to the Elasticsearch server
(http://www.elasticsearch.org/).
The first thing you do is to create a Client. If you have Elasticsearch
installed and running with its default settings
(i.e. available at http://127.0.0.1:9200), all you need to do is:
client, err := elastic.NewClient()
if err != nil {
// Handle error
}
If your Elasticsearch server is running on a different IP and/or port,
just provide a URL to NewClient:
// Create a client and connect to http://192.168.2.10:9201
client, err := elastic.NewClient(elastic.SetURL("http://192.168.2.10:9201"))
if err != nil {
// Handle error
}
You can pass many more configuration parameters to NewClient. Review the
documentation of NewClient for more information.
If no Elasticsearch server is available, services will fail when creating
a new request and will return ErrNoClient.
A Client provides services. The services usually come with a variety of
methods to prepare the query and a Do function to execute it against the
Elasticsearch REST interface and return a response. Here is an example
of the IndexExists service that checks if a given index already exists.
exists, err := client.IndexExists("twitter").Do()
if err != nil {
// Handle error
}
if !exists {
// Index does not exist yet.
}
Look up the documentation for Client to get an idea of the services provided
and what kinds of responses you get when executing the Do function of a service.
Also see the wiki on Github for more details.
*/
package elastic

View File

@ -0,0 +1,63 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"net/http"
)
var (
// ErrMissingIndex is returned e.g. from DeleteService if the index is missing.
ErrMissingIndex = errors.New("elastic: index is missing")
// ErrMissingType is returned e.g. from DeleteService if the type is missing.
ErrMissingType = errors.New("elastic: type is missing")
// ErrMissingId is returned e.g. from DeleteService if the document identifier is missing.
ErrMissingId = errors.New("elastic: id is missing")
)
func checkResponse(res *http.Response) error {
// 200-299 and 404 are valid status codes
if (res.StatusCode >= 200 && res.StatusCode <= 299) || res.StatusCode == http.StatusNotFound {
return nil
}
if res.Body == nil {
return fmt.Errorf("elastic: Error %d (%s)", res.StatusCode, http.StatusText(res.StatusCode))
}
slurp, err := ioutil.ReadAll(res.Body)
if err != nil {
return fmt.Errorf("elastic: Error %d (%s) when reading body: %v", res.StatusCode, http.StatusText(res.StatusCode), err)
}
errReply := new(Error)
err = json.Unmarshal(slurp, errReply)
if err != nil {
return fmt.Errorf("elastic: Error %d (%s)", res.StatusCode, http.StatusText(res.StatusCode))
}
if errReply != nil {
if errReply.Status == 0 {
errReply.Status = res.StatusCode
}
return errReply
}
return fmt.Errorf("elastic: Error %d (%s)", res.StatusCode, http.StatusText(res.StatusCode))
}
type Error struct {
Status int `json:"status"`
Message string `json:"error"`
}
func (e *Error) Error() string {
if e.Message != "" {
return fmt.Sprintf("elastic: Error %d (%s): %s", e.Status, http.StatusText(e.Status), e.Message)
} else {
return fmt.Sprintf("elastic: Error %d (%s)", e.Status, http.StatusText(e.Status))
}
}

View File

@ -0,0 +1,74 @@
package elastic
import (
"bufio"
"fmt"
"net/http"
"strings"
"testing"
)
func TestResponseError(t *testing.T) {
message := "Something went seriously wrong."
raw := "HTTP/1.1 500 Internal Server Error\r\n" +
"\r\n" +
`{"status":500,"error":"` + message + `"}` + "\r\n"
r := bufio.NewReader(strings.NewReader(raw))
resp, err := http.ReadResponse(r, nil)
if err != nil {
t.Fatal(err)
}
err = checkResponse(resp)
if err == nil {
t.Fatalf("expected error; got: %v", err)
}
// Check for correct error message
expected := fmt.Sprintf("elastic: Error %d (%s): %s", resp.StatusCode, http.StatusText(resp.StatusCode), message)
got := err.Error()
if got != expected {
t.Fatalf("expected %q; got: %q", expected, got)
}
// Check that error is of type *elastic.Error, which contains additional information
e, ok := err.(*Error)
if !ok {
t.Fatal("expected error to be of type *elastic.Error")
}
if e.Status != resp.StatusCode {
t.Fatalf("expected status code %d; got: %d", resp.StatusCode, e.Status)
}
if e.Message != message {
t.Fatalf("expected error message %q; got: %q", message, e.Message)
}
}
func TestResponseErrorHTML(t *testing.T) {
raw := "HTTP/1.1 413 Request Entity Too Large\r\n" +
"\r\n" +
`<html>
<head><title>413 Request Entity Too Large</title></head>
<body bgcolor="white">
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>nginx/1.6.2</center>
</body>
</html>` + "\r\n"
r := bufio.NewReader(strings.NewReader(raw))
resp, err := http.ReadResponse(r, nil)
if err != nil {
t.Fatal(err)
}
err = checkResponse(resp)
if err == nil {
t.Fatalf("expected error; got: %v", err)
}
// Check for correct error message
expected := fmt.Sprintf("elastic: Error %d (%s)", http.StatusRequestEntityTooLarge, http.StatusText(http.StatusRequestEntityTooLarge))
got := err.Error()
if got != expected {
t.Fatalf("expected %q; got: %q", expected, got)
}
}

View File

@ -0,0 +1,547 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic_test
import (
"encoding/json"
"fmt"
"log"
"os"
"reflect"
"time"
elastic "gopkg.in/olivere/elastic.v2"
)
type Tweet struct {
User string `json:"user"`
Message string `json:"message"`
Retweets int `json:"retweets"`
Image string `json:"image,omitempty"`
Created time.Time `json:"created,omitempty"`
Tags []string `json:"tags,omitempty"`
Location string `json:"location,omitempty"`
Suggest *elastic.SuggestField `json:"suggest_field,omitempty"`
}
func Example() {
errorlog := log.New(os.Stdout, "APP ", log.LstdFlags)
// Obtain a client. You can provide your own HTTP client here.
client, err := elastic.NewClient(elastic.SetErrorLog(errorlog))
if err != nil {
// Handle error
panic(err)
}
// Trace request and response details like this
//client.SetTracer(log.New(os.Stdout, "", 0))
// Ping the Elasticsearch server to get e.g. the version number
info, code, err := client.Ping().Do()
if err != nil {
// Handle error
panic(err)
}
fmt.Printf("Elasticsearch returned with code %d and version %s", code, info.Version.Number)
// Getting the ES version number is quite common, so there's a shortcut
esversion, err := client.ElasticsearchVersion("http://127.0.0.1:9200")
if err != nil {
// Handle error
panic(err)
}
fmt.Printf("Elasticsearch version %s", esversion)
// Use the IndexExists service to check if a specified index exists.
exists, err := client.IndexExists("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
if !exists {
// Create a new index.
createIndex, err := client.CreateIndex("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
if !createIndex.Acknowledged {
// Not acknowledged
}
}
// Index a tweet (using JSON serialization)
tweet1 := Tweet{User: "olivere", Message: "Take Five", Retweets: 0}
put1, err := client.Index().
Index("twitter").
Type("tweet").
Id("1").
BodyJson(tweet1).
Do()
if err != nil {
// Handle error
panic(err)
}
fmt.Printf("Indexed tweet %s to index %s, type %s\n", put1.Id, put1.Index, put1.Type)
// Index a second tweet (by string)
tweet2 := `{"user" : "olivere", "message" : "It's a Raggy Waltz"}`
put2, err := client.Index().
Index("twitter").
Type("tweet").
Id("2").
BodyString(tweet2).
Do()
if err != nil {
// Handle error
panic(err)
}
fmt.Printf("Indexed tweet %s to index %s, type %s\n", put2.Id, put2.Index, put2.Type)
// Get tweet with specified ID
get1, err := client.Get().
Index("twitter").
Type("tweet").
Id("1").
Do()
if err != nil {
// Handle error
panic(err)
}
if get1.Found {
fmt.Printf("Got document %s in version %d from index %s, type %s\n", get1.Id, get1.Version, get1.Index, get1.Type)
}
// Flush to make sure the documents got written.
_, err = client.Flush().Index("twitter").Do()
if err != nil {
panic(err)
}
// Search with a term query
termQuery := elastic.NewTermQuery("user", "olivere")
searchResult, err := client.Search().
Index("twitter"). // search in index "twitter"
Query(&termQuery). // specify the query
Sort("user", true). // sort by "user" field, ascending
From(0).Size(10). // take documents 0-9
Pretty(true). // pretty print request and response JSON
Do() // execute
if err != nil {
// Handle error
panic(err)
}
// searchResult is of type SearchResult and returns hits, suggestions,
// and all kinds of other information from Elasticsearch.
fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)
// Each is a convenience function that iterates over hits in a search result.
// It makes sure you don't need to check for nil values in the response.
// However, it ignores errors in serialization. If you want full control
// over iterating the hits, see below.
var ttyp Tweet
for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) {
t := item.(Tweet)
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
// TotalHits is another convenience function that works even when something goes wrong.
fmt.Printf("Found a total of %d tweets\n", searchResult.TotalHits())
// Here's how you iterate through results with full control over each step.
if searchResult.Hits != nil {
fmt.Printf("Found a total of %d tweets\n", searchResult.Hits.TotalHits)
// Iterate through results
for _, hit := range searchResult.Hits.Hits {
// hit.Index contains the name of the index
// Deserialize hit.Source into a Tweet (could also be just a map[string]interface{}).
var t Tweet
err := json.Unmarshal(*hit.Source, &t)
if err != nil {
// Deserialization failed
}
// Work with tweet
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
} else {
// No hits
fmt.Print("Found no tweets\n")
}
// Update a tweet by the update API of Elasticsearch.
// We just increment the number of retweets.
update, err := client.Update().Index("twitter").Type("tweet").Id("1").
Script("ctx._source.retweets += num").
ScriptParams(map[string]interface{}{"num": 1}).
Upsert(map[string]interface{}{"retweets": 0}).
Do()
if err != nil {
// Handle error
panic(err)
}
fmt.Printf("New version of tweet %q is now %d", update.Id, update.Version)
// ...
// Delete an index.
deleteIndex, err := client.DeleteIndex("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
if !deleteIndex.Acknowledged {
// Not acknowledged
}
}
func ExampleClient_NewClient_default() {
// Obtain a client to the Elasticsearch instance on http://localhost:9200.
client, err := elastic.NewClient()
if err != nil {
// Handle error
fmt.Printf("connection failed: %v\n", err)
} else {
fmt.Println("connected")
}
_ = client
// Output:
// connected
}
func ExampleClient_NewClient_cluster() {
// Obtain a client for an Elasticsearch cluster of two nodes,
// running on 10.0.1.1 and 10.0.1.2.
client, err := elastic.NewClient(elastic.SetURL("http://10.0.1.1:9200", "http://10.0.1.2:9200"))
if err != nil {
// Handle error
panic(err)
}
_ = client
}
func ExampleClient_NewClient_manyOptions() {
// Obtain a client for an Elasticsearch cluster of two nodes,
// running on 10.0.1.1 and 10.0.1.2. Do not run the sniffer.
// Set the healthcheck interval to 10s. When requests fail,
// retry 5 times. Print error messages to os.Stderr and informational
// messages to os.Stdout.
client, err := elastic.NewClient(
elastic.SetURL("http://10.0.1.1:9200", "http://10.0.1.2:9200"),
elastic.SetSniff(false),
elastic.SetHealthcheckInterval(10*time.Second),
elastic.SetMaxRetries(5),
elastic.SetErrorLog(log.New(os.Stderr, "ELASTIC ", log.LstdFlags)),
elastic.SetInfoLog(log.New(os.Stdout, "", log.LstdFlags)))
if err != nil {
// Handle error
panic(err)
}
_ = client
}
func ExampleIndexExistsService() {
// Get a client to the local Elasticsearch instance.
client, err := elastic.NewClient()
if err != nil {
// Handle error
panic(err)
}
// Use the IndexExists service to check if the index "twitter" exists.
exists, err := client.IndexExists("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
if exists {
// ...
}
}
func ExampleCreateIndexService() {
// Get a client to the local Elasticsearch instance.
client, err := elastic.NewClient()
if err != nil {
// Handle error
panic(err)
}
// Create a new index.
createIndex, err := client.CreateIndex("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
if !createIndex.Acknowledged {
// Not acknowledged
}
}
func ExampleDeleteIndexService() {
// Get a client to the local Elasticsearch instance.
client, err := elastic.NewClient()
if err != nil {
// Handle error
panic(err)
}
// Delete an index.
deleteIndex, err := client.DeleteIndex("twitter").Do()
if err != nil {
// Handle error
panic(err)
}
if !deleteIndex.Acknowledged {
// Not acknowledged
}
}
func ExampleSearchService() {
// Get a client to the local Elasticsearch instance.
client, err := elastic.NewClient()
if err != nil {
// Handle error
panic(err)
}
// Search with a term query
termQuery := elastic.NewTermQuery("user", "olivere")
searchResult, err := client.Search().
Index("twitter"). // search in index "twitter"
Query(&termQuery). // specify the query
Sort("user", true). // sort by "user" field, ascending
From(0).Size(10). // take documents 0-9
Pretty(true). // pretty print request and response JSON
Do() // execute
if err != nil {
// Handle error
panic(err)
}
// searchResult is of type SearchResult and returns hits, suggestions,
// and all kinds of other information from Elasticsearch.
fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)
// Number of hits
if searchResult.Hits != nil {
fmt.Printf("Found a total of %d tweets\n", searchResult.Hits.TotalHits)
// Iterate through results
for _, hit := range searchResult.Hits.Hits {
// hit.Index contains the name of the index
// Deserialize hit.Source into a Tweet (could also be just a map[string]interface{}).
var t Tweet
err := json.Unmarshal(*hit.Source, &t)
if err != nil {
// Deserialization failed
}
// Work with tweet
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
} else {
// No hits
fmt.Print("Found no tweets\n")
}
}
func ExampleAggregations() {
// Get a client to the local Elasticsearch instance.
client, err := elastic.NewClient()
if err != nil {
// Handle error
panic(err)
}
// Create an aggregation for users and a sub-aggregation for a date histogram of tweets (per year).
timeline := elastic.NewTermsAggregation().Field("user").Size(10).OrderByCountDesc()
histogram := elastic.NewDateHistogramAggregation().Field("created").Interval("year")
timeline = timeline.SubAggregation("history", histogram)
// Search with a term query
searchResult, err := client.Search().
Index("twitter"). // search in index "twitter"
Query(elastic.NewMatchAllQuery()). // return all results, but ...
SearchType("count"). // ... do not return hits, just the count
Aggregation("timeline", timeline). // add our aggregation to the query
Pretty(true). // pretty print request and response JSON
Do() // execute
if err != nil {
// Handle error
panic(err)
}
// Access "timeline" aggregate in search result.
agg, found := searchResult.Aggregations.Terms("timeline")
if !found {
log.Fatalf("we sould have a terms aggregation called %q", "timeline")
}
for _, userBucket := range agg.Buckets {
// Every bucket should have the user field as key.
user := userBucket.Key
// The sub-aggregation history should have the number of tweets per year.
histogram, found := userBucket.DateHistogram("history")
if found {
for _, year := range histogram.Buckets {
fmt.Printf("user %q has %d tweets in %q\n", user, year.DocCount, year.KeyAsString)
}
}
}
}
func ExampleSearchResult() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Do a search
searchResult, err := client.Search().Index("twitter").Query(elastic.NewMatchAllQuery()).Do()
if err != nil {
panic(err)
}
// searchResult is of type SearchResult and returns hits, suggestions,
// and all kinds of other information from Elasticsearch.
fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)
// Each is a utility function that iterates over hits in a search result.
// It makes sure you don't need to check for nil values in the response.
// However, it ignores errors in serialization. If you want full control
// over iterating the hits, see below.
var ttyp Tweet
for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) {
t := item.(Tweet)
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
fmt.Printf("Found a total of %d tweets\n", searchResult.TotalHits())
// Here's how you iterate hits with full control.
if searchResult.Hits != nil {
fmt.Printf("Found a total of %d tweets\n", searchResult.Hits.TotalHits)
// Iterate through results
for _, hit := range searchResult.Hits.Hits {
// hit.Index contains the name of the index
// Deserialize hit.Source into a Tweet (could also be just a map[string]interface{}).
var t Tweet
err := json.Unmarshal(*hit.Source, &t)
if err != nil {
// Deserialization failed
}
// Work with tweet
fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
}
} else {
// No hits
fmt.Print("Found no tweets\n")
}
}
func ExamplePutTemplateService() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Create search template
tmpl := `{"template":{"query":{"match":{"title":"{{query_string}}"}}}}`
// Create template
resp, err := client.PutTemplate().
Id("my-search-template"). // Name of the template
BodyString(tmpl). // Search template itself
Do() // Execute
if err != nil {
panic(err)
}
if resp.Created {
fmt.Println("search template created")
}
}
func ExampleGetTemplateService() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Get template stored under "my-search-template"
resp, err := client.GetTemplate().Id("my-search-template").Do()
if err != nil {
panic(err)
}
fmt.Printf("search template is: %q\n", resp.Template)
}
func ExampleDeleteTemplateService() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Delete template
resp, err := client.DeleteTemplate().Id("my-search-template").Do()
if err != nil {
panic(err)
}
if resp != nil && resp.Found {
fmt.Println("template deleted")
}
}
func ExampleClusterHealthService() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Get cluster health
res, err := client.ClusterHealth().Index("twitter").Do()
if err != nil {
panic(err)
}
if res == nil {
panic(err)
}
fmt.Printf("Cluster status is %q\n", res.Status)
}
func ExampleClusterHealthService_WaitForGreen() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Wait for status green
res, err := client.ClusterHealth().WaitForStatus("green").Timeout("15s").Do()
if err != nil {
panic(err)
}
if res.TimedOut {
fmt.Printf("time out waiting for cluster status %q\n", "green")
} else {
fmt.Printf("cluster status is %q\n", res.Status)
}
}
func ExampleClusterStateService() {
client, err := elastic.NewClient()
if err != nil {
panic(err)
}
// Get cluster state
res, err := client.ClusterState().Metric("version").Do()
if err != nil {
panic(err)
}
fmt.Printf("Cluster %q has version %d", res.ClusterName, res.Version)
}

View File

@ -0,0 +1,176 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
"net/http"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// ExistsService checks if a document exists.
//
// See http://www.elastic.co/guide/en/elasticsearch/reference/current/docs-get.html
// for details.
type ExistsService struct {
client *Client
pretty bool
id string
index string
typ string
parent string
preference string
realtime *bool
refresh *bool
routing string
}
// NewExistsService creates a new ExistsService.
func NewExistsService(client *Client) *ExistsService {
return &ExistsService{
client: client,
}
}
// Id is the document ID.
func (s *ExistsService) Id(id string) *ExistsService {
s.id = id
return s
}
// Index is the name of the index.
func (s *ExistsService) Index(index string) *ExistsService {
s.index = index
return s
}
// Type is the type of the document (use `_all` to fetch the first
// document matching the ID across all types).
func (s *ExistsService) Type(typ string) *ExistsService {
s.typ = typ
return s
}
// Parent is the ID of the parent document.
func (s *ExistsService) Parent(parent string) *ExistsService {
s.parent = parent
return s
}
// Preference specifies the node or shard the operation should be
// performed on (default: random).
func (s *ExistsService) Preference(preference string) *ExistsService {
s.preference = preference
return s
}
// Realtime specifies whether to perform the operation in realtime or search mode.
func (s *ExistsService) Realtime(realtime bool) *ExistsService {
s.realtime = &realtime
return s
}
// Refresh the shard containing the document before performing the operation.
func (s *ExistsService) Refresh(refresh bool) *ExistsService {
s.refresh = &refresh
return s
}
// Routing is the specific routing value.
func (s *ExistsService) Routing(routing string) *ExistsService {
s.routing = routing
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *ExistsService) Pretty(pretty bool) *ExistsService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *ExistsService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/{index}/{type}/{id}", map[string]string{
"id": s.id,
"index": s.index,
"type": s.typ,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.parent != "" {
params.Set("parent", s.parent)
}
if s.preference != "" {
params.Set("preference", s.preference)
}
if s.realtime != nil {
params.Set("realtime", fmt.Sprintf("%v", *s.realtime))
}
if s.refresh != nil {
params.Set("refresh", fmt.Sprintf("%v", *s.refresh))
}
if s.routing != "" {
params.Set("routing", s.routing)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *ExistsService) Validate() error {
var invalid []string
if s.id == "" {
invalid = append(invalid, "Id")
}
if s.index == "" {
invalid = append(invalid, "Index")
}
if s.typ == "" {
invalid = append(invalid, "Type")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *ExistsService) Do() (bool, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return false, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return false, err
}
// Get HTTP response
res, err := s.client.PerformRequest("HEAD", path, params, nil)
if err != nil {
return false, err
}
// Evaluate operation response
switch res.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
default:
return false, fmt.Errorf("elastic: got HTTP code %d when it should have been either 200 or 404", res.StatusCode)
}
}

View File

@ -0,0 +1,19 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import "testing"
func TestExists(t *testing.T) {
client := setupTestClientAndCreateIndexAndAddDocs(t) //, SetTraceLog(log.New(os.Stdout, "", 0)))
exists, err := client.Exists().Index(testIndexName).Type("comment").Id("1").Parent("tweet").Do()
if err != nil {
t.Fatal(err)
}
if !exists {
t.Fatal("expected document to exist")
}
}

View File

@ -0,0 +1,329 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// ExplainService computes a score explanation for a query and
// a specific document.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html.
type ExplainService struct {
client *Client
pretty bool
id string
index string
typ string
q string
routing string
lenient *bool
analyzer string
df string
fields []string
lowercaseExpandedTerms *bool
xSourceInclude []string
analyzeWildcard *bool
parent string
preference string
xSource []string
defaultOperator string
xSourceExclude []string
source string
bodyJson interface{}
bodyString string
}
// NewExplainService creates a new ExplainService.
func NewExplainService(client *Client) *ExplainService {
return &ExplainService{
client: client,
xSource: make([]string, 0),
xSourceExclude: make([]string, 0),
fields: make([]string, 0),
xSourceInclude: make([]string, 0),
}
}
// Id is the document ID.
func (s *ExplainService) Id(id string) *ExplainService {
s.id = id
return s
}
// Index is the name of the index.
func (s *ExplainService) Index(index string) *ExplainService {
s.index = index
return s
}
// Type is the type of the document.
func (s *ExplainService) Type(typ string) *ExplainService {
s.typ = typ
return s
}
// Source is the URL-encoded query definition (instead of using the request body).
func (s *ExplainService) Source(source string) *ExplainService {
s.source = source
return s
}
// XSourceExclude is a list of fields to exclude from the returned _source field.
func (s *ExplainService) XSourceExclude(xSourceExclude ...string) *ExplainService {
s.xSourceExclude = make([]string, 0)
s.xSourceExclude = append(s.xSourceExclude, xSourceExclude...)
return s
}
// Lenient specifies whether format-based query failures
// (such as providing text to a numeric field) should be ignored.
func (s *ExplainService) Lenient(lenient bool) *ExplainService {
s.lenient = &lenient
return s
}
// Query in the Lucene query string syntax.
func (s *ExplainService) Q(q string) *ExplainService {
s.q = q
return s
}
// Routing sets a specific routing value.
func (s *ExplainService) Routing(routing string) *ExplainService {
s.routing = routing
return s
}
// AnalyzeWildcard specifies whether wildcards and prefix queries
// in the query string query should be analyzed (default: false).
func (s *ExplainService) AnalyzeWildcard(analyzeWildcard bool) *ExplainService {
s.analyzeWildcard = &analyzeWildcard
return s
}
// Analyzer is the analyzer for the query string query.
func (s *ExplainService) Analyzer(analyzer string) *ExplainService {
s.analyzer = analyzer
return s
}
// Df is the default field for query string query (default: _all).
func (s *ExplainService) Df(df string) *ExplainService {
s.df = df
return s
}
// Fields is a list of fields to return in the response.
func (s *ExplainService) Fields(fields ...string) *ExplainService {
s.fields = make([]string, 0)
s.fields = append(s.fields, fields...)
return s
}
// LowercaseExpandedTerms specifies whether query terms should be lowercased.
func (s *ExplainService) LowercaseExpandedTerms(lowercaseExpandedTerms bool) *ExplainService {
s.lowercaseExpandedTerms = &lowercaseExpandedTerms
return s
}
// XSourceInclude is a list of fields to extract and return from the _source field.
func (s *ExplainService) XSourceInclude(xSourceInclude ...string) *ExplainService {
s.xSourceInclude = make([]string, 0)
s.xSourceInclude = append(s.xSourceInclude, xSourceInclude...)
return s
}
// DefaultOperator is the default operator for query string query (AND or OR).
func (s *ExplainService) DefaultOperator(defaultOperator string) *ExplainService {
s.defaultOperator = defaultOperator
return s
}
// Parent is the ID of the parent document.
func (s *ExplainService) Parent(parent string) *ExplainService {
s.parent = parent
return s
}
// Preference specifies the node or shard the operation should be performed on (default: random).
func (s *ExplainService) Preference(preference string) *ExplainService {
s.preference = preference
return s
}
// XSource is true or false to return the _source field or not, or a list of fields to return.
func (s *ExplainService) XSource(xSource ...string) *ExplainService {
s.xSource = make([]string, 0)
s.xSource = append(s.xSource, xSource...)
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *ExplainService) Pretty(pretty bool) *ExplainService {
s.pretty = pretty
return s
}
// Query sets a query definition using the Query DSL.
func (s *ExplainService) Query(query Query) *ExplainService {
body := make(map[string]interface{})
body["query"] = query.Source()
s.bodyJson = body
return s
}
// BodyJson sets the query definition using the Query DSL.
func (s *ExplainService) BodyJson(body interface{}) *ExplainService {
s.bodyJson = body
return s
}
// BodyString sets the query definition using the Query DSL as a string.
func (s *ExplainService) BodyString(body string) *ExplainService {
s.bodyString = body
return s
}
// buildURL builds the URL for the operation.
func (s *ExplainService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/{index}/{type}/{id}/_explain", map[string]string{
"id": s.id,
"index": s.index,
"type": s.typ,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if len(s.xSource) > 0 {
params.Set("_source", strings.Join(s.xSource, ","))
}
if s.defaultOperator != "" {
params.Set("default_operator", s.defaultOperator)
}
if s.parent != "" {
params.Set("parent", s.parent)
}
if s.preference != "" {
params.Set("preference", s.preference)
}
if s.source != "" {
params.Set("source", s.source)
}
if len(s.xSourceExclude) > 0 {
params.Set("_source_exclude", strings.Join(s.xSourceExclude, ","))
}
if s.lenient != nil {
params.Set("lenient", fmt.Sprintf("%v", *s.lenient))
}
if s.q != "" {
params.Set("q", s.q)
}
if s.routing != "" {
params.Set("routing", s.routing)
}
if len(s.fields) > 0 {
params.Set("fields", strings.Join(s.fields, ","))
}
if s.lowercaseExpandedTerms != nil {
params.Set("lowercase_expanded_terms", fmt.Sprintf("%v", *s.lowercaseExpandedTerms))
}
if len(s.xSourceInclude) > 0 {
params.Set("_source_include", strings.Join(s.xSourceInclude, ","))
}
if s.analyzeWildcard != nil {
params.Set("analyze_wildcard", fmt.Sprintf("%v", *s.analyzeWildcard))
}
if s.analyzer != "" {
params.Set("analyzer", s.analyzer)
}
if s.df != "" {
params.Set("df", s.df)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *ExplainService) Validate() error {
var invalid []string
if s.index == "" {
invalid = append(invalid, "Index")
}
if s.typ == "" {
invalid = append(invalid, "Type")
}
if s.id == "" {
invalid = append(invalid, "Id")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *ExplainService) Do() (*ExplainResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Setup HTTP request body
var body interface{}
if s.bodyJson != nil {
body = s.bodyJson
} else {
body = s.bodyString
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, body)
if err != nil {
return nil, err
}
// Return operation response
ret := new(ExplainResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// ExplainResponse is the response of ExplainService.Do.
type ExplainResponse struct {
Index string `json:"_index"`
Type string `json:"_type"`
Id string `json:"_id"`
Matched bool `json:"matched"`
Explanation map[string]interface{} `json:"explanation"`
}

View File

@ -0,0 +1,41 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import "testing"
func TestExplain(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
// Add a document
indexResult, err := client.Index().
Index(testIndexName).
Type("tweet").
Id("1").
BodyJson(&tweet1).
Refresh(true).
Do()
if err != nil {
t.Fatal(err)
}
if indexResult == nil {
t.Errorf("expected result to be != nil; got: %v", indexResult)
}
// Explain
query := NewTermQuery("user", "olivere")
expl, err := client.Explain(testIndexName, "tweet", "1").Query(query).Do()
if err != nil {
t.Fatal(err)
}
if expl == nil {
t.Fatal("expected to return an explanation")
}
if !expl.Matched {
t.Errorf("expected matched to be %v; got: %v", true, expl.Matched)
}
}

View File

@ -0,0 +1,74 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"net/url"
"strings"
)
type FetchSourceContext struct {
fetchSource bool
transformSource bool
includes []string
excludes []string
}
func NewFetchSourceContext(fetchSource bool) *FetchSourceContext {
return &FetchSourceContext{
fetchSource: fetchSource,
includes: make([]string, 0),
excludes: make([]string, 0),
}
}
func (fsc *FetchSourceContext) FetchSource() bool {
return fsc.fetchSource
}
func (fsc *FetchSourceContext) SetFetchSource(fetchSource bool) {
fsc.fetchSource = fetchSource
}
func (fsc *FetchSourceContext) Include(includes ...string) *FetchSourceContext {
fsc.includes = append(fsc.includes, includes...)
return fsc
}
func (fsc *FetchSourceContext) Exclude(excludes ...string) *FetchSourceContext {
fsc.excludes = append(fsc.excludes, excludes...)
return fsc
}
func (fsc *FetchSourceContext) TransformSource(transformSource bool) *FetchSourceContext {
fsc.transformSource = transformSource
return fsc
}
func (fsc *FetchSourceContext) Source() interface{} {
if !fsc.fetchSource {
return false
}
return map[string]interface{}{
"includes": fsc.includes,
"excludes": fsc.excludes,
}
}
// Query returns the parameters in a form suitable for a URL query string.
func (fsc *FetchSourceContext) Query() url.Values {
params := url.Values{}
if !fsc.fetchSource {
params.Add("_source", "false")
return params
}
if len(fsc.includes) > 0 {
params.Add("_source_include", strings.Join(fsc.includes, ","))
}
if len(fsc.excludes) > 0 {
params.Add("_source_exclude", strings.Join(fsc.excludes, ","))
}
return params
}

View File

@ -0,0 +1,92 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"testing"
)
func TestFetchSourceContextNoFetchSource(t *testing.T) {
builder := NewFetchSourceContext(false)
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `false`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestFetchSourceContextNoFetchSourceIgnoreIncludesAndExcludes(t *testing.T) {
builder := NewFetchSourceContext(false).Include("a", "b").Exclude("c")
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `false`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestFetchSourceContextFetchSource(t *testing.T) {
builder := NewFetchSourceContext(true)
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"excludes":[],"includes":[]}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestFetchSourceContextFetchSourceWithIncludesAndExcludes(t *testing.T) {
builder := NewFetchSourceContext(true).Include("a", "b").Exclude("c")
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"excludes":["c"],"includes":["a","b"]}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestFetchSourceContextQueryDefaults(t *testing.T) {
builder := NewFetchSourceContext(true)
values := builder.Query()
got := values.Encode()
expected := ""
if got != expected {
t.Errorf("expected %q; got: %q", expected, got)
}
}
func TestFetchSourceContextQueryNoFetchSource(t *testing.T) {
builder := NewFetchSourceContext(false)
values := builder.Query()
got := values.Encode()
expected := "_source=false"
if got != expected {
t.Errorf("expected %q; got: %q", expected, got)
}
}
func TestFetchSourceContextQueryFetchSourceWithIncludesAndExcludes(t *testing.T) {
builder := NewFetchSourceContext(true).Include("a", "b").Exclude("c")
values := builder.Query()
got := values.Encode()
expected := "_source_exclude=c&_source_include=a%2Cb"
if got != expected {
t.Errorf("expected %q; got: %q", expected, got)
}
}

View File

@ -0,0 +1,9 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
type Filter interface {
Source() interface{}
}

View File

@ -0,0 +1,167 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// Flush allows to flush one or more indices. The flush process of an index
// basically frees memory from the index by flushing data to the index
// storage and clearing the internal transaction log.
//
// See http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-flush.html
// for details.
type FlushService struct {
client *Client
indices []string
force *bool
full *bool
waitIfOngoing *bool
ignoreUnavailable *bool
allowNoIndices *bool
expandWildcards string
}
func NewFlushService(client *Client) *FlushService {
builder := &FlushService{
client: client,
}
return builder
}
func (s *FlushService) Index(index string) *FlushService {
if s.indices == nil {
s.indices = make([]string, 0)
}
s.indices = append(s.indices, index)
return s
}
func (s *FlushService) Indices(indices ...string) *FlushService {
if s.indices == nil {
s.indices = make([]string, 0)
}
s.indices = append(s.indices, indices...)
return s
}
// Force specifies whether to force a flush even if it is not necessary.
func (s *FlushService) Force(force bool) *FlushService {
s.force = &force
return s
}
// Full, when set to true, creates a new index writer for the index and
// refreshes all settings related to the index.
func (s *FlushService) Full(full bool) *FlushService {
s.full = &full
return s
}
// WaitIfOngoing will block until the flush can be executed (if set to true)
// if another flush operation is already executing. The default is false
// and will cause an exception to be thrown on the shard level if another
// flush operation is already running. [1.4.0.Beta1]
func (s *FlushService) WaitIfOngoing(wait bool) *FlushService {
s.waitIfOngoing = &wait
return s
}
// IgnoreUnavailable specifies whether concrete indices should be ignored
// when unavailable (e.g. missing or closed).
func (s *FlushService) IgnoreUnavailable(ignoreUnavailable bool) *FlushService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// AllowNoIndices specifies whether to ignore if a wildcard expression
// yields no indices. This includes the _all index or when no indices
// have been specified.
func (s *FlushService) AllowNoIndices(allowNoIndices bool) *FlushService {
s.allowNoIndices = &allowNoIndices
return s
}
// ExpandWildcards specifies whether to expand wildcards to concrete indices
// that are open, closed, or both. Use one of "open", "closed", "none", or "all".
func (s *FlushService) ExpandWildcards(expandWildcards string) *FlushService {
s.expandWildcards = expandWildcards
return s
}
// Do executes the service.
func (s *FlushService) Do() (*FlushResult, error) {
// Build url
path := "/"
// Indices part
if len(s.indices) > 0 {
indexPart := make([]string, 0)
for _, index := range s.indices {
index, err := uritemplates.Expand("{index}", map[string]string{
"index": index,
})
if err != nil {
return nil, err
}
indexPart = append(indexPart, index)
}
path += strings.Join(indexPart, ",") + "/"
}
path += "_flush"
// Parameters
params := make(url.Values)
if s.force != nil {
params.Set("force", fmt.Sprintf("%v", *s.force))
}
if s.full != nil {
params.Set("full", fmt.Sprintf("%v", *s.full))
}
if s.waitIfOngoing != nil {
params.Set("wait_if_ongoing", fmt.Sprintf("%v", *s.waitIfOngoing))
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
// Get response
res, err := s.client.PerformRequest("POST", path, params, nil)
if err != nil {
return nil, err
}
// Return result
ret := new(FlushResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of a flush request.
type shardsInfo struct {
Total int `json:"total"`
Successful int `json:"successful"`
Failed int `json:"failed"`
}
type FlushResult struct {
Shards shardsInfo `json:"_shards"`
}

View File

@ -0,0 +1,22 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestFlush(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
// Flush all indices
res, err := client.Flush().Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Errorf("expected res to be != nil; got: %v", res)
}
}

View File

@ -0,0 +1,47 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
"strconv"
"strings"
)
// GeoPoint is a geographic position described via latitude and longitude.
type GeoPoint struct {
Lat, Lon float64
}
// Source returns the object to be serialized in Elasticsearch DSL.
func (pt *GeoPoint) Source() map[string]float64 {
return map[string]float64{
"lat": pt.Lat,
"lon": pt.Lon,
}
}
// GeoPointFromLatLon initializes a new GeoPoint by latitude and longitude.
func GeoPointFromLatLon(lat, lon float64) *GeoPoint {
return &GeoPoint{Lat: lat, Lon: lon}
}
// GeoPointFromString initializes a new GeoPoint by a string that is
// formatted as "{latitude},{longitude}", e.g. "40.10210,-70.12091".
func GeoPointFromString(latLon string) (*GeoPoint, error) {
latlon := strings.SplitN(latLon, ",", 2)
if len(latlon) != 2 {
return nil, fmt.Errorf("elastic: %s is not a valid geo point string", latLon)
}
lat, err := strconv.ParseFloat(latlon[0], 64)
if err != nil {
return nil, err
}
lon, err := strconv.ParseFloat(latlon[1], 64)
if err != nil {
return nil, err
}
return &GeoPoint{Lat: lat, Lon: lon}, nil
}

View File

@ -0,0 +1,24 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"testing"
)
func TestGeoPointSource(t *testing.T) {
pt := GeoPoint{Lat: 40, Lon: -70}
data, err := json.Marshal(pt.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"lat":40,"lon":-70}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}

View File

@ -0,0 +1,223 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type GetService struct {
client *Client
index string
typ string
id string
routing string
preference string
fields []string
refresh *bool
realtime *bool
fsc *FetchSourceContext
versionType string
version *int64
ignoreErrorsOnGeneratedFields *bool
}
func NewGetService(client *Client) *GetService {
builder := &GetService{
client: client,
typ: "_all",
}
return builder
}
func (b *GetService) String() string {
return fmt.Sprintf("[%v][%v][%v]: routing [%v]",
b.index,
b.typ,
b.id,
b.routing)
}
func (b *GetService) Index(index string) *GetService {
b.index = index
return b
}
func (b *GetService) Type(typ string) *GetService {
b.typ = typ
return b
}
func (b *GetService) Id(id string) *GetService {
b.id = id
return b
}
func (b *GetService) Parent(parent string) *GetService {
if b.routing == "" {
b.routing = parent
}
return b
}
func (b *GetService) Routing(routing string) *GetService {
b.routing = routing
return b
}
func (b *GetService) Preference(preference string) *GetService {
b.preference = preference
return b
}
func (b *GetService) Fields(fields ...string) *GetService {
if b.fields == nil {
b.fields = make([]string, 0)
}
b.fields = append(b.fields, fields...)
return b
}
func (s *GetService) FetchSource(fetchSource bool) *GetService {
if s.fsc == nil {
s.fsc = NewFetchSourceContext(fetchSource)
} else {
s.fsc.SetFetchSource(fetchSource)
}
return s
}
func (s *GetService) FetchSourceContext(fetchSourceContext *FetchSourceContext) *GetService {
s.fsc = fetchSourceContext
return s
}
func (b *GetService) Refresh(refresh bool) *GetService {
b.refresh = &refresh
return b
}
func (b *GetService) Realtime(realtime bool) *GetService {
b.realtime = &realtime
return b
}
func (b *GetService) VersionType(versionType string) *GetService {
b.versionType = versionType
return b
}
func (b *GetService) Version(version int64) *GetService {
b.version = &version
return b
}
func (b *GetService) IgnoreErrorsOnGeneratedFields(ignore bool) *GetService {
b.ignoreErrorsOnGeneratedFields = &ignore
return b
}
// Validate checks if the operation is valid.
func (s *GetService) Validate() error {
var invalid []string
if s.id == "" {
invalid = append(invalid, "Id")
}
if s.index == "" {
invalid = append(invalid, "Index")
}
if s.typ == "" {
invalid = append(invalid, "Type")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
func (b *GetService) Do() (*GetResult, error) {
// Check pre-conditions
if err := b.Validate(); err != nil {
return nil, err
}
// Build url
path, err := uritemplates.Expand("/{index}/{type}/{id}", map[string]string{
"index": b.index,
"type": b.typ,
"id": b.id,
})
if err != nil {
return nil, err
}
params := make(url.Values)
if b.realtime != nil {
params.Add("realtime", fmt.Sprintf("%v", *b.realtime))
}
if len(b.fields) > 0 {
params.Add("fields", strings.Join(b.fields, ","))
}
if b.routing != "" {
params.Add("routing", b.routing)
}
if b.preference != "" {
params.Add("preference", b.preference)
}
if b.refresh != nil {
params.Add("refresh", fmt.Sprintf("%v", *b.refresh))
}
if b.realtime != nil {
params.Add("realtime", fmt.Sprintf("%v", *b.realtime))
}
if b.ignoreErrorsOnGeneratedFields != nil {
params.Add("ignore_errors_on_generated_fields", fmt.Sprintf("%v", *b.ignoreErrorsOnGeneratedFields))
}
if len(b.fields) > 0 {
params.Add("_fields", strings.Join(b.fields, ","))
}
if b.version != nil {
params.Add("version", fmt.Sprintf("%d", *b.version))
}
if b.versionType != "" {
params.Add("version_type", b.versionType)
}
if b.fsc != nil {
for k, values := range b.fsc.Query() {
params.Add(k, strings.Join(values, ","))
}
}
// Get response
res, err := b.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return result
ret := new(GetResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of a get request.
type GetResult struct {
Index string `json:"_index"`
Type string `json:"_type"`
Id string `json:"_id"`
Version int64 `json:"_version,omitempty"`
Source *json.RawMessage `json:"_source,omitempty"`
Found bool `json:"found,omitempty"`
Fields map[string]interface{} `json:"fields,omitempty"`
Error string `json:"error,omitempty"` // used only in MultiGet
}

View File

@ -0,0 +1,172 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// GetMappingService retrieves the mapping definitions for an index or
// index/type. See at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-get-mapping.html.
type GetMappingService struct {
client *Client
pretty bool
index []string
typ []string
local *bool
ignoreUnavailable *bool
allowNoIndices *bool
expandWildcards string
}
// NewGetMappingService creates a new GetMappingService.
func NewGetMappingService(client *Client) *GetMappingService {
return &GetMappingService{
client: client,
index: make([]string, 0),
typ: make([]string, 0),
}
}
// Index is a list of index names.
func (s *GetMappingService) Index(index ...string) *GetMappingService {
s.index = append(s.index, index...)
return s
}
// Type is a list of document types.
func (s *GetMappingService) Type(typ ...string) *GetMappingService {
s.typ = append(s.typ, typ...)
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard indices
// expression resolves into no concrete indices.
// This includes `_all` string or when no indices have been specified.
func (s *GetMappingService) AllowNoIndices(allowNoIndices bool) *GetMappingService {
s.allowNoIndices = &allowNoIndices
return s
}
// ExpandWildcards indicates whether to expand wildcard expression to
// concrete indices that are open, closed or both..
func (s *GetMappingService) ExpandWildcards(expandWildcards string) *GetMappingService {
s.expandWildcards = expandWildcards
return s
}
// Local indicates whether to return local information, do not retrieve
// the state from master node (default: false).
func (s *GetMappingService) Local(local bool) *GetMappingService {
s.local = &local
return s
}
// IgnoreUnavailable indicates whether specified concrete indices should be
// ignored when unavailable (missing or closed).
func (s *GetMappingService) IgnoreUnavailable(ignoreUnavailable bool) *GetMappingService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *GetMappingService) Pretty(pretty bool) *GetMappingService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *GetMappingService) buildURL() (string, url.Values, error) {
var index, typ []string
if len(s.index) > 0 {
index = s.index
} else {
index = []string{"_all"}
}
if len(s.typ) > 0 {
typ = s.typ
} else {
typ = []string{"_all"}
}
// Build URL
path, err := uritemplates.Expand("/{index}/_mapping/{type}", map[string]string{
"index": strings.Join(index, ","),
"type": strings.Join(typ, ","),
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *GetMappingService) Validate() error {
return nil
}
// Do executes the operation. When successful, it returns a json.RawMessage.
// If you specify an index, Elasticsearch returns HTTP status 404.
// if you specify a type that does not exist, Elasticsearch returns
// an empty map.
func (s *GetMappingService) Do() (map[string]interface{}, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
var ret map[string]interface{}
if err := json.Unmarshal(res.Body, &ret); err != nil {
return nil, err
}
return ret, nil
}

View File

@ -0,0 +1,50 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestGetMappingURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Indices []string
Types []string
Expected string
}{
{
[]string{},
[]string{},
"/_all/_mapping/_all",
},
{
[]string{},
[]string{"tweet"},
"/_all/_mapping/tweet",
},
{
[]string{"twitter"},
[]string{"tweet"},
"/twitter/_mapping/tweet",
},
{
[]string{"store-1", "store-2"},
[]string{"tweet", "user"},
"/store-1%2Cstore-2/_mapping/tweet%2Cuser",
},
}
for _, test := range tests {
path, _, err := client.GetMapping().Index(test.Indices...).Type(test.Types...).buildURL()
if err != nil {
t.Fatal(err)
}
if path != test.Expected {
t.Errorf("expected %q; got: %q", test.Expected, path)
}
}
}

View File

@ -0,0 +1,113 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// GetTemplateService reads a search template.
// It is documented at http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-template.html.
type GetTemplateService struct {
client *Client
pretty bool
id string
version interface{}
versionType string
}
// NewGetTemplateService creates a new GetTemplateService.
func NewGetTemplateService(client *Client) *GetTemplateService {
return &GetTemplateService{
client: client,
}
}
// Id is the template ID.
func (s *GetTemplateService) Id(id string) *GetTemplateService {
s.id = id
return s
}
// Version is an explicit version number for concurrency control.
func (s *GetTemplateService) Version(version interface{}) *GetTemplateService {
s.version = version
return s
}
// VersionType is a specific version type.
func (s *GetTemplateService) VersionType(versionType string) *GetTemplateService {
s.versionType = versionType
return s
}
// buildURL builds the URL for the operation.
func (s *GetTemplateService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_search/template/{id}", map[string]string{
"id": s.id,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.version != nil {
params.Set("version", fmt.Sprintf("%v", s.version))
}
if s.versionType != "" {
params.Set("version_type", s.versionType)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *GetTemplateService) Validate() error {
var invalid []string
if s.id == "" {
invalid = append(invalid, "Id")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation and returns the template.
func (s *GetTemplateService) Do() (*GetTemplateResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return result
ret := new(GetTemplateResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
type GetTemplateResponse struct {
Template string `json:"template"`
}

View File

@ -0,0 +1,51 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestGetPutDeleteTemplate(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
// This is a search template, not an index template!
tmpl := `{
"template": {
"query" : { "term" : { "{{my_field}}" : "{{my_value}}" } },
"size" : "{{my_size}}"
},
"params":{
"my_field" : "user",
"my_value" : "olivere",
"my_size" : 5
}
}`
putres, err := client.PutTemplate().Id("elastic-template").BodyString(tmpl).Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if putres == nil {
t.Fatalf("expected response; got: %v", putres)
}
if !putres.Created {
t.Fatalf("expected template to be created; got: %v", putres.Created)
}
// Always delete template
defer client.DeleteTemplate().Id("elastic-template").Do()
// Get template
getres, err := client.GetTemplate().Id("elastic-template").Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if getres == nil {
t.Fatalf("expected response; got: %v", getres)
}
if getres.Template == "" {
t.Errorf("expected template %q; got: %q", tmpl, getres.Template)
}
}

View File

@ -0,0 +1,168 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"testing"
)
func TestGet(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
// Get document 1
res, err := client.Get().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if res.Found != true {
t.Errorf("expected Found = true; got %v", res.Found)
}
if res.Source == nil {
t.Errorf("expected Source != nil; got %v", res.Source)
}
// Get non existent document 99
res, err = client.Get().Index(testIndexName).Type("tweet").Id("99").Do()
if err != nil {
t.Fatal(err)
}
if res.Found != false {
t.Errorf("expected Found = false; got %v", res.Found)
}
if res.Source != nil {
t.Errorf("expected Source == nil; got %v", res.Source)
}
}
func TestGetWithSourceFiltering(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
// Get document 1, without source
res, err := client.Get().Index(testIndexName).Type("tweet").Id("1").FetchSource(false).Do()
if err != nil {
t.Fatal(err)
}
if res.Found != true {
t.Errorf("expected Found = true; got %v", res.Found)
}
if res.Source != nil {
t.Errorf("expected Source == nil; got %v", res.Source)
}
// Get document 1, exclude Message field
fsc := NewFetchSourceContext(true).Exclude("message")
res, err = client.Get().Index(testIndexName).Type("tweet").Id("1").FetchSourceContext(fsc).Do()
if err != nil {
t.Fatal(err)
}
if res.Found != true {
t.Errorf("expected Found = true; got %v", res.Found)
}
if res.Source == nil {
t.Errorf("expected Source != nil; got %v", res.Source)
}
var tw tweet
err = json.Unmarshal(*res.Source, &tw)
if err != nil {
t.Fatal(err)
}
if tw.User != "olivere" {
t.Errorf("expected user %q; got: %q", "olivere", tw.User)
}
if tw.Message != "" {
t.Errorf("expected message %q; got: %q", "", tw.Message)
}
}
func TestGetWithFields(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").Timestamp("12345").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
// Get document 1, specifying fields
res, err := client.Get().Index(testIndexName).Type("tweet").Id("1").Fields("message", "_timestamp").Do()
if err != nil {
t.Fatal(err)
}
if res.Found != true {
t.Errorf("expected Found = true; got %v", res.Found)
}
timestamp, ok := res.Fields["_timestamp"].(float64)
if !ok {
t.Fatalf("Cannot retrieve \"_timestamp\" field from document")
}
if timestamp != 12345 {
t.Fatalf("Expected timestamp %v; got %v", 12345, timestamp)
}
messageField, ok := res.Fields["message"]
if !ok {
t.Fatalf("Cannot retrieve \"message\" field from document")
}
// Depending on the version of elasticsearch the message field will be returned
// as a string or a slice of strings. This test works in both cases.
messageString, ok := messageField.(string)
if !ok {
messageArray, ok := messageField.([]interface{})
if ok {
messageString, ok = messageArray[0].(string)
}
if !ok {
t.Fatalf("\"message\" field should be a string or a slice of strings")
}
}
if messageString != tweet1.Message {
t.Errorf("Expected message %s; got %s", tweet1.Message, messageString)
}
}
func TestGetFailsWithMissingParams(t *testing.T) {
// Mitigate against http://stackoverflow.com/questions/27491738/elasticsearch-go-index-failures-no-feature-for-name
client := setupTestClientAndCreateIndex(t)
if _, err := client.Get().Do(); err == nil {
t.Fatal("expected Get to fail")
}
if _, err := client.Get().Index(testIndexName).Do(); err == nil {
t.Fatal("expected Get to fail")
}
if _, err := client.Get().Type("tweet").Do(); err == nil {
t.Fatal("expected Get to fail")
}
if _, err := client.Get().Id("1").Do(); err == nil {
t.Fatal("expected Get to fail")
}
if _, err := client.Get().Index(testIndexName).Type("tweet").Do(); err == nil {
t.Fatal("expected Get to fail")
}
/*
if _, err := client.Get().Index(testIndexName).Id("1").Do(); err == nil {
t.Fatal("expected Get to fail")
}
*/
if _, err := client.Get().Type("tweet").Id("1").Do(); err == nil {
t.Fatal("expected Get to fail")
}
}

View File

@ -0,0 +1,496 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
// Highlight allows highlighting search results on one or more fields.
// For details, see:
// http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-highlighting.html
type Highlight struct {
fields []*HighlighterField
tagsSchema *string
highlightFilter *bool
fragmentSize *int
numOfFragments *int
preTags []string
postTags []string
order *string
encoder *string
requireFieldMatch *bool
boundaryMaxScan *int
boundaryChars []rune
highlighterType *string
fragmenter *string
highlightQuery Query
noMatchSize *int
phraseLimit *int
options map[string]interface{}
forceSource *bool
useExplicitFieldOrder bool
}
func NewHighlight() *Highlight {
hl := &Highlight{
fields: make([]*HighlighterField, 0),
preTags: make([]string, 0),
postTags: make([]string, 0),
boundaryChars: make([]rune, 0),
options: make(map[string]interface{}),
}
return hl
}
func (hl *Highlight) Fields(fields ...*HighlighterField) *Highlight {
hl.fields = append(hl.fields, fields...)
return hl
}
func (hl *Highlight) Field(name string) *Highlight {
field := NewHighlighterField(name)
hl.fields = append(hl.fields, field)
return hl
}
func (hl *Highlight) TagsSchema(schemaName string) *Highlight {
hl.tagsSchema = &schemaName
return hl
}
func (hl *Highlight) HighlightFilter(highlightFilter bool) *Highlight {
hl.highlightFilter = &highlightFilter
return hl
}
func (hl *Highlight) FragmentSize(fragmentSize int) *Highlight {
hl.fragmentSize = &fragmentSize
return hl
}
func (hl *Highlight) NumOfFragments(numOfFragments int) *Highlight {
hl.numOfFragments = &numOfFragments
return hl
}
func (hl *Highlight) Encoder(encoder string) *Highlight {
hl.encoder = &encoder
return hl
}
func (hl *Highlight) PreTags(preTags ...string) *Highlight {
hl.preTags = make([]string, 0)
hl.preTags = append(hl.preTags, preTags...)
return hl
}
func (hl *Highlight) PostTags(postTags ...string) *Highlight {
hl.postTags = make([]string, 0)
hl.postTags = append(hl.postTags, postTags...)
return hl
}
func (hl *Highlight) Order(order string) *Highlight {
hl.order = &order
return hl
}
func (hl *Highlight) RequireFieldMatch(requireFieldMatch bool) *Highlight {
hl.requireFieldMatch = &requireFieldMatch
return hl
}
func (hl *Highlight) BoundaryMaxScan(boundaryMaxScan int) *Highlight {
hl.boundaryMaxScan = &boundaryMaxScan
return hl
}
func (hl *Highlight) BoundaryChars(boundaryChars ...rune) *Highlight {
hl.boundaryChars = make([]rune, 0)
hl.boundaryChars = append(hl.boundaryChars, boundaryChars...)
return hl
}
func (hl *Highlight) HighlighterType(highlighterType string) *Highlight {
hl.highlighterType = &highlighterType
return hl
}
func (hl *Highlight) Fragmenter(fragmenter string) *Highlight {
hl.fragmenter = &fragmenter
return hl
}
func (hl *Highlight) HighlighQuery(highlightQuery Query) *Highlight {
hl.highlightQuery = highlightQuery
return hl
}
func (hl *Highlight) NoMatchSize(noMatchSize int) *Highlight {
hl.noMatchSize = &noMatchSize
return hl
}
func (hl *Highlight) Options(options map[string]interface{}) *Highlight {
hl.options = options
return hl
}
func (hl *Highlight) ForceSource(forceSource bool) *Highlight {
hl.forceSource = &forceSource
return hl
}
func (hl *Highlight) UseExplicitFieldOrder(useExplicitFieldOrder bool) *Highlight {
hl.useExplicitFieldOrder = useExplicitFieldOrder
return hl
}
// Creates the query source for the bool query.
func (hl *Highlight) Source() interface{} {
// Returns the map inside of "highlight":
// "highlight":{
// ... this ...
// }
source := make(map[string]interface{})
if hl.tagsSchema != nil {
source["tags_schema"] = *hl.tagsSchema
}
if hl.preTags != nil && len(hl.preTags) > 0 {
source["pre_tags"] = hl.preTags
}
if hl.postTags != nil && len(hl.postTags) > 0 {
source["post_tags"] = hl.postTags
}
if hl.order != nil {
source["order"] = *hl.order
}
if hl.highlightFilter != nil {
source["highlight_filter"] = *hl.highlightFilter
}
if hl.fragmentSize != nil {
source["fragment_size"] = *hl.fragmentSize
}
if hl.numOfFragments != nil {
source["number_of_fragments"] = *hl.numOfFragments
}
if hl.encoder != nil {
source["encoder"] = *hl.encoder
}
if hl.requireFieldMatch != nil {
source["require_field_match"] = *hl.requireFieldMatch
}
if hl.boundaryMaxScan != nil {
source["boundary_max_scan"] = *hl.boundaryMaxScan
}
if hl.boundaryChars != nil && len(hl.boundaryChars) > 0 {
source["boundary_chars"] = hl.boundaryChars
}
if hl.highlighterType != nil {
source["type"] = *hl.highlighterType
}
if hl.fragmenter != nil {
source["fragmenter"] = *hl.fragmenter
}
if hl.highlightQuery != nil {
source["highlight_query"] = hl.highlightQuery.Source()
}
if hl.noMatchSize != nil {
source["no_match_size"] = *hl.noMatchSize
}
if hl.phraseLimit != nil {
source["phrase_limit"] = *hl.phraseLimit
}
if hl.options != nil && len(hl.options) > 0 {
source["options"] = hl.options
}
if hl.forceSource != nil {
source["force_source"] = *hl.forceSource
}
if hl.fields != nil && len(hl.fields) > 0 {
if hl.useExplicitFieldOrder {
// Use a slice for the fields
fields := make([]map[string]interface{}, 0)
for _, field := range hl.fields {
fmap := make(map[string]interface{})
fmap[field.Name] = field.Source()
fields = append(fields, fmap)
}
source["fields"] = fields
} else {
// Use a map for the fields
fields := make(map[string]interface{}, 0)
for _, field := range hl.fields {
fields[field.Name] = field.Source()
}
source["fields"] = fields
}
}
return source
/*
highlightS := make(map[string]interface{})
if hl.tagsSchema != "" {
highlightS["tags_schema"] = hl.tagsSchema
}
if len(hl.preTags) > 0 {
highlightS["pre_tags"] = hl.preTags
}
if len(hl.postTags) > 0 {
highlightS["post_tags"] = hl.postTags
}
if hl.order != "" {
highlightS["order"] = hl.order
}
if hl.encoder != "" {
highlightS["encoder"] = hl.encoder
}
if hl.requireFieldMatch != nil {
highlightS["require_field_match"] = *hl.requireFieldMatch
}
if hl.highlighterType != "" {
highlightS["type"] = hl.highlighterType
}
if hl.fragmenter != "" {
highlightS["fragmenter"] = hl.fragmenter
}
if hl.highlightQuery != nil {
highlightS["highlight_query"] = hl.highlightQuery.Source()
}
if hl.noMatchSize != nil {
highlightS["no_match_size"] = *hl.noMatchSize
}
if len(hl.options) > 0 {
highlightS["options"] = hl.options
}
if hl.forceSource != nil {
highlightS["force_source"] = *hl.forceSource
}
if len(hl.fields) > 0 {
fieldsS := make(map[string]interface{})
for _, field := range hl.fields {
fieldsS[field.Name] = field.Source()
}
highlightS["fields"] = fieldsS
}
return highlightS
*/
}
// HighlighterField specifies a highlighted field.
type HighlighterField struct {
Name string
preTags []string
postTags []string
fragmentSize int
fragmentOffset int
numOfFragments int
highlightFilter *bool
order *string
requireFieldMatch *bool
boundaryMaxScan int
boundaryChars []rune
highlighterType *string
fragmenter *string
highlightQuery Query
noMatchSize *int
matchedFields []string
phraseLimit *int
options map[string]interface{}
forceSource *bool
/*
Name string
preTags []string
postTags []string
fragmentSize int
numOfFragments int
fragmentOffset int
highlightFilter *bool
order string
requireFieldMatch *bool
boundaryMaxScan int
boundaryChars []rune
highlighterType string
fragmenter string
highlightQuery Query
noMatchSize *int
matchedFields []string
options map[string]interface{}
forceSource *bool
*/
}
func NewHighlighterField(name string) *HighlighterField {
return &HighlighterField{
Name: name,
preTags: make([]string, 0),
postTags: make([]string, 0),
fragmentSize: -1,
fragmentOffset: -1,
numOfFragments: -1,
boundaryMaxScan: -1,
boundaryChars: make([]rune, 0),
matchedFields: make([]string, 0),
options: make(map[string]interface{}),
}
}
func (f *HighlighterField) PreTags(preTags ...string) *HighlighterField {
f.preTags = make([]string, 0)
f.preTags = append(f.preTags, preTags...)
return f
}
func (f *HighlighterField) PostTags(postTags ...string) *HighlighterField {
f.postTags = make([]string, 0)
f.postTags = append(f.postTags, postTags...)
return f
}
func (f *HighlighterField) FragmentSize(fragmentSize int) *HighlighterField {
f.fragmentSize = fragmentSize
return f
}
func (f *HighlighterField) FragmentOffset(fragmentOffset int) *HighlighterField {
f.fragmentOffset = fragmentOffset
return f
}
func (f *HighlighterField) NumOfFragments(numOfFragments int) *HighlighterField {
f.numOfFragments = numOfFragments
return f
}
func (f *HighlighterField) HighlightFilter(highlightFilter bool) *HighlighterField {
f.highlightFilter = &highlightFilter
return f
}
func (f *HighlighterField) Order(order string) *HighlighterField {
f.order = &order
return f
}
func (f *HighlighterField) RequireFieldMatch(requireFieldMatch bool) *HighlighterField {
f.requireFieldMatch = &requireFieldMatch
return f
}
func (f *HighlighterField) BoundaryMaxScan(boundaryMaxScan int) *HighlighterField {
f.boundaryMaxScan = boundaryMaxScan
return f
}
func (f *HighlighterField) BoundaryChars(boundaryChars ...rune) *HighlighterField {
f.boundaryChars = make([]rune, 0)
f.boundaryChars = append(f.boundaryChars, boundaryChars...)
return f
}
func (f *HighlighterField) HighlighterType(highlighterType string) *HighlighterField {
f.highlighterType = &highlighterType
return f
}
func (f *HighlighterField) Fragmenter(fragmenter string) *HighlighterField {
f.fragmenter = &fragmenter
return f
}
func (f *HighlighterField) HighlightQuery(highlightQuery Query) *HighlighterField {
f.highlightQuery = highlightQuery
return f
}
func (f *HighlighterField) NoMatchSize(noMatchSize int) *HighlighterField {
f.noMatchSize = &noMatchSize
return f
}
func (f *HighlighterField) Options(options map[string]interface{}) *HighlighterField {
f.options = options
return f
}
func (f *HighlighterField) MatchedFields(matchedFields ...string) *HighlighterField {
f.matchedFields = make([]string, 0)
f.matchedFields = append(f.matchedFields, matchedFields...)
return f
}
func (f *HighlighterField) PhraseLimit(phraseLimit int) *HighlighterField {
f.phraseLimit = &phraseLimit
return f
}
func (f *HighlighterField) ForceSource(forceSource bool) *HighlighterField {
f.forceSource = &forceSource
return f
}
func (f *HighlighterField) Source() interface{} {
source := make(map[string]interface{})
if f.preTags != nil && len(f.preTags) > 0 {
source["pre_tags"] = f.preTags
}
if f.postTags != nil && len(f.postTags) > 0 {
source["post_tags"] = f.postTags
}
if f.fragmentSize != -1 {
source["fragment_size"] = f.fragmentSize
}
if f.numOfFragments != -1 {
source["number_of_fragments"] = f.numOfFragments
}
if f.fragmentOffset != -1 {
source["fragment_offset"] = f.fragmentOffset
}
if f.highlightFilter != nil {
source["highlight_filter"] = *f.highlightFilter
}
if f.order != nil {
source["order"] = *f.order
}
if f.requireFieldMatch != nil {
source["require_field_match"] = *f.requireFieldMatch
}
if f.boundaryMaxScan != -1 {
source["boundary_max_scan"] = f.boundaryMaxScan
}
if f.boundaryChars != nil && len(f.boundaryChars) > 0 {
source["boundary_chars"] = f.boundaryChars
}
if f.highlighterType != nil {
source["type"] = *f.highlighterType
}
if f.fragmenter != nil {
source["fragmenter"] = *f.fragmenter
}
if f.highlightQuery != nil {
source["highlight_query"] = f.highlightQuery.Source()
}
if f.noMatchSize != nil {
source["no_match_size"] = *f.noMatchSize
}
if f.matchedFields != nil && len(f.matchedFields) > 0 {
source["matched_fields"] = f.matchedFields
}
if f.phraseLimit != nil {
source["phrase_limit"] = *f.phraseLimit
}
if f.options != nil && len(f.options) > 0 {
source["options"] = f.options
}
if f.forceSource != nil {
source["force_source"] = *f.forceSource
}
return source
}

View File

@ -0,0 +1,168 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
_ "net/http"
"testing"
)
func TestHighlighterField(t *testing.T) {
field := NewHighlighterField("grade")
data, err := json.Marshal(field.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestHighlighterFieldWithOptions(t *testing.T) {
field := NewHighlighterField("grade").FragmentSize(2).NumOfFragments(1)
data, err := json.Marshal(field.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"fragment_size":2,"number_of_fragments":1}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestHighlightWithStringField(t *testing.T) {
builder := NewHighlight().Field("grade")
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"fields":{"grade":{}}}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestHighlightWithFields(t *testing.T) {
gradeField := NewHighlighterField("grade")
builder := NewHighlight().Fields(gradeField)
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"fields":{"grade":{}}}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestHighlightWithMultipleFields(t *testing.T) {
gradeField := NewHighlighterField("grade")
colorField := NewHighlighterField("color")
builder := NewHighlight().Fields(gradeField, colorField)
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"fields":{"color":{},"grade":{}}}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestHighlighterWithExplicitFieldOrder(t *testing.T) {
gradeField := NewHighlighterField("grade").FragmentSize(2)
colorField := NewHighlighterField("color").FragmentSize(2).NumOfFragments(1)
builder := NewHighlight().Fields(gradeField, colorField).UseExplicitFieldOrder(true)
data, err := json.Marshal(builder.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"fields":[{"grade":{"fragment_size":2}},{"color":{"fragment_size":2,"number_of_fragments":1}}]}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestHighlightWithTermQuery(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun to do."}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Specify highlighter
hl := NewHighlight()
hl = hl.Fields(NewHighlighterField("message"))
hl = hl.PreTags("<em>").PostTags("</em>")
// Match all should return all documents
query := NewPrefixQuery("message", "golang")
searchResult, err := client.Search().
Index(testIndexName).
Highlight(hl).
Query(&query).
Do()
if err != nil {
t.Fatal(err)
}
if searchResult.Hits == nil {
t.Fatalf("expected SearchResult.Hits != nil; got nil")
}
if searchResult.Hits.TotalHits != 1 {
t.Fatalf("expected SearchResult.Hits.TotalHits = %d; got %d", 1, searchResult.Hits.TotalHits)
}
if len(searchResult.Hits.Hits) != 1 {
t.Fatalf("expected len(SearchResult.Hits.Hits) = %d; got %d", 1, len(searchResult.Hits.Hits))
}
hit := searchResult.Hits.Hits[0]
var tw tweet
if err := json.Unmarshal(*hit.Source, &tw); err != nil {
t.Fatal(err)
}
if hit.Highlight == nil || len(hit.Highlight) == 0 {
t.Fatal("expected hit to have a highlight; got nil")
}
if hl, found := hit.Highlight["message"]; found {
if len(hl) != 1 {
t.Fatalf("expected to have one highlight for field \"message\"; got %d", len(hl))
}
expected := "Welcome to <em>Golang</em> and Elasticsearch."
if hl[0] != expected {
t.Errorf("expected to have highlight \"%s\"; got \"%s\"", expected, hl[0])
}
} else {
t.Fatal("expected to have a highlight on field \"message\"; got none")
}
}

View File

@ -0,0 +1,217 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndexResult is the result of indexing a document in Elasticsearch.
type IndexResult struct {
Index string `json:"_index"`
Type string `json:"_type"`
Id string `json:"_id"`
Version int `json:"_version"`
Created bool `json:"created"`
}
// IndexService adds documents to Elasticsearch.
type IndexService struct {
client *Client
index string
_type string
id string
routing string
parent string
opType string
refresh *bool
version *int64
versionType string
timestamp string
ttl string
timeout string
bodyString string
bodyJson interface{}
pretty bool
}
func NewIndexService(client *Client) *IndexService {
builder := &IndexService{
client: client,
}
return builder
}
func (b *IndexService) Index(name string) *IndexService {
b.index = name
return b
}
func (b *IndexService) Type(_type string) *IndexService {
b._type = _type
return b
}
func (b *IndexService) Id(id string) *IndexService {
b.id = id
return b
}
func (b *IndexService) Routing(routing string) *IndexService {
b.routing = routing
return b
}
func (b *IndexService) Parent(parent string) *IndexService {
b.parent = parent
return b
}
// OpType is either "create" or "index" (the default).
func (b *IndexService) OpType(opType string) *IndexService {
b.opType = opType
return b
}
func (b *IndexService) Refresh(refresh bool) *IndexService {
b.refresh = &refresh
return b
}
func (b *IndexService) Version(version int64) *IndexService {
b.version = &version
return b
}
// VersionType is either "internal" (default), "external",
// "external_gt", "external_gte", or "force".
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#_version_types
// for details.
func (b *IndexService) VersionType(versionType string) *IndexService {
b.versionType = versionType
return b
}
func (b *IndexService) Timestamp(timestamp string) *IndexService {
b.timestamp = timestamp
return b
}
func (b *IndexService) TTL(ttl string) *IndexService {
b.ttl = ttl
return b
}
func (b *IndexService) Timeout(timeout string) *IndexService {
b.timeout = timeout
return b
}
func (b *IndexService) BodyString(body string) *IndexService {
b.bodyString = body
return b
}
func (b *IndexService) BodyJson(json interface{}) *IndexService {
b.bodyJson = json
return b
}
func (b *IndexService) Pretty(pretty bool) *IndexService {
b.pretty = pretty
return b
}
func (b *IndexService) Do() (*IndexResult, error) {
// Build url
var path, method string
if b.id != "" {
// Create document with manual id
method = "PUT"
path = "/{index}/{type}/{id}"
} else {
// Automatic ID generation
// See: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation
method = "POST"
path = "/{index}/{type}/"
}
path, err := uritemplates.Expand(path, map[string]string{
"index": b.index,
"type": b._type,
"id": b.id,
})
if err != nil {
return nil, err
}
// Parameters
params := make(url.Values)
if b.pretty {
params.Set("pretty", "true")
}
if b.routing != "" {
params.Set("routing", b.routing)
}
if b.parent != "" {
params.Set("parent", b.parent)
}
if b.opType != "" {
params.Set("op_type", b.opType)
}
if b.refresh != nil && *b.refresh {
params.Set("refresh", "true")
}
if b.version != nil {
params.Set("version", fmt.Sprintf("%d", *b.version))
}
if b.versionType != "" {
params.Set("version_type", b.versionType)
}
if b.timestamp != "" {
params.Set("timestamp", b.timestamp)
}
if b.ttl != "" {
params.Set("ttl", b.ttl)
}
if b.timeout != "" {
params.Set("timeout", b.timeout)
}
/*
routing string
parent string
opType string
refresh *bool
version *int64
versionType string
timestamp string
ttl string
*/
// Body
var body interface{}
if b.bodyJson != nil {
body = b.bodyJson
} else {
body = b.bodyString
}
// Get response
res, err := b.client.PerformRequest(method, path, params, body)
if err != nil {
return nil, err
}
// Return result
ret := new(IndexResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}

View File

@ -0,0 +1,145 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// CloseIndexService closes an index.
// See documentation at http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/indices-open-close.html.
type CloseIndexService struct {
client *Client
pretty bool
index string
ignoreUnavailable *bool
allowNoIndices *bool
expandWildcards string
timeout string
masterTimeout string
}
// NewCloseIndexService creates a new CloseIndexService.
func NewCloseIndexService(client *Client) *CloseIndexService {
return &CloseIndexService{client: client}
}
// Index is the name of the index.
func (s *CloseIndexService) Index(index string) *CloseIndexService {
s.index = index
return s
}
// Timeout is an explicit operation timeout.
func (s *CloseIndexService) Timeout(timeout string) *CloseIndexService {
s.timeout = timeout
return s
}
// MasterTimeout specifies the timeout for connection to master.
func (s *CloseIndexService) MasterTimeout(masterTimeout string) *CloseIndexService {
s.masterTimeout = masterTimeout
return s
}
// IgnoreUnavailable indicates whether specified concrete indices should be
// ignored when unavailable (missing or closed).
func (s *CloseIndexService) IgnoreUnavailable(ignoreUnavailable bool) *CloseIndexService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard indices
// expression resolves into no concrete indices. (This includes `_all` string or when no indices have been specified).
func (s *CloseIndexService) AllowNoIndices(allowNoIndices bool) *CloseIndexService {
s.allowNoIndices = &allowNoIndices
return s
}
// ExpandWildcards indicates whether to expand wildcard expression to
// concrete indices that are open, closed or both.
func (s *CloseIndexService) ExpandWildcards(expandWildcards string) *CloseIndexService {
s.expandWildcards = expandWildcards
return s
}
// buildURL builds the URL for the operation.
func (s *CloseIndexService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/{index}/_close", map[string]string{
"index": s.index,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *CloseIndexService) Validate() error {
var invalid []string
if s.index == "" {
invalid = append(invalid, "Index")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *CloseIndexService) Do() (*CloseIndexResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("POST", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(CloseIndexResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// CloseIndexResponse is the response of CloseIndexService.Do.
type CloseIndexResponse struct {
Acknowledged bool `json:"acknowledged"`
}

View File

@ -0,0 +1,50 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type IndexExistsService struct {
client *Client
index string
}
func NewIndexExistsService(client *Client) *IndexExistsService {
builder := &IndexExistsService{
client: client,
}
return builder
}
func (b *IndexExistsService) Index(index string) *IndexExistsService {
b.index = index
return b
}
func (b *IndexExistsService) Do() (bool, error) {
// Build url
path, err := uritemplates.Expand("/{index}", map[string]string{
"index": b.index,
})
if err != nil {
return false, err
}
// Get response
res, err := b.client.PerformRequest("HEAD", path, nil, nil)
if err != nil {
return false, err
}
if res.StatusCode == 200 {
return true, nil
} else if res.StatusCode == 404 {
return false, nil
}
return false, fmt.Errorf("elastic: got HTTP code %d when it should have been either 200 or 404", res.StatusCode)
}

View File

@ -0,0 +1,186 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// IndicesGetService retrieves information about one or more indices.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-get-index.html.
type IndicesGetService struct {
client *Client
pretty bool
index []string
feature []string
expandWildcards string
local *bool
ignoreUnavailable *bool
allowNoIndices *bool
}
// NewIndicesGetService creates a new IndicesGetService.
func NewIndicesGetService(client *Client) *IndicesGetService {
return &IndicesGetService{
client: client,
index: make([]string, 0),
feature: make([]string, 0),
}
}
// Index is a list of index names. Use _all to retrieve information about
// all indices of a cluster.
func (s *IndicesGetService) Index(index ...string) *IndicesGetService {
s.index = append(s.index, index...)
return s
}
// Feature is a list of features (e.g. _settings,_mappings,_warmers, and _aliases).
func (s *IndicesGetService) Feature(feature ...string) *IndicesGetService {
s.feature = append(s.feature, feature...)
return s
}
// ExpandWildcards indicates whether wildcard expressions should
// get expanded to open or closed indices (default: open).
func (s *IndicesGetService) ExpandWildcards(expandWildcards string) *IndicesGetService {
s.expandWildcards = expandWildcards
return s
}
// Local indicates whether to return local information (do not retrieve
// the state from master node (default: false)).
func (s *IndicesGetService) Local(local bool) *IndicesGetService {
s.local = &local
return s
}
// IgnoreUnavailable indicates whether to ignore unavailable indexes (default: false).
func (s *IndicesGetService) IgnoreUnavailable(ignoreUnavailable bool) *IndicesGetService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard expression
// resolves to no concrete indices (default: false).
func (s *IndicesGetService) AllowNoIndices(allowNoIndices bool) *IndicesGetService {
s.allowNoIndices = &allowNoIndices
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesGetService) Pretty(pretty bool) *IndicesGetService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesGetService) buildURL() (string, url.Values, error) {
var err error
var path string
var index []string
if len(s.index) > 0 {
index = s.index
} else {
index = []string{"_all"}
}
if len(s.feature) > 0 {
// Build URL
path, err = uritemplates.Expand("/{index}/{feature}", map[string]string{
"index": strings.Join(index, ","),
"feature": strings.Join(s.feature, ","),
})
} else {
// Build URL
path, err = uritemplates.Expand("/{index}", map[string]string{
"index": strings.Join(index, ","),
})
}
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesGetService) Validate() error {
var invalid []string
if len(s.index) == 0 {
invalid = append(invalid, "Index")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *IndicesGetService) Do() (map[string]*IndicesGetResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
var ret map[string]*IndicesGetResponse
if err := json.Unmarshal(res.Body, &ret); err != nil {
return nil, err
}
return ret, nil
}
// IndicesGetResponse is part of the response of IndicesGetService.Do.
type IndicesGetResponse struct {
Aliases map[string]interface{} `json:"aliases"`
Mappings map[string]interface{} `json:"mappings"`
Settings map[string]interface{} `json:"settings"`
Warmers map[string]interface{} `json:"warmers"`
}

View File

@ -0,0 +1,189 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// IndicesGetSettingsService allows to retrieve settings of one
// or more indices.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/indices-get-settings.html.
type IndicesGetSettingsService struct {
client *Client
pretty bool
index []string
name []string
ignoreUnavailable *bool
allowNoIndices *bool
expandWildcards string
flatSettings *bool
local *bool
}
// NewIndicesGetSettingsService creates a new IndicesGetSettingsService.
func NewIndicesGetSettingsService(client *Client) *IndicesGetSettingsService {
return &IndicesGetSettingsService{
client: client,
index: make([]string, 0),
name: make([]string, 0),
}
}
// Index is a list of index names; use `_all` or empty string to perform the operation on all indices.
func (s *IndicesGetSettingsService) Index(index ...string) *IndicesGetSettingsService {
s.index = append(s.index, index...)
return s
}
// Name are the names of the settings that should be included.
func (s *IndicesGetSettingsService) Name(name ...string) *IndicesGetSettingsService {
s.name = append(s.name, name...)
return s
}
// IgnoreUnavailable indicates whether specified concrete indices should
// be ignored when unavailable (missing or closed).
func (s *IndicesGetSettingsService) IgnoreUnavailable(ignoreUnavailable bool) *IndicesGetSettingsService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard indices
// expression resolves into no concrete indices.
// (This includes `_all` string or when no indices have been specified).
func (s *IndicesGetSettingsService) AllowNoIndices(allowNoIndices bool) *IndicesGetSettingsService {
s.allowNoIndices = &allowNoIndices
return s
}
// ExpandWildcards indicates whether to expand wildcard expression
// to concrete indices that are open, closed or both.
// Options: open, closed, none, all. Default: open,closed.
func (s *IndicesGetSettingsService) ExpandWildcards(expandWildcards string) *IndicesGetSettingsService {
s.expandWildcards = expandWildcards
return s
}
// FlatSettings indicates whether to return settings in flat format (default: false).
func (s *IndicesGetSettingsService) FlatSettings(flatSettings bool) *IndicesGetSettingsService {
s.flatSettings = &flatSettings
return s
}
// Local indicates whether to return local information, do not retrieve
// the state from master node (default: false).
func (s *IndicesGetSettingsService) Local(local bool) *IndicesGetSettingsService {
s.local = &local
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesGetSettingsService) Pretty(pretty bool) *IndicesGetSettingsService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesGetSettingsService) buildURL() (string, url.Values, error) {
var err error
var path string
var index []string
if len(s.index) > 0 {
index = s.index
} else {
index = []string{"_all"}
}
if len(s.name) > 0 {
// Build URL
path, err = uritemplates.Expand("/{index}/_settings/{name}", map[string]string{
"index": strings.Join(index, ","),
"name": strings.Join(s.name, ","),
})
} else {
// Build URL
path, err = uritemplates.Expand("/{index}/_settings", map[string]string{
"index": strings.Join(index, ","),
})
}
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
if s.flatSettings != nil {
params.Set("flat_settings", fmt.Sprintf("%v", *s.flatSettings))
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesGetSettingsService) Validate() error {
return nil
}
// Do executes the operation.
func (s *IndicesGetSettingsService) Do() (map[string]*IndicesGetSettingsResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
var ret map[string]*IndicesGetSettingsResponse
if err := json.Unmarshal(res.Body, &ret); err != nil {
return nil, err
}
return ret, nil
}
// IndicesGetSettingsResponse is the response of IndicesGetSettingsService.Do.
type IndicesGetSettingsResponse struct {
Settings map[string]interface{} `json:"settings"`
}

View File

@ -0,0 +1,81 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestIndexGetSettingsURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Indices []string
Names []string
Expected string
}{
{
[]string{},
[]string{},
"/_all/_settings",
},
{
[]string{},
[]string{"index.merge.*"},
"/_all/_settings/index.merge.%2A",
},
{
[]string{"twitter-*"},
[]string{"index.merge.*", "_settings"},
"/twitter-%2A/_settings/index.merge.%2A%2C_settings",
},
{
[]string{"store-1", "store-2"},
[]string{"index.merge.*", "_settings"},
"/store-1%2Cstore-2/_settings/index.merge.%2A%2C_settings",
},
}
for _, test := range tests {
path, _, err := client.IndexGetSettings().Index(test.Indices...).Name(test.Names...).buildURL()
if err != nil {
t.Fatal(err)
}
if path != test.Expected {
t.Errorf("expected %q; got: %q", test.Expected, path)
}
}
}
func TestIndexGetSettingsService(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
esversion, err := client.ElasticsearchVersion(DefaultURL)
if err != nil {
t.Fatal(err)
}
if esversion < "1.4.0" {
t.Skip("Index Get API is available since 1.4")
return
}
res, err := client.IndexGetSettings().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatalf("expected result; got: %v", res)
}
info, found := res[testIndexName]
if !found {
t.Fatalf("expected index %q to be found; got: %v", testIndexName, found)
}
if info == nil {
t.Fatalf("expected index %q to be != nil; got: %v", testIndexName, info)
}
if info.Settings == nil {
t.Fatalf("expected index settings of %q to be != nil; got: %v", testIndexName, info.Settings)
}
}

View File

@ -0,0 +1,84 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestIndexGetURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Indices []string
Features []string
Expected string
}{
{
[]string{},
[]string{},
"/_all",
},
{
[]string{},
[]string{"_mappings"},
"/_all/_mappings",
},
{
[]string{"twitter"},
[]string{"_mappings", "_settings"},
"/twitter/_mappings%2C_settings",
},
{
[]string{"store-1", "store-2"},
[]string{"_mappings", "_settings"},
"/store-1%2Cstore-2/_mappings%2C_settings",
},
}
for _, test := range tests {
path, _, err := client.IndexGet().Index(test.Indices...).Feature(test.Features...).buildURL()
if err != nil {
t.Fatal(err)
}
if path != test.Expected {
t.Errorf("expected %q; got: %q", test.Expected, path)
}
}
}
func TestIndexGetService(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
esversion, err := client.ElasticsearchVersion(DefaultURL)
if err != nil {
t.Fatal(err)
}
if esversion < "1.4.0" {
t.Skip("Index Get API is available since 1.4")
return
}
res, err := client.IndexGet().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatalf("expected result; got: %v", res)
}
info, found := res[testIndexName]
if !found {
t.Fatalf("expected index %q to be found; got: %v", testIndexName, found)
}
if info == nil {
t.Fatalf("expected index %q to be != nil; got: %v", testIndexName, info)
}
if info.Mappings == nil {
t.Errorf("expected mappings to be != nil; got: %v", info.Mappings)
}
if info.Settings == nil {
t.Errorf("expected settings to be != nil; got: %v", info.Settings)
}
}

View File

@ -0,0 +1,146 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// OpenIndexService opens an index.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/indices-open-close.html.
type OpenIndexService struct {
client *Client
pretty bool
index string
expandWildcards string
timeout string
masterTimeout string
ignoreUnavailable *bool
allowNoIndices *bool
}
// NewOpenIndexService creates a new OpenIndexService.
func NewOpenIndexService(client *Client) *OpenIndexService {
return &OpenIndexService{client: client}
}
// Index is the name of the index to open.
func (s *OpenIndexService) Index(index string) *OpenIndexService {
s.index = index
return s
}
// Timeout is an explicit operation timeout.
func (s *OpenIndexService) Timeout(timeout string) *OpenIndexService {
s.timeout = timeout
return s
}
// MasterTimeout specifies the timeout for connection to master.
func (s *OpenIndexService) MasterTimeout(masterTimeout string) *OpenIndexService {
s.masterTimeout = masterTimeout
return s
}
// IgnoreUnavailable indicates whether specified concrete indices should
// be ignored when unavailable (missing or closed).
func (s *OpenIndexService) IgnoreUnavailable(ignoreUnavailable bool) *OpenIndexService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard indices
// expression resolves into no concrete indices.
// (This includes `_all` string or when no indices have been specified).
func (s *OpenIndexService) AllowNoIndices(allowNoIndices bool) *OpenIndexService {
s.allowNoIndices = &allowNoIndices
return s
}
// ExpandWildcards indicates whether to expand wildcard expression to
// concrete indices that are open, closed or both..
func (s *OpenIndexService) ExpandWildcards(expandWildcards string) *OpenIndexService {
s.expandWildcards = expandWildcards
return s
}
// buildURL builds the URL for the operation.
func (s *OpenIndexService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/{index}/_open", map[string]string{
"index": s.index,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *OpenIndexService) Validate() error {
var invalid []string
if s.index == "" {
invalid = append(invalid, "Index")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *OpenIndexService) Do() (*OpenIndexResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("POST", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(OpenIndexResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// OpenIndexResponse is the response of OpenIndexService.Do.
type OpenIndexResponse struct {
Acknowledged bool `json:"acknowledged"`
}

View File

@ -0,0 +1,552 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"os"
"testing"
"time"
)
const (
testIndexName = "elastic-test"
testIndexName2 = "elastic-test2"
testMapping = `
{
"settings":{
"number_of_shards":1,
"number_of_replicas":0
},
"mappings":{
"_default_": {
"_timestamp": {
"enabled": true,
"store": "yes"
},
"_ttl": {
"enabled": true,
"store": "yes"
}
},
"tweet":{
"properties":{
"tags":{
"type":"string"
},
"location":{
"type":"geo_point"
},
"suggest_field":{
"type":"completion",
"payloads":true
}
}
},
"comment":{
"_parent": {
"type": "tweet"
}
}
}
}
`
)
type tweet struct {
User string `json:"user"`
Message string `json:"message"`
Retweets int `json:"retweets"`
Image string `json:"image,omitempty"`
Created time.Time `json:"created,omitempty"`
Tags []string `json:"tags,omitempty"`
Location string `json:"location,omitempty"`
Suggest *SuggestField `json:"suggest_field,omitempty"`
}
func (t tweet) String() string {
return fmt.Sprintf("tweet{User:%q,Message:%q,Retweets:%d}", t.User, t.Message, t.Retweets)
}
type comment struct {
User string `json:"user"`
Comment string `json:"comment"`
Created time.Time `json:"created,omitempty"`
}
func (c comment) String() string {
return fmt.Sprintf("comment{User:%q,Comment:%q}", c.User, c.Comment)
}
func isTravis() bool {
return os.Getenv("TRAVIS") != ""
}
func travisGoVersion() string {
return os.Getenv("TRAVIS_GO_VERSION")
}
type logger interface {
Error(args ...interface{})
Errorf(format string, args ...interface{})
Fatal(args ...interface{})
Fatalf(format string, args ...interface{})
Fail()
FailNow()
Log(args ...interface{})
Logf(format string, args ...interface{})
}
func setupTestClient(t logger, options ...ClientOptionFunc) (client *Client) {
var err error
client, err = NewClient(options...)
if err != nil {
t.Fatal(err)
}
client.DeleteIndex(testIndexName).Do()
client.DeleteIndex(testIndexName2).Do()
return client
}
func setupTestClientAndCreateIndex(t logger, options ...ClientOptionFunc) *Client {
client := setupTestClient(t, options...)
// Create index
createIndex, err := client.CreateIndex(testIndexName).Body(testMapping).Do()
if err != nil {
t.Fatal(err)
}
if createIndex == nil {
t.Errorf("expected result to be != nil; got: %v", createIndex)
}
// Create second index
createIndex2, err := client.CreateIndex(testIndexName2).Body(testMapping).Do()
if err != nil {
t.Fatal(err)
}
if createIndex2 == nil {
t.Errorf("expected result to be != nil; got: %v", createIndex2)
}
return client
}
func setupTestClientAndCreateIndexAndLog(t logger, options ...ClientOptionFunc) *Client {
return setupTestClientAndCreateIndex(t, SetTraceLog(log.New(os.Stdout, "", 0)))
}
func setupTestClientAndCreateIndexAndAddDocs(t logger, options ...ClientOptionFunc) *Client {
client := setupTestClientAndCreateIndex(t, options...)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
comment1 := comment{User: "nico", Comment: "You bet."}
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").Routing("someroutingkey").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("comment").Id("1").Parent("3").BodyJson(&comment1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
return client
}
func TestIndexLifecycle(t *testing.T) {
client := setupTestClient(t)
// Create index
createIndex, err := client.CreateIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !createIndex.Acknowledged {
t.Errorf("expected CreateIndexResult.Acknowledged %v; got %v", true, createIndex.Acknowledged)
}
// Check if index exists
indexExists, err := client.IndexExists(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !indexExists {
t.Fatalf("index %s should exist, but doesn't\n", testIndexName)
}
// Delete index
deleteIndex, err := client.DeleteIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !deleteIndex.Acknowledged {
t.Errorf("expected DeleteIndexResult.Acknowledged %v; got %v", true, deleteIndex.Acknowledged)
}
// Check if index exists
indexExists, err = client.IndexExists(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if indexExists {
t.Fatalf("index %s should not exist, but does\n", testIndexName)
}
}
func TestIndexExistScenarios(t *testing.T) {
client := setupTestClient(t)
// Should return false if index does not exist
indexExists, err := client.IndexExists(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if indexExists {
t.Fatalf("expected index exists to return %v, got %v", false, indexExists)
}
// Create index
createIndex, err := client.CreateIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !createIndex.Acknowledged {
t.Errorf("expected CreateIndexResult.Ack %v; got %v", true, createIndex.Acknowledged)
}
// Should return true if index does not exist
indexExists, err = client.IndexExists(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !indexExists {
t.Fatalf("expected index exists to return %v, got %v", true, indexExists)
}
}
// TODO(oe): Find out why this test fails on Travis CI.
/*
func TestIndexOpenAndClose(t *testing.T) {
client := setupTestClient(t)
// Create index
createIndex, err := client.CreateIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !createIndex.Acknowledged {
t.Errorf("expected CreateIndexResult.Acknowledged %v; got %v", true, createIndex.Acknowledged)
}
defer func() {
// Delete index
deleteIndex, err := client.DeleteIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !deleteIndex.Acknowledged {
t.Errorf("expected DeleteIndexResult.Acknowledged %v; got %v", true, deleteIndex.Acknowledged)
}
}()
waitForYellow := func() {
// Wait for status yellow
res, err := client.ClusterHealth().WaitForStatus("yellow").Timeout("15s").Do()
if err != nil {
t.Fatal(err)
}
if res != nil && res.TimedOut {
t.Fatalf("cluster time out waiting for status %q", "yellow")
}
}
// Wait for cluster
waitForYellow()
// Close index
cresp, err := client.CloseIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !cresp.Acknowledged {
t.Fatalf("expected close index of %q to be acknowledged\n", testIndexName)
}
// Wait for cluster
waitForYellow()
// Open index again
oresp, err := client.OpenIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !oresp.Acknowledged {
t.Fatalf("expected open index of %q to be acknowledged\n", testIndexName)
}
}
*/
func TestDocumentLifecycle(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
// Add a document
indexResult, err := client.Index().
Index(testIndexName).
Type("tweet").
Id("1").
BodyJson(&tweet1).
Do()
if err != nil {
t.Fatal(err)
}
if indexResult == nil {
t.Errorf("expected result to be != nil; got: %v", indexResult)
}
// Exists
exists, err := client.Exists().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if !exists {
t.Errorf("expected exists %v; got %v", true, exists)
}
// Get document
getResult, err := client.Get().
Index(testIndexName).
Type("tweet").
Id("1").
Do()
if err != nil {
t.Fatal(err)
}
if getResult.Index != testIndexName {
t.Errorf("expected GetResult.Index %q; got %q", testIndexName, getResult.Index)
}
if getResult.Type != "tweet" {
t.Errorf("expected GetResult.Type %q; got %q", "tweet", getResult.Type)
}
if getResult.Id != "1" {
t.Errorf("expected GetResult.Id %q; got %q", "1", getResult.Id)
}
if getResult.Source == nil {
t.Errorf("expected GetResult.Source to be != nil; got nil")
}
// Decode the Source field
var tweetGot tweet
err = json.Unmarshal(*getResult.Source, &tweetGot)
if err != nil {
t.Fatal(err)
}
if tweetGot.User != tweet1.User {
t.Errorf("expected Tweet.User to be %q; got %q", tweet1.User, tweetGot.User)
}
if tweetGot.Message != tweet1.Message {
t.Errorf("expected Tweet.Message to be %q; got %q", tweet1.Message, tweetGot.Message)
}
// Delete document again
deleteResult, err := client.Delete().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if deleteResult == nil {
t.Errorf("expected result to be != nil; got: %v", deleteResult)
}
// Exists
exists, err = client.Exists().Index(testIndexName).Type("tweet").Id("1").Do()
if err != nil {
t.Fatal(err)
}
if exists {
t.Errorf("expected exists %v; got %v", false, exists)
}
}
func TestDocumentLifecycleWithAutomaticIDGeneration(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
// Add a document
indexResult, err := client.Index().
Index(testIndexName).
Type("tweet").
BodyJson(&tweet1).
Do()
if err != nil {
t.Fatal(err)
}
if indexResult == nil {
t.Errorf("expected result to be != nil; got: %v", indexResult)
}
if indexResult.Id == "" {
t.Fatalf("expected Es to generate an automatic ID, got: %v", indexResult.Id)
}
id := indexResult.Id
// Exists
exists, err := client.Exists().Index(testIndexName).Type("tweet").Id(id).Do()
if err != nil {
t.Fatal(err)
}
if !exists {
t.Errorf("expected exists %v; got %v", true, exists)
}
// Get document
getResult, err := client.Get().
Index(testIndexName).
Type("tweet").
Id(id).
Do()
if err != nil {
t.Fatal(err)
}
if getResult.Index != testIndexName {
t.Errorf("expected GetResult.Index %q; got %q", testIndexName, getResult.Index)
}
if getResult.Type != "tweet" {
t.Errorf("expected GetResult.Type %q; got %q", "tweet", getResult.Type)
}
if getResult.Id != id {
t.Errorf("expected GetResult.Id %q; got %q", id, getResult.Id)
}
if getResult.Source == nil {
t.Errorf("expected GetResult.Source to be != nil; got nil")
}
// Decode the Source field
var tweetGot tweet
err = json.Unmarshal(*getResult.Source, &tweetGot)
if err != nil {
t.Fatal(err)
}
if tweetGot.User != tweet1.User {
t.Errorf("expected Tweet.User to be %q; got %q", tweet1.User, tweetGot.User)
}
if tweetGot.Message != tweet1.Message {
t.Errorf("expected Tweet.Message to be %q; got %q", tweet1.Message, tweetGot.Message)
}
// Delete document again
deleteResult, err := client.Delete().Index(testIndexName).Type("tweet").Id(id).Do()
if err != nil {
t.Fatal(err)
}
if deleteResult == nil {
t.Errorf("expected result to be != nil; got: %v", deleteResult)
}
// Exists
exists, err = client.Exists().Index(testIndexName).Type("tweet").Id(id).Do()
if err != nil {
t.Fatal(err)
}
if exists {
t.Errorf("expected exists %v; got %v", false, exists)
}
}
func TestIndexCreateExistsOpenCloseDelete(t *testing.T) {
// TODO: Find out how to make these test robust
t.Skip("test fails regularly with 409 (Conflict): " +
"IndexPrimaryShardNotAllocatedException[[elastic-test] " +
"primary not allocated post api... skipping")
client := setupTestClient(t)
// Create index
createIndex, err := client.CreateIndex(testIndexName).Body(testMapping).Do()
if err != nil {
t.Fatal(err)
}
if createIndex == nil {
t.Fatalf("expected response; got: %v", createIndex)
}
if !createIndex.Acknowledged {
t.Errorf("expected ack for creating index; got: %v", createIndex.Acknowledged)
}
// Exists
indexExists, err := client.IndexExists(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !indexExists {
t.Fatalf("expected index exists=%v; got %v", true, indexExists)
}
// Flush
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Close index
closeIndex, err := client.CloseIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if closeIndex == nil {
t.Fatalf("expected response; got: %v", closeIndex)
}
if !closeIndex.Acknowledged {
t.Errorf("expected ack for closing index; got: %v", closeIndex.Acknowledged)
}
// Open index
openIndex, err := client.OpenIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if openIndex == nil {
t.Fatalf("expected response; got: %v", openIndex)
}
if !openIndex.Acknowledged {
t.Errorf("expected ack for opening index; got: %v", openIndex.Acknowledged)
}
// Flush
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Delete index
deleteIndex, err := client.DeleteIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if deleteIndex == nil {
t.Fatalf("expected response; got: %v", deleteIndex)
}
if !deleteIndex.Acknowledged {
t.Errorf("expected ack for deleting index; got %v", deleteIndex.Acknowledged)
}
}

View File

@ -0,0 +1,122 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndicesDeleteTemplateService deletes index templates.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/indices-templates.html.
type IndicesDeleteTemplateService struct {
client *Client
pretty bool
name string
timeout string
masterTimeout string
}
// NewIndicesDeleteTemplateService creates a new IndicesDeleteTemplateService.
func NewIndicesDeleteTemplateService(client *Client) *IndicesDeleteTemplateService {
return &IndicesDeleteTemplateService{
client: client,
}
}
// Name is the name of the template.
func (s *IndicesDeleteTemplateService) Name(name string) *IndicesDeleteTemplateService {
s.name = name
return s
}
// Timeout is an explicit operation timeout.
func (s *IndicesDeleteTemplateService) Timeout(timeout string) *IndicesDeleteTemplateService {
s.timeout = timeout
return s
}
// MasterTimeout specifies the timeout for connection to master.
func (s *IndicesDeleteTemplateService) MasterTimeout(masterTimeout string) *IndicesDeleteTemplateService {
s.masterTimeout = masterTimeout
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesDeleteTemplateService) Pretty(pretty bool) *IndicesDeleteTemplateService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesDeleteTemplateService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_template/{name}", map[string]string{
"name": s.name,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesDeleteTemplateService) Validate() error {
var invalid []string
if s.name == "" {
invalid = append(invalid, "Name")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *IndicesDeleteTemplateService) Do() (*IndicesDeleteTemplateResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("DELETE", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(IndicesDeleteTemplateResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// IndicesDeleteTemplateResponse is the response of IndicesDeleteTemplateService.Do.
type IndicesDeleteTemplateResponse struct {
Acknowledged bool `json:"acknowledged,omitempty"`
}

View File

@ -0,0 +1,107 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndicesExistsTemplateService checks if a given template exists.
// See http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html#indices-templates-exists
// for documentation.
type IndicesExistsTemplateService struct {
client *Client
pretty bool
name string
local *bool
}
// NewIndicesExistsTemplateService creates a new IndicesExistsTemplateService.
func NewIndicesExistsTemplateService(client *Client) *IndicesExistsTemplateService {
return &IndicesExistsTemplateService{
client: client,
}
}
// Name is the name of the template.
func (s *IndicesExistsTemplateService) Name(name string) *IndicesExistsTemplateService {
s.name = name
return s
}
// Local indicates whether to return local information, i.e. do not retrieve
// the state from master node (default: false).
func (s *IndicesExistsTemplateService) Local(local bool) *IndicesExistsTemplateService {
s.local = &local
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesExistsTemplateService) Pretty(pretty bool) *IndicesExistsTemplateService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesExistsTemplateService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_template/{name}", map[string]string{
"name": s.name,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesExistsTemplateService) Validate() error {
var invalid []string
if s.name == "" {
invalid = append(invalid, "Name")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *IndicesExistsTemplateService) Do() (bool, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return false, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return false, err
}
// Get HTTP response
res, err := s.client.PerformRequest("HEAD", path, params, nil)
if err != nil {
return false, err
}
if res.StatusCode == 200 {
return true, nil
} else if res.StatusCode == 404 {
return false, nil
}
return false, fmt.Errorf("elastic: got HTTP code %d when it should have been either 200 or 404", res.StatusCode)
}

View File

@ -0,0 +1,68 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestIndexExistsTemplate(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tmpl := `{
"template":"elastic-test*",
"settings":{
"number_of_shards":1,
"number_of_replicas":0
},
"mappings":{
"tweet":{
"properties":{
"tags":{
"type":"string"
},
"location":{
"type":"geo_point"
},
"suggest_field":{
"type":"completion",
"payloads":true
}
}
}
}
}`
putres, err := client.IndexPutTemplate("elastic-template").BodyString(tmpl).Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if putres == nil {
t.Fatalf("expected response; got: %v", putres)
}
if !putres.Acknowledged {
t.Fatalf("expected index template to be ack'd; got: %v", putres.Acknowledged)
}
// Always delete template
defer client.IndexDeleteTemplate("elastic-template").Do()
// Check if template exists
exists, err := client.IndexTemplateExists("elastic-template").Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if !exists {
t.Fatalf("expected index template %q to exist; got: %v", "elastic-template", exists)
}
// Get template
getres, err := client.IndexGetTemplate("elastic-template").Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if getres == nil {
t.Fatalf("expected to get index template %q; got: %v", "elastic-template", getres)
}
}

View File

@ -0,0 +1,155 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndicesExistsTypeService checks if one or more types exist in one or more indices.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-types-exists.html.
type IndicesExistsTypeService struct {
client *Client
pretty bool
index []string
typ []string
allowNoIndices *bool
expandWildcards string
local *bool
ignoreUnavailable *bool
}
// NewIndicesExistsTypeService creates a new IndicesExistsTypeService.
func NewIndicesExistsTypeService(client *Client) *IndicesExistsTypeService {
return &IndicesExistsTypeService{
client: client,
index: make([]string, 0),
typ: make([]string, 0),
}
}
// Index is a list of index names; use `_all` to check the types across all indices.
func (s *IndicesExistsTypeService) Index(index ...string) *IndicesExistsTypeService {
s.index = append(s.index, index...)
return s
}
// Type is a list of document types to check.
func (s *IndicesExistsTypeService) Type(typ ...string) *IndicesExistsTypeService {
s.typ = append(s.typ, typ...)
return s
}
// IgnoreUnavailable indicates whether specified concrete indices should be
// ignored when unavailable (missing or closed).
func (s *IndicesExistsTypeService) IgnoreUnavailable(ignoreUnavailable bool) *IndicesExistsTypeService {
s.ignoreUnavailable = &ignoreUnavailable
return s
}
// AllowNoIndices indicates whether to ignore if a wildcard indices
// expression resolves into no concrete indices.
// (This includes `_all` string or when no indices have been specified).
func (s *IndicesExistsTypeService) AllowNoIndices(allowNoIndices bool) *IndicesExistsTypeService {
s.allowNoIndices = &allowNoIndices
return s
}
// ExpandWildcards indicates whether to expand wildcard expression to
// concrete indices that are open, closed or both.
func (s *IndicesExistsTypeService) ExpandWildcards(expandWildcards string) *IndicesExistsTypeService {
s.expandWildcards = expandWildcards
return s
}
// Local specifies whether to return local information, i.e. do not retrieve
// the state from master node (default: false).
func (s *IndicesExistsTypeService) Local(local bool) *IndicesExistsTypeService {
s.local = &local
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesExistsTypeService) Pretty(pretty bool) *IndicesExistsTypeService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesExistsTypeService) buildURL() (string, url.Values, error) {
if err := s.Validate(); err != nil {
return "", url.Values{}, err
}
// Build URL
path, err := uritemplates.Expand("/{index}/{type}", map[string]string{
"type": strings.Join(s.typ, ","),
"index": strings.Join(s.index, ","),
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.expandWildcards != "" {
params.Set("expand_wildcards", s.expandWildcards)
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
if s.ignoreUnavailable != nil {
params.Set("ignore_unavailable", fmt.Sprintf("%v", *s.ignoreUnavailable))
}
if s.allowNoIndices != nil {
params.Set("allow_no_indices", fmt.Sprintf("%v", *s.allowNoIndices))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesExistsTypeService) Validate() error {
var invalid []string
if len(s.index) == 0 {
invalid = append(invalid, "Index")
}
if len(s.typ) == 0 {
invalid = append(invalid, "Type")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *IndicesExistsTypeService) Do() (bool, error) {
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return false, err
}
// Get HTTP response
res, err := s.client.PerformRequest("HEAD", path, params, nil)
if err != nil {
return false, err
}
// Return operation response
if res.StatusCode == 200 {
return true, nil
} else if res.StatusCode == 404 {
return false, nil
}
return false, fmt.Errorf("elastic: got HTTP code %d when it should have been either 200 or 404", res.StatusCode)
}

View File

@ -0,0 +1,121 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestTypeExistsBuildURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Indices []string
Types []string
Expected string
ExpectValidateFailure bool
}{
{
[]string{},
[]string{},
"",
true,
},
{
[]string{"index1"},
[]string{},
"",
true,
},
{
[]string{},
[]string{"type1"},
"",
true,
},
{
[]string{"index1"},
[]string{"type1"},
"/index1/type1",
false,
},
{
[]string{"index1", "index2"},
[]string{"type1"},
"/index1%2Cindex2/type1",
false,
},
{
[]string{"index1", "index2"},
[]string{"type1", "type2"},
"/index1%2Cindex2/type1%2Ctype2",
false,
},
}
for i, test := range tests {
err := client.TypeExists().Index(test.Indices...).Type(test.Types...).Validate()
if err == nil && test.ExpectValidateFailure {
t.Errorf("case #%d: expected validate to fail", i+1)
continue
}
if err != nil && !test.ExpectValidateFailure {
t.Errorf("case #%d: expected validate to succeed", i+1)
continue
}
if !test.ExpectValidateFailure {
path, _, err := client.TypeExists().Index(test.Indices...).Type(test.Types...).buildURL()
if err != nil {
t.Fatalf("case #%d: %v", i+1, err)
}
if path != test.Expected {
t.Errorf("case #%d: expected %q; got: %q", i+1, test.Expected, path)
}
}
}
}
func TestTypeExists(t *testing.T) {
client := setupTestClient(t)
// Create index with tweet type
createIndex, err := client.CreateIndex(testIndexName).Body(testMapping).Do()
if err != nil {
t.Fatal(err)
}
if createIndex == nil {
t.Errorf("expected result to be != nil; got: %v", createIndex)
}
if !createIndex.Acknowledged {
t.Errorf("expected CreateIndexResult.Acknowledged %v; got %v", true, createIndex.Acknowledged)
}
// Check if type exists
exists, err := client.TypeExists().Index(testIndexName).Type("tweet").Do()
if err != nil {
t.Fatal(err)
}
if !exists {
t.Fatalf("type %s should exist in index %s, but doesn't\n", "tweet", testIndexName)
}
// Delete index
deleteIndex, err := client.DeleteIndex(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if !deleteIndex.Acknowledged {
t.Errorf("expected DeleteIndexResult.Acknowledged %v; got %v", true, deleteIndex.Acknowledged)
}
// Check if type exists
exists, err = client.TypeExists().Index(testIndexName).Type("tweet").Do()
if err != nil {
t.Fatal(err)
}
if exists {
t.Fatalf("type %s should not exist in index %s, but it does\n", "tweet", testIndexName)
}
}

View File

@ -0,0 +1,128 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndicesGetTemplateService returns an index template.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/indices-templates.html.
type IndicesGetTemplateService struct {
client *Client
pretty bool
name []string
flatSettings *bool
local *bool
}
// NewIndicesGetTemplateService creates a new IndicesGetTemplateService.
func NewIndicesGetTemplateService(client *Client) *IndicesGetTemplateService {
return &IndicesGetTemplateService{
client: client,
name: make([]string, 0),
}
}
// Name is the name of the index template.
func (s *IndicesGetTemplateService) Name(name ...string) *IndicesGetTemplateService {
s.name = append(s.name, name...)
return s
}
// FlatSettings is returns settings in flat format (default: false).
func (s *IndicesGetTemplateService) FlatSettings(flatSettings bool) *IndicesGetTemplateService {
s.flatSettings = &flatSettings
return s
}
// Local indicates whether to return local information, i.e. do not retrieve
// the state from master node (default: false).
func (s *IndicesGetTemplateService) Local(local bool) *IndicesGetTemplateService {
s.local = &local
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesGetTemplateService) Pretty(pretty bool) *IndicesGetTemplateService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesGetTemplateService) buildURL() (string, url.Values, error) {
// Build URL
var err error
var path string
if len(s.name) > 0 {
path, err = uritemplates.Expand("/_template/{name}", map[string]string{
"name": strings.Join(s.name, ","),
})
} else {
path = "/_template"
}
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.flatSettings != nil {
params.Set("flat_settings", fmt.Sprintf("%v", *s.flatSettings))
}
if s.local != nil {
params.Set("local", fmt.Sprintf("%v", *s.local))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesGetTemplateService) Validate() error {
return nil
}
// Do executes the operation.
func (s *IndicesGetTemplateService) Do() (map[string]*IndicesGetTemplateResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
var ret map[string]*IndicesGetTemplateResponse
if err := json.Unmarshal(res.Body, &ret); err != nil {
return nil, err
}
return ret, nil
}
// IndicesGetTemplateResponse is the response of IndicesGetTemplateService.Do.
type IndicesGetTemplateResponse struct {
Order int `json:"order,omitempty"`
Template string `json:"template,omitempty"`
Settings map[string]interface{} `json:"settings,omitempty"`
Mappings map[string]interface{} `json:"mappings,omitempty"`
Aliases map[string]interface{} `json:"aliases,omitempty"`
}

View File

@ -0,0 +1,41 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestIndexGetTemplateURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Names []string
Expected string
}{
{
[]string{},
"/_template",
},
{
[]string{"index1"},
"/_template/index1",
},
{
[]string{"index1", "index2"},
"/_template/index1%2Cindex2",
},
}
for _, test := range tests {
path, _, err := client.IndexGetTemplate().Name(test.Names...).buildURL()
if err != nil {
t.Fatal(err)
}
if path != test.Expected {
t.Errorf("expected %q; got: %q", test.Expected, path)
}
}
}

View File

@ -0,0 +1,179 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndicesPutTemplateService creates or updates index mappings.
// See http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/indices-templates.html.
type IndicesPutTemplateService struct {
client *Client
pretty bool
name string
order interface{}
create *bool
timeout string
masterTimeout string
flatSettings *bool
bodyJson interface{}
bodyString string
}
// NewIndicesPutTemplateService creates a new IndicesPutTemplateService.
func NewIndicesPutTemplateService(client *Client) *IndicesPutTemplateService {
return &IndicesPutTemplateService{
client: client,
}
}
// Name is the name of the index template.
func (s *IndicesPutTemplateService) Name(name string) *IndicesPutTemplateService {
s.name = name
return s
}
// Timeout is an explicit operation timeout.
func (s *IndicesPutTemplateService) Timeout(timeout string) *IndicesPutTemplateService {
s.timeout = timeout
return s
}
// MasterTimeout specifies the timeout for connection to master.
func (s *IndicesPutTemplateService) MasterTimeout(masterTimeout string) *IndicesPutTemplateService {
s.masterTimeout = masterTimeout
return s
}
// FlatSettings indicates whether to return settings in flat format (default: false).
func (s *IndicesPutTemplateService) FlatSettings(flatSettings bool) *IndicesPutTemplateService {
s.flatSettings = &flatSettings
return s
}
// Order is the order for this template when merging multiple matching ones
// (higher numbers are merged later, overriding the lower numbers).
func (s *IndicesPutTemplateService) Order(order interface{}) *IndicesPutTemplateService {
s.order = order
return s
}
// Create indicates whether the index template should only be added if
// new or can also replace an existing one.
func (s *IndicesPutTemplateService) Create(create bool) *IndicesPutTemplateService {
s.create = &create
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesPutTemplateService) Pretty(pretty bool) *IndicesPutTemplateService {
s.pretty = pretty
return s
}
// BodyJson is documented as: The template definition.
func (s *IndicesPutTemplateService) BodyJson(body interface{}) *IndicesPutTemplateService {
s.bodyJson = body
return s
}
// BodyString is documented as: The template definition.
func (s *IndicesPutTemplateService) BodyString(body string) *IndicesPutTemplateService {
s.bodyString = body
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesPutTemplateService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_template/{name}", map[string]string{
"name": s.name,
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if s.order != nil {
params.Set("order", fmt.Sprintf("%v", s.order))
}
if s.create != nil {
params.Set("create", fmt.Sprintf("%v", *s.create))
}
if s.timeout != "" {
params.Set("timeout", s.timeout)
}
if s.masterTimeout != "" {
params.Set("master_timeout", s.masterTimeout)
}
if s.flatSettings != nil {
params.Set("flat_settings", fmt.Sprintf("%v", *s.flatSettings))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesPutTemplateService) Validate() error {
var invalid []string
if s.name == "" {
invalid = append(invalid, "Name")
}
if s.bodyString == "" && s.bodyJson == nil {
invalid = append(invalid, "BodyJson")
}
if len(invalid) > 0 {
return fmt.Errorf("missing required fields: %v", invalid)
}
return nil
}
// Do executes the operation.
func (s *IndicesPutTemplateService) Do() (*IndicesPutTemplateResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Setup HTTP request body
var body interface{}
if s.bodyJson != nil {
body = s.bodyJson
} else {
body = s.bodyString
}
// Get HTTP response
res, err := s.client.PerformRequest("PUT", path, params, body)
if err != nil {
return nil, err
}
// Return operation response
ret := new(IndicesPutTemplateResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// IndicesPutTemplateResponse is the response of IndicesPutTemplateService.Do.
type IndicesPutTemplateResponse struct {
Acknowledged bool `json:"acknowledged,omitempty"`
}

View File

@ -0,0 +1,385 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
// IndicesStatsService provides stats on various metrics of one or more
// indices. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-stats.html.
type IndicesStatsService struct {
client *Client
pretty bool
metric []string
index []string
level string
types []string
completionFields []string
fielddataFields []string
fields []string
groups []string
human *bool
}
// NewIndicesStatsService creates a new IndicesStatsService.
func NewIndicesStatsService(client *Client) *IndicesStatsService {
return &IndicesStatsService{
client: client,
index: make([]string, 0),
metric: make([]string, 0),
completionFields: make([]string, 0),
fielddataFields: make([]string, 0),
fields: make([]string, 0),
groups: make([]string, 0),
types: make([]string, 0),
}
}
// Metric limits the information returned the specific metrics. Options are:
// docs, store, indexing, get, search, completion, fielddata, flush, merge,
// query_cache, refresh, suggest, and warmer.
func (s *IndicesStatsService) Metric(metric ...string) *IndicesStatsService {
s.metric = append(s.metric, metric...)
return s
}
// Index is the list of index names; use `_all` or empty string to perform
// the operation on all indices.
func (s *IndicesStatsService) Index(index ...string) *IndicesStatsService {
s.index = append(s.index, index...)
return s
}
// Level returns stats aggregated at cluster, index or shard level.
func (s *IndicesStatsService) Level(level string) *IndicesStatsService {
s.level = level
return s
}
// Types is a list of document types for the `indexing` index metric.
func (s *IndicesStatsService) Types(types ...string) *IndicesStatsService {
s.types = append(s.types, types...)
return s
}
// CompletionFields is a list of fields for `fielddata` and `suggest`
// index metric (supports wildcards).
func (s *IndicesStatsService) CompletionFields(completionFields ...string) *IndicesStatsService {
s.completionFields = append(s.completionFields, completionFields...)
return s
}
// FielddataFields is a list of fields for `fielddata` index metric (supports wildcards).
func (s *IndicesStatsService) FielddataFields(fielddataFields ...string) *IndicesStatsService {
s.fielddataFields = append(s.fielddataFields, fielddataFields...)
return s
}
// Fields is a list of fields for `fielddata` and `completion` index metric
// (supports wildcards).
func (s *IndicesStatsService) Fields(fields ...string) *IndicesStatsService {
s.fields = append(s.fields, fields...)
return s
}
// Groups is a list of search groups for `search` index metric.
func (s *IndicesStatsService) Groups(groups ...string) *IndicesStatsService {
s.groups = append(s.groups, groups...)
return s
}
// Human indicates whether to return time and byte values in human-readable format..
func (s *IndicesStatsService) Human(human bool) *IndicesStatsService {
s.human = &human
return s
}
// Pretty indicates that the JSON response be indented and human readable.
func (s *IndicesStatsService) Pretty(pretty bool) *IndicesStatsService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *IndicesStatsService) buildURL() (string, url.Values, error) {
var err error
var path string
if len(s.index) > 0 && len(s.metric) > 0 {
path, err = uritemplates.Expand("/{index}/_stats/{metric}", map[string]string{
"index": strings.Join(s.index, ","),
"metric": strings.Join(s.metric, ","),
})
} else if len(s.index) > 0 {
path, err = uritemplates.Expand("/{index}/_stats", map[string]string{
"index": strings.Join(s.index, ","),
})
} else if len(s.metric) > 0 {
path, err = uritemplates.Expand("/_stats/{metric}", map[string]string{
"metric": strings.Join(s.metric, ","),
})
} else {
path = "/_stats"
}
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.pretty {
params.Set("pretty", "1")
}
if len(s.groups) > 0 {
params.Set("groups", strings.Join(s.groups, ","))
}
if s.human != nil {
params.Set("human", fmt.Sprintf("%v", *s.human))
}
if s.level != "" {
params.Set("level", s.level)
}
if len(s.types) > 0 {
params.Set("types", strings.Join(s.types, ","))
}
if len(s.completionFields) > 0 {
params.Set("completion_fields", strings.Join(s.completionFields, ","))
}
if len(s.fielddataFields) > 0 {
params.Set("fielddata_fields", strings.Join(s.fielddataFields, ","))
}
if len(s.fields) > 0 {
params.Set("fields", strings.Join(s.fields, ","))
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *IndicesStatsService) Validate() error {
return nil
}
// Do executes the operation.
func (s *IndicesStatsService) Do() (*IndicesStatsResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(IndicesStatsResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// IndicesStatsResponse is the response of IndicesStatsService.Do.
type IndicesStatsResponse struct {
// Shards provides information returned from shards.
Shards shardsInfo `json:"_shards"`
// All provides summary stats about all indices.
All *IndexStats `json:"_all,omitempty"`
// Indices provides a map into the stats of an index. The key of the
// map is the index name.
Indices map[string]*IndexStats `json:"indices,omitempty"`
}
// IndexStats is index stats for a specific index.
type IndexStats struct {
Primaries *IndexStatsDetails `json:"primaries,omitempty"`
Total *IndexStatsDetails `json:"total,omitempty"`
}
type IndexStatsDetails struct {
Docs *IndexStatsDocs `json:"docs,omitempty"`
Store *IndexStatsStore `json:"store,omitempty"`
Indexing *IndexStatsIndexing `json:"indexing,omitempty"`
Get *IndexStatsGet `json:"get,omitempty"`
Search *IndexStatsSearch `json:"search,omitempty"`
Merges *IndexStatsMerges `json:"merges,omitempty"`
Refresh *IndexStatsRefresh `json:"refresh,omitempty"`
Flush *IndexStatsFlush `json:"flush,omitempty"`
Warmer *IndexStatsWarmer `json:"warmer,omitempty"`
FilterCache *IndexStatsFilterCache `json:"filter_cache,omitempty"`
IdCache *IndexStatsIdCache `json:"id_cache,omitempty"`
Fielddata *IndexStatsFielddata `json:"fielddata,omitempty"`
Percolate *IndexStatsPercolate `json:"percolate,omitempty"`
Completion *IndexStatsCompletion `json:"completion,omitempty"`
Segments *IndexStatsSegments `json:"segments,omitempty"`
Translog *IndexStatsTranslog `json:"translog,omitempty"`
Suggest *IndexStatsSuggest `json:"suggest,omitempty"`
QueryCache *IndexStatsQueryCache `json:"query_cache,omitempty"`
}
type IndexStatsDocs struct {
Count int64 `json:"count,omitempty"`
Deleted int64 `json:"deleted,omitempty"`
}
type IndexStatsStore struct {
Size string `json:"size,omitempty"` // human size, e.g. 119.3mb
SizeInBytes int64 `json:"size_in_bytes,omitempty"`
ThrottleTime string `json:"throttle_time,omitempty"` // human time, e.g. 0s
ThrottleTimeInMillis int64 `json:"throttle_time_in_millis,omitempty"`
}
type IndexStatsIndexing struct {
IndexTotal int64 `json:"index_total,omitempty"`
IndexTime string `json:"index_time,omitempty"`
IndexTimeInMillis int64 `json:"index_time_in_millis,omitempty"`
IndexCurrent int64 `json:"index_current,omitempty"`
DeleteTotal int64 `json:"delete_total,omitempty"`
DeleteTime string `json:"delete_time,omitempty"`
DeleteTimeInMillis int64 `json:"delete_time_in_millis,omitempty"`
DeleteCurrent int64 `json:"delete_current,omitempty"`
NoopUpdateTotal int64 `json:"noop_update_total,omitempty"`
IsThrottled bool `json:"is_throttled,omitempty"`
ThrottleTime string `json:"throttle_time,omitempty"`
ThrottleTimeInMillis int64 `json:"throttle_time_in_millis,omitempty"`
}
type IndexStatsGet struct {
Total int64 `json:"total,omitempty"`
GetTime string `json:"get_time,omitempty"`
TimeInMillis int64 `json:"time_in_millis,omitempty"`
ExistsTotal int64 `json:"exists_total,omitempty"`
ExistsTime string `json:"exists_time,omitempty"`
ExistsTimeInMillis int64 `json:"exists_time_in_millis,omitempty"`
MissingTotal int64 `json:"missing_total,omitempty"`
MissingTime string `json:"missing_time,omitempty"`
MissingTimeInMillis int64 `json:"missing_time_in_millis,omitempty"`
Current int64 `json:"current,omitempty"`
}
type IndexStatsSearch struct {
OpenContexts int64 `json:"open_contexts,omitempty"`
QueryTotal int64 `json:"query_total,omitempty"`
QueryTime string `json:"query_time,omitempty"`
QueryTimeInMillis int64 `json:"query_time_in_millis,omitempty"`
QueryCurrent int64 `json:"query_current,omitempty"`
FetchTotal int64 `json:"fetch_total,omitempty"`
FetchTime string `json:"fetch_time,omitempty"`
FetchTimeInMillis int64 `json:"fetch_time_in_millis,omitempty"`
FetchCurrent int64 `json:"fetch_current,omitempty"`
}
type IndexStatsMerges struct {
Current int64 `json:"current,omitempty"`
CurrentDocs int64 `json:"current_docs,omitempty"`
CurrentSize string `json:"current_size,omitempty"`
CurrentSizeInBytes int64 `json:"current_size_in_bytes,omitempty"`
Total int64 `json:"total,omitempty"`
TotalTime string `json:"total_time,omitempty"`
TotalTimeInMillis int64 `json:"total_time_in_millis,omitempty"`
TotalDocs int64 `json:"total_docs,omitempty"`
TotalSize string `json:"total_size,omitempty"`
TotalSizeInBytes int64 `json:"total_size_in_bytes,omitempty"`
}
type IndexStatsRefresh struct {
Total int64 `json:"total,omitempty"`
TotalTime string `json:"total_time,omitempty"`
TotalTimeInMillis int64 `json:"total_time_in_millis,omitempty"`
}
type IndexStatsFlush struct {
Total int64 `json:"total,omitempty"`
TotalTime string `json:"total_time,omitempty"`
TotalTimeInMillis int64 `json:"total_time_in_millis,omitempty"`
}
type IndexStatsWarmer struct {
Current int64 `json:"current,omitempty"`
Total int64 `json:"total,omitempty"`
TotalTime string `json:"total_time,omitempty"`
TotalTimeInMillis int64 `json:"total_time_in_millis,omitempty"`
}
type IndexStatsFilterCache struct {
MemorySize string `json:"memory_size,omitempty"`
MemorySizeInBytes int64 `json:"memory_size_in_bytes,omitempty"`
Evictions int64 `json:"evictions,omitempty"`
}
type IndexStatsIdCache struct {
MemorySize string `json:"memory_size,omitempty"`
MemorySizeInBytes int64 `json:"memory_size_in_bytes,omitempty"`
}
type IndexStatsFielddata struct {
MemorySize string `json:"memory_size,omitempty"`
MemorySizeInBytes int64 `json:"memory_size_in_bytes,omitempty"`
Evictions int64 `json:"evictions,omitempty"`
}
type IndexStatsPercolate struct {
Total int64 `json:"total,omitempty"`
GetTime string `json:"get_time,omitempty"`
TimeInMillis int64 `json:"time_in_millis,omitempty"`
Current int64 `json:"current,omitempty"`
MemorySize string `json:"memory_size,omitempty"`
MemorySizeInBytes int64 `json:"memory_size_in_bytes,omitempty"`
Queries int64 `json:"queries,omitempty"`
}
type IndexStatsCompletion struct {
Size string `json:"size,omitempty"`
SizeInBytes int64 `json:"size_in_bytes,omitempty"`
}
type IndexStatsSegments struct {
Count int64 `json:"count,omitempty"`
Memory string `json:"memory,omitempty"`
MemoryInBytes int64 `json:"memory_in_bytes,omitempty"`
IndexWriterMemory string `json:"index_writer_memory,omitempty"`
IndexWriterMemoryInBytes int64 `json:"index_writer_memory_in_bytes,omitempty"`
IndexWriterMaxMemory string `json:"index_writer_max_memory,omitempty"`
IndexWriterMaxMemoryInBytes int64 `json:"index_writer_max_memory_in_bytes,omitempty"`
VersionMapMemory string `json:"version_map_memory,omitempty"`
VersionMapMemoryInBytes int64 `json:"version_map_memory_in_bytes,omitempty"`
FixedBitSetMemory string `json:"fixed_bit_set,omitempty"`
FixedBitSetMemoryInBytes int64 `json:"fixed_bit_set_memory_in_bytes,omitempty"`
}
type IndexStatsTranslog struct {
Operations int64 `json:"operations,omitempty"`
Size string `json:"size,omitempty"`
SizeInBytes int64 `json:"size_in_bytes,omitempty"`
}
type IndexStatsSuggest struct {
Total int64 `json:"total,omitempty"`
Time string `json:"time,omitempty"`
TimeInMillis int64 `json:"time_in_millis,omitempty"`
Current int64 `json:"current,omitempty"`
}
type IndexStatsQueryCache struct {
MemorySize string `json:"memory_size,omitempty"`
MemorySizeInBytes int64 `json:"memory_size_in_bytes,omitempty"`
Evictions int64 `json:"evictions,omitempty"`
HitCount int64 `json:"hit_count,omitempty"`
MissCount int64 `json:"miss_count,omitempty"`
}

View File

@ -0,0 +1,85 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestIndexStatsBuildURL(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tests := []struct {
Indices []string
Metrics []string
Expected string
}{
{
[]string{},
[]string{},
"/_stats",
},
{
[]string{"index1"},
[]string{},
"/index1/_stats",
},
{
[]string{},
[]string{"metric1"},
"/_stats/metric1",
},
{
[]string{"index1"},
[]string{"metric1"},
"/index1/_stats/metric1",
},
{
[]string{"index1", "index2"},
[]string{"metric1"},
"/index1%2Cindex2/_stats/metric1",
},
{
[]string{"index1", "index2"},
[]string{"metric1", "metric2"},
"/index1%2Cindex2/_stats/metric1%2Cmetric2",
},
}
for i, test := range tests {
path, _, err := client.IndexStats().Index(test.Indices...).Metric(test.Metrics...).buildURL()
if err != nil {
t.Fatalf("case #%d: %v", i+1, err)
}
if path != test.Expected {
t.Errorf("case #%d: expected %q; got: %q", i+1, test.Expected, path)
}
}
}
func TestIndexStats(t *testing.T) {
client := setupTestClientAndCreateIndexAndAddDocs(t)
stats, err := client.IndexStats(testIndexName).Do()
if err != nil {
t.Fatalf("expected no error; got: %v", err)
}
if stats == nil {
t.Fatalf("expected response; got: %v", stats)
}
stat, found := stats.Indices[testIndexName]
if !found {
t.Fatalf("expected stats about index %q; got: %v", testIndexName, found)
}
if stat.Total == nil {
t.Fatalf("expected total to be != nil; got: %v", stat.Total)
}
if stat.Total.Docs == nil {
t.Fatalf("expected total docs to be != nil; got: %v", stat.Total.Docs)
}
if stat.Total.Docs.Count == 0 {
t.Fatalf("expected total docs count to be > 0; got: %d", stat.Total.Docs.Count)
}
}

View File

@ -0,0 +1,156 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
// InnerHit implements a simple join for parent/child, nested, and even
// top-level documents in Elasticsearch.
// It is an experimental feature for Elasticsearch versions 1.5 (or greater).
// See http://www.elastic.co/guide/en/elasticsearch/reference/1.5/search-request-inner-hits.html
// for documentation.
//
// See the tests for SearchSource, HasChildFilter, HasChildQuery,
// HasParentFilter, HasParentQuery, NestedFilter, and NestedQuery
// for usage examples.
type InnerHit struct {
source *SearchSource
path string
typ string
name string
}
// NewInnerHit creates a new InnerHit.
func NewInnerHit() *InnerHit {
return &InnerHit{source: NewSearchSource()}
}
func (hit *InnerHit) Path(path string) *InnerHit {
hit.path = path
return hit
}
func (hit *InnerHit) Type(typ string) *InnerHit {
hit.typ = typ
return hit
}
func (hit *InnerHit) Query(query Query) *InnerHit {
hit.source.Query(query)
return hit
}
func (hit *InnerHit) From(from int) *InnerHit {
hit.source.From(from)
return hit
}
func (hit *InnerHit) Size(size int) *InnerHit {
hit.source.Size(size)
return hit
}
func (hit *InnerHit) TrackScores(trackScores bool) *InnerHit {
hit.source.TrackScores(trackScores)
return hit
}
func (hit *InnerHit) Explain(explain bool) *InnerHit {
hit.source.Explain(explain)
return hit
}
func (hit *InnerHit) Version(version bool) *InnerHit {
hit.source.Version(version)
return hit
}
func (hit *InnerHit) Field(fieldName string) *InnerHit {
hit.source.Field(fieldName)
return hit
}
func (hit *InnerHit) Fields(fieldNames ...string) *InnerHit {
hit.source.Fields(fieldNames...)
return hit
}
func (hit *InnerHit) NoFields() *InnerHit {
hit.source.NoFields()
return hit
}
func (hit *InnerHit) FetchSource(fetchSource bool) *InnerHit {
hit.source.FetchSource(fetchSource)
return hit
}
func (hit *InnerHit) FetchSourceContext(fetchSourceContext *FetchSourceContext) *InnerHit {
hit.source.FetchSourceContext(fetchSourceContext)
return hit
}
func (hit *InnerHit) FieldDataFields(fieldDataFields ...string) *InnerHit {
hit.source.FieldDataFields(fieldDataFields...)
return hit
}
func (hit *InnerHit) FieldDataField(fieldDataField string) *InnerHit {
hit.source.FieldDataField(fieldDataField)
return hit
}
func (hit *InnerHit) ScriptFields(scriptFields ...*ScriptField) *InnerHit {
hit.source.ScriptFields(scriptFields...)
return hit
}
func (hit *InnerHit) ScriptField(scriptField *ScriptField) *InnerHit {
hit.source.ScriptField(scriptField)
return hit
}
func (hit *InnerHit) Sort(field string, ascending bool) *InnerHit {
hit.source.Sort(field, ascending)
return hit
}
func (hit *InnerHit) SortWithInfo(info SortInfo) *InnerHit {
hit.source.SortWithInfo(info)
return hit
}
func (hit *InnerHit) SortBy(sorter ...Sorter) *InnerHit {
hit.source.SortBy(sorter...)
return hit
}
func (hit *InnerHit) Highlight(highlight *Highlight) *InnerHit {
hit.source.Highlight(highlight)
return hit
}
func (hit *InnerHit) Highlighter() *Highlight {
return hit.source.Highlighter()
}
func (hit *InnerHit) Name(name string) *InnerHit {
hit.name = name
return hit
}
func (hit *InnerHit) Source() interface{} {
source, ok := hit.source.Source().(map[string]interface{})
if !ok {
return nil
}
// Notice that hit.typ and hit.path are not exported here.
// They are only used with SearchSource and serialized there.
if hit.name != "" {
source["name"] = hit.name
}
return source
}

View File

@ -0,0 +1,36 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"testing"
)
func TestInnerHitEmpty(t *testing.T) {
hit := NewInnerHit()
data, err := json.Marshal(hit.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}
func TestInnerHitWithName(t *testing.T) {
hit := NewInnerHit().Name("comments")
data, err := json.Marshal(hit.Source())
if err != nil {
t.Fatalf("marshaling to JSON failed: %v", err)
}
got := string(data)
expected := `{"name":"comments"}`
if got != expected {
t.Errorf("expected\n%s\n,got:\n%s", expected, got)
}
}

View File

@ -0,0 +1,194 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
)
type MultiGetService struct {
client *Client
preference string
realtime *bool
refresh *bool
items []*MultiGetItem
}
func NewMultiGetService(client *Client) *MultiGetService {
builder := &MultiGetService{
client: client,
items: make([]*MultiGetItem, 0),
}
return builder
}
func (b *MultiGetService) Preference(preference string) *MultiGetService {
b.preference = preference
return b
}
func (b *MultiGetService) Refresh(refresh bool) *MultiGetService {
b.refresh = &refresh
return b
}
func (b *MultiGetService) Realtime(realtime bool) *MultiGetService {
b.realtime = &realtime
return b
}
func (b *MultiGetService) Add(items ...*MultiGetItem) *MultiGetService {
b.items = append(b.items, items...)
return b
}
func (b *MultiGetService) Source() interface{} {
source := make(map[string]interface{})
items := make([]interface{}, len(b.items))
for i, item := range b.items {
items[i] = item.Source()
}
source["docs"] = items
return source
}
func (b *MultiGetService) Do() (*MultiGetResult, error) {
// Build url
path := "/_mget"
params := make(url.Values)
if b.realtime != nil {
params.Add("realtime", fmt.Sprintf("%v", *b.realtime))
}
if b.preference != "" {
params.Add("preference", b.preference)
}
if b.refresh != nil {
params.Add("refresh", fmt.Sprintf("%v", *b.refresh))
}
// Set body
body := b.Source()
// Get response
res, err := b.client.PerformRequest("GET", path, params, body)
if err != nil {
return nil, err
}
// Return result
ret := new(MultiGetResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Multi Get Item --
// MultiGetItem is a single document to retrieve via the MultiGetService.
type MultiGetItem struct {
index string
typ string
id string
routing string
fields []string
version *int64 // see org.elasticsearch.common.lucene.uid.Versions
versionType string // see org.elasticsearch.index.VersionType
fsc *FetchSourceContext
}
func NewMultiGetItem() *MultiGetItem {
return &MultiGetItem{}
}
func (item *MultiGetItem) Index(index string) *MultiGetItem {
item.index = index
return item
}
func (item *MultiGetItem) Type(typ string) *MultiGetItem {
item.typ = typ
return item
}
func (item *MultiGetItem) Id(id string) *MultiGetItem {
item.id = id
return item
}
func (item *MultiGetItem) Routing(routing string) *MultiGetItem {
item.routing = routing
return item
}
func (item *MultiGetItem) Fields(fields ...string) *MultiGetItem {
if item.fields == nil {
item.fields = make([]string, 0)
}
item.fields = append(item.fields, fields...)
return item
}
// Version can be MatchAny (-3), MatchAnyPre120 (0), NotFound (-1),
// or NotSet (-2). These are specified in org.elasticsearch.common.lucene.uid.Versions.
// The default in Elasticsearch is MatchAny (-3).
func (item *MultiGetItem) Version(version int64) *MultiGetItem {
item.version = &version
return item
}
// VersionType can be "internal", "external", "external_gt", "external_gte",
// or "force". See org.elasticsearch.index.VersionType in Elasticsearch source.
// It is "internal" by default.
func (item *MultiGetItem) VersionType(versionType string) *MultiGetItem {
item.versionType = versionType
return item
}
func (item *MultiGetItem) FetchSource(fetchSourceContext *FetchSourceContext) *MultiGetItem {
item.fsc = fetchSourceContext
return item
}
// Source returns the serialized JSON to be sent to Elasticsearch as
// part of a MultiGet search.
func (item *MultiGetItem) Source() interface{} {
source := make(map[string]interface{})
source["_id"] = item.id
if item.index != "" {
source["_index"] = item.index
}
if item.typ != "" {
source["_type"] = item.typ
}
if item.fsc != nil {
source["_source"] = item.fsc.Source()
}
if item.fields != nil {
source["fields"] = item.fields
}
if item.routing != "" {
source["_routing"] = item.routing
}
if item.version != nil {
source["version"] = fmt.Sprintf("%d", *item.version)
}
if item.versionType != "" {
source["version_type"] = item.versionType
}
return source
}
// -- Result of a Multi Get request.
type MultiGetResult struct {
Docs []*GetResult `json:"docs,omitempty"`
}

View File

@ -0,0 +1,95 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"testing"
)
func TestMultiGet(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
// Add some documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Count documents
count, err := client.Count(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
if count != 3 {
t.Errorf("expected Count = %d; got %d", 3, count)
}
// Get documents 1 and 3
res, err := client.MultiGet().
Add(NewMultiGetItem().Index(testIndexName).Type("tweet").Id("1")).
Add(NewMultiGetItem().Index(testIndexName).Type("tweet").Id("3")).
Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatal("expected result to be != nil; got nil")
}
if res.Docs == nil {
t.Fatal("expected result docs to be != nil; got nil")
}
if len(res.Docs) != 2 {
t.Fatalf("expected to have 2 docs; got %d", len(res.Docs))
}
item := res.Docs[0]
if item.Error != "" {
t.Errorf("expected no error on item 0; got %q", item.Error)
}
if item.Source == nil {
t.Errorf("expected Source != nil; got %v", item.Source)
}
var doc tweet
if err := json.Unmarshal(*item.Source, &doc); err != nil {
t.Fatalf("expected to unmarshal item Source; got %v", err)
}
if doc.Message != tweet1.Message {
t.Errorf("expected Message of first tweet to be %q; got %q", tweet1.Message, doc.Message)
}
item = res.Docs[1]
if item.Error != "" {
t.Errorf("expected no error on item 1; got %q", item.Error)
}
if item.Source == nil {
t.Errorf("expected Source != nil; got %v", item.Source)
}
if err := json.Unmarshal(*item.Source, &doc); err != nil {
t.Fatalf("expected to unmarshal item Source; got %v", err)
}
if doc.Message != tweet3.Message {
t.Errorf("expected Message of second tweet to be %q; got %q", tweet3.Message, doc.Message)
}
}

View File

@ -0,0 +1,101 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
)
// MultiSearch executes one or more searches in one roundtrip.
// See http://www.elasticsearch.org/guide/reference/api/multi-search/
type MultiSearchService struct {
client *Client
requests []*SearchRequest
indices []string
pretty bool
routing string
preference string
}
func NewMultiSearchService(client *Client) *MultiSearchService {
builder := &MultiSearchService{
client: client,
requests: make([]*SearchRequest, 0),
indices: make([]string, 0),
}
return builder
}
func (s *MultiSearchService) Add(requests ...*SearchRequest) *MultiSearchService {
s.requests = append(s.requests, requests...)
return s
}
func (s *MultiSearchService) Index(index string) *MultiSearchService {
s.indices = append(s.indices, index)
return s
}
func (s *MultiSearchService) Indices(indices ...string) *MultiSearchService {
s.indices = append(s.indices, indices...)
return s
}
func (s *MultiSearchService) Pretty(pretty bool) *MultiSearchService {
s.pretty = pretty
return s
}
func (s *MultiSearchService) Do() (*MultiSearchResult, error) {
// Build url
path := "/_msearch"
// Parameters
params := make(url.Values)
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
// Set body
lines := make([]string, 0)
for _, sr := range s.requests {
// Set default indices if not specified in the request
if !sr.HasIndices() && len(s.indices) > 0 {
sr = sr.Indices(s.indices...)
}
header, err := json.Marshal(sr.header())
if err != nil {
return nil, err
}
body, err := json.Marshal(sr.body())
if err != nil {
return nil, err
}
lines = append(lines, string(header))
lines = append(lines, string(body))
}
body := strings.Join(lines, "\n") + "\n" // Don't forget trailing \n
// Get response
res, err := s.client.PerformRequest("GET", path, params, body)
if err != nil {
return nil, err
}
// Return result
ret := new(MultiSearchResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
type MultiSearchResult struct {
Responses []*SearchResult `json:"responses,omitempty"`
}

View File

@ -0,0 +1,197 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
_ "net/http"
"testing"
)
func TestMultiSearch(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{
User: "olivere",
Message: "Welcome to Golang and Elasticsearch.",
Tags: []string{"golang", "elasticsearch"},
}
tweet2 := tweet{
User: "olivere",
Message: "Another unrelated topic.",
Tags: []string{"golang"},
}
tweet3 := tweet{
User: "sandrae",
Message: "Cycling is fun.",
Tags: []string{"sports", "cycling"},
}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Spawn two search queries with one roundtrip
q1 := NewMatchAllQuery()
q2 := NewTermQuery("tags", "golang")
sreq1 := NewSearchRequest().Indices(testIndexName, testIndexName2).
Source(NewSearchSource().Query(q1).Size(10))
sreq2 := NewSearchRequest().Index(testIndexName).Type("tweet").
Source(NewSearchSource().Query(q2))
searchResult, err := client.MultiSearch().
Add(sreq1, sreq2).
Do()
if err != nil {
t.Fatal(err)
}
if searchResult.Responses == nil {
t.Fatal("expected responses != nil; got nil")
}
if len(searchResult.Responses) != 2 {
t.Fatalf("expected 2 responses; got %d", len(searchResult.Responses))
}
sres := searchResult.Responses[0]
if sres.Hits == nil {
t.Errorf("expected Hits != nil; got nil")
}
if sres.Hits.TotalHits != 3 {
t.Errorf("expected Hits.TotalHits = %d; got %d", 3, sres.Hits.TotalHits)
}
if len(sres.Hits.Hits) != 3 {
t.Errorf("expected len(Hits.Hits) = %d; got %d", 3, len(sres.Hits.Hits))
}
for _, hit := range sres.Hits.Hits {
if hit.Index != testIndexName {
t.Errorf("expected Hits.Hit.Index = %q; got %q", testIndexName, hit.Index)
}
item := make(map[string]interface{})
err := json.Unmarshal(*hit.Source, &item)
if err != nil {
t.Fatal(err)
}
}
sres = searchResult.Responses[1]
if sres.Hits == nil {
t.Errorf("expected Hits != nil; got nil")
}
if sres.Hits.TotalHits != 2 {
t.Errorf("expected Hits.TotalHits = %d; got %d", 2, sres.Hits.TotalHits)
}
if len(sres.Hits.Hits) != 2 {
t.Errorf("expected len(Hits.Hits) = %d; got %d", 2, len(sres.Hits.Hits))
}
for _, hit := range sres.Hits.Hits {
if hit.Index != testIndexName {
t.Errorf("expected Hits.Hit.Index = %q; got %q", testIndexName, hit.Index)
}
item := make(map[string]interface{})
err := json.Unmarshal(*hit.Source, &item)
if err != nil {
t.Fatal(err)
}
}
}
func TestMultiSearchWithOneRequest(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{
User: "olivere",
Message: "Welcome to Golang and Elasticsearch.",
Tags: []string{"golang", "elasticsearch"},
}
tweet2 := tweet{
User: "olivere",
Message: "Another unrelated topic.",
Tags: []string{"golang"},
}
tweet3 := tweet{
User: "sandrae",
Message: "Cycling is fun.",
Tags: []string{"sports", "cycling"},
}
// Add all documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Spawn two search queries with one roundtrip
query := NewMatchAllQuery()
source := NewSearchSource().Query(query).Size(10)
sreq := NewSearchRequest().Source(source)
searchResult, err := client.MultiSearch().
Index(testIndexName).
Add(sreq).
Do()
if err != nil {
t.Fatal(err)
}
if searchResult.Responses == nil {
t.Fatal("expected responses != nil; got nil")
}
if len(searchResult.Responses) != 1 {
t.Fatalf("expected 1 responses; got %d", len(searchResult.Responses))
}
sres := searchResult.Responses[0]
if sres.Hits == nil {
t.Errorf("expected Hits != nil; got nil")
}
if sres.Hits.TotalHits != 3 {
t.Errorf("expected Hits.TotalHits = %d; got %d", 3, sres.Hits.TotalHits)
}
if len(sres.Hits.Hits) != 3 {
t.Errorf("expected len(Hits.Hits) = %d; got %d", 3, len(sres.Hits.Hits))
}
for _, hit := range sres.Hits.Hits {
if hit.Index != testIndexName {
t.Errorf("expected Hits.Hit.Index = %q; got %q", testIndexName, hit.Index)
}
item := make(map[string]interface{})
err := json.Unmarshal(*hit.Source, &item)
if err != nil {
t.Fatal(err)
}
}
}

View File

@ -0,0 +1,311 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"log"
"net/url"
"strings"
"time"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
var (
_ = fmt.Print
_ = log.Print
_ = strings.Index
_ = uritemplates.Expand
_ = url.Parse
)
// NodesInfoService allows to retrieve one or more or all of the
// cluster nodes information.
// It is documented at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-info.html.
type NodesInfoService struct {
client *Client
pretty bool
nodeId []string
metric []string
flatSettings *bool
human *bool
}
// NewNodesInfoService creates a new NodesInfoService.
func NewNodesInfoService(client *Client) *NodesInfoService {
return &NodesInfoService{
client: client,
nodeId: []string{"_all"},
metric: []string{"_all"},
}
}
// NodeId is a list of node IDs or names to limit the returned information.
// Use "_local" to return information from the node you're connecting to,
// leave empty to get information from all nodes.
func (s *NodesInfoService) NodeId(nodeId ...string) *NodesInfoService {
s.nodeId = make([]string, 0)
s.nodeId = append(s.nodeId, nodeId...)
return s
}
// Metric is a list of metrics you wish returned. Leave empty to return all.
// Valid metrics are: settings, os, process, jvm, thread_pool, network,
// transport, http, and plugins.
func (s *NodesInfoService) Metric(metric ...string) *NodesInfoService {
s.metric = make([]string, 0)
s.metric = append(s.metric, metric...)
return s
}
// FlatSettings returns settings in flat format (default: false).
func (s *NodesInfoService) FlatSettings(flatSettings bool) *NodesInfoService {
s.flatSettings = &flatSettings
return s
}
// Human indicates whether to return time and byte values in human-readable format.
func (s *NodesInfoService) Human(human bool) *NodesInfoService {
s.human = &human
return s
}
// Pretty indicates whether to indent the returned JSON.
func (s *NodesInfoService) Pretty(pretty bool) *NodesInfoService {
s.pretty = pretty
return s
}
// buildURL builds the URL for the operation.
func (s *NodesInfoService) buildURL() (string, url.Values, error) {
// Build URL
path, err := uritemplates.Expand("/_nodes/{node_id}/{metric}", map[string]string{
"node_id": strings.Join(s.nodeId, ","),
"metric": strings.Join(s.metric, ","),
})
if err != nil {
return "", url.Values{}, err
}
// Add query string parameters
params := url.Values{}
if s.flatSettings != nil {
params.Set("flat_settings", fmt.Sprintf("%v", *s.flatSettings))
}
if s.human != nil {
params.Set("human", fmt.Sprintf("%v", *s.human))
}
if s.pretty {
params.Set("pretty", "1")
}
return path, params, nil
}
// Validate checks if the operation is valid.
func (s *NodesInfoService) Validate() error {
return nil
}
// Do executes the operation.
func (s *NodesInfoService) Do() (*NodesInfoResponse, error) {
// Check pre-conditions
if err := s.Validate(); err != nil {
return nil, err
}
// Get URL for request
path, params, err := s.buildURL()
if err != nil {
return nil, err
}
// Get HTTP response
res, err := s.client.PerformRequest("GET", path, params, nil)
if err != nil {
return nil, err
}
// Return operation response
ret := new(NodesInfoResponse)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// NodesInfoResponse is the response of NodesInfoService.Do.
type NodesInfoResponse struct {
ClusterName string `json:"cluster_name"`
Nodes map[string]*NodesInfoNode `json:"nodes"`
}
type NodesInfoNode struct {
// Name of the node, e.g. "Mister Fear"
Name string `json:"name"`
// TransportAddress, e.g. "inet[/127.0.0.1:9300]"
TransportAddress string `json:"transport_address"`
// Host is the host name, e.g. "macbookair"
Host string `json:"host"`
// IP is the IP address, e.g. "192.168.1.2"
IP string `json:"ip"`
// Version is the Elasticsearch version running on the node, e.g. "1.4.3"
Version string `json:"version"`
// Build is the Elasticsearch build, e.g. "36a29a7"
Build string `json:"build"`
// HTTPAddress, e.g. "inet[/127.0.0.1:9200]"
HTTPAddress string `json:"http_address"`
// HTTPSAddress, e.g. "inet[/127.0.0.1:9200]"
HTTPSAddress string `json:"https_address"`
// Settings of the node, e.g. paths and pidfile.
Settings map[string]interface{} `json:"settings"`
// OS information, e.g. CPU and memory.
OS *NodesInfoNodeOS `json:"os"`
// Process information, e.g. max file descriptors.
Process *NodesInfoNodeProcess `json:"process"`
// JVM information, e.g. VM version.
JVM *NodesInfoNodeProcess `json:"jvm"`
// ThreadPool information.
ThreadPool *NodesInfoNodeThreadPool `json:"thread_pool"`
// Network information.
Network *NodesInfoNodeNetwork `json:"network"`
// Network information.
Transport *NodesInfoNodeTransport `json:"transport"`
// HTTP information.
HTTP *NodesInfoNodeHTTP `json:"http"`
// Plugins information.
Plugins []*NodesInfoNodePlugin `json:"plugins"`
}
type NodesInfoNodeOS struct {
RefreshInterval string `json:"refresh_interval"` // e.g. 1s
RefreshIntervalInMillis int `json:"refresh_interval_in_millis"` // e.g. 1000
AvailableProcessors int `json:"available_processors"` // e.g. 4
// CPU information
CPU struct {
Vendor string `json:"vendor"` // e.g. Intel
Model string `json:"model"` // e.g. iMac15,1
MHz int `json:"mhz"` // e.g. 3500
TotalCores int `json:"total_cores"` // e.g. 4
TotalSockets int `json:"total_sockets"` // e.g. 4
CoresPerSocket int `json:"cores_per_socket"` // e.g. 16
CacheSizeInBytes int `json:"cache_size_in_bytes"` // e.g. 256
} `json:"cpu"`
// Mem information
Mem struct {
Total string `json:"total"` // e.g. 16gb
TotalInBytes int `json:"total_in_bytes"` // e.g. 17179869184
} `json:"mem"`
// Swap information
Swap struct {
Total string `json:"total"` // e.g. 1gb
TotalInBytes int `json:"total_in_bytes"` // e.g. 1073741824
} `json:"swap"`
}
type NodesInfoNodeProcess struct {
RefreshInterval string `json:"refresh_interval"` // e.g. 1s
RefreshIntervalInMillis int `json:"refresh_interval_in_millis"` // e.g. 1000
ID int `json:"id"` // process id, e.g. 87079
MaxFileDescriptors int `json:"max_file_descriptors"` // e.g. 32768
Mlockall bool `json:"mlockall"` // e.g. false
}
type NodesInfoNodeJVM struct {
PID int `json:"pid"` // process id, e.g. 87079
Version string `json:"version"` // e.g. "1.8.0_25"
VMName string `json:"vm_name"` // e.g. "Java HotSpot(TM) 64-Bit Server VM"
VMVersion string `json:"vm_version"` // e.g. "25.25-b02"
VMVendor string `json:"vm_vendor"` // e.g. "Oracle Corporation"
StartTime time.Time `json:"start_time"` // e.g. "2015-01-03T15:18:30.982Z"
StartTimeInMillis int64 `json:"start_time_in_millis"`
// Mem information
Mem struct {
HeapInit string `json:"heap_init"` // e.g. 1gb
HeapInitInBytes int `json:"heap_init_in_bytes"`
HeapMax string `json:"heap_max"` // e.g. 4gb
HeapMaxInBytes int `json:"heap_max_in_bytes"`
NonHeapInit string `json:"non_heap_init"` // e.g. 2.4mb
NonHeapInitInBytes int `json:"non_heap_init_in_bytes"`
NonHeapMax string `json:"non_heap_max"` // e.g. 0b
NonHeapMaxInBytes int `json:"non_heap_max_in_bytes"`
DirectMax string `json:"direct_max"` // e.g. 4gb
DirectMaxInBytes int `json:"direct_max_in_bytes"`
} `json:"mem"`
GCCollectors []string `json:"gc_collectors"` // e.g. ["ParNew"]
MemoryPools []string `json:"memory_pools"` // e.g. ["Code Cache", "Metaspace"]
}
type NodesInfoNodeThreadPool struct {
Percolate *NodesInfoNodeThreadPoolSection `json:"percolate"`
Bench *NodesInfoNodeThreadPoolSection `json:"bench"`
Listener *NodesInfoNodeThreadPoolSection `json:"listener"`
Index *NodesInfoNodeThreadPoolSection `json:"index"`
Refresh *NodesInfoNodeThreadPoolSection `json:"refresh"`
Suggest *NodesInfoNodeThreadPoolSection `json:"suggest"`
Generic *NodesInfoNodeThreadPoolSection `json:"generic"`
Warmer *NodesInfoNodeThreadPoolSection `json:"warmer"`
Search *NodesInfoNodeThreadPoolSection `json:"search"`
Flush *NodesInfoNodeThreadPoolSection `json:"flush"`
Optimize *NodesInfoNodeThreadPoolSection `json:"optimize"`
Management *NodesInfoNodeThreadPoolSection `json:"management"`
Get *NodesInfoNodeThreadPoolSection `json:"get"`
Merge *NodesInfoNodeThreadPoolSection `json:"merge"`
Bulk *NodesInfoNodeThreadPoolSection `json:"bulk"`
Snapshot *NodesInfoNodeThreadPoolSection `json:"snapshot"`
}
type NodesInfoNodeThreadPoolSection struct {
Type string `json:"type"` // e.g. fixed
Min int `json:"min"` // e.g. 4
Max int `json:"max"` // e.g. 4
KeepAlive string `json:"keep_alive"` // e.g. "5m"
QueueSize interface{} `json:"queue_size"` // e.g. "1k" or -1
}
type NodesInfoNodeNetwork struct {
RefreshInterval string `json:"refresh_interval"` // e.g. 1s
RefreshIntervalInMillis int `json:"refresh_interval_in_millis"` // e.g. 1000
PrimaryInterface struct {
Address string `json:"address"` // e.g. 192.168.1.2
Name string `json:"name"` // e.g. en0
MACAddress string `json:"mac_address"` // e.g. 11:22:33:44:55:66
} `json:"primary_interface"`
}
type NodesInfoNodeTransport struct {
BoundAddress string `json:"bound_address"` // e.g. inet[/127.0.0.1:9300]
PublishAddress string `json:"publish_address"` // e.g. inet[/127.0.0.1:9300]
}
type NodesInfoNodeHTTP struct {
BoundAddress string `json:"bound_address"` // e.g. inet[/127.0.0.1:9300]
PublishAddress string `json:"publish_address"` // e.g. inet[/127.0.0.1:9300]
MaxContentLength string `json:"max_content_length"` // e.g. "100mb"
MaxContentLengthInBytes int64 `json:"max_content_length_in_bytes"`
}
type NodesInfoNodePlugin struct {
Name string `json:"name"`
Description string `json:"description"`
Site bool `json:"site"`
JVM bool `json:"jvm"`
URL string `json:"url"` // e.g. /_plugin/dummy/
}

View File

@ -0,0 +1,40 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import "testing"
func TestNodesInfo(t *testing.T) {
client, err := NewClient()
if err != nil {
t.Fatal(err)
}
info, err := client.NodesInfo().Do()
if err != nil {
t.Fatal(err)
}
if info == nil {
t.Fatal("expected nodes info")
}
if info.ClusterName == "" {
t.Errorf("expected cluster name; got: %q", info.ClusterName)
}
if len(info.Nodes) == 0 {
t.Errorf("expected some nodes; got: %d", len(info.Nodes))
}
for id, node := range info.Nodes {
if id == "" {
t.Errorf("expected node id; got: %q", id)
}
if node == nil {
t.Fatalf("expected node info; got: %v", node)
}
if node.IP == "" {
t.Errorf("expected node IP; got: %q", node.IP)
}
}
}

View File

@ -0,0 +1,135 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"gopkg.in/olivere/elastic.v2/uritemplates"
)
type OptimizeService struct {
client *Client
indices []string
maxNumSegments *int
onlyExpungeDeletes *bool
flush *bool
waitForMerge *bool
force *bool
pretty bool
}
func NewOptimizeService(client *Client) *OptimizeService {
builder := &OptimizeService{
client: client,
indices: make([]string, 0),
}
return builder
}
func (s *OptimizeService) Index(index string) *OptimizeService {
s.indices = append(s.indices, index)
return s
}
func (s *OptimizeService) Indices(indices ...string) *OptimizeService {
s.indices = append(s.indices, indices...)
return s
}
func (s *OptimizeService) MaxNumSegments(maxNumSegments int) *OptimizeService {
s.maxNumSegments = &maxNumSegments
return s
}
func (s *OptimizeService) OnlyExpungeDeletes(onlyExpungeDeletes bool) *OptimizeService {
s.onlyExpungeDeletes = &onlyExpungeDeletes
return s
}
func (s *OptimizeService) Flush(flush bool) *OptimizeService {
s.flush = &flush
return s
}
func (s *OptimizeService) WaitForMerge(waitForMerge bool) *OptimizeService {
s.waitForMerge = &waitForMerge
return s
}
func (s *OptimizeService) Force(force bool) *OptimizeService {
s.force = &force
return s
}
func (s *OptimizeService) Pretty(pretty bool) *OptimizeService {
s.pretty = pretty
return s
}
func (s *OptimizeService) Do() (*OptimizeResult, error) {
// Build url
path := "/"
// Indices part
indexPart := make([]string, 0)
for _, index := range s.indices {
index, err := uritemplates.Expand("{index}", map[string]string{
"index": index,
})
if err != nil {
return nil, err
}
indexPart = append(indexPart, index)
}
if len(indexPart) > 0 {
path += strings.Join(indexPart, ",")
}
path += "/_optimize"
// Parameters
params := make(url.Values)
if s.maxNumSegments != nil {
params.Set("max_num_segments", fmt.Sprintf("%d", *s.maxNumSegments))
}
if s.onlyExpungeDeletes != nil {
params.Set("only_expunge_deletes", fmt.Sprintf("%v", *s.onlyExpungeDeletes))
}
if s.flush != nil {
params.Set("flush", fmt.Sprintf("%v", *s.flush))
}
if s.waitForMerge != nil {
params.Set("wait_for_merge", fmt.Sprintf("%v", *s.waitForMerge))
}
if s.force != nil {
params.Set("force", fmt.Sprintf("%v", *s.force))
}
if s.pretty {
params.Set("pretty", fmt.Sprintf("%v", s.pretty))
}
// Get response
res, err := s.client.PerformRequest("POST", path, params, nil)
if err != nil {
return nil, err
}
// Return result
ret := new(OptimizeResult)
if err := json.Unmarshal(res.Body, ret); err != nil {
return nil, err
}
return ret, nil
}
// -- Result of an optimize request.
type OptimizeResult struct {
Shards shardsInfo `json:"_shards,omitempty"`
}

View File

@ -0,0 +1,47 @@
// Copyright 2012-2015 Oliver Eilhard. All rights reserved.
// Use of this source code is governed by a MIT-license.
// See http://olivere.mit-license.org/license.txt for details.
package elastic
import (
"testing"
)
func TestOptimize(t *testing.T) {
client := setupTestClientAndCreateIndex(t)
tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
tweet2 := tweet{User: "olivere", Message: "Another unrelated topic."}
tweet3 := tweet{User: "sandrae", Message: "Cycling is fun."}
// Add some documents
_, err := client.Index().Index(testIndexName).Type("tweet").Id("1").BodyJson(&tweet1).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("2").BodyJson(&tweet2).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Index().Index(testIndexName).Type("tweet").Id("3").BodyJson(&tweet3).Do()
if err != nil {
t.Fatal(err)
}
_, err = client.Flush().Index(testIndexName).Do()
if err != nil {
t.Fatal(err)
}
// Optimize documents
res, err := client.Optimize(testIndexName, testIndexName2).Do()
if err != nil {
t.Fatal(err)
}
if res == nil {
t.Fatal("expected result; got nil")
}
}

Some files were not shown because too many files have changed in this diff Show More