Indexing and Searching Geo points using Elasticsearch

Elasticsearch allows users to index geo points as a part of document. The Elasticsearch provides an inbuilt datatype named ‘geo_point’ to get the latitude and longitude data indexed.

Let’s see how the geo points can be stored and searched with a simple example.

Mapping data:

curl -XPUT http://localhost:9200/simplyjava/user -d '{

                " user":{

                        "properties": {

                        "location":{

                             "type":" geo_point"

                        }

          }

    }

}’

Indexing data:

The data for geo_points can be indexed using the below code snippet.

curl -XPOST http://localhost:9200/simplyjava/user/testuser1 -d '{

"location":{

                                     "lat":"80.234",

                                     "lon":"-120.4"   

}

}’

Searching data:

Below code can be used to retrieve the list of people who are within a particular mile radius from a geo location.

curl -XGET 'http://localhost:9200/simplyjava/user/_search?pretty=true' -d '{

  "query": {

"filtered":{

                   "filter" : {

                         "geo_distance" : {

                              "distance" : "1000miles",

                                      "location" : {

                                     "lat" : 70.12,

                                      "lon" : -120.4

                                     }

                         }

            }

       }

  }

}'

Elasticsearch Response:

{

   "took": 131,

   "timed_out": false,

   "_shards": {

      "total": 5,

      "successful": 5,

      "failed": 0

   },

   "hits": {

      "total": 1,

      "max_score": 1,

      "hits": [

         {

            "_index": "simplyjava",

            "_type": "user",

            "_id": "1",

            "_score": 1,

            "_source": {

               "location": {

                  "lat": "80.234",

                  "lon": "-120.4"

               }

            }

         }

      ]

   }

}

Create explicit mappings in Elasticsearch index

An Elasticsearch index can have more than one type. Each type will have its own set of fields. In this post, let’s see how fields can be mapped to various datatypes in Elasticsearch.

Elasticsearch will do its own mapping. Additionally, we can tell it in which way the fields in a document should be stored, indexed and analyzed.

The field that needs to be treated as string or number or geolocation can be specified. The type of analyzer to be used for a particular field and when it should be applied can be defined in field mapping. (Analyzers will be added as settings to an index and it will be applicable for all types. It need not be added for individual document type in Elasticsearch index.)

Field mapping using REST Service:

curl -XPUT http://localhost:9200/simplyjava/user -d '{

"user":{

            "properties": {

                        "name":{

                        "type":"string","search_analyzer":"a1"

                        },

                        "age":{

                        "type":"long"

                        },

                        "dateofbirth":{

                        "type":"date",

                        "format":"yyyy-MM-dd"

                        },

                        "location":{

                        "type": "geo_point"

                        }

            }

}'

Field mapping using Java API:

XContentBuilder mappingBuilder = XContentFactory.jsonBuilder()

                                                                    .startObject().startObject("user")

                                          .startObject("properties")

                                          .startObject("name")

                                            .field("type", "string")

                                            .field("search_analyzer", "a1")

                                          .endObject()

                                          .startObject("age")

                                            .field("type","long")

                                          .endObject()

                                          .startObject("dateofbirth")

                                            .field("type","date")

                                            .field("format","yyyy-MM-dd")

                                          .endObject()

                                          .startObject("location")

                                            .field("type","geo_point")

                                          .endObject()

                                         .endObject()

                                       .endObject()

                                      .endObject();

PutMappingResponse response =client.admin().indices().preparePutMapping("simplyjava").

setType("user").setSource(mappingBuilder).execute().actionGet();

  Note:

 Mappings of indexed document fields cannot be updated. The documents should be deleted and re-indexed in order to get the mapping type changed for the fields in a type.

Delete Index in Elasticsearch

An existing index/type/document in Elasticsearch can be deleted either by using REST service or by using the Java API invocation.

Below are the scripts to delete the index.

Delete complete index :

REST  Service:

$ curl -XDELETE 'http://localhost:9200/<indexname>'

Java API :

client.admin().indices().delete(new DeleteIndexRequest("<indexname>")).actionGet();

Delete a type in index   :

REST Service :

$ curl -XDELETE 'http://localhost:9200/<indexname>/<typename>'

Java API :

client.prepareDelete().setIndex("<indexname>").setType("<typename>").setId("*").execute().actionGet();

Delete a particular document :

REST Service :

$ curl -XDELETE 'http://localhost:9200/<indexname>/<typename>/<documentId>

Java API:

client.prepareDelete().setIndex("<indexname>").setType("<typename>").setId("<documentId>").execute().actionGet();

Note:

The delete operation cannot be rolled back at any point. So make sure to double check the delete command before execution.

Create Elasticsearch index using RESTFul Service and Java API

Elasticsearch is used for faster retrieval of data from a stored index. An index can be created either by using the RESTFul service or Java APIs exposed by Elasticsearch. In this blog, let us see what an index is and how to create it.

Basically, an Elasticsearch index is more like a database with multiple types (tables) in it. In relational model, an index can be related to database and type can be related to a table in the database. All the documents that are indexed are similar to the rows in a database table.

An elastic search cluster can have more than one index.

The below code will create an index with name ‘simplyjava’ in the Elasticsearch cluster. We can add our own settings in the created index. By settings, I mean that the type of analyzers, number of shards and number of replicas to be used by index. This will be covered in upcoming posts.

Creating an index using Elasticsearch RESTFul service:

$ curl -XPUT 'http://localhost:9200/simplyjava/'

Creating an index using Elasticsearch Java API:

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").build();

TransportClient transportClient = new TransportClient(settings);

transportClient = transportClient.addTransportAddress(new InetSocketTransportAddress("localhost", 9300));

Client  client = (Client) transportClient;

CreateIndexRequestBuilder createIndex = client.admin().indices().prepareCreate("simplyjava");

CreateIndexResponse response = createIndex.execute().actionGet();

Note : Even though both the codes do the same functionality you can see that the port value used is different. That is because, 9300 is the port where the TransportClient resides and 9200 is the http port configured in Elasticsearch.