Era Software

No results match your query

2. Write and query data

Estimated reading time: 3 minutes
  • tutorial
  • self-hosted

Now that you've installed self-hosted EraSearch, you’re ready to start managing and learning from your log data. In this guide, you'll write and query data using the EraSearch REST API.

Note: This series works with self-hosted EraSearch. If you're looking for the fastest way to set up and use EraSearch, get started with EraSearch on EraCloud.

Before you begin
Copy
Copied!

Connect your machine to your EraSearch deployment using local port forwarding. Enter this command in your terminal, replacing NAMESPACE_NAME and NAME with your values from the previous guide:

Copy
Copied!
$ kubectl port-forward -n NAMESPACE_NAME svc/NAME-quarry-deployment 9200:9200

The response is similar to this output:

Copy
Copied!
Forwarding from 127.0.0.1:9200 -> 9200

Writing data
Copy
Copied!

The EraSearch REST API is similar to Elasticsearch's API. All writes go to the /_bulk HTTP endpoint, which accepts JSON-formatted data. This page uses cURL to show how the API works, but you can use any language or framework to make HTTP requests to your database.

To write sample data to EraSearch, paste the command below in your terminal. The example writes one document

document

A document is a JSON object made up of data. In EraSearch, all documents have a unique identifier (_id) and a timestamp (_ts). Most documents include additional fields. Here's an example of a document:

Copy
Copied!
{"_id":4248176661010579457,"_line":"access","response":200,"_ts":1634060854000}
to the my_era_logs index

index

An index is a group of similar documents. With EraSearch, you can query documents in one or more indexes to optimize your searches.

. The document has one field

field

A field is data stored as a key and value pair.

with the key _line, and EraSearch creates the index for you if the index doesn't exist.

Copy
Copied!
$ curl -XPOST 'http://localhost:9200/_bulk' \
    -d '{"index":{"_index":"my_era_logs"}}
        {"_line": "my first log line"}'

Note: _line is a recognized field key in EraSearch. The database auto-parses _line values and stores them as distinct strings for future queries. For example, EraSearch stores the field value above as ["my","first","log","line"]. By default, EraSearch doesn't auto-parse the values of other field keys such as line or _logline.

EraSearch's response looks like the example below, including this information about the write:

  • took - An integer showing the time EraSearch takes in milliseconds to complete all writes.
  • errors - A boolean value set to false if all writes succeed.
  • _id - A unique numerical identifier for the document.
  • status - An HTTP status code for the write.
Copy
Copied!
{
  "took": 453,
  "errors": false,
  "items": [
    {
      "index": {
        "_index": "my_era_logs",
        "_type": "_doc",
        "_id": "8802999840880787456",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 1,
          "successful": 1,
          "failed": 0
        },
        "status": 201
      }
    }
  ]
}

Reading data
Copy
Copied!

You can now query the data using EraSearch's GET /_search HTTP endpoint. While the example below uses cURL, the GET /_search and POST /_msearch endpoints work with several Elasticsearch clients and frameworks.

EraSearch supports most of Elasticsearch's query string syntax. You can use the same syntax to query data based on keywords, ranges, booleans, and wildcards.

For example, enter this command to view data in the my_era_logs index where the _line field includes log:

Copy
Copied!
$ curl 'http://localhost:9200/my_era_logs/_search?q=_line:log'

EraSearch's response is similar to the JSON output below. It has information about the request, including:

  • took - The time it takes to serve the request, in milliseconds.

  • _ts - The document's epoch timestamp.

    If the original write request didn't specify a timestamp, _ts is the time the server received the write.

Copy
Copied!
{
  "took": 2,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "my_era_logs",
        "_id": "8802999840880787456",
        "_score": 1,
        "_source": {
          "_line": "my first log line",
          "_ts": 1649347314721,
          "_lid": 8802999840880787000
        }
      }
    ]
  }
}

Next steps
Copy
Copied!

Now that you have the basics, visit the list of write integrations to see how to get real-time data into EraSearch. For more information about exploring data and managing EraSearch, visit: