Era Software

Writing data with Cloudflare

Estimated reading time: 3 minutes

This integration is not yet generally available. Please reach out if you would like to use this integration or have any questions.

This page shows how to use Cloudflare's Logpush feature to ingest logs directly into EraSearch. In this guide, you'll:

  • Configure Cloudflare to push logs directly to EraSearch leveraging the Splunk HEC format
  • View the logs in the EraSearch UI

Before you begin

This content is intended for engineers and developers using EraSearch either on EraCloud or in a self-hosted environment. To create an EraCloud account, visit the Getting started series. You'll need your EraSearch Service URI and API key to complete the steps below.

In addition, you'll need administrative access to a Cloudflare Enterprise account to configure the Logpush integration.

Instructions

Step 1: Go to the Logpush job configuration page in Cloudflare

From the Cloudflare dashboard, after selecting a domain you want to use with Logpush:

  1. Navigate to the Analytics section
  2. From Analytics, choose the Logs sub-section
  3. From Logs, select Connect a Service to create the integration

Step 2: Configure the data fields for your Logpush job

From the Select Data Set screen, choose the HTTP requests option.

From the Select Data Fields screen, select all available fields using the checkbox at the top.

If you already know what data fields you want to include or exclude from the integration, feel free to only select those items. If you are new to this integration, we recommend starting with all options.

Step 3: Configure the destination for your Logpush job

From the Select Destination screen, choose the Splunk option.

Note: This guide uses the Splunk option to integrate with EraSearch. That workflow is possible because the EraSearch REST API supports ingesting data in Splunk's HEC format.

From the Enter Destination Info screen, fill in the following details:

  • Splunk raw HTTP Event Collector URL - This will be your EraSearch Service URI with a suffix of /services/collector/raw. For example, if your EraSearch URI is https://era.example.com, then set this option to https://era.example.com/services/collector/raw.

  • Channel ID - A random UUID or string identifier used to identify this stream of data. You'll need to generate this yourself using a random string or UUID generator.

  • Auth Token - This is the token set in the Authorization header of the incoming requests. To have EraSearch accept the data, this needs to be set to a valid authorization header.

    • For EraCloud users, this is Bearer%20${YOUR_ERACLOUD_TOKEN}, where ${YOUR_ERACLOUD_TOKEN} is a valid token.
    • For self-hosted deployments, insert a valid basic HTTP authorization header.
  • Source Type - The source type, set to cloudflare:json.

Step 4: Enable your Logpush job

With the destination information configured, all that's left to do is enable the integration by selecting Push.

Once completed, data should now be flowing from Cloudflare into your EraSearch database. In the next section, you'll view your data.

Step 5: View your data in the EraSearch UI

To view your log data:

  • For EraCloud users, sign in to your EraCloud account and click EraSearch UI.
  • For self-hosted EraSearch users, navigate to the respective EraSearch user interface.

Next, navigate to the FILTERS tab and select the logs-cloudflare index. You may need to refresh the UI if the index you specified is new.

The EraSearch UI displays your log data organized by time. Each document has a unique numerical identifier (_lid) and has data stored in several field key-value pairs.

Next steps

That's it. Your EraSearch instance is now receiving real-time log data. To learn more about writing data to EraSearch, visit:

For more information about Cloudflare, including what logs you can collect, visit these pages: