# GeoMesa NiFi Quick Start¶

This tutorial provides an example implementation for using GeoMesa with NiFi. This walk-through will guide you in setting up the components required for ingesting GDELT files into GeoMesa running on Accumulo.

## Prerequisites¶

Before you begin, you must have the following:

This quick start operates by reading CSV files from the local filesystem, and writing them to Accumulo using the PutGeoMesaAccumulo processor.

## Obtain GDELT data¶

In this tutorial we will be ingesting GDELT data. If you already have some GDELT data downloaded, then you may skip this section.

The GDELT Event database provides a comprehensive time- and location-indexed archive of events reported in broadcast, print, and web news media worldwide from 1979 to today. You can download raw GDELT data files at http://data.gdeltproject.org/events/index.html.

GeoMesa ships with the ability to parse GDELT data, and a script for downloading it. For more details, see Global Database of Events, Language, and Tone (GDELT).

Note

GDELT is available in two different formats, the original and version 2. GeoMesa provides converters for both formats, but take note which format you download, as the converter name will differ.

## Download and Build the GeoMesa NiFi project¶

Pick a reasonable directory on your machine, and run:

$git clone https://github.com/geomesa/geomesa-nifi.git$ cd geomesa-nifi


To build, run

### Install the SFTs and Converters¶

Additionally we need to copy the geomesa-tools_2.11-$VERSION-data.jar from the geomesa binary distribution to the lib/ of the NiFi installation. The geomesa-tools_2.11-$VERSION-data.jar is located in the dist/converters directory of the geomesa Accumulo binary distribution. This jar contains the SimpleFeatureType and converter definitions needed for GeoMesa to ingest the GDELT data. You can obtain the binary distribution from GitHub, or you may build it locally from source.

## Create a NiFi Flow¶

If you are not familiar with NiFi, follow the Getting Started guide to familiarize yourself. The rest of this tutorial assumes a basic understanding of NiFi.

Create the GeoMesa processor by dragging a new processor to your flow, and selecting ‘PutGeoMesaAccumulo’. Select the processor and click the ‘configure’ button to configure it. On the properties page, fill out the following values:

• Mode: Converter
• SftName: gdelt or gdelt2 (depending on what version of data you downloaded)
• ConverterName: gdelt or gdelt2 (depending on what version of data you downloaded)
• FeatureNameOverride: gdelt
• accumulo.instance.id: the ID of your Accumulo instance
• accumulo.zookeepers: the zookeeper hosts of your Accumulo instance
• accumulo.user: an Accumulo user with create and write permissions
• accumulo.catalog: the catalog table you want to ingest data into

Warning

If you have set up the GeoMesa Accumulo distributed runtime to be isolated within a namespace, as described in Installing the Accumulo Distributed Runtime Library, the value of accumulo.catalog should include the namespace (e.g. myNamespace.gdelt).

Now create a ‘GetFile’ processor, and hook it up as the input to the GeoMesa processor.

Once both processors are configured, you can start them through the NiFi UI.

## Ingest the Data¶

Assuming you have configured your NiFi flow with a ‘GetFile’ processor, you can ingest data by copying the GDELT data you downloaded into the processor’s configured input path. Note that you will need to use plain files - if the files are zipped, unzip them before ingesting.

You should see the data pass through the NiFi flow and be ingested.

## Visualize the Data¶

Once the data has been ingested, you can use GeoServer to visualize it on a map. Follow the instructions in the Accumulo quick-start tutorial, Visualize Data (optional).