17.1. Installing GeoMesa Kafka


GeoMesa currently supports Kafka versions 2.0 and later.


The examples below expect a version to be set in the environment:

$ export TAG="4.0.5"
$ export VERSION="2.12-${TAG}" # note: 2.12 is the Scala build version

17.1.1. Installing from the Binary Distribution

GeoMesa Kafka artifacts are available for download or can be built from source. The easiest way to get started is to download the most recent binary version from GitHub.

Download and extract it somewhere convenient:

# download and unpackage the most recent distribution:
$ wget "https://github.com/locationtech/geomesa/releases/download/geomesa-${TAG}/geomesa-kafka_${VERSION}-bin.tar.gz"
$ tar xvf geomesa-kafka_${VERSION}-bin.tar.gz
$ cd geomesa-kafka_${VERSION}

17.1.2. Building from Source

GeoMesa Kafka may also be built from source. For more information, refer to the instructions on GitHub. The remainder of the instructions in this chapter assume the use of the binary GeoMesa distribution.

If you have built from source, the Kafka distribution is created in the target directory of the geomesa-kafka/geomesa-kafka-dist module.

17.1.3. Setting up the Kafka Command Line Tools

GeoMesa comes with a set of command line tools for managing Kafka features. In the Kafka distribution the tools are located in geomesa-kafka_${VERSION}/bin/.

If the environment variables KAFKA_HOME and ZOOKEEPER_HOME are set, then GeoMesa will load the appropriate JARs and configuration files from those locations and no further configuration is required. Otherwise, you will be prompted to download the appropriate JARs the first time you invoke the tools. Environment variables can be specified in conf/*-env.sh and dependency versions can be specified in conf/dependencies.sh.

GeoMesa also provides the ability to add additional JARs to the classpath using the environmental variable $GEOMESA_EXTRA_CLASSPATHS. GeoMesa will prepend the contents of this environmental variable to the computed classpath, giving it highest precedence in the classpath. Users can provide directories of jar files or individual files using a colon (:) as a delimiter. These entries will also be added the the map-reduce libjars variable.

Due to licensing restrictions, dependencies for shape file support must be separately installed. Do this with the following command:

$ ./bin/install-shapefile-support.sh

Test the command that invokes the GeoMesa Tools:

$ geomesa-kafka

The output should look like this:

Usage: geomesa-kafka [command] [command options]

17.1.4. Installing GeoMesa Kafka in GeoServer


See GeoServer Versions to ensure that GeoServer is compatible with your GeoMesa version. Installing GeoServer

As described in section GeoMesa and GeoServer, GeoMesa implements a GeoTools-compatible data store. This makes it possible to use GeoMesa Kafka as a data store in GeoServer. GeoServer’s web site includes installation instructions for GeoServer.

After GeoServer is running, you will also need to install the WPS plugin to your GeoServer instance. The GeoServer WPS Plugin must match the version of GeoServer instance. The GeoServer website includes instructions for downloading and installing the WPS plugin.


If using Tomcat as a web server, it will most likely be necessary to pass some custom options:

export CATALINA_OPTS="-Xmx8g -XX:MaxPermSize=512M -Duser.timezone=UTC \
-server -Djava.awt.headless=true"

The value of -Xmx should be as large as your system will permit; this is especially important for the Kafka plugin. Be sure to restart Tomcat for changes to take place. Installing the GeoMesa Kafka Data Store

To install the GeoMesa data store, extract the contents of the geomesa-kafka-gs-plugin_${VERSION}-install.tar.gz file in geomesa-kafka_${VERSION}/dist/gs-plugins/ in the binary distribution or geomesa-kafka/geomesa-kafka-gs-plugin/target/ in the source distribution into your GeoServer’s lib directory:

$ tar -xzvf \
  geomesa-kafka_${VERSION}/dist/gs-plugins/geomesa-kafka-gs-plugin_${VERSION}-install.tar.gz \
  -C /path/to/geoserver/webapps/geoserver/WEB-INF/lib

Next, install the JARs for Kafka and Zookeeper. By default, JARs will be downloaded from Maven central. You may override this by setting the environment variable GEOMESA_MAVEN_URL. If you do not have an internet connection you can download the JARs manually.

Edit the file geomesa-kafka_${VERSION}/conf/dependencies.sh to set the versions of Kafka and Zookeeper to match the target environment, and then run the script:

$ ./bin/install-dependencies.sh /path/to/geoserver/webapps/geoserver/WEB-INF/lib


Ensure that the Scala version of both GeoMesa and Kafka match to avoid compatibility errors.

Restart GeoServer after the JARs are installed.