16.10. Extending the Converter Library

There are two ways to extend the converter library - adding new transformation functions and adding new data formats.

16.10.1. Adding New Transformation Functions

To add new transformation functions, create a TransformationFunctionFactory and register it in META-INF/services/org.locationtech.geomesa.convert.TransformationFunctionFactory. For example, here’s how to add a new transformation function that computes a SHA-256 hash.

import org.locationtech.geomesa.convert.TransformerFunctionFactory
import org.locationtech.geomesa.convert.TransformerFn

class SHAFunctionFactory extends TransformerFunctionFactory {
  override def functions = Seq(sha256fn)
  val sha256fn = TransformerFn("sha256") { args =>
    Hashing.sha256().hashBytes(args(0).asInstanceOf[Array[Byte]])
  }
}

The sha256 function can then be used in a field as shown.

fields: [
   { name = "hash", transform = "sha256(stringToBytes($0))" }
]

16.10.2. Adding New Data Formats

To add new data formats, implement the SimpleFeatureConverterFactory and SimpleFeatureConverter interfaces and register them in META-INF/services appropriately. See org.locationtech.geomesa.convert.avro.Avro2SimpleFeatureConverter for an example.

16.10.3. Adding Functions to the Geomesa Classpath

After creating a JAR file with your transformation function and factory you can add these to the GEOMESA_EXTRA_CLASSPATHS environmental variable in order to expose them to the command line tools and distributed (mapreduce) ingest jobs.

A example of ingest with a transforms on the classpath is below:

GEOMESA_EXTRA_CLASSPATHS="/tmp/custom-transformer-1.0.0.jar" bin/geomesa ingest -u <user-name>
-p <password> -s <sft-name> -C <converter-name> -c geomesa.catalog hdfs://localhost:9000/data/example.csv

You can also verify the classpath is properly configured with the tools:

GEOMESA_EXTRA_CLASSPATHS="/tmp/custom-transformer-1.0.0.jar" bin/geomesa classpath