16.10. Extending the Converter Library¶
There are two ways to extend the converter library - adding new transformation functions and adding new data formats.
16.10.1. Adding New Transformation Functions¶
To add new transformation functions, create a
TransformationFunctionFactory
and register it in
META-INF/services/org.locationtech.geomesa.convert.TransformationFunctionFactory
.
For example, here’s how to add a new transformation function that
computes a SHA-256 hash.
import org.locationtech.geomesa.convert.TransformerFunctionFactory
import org.locationtech.geomesa.convert.TransformerFn
class SHAFunctionFactory extends TransformerFunctionFactory {
override def functions = Seq(sha256fn)
val sha256fn = TransformerFn("sha256") { args =>
Hashing.sha256().hashBytes(args(0).asInstanceOf[Array[Byte]])
}
}
The sha256
function can then be used in a field as shown.
fields: [
{ name = "hash", transform = "sha256(stringToBytes($0))" }
]
16.10.2. Adding New Data Formats¶
To add new data formats, implement the SimpleFeatureConverterFactory
and SimpleFeatureConverter
interfaces and register them in
META-INF/services
appropriately. See
org.locationtech.geomesa.convert.avro.Avro2SimpleFeatureConverter
for an example.
16.10.3. Adding Functions to the Geomesa Classpath¶
After creating a JAR file with your transformation function and factory
you can add these to the GEOMESA_EXTRA_CLASSPATHS
environmental variable
in order to expose them to the command line tools and distributed (mapreduce)
ingest jobs.
A example of ingest with a transforms on the classpath is below:
GEOMESA_EXTRA_CLASSPATHS="/tmp/custom-transformer-1.0.0.jar" bin/geomesa ingest -u <user-name>
-p <password> -s <sft-name> -C <converter-name> -c geomesa.catalog hdfs://localhost:9000/data/example.csv
You can also verify the classpath is properly configured with the tools:
GEOMESA_EXTRA_CLASSPATHS="/tmp/custom-transformer-1.0.0.jar" bin/geomesa classpath