Releases

v0.0.4 latest

This release introduces Apache Flink 2.0+ support and Java 17/21 compatibility. All engine modules have been tested against Flink 2.0, 2.1, and 2.2 previews. The SQL Gateway integration now works seamlessly with Flink 2.x's redesigned Table API and type system.

What's New

• Flink 2.0+ support — the SDK now officially supports Apache Flink versions 2.0, 2.1, and 2.2. Users running Flink 1.x should continue using v1.0.17.
• Java 17 and 21 compatibility — all modules have been recompiled and tested against Java 17 (LTS) and Java 21 (LTS). The SDK no longer supports Java 11.
• Updated shaded JAR configuration — the shade plugin now properly excludes Flink 2.x dependencies while bundling all runtime engine libraries.
• Improved type inference — better handling of Flink 2.x's new type extraction system for UDF registration.
• Enhanced error messages — more descriptive errors when model loading or inference fails, including the specific model name and path.

Breaking Changes

• Java 11 is no longer supported. Upgrade your Flink cluster to Java 17 or 21 before deploying v0.0.4.
• Flink 1.x is not supported. Use v1.0.17 or earlier for Flink 1.15–1.20 clusters.
• The ml-inference-core module now requires flink-streaming-java:2.0.0+ as a provided dependency.

Maven coordinates

<dependency>
  <groupId>com.codedstreams</groupId>
  <artifactId>otter-stream-sql</artifactId>
  <version>0.0.4</version>
</dependency>

<!-- Or the core module -->
<dependency>
  <groupId>com.codedstreams</groupId>
  <artifactId>ml-inference-core</artifactId>
  <version>0.0.4</version>
</dependency>

v1.0.17 stable for Flink 1.x

This release adds the otter-stream-sql module — full Flink SQL integration with a scalar UDF, a lookup table function, a dynamic table connector, and MinIO/S3 model loading support. It also introduces the shade-plugin configuration that bundles all runtime dependencies into the *-flink-udf.jar classifier artifact required for SQL Gateway deployment.

Note: This version is intended for Flink 1.15–1.20 clusters. For Flink 2.0+, please use v0.0.4 or newer.

What is new

• otter-stream-sql module — MLInferenceFunction (ScalarFunction), MLInferenceLookupFunction (TableFunction), MLInferenceDynamicTableFactory (connector identifier: ml-inference).
• MinIO model loader — MinioModelLoader downloads SavedModel directories and single-file formats (ONNX, XGBoost, PMML) from any S3-compatible endpoint at engine initialisation time.
• Shaded JAR classifier — running mvn package inside otter-stream-sql now produces both the thin JAR and the *-flink-udf.jar shaded artifact with all runtime deps bundled.
• SqlInferenceConfig — typed DDL option parser covering model path, format, MinIO credentials, cache TTL, batch size, async flag, and retry policy.
• ModelCache TTL default raised from 30 to 60 minutes.
• Caffeine upgraded to 3.1.8 to resolve a thread-local memory leak under high eviction rates.

Bug fixes

• Fixed NullPointerException in MLInferenceFunction.eval() when InferenceResult.getOutputs() returned an empty map rather than null.
• Fixed ClassCastException when the first output tensor was an int[] rather than a float[] or double[].
• Fixed serialisation of SqlInferenceConfig across Flink task slot boundaries.

Maven coordinates

<dependency>
  <groupId>com.codedstreams</groupId>
  <artifactId>otter-stream-sql</artifactId>
  <version>1.0.17</version>
</dependency>

v1.0.15

Maintenance release. Upgraded ONNX Runtime to 1.23.2 and resolved a memory-leak in the TensorFlow engine when SavedModelBundle was not closed on engine eviction from ModelCache.

Changes

• ONNX Runtime upgraded from 1.16.3 to 1.23.2.
• TensorFlowInferenceEngine.close() now properly calls SavedModelBundle.close() on cache eviction.
• XGBoost4J upgraded to 3.1.1.
• AWS SDK upgraded to 2.21.29.
• Added EngineCapabilities interface so callers can query whether an engine supports GPU acceleration before committing to it.
• Improved error messages on InferenceException to include the model name and path that failed.

v1.0.12

Added the otter-stream-remote module and otter-stream-pytorch module. Introduced Micrometer metrics instrumentation across all engines.

Changes

• New module: otter-stream-remote — HTTP and gRPC clients for SageMaker, Vertex AI, Azure ML, and arbitrary REST endpoints.
• New module: otter-stream-pytorch — TorchScript inference via Deep Java Library 0.25.0.
• All engines now emit Micrometer metrics: inference.latency, inference.requests.total, inference.errors.total.
• ModelCache emits cache.hits and cache.misses counters.
• Flink version in parent POM updated from 1.16.0 to 1.17.0.

v1.0.0 initial release

Initial public release. Established the core architecture: InferenceEngine interface, InferenceResult and InferenceException value types, ModelCache singleton, InferenceConfig builder, and the Flink AsyncModelInferenceFunction wrapper. Shipped with ONNX, TensorFlow, XGBoost, and PMML engine implementations.

Compatibility Matrix

Upgrade Notes

From 1.0.17 to 0.0.4 (Flink 1.x → Flink 2.0+)

• Upgrade your Flink cluster to 2.0+ before deploying v0.0.4.
• Upgrade your JVM to Java 17 or 21 — Java 11 is no longer supported.
• Rebuild your shaded JAR using the v0.0.4 parent POM. The shade configuration has been updated to exclude Flink 2.x dependencies correctly.
• If you are using Scala 2.13 features, ensure your custom UDFs are compiled against Scala 2.13.
• Test your SQL pipelines in a staging environment before production — Flink 2.x introduces several API changes in the Table API and type system.

From 1.0.15 to 1.0.17

• The otter-stream-sql module is new in 1.0.17. No changes to existing module APIs.
• If you were using ModelCache directly, the TTL default has changed from 30 to 60 minutes. Override via SET 'otter.model.cache.ttl-minutes' = '30' if you need the old behaviour.
• Rebuild your shaded JAR — the new shade configuration in otter-stream-sql/pom.xml is required for SQL Gateway deployment. The old thin JAR will produce a NoClassDefFoundError at registration time.

From 1.0.12 to 1.0.15

• ONNX Runtime 1.23.2 drops support for AVX without AVX2. If your CPUs do not support AVX2 pin to onnxruntime:1.16.3 in your dependency overrides.
• The TensorFlowInferenceEngine now calls close() automatically on cache eviction. If you were managing the lifecycle manually via engine.close(), remove those calls to avoid double-close errors.