Releases

v1.0.17 latest

This release adds the otter-stream-sql module — full Flink SQL integration with a scalar UDF, a lookup table function, a dynamic table connector, and MinIO/S3 model loading support. It also introduces the shade-plugin configuration that bundles all runtime dependencies into the *-flink-udf.jar classifier artifact required for SQL Gateway deployment.

What is new

• otter-stream-sql module — MLInferenceFunction (ScalarFunction), MLInferenceLookupFunction (TableFunction), MLInferenceDynamicTableFactory (connector identifier: ml-inference).
• MinIO model loader — MinioModelLoader downloads SavedModel directories and single-file formats (ONNX, XGBoost, PMML) from any S3-compatible endpoint at engine initialisation time.
• Shaded JAR classifier — running mvn package inside otter-stream-sql now produces both the thin JAR and the *-flink-udf.jar shaded artifact with all runtime deps bundled.
• SqlInferenceConfig — typed DDL option parser covering model path, format, MinIO credentials, cache TTL, batch size, async flag, and retry policy.
• ModelCache TTL default raised from 30 to 60 minutes.
• Caffeine upgraded to 3.1.8 to resolve a thread-local memory leak under high eviction rates.

Bug fixes

• Fixed NullPointerException in MLInferenceFunction.eval() when InferenceResult.getOutputs() returned an empty map rather than null.
• Fixed ClassCastException when the first output tensor was an int[] rather than a float[] or double[].
• Fixed serialisation of SqlInferenceConfig across Flink task slot boundaries.

Maven coordinates

<dependency>
  <groupId>com.codedstreams</groupId>
  <artifactId>otter-stream-sql</artifactId>
  <version>1.0.17</version>
</dependency>

v1.0.15

Maintenance release. Upgraded ONNX Runtime to 1.23.2 and resolved a memory-leak in the TensorFlow engine when SavedModelBundle was not closed on engine eviction from ModelCache.

Changes

• ONNX Runtime upgraded from 1.16.3 to 1.23.2.
• TensorFlowInferenceEngine.close() now properly calls SavedModelBundle.close() on cache eviction.
• XGBoost4J upgraded to 3.1.1.
• AWS SDK upgraded to 2.21.29.
• Added EngineCapabilities interface so callers can query whether an engine supports GPU acceleration before committing to it.
• Improved error messages on InferenceException to include the model name and path that failed.

Maven coordinates

<dependency>
  <groupId>com.codedstreams</groupId>
  <artifactId>ml-inference-core</artifactId>
  <version>1.0.15</version>
</dependency>

v1.0.12

Added the otter-stream-remote module and otter-stream-pytorch module. Introduced Micrometer metrics instrumentation across all engines.

Changes

• New module: otter-stream-remote — HTTP and gRPC clients for SageMaker, Vertex AI, Azure ML, and arbitrary REST endpoints.
• New module: otter-stream-pytorch — TorchScript inference via Deep Java Library 0.25.0.
• All engines now emit Micrometer metrics: inference.latency, inference.requests.total, inference.errors.total.
• ModelCache emits cache.hits and cache.misses counters.
• Flink version in parent POM updated from 1.16.0 to 1.17.0.

v1.0.0 initial release

Initial public release. Established the core architecture: InferenceEngine interface, InferenceResult and InferenceException value types, ModelCache singleton, InferenceConfig builder, and the Flink AsyncModelInferenceFunction wrapper. Shipped with ONNX, TensorFlow, XGBoost, and PMML engine implementations.

Compatibility Matrix

Upgrade Notes

From 1.0.15 to 1.0.17

• The otter-stream-sql module is new in 1.0.17. No changes to existing module APIs.
• If you were using ModelCache directly, the TTL default has changed from 30 to 60 minutes. Override via SET 'otter.model.cache.ttl-minutes' = '30' if you need the old behaviour.
• Rebuild your shaded JAR — the new shade configuration in otter-stream-sql/pom.xml is required for SQL Gateway deployment. The old thin JAR will produce a NoClassDefFoundError at registration time.

From 1.0.12 to 1.0.15

• ONNX Runtime 1.23.2 drops support for AVX without AVX2. If your CPUs do not support AVX2 pin to onnxruntime:1.16.3 in your dependency overrides.
• The TensorFlowInferenceEngine now calls close() automatically on cache eviction. If you were managing the lifecycle manually via engine.close(), remove those calls to avoid double-close errors.

From 1.0.0 to 1.0.12

• Parent POM Flink version bumped from 1.16.0 to 1.17.0. Rebuild all modules against your target Flink version before deploying.
• InferenceConfig now requires modelConfig(ModelConfig) to be set; previously modelPath could be set directly on the top-level builder. Update call sites accordingly.