otter-stream-remote

Cloud Services

Remote inference clients for cloud ML services and HTTP/gRPC endpoints.

Module Overview

The Remote module provides integration with cloud ML services and custom inference endpoints via HTTP/gRPC. It supports major cloud providers including AWS SageMaker, Google Vertex AI, Azure ML, and custom REST/gRPC endpoints.

🌐

HTTP Client

REST API

Generic HTTP/REST client for custom model serving endpoints.

  • OkHttp-based connection pooling
  • Configurable timeouts and retries
  • Custom authentication headers
  • JSON request/response handling
📡

gRPC Client

High Performance

gRPC client for TensorFlow Serving, NVIDIA Triton, and custom gRPC servers.

  • HTTP/2 multiplexing and streaming
  • Binary protocol for efficiency
  • TLS/authentication support
  • Type-safe from protobuf definitions
☁️

AWS SageMaker

AWS

Native integration with AWS SageMaker hosted model endpoints.

  • SageMaker Runtime API integration
  • IAM role or static credentials
  • Automatic retry with exponential backoff
  • Multi-region support
🔷

Google Vertex AI

GCP

Integration with Google Cloud Vertex AI PredictionService API.

  • Application Default Credentials
  • gRPC-based communication
  • Native batch prediction support
  • Multi-region deployment

Implementing Remote Inference

Connect to cloud ML services and custom endpoints.

// AWS SageMaker
ModelConfig sagemakerConfig = ModelConfig.builder()
    .modelId("sagemaker-model")
    .endpointUrl("my-sagemaker-endpoint")
    .format(ModelFormat.SAGEMAKER)
    .authConfig(AuthConfig.builder()
        .apiKey("ACCESS_KEY:SECRET_KEY")
        .build())
    .build();

SageMakerInferenceClient client = new SageMakerInferenceClient();
client.initialize(sagemakerConfig);

// Google Vertex AI
InferenceConfig vertexConfig = InferenceConfig.builder()
    .modelConfig(ModelConfig.builder()
        .modelName("vertex-model")
        .build())
    .engineOption("endpoint", "projects/my-project/locations/us-central1/endpoints/123")
    .engineOption("project_id", "my-project")
    .engineOption("location", "us-central1")
    .build();

VertexAIInferenceClient vertexClient = new VertexAIInferenceClient(vertexConfig);

// HTTP REST Endpoint
ModelConfig httpConfig = ModelConfig.builder()
    .modelId("custom-api")
    .endpointUrl("https://api.myservice.com/v1/predict")
    .format(ModelFormat.REMOTE_HTTP)
    .authConfig(AuthConfig.builder()
        .addHeader("Authorization", "Bearer token123")
        .build())
    .build();

HttpInferenceClient httpClient = new HttpInferenceClient();
httpClient.initialize(httpConfig);

Quick Links

🌐 HTTP Client Details 📡 gRPC Client Details ☁️ AWS SageMaker 📚 Full JavaDocs 🏠 Back to Home 📋 ← PMML Module 📋 Examples Module →

Maven Dependency

<dependency>
    <groupId>com.codedstreams</groupId>
    <artifactId>otter-stream-remote</artifactId>
    <version>1.0.16</version>
</dependency>