Package com.codedstream.otterstream.remote

Class RemoteInferenceEngine

  • All Implemented Interfaces:
    InferenceEngine<Void>
    Direct Known Subclasses:
    HttpInferenceClient, SageMakerInferenceClient
    public abstract class RemoteInferenceEngine
extends Object
implements InferenceEngine<Void>
    Abstract base class for remote inference engines communicating with external model endpoints.

    Provides common functionality for remote inference implementations including:

    • Basic initialization with endpoint URL
    • Default batch inference implementation (sequential processing)
    • Connection state management
    • Resource cleanup framework

    Extending This Class:

    Subclasses should implement:

    1. initialize(ModelConfig) - Set up remote connection
    2. infer(Map) - Send single inference request
    3. validateConnection() - Verify endpoint availability
    4. close() - Clean up resources

    Batch Inference:

    The default inferBatch(Map[]) implementation processes requests sequentially. Subclasses should override this method if the remote endpoint supports native batch processing for better performance.

    Usage Pattern:

    {@code
 public class MyRemoteEngine extends RemoteInferenceEngine {
    Since:
    1.0.0
    Author:
    Nestor Martourez, Sr Software and Data Streaming Engineer @ CodedStreams
    See Also:
    InferenceEngine, HttpInferenceClient, SageMakerInferenceClient

    • Field Detail

      • initialized

        protected boolean initialized

      • endpointUrl

        protected String endpointUrl

    • Constructor Detail

      • RemoteInferenceEngine

        public RemoteInferenceEngine()

    • Method Detail

      • infer

        public abstract InferenceResult infer​(Map<String,​Object> inputs)
                               throws InferenceException
        Performs single inference on remote endpoint (abstract).

        Subclasses must implement this method to send inference requests to the remote endpoint and parse responses.

        Specified by:
        infer in interface InferenceEngine<Void>
        Parameters:
        inputs - map of input names to values
        Returns:
        inference result containing outputs and timing
        Throws:
        InferenceException - if inference fails

      • inferBatch

        public InferenceResult inferBatch​(Map<String,​Object>[] batchInputs)
                           throws InferenceException
        Performs batch inference using sequential processing.

        Default implementation processes batch inputs sequentially by calling infer(Map) for each input. Outputs are aggregated with index suffixes to maintain traceability.

        Output Format:

         {
   "output1_0": value1,
   "output2_0": value2,
   "output1_1": value3,
   "output2_1": value4,
   ...
 }

        Performance Note: Sequential processing is inefficient for large batches. Override this method if the remote endpoint supports native batch processing.

        Specified by:
        inferBatch in interface InferenceEngine<Void>
        Parameters:
        batchInputs - array of input maps for batch processing
        Returns:
        aggregated inference result containing all batch outputs
        Throws:
        InferenceException - if any inference in the batch fails

      • isReady

        public boolean isReady()
        Checks if the engine is ready for inference.
        Specified by:
        isReady in interface InferenceEngine<Void>
        Returns:
        true if initialized and endpoint URL is set

      • close

        public void close()
           throws InferenceException
        Closes the engine and releases resources.

        Base implementation marks engine as not initialized and clears endpoint URL. Subclasses should override to clean up their own resources (close HTTP clients, database connections, etc.), then call super.close().

        Specified by:
        close in interface InferenceEngine<Void>
        Throws:
        InferenceException - if cleanup fails

      • validateConnection

        public abstract boolean validateConnection()
                                    throws InferenceException
        Validates connection to remote endpoint (abstract).

        Subclasses must implement this method to test connectivity to the remote endpoint. This is useful for health checks and connection pooling.

        Returns:
        true if connection is successful
        Throws:
        InferenceException - if validation fails (network error, timeout)

      • getCapabilities

        public InferenceEngine.EngineCapabilities getCapabilities()
        Gets engine capabilities for remote endpoints.

        Default capabilities indicate:

        • Not a local engine (remote = false)
        • No native batch support (batch = false)
        • Max batch size of 1 (sequential processing)
        • Async operations supported (async = true)

        Subclasses should override to provide accurate capabilities for their specific remote endpoint.

        Specified by:
        getCapabilities in interface InferenceEngine<Void>
        Returns:
        default engine capabilities for remote inference