Package com.codedstream.otterstream.inference.config

Class InferenceConfig.Builder

java.lang.Object

com.codedstream.otterstream.inference.config.InferenceConfig.Builder

Enclosing class:

InferenceConfig

public static class InferenceConfig.Builder
extends Object

Builder for creating InferenceConfig instances with sensible defaults.

Constructor Summary

Constructors

Constructor	Description
Builder()

Method Summary

Modifier and Type	Method	Description
InferenceConfig.Builder	batchSize(int batchSize)	Sets the batch size for inference operations.
InferenceConfig	build()	Builds the InferenceConfig instance.
InferenceConfig.Builder	enableMetrics(boolean enableMetrics)	Enables or disables metrics collection.
InferenceConfig.Builder	engineOptions(Map<String,Object> engineOptions)	Sets engine-specific options.
InferenceConfig.Builder	maxRetries(int maxRetries)	Sets maximum retry attempts for failed inferences.
InferenceConfig.Builder	modelConfig(ModelConfig modelConfig)	Sets the model configuration.
InferenceConfig.Builder	timeout(Duration duration)	Sets the inference timeout using Duration.
InferenceConfig.Builder	timeoutMs(long timeoutMs)	Sets the inference timeout in milliseconds.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder()

Method Detail

modelConfig

public InferenceConfig.Builder modelConfig(ModelConfig modelConfig)

Sets the model configuration.

Parameters:

modelConfig - model configuration

Returns:

this builder

batchSize

public InferenceConfig.Builder batchSize(int batchSize)

Sets the batch size for inference operations.

Larger batch sizes improve throughput but increase latency. Default is 1 (no batching).

Parameters:

batchSize - number of records to batch (must be > 0)

Returns:

this builder

timeoutMs

public InferenceConfig.Builder timeoutMs(long timeoutMs)

Sets the inference timeout in milliseconds.

Parameters:

timeoutMs - timeout in milliseconds

Returns:

this builder

timeout

public InferenceConfig.Builder timeout(Duration duration)

Sets the inference timeout using Duration.

Parameters:

duration - timeout duration

Returns:

this builder

maxRetries

public InferenceConfig.Builder maxRetries(int maxRetries)

Sets maximum retry attempts for failed inferences.

Parameters:

maxRetries - maximum retries (0 for no retries)

Returns:

this builder

enableMetrics

public InferenceConfig.Builder enableMetrics(boolean enableMetrics)

Enables or disables metrics collection.

When enabled, collects latency, throughput, and error metrics.

Parameters:

enableMetrics - true to enable metrics

Returns:

this builder

engineOptions

public InferenceConfig.Builder engineOptions(Map<String,Object> engineOptions)

Sets engine-specific options.

Options vary by engine (TensorFlow, ONNX, PyTorch, etc.)

Parameters:

engineOptions - map of option name-value pairs

Returns:

this builder

build

public InferenceConfig build()

Builds the InferenceConfig instance.

Returns:

configured InferenceConfig

Throws:

NullPointerException - if modelConfig is not set