Package com.codedstream.otterstream.inference.metrics

Class InferenceMetrics

  • public class InferenceMetrics
extends Object
    Collects and records metrics for ML inference operations.

    Tracks key performance indicators including:

    • Request counts (total, success, failure)
    • Latency distribution
    • Cache hit/miss rates
    • Batch sizes

    Integration with Monitoring:

    Metrics are exported via Micrometer and can be viewed in:

    • Prometheus + Grafana
    • InfluxDB
    • DataDog
    • New Relic
    • Any Micrometer-compatible backend

    Usage Example:

    
 // Setup with Prometheus
 PrometheusMeterRegistry registry = new PrometheusMeterRegistry(PrometheusConfig.DEFAULT);
 InferenceMetrics metrics = new InferenceMetrics(registry, "fraud-detector");

 // Record inference
 long startTime = System.currentTimeMillis();
 InferenceResult result = engine.infer(inputs);
 long duration = System.currentTimeMillis() - startTime;
 metrics.recordInference(duration, result.isSuccess());

 // Record cache hit
 if (cachedResult != null) {
     metrics.recordCacheHit();
 }
    Since:
    1.0.0
    Author:
    Nestor Martourez, Sr Software and Data Streaming Engineer @ CodedStreams
    See Also:
    MetricsCollector

    • Constructor Detail

      • InferenceMetrics

        public InferenceMetrics​(io.micrometer.core.instrument.MeterRegistry meterRegistry,
                        String modelId)
        Constructs inference metrics collector.
        Parameters:
        meterRegistry - the meter registry for metric export
        modelId - identifier for this model (used in metric tags)

    • Method Detail

      • recordInference

        public void recordInference​(long durationMs,
                            boolean success)
        Records an inference operation with duration and outcome.
        Parameters:
        durationMs - inference duration in milliseconds
        success - whether inference succeeded

      • recordCacheHit

        public void recordCacheHit()
        Records a cache hit event.

      • recordCacheMiss

        public void recordCacheMiss()
        Records a cache miss event.

      • recordBatchSize

        public void recordBatchSize​(int batchSize)
        Records the size of a batch inference operation.
        Parameters:
        batchSize - number of records in the batch