Package com.codedstreams.otterstreams.sql.async

Class AsyncInferenceExecutor

java.lang.Object

com.codedstreams.otterstreams.sql.async.AsyncInferenceExecutor

public class AsyncInferenceExecutor
extends Object

AsyncInferenceExecutor provides a simple abstraction for executing tasks asynchronously using a fixed-size thread pool. It is designed for running inference or other long-running tasks concurrently without blocking the main thread.

This class manages an internal ExecutorService and allows submission of tasks as Supplier instances. Results of tasks are returned as CompletableFuture objects, enabling asynchronous processing and easy composition of multiple tasks.

Typical usage:

 AsyncInferenceExecutor executor = new AsyncInferenceExecutor(4);

 // Submit a single task
 CompletableFuture<ResultType> future = executor.submit(() -> performInference(input));
 future.thenAccept(result -> handleResult(result))
       .exceptionally(ex -> { handleError(ex); return null; });

 // Submit multiple tasks and combine results
 List<Supplier<ResultType>> tasks = List.of(
     () -> performInference(input1),
     () -> performInference(input2),
     () -> performInference(input3)
 );

 List<CompletableFuture<ResultType>> futures = tasks.stream()
     .map(task -> executor.submit(task)
                         .exceptionally(ex -> {
                             handleError(ex); // Handle exception for this task
                             return null; // Or return a default value
                         }))
     .collect(Collectors.toList());

 CompletableFuture<Void> allDone = CompletableFuture.allOf(
     futures.toArray(new CompletableFuture[0])
 );

 allDone.thenRun(() -> {
     List<ResultType> results = futures.stream()
         .map(CompletableFuture::join) // join will return null for failed tasks
         .collect(Collectors.toList());
     handleResults(results);
 });

 executor.shutdown();
 

Notes:

• Exceptions thrown by individual tasks can be handled using CompletableFuture.exceptionally(java.util.function.Function<java.lang.Throwable, ? extends T>).
• Using CompletableFuture.join() will propagate unchecked exceptions; returning a default value in exceptionally is a safe way to avoid stopping the whole batch.
• The shutdown() method should be called when no further tasks need to be submitted.

Constructor Summary

Constructors

Constructor	Description
AsyncInferenceExecutor(int poolSize)	Creates a new AsyncInferenceExecutor with a fixed-size thread pool.

Method Summary

Modifier and Type	Method	Description
void	shutdown()	Shuts down the executor, preventing new tasks from being submitted.
<T> CompletableFuture<T>	submit(Supplier<T> task)	Submits a task for asynchronous execution.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AsyncInferenceExecutor

public AsyncInferenceExecutor(int poolSize)

Creates a new AsyncInferenceExecutor with a fixed-size thread pool.

Parameters:

poolSize - the number of threads to maintain in the pool

Method Detail

submit

public <T> CompletableFuture<T> submit(Supplier<T> task)

Submits a task for asynchronous execution.

The task is executed on the internal thread pool, and the result is returned as a CompletableFuture. Any exception thrown during execution will be captured by the CompletableFuture and can be handled using CompletableFuture.exceptionally(java.util.function.Function).

Type Parameters:

T - the type of the result produced by the task

Parameters:

task - a Supplier representing the task to execute

Returns:

a CompletableFuture representing the pending completion of the task

shutdown

public void shutdown()

Shuts down the executor, preventing new tasks from being submitted.

Previously submitted tasks continue to execute. Use this method when you no longer need to submit any tasks to the executor. For immediate shutdown of running tasks, consider using ExecutorService.shutdownNow() on a custom implementation.