Class GoogleAiGeminiBatchChatModel

java.lang.Object
dev.langchain4j.model.googleai.GoogleAiGeminiBatchChatModel
@Experimental public final class GoogleAiGeminiBatchChatModel extends Object
Provides an interface for interacting with the Gemini Batch API, an asynchronous service designed for processing large volumes of requests at a reduced cost (50% of standard). It is ideal for non-urgent, large-scale tasks like data pre-processing or evaluations, with a Service Level Objective (SLO) of 24-hour turnaround, though completion is often much quicker.

  • Method Details

    • createBatchInline

      public BatchRequestResponse.BatchResponse<dev.langchain4j.model.chat.response.ChatResponse> createBatchInline(String displayName, @Nullable Long priority, List<dev.langchain4j.model.chat.request.ChatRequest> requests)
      Creates and enqueues a batch of content generation requests for asynchronous processing.

      This method submits multiple chat requests as a single batch operation to the Gemini API. All requests in the batch must use the same model. The batch will be processed asynchronously, and the initial response will typically be in a BatchRequestResponse.BatchIncomplete state.

      Batch processing offers a 50% cost reduction compared to real-time requests and has a 24-hour turnaround SLO, making it ideal for large-scale, non-urgent tasks.

      Note: The inline API allows for a total request size of 20MB or under. Larger requests should use the File API

      Parameters:
      displayName - a user-defined name for the batch, used for identification
      priority - optional priority for the batch; batches with higher priority values are processed before those with lower values; negative values are allowed; defaults to 0 if null
      requests - a list of chat requests to be processed in the batch; all requests must use the same model
      Returns:
      a BatchRequestResponse.BatchResponse representing the initial state of the batch operation, typically BatchRequestResponse.BatchIncomplete
      Throws:
      IllegalArgumentException - if the requests contain different models

    • createBatchFromFile

      public BatchRequestResponse.BatchResponse<dev.langchain4j.model.chat.response.ChatResponse> createBatchFromFile(String displayName, GeminiFiles.GeminiFile file)
      Creates a batch of chat requests from an uploaded file.

      This method allows you to create a batch job using a JSONL file that has been previously uploaded to the Gemini Files API. This is useful for larger batches that exceed the 20 MB inline request limit.

      The file must contain batch requests in JSONL format, where each line is a JSON object with a "key" and "request" field. You can use writeBatchToFile(JsonLinesWriter, Iterable) to create properly formatted JSONL files.

      Parameters:
      displayName - a user-defined name for the batch, used for identification
      file - the GeminiFile object representing the uploaded file containing batch requests
      Returns:
      a BatchRequestResponse.BatchResponse representing the initial state of the batch operation, typically BatchRequestResponse.BatchIncomplete
      See Also:

    • writeBatchToFile

      public void writeBatchToFile(JsonLinesWriter writer, Iterable<BatchRequestResponse.BatchFileRequest<dev.langchain4j.model.chat.request.ChatRequest>> requests) throws IOException
      Writes a batch of chat requests to a JSONL file for later upload and processing.

      This method serializes chat requests into JSONL (JSON Lines) format, where each line contains a single request wrapped in a BatchRequestResponse.BatchFileRequest with a unique key. The resulting file can be uploaded using the Gemini Files API and then used to create a batch job via createBatchFromFile(String, GeminiFile).

      Each request is converted to the internal Gemini format before being written to the file.

      Example usage:

      Path batchFile = Files.createTempFile("batch", ".jsonl");
try (JsonLinesWriter writer = new StreamingJsonLinesWriter(batchFile)) {
    List<BatchFileRequest<ChatRequest>> requests = List.of(
        new BatchFileRequest<>("request-1", ChatRequest.builder()
            .messages(UserMessage.from("Question 1"))
            .build()),
        new BatchFileRequest<>("request-2", ChatRequest.builder()
            .messages(UserMessage.from("Question 2"))
            .build())
    );
    batchModel.writeBatchToFile(writer, requests);
}
      Parameters:
      writer - the JsonLinesWriter to which the batch requests will be written
      requests - an iterable collection of BatchFileRequest objects containing ChatRequest instances, each with a unique key identifier
      Throws:
      IOException - if an I/O error occurs while writing to the writer
      See Also:

    • retrieveBatchResults

      public BatchRequestResponse.BatchResponse<dev.langchain4j.model.chat.response.ChatResponse> retrieveBatchResults(BatchRequestResponse.BatchName name)
      Retrieves the current state and results of a batch operation.

      This method polls the Gemini API to get the latest state of a previously created batch. The response can be:

      Clients should poll this method at intervals to check the operation status until completion.

      Parameters:
      name - the name of the batch operation to retrieve, obtained from the initial createBatchInline(String, Long, List) call
      Returns:
      a BatchRequestResponse.BatchResponse representing the current state of the batch operation

    • cancelBatchJob

      public void cancelBatchJob(BatchRequestResponse.BatchName name)
      Cancels a batch operation that is currently pending or running.

      This method attempts to cancel a batch job. Cancellation is only possible for batches that are in BatchRequestResponse.BatchJobState.BATCH_STATE_PENDING or BatchRequestResponse.BatchJobState.BATCH_STATE_RUNNING state. Batches that have already completed, failed, or been cancelled cannot be cancelled.

      Parameters:
      name - the name of the batch operation to cancel
      Throws:
      dev.langchain4j.exception.HttpException - if the batch cannot be cancelled (e.g., already completed, already cancelled, or does not exist)

    • deleteBatchJob

      public void deleteBatchJob(BatchRequestResponse.BatchName name)
      Deletes a batch job from the system.

      This removes the batch job but does not cancel it if still running. Use cancelBatchJob(BatchName) to cancel a running batch.

      Parameters:
      name - the name of the batch job to delete
      Throws:
      RuntimeException - if the batch job cannot be deleted or does not exist

    • listBatchJobs

      public BatchRequestResponse.BatchList<dev.langchain4j.model.chat.response.ChatResponse> listBatchJobs(@Nullable Integer pageSize, @Nullable String pageToken)
      Lists batch jobs with optional pagination.
      Parameters:
      pageSize - the maximum number of batch jobs to return; if null, uses server default
      pageToken - token for retrieving a specific page from BatchRequestResponse.BatchList.pageToken(); if null, returns the first page
      Returns:
      a BatchRequestResponse.BatchList containing batch responses and a token for the next page
      Throws:
      RuntimeException - if the server does not support this operation

    • builder

      public static GoogleAiGeminiBatchChatModel.Builder builder()