云信AI部署

Text Generation Inference实战：如何用Java开发高效知识库应用

云信安装大师
90
AI 质量分
11 5 月, 2025
7 分钟阅读
0 阅读

Text Generation Inference实战：如何用Java开发高效知识库应用

引言

在当今信息爆炸的时代，构建高效的知识库应用成为企业和个人的迫切需求。Text Generation Inference (TGI) 是Hugging Face推出的高性能文本生成推理服务，结合Java强大的生态系统，我们可以开发出既高效又易于维护的知识库应用。本文将带你从零开始，使用Java和TGI构建一个完整的知识库应用。

准备工作

环境要求

  1. Java 11或更高版本
  2. Maven 3.6+
  3. Docker (用于运行TGI服务)
  4. 至少16GB内存(运行大型语言模型需要)

前置知识

  • 基础的Java编程能力
  • REST API的基本概念
  • 了解基本的自然语言处理概念

第一步：搭建TGI服务

1.1 安装Docker

如果你还没有安装Docker，请先安装：

代码片段 
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

# CentOS/RHEL
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce docker-ce-cli containerd.io

1.2 启动TGI服务

我们使用Hugging Face提供的官方镜像：

代码片段 
docker run -d --name tgi \
  -p 8080:80 \
  -e MODEL_ID=google/flan-t5-large \
  ghcr.io/huggingface/text-generation-inference:latest

参数说明：
– MODEL_ID: 指定要使用的模型，这里使用Google的flan-t5-large模型
– 8080:80: 将容器内部的80端口映射到主机的8080端口

实践经验：对于生产环境，建议使用GPU加速的版本并配置适当的资源限制：

代码片段 
docker run -d --gpus all --shm-size 1g \
  -p 8080:80 \
  -e MODEL_ID=google/flan-t5-large \
  ghcr.io/huggingface/text-generation-inference:latest

第二步：创建Java项目

2.1 Maven项目初始化

创建一个新的Maven项目，添加以下依赖到pom.xml：

代码片段 
<dependencies>
    <!-- HTTP客户端 -->
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.5.13</version>
    </dependency>

    <!-- JSON处理 -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.13.3</version>
    </dependency>

    <!-- Lombok简化代码 -->
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.24</version>
        <scope>provided</scope>
    </dependency>

    <!-- Logging -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.7.36</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.2.11</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <!-- Java版本设置 -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>11</source>
                <target>11</target>
            </configuration>
        </plugin>
    </plugins>
</build>

第三步：实现TGI客户端

3.1 DTO类定义

首先定义请求和响应的DTO类：

代码片段 
import lombok.Data;

@Data
public class TGIRequest {
    private String inputs;           // 输入的文本提示
    private Parameters parameters;   // 生成参数

    @Data
    public static class Parameters {
        private Integer max_new_tokens =  200;   //最大生成token数
        private Double temperature =  0.7;      //温度参数(控制随机性)
        private Double top_p =  0.9;            //top-p采样参数

        // Getters and setters...

        public Parameters() {}

        public Parameters(Integer max_new_tokens, Double temperature, Double top_p) {
            this.max_new_tokens = max_new_tokens;
            this.temperature = temperature;
            this.top_p = top_p;
        }
    }
}

@Data 
public class TGIResponse {
    private String generated_text;   //生成的文本

    // Getters and setters...
}

3.2 HTTP客户端实现

创建一个封装了TGI API调用的服务类：

代码片段 
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import com.fasterxml.jackson.databind.ObjectMapper;

public class TGIClient {

    private static final String TGI_API_URL = "http://localhost:8080/generate";

    private final ObjectMapper objectMapper = new ObjectMapper();

    public String generateText(String prompt) throws Exception {

        // Prepare request payload with default parameters
        TGIRequest request = new TGIRequest();
        request.setInputs(prompt);

        // Create HTTP client and request object
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost httpPost = new HttpPost(TGI_API_URL);

            // Set headers and body
            httpPost.setHeader("Content-Type", "application/json");
            httpPost.setEntity(new StringEntity(objectMapper.writeValueAsString(request)));

            // Execute request and process response
            try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
                HttpEntity entity = response.getEntity();
                if (entity != null) {
                    String result = EntityUtils.toString(entity);
                    TGIResponse tgiResponse = objectMapper.readValue(result, TGIResponse.class);
                    return tgiResponse.getGenerated_text();
                }
                throw new RuntimeException("Empty response from TGI server");
            }

         } catch (Exception e) {
             throw new RuntimeException("Error calling TGI service", e);
         }
     }
}

原理说明：
– max_new_tokens:控制生成文本的最大长度(以token为单位)
– temperature:控制输出的随机性(值越高输出越多样)
– top_p:核采样参数，控制候选词的概率分布范围

第四步：构建知识库应用核心逻辑

4.1 KnowledgeBaseService实现

代码片段 
import java.util.HashMap;
import java.util.Map;

public class KnowledgeBaseService {

    private final TGIClient tgiClient;

    // In-memory knowledge store (for demo purposes)
    private final Map<String, String> knowledgeStore = new HashMap<>();

    public KnowledgeBaseService(TGIClient tgiClient) {
         this.tgiClient = tgiClient;
         initializeSampleData();
     }

     private void initializeSampleData() {
         knowledgeStore.put("company_history", 
             "Our company was founded in  2010 by John Smith and Jane Doe.");
         knowledgeStore.put("products", 
             "We offer three main products: AI Assistant, Data Analyzer, and Cloud Storage.");
         knowledgeStore.put("contact", 
             "You can reach us at support@example.com or call +123456789.");
     }

     public String queryKnowledgeBase(String question) throws Exception {
         // Step  1: Analyze the question to determine the context/keyword(s)
         String contextPrompt = "Extract the main keyword from this question for knowledge base lookup:\n" + 
                               question + "\nKeyword:";

         String keyword = tgiClient.generateText(contextPrompt).trim().toLowerCase();

         System.out.println("Detected keyword: " + keyword);

         // Step  2: Check if we have direct answer in knowledge store
         if (knowledgeStore.containsKey(keyword)) { 
             return knowledgeStore.get(keyword);
         }

         // Step  3: If no direct answer, use LLM to generate answer based on available data 
         StringBuilder promptBuilder = new StringBuilder();
         promptBuilder.append("Answer the following question based on the given context:\n");
         promptBuilder.append("Question:").append(question).append("\n");

         if (!knowledgeStore.isEmpty()) { 
             promptBuilder.append("Context:\n");
             knowledgeStore.forEach((k, v) -> promptBuilder.append(k).append(":").append(v).append("\n"));
          }

          return tgiClient.generateText(promptBuilder.toString());
      }
}

实践经验：
1. 关键词提取：先让LLM提取问题的关键词，可以提高知识库匹配的准确性。
2. 分层回答：优先从结构化知识库中获取答案，没有匹配时再使用LLM生成。
3. 上下文构建：将相关知识作为上下文提供给LLM可以提高回答质量。

第五步：创建主程序测试功能

Main.java实现示例

代码片段 
public class Main {

     public static void main(String[] args) { 
          try { 
               TGIClient client = new TGIClient();
               KnowledgeBaseService kbService = new KnowledgeBaseService(client);

               System.out.println("Knowledge Base Application Started");
               System.out.println("Type 'exit' to quit");

               Scanner scanner = new Scanner(System.in); 
               while (true) { 
                   System.out.print("\nAsk a question > ");
                   String input = scanner.nextLine(); 

                   if ("exit".equalsIgnoreCase(input)) { 
                       break; 
                   } 

                   try { 
                       long startTime = System.currentTimeMillis();  
                       String answer = kbService.queryKnowledgeBase(input);  
                       long endTime = System.currentTimeMillis();  

                       System.out.println("\nAnswer:");  
                       System.out.println(answer);  
                       System.out.printf("\nGenerated in %d ms\n", endTime - startTime);  
                   } catch (Exception e) {  
                       System.err.println("Error processing your question: " + e.getMessage());  
                   }  
               }  

               scanner.close();  
           } catch (Exception e) {  
               e.printStackTrace();  
           }  
      }  
}

API性能优化技巧

为了提高响应速度和服务稳定性，我们可以采用以下优化策略：

缓存层实现

代码片段 
// Add to KnowledgeBaseService.java 

private final Map<String, CacheEntry<String>> responseCache;

public KnowledgeBaseService(TGIClient tgiClient) {
     this.tgiClient = tgiClient;
     this.responseCache = new ConcurrentHashMap<>();
     initializeSampleData();
}

private static class CacheEntry<V> {
     V value;
     long timestamp;

     CacheEntry(V value) {
          this.value = value;
          this.timestamp = System.currentTimeMillis();
     }
}

public String queryKnowledgeBase(String question) throws Exception { 
     // Check cache first (simple hash of the question as key)
     String cacheKey = Integer.toHexString(question.hashCode());

     CacheEntry<String> cachedAnswer = responseCache.get(cacheKey);

     if (cachedAnswer != null && !isCacheExpired(cachedAnswer)) { 
          return cachedAnswer.value + "\n[From cache]";
      }

      // ... original processing logic ...

      // Store in cache before returning with a default expiry of  5 minutes   
      responseCache.put(cacheKey, new CacheEntry<>(answer));

      return answer;   
}  

private boolean isCacheExpired(CacheEntry<?> entry) {   
     return System.currentTimeMillis() - entry.timestamp > TimeUnit.MINUTES.toMillis(5);   
}

批处理请求

对于高并发场景，可以批量发送请求以提高吞吐量：

代码片段 
// Add to TGIClient.java  

public List<String> batchGenerate(List<String> prompts) throws Exception {   

     List<TGIRequestBatchItem<T>> batchItems =
          prompts.stream()
              .map(prompt -> new TGIRequestBatchItem<>(prompt))
              .collect(Collectors.toList());

      try (CloseableHttpClient httpClient =
           HttpClients.custom()
              .setMaxConnPerRoute(20)
              .setMaxConnTotal(100)
              .build()) {

           HttpPost httpPost =
              new HttpPost(TGI_API_URL + "/batch");  

           httpPost.setHeader(
              "Content-Type",
              "application/json");  

           httpPost.setEntity(
              new StringEntity(
                  objectMapper.writeValueAsString(batchItems)));

           try (CloseableHttpResponse response =
                httpClient.execute(httpPost)) {

                HttpEntity entity =
                   response.getEntity();

                if (entity != null) {

                     List<TGIBatchResponseItem<T>> batchResponses =
                        Arrays.asList(
                            objectMapper.readValue(
                                EntityUtils.toString(entity),
                                TGIBatchResponseItem[].class));

                     return batchResponses.stream()
                        .map(TGIBatchResponseItem::getGenerated_text)
                        .collect(Collectors.toList());

                 }

                 throw new RuntimeException(
                    "Empty batch response from server");

             }

       } catch (Exception ex) {

           throw ex;

       }

}

部署架构建议

对于生产环境部署，推荐采用以下架构：

代码片段 
[Load Balancer]
       |
       v   
[API Gateway] -> [Auth Service]   
       |   
       v   
[Application Servers] -> [Redis Cache]   
       |   
       v   
[TGI Cluster] <- [Model Registry]

关键组件说明：
– 负载均衡器:分发流量到多个应用服务器实例。
– API网关:处理认证、限流、日志等横切关注点。
– Redis缓存:存储高频查询结果减少LLM调用。
– TGI集群:多实例部署确保高可用性。

常见问题解决

Q1:TGI服务响应慢怎么办？

A:
1️⃣检查Docker资源分配是否充足：

代码片段 
docker stats #查看容器资源使用情况。

2️⃣降低模型精度换取速度：

代码片段 
docker run ... -e QUANTIZE=bitsandbytes-nf4 ...

3️⃣启用连续批处理：

代码片段 
docker run ... -e MAX_BATCH_PREFILL_TOKENS=2048 ...

Q2:如何提高答案准确性？

A:
🔹优化提示工程:

代码片段 
//在KnowledgeBaseService中添加更多上下文提示:
String systemPrompt="你是一个专业的知识库助手..."+questionContext;.

🔹实施RAG模式:
“`java.
//检索增强生成示例:
List=vectorStore.search(questionEmbedding);.
String augmentedPrompt=combine(documents,question);.
return tgi.generate(augmentedPrompt);.

代码片段 
🔹添加后处理校验:
```java.
//验证生成的答案是否合理.
if(!validateAnswer(answer)){
return fallbackResponse();.
}.

总结

本文详细介绍了如何使用Java和Text Generation Inference构建高效的知识库应用。关键要点包括:

✅ TGI服务的Docker化部署与配置

✅ Java客户端的完整实现与优化技巧

✅分层知识检索策略的设计

✅性能优化与生产环境建议

通过结合结构化知识存储和LLM的生成能力，我们能够构建既准确又灵活的知识服务系统。下一步可以考虑集成向量数据库实现真正的RAG架构。

完整项目代码已托管至GitHub:your-repo-link(示例链接)。

原创 高质量