From 395feeb458ce9f8cfa7ca0519adc9d4500688101 Mon Sep 17 00:00:00 2001 From: Fei Huang Date: Mon, 28 Apr 2025 15:37:29 +0000 Subject: [PATCH] Update README.md --- README.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index b1f8b46..7520645 100644 --- a/README.md +++ b/README.md @@ -80,21 +80,25 @@ print("thinking content:", thinking_content) print("content:", content) ``` -For deployment, you can use `vllm>=0.8.5` or `sglang>=0.4.5.post2` to create an OpenAI-compatible API endpoint: +For deployment, you can use `sglang>=0.4.6.post1` or `vllm>=0.8.4` or to create an OpenAI-compatible API endpoint: +- SGLang: + ```shell + python -m sglang.launch_server --model-path Qwen/Qwen3-32B --reasoning-parser qwen3 + ``` - vLLM: ```shell vllm serve Qwen/Qwen3-32B --enable-reasoning --reasoning-parser deepseek_r1 ``` -- SGLang: - ```shell - python -m sglang.launch_server --model-path Qwen/Qwen3-32B --reasoning-parser deepseek-r1 - ``` + +For local use, applications such as llama.cpp, Ollama, LMStudio, and MLX-LM have also supported Qwen3. + + ## Switching Between Thinking and Non-Thinking Mode > [!TIP] -> The `enable_thinking` switch is also available in APIs created by vLLM and SGLang. -> Please refer to our documentation for [vLLM](https://qwen.readthedocs.io/en/latest/deployment/vllm.html#thinking-non-thinking-modes) and [SGLang](https://qwen.readthedocs.io/en/latest/deployment/sglang.html#thinking-non-thinking-modes) users. +> The `enable_thinking` switch is also available in APIs created by SGLang and vLLM. +> Please refer to our documentation for [SGLang](https://qwen.readthedocs.io/en/latest/deployment/sglang.html#thinking-non-thinking-modes) and [vLLM](https://qwen.readthedocs.io/en/latest/deployment/vllm.html#thinking-non-thinking-modes) users. ### `enable_thinking=True`