Update README.md
This commit is contained in:
parent
8874310d3a
commit
81b9058705
18
README.md
18
README.md
@ -83,21 +83,24 @@ print("thinking content:", thinking_content)
|
||||
print("content:", content)
|
||||
```
|
||||
|
||||
For deployment, you can use `vllm>=0.8.5` or `sglang>=0.4.5.post2` to create an OpenAI-compatible API endpoint:
|
||||
For deployment, you can use `sglang>=0.4.6.post1` or `vllm>=0.8.4` or to create an OpenAI-compatible API endpoint:
|
||||
- SGLang:
|
||||
```shell
|
||||
python -m sglang.launch_server --model-path Qwen/Qwen3-0.6B --reasoning-parser qwen3
|
||||
```
|
||||
- vLLM:
|
||||
```shell
|
||||
vllm serve Qwen/Qwen3-0.6B --enable-reasoning --reasoning-parser deepseek_r1
|
||||
```
|
||||
- SGLang:
|
||||
```shell
|
||||
python -m sglang.launch_server --model-path Qwen/Qwen3-0.6B --reasoning-parser deepseek-r1
|
||||
```
|
||||
|
||||
For local use, applications such as llama.cpp, Ollama, LMStudio, and MLX-LM have also supported Qwen3.
|
||||
|
||||
|
||||
## Switching Between Thinking and Non-Thinking Mode
|
||||
|
||||
> [!TIP]
|
||||
> The `enable_thinking` switch is also available in APIs created by vLLM and SGLang.
|
||||
> Please refer to our documentation for [vLLM](https://qwen.readthedocs.io/en/latest/deployment/vllm.html#thinking-non-thinking-modes) and [SGLang](https://qwen.readthedocs.io/en/latest/deployment/sglang.html#thinking-non-thinking-modes) users.
|
||||
> The `enable_thinking` switch is also available in APIs created by SGLang and vLLM.
|
||||
> Please refer to our documentation for [SGLang](https://qwen.readthedocs.io/en/latest/deployment/sglang.html#thinking-non-thinking-modes) and [vLLM](https://qwen.readthedocs.io/en/latest/deployment/vllm.html#thinking-non-thinking-modes) users.
|
||||
|
||||
### `enable_thinking=True`
|
||||
|
||||
@ -117,7 +120,6 @@ In this mode, the model will generate think content wrapped in a `<think>...</th
|
||||
> [!NOTE]
|
||||
> For thinking mode, use `Temperature=0.6`, `TopP=0.95`, `TopK=20`, and `MinP=0` (the default setting in `generation_config.json`). **DO NOT use greedy decoding**, as it can lead to performance degradation and endless repetitions. For more detailed guidance, please refer to the [Best Practices](#best-practices) section.
|
||||
|
||||
|
||||
### `enable_thinking=False`
|
||||
|
||||
We provide a hard switch to strictly disable the model's thinking behavior, aligning its functionality with the previous Qwen2.5-Instruct models. This mode is particularly useful in scenarios where disabling thinking is essential for enhancing efficiency.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user