public/model-qwen2-5-72b-instruct-gguf
62
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Cherrytest 2024-09-18 15:58:28 +00:00
parent f9998697de
commit d31b9cff50
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -42,14 +42,14 @@ Check out our [llama.cpp documentation](https://qwen.readthedocs.io/en/latest/ru
We advise you to clone [`llama.cpp`](https://github.com/ggerganov/llama.cpp) and install it following the official guide. We follow the latest version of llama.cpp.
In the following demonstration, we assume that you are running commands under the repository `llama.cpp`.
Since cloning the entire repo may be inefficient, you can manually download the GGUF file that you need or use `huggingface-cli`:
Since cloning the entire repo may be inefficient, you can manually download the GGUF file that you need or use `modelscope`:
1. Install
```shell
pip install -U huggingface_hub
pip install -U modelscope
```
2. Download:
```shell
huggingface-cli download Qwen/Qwen2.5-72B-Instruct-GGUF --include "qwen2.5-72b-instruct-q5_k_m*.gguf" --local-dir . --local-dir-use-symlinks False
modelscope download --model=qwen/Qwen2.5-72B-Instruct-GGUF --include "qwen2.5-72b-instruct-q5_k_m*.gguf" --local_dir .
```
For large files, we split them into multiple segments due to the limitation of file upload. They share a prefix, with a suffix indicating its index. For examples, `qwen2.5-72b-instruct-q5_k_m-00001-of-00014.gguf` to `qwen2.5-72b-instruct-q5_k_m-00014-of-00014.gguf`. The above command will download all of them.
3. (Optional) Merge: