NexVeridian/nexveridian-web
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

dwq

Browse source
This commit is contained in:
Elijah McMorris 2025-07-15 22:06:20 -07:00
parent 7f13e0ff91
commit 47b2ebf74c
Signed by: NexVeridian
SSH key fingerprint: SHA256:bsA1SKZxuEcEVHAy3gY1HUeM5ykRJl0U0kQHQn0hMg8
2 changed files with 80 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

12
content/blog/llm-inference-benchmarks.md
View file

@ -56,12 +56,14 @@ https://huggingface.co/mlx-community
git clone git@github.com:NexVeridian/NexVeridian-web.git
```bash
just uv
just mlx_create "Qwen/QwQ-32B" "4 6 8" "/Users/elijahmcmorris/.cache/lm-studio/models" "mlx-community" fasle false
# or
uv venv
uv pip install huggingface_hub hf_transfer mlx_lm
uv run huggingface-cli login
just mlx_create "Qwen/QwQ-32B" "4 6 8" "/Users/elijahmcmorris/.cache/lm-studio/models" "mlx-community" "false"
# or
uv run mlx_lm.convert --hf-path Qwen/QwQ-32B -q --q-bits 4 --upload-repo mlx-community/QwQ-32B-4bit --mlx-path /Users/elijahmcmorris/.cache/lm-studio/models/mlx-community/QwQ-32B-4bit
```
or use https://huggingface.co/spaces/mlx-community/mlx-my-repo
@ -77,12 +79,6 @@ or use https://huggingface.co/spaces/mlx-community/mlx-my-repo
|------|-------|-------|-------|----------|
| 0.7 | 0.00 | 0.80 | 20 | 1.5 |
## QWQ
| Temp | Min P | Top P | Repeat P |
|------|-------|-------|----------|
| 0.7 | 0.05 | 0.95 | - |
| 1.5 | 0.10 | 1.00 | - |
## Prompt Template
```
{%- if tools %}