Text Generation
Transformers
Safetensors
qwen3_moe
european
nordic
norwegian
swedish
danish
icelandic
multilingual
Mixture of Experts
qwen3
conversational
Instructions to use bineric/lynx-instruct-30b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bineric/lynx-instruct-30b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bineric/lynx-instruct-30b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bineric/lynx-instruct-30b") model = AutoModelForCausalLM.from_pretrained("bineric/lynx-instruct-30b") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bineric/lynx-instruct-30b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bineric/lynx-instruct-30b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bineric/lynx-instruct-30b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/bineric/lynx-instruct-30b
- SGLang
How to use bineric/lynx-instruct-30b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bineric/lynx-instruct-30b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bineric/lynx-instruct-30b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bineric/lynx-instruct-30b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bineric/lynx-instruct-30b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use bineric/lynx-instruct-30b with Docker Model Runner:
docker model run hf.co/bineric/lynx-instruct-30b
File size: 3,290 Bytes
50b9f98 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 | <svg width="700" height="400" xmlns="http://www.w3.org/2000/svg">
<style>
.title { font: bold 18px sans-serif; fill: #333; }
.label { font: 12px sans-serif; fill: #333; }
.value { font: 11px sans-serif; fill: #333; }
.axis { font: 11px sans-serif; fill: #666; }
.legend { font: 13px sans-serif; fill: #333; }
</style>
<!-- Background -->
<rect width="700" height="400" fill="#fafafa"/>
<!-- Title -->
<text x="350" y="30" text-anchor="middle" class="title">8-bit vs 4-bit Quantization (Norwegian)</text>
<!-- Legend -->
<rect x="480" y="50" width="16" height="16" fill="#3b82f6"/>
<text x="502" y="63" class="legend">8-bit</text>
<rect x="560" y="50" width="16" height="16" fill="#94a3b8"/>
<text x="582" y="63" class="legend">4-bit</text>
<!-- Y-axis labels -->
<text x="70" y="100" text-anchor="end" class="label">Sentiment</text>
<text x="70" y="145" text-anchor="end" class="label">NER (nb)</text>
<text x="70" y="190" text-anchor="end" class="label">NER (nn)</text>
<text x="70" y="235" text-anchor="end" class="label">Reading</text>
<text x="70" y="280" text-anchor="end" class="label">Summary</text>
<text x="70" y="325" text-anchor="end" class="label">Common Sense</text>
<text x="70" y="370" text-anchor="end" class="label">Ling. Accept.</text>
<!-- Bars - Sentiment -->
<rect x="80" y="85" width="306" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="105" width="298.2" height="18" fill="#94a3b8" rx="2"/>
<text x="392" y="98" class="value">51.0%</text>
<text x="384" y="118" class="value">49.7%</text>
<!-- Bars - NER nb -->
<rect x="80" y="130" width="394.2" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="150" width="390.6" height="18" fill="#94a3b8" rx="2"/>
<text x="480" y="143" class="value">65.7%</text>
<text x="476" y="163" class="value">65.1%</text>
<!-- Bars - NER nn -->
<rect x="80" y="175" width="426" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="195" width="419.4" height="18" fill="#94a3b8" rx="2"/>
<text x="512" y="188" class="value">71.0%</text>
<text x="505" y="208" class="value">69.9%</text>
<!-- Bars - Reading -->
<rect x="80" y="220" width="367.2" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="240" width="353.4" height="18" fill="#94a3b8" rx="2"/>
<text x="453" y="233" class="value">61.2%</text>
<text x="439" y="253" class="value">58.9%</text>
<!-- Bars - Summary -->
<rect x="80" y="265" width="380.4" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="285" width="378.6" height="18" fill="#94a3b8" rx="2"/>
<text x="466" y="278" class="value">63.4%</text>
<text x="464" y="298" class="value">63.1%</text>
<!-- Bars - Common Sense -->
<rect x="80" y="310" width="415.8" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="330" width="411" height="18" fill="#94a3b8" rx="2"/>
<text x="501" y="323" class="value">69.3%</text>
<text x="497" y="343" class="value">68.5%</text>
<!-- Bars - Linguistic Acceptability (biggest difference) -->
<rect x="80" y="355" width="218.4" height="18" fill="#3b82f6" rx="2"/>
<rect x="80" y="375" width="178.8" height="18" fill="#94a3b8" rx="2"/>
<text x="304" y="368" class="value">36.4% (+6.6%)</text>
<text x="264" y="388" class="value">29.8%</text>
</svg> |