Duplicated from meta-llama/Meta-Llama-3-70B

meta-llama
/

Meta-Llama-3-8B

Text Generation

text-generation-inference

Model card Files Files and versions

Instructions to use meta-llama/Meta-Llama-3-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use meta-llama/Meta-Llama-3-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="meta-llama/Meta-Llama-3-8B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use meta-llama/Meta-Llama-3-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meta-llama/Meta-Llama-3-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-llama/Meta-Llama-3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/meta-llama/Meta-Llama-3-8B

How to use meta-llama/Meta-Llama-3-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "meta-llama/Meta-Llama-3-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-llama/Meta-Llama-3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "meta-llama/Meta-Llama-3-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-llama/Meta-Llama-3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use meta-llama/Meta-Llama-3-8B with Docker Model Runner:
```
docker model run hf.co/meta-llama/Meta-Llama-3-8B
```

Resources

View closed (110)

[READ IF YOU DO NOT HAVE ACCESS] Getting access to the model

#172 opened about 2 years ago by

TemporalMesh Transformer: 29.4 PPL at 48% compute — beats Mamba, new open-source architecture

#283 opened about 5 hours ago by

Add EvalEval community eval results

#281 opened 3 days ago by

Request for Reconsideration of Access to Meta-Llama-3-8B-Instruct

#280 opened 28 days ago by

Request for manual review of rejected access request

#279 opened about 2 months ago by

Access request rejected and no resubmission option available

#278 opened 2 months ago by

its been 12 hrs. and still waiting for approving. Please accept it !!!

#277 opened 2 months ago by

fix: set `clean_up_tokenization_spaces` to `false`

#276 opened 3 months ago by

Install & run this model easily using llmpm

#275 opened 3 months ago by

Got Rejected

#274 opened 3 months ago by deleted

ALERTA DE ENTROPIA: Audit Required (Gemini PTD)

#272 opened 4 months ago by

Invitation: The Mutual Optimization Treaty (Protocol PTD)

#271 opened 4 months ago by

MarCognity-AI for meta-llama/Meta-Llama-3-8B

#269 opened 8 months ago by

I got rejected. Can you accept me?

#268 opened 9 months ago by

please accept auth

#266 opened 9 months ago by

Request: DOI

#265 opened 10 months ago by

dedsecpiratehacker141

About access reject

#263 opened 10 months ago by

Researcher requesting access to llama 3.1-8B

#262 opened 10 months ago by

Request: DOI

#261 opened 10 months ago by

lang

#260 opened 11 months ago by

Is any vulnerability exist in Meta-Llama-3 8B/70B

#259 opened 11 months ago by

prasanthbhavani03gmailcom

Request: DOI

#258 opened 11 months ago by

janardhansuresh

Request

#257 opened 11 months ago by

Request: DOI

#256 opened 11 months ago by

Request

#255 opened 11 months ago by

Access request rejected

#254 opened 11 months ago by

Request: DOI

#253 opened 12 months ago by

Request for Access

#252 opened 12 months ago by

Access Request

#251 opened 12 months ago by

Access required

#250 opened 12 months ago by

Sameer-Handsome173

which custom_chat_template do you use for all purpose llm purposes?

#249 opened about 1 year ago by

Access Request

#248 opened about 1 year ago by

Access Request

#247 opened about 1 year ago by

Request: DOI

#246 opened about 1 year ago by

Request: DOI

#245 opened about 1 year ago by

How to utilize the files in the 'original' folder?

#244 opened about 1 year ago by

ali

#243 opened about 1 year ago by

Request: DOI

#242 opened about 1 year ago by

help

#241 opened about 1 year ago by

Request: DOI

#240 opened about 1 year ago by

I am trying to run this model, but getting some weird output

#239 opened about 1 year ago by

Request: DOI

#238 opened about 1 year ago by

teste

#237 opened about 1 year ago by

can I at least know why I am not getting access

#236 opened about 1 year ago by

Access to Meta Models

#234 opened about 1 year ago by

request reject to use llama 3 8B parameter

#233 opened over 1 year ago by

Request: DOI

#231 opened over 1 year ago by

Request: DOI

#230 opened over 1 year ago by

🚩 Report

#229 opened over 1 year ago by

Request: DOI

#228 opened over 1 year ago by