curl -fsSL https://ollama.com/install.sh | sh安装完成后验证:
ollama --versionollama run llama3✅ 默认是 Llama 3 8B
✅ 首次会自动下载模型(约 4.7GB)
ollama run llama3:8b
ollama run llama3:70b # 需要大显存ollama list # 查看已下载模型
ollama rm llama3 # 删除模型
ollama serve # 启动 API 服务pip install ollamaimport ollama
response = ollama.chat(
model='llama3',
messages=[{'role': 'user', 'content': '你好'}]
)
print(response['message']['content'])适合有 GPU + CUDA 的环境
pip install transformers accelerate torchfrom transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto"
)
inputs = tokenizer("你好", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))⚠️ 需要:
| 模型 | 最低显存 |
|---|---|
| Llama 3 8B | 8GB(可跑) |
| Llama 3 70B | 40GB+ |
export OLLAMA_MODELS=/path/to/big/disk可以换成:
ollama run llama3-chinesecurl -fsSL https://ollama.com/install.sh | sh
ollama run llama3如果你告诉我:
我可以给你更精确的方案。