Skip to main content

Oobabooga Text Web API Tutorial

Install + Import LiteLLM​

!uv add litellm
from litellm import completion 
import os

Call your oobabooga model​

Remember to set your api_base

response = completion(
  model="oobabooga/WizardCoder-Python-7B-V1.0-GPTQ",
  messages=[{ "content": "can you write a binary tree traversal preorder","role": "user"}], 
  api_base="http://localhost:5000",
  max_tokens=4000
)

See your response​

print(response)

Credits to Shuai Shao, for this tutorial.

πŸš…
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails β€” built for production.
Learn more β†’