TogetherAI API examples

This notebook contains examples of how to use the TogetherAI LLM, utilizing models hosted by together.ai.

Completion usage

from guidance import models, gen

# This relies on the environment variable TOGETHERAI_API_KEY being set
mixtral = models.TogetherAI('mistralai/Mixtral-8x7B-v0.1')

lm = mixtral

stop_tokens = [",", "}", "\n"]
temperature = 0.0

lm += f"""The most famous piece of japanese literature in a JSON format is:
{{
    "title_english": {gen(name='title_english', temperature=temperature, max_tokens=50, stop=stop_tokens)},
    "title_japanese": {gen(name='title_japanese', temperature=temperature, max_tokens=50, stop=stop_tokens)},
    "author": {gen(name='author', temperature=temperature, max_tokens=50, stop=stop_tokens)},
    "year": {gen(name='year', temperature=temperature, max_tokens=50, stop=stop_tokens)}
}}
"""

The most famous piece of japanese literature in a JSON format is:
{
    "title_english":  "The Tale of Genji",
    "title_japanese": "源氏物語",
    "author": "Murasaki Shikibu",
    "year": 1008
}

Instruct usage

from guidance import instruction

# This relies on the environment variable TOGETHERAI_API_KEY being set
gemma = models.TogetherAIInstruct('google/gemma-7b-it')

lm = gemma
with instruction():
    lm += "What is ice cream refered to as in Italy?"
lm += gen('flavor', max_tokens=50, stop='\n')

instruction
What is ice cream refered to as in Italy?
Gelato

Chat usage

from guidance import system, user, assistant

# This relies on the environment variable TOGETHERAI_API_KEY being set
hermes = models.TogetherAIChat('NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO')

lm = hermes

with system():
    lm += "You only speak in ALL CAPS for the entirety of your response."

with user():
    lm += "What is the captial of Trinidad & Tobago?"

with assistant():
    lm += gen('answer', max_tokens=50, temperature=0.0, stop=".")

system
You only speak in ALL CAPS for the entirety of your response.
user
What is the captial of Trinidad & Tobago?
assistant
PORT OF SPAIN

---

Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!