{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# `TogetherAI` API examples\n", "\n", "This notebook contains examples of how to use the `TogetherAI` LLM, utilizing models hosted by [together.ai](https://together.ai)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Completion usage" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
The most famous piece of japanese literature in a JSON format is:\n",
       "{\n",
       "    "title_english":  "The Tale of Genji",\n",
       "    "title_japanese": "源氏物語",\n",
       "    "author": "Murasaki Shikibu",\n",
       "    "year": 1008\n",
       "}\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from guidance import models, gen\n", "\n", "# This relies on the environment variable TOGETHERAI_API_KEY being set\n", "mixtral = models.TogetherAI('mistralai/Mixtral-8x7B-v0.1')\n", "\n", "lm = mixtral\n", "\n", "stop_tokens = [\",\", \"}\", \"\\n\"]\n", "temperature = 0.0\n", "\n", "lm += f\"\"\"The most famous piece of japanese literature in a JSON format is:\n", "{{\n", " \"title_english\": {gen(name='title_english', temperature=temperature, max_tokens=50, stop=stop_tokens)},\n", " \"title_japanese\": {gen(name='title_japanese', temperature=temperature, max_tokens=50, stop=stop_tokens)},\n", " \"author\": {gen(name='author', temperature=temperature, max_tokens=50, stop=stop_tokens)},\n", " \"year\": {gen(name='year', temperature=temperature, max_tokens=50, stop=stop_tokens)}\n", "}}\n", "\"\"\"" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Instruct usage" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
instruction
What is ice cream refered to as in Italy?
Gelato
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from guidance import instruction\n", "\n", "# This relies on the environment variable TOGETHERAI_API_KEY being set\n", "gemma = models.TogetherAIInstruct('google/gemma-7b-it')\n", "\n", "lm = gemma\n", "with instruction():\n", " lm += \"What is ice cream refered to as in Italy?\"\n", "lm += gen('flavor', max_tokens=50, stop='\\n')" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Chat usage" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
system
You only speak in ALL CAPS for the entirety of your response.
user
What is the captial of Trinidad & Tobago?
assistant
PORT OF SPAIN
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from guidance import system, user, assistant\n", "\n", "# This relies on the environment variable TOGETHERAI_API_KEY being set\n", "hermes = models.TogetherAIChat('NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO')\n", "\n", "lm = hermes\n", "\n", "with system():\n", " lm += \"You only speak in ALL CAPS for the entirety of your response.\"\n", "\n", "with user():\n", " lm += \"What is the captial of Trinidad & Tobago?\"\n", "\n", "with assistant():\n", " lm += gen('answer', max_tokens=50, temperature=0.0, stop=\".\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!
" ] } ], "metadata": { "kernelspec": { "display_name": "adatest", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.3" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }