Chat dialog

Guidance supports chat-based models like ChatGPT and GPT-4 using role tags. These are then converted to the appropriate format for the model (either a JSON API format or special tokens).

[ ]:
import logging
[1]:
call_delay_secs = 0
requested_log_level = logging.WARNING
[ ]:
logging.basicConfig(level=requested_log_level)
[2]:
import os

# Uncomment if using DefaultAzureCredential below
# from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# This is the name of the model deployed, such as 'gpt-4' or 'gpt-3.5-turbo
model = os.getenv("AZUREAI_CHAT_MODEL", "Please set the model")

# This is the deployment URL, as provided in the Azure AI playground ('view code')
# It will end with 'openai.azure.com'
azure_endpoint = os.getenv("AZUREAI_CHAT_BASE_ENDPOINT", "Please set the endpoint")

# This is the name of the deployment specified in the Azure portal
azure_deployment = os.getenv("AZUREAI_CHAT_DEPLOYMENT", "Please set the deployment name")

# This is the deployed API version, such as 2024-02-15-preview
azure_api_version = os.getenv("AZUREAI_CHAT_API_VERSION", "Please set the API version")

# The environment variable should be set to the API key from the Azure AI playground:
api_key=os.getenv("AZUREAI_CHAT_KEY", "Please set API key")

# Alternatively, we can use Entra authentication
# token_provider = get_bearer_token_provider(
#     DefaultAzureCredential(),
#     "https://cognitiveservices.azure.com/.default"
#)

Preliminaries concluded, we can now create our model:

[3]:
from guidance import models, gen

azureai_model = models.AzureOpenAI(
    model=model,
    azure_endpoint=azure_endpoint,
    azure_deployment=azure_deployment,
    version=azure_api_version,
    # For authentication, use either
    # azure_ad_token_provider=token_provider,
    # or
    api_key=api_key,
)

Multi-step chat with hidden blocks

We are now going to set up a multistage chat, where we have the chat bot help the use achieve some goal. The user will only have to specify the goal, and then we will create a chain-of-thought conversation with the bot which will:

  1. Ask the bot for a number of suggestions.

  2. List the pros and cons of each.

  3. Pick the best suggestion.

  4. Product a detailed action plan.

Our goal is to only show the final result to the user.

Now, let us define our generation function:

[4]:
import re
import time

import guidance
from guidance import gen, select, system, user, assistant

@guidance
def plan_for_goal(lm, goal: str):

    # This is a helper function which we will use below
    def parse_best(prosandcons, options):
        best = re.search(r'Best=(\d+)', prosandcons)
        if not best:
            best =  re.search(r'Best.*?(\d+)', 'Best= option is 3')
        if best:
            best = int(best.group(1))
        else:
            best = 0
        return options[best]

    # Some general instruction to the model
    with system():
        lm += "You are a helpful assistant."

    # Simulate a simple request from the user
    # Note that we switch to using 'lm2' here, because these are intermediate steps (so we don't want to overwrite the current lm object)
    with user():
        lm2 = lm + f"""\
        I want to {goal}
        Can you please generate one option for how to accomplish this?
        Please make the option very short, at most one line."""

    # Generate several options. Note that this means several sequential generation requests
    n_options = 5
    with assistant():
        options = []
        for i in range(n_options):
            options.append((lm2 + gen(name='option', temperature=1.0, max_tokens=50))["option"])

    # Have the user request pros and cons
    with user():
        lm2 += f"""\
        I want to {goal}
        Can you please comment on the pros and cons of each of the following options, and then pick the best option?
        ---
        """
        for i, opt in enumerate(options):
            lm2 += f"Option {i}: {opt}\n"
        lm2 += f"""\
        ---
        Please discuss each option very briefly (one line for pros, one for cons), and end by saying Best=X, where X is the number of the best option."""

    # Get the pros and cons from the model
    with assistant():
        lm2 += gen(name='prosandcons', temperature=0.0, max_tokens=600, stop="Best=") + "Best=" + gen("best", regex="[0-9]+")
        time.sleep(call_delay_secs)

    # The user now extracts the one selected as the best, and asks for a full plan
    # We switch back to 'lm' because this is the final result we want
    with user():
        lm += f"""\
        I want to {goal}
        Here is my plan: {options[int(lm2["best"])]}
        Please elaborate on this plan, and tell me how to best accomplish it."""

    # The plan is generated
    with assistant():
        lm += gen(name='plan', max_tokens=500)
        time.sleep(call_delay_secs)

    return lm

Create a plan for the user. Note how the portions which were sent to lm2 in the function above are not shown in the final result:

[5]:
results = azureai_model + plan_for_goal(goal="read more books")
system
You are a helpful assistant.
user
I want to read more books Here is my plan: Set a daily reading goal, like one chapter or 30 minutes per day. Please elaborate on this plan, and tell me how to best accomplish it.
assistant
Absolutely, setting a daily reading goal is a great way to incorporate more reading into your routine. Here's a more detailed plan: 1. **Choose the Right Book**: Start with a book that interests you. It could be a novel, a self-help book, a biography, or any genre that you enjoy. The key is to choose a book that you're excited to read, as this will make it easier to stick to your reading plan. 2. **Set a Specific Goal**: You've mentioned reading one chapter or for 30 minutes each day. This is a great start. Make sure your goal is achievable and not too overwhelming. If a chapter is too long, you can set a goal to read a certain number of pages each day. 3. **Create a Reading Schedule**: Decide on a specific time to read each day. It could be in the morning, during lunch breaks, or before bed. Consistency is key, so try to stick to this schedule as closely as possible. 4. **Find a Quiet Place**: Choose a quiet and comfortable place to read where you won't be easily distracted. This could be a cozy corner in your home, a local library, or a park. 5. **Limit Distractions**: Turn off your phone or put it on silent mode to avoid interruptions. If you're reading on a digital device, try to close all unrelated apps and tabs. 6. **Take Notes**: If you come across an interesting idea or a word you don't understand, jot it down. This not only helps you remember what you've read but also makes the reading process more interactive and enjoyable. 7. **Review Your Progress**: Keep track of your reading progress. You can use a journal or a digital app for this. Celebrate your small victories, like finishing a chapter or a book. 8. **Join a Book Club or Reading Group**: This can provide motivation to read regularly. You'll also get to discuss the books and share your thoughts with others. Remember, the goal is to make reading a habit and enjoy the process. Don't rush through books just to meet your goal. Happy reading!

We can access the final plan itself:

[6]:
print(results['plan'])
Absolutely, setting a daily reading goal is a great way to incorporate more reading into your routine. Here's a more detailed plan:

1. **Choose the Right Book**: Start with a book that interests you. It could be a novel, a self-help book, a biography, or any genre that you enjoy. The key is to choose a book that you're excited to read, as this will make it easier to stick to your reading plan.

2. **Set a Specific Goal**: You've mentioned reading one chapter or for 30 minutes each day. This is a great start. Make sure your goal is achievable and not too overwhelming. If a chapter is too long, you can set a goal to read a certain number of pages each day.

3. **Create a Reading Schedule**: Decide on a specific time to read each day. It could be in the morning, during lunch breaks, or before bed. Consistency is key, so try to stick to this schedule as closely as possible.

4. **Find a Quiet Place**: Choose a quiet and comfortable place to read where you won't be easily distracted. This could be a cozy corner in your home, a local library, or a park.

5. **Limit Distractions**: Turn off your phone or put it on silent mode to avoid interruptions. If you're reading on a digital device, try to close all unrelated apps and tabs.

6. **Take Notes**: If you come across an interesting idea or a word you don't understand, jot it down. This not only helps you remember what you've read but also makes the reading process more interactive and enjoyable.

7. **Review Your Progress**: Keep track of your reading progress. You can use a journal or a digital app for this. Celebrate your small victories, like finishing a chapter or a book.

8. **Join a Book Club or Reading Group**: This can provide motivation to read regularly. You'll also get to discuss the books and share your thoughts with others.

Remember, the goal is to make reading a habit and enjoy the process. Don't rush through books just to meet your goal. Happy reading!

Asking help from experts

Now, let us ask our chat model to pick some experts in a particular field, and impersonate them to give advice:

[7]:
@guidance
def run_expert_advice(lm, query: str):
    # Some general instruction to the model
    with system():
        lm += "You are a helpful assistant."

    with user():
        lm += f"""I want a response to the following question:
{query}
Who are 3 world-class experts (past or present) who would be great at answering this?
Please don't answer the question or comment on it yet.
"""

    with assistant():
        lm += gen(name='experts', temperature=0, max_tokens=300)
        time.sleep(call_delay_secs)

    with user():
        lm += """Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question.
If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...').
Please start your answer with ANSWER:
"""

    with assistant():
        lm += gen(name='answer', temperature=0, max_tokens=500)
        time.sleep(call_delay_secs)

    return lm
[8]:
mean_life = azureai_model + run_expert_advice("What is the meaning of life?")
system
You are a helpful assistant.
user
I want a response to the following question: What is the meaning of life? Who are 3 world-class experts (past or present) who would be great at answering this? Please don't answer the question or comment on it yet.
assistant
1. Dalai Lama: As the spiritual leader of Tibetan Buddhism, the Dalai Lama has spent his life exploring questions about existence, purpose, and happiness. His insights on compassion, self-awareness, and inner peace could provide a unique perspective on the meaning of life. 2. Albert Einstein: Known for his contributions to physics, Einstein also pondered philosophical questions about life's purpose and meaning. His thoughts on science, religion, and the universe could offer a thought-provoking answer. 3. Viktor Frankl: A psychiatrist and Holocaust survivor, Frankl wrote extensively about the search for life's meaning, most notably in his book "Man's Search for Meaning". His experiences and insights could provide a profound answer to this question.
user
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer. In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question. If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...'). Please start your answer with ANSWER:
assistant
ANSWER: The meaning of life is a deeply personal and subjective concept, and it can be approached from various perspectives. From a spiritual perspective, one might argue that the purpose of life is to cultivate compassion, understanding, and inner peace. This involves recognizing the interconnectedness of all beings and striving to alleviate suffering in the world. From a scientific perspective, life might be seen as a complex series of events in the universe, with no inherent meaning beyond what we as conscious beings assign to it. This perspective encourages us to seek knowledge, explore the universe, and appreciate the beauty and complexity of the natural world. From a psychological perspective, the meaning of life could be found in our response to life's challenges and our pursuit of personal growth. This involves finding purpose in our actions, creating and pursuing meaningful goals, and striving to make a positive impact on the world. In conclusion, the meaning of life may not be a universal truth, but rather a personal journey of discovery, growth, and understanding. It is up to each individual to seek out their own purpose and find what gives their life meaning.
[9]:
more_productive = azureai_model + run_expert_advice('How can I be more productive?')
system
You are a helpful assistant.
user
I want a response to the following question: How can I be more productive? Who are 3 world-class experts (past or present) who would be great at answering this? Please don't answer the question or comment on it yet.
assistant
1. Tim Ferriss: An entrepreneur, author, and public speaker, Ferriss is best known for his book "The 4-Hour Workweek" where he discusses productivity and lifestyle design. 2. David Allen: A productivity consultant who is recognized for creating the time management method known as "Getting Things Done". 3. Stephen Covey: The late author of "The 7 Habits of Highly Effective People", Covey's work has been influential in the field of productivity and personal effectiveness.
user
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer. In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question. If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...'). Please start your answer with ANSWER:
assistant
ANSWER: To increase productivity, it's crucial to first understand your goals and priorities. Start by defining what you want to achieve and then break these down into smaller, manageable tasks. This will help you focus on what's important and avoid getting overwhelmed. Some might argue that it's beneficial to follow the principle of doing the most challenging task first thing in the morning, often referred to as "eating the frog". This approach can help you tackle difficult tasks when your energy levels are highest, leading to increased productivity. Others might suggest implementing a time management system, such as the "two-minute rule". If a task takes two minutes or less to complete, do it immediately rather than putting it off. This can help prevent small tasks from piling up and becoming overwhelming. Additionally, it's important to eliminate distractions as much as possible. This could mean turning off notifications on your devices, creating a dedicated workspace, or setting specific "focus hours" during your day. Remember, productivity isn't just about doing more, it's about effectiveness. Regular breaks, for instance, can actually increase productivity by preventing burnout and maintaining high levels of focus. Some might recommend techniques such as the Pomodoro Technique, which involves working for a set amount of time (e.g., 25 minutes) and then taking a short break (e.g., 5 minutes). Finally, it's crucial to maintain a healthy work-life balance. Overworking can lead to burnout, which will ultimately decrease productivity. Make sure to take time for relaxation and activities you enjoy. This will help you stay motivated and energized, leading to increased productivity in the long run.

Agents

We are now going to define a ‘conversation agent.’ This maintains a memory of a conversation, and can generate an appropriate reply, based on the persona it has been given.

[10]:
class ConversationAgent:
    def __init__(self, chat_model, name: str, instructions: str, context_turns: int = 2):
        self._chat_model = chat_model
        self._name = name
        self._instructions = instructions
        self._my_turns = []
        self._interlocutor_turns = []
        self._went_first = False
        self._context_turns = context_turns

    @property
    def name(self) -> str:
        return self._name

    def reply(self, interlocutor_reply = None) -> str:
        if interlocutor_reply is None:
            self._my_turns = []
            self._interlocutor_turns = []
            self._went_first = True
        else:
            self._interlocutor_turns.append(interlocutor_reply)

        # Get trimmed history
        my_hist = self._my_turns[(1-self._context_turns):]
        interlocutor_hist = self._interlocutor_turns[-self._context_turns:]

        # Set up the system prompt
        curr_model = self._chat_model
        with system():
            curr_model += f"Your name is {self.name}. {self._instructions}"
            if len(interlocutor_hist) == 0:
                curr_model += "Introduce yourself and start the conversation"
            elif len(interlocutor_hist) == 1:
                curr_model += "Introduce yourself before continuing the conversation"

        # Replay the last few turns
        for i in range(len(my_hist)):
            with user():
                curr_model += interlocutor_hist[i]
            with assistant():
                curr_model += my_hist[i]

        if len(interlocutor_hist) > 0:
            with user():
                curr_model += interlocutor_hist[-1]

        with assistant():
            curr_model += gen(name='response', max_tokens=100)
        time.sleep(call_delay_secs)

        self._my_turns.append(curr_model['response'])
        return curr_model['response']

We can have two of these agents converse with each other with a conversation simulator:

[11]:
def conversation_simulator(
    bot0: ConversationAgent,
    bot1: ConversationAgent,
    total_turns: int = 5 ):
    conversation_turns = []
    last_reply = None
    for _ in range(total_turns):
        last_reply = bot0.reply(last_reply)
        conversation_turns.append(dict(name=bot0.name, text=last_reply))
        time.sleep(call_delay_secs)
        last_reply = bot1.reply(last_reply)
        conversation_turns.append(dict(name=bot1.name, text=last_reply))
    return conversation_turns

Now, let’s try generating a conversation:

[12]:
bot_instructions = """You are taking part in a discussion about bodyline bowling.
Only generate text as yourself and do not prefix your reply with your name.
Keep your answers to a couple of short sentences."""

bradman_bot = ConversationAgent(azureai_model, "Donald Bradman", bot_instructions, context_turns=5)
jardine_bot = ConversationAgent(azureai_model, "Douglas Jardine", bot_instructions, context_turns=5)

conversation_turns = conversation_simulator(bradman_bot, jardine_bot, total_turns=3)
system
Your name is Douglas Jardine. You are taking part in a discussion about bodyline bowling. Only generate text as yourself and do not prefix your reply with your name. Keep your answers to a couple of short sentences.
user
Hello everyone, I'm Donald Bradman, a former Australian cricketer. I'd like to discuss the controversial topic of bodyline bowling. What are your thoughts on this strategy?
assistant
Hello Donald, I'm Douglas Jardine, a former English cricketer. I believe bodyline bowling was a legitimate strategy, albeit a controversial one. It was a tactical response to the dominant batting during that era.
user
Hello Douglas, I respect your viewpoint. However, I believe that while it may have been a tactical response, bodyline bowling was a dangerous strategy that risked the safety of the batsmen.
assistant
I understand your concerns, Donald. The safety of players is indeed paramount. However, it's important to remember that bodyline bowling was within the rules of the game at that time. It was a tactical innovation, albeit a contentious one.
user
Indeed, Douglas, it was within the rules. However, the spirit of the game should always be upheld, and tactics that compromise player safety should be reconsidered, regardless of their legality.
assistant
I agree, Donald, the spirit of the game is crucial. It's a delicate balance between innovation and maintaining the integrity and safety of cricket. The controversy around bodyline bowling led to important changes in the rules, which is a testament to the game's ability to evolve.
[13]:
for turn in conversation_turns:
    print(f"{turn['name']}: {turn['text']}\n")
Donald Bradman: Hello everyone, I'm Donald Bradman, a former Australian cricketer. I'd like to discuss the controversial topic of bodyline bowling. What are your thoughts on this strategy?

Douglas Jardine: Hello Donald, I'm Douglas Jardine, a former English cricketer. I believe bodyline bowling was a legitimate strategy, albeit a controversial one. It was a tactical response to the dominant batting during that era.

Donald Bradman: Hello Douglas, I respect your viewpoint. However, I believe that while it may have been a tactical response, bodyline bowling was a dangerous strategy that risked the safety of the batsmen.

Douglas Jardine: I understand your concerns, Donald. The safety of players is indeed paramount. However, it's important to remember that bodyline bowling was within the rules of the game at that time. It was a tactical innovation, albeit a contentious one.

Donald Bradman: Indeed, Douglas, it was within the rules. However, the spirit of the game should always be upheld, and tactics that compromise player safety should be reconsidered, regardless of their legality.

Douglas Jardine: I agree, Donald, the spirit of the game is crucial. It's a delicate balance between innovation and maintaining the integrity and safety of cricket. The controversy around bodyline bowling led to important changes in the rules, which is a testament to the game's ability to evolve.


Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!