guidance.models.VertexAI

class guidance.models.VertexAI(model, tokenizer=None, echo=True, max_streaming_tokens=None, timeout=0.5, compute_log_probs=False, engine_class=None, **kwargs)

__init__(model, tokenizer=None, echo=True, max_streaming_tokens=None, timeout=0.5, compute_log_probs=False, engine_class=None, **kwargs)

Build a new VertexAI model object that represents a model in a given state.

Methods

__init__(model[, tokenizer, echo, ...])	Build a new VertexAI model object that represents a model in a given state.
copy()	Create a shallow copy of the model object.
delattr(key)	Return a new model with the given attribute deleted.
get(key[, default])	Return the value of a variable, or a default value if the variable is not present.
log_prob(key[, default])	Return the log prob of a variable, or a default value if the variable is not present.
remove(key)	Return a new model with the given variable deleted.
reset([clear_variables])	This resets the state of the model object.
set(key, value)	Return a new model with the given variable value set.
setattr(key, value)	Return a new model with the given model attribute set.
stream()

Attributes

active_role_end	The default end patterns we should use for gen calls.
open_blocks

property active_role_end

The default end patterns we should use for gen calls. TODO: move this logic into the gen call…we can do with if we allow model_variables to run functions.

These patterns are computed dynamically by the model object because they can depend on what the current open roles are, which is something

copy()

Create a shallow copy of the model object.

delattr(key)

Return a new model with the given attribute deleted.

Parameters:

keystr

The attribute name to remove.

get(key, default=None)

Return the value of a variable, or a default value if the variable is not present.

Parameters:

keystr

The name of the variable.

defaultany

The value to return if the variable is not current set.

log_prob(key, default=None)

Return the log prob of a variable, or a default value if the variable is not present.

Parameters:

keystr

The name of the variable.

defaultany

The value to return if the variable is not current set.

remove(key)

Return a new model with the given variable deleted.

Parameters:

keystr

The variable name to remove.

reset(clear_variables=True)

This resets the state of the model object.

Parameters:

clear_variablesbool

If we should clear all the model object’s variables in addition to reseting the byte state.

set(key, value)

Return a new model with the given variable value set.

Parameters:

keystr

The name of the variable to be set.

valueany

The value to set the variable to.

setattr(key, value)

Return a new model with the given model attribute set.

Parameters:

keystr

The name of the attribute to be set.

valueany

The value to set the attribute to.