guidance.models.LlamaCpp

class guidance.models.LlamaCpp(model=None, echo=True, chat_template=None, enable_backtrack=True, enable_ff_tokens=True, enable_monitoring=True, sampling_params: SamplingParams | None = None, **llama_cpp_kwargs)

__init__(model=None, echo=True, chat_template=None, enable_backtrack=True, enable_ff_tokens=True, enable_monitoring=True, sampling_params: SamplingParams | None = None, **llama_cpp_kwargs)

Build a new LlamaCpp model object that represents a model in a given state.

Methods

__init__([model, echo, chat_template, ...])	Build a new LlamaCpp model object that represents a model in a given state.
copy()
get(key[, default])	Return the value of a variable, or a default value if the variable is not present.
log_prob(key[, default])	Return the log probability of a variable, or a default value if the variable is not present.
remove(key)	Return a new model with the given variable deleted.
set(key, value)	Return a new model with the given variable value set.
stream()	Return a new model stream object that delays execution until it is iterated over.
with_sampling_params(sampling_params)	Return a new model with the given sampling parameters set.
with_step_config(step_config)	Return a new model with step interjection configured (engine-backed models only).

get(key: str, default: D | None = None) → str | list[str] | None | D

Return the value of a variable, or a default value if the variable is not present.

Parameters:

keystr

The name of the variable.

defaultAny

The value to return if the variable is not current set.

log_prob(key: str, default: D | None = None) → float | list[float | None] | None | D

Return the log probability of a variable, or a default value if the variable is not present.

Parameters:

keystr

The name of the variable.

defaultAny

The value to return if the variable is not current set.

remove(key: str) → Self

Return a new model with the given variable deleted.

Parameters:

keystr

The variable name to remove.

set(key: str, value: str | list[str]) → Self

Return a new model with the given variable value set.

Parameters:

keystr

The name of the variable to be set.

valuestr

The value to set the variable to.

stream() → ModelStream

Return a new model stream object that delays execution until it is iterated over.

with_sampling_params(sampling_params: SamplingParams) → Self

Return a new model with the given sampling parameters set.

with_step_config(step_config: StepConfig) → Self

Return a new model with step interjection configured (engine-backed models only).