More docs, fixes, performance improvements. Added Offline grammar debugging guide in README (thanks @biefan!) (#1462) Fixed Workaround for OnnxRT-GenAI runtime failing to load a model (#1419) Correct handling of notebook normalization for stderr and stdout (#1423) Changed Updated llguidance to 1.6.1 -- support for URI format for JSON strings; perf improvements (#1429)
Introduces monitor guided inference and onnxruntime-genai support. Numerous fixes. Added Support for Python 3.14 (#1385) Ability to add inference-time semantic verification via monitor guided inference (#1391) by @naga86 Added onnxruntime-genai backend (#1366) Removed Dropped 3.9 support Fixed Remove code dealing with deprecated transformers.HybridCache (#1408) Ensure setup.py has consistent trans…
Guidance 0.3.0 More model providers, better LARK exposure, re-done custom tools. Breaking Changes Removed old-style @guidance-based tool calls (in favor of more generic OpenAI-style tools) Temp removal of calling support for local models (PR planned to restore support in-line with new tools) Added Added support for groq and mistralai APIs via litellm Added experimental sglang model Added experimen…
Guidance 0.2.5 Special tokens can be user-defined. Numerous grammar fixes. Candidate tokens rendered for all token types. Added Widget has dark mode (mainly for exporting). special_token function added for adding in <think> and similar. Changed top_k exposed when echo=True for all token types (including fast-forward, inputs). Use greedy samplng if temp is smaller than 1e-5. Fixed Normalized captur…
Guidance 0.2.4 Better sampling, better metrics, llama-cpp-python fixes (update to latest please!), uncountable visualization fixes. Added Allow changing sampling_params (top_p/top_k, min_p, repetition_penalty) on the fly Model.with_sampling_params(...) Add top_k tokens back into vis after temporary removal in previous refactor Removed Model.token_count removed in favor of (currently private) Model…
Guidance 0.2.3 We have a performance hotfix, and then we snuck in some extras. Added Added Llama3.2 chat template Removed Deleted some dead code, in particular sample_with_temperature from Engine classes Changed Switched top-k (for widget) implementation to use a priority-queue instead of a full sort, saving a few milliseconds per token when widget/vis is turned on Fixed Fix performance regression…
Guidance 0.2.2 Better support for OpenAI, we fixed some bugs too (we tried to at least)! Included are 0.2.1 changes as we didn't record it previously. Added Token probabilities and backtracking support in the Jupyter widget. Support for offline notebook rendering. Higher-level AST for representing various input/output types for models (including JSON outputs, image inputs, special tokens, selects,…
0.2.1 release
What's Changed Happy new year! This release contains a significant overhaul of many elements of guidance, including: Visualization Overhaul: Guidance's in-line visualizations have been rewritten from scratch. TL;DR: We added a lot of code to support better model metrics and visualization. Check it out: Significantly Faster, Rust-based Grammar Engine: Guidance's core grammar processing has been ful…
What's Changed Replace Python-based EarleyParser with lexeme-based rust implementation by @hudson-ai & @mmoskal in #951 This pre-release version of guidance uses a new Rust-based Earley lexer + parser that offers significant speedups and bugfixes to guidance grammar parsing. See PR #951 for full details of the optimizations. Expect some changes from current Guidance (v0.1.16) behavior during pre-r…