What's Changed The most exciting feature that this release of llamafile brings is the first version of transcribefile! 🎉 Our transcribefile is nothing more than a cosmocc-build of the CLI tool from transcribe.cpp, @cjpais' ggml-based Speech-to-Text library that supports 16+ model families 💙 For more details, see CJ's post and Mozilla.ai's annoucement. In addition to this, a new sync with upstream…
What's Changed This release is mainly a bugfix to a regression introduced in 0.10.2 (many many thanks to @niutech for opening the issue! 🙏) As releases become more frequent, we are working to better automate the process and making this more open. This PR brings a new llamafile/release.sh script that's the one used to create all the new release files (we'll be testing it until the next release befo…
What's Changed Add gpu accel probe by @aittalam in #953 Added executable dir to list of dylib valid paths by @aittalam in #959 Update README with Windows executable size note by @ParthProLegend in #962 Update llama.cpp submodule to 7b8443ac7 by @aittalam in #951 Reduce cuda dylibs size by @aittalam in #963 Clarify Linux GPU offload diagnostics by @adminlip in #967 docs: rename example_llamfiles to…
What's Changed Summary: Added support for vulkan dylibs Added windows build scripts -> we now have cuda, rocm, vulkan both as .so and as .dll libraries Updated llama.cpp submodule to 5e9c63546 -> we now have llama.cpp support for new models (e.g. gemma-4, bonsai, Qwen3.6) and functionalities (e.g. llama.cpp internal tools for agents) Details: Fix README_0.10.0 by @aittalam in #918 Add support for…
llamafile versions starting from 0.10.0 use a new build system, aimed at keeping our code more easily aligned with the latest versions of llama.cpp. This means they support more recent models and functionalities, but at the same time they might be missing some of the features you were accustomed to (check out this doc for a high-level description of what has been done). If you liked the "classic e…
What's Changed Fix link to troubleshooting guide by @rsanheim in #740 Preserve URL path when building relative URLs in JS by @dmcardle in #733 Add Plaintext output option to LocalScore + Respect NO_COLOR env var by @cjpais in #742 Update README.md, fix llama 8B table stats by @cjrh in #745 Add phi4 support by @cjpais in #744 Qwen3 Support by @cjpais in #743 New Contributors @rsanheim made their fi…
Llamafile Llamafile v0.9.2 is a significant release. It adds support for: DeepSeek Distil R1 Models Gemma 3 IBM Granite LocalScore In addition 0.9.2 introduces LocalScore, a benchmarking utility and website. LocalScore is an open-source tool that both benchmarks how fast Large Language Models (LLMs) run on your specific hardware and serves as a repository for these results. We created LocalScore t…
This release adds support for DeepSeek Distil models. It improves some documentation, and fixes a segfault when running with an Nvidia GPU. What's Changed Update Makefile: Fix PHONY from check to cosmocc and cosmocc-ci respectively by @mofosyne in #683 Updated README to reflect WSL 2 command for Windows 11 by @peteski22 in #685 Add Support for DeepSeek-R1 models by @Xydane in #687 Revert Cosmopoli…
We've solved all the known issues with the new llamafiler server, and improved its web gui. In addition to the llamafiler binary below, the new server is available as llamafile --server --v2. Its manual can be accessed via llamafile --server --v2 --help. e64c7e2 Include llamafiler in llamafile binary a8fd4d2 Improve management of multiple slots 4158265 Show progress bar for prompt processing in we…
llamafiler has a new web UI which supports two modes of operation: chatbot and raw completion. Its syntax highlighting is just as advanced as the CLI chatbot. It looks much nicer than the old web ui. In a future release, llamafiler will be folded into llamafile to replace the old server. 988c9ec Introduce raw completions web ui 241bf21 Introduce /v1/completions endpoint in new server 6d89f8f Add b…
Add Julia syntax highlighting support Fix possible crash on Windows due to MT bug Improve accuracy of chatbot context window management The new llamafiler server now supports GPU. Pass the -ngl 999 flag. The new llamafiler server's /v1/chat/completions endpoint now supports prompt caching. It may be configured using the --slots COUNT and --ctx-size TOKENS flags.