promptfoo

About promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

What you should know about promptfoo

promptfoo — Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.. It is categorized under AI Tools and primarily built with TypeScript. The project has gathered 20,823 stars and 1,804 forks on GitHub, indicating strong adoption among developers.

Pricing & licensing: This tool is offered free of charge , released under the MIT license. The source code is openly available on GitHub, allowing engineers to audit, contribute, or fork as needed.

Use cases & topics: promptfoo is associated with the following topics: ci, ci-cd, cicd, evaluation, evaluation-framework, llm, llm-eval, llm-evaluation. Teams working in ci / ci-cd / cicd spaces typically evaluate this kind of tool when scoping new architecture decisions or replacing legacy components.

Getting started: Check out the official GitHub repository for installation steps, configuration examples, and the latest release notes. Most teams hit value within the first week if the tool aligns with their existing AI Tools stack.

Editor's note from Fanny Engriana (Founder, Wardigi Digital Agency): when evaluating tools in the AI Tools category for our agency clients, we look at three things first — license clarity, community size, and active maintenance. Tools with explicit license terms and ongoing commits tend to remain viable across multi-year projects.

Related Tools