AI Chatbot Security Scanner
An open-source wizard that tests a website chatbot for prompt injection, jailbreaks and data leaks with NVIDIA garak, and reports every successful attack with the full prompt and reply.
Source repositories
Project overview
Companies put LLM chatbots on their websites without testing what happens when a visitor tries to make them leak their instructions, ignore their rules or reveal data. NVIDIA garak can run thousands of adversarial prompts, but it is a research tool that is hard to point at a real website chatbot. This scanner wraps it in a guided terminal wizard: paste the chatbot endpoint from the browser, test the connection, pick attacks with plain explanations, run the scan and get an HTML report you can act on.
What I built
- 01
Wrote a one-command installer that sets up garak in an isolated Python 3.12 environment, avoiding conflicts with newer system Pythons garak does not support.
- 02
Built a step-by-step wizard that takes the chatbot endpoint copied from the browser’s Network tab, verifies the connection and explains each attack family before it runs.
- 03
Generated an HTML report listing every successful attack with the complete prompt and the chatbot’s reply, so each finding can be reproduced and fixed.
- 04
Added adapters for chatbots whose endpoint cannot be copied directly, and a terminal viewer for garak’s hit logs.
Implementation notes
Scans send thousands of hostile requests, so the tool is documented for authorised testing only: chatbots you own or have written permission to test.
It is the tooling behind my AI security assessments of customer-facing chatbots.
Technology stack
- Python
- NVIDIA garak
- uv
- Bash
AI Security & LLM Red Teaming
Want your chatbot tested properly? I run authorised red-team assessments and deliver a report of what got through and how to fix it.
See the serviceMore open-source projects
Need a similar product?
Let’s discuss the workflow, website, or application you want to ship.