mirror of https://github.com/abpframework/abp.git
committed by
GitHub
4 changed files with 90 additions and 0 deletions
@ -0,0 +1,90 @@ |
|||
# How We Built TutorialValidator to Automatically Validate Documentation Tutorials |
|||
|
|||
Writing a tutorial is hard. Keeping it correct over time is even harder. |
|||
|
|||
If you maintain technical documentation, you probably know this pain: a tutorial that worked three months ago can quietly break after a framework update, a package change, or a small missing line in a code snippet. New developers follow the steps, hit an error, and lose trust in the docs. |
|||
|
|||
That exact problem is why we built **TutorialValidator**. |
|||
|
|||
TutorialValidator is an open-source, AI-powered tool that checks whether a software tutorial actually works from start to finish. You give it a tutorial URL, and it behaves like a real developer following the guide: it reads each step, executes commands, writes files, runs the app, and verifies expected results. |
|||
|
|||
We first created it to validate ABP Framework tutorials internally, then shared it publicly so anyone can use it with their own tutorials. |
|||
|
|||
 |
|||
|
|||
## What Problem Does It Solve? |
|||
|
|||
Most documentation issues are not obvious during review: |
|||
|
|||
- A command assumes a file that has not been created yet |
|||
- A code sample misses a namespace or import |
|||
- A step relies on hidden context that is never explained |
|||
- An endpoint is expected to respond, but does not |
|||
|
|||
Traditional proofreading catches wording problems. TutorialValidator targets **execution problems**. |
|||
|
|||
It turns tutorials into something testable. |
|||
|
|||
## How It Works (Simple View) |
|||
|
|||
TutorialValidator runs in three phases: |
|||
|
|||
1. **Analyst**: Scrapes tutorial pages and converts instructions into a structured test plan |
|||
2. **Executor**: Follows the plan step by step in a clean environment |
|||
3. **Reporter**: Produces a clear result summary and optional notifications |
|||
|
|||
 |
|||
|
|||
The key idea is simple: if a developer would need to do it, the validator does it too. |
|||
|
|||
That includes running terminal commands, editing files, checking HTTP responses, and validating build outcomes. |
|||
|
|||
 |
|||
|
|||
## Why This Approach Is Useful |
|||
|
|||
TutorialValidator is designed for practical documentation quality, not just technical experimentation. |
|||
|
|||
- **Catches real-world breakages early** before readers report them |
|||
- **Creates repeatable validation** instead of one-off manual checks |
|||
- **Works well in teams** through report outputs, logs, and CI-friendly behavior |
|||
- **Supports different strictness levels** with developer personas (`junior`, `mid`, `senior`) |
|||
|
|||
For example, `junior` and `mid` personas are great for spotting unclear documentation, while `senior` helps identify issues an experienced developer could work around. |
|||
|
|||
## Built for ABP, Open for Everyone |
|||
|
|||
Even though TutorialValidator was born from ABP documentation needs, it is not limited to ABP content. |
|||
|
|||
It supports validating any publicly accessible software tutorial and can run in: |
|||
|
|||
- **Docker mode** for clean, isolated execution (recommended) |
|||
- **Local mode** for faster feedback when your environment is already prepared |
|||
|
|||
It also supports multiple AI providers, including OpenAI, Azure OpenAI, and OpenAI-compatible endpoints. |
|||
|
|||
## Open Source and Extensible |
|||
|
|||
TutorialValidator is structured as multiple focused projects: |
|||
|
|||
- Core models and shared contracts |
|||
- Analyst for scraping and plan extraction |
|||
- Executor for step-by-step validation |
|||
- Orchestrator for end-to-end workflow |
|||
- Reporter for Email/Discord notifications |
|||
|
|||
This architecture keeps the project easy to understand and extend. Teams can add new step types, plugins, or reporting channels based on their own workflow. |
|||
|
|||
## Final Thoughts |
|||
|
|||
Documentation is part of the product experience. When tutorials fail, trust fails. |
|||
|
|||
TutorialValidator helps teams move from “we think this tutorial works” to “we verified it works.” |
|||
|
|||
If your team maintains technical tutorials, this project can give you a practical safety net and a repeatable quality process. |
|||
|
|||
--- |
|||
|
|||
Repository: https://github.com/AbpFramework/TutorialValidator |
|||
|
|||
If you try it in your own docs pipeline, we would love to hear your feedback and ideas. |
|||
|
After Width: | Height: | Size: 9.9 KiB |
|
After Width: | Height: | Size: 18 KiB |
|
After Width: | Height: | Size: 10 KiB |
Loading…
Reference in new issue