5 min read
Running suites from CI and on a schedule, and keeping flaky tests honest
Gate pull requests on VibeQA, run every suite nightly, hear about new failures, and handle flaky tests in CI with history and quarantine, not by hiding bugs.
Blog
How we design tests, re-check fixes and keep a red run honest.
New to the blog? Start with What automation testing is, and why end-to-end suites decay
Why we run a fix three times before we call it fixed
One green run can be luck. VibeQA runs the repro test three times to verify bug fixes, then turns the outcomes into one verdict. Here is what each one means.
From a failing run to a Jira ticket, and back
Jira defect tracking with a repro test: how a failing run becomes a Jira Bug, what VibeQA writes on the ticket, and where the Jira connection stops today.
A red run should mean your app is broken
Test failure vs infrastructure failure: why VibeQA gives a run six outcomes instead of pass and fail, shows its own errors in violet, and retries only those.
What happens when you press run
How VibeQA runs end-to-end tests: the queue, a fresh sandbox per run, test accounts whose passwords Tokay never sees, and where to look when a test fails.
Reviewing a test design with Tokay
What test case review looks like in VibeQA: hand edits, @Tokay comments that come back as suggestions, "Looks right" per case, and the checks before approval.
Design the tests first, write the code second
Test case design before code: how Tokay turns a requirement into a table of cases you can review, and how VibeQA measures coverage by a fixed rule.
From sign-up to the first run
How to set up automated testing for a web app in VibeQA: sign in with a link, create a project, let Tokay draft suites, review them and start a first run.
What is different about VibeQA
What sets VibeQA apart from other AI testing tools: you review a test design, a person approves every version, and a red run means your app broke.
Meet VibeQA and Tokay
VibeQA is AI test automation for web apps: Tokay drafts the test design and the code, people approve both. The whole loop, from Jira ticket to verified fix.
What automation testing is, and why end-to-end suites decay
Automation testing explained: what unit, API and end-to-end tests each check, what they buy you, and why browser suites so often stop being trusted.
