4 min readĐọc bằng tiếng Việt
VibeQA in three minutes
Three short walkthroughs of the real dashboard: Tokay proposes suites for a new project, a team reviews the cases, and a failing run becomes a defect that is re-verified.
The other posts on this blog explain one part of VibeQA at a time. This one shows the whole loop instead, in three videos of about a minute each, recorded from the real dashboard. Each video has a short section before it that says what happens in it and why that step is there.
1. A new project: Tokay proposes, you decide
A team creates a project called Seller center, gives it the app's address, uploads a requirements document and links a page. Tokay reads the documents, then explores the app: it opens every menu first, so it knows which pages exist, and then looks at the ones that matter. From that it proposes suites, one per feature and one per business flow that crosses features, such as listing a product and selling it.
Nothing is written yet. Each proposal lists what Tokay saw in the app, for example that a price of 0 shows "Enter a price". A person confirms or drops each of those observations before the proposal can be approved, because whatever is confirmed becomes the expected result: confirm a bug here and the tests will expect the bug. Only after a person approves a proposal does Tokay write its suite, and nothing runs until a person approves that too.
This step exists so that the first suites come from your documents and your app, not from a blank page, while every decision about what to test stays with a person.
2. Review: one case at a time
Each suite starts as a test design. Every case is a row of that design: concrete test data for a
scenario, written as Given, When, Then. A reviewer reads them one by one and marks a case "Looks
right", or asks for a change in the case's thread. In the video, the reviewer asks @Tokay to cover
the 255-character limit on the delivery address. Tokay does not edit the design; it answers in the
thread with a suggestion, a new 256-character class and a new row, and the reviewer accepts it.
Coverage is computed, not declared: each choice of every input appears in some case and every boundary is tested. Neither Tokay nor the reviewer can say a design is complete; the coverage view says so once the rule holds. A case that changes after someone marked it goes back to review. Reviewing a test design with Tokay covers the rest of the review desk.
Reviewing the design instead of the code is the point. A product manager can check a row of test data; few of them can check a Playwright file.
3. A failing run becomes a tracked defect
The last video starts a run. Results stream in test by test while the other suites pass. Then Checkout fails, and the run says why in plain words: the sticky cart total covers the Pay button at this window size, so the click lands on the total. A failure on our side, such as a sandbox or the model provider, would be an infrastructure error and retried by the queue; it is never shown as a test failure.
Report defect turns the failing suite version into the repro: a test that fails while the bug is there and passes once it is fixed. In the video the defect is kept in VibeQA only, with no Jira, so the re-verify is started by hand with Re-verify now. It runs the repro three times in fresh sandboxes, and the verdict comes from a fixed rule, not from anyone's judgement: three passes out of three is Fixed.
With Jira connected, you do not press anything. When the ticket moves to Ready for QA, the re-verify starts by itself and the verdict is posted back to the ticket as a comment. From a failing run to a Jira ticket, and back shows that setup, and Why we run a fix three times explains the verdicts.
What the videos leave out
- Sign-up and the project settings: test accounts, environments and schedules are in From sign-up to first run.
- Writing the code: Tokay generates one test per automated case and dry-runs it once before a person approves the version. The videos skip that wait.
- Web apps only: Android and iOS do not run yet.