Compare · Checked Oct 19, 2026
Crewly vs DIY GitHub Actions + Playwright
You can run Playwright on every pull request with GitHub Actions and make the check required. Crewly uses that same run. What it adds is the link to an approved scope, the lock on the approved test, the test-change flags and a page your client can open.
| Topic | DIY GitHub Actions + Playwright | Crewly |
|---|---|---|
| Running the test | npm init playwright@latest offers to add a GitHub Actions workflow that runs your tests and uploads an HTML report as an artifact. | The same: your CI runs Playwright. Crewly reads the results artifact and never runs your code. |
| Blocking a merge | Required status checks must be successful, skipped or neutral before a merge, and can be pinned to a specific GitHub App. | "Crewly Receipt" is a check run, so you can make it required the same way. |
| Who can change the test | Anyone who opens the pull request, an agent included, can edit the tests. CODEOWNERS with "Require review from Code Owners" blocks the merge until an owner approves, not the edit. For private repos that is listed on Pro, Team or Enterprise Cloud. | The test is approved with the scope and locked by hash. Changing it fails the receipt. |
| Weakened tests | Not built in. You would write and maintain these checks yourself. | Flags deleted test files, added skips, fewer assertions and CI workflow edits. |
| What a client can see | The HTML report is a zip artifact (kept 30 days in the generated workflow): you download it, extract it, and open it with npx playwright show-report. | A client page with the approved scope, what was checked and what was not. No login. |
| Cost | Private repos include monthly Actions minutes (2,000 on Free, 3,000 on Team); Linux 2-core minutes beyond that are listed at $0.006 each. Public repos are free. | Free 14-day pilot, then $99 per client project a month. The receipt workflow still runs in your CI, on your minutes. |
DIY GitHub Actions + Playwright may fit better if
- One team, no client to report to, and you control who edits tests.
- You are happy to build and maintain approval links and test-change checks yourself.
Crewly may fit better if
- Agents open the pull requests, and the test needs to be agreed before they start.
- A client needs a plain-language record of each delivery.
Sources
Everything about DIY GitHub Actions + Playwright on this page comes from these public pages, read on Oct 19, 2026. Products change; if something here is out of date, email kidwiz404@gmail.com and we will fix it.