Risk‑based guidance for QA engineers who need to replace an established Selenium suite with Playwright while keeping behavioral coverage visible and auditable.
1. Start with an Inventory & Baseline
Before any code moves, capture what you are testing. A concrete inventory reduces the chance of “silent” coverage loss.
- Dimension: Business journeys What to record: End‑to‑end flows (e.g., “checkout”, “user onboarding”) Why it matters: Defines the critical user paths whose coverage must be preserved and re-verified.
- Dimension: Browser matrix What to record: Browser engines, branded channels, operating systems, and versions currently covered by Selenium Why it matters: Playwright supports Chromium, Firefox, and WebKit plus selected branded Chromium channels; verify the required matrix explicitly because WebKit is not identical to Safari and engine coverage is not automatic browser-version parity.
- Dimension: Assertions What to record: Types (visibility, text, HTTP status, DB state) and locations (page objects, test bodies) Why it matters: Makes verification intent visible so equivalent evidence can be designed in the new suite.
- Dimension: Test data & setup What to record: Fixtures, API seeding, test‑specific accounts Why it matters: Ensures the same pre‑conditions are reproduced in Playwright.
- Dimension: Side‑effects What to record: Cookies, localStorage, server state changes Why it matters: Prevents hidden dependencies from disappearing.
- Dimension: Historical failures What to record: Flaky patterns, known bugs, failure rates per test Why it matters: Provides a “flake budget” that can be transferred to the new suite.
- Dimension: Timing & waits What to record: Explicit
Thread.sleep,WebDriverWait, custom retries Why it matters: Highlights where Playwright’s auto‑waiting can replace fragile code. - Dimension: Evidence artifacts What to record: Screenshots, logs, JUnit XML, CI reports Why it matters: Baseline artifacts are needed for later comparison.
Export this inventory to a shared spreadsheet or a lightweight markdown table that the whole team can edit. Treat it as the definition of coverage for the migration project.
2. Build a Parity Ledger
A parity ledger is a living document that maps each Selenium scenario to its Playwright counterpart and records the equivalence status.
- Selenium Test ID:
S-Login-001Playwright Test ID:PW-Login-001Business Intent: Verify successful login with valid credentials Expected Differences: Playwright uses locator actionability checks; Selenium used an explicit wait Owner: QA owner Evidence (trace, screenshot): Playwright trace + assertion result Rollback Status: ✅ Ready - Selenium Test ID:
S-Search-045Playwright Test ID:PW-Search-045Business Intent: Search returns at least one result Expected Differences: Assertion moved from page object to test body Owner: QA owner Evidence (trace, screenshot): Screenshot + JUnit result Rollback Status: ⏳ Pending
How to use the ledger
- Create the entry as soon as a Playwright test is drafted.
- Mark “Expected Differences” for any intentional change (e.g., using Playwright’s
expect(page).toHaveURL(url)instead of Selenium’sassertEquals(url, expectedUrl)). - Collect evidence – Playwright’s built‑in trace (
trace.zip) or screenshots are attached to the ledger entry. - Set rollback status –
✅ Readywhen the new test has been run side‑by‑side with the old one and the results match the baseline; otherwise keep it⏳ Pending.
The ledger is an audit trail of the evidence collected; it supports a coverage decision but cannot by itself prove that no behavior was missed.
3. Replace Explicit Waits – Use Playwright’s Actionability & Auto‑Retry
Selenium tests often contain patterns such as:
new WebDriverWait(driver, Duration.ofSeconds(10))
.until(ExpectedConditions.elementToBeClickable(By.id("submit")));
driver.findElement(By.id("submit")).click();Playwright’s philosophy is “wait for the right thing automatically”. The equivalent in TypeScript is:
await page.getByRole('button', { name: 'Submit' }).click(); // waits for the locator to satisfy actionability checksKey points
- Selenium pattern:
Thread.sleep(3000)Playwright replacement: Avoid; rely on locator actions, web-first assertions, or a specific observable condition. KeepwaitForTimeoutonly for temporary debugging, not production tests. Risk note: Sleeping hides real flakiness. - Selenium pattern:
WebDriverWait+ExpectedConditionsPlaywright replacement: Direct interaction (click,fill,selectOption) orawait expect(locator).toBeVisible()Risk note: Auto‑retry asserts visibility, not business correctness. - Selenium pattern: Custom retry loops Playwright replacement: Remove; Playwright already retries actions until they become actionable (see Playwright best practices) Risk note: Over‑retrying can mask real defects.
Important: Auto‑waiting does not guarantee that the UI state matches the business rule. Keep explicit assertions that encode the expected outcome (e.g., “order confirmation number appears”).
References: Playwright documentation on auto-waiting and locators.
4. Preserve Page / Component Abstractions
If your Selenium suite already uses a Page Object Model (POM), you can migrate the structure while moving the implementation to Playwright.
// src/pages/LoginPage.ts
import { Locator, Page } from '@playwright/test';
export class LoginPage {
readonly page: Page;
readonly username: Locator;
readonly password: Locator;
readonly submitBtn: Locator;
constructor(page: Page) {
this.page = page;
this.username = page.getByLabel('Username');
this.password = page.getByLabel('Password');
this.submitBtn = page.getByRole('button', { name: 'Login' });
}
async login(user: string, pass: string) {
await this.username.fill(user);
await this.password.fill(pass);
await this.submitBtn.click();
}
}Keep business assertions in the test files rather than inside page objects. This follows Selenium’s documented Page Object guidance and keeps verification intent visible; Playwright page objects can still encapsulate locators and user-facing operations.
5. Fixture & Browser‑Context Isolation
Playwright’s test runner (@playwright/test) provides built‑in fixtures for browser, context, and page isolation. A minimal playwright.config.ts that defines isolated tests and three browser-engine projects might look like:
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
timeout: 30_000,
expect: { timeout: 5_000 },
fullyParallel: true,
retries: process.env.CI ? 2 : 0,
reporter: [['list'], ['html']],
use: {
headless: true,
viewport: { width: 1280, height: 720 },
ignoreHTTPSErrors: true,
trace: 'on-first-retry',
},
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
],
});Key take‑aways
- Each test gets a fresh browser context, preventing cross‑test state leakage (cookies, storage).
- The
traceoption records a full execution trace on the first retry, which is valuable for the parity ledger. - Parallel execution (
fullyParallel) can reduce elapsed CI time, but resource contention and test-data isolation must be measured in the target environment.
6. Cross‑Browser Projects & CI Integration
GitHub Actions can run the three Playwright projects in parallel, similar to a Selenium Grid. A concise workflow snippet:
name: Playwright Tests
on: [push, pull_request]
jobs:
e2e:
runs-on: ubuntu-latest
strategy:
matrix:
browser: [chromium, firefox, webkit]
steps:
- uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: '20'
- run: npm ci
- run: npx playwright install --with-deps ${{ matrix.browser }}
- run: npx playwright test --project=${{ matrix.browser }}
env:
CI: trueThe workflow uses an official GitHub Actions matrix to run the configured Playwright projects separately; publish reports and traces as workflow artifacts when the team needs retained evidence.
7. Dual‑Running Old & New Suites
To prove that coverage is unchanged, run both suites on the same commit:
- Execute Selenium suite → generate JUnit XML, screenshots, and a failure report.
- Execute Playwright suite → generate the same artifacts (Playwright can output JUnit XML via
--reporter=junit). - Compare the mapped business intent and expected outcome automatically. Treat pass/fail differences as investigation signals, not proof of equivalence, because timing, data, or environment may differ between runs.
If a discrepancy appears, flag the corresponding parity‑ledger entry for investigation before proceeding.
8. Delta Classification & Flake Budget
When differences surface, classify them:
- Category: True regression Description: Business behavior changed (e.g., missing element) Action: Fix the application or update the test intent.
- Category: Expected implementation difference Description: The tools synchronize differently, while the verified business outcome is the same Action: Record the reason and retain explicit business assertions.
- Category: Flake Description: Intermittent failure not reproduced in the other suite Action: Add to the flake budget; consider retry settings or stabilisation work.
Define a team-owned flake budget from the current baseline and track it in the ledger. Playwright retries can help classify a test as flaky, but retries must not be used to hide instability; the first-run failure and retry outcome should both remain visible.
9. Phased Cut‑Over & Rollback Criteria
A risk‑averse migration proceeds in stages:
- Phase: Pilot Goal: Migrate a small set of critical journeys Success Metric: Required ledger entries are
✅ Readyand differences are explained Rollback Trigger: Any unexplained mismatch in a critical journey - Phase: Expand Goal: Add medium-risk journeys in controlled batches Success Metric: Agreed flake budget and evidence-retention criteria are met Rollback Trigger: Regression or instability exceeds the team baseline
- Phase: Full Cut-Over Goal: Retire the Selenium jobs only after the agreed observation window Success Metric: All in-scope ledger entries are ready and rollback has been rehearsed Rollback Trigger: A critical regression or missing coverage is discovered
Rollback means re-enabling the preserved Selenium CI job through a reviewed configuration change while the Playwright implementation remains available for diagnosis and correction.
10. Migration QA Checklist
- ✅ Item: Inventory captured Description: Business journeys, browsers, assertions, data, side‑effects, historical failures.
- ✅ Item: Parity ledger created Description: Every Selenium test mapped to a Playwright test, with owner and evidence.
- ✅ Item: Locators reviewed Description: Use Playwright’s role‑based and text‑based locators; avoid brittle CSS/XPath where possible.
- ✅ Item: Explicit waits removed Description: Replaced by auto‑waiting or purposeful
expectchecks. - ✅ Item: Assertions kept in tests Description: Business‑level verification remains outside page objects.
- ✅ Item: Fixtures defined Description: Browser, context, and page isolation configured.
- ✅ Item: Cross‑browser projects added Description: Chromium, Firefox, WebKit defined in
playwright.config.ts. - ✅ Item: Trace collection enabled Description:
trace: 'on-first-retry'for evidence. - ✅ Item: Dual‑run CI pipeline Description: Selenium and Playwright suites executed on each PR.
- ✅ Item: Delta classification process Description: True regression, expected implementation difference, and flake handling documented.
- ✅ Item: Flake budget agreed Description: Team-defined from the baseline and monitored without hiding first-run failures.
- ✅ Item: Phased cut‑over plan Description: Pilot, expand, full cut‑over with rollback criteria.
- ✅ Item: Documentation updated Description: Test design guidelines reflect Playwright best practices.
11. Where Automation Helps
- Area: Baseline generation Automation benefit: Selenium test runs can be scripted to produce baseline screenshots, logs, and JUnit results for later comparison.
- Area: Parity‑ledger diff Automation benefit: A small Node script can compare JUnit XML from both suites and flag mismatches automatically.
- Area: Flake investigation Automation benefit: Playwright Trace Viewer exposes the action timeline, DOM snapshots, network activity, and attachments that help diagnose intermittent failures.
- Area: Cross‑browser reporting Automation benefit: GitHub Actions matrix produces per‑browser reports without manual orchestration.
- Area: Rollback validation Automation benefit: A reviewed CI flag or preserved job makes rollback testable without deleting either suite prematurely.
Automation does not replace human judgment about what constitutes business correctness; it merely makes the migration evidence reproducible and auditable.
12. Conclusion
Migrating an established Selenium suite to Playwright is not a simple copy‑paste exercise. By inventorying the existing coverage, building a parity ledger, and running both suites side‑by‑side, teams can quantify risk and demonstrate that no behavioral gaps have been introduced. Playwright’s auto‑waiting, robust tracing, and built‑in fixtures simplify test maintenance, but they must be complemented with explicit assertions that capture the true business intent. A phased cut‑over, clear rollback criteria, and a disciplined flake budget keep the migration under control. When executed with this risk-based approach, the transition keeps migration risk visible and gives reviewers reproducible evidence for deciding when each critical journey is ready to cut over.
Review Questions
- What three pieces of information should be captured for each Selenium test before migration?
- How does Playwright’s auto‑waiting differ from Selenium’s explicit
WebDriverWait, and why must assertions still be explicit? - What is the purpose of a parity ledger, and which fields are essential for auditability?
- Describe one concrete way to use Playwright’s tracing feature in the migration process.
- List two criteria that would trigger a rollback during the phased cut‑over.
Sources
- Playwright documentation – Locators, Auto‑waiting, Tracing, Best Practices: https://playwright.dev/docs/locators
- Playwright documentation – Configuration and Projects: https://playwright.dev/docs/test-configuration
- Selenium documentation – WebDriverWait and ExpectedConditions: https://www.selenium.dev/documentation/webdriver/waits/
- GitHub Actions documentation – Matrix builds and workflow syntax: https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-workflow
Leave a comment
Share your thoughts about the article. No account is required.
Comments
Loading comments…