Testing web apps in C# using Playwright | Applitools Documentation

On this page

This quickstart will show you how to visually test web apps in C# using Playwright. Visual testing can help you catch problems that traditional automation struggles to find.

After preparing your environment, this three-step quickstart should take about 15 minutes to complete.

Need extra help?
If you get stuck on this example, don't suffer in silence! Please reach out to us to get things working. We can also help you get Applitools running in your own project.

Step 1: Preparing your environment

You'll need a few things to run this quickstart:

1. An Applitools account, which you can register for free.

2. The .NET 7 SDK for building and running the project.

3. A good C# editor:

• Use Microsoft Visual Studio if you want to drive development through a full IDE. Visual Studio comes bundled with the .NET SDK and all the tools you need out of the box. However, you will need to add a .runsettings file or overwrite some variables in the code to run the tests through Visual Studio.
  • Use Visual Studio Code if you want a faster experience driven more by the command line. You will need to install the C# extension and use the dotnet CLI tool from the terminal. You will also need to install the .NET SDK separately. Setting environment variables to run tests will be easier.

4. An up-to-date version of Google Chrome.

5. A corresponding version of ChromeDriver.

Installing ChromeDriver

The major version numbers of Chrome and ChromeDriver must be the same. Otherwise, Selenium WebDriver may raise an exception when it is initialized. For example, Chrome v101 requires ChromeDriver v101. ChromeDriver must be installed into a directory covered by the system PATH variable. Follow the instructions on Selenium's Install browser drivers page. On macOS and Linux, the recommended location for the chromedriver executable is the /usr/local/bin directory. You can test that ChromeDriver is working by running the chromedriver -v command to print its version.

Step 2: Getting your example project

Downloading the example project

The example project is located at https://github.com/applitools/example-playwright-csharp. Clone this repository to your local machine:

git clone https://github.com/applitools/example-playwright-csharp.git
cd example-playwright-csharp

Don't have Git?

Instead of running git clone, you can download the project as a ZIP file and extract it.

Installing the dependencies

The example project uses NuGet for package management. All dependencies are listed in the project's .csproj file. They should be automatically downloaded whenever the project is built. You can explicitly restore NuGet packages from either Visual Studio or the command line by following this guide: Restore packages with NuGet Package Restore.

Adding Applitools to Another .NET Project?

This example project already has the Applitools Eyes SDK as a dependency. There are two NuGet packages for the Applitools Eyes SDK for different Selenium versions:

• Eyes.Selenium4 uses Selenium 4
• Eyes.Selenium uses Selenium 3

Pick the Selenium version you want, and add the appropriate package to your .NET project. To learn how to add NuGet packages to projects, follow the official guides for Visual Studio for Windows, Visual Studio for Mac, or the dotnet CLI.

Deciding how to run tests

The test uses Selenium WebDriver to automate the browser. There are two places to run your Selenium WebDriver session:

1. On your local machine (which is the default option)
2. On Applitools Execution Cloud

If you run WebDriver locally, then you need to set it up and manage it yourself. If you use Execution Cloud, then Applitools will manage WebDriver for you. It will also automatically heal broken locators and wait for elements to be ready. If you would like to try Execution Cloud, please request access. The docs for Execution Cloud provide more information.

There are two ways to test the visual snapshots captured by the test:

1. Using Applitools Ultrafast Grid for cross-browser testing in the cloud
2. Using Applitools Classic runner on your local machine

Walking through the code

The project contains one visual test case, which is located at Applitools.Example.Tests/AcmeBankTest.cs.

All test setup and cleanup is handled by hook methods. The variables at the top control how tests will run:

• Set USE_ULTRAFAST_GRID to true to use Ultrafast Grid or false to use the Classic runner.

Setup varies slightly for these different options. In-line comments explain every section of its code. Read it top to bottom to understand how it works:

using System;
using System.Threading.Tasks;
using Applitools;
using Applitools.Playwright;
using Applitools.Playwright.Fluent;
using Applitools.Utils.Geometry;
using Applitools.VisualGrid;
using Microsoft.Playwright;
using NUnit.Framework;
using BrowserType = Applitools.BrowserType;

namespace ExamplePlaywrightCSharp;

public class AcmeBankTests
{
    // This NUnit test case class contains everything needed to run a full visual test against the ACME bank site.
    // It runs the test once locally.
    // If you use the Ultrafast Grid, then it performs cross-browser testing against multiple unique browsers.

#pragma warning disable CS8618

// Runner Settings.
    // These could be set by environment variables or other input mechanisms.
    // They are hard-coded here to keep the example project simple.
    private static readonly bool UseUltrafastGrid = true;

// Test control inputs to read once and share for all tests
    private static string? ApplitoolsApiKey;
    private static bool Headless;

// Applitools objects to share for all tests
    private static BatchInfo Batch;
    private static Configuration Config;
    private static PlaywrightEyesRunner Runner;

// Test-specific objects
    private static IPlaywright Playwright;
    private static IBrowser Browser;
    private IBrowserContext Context;
    private IPage Page;
    private Eyes Eyes;

#pragma warning restore CS8618

[OneTimeSetUp]
    public static async Task SetUpConfigAndRunner()
    {
        // This method sets up the configuration for running visual tests.
        // The configuration is shared by all tests in a test suite, so it belongs in a `BeforeAll` method.
        // If you have more than one test class, then you should abstract this configuration to avoid duplication.

// Read the Applitools API key from an environment variable.
        ApplitoolsApiKey = Environment.GetEnvironmentVariable("APPLITOOLS_API_KEY");

// Read the headless mode setting from an environment variable.
        // Use headless mode for Continuous Integration (CI) execution.
        // Use headed mode for local development.
        Headless = bool.Parse(Environment.GetEnvironmentVariable("HEADLESS") ?? "true");

if (UseUltrafastGrid)
        {
            // Create the runner for the Ultrafast Grid.
            // Concurrency refers to the number of visual checkpoints Applitools will perform in parallel.
            // Warning: If you have a free account, then concurrency will be limited to 1.
            Runner = new VisualGridRunner(new RunnerOptions().TestConcurrency(5));
        }
        else
        {
            // Create the Classic runner.
            Runner = new ClassicRunner();
        }

// Create a new batch for tests.
        // A batch is the collection of visual checkpoints for a test suite.
        // Batches are displayed in the Eyes Test Manager, so use meaningful names.
        string runnerName = (UseUltrafastGrid) ? "Ultrafast Grid" : "Classic runner";
        Batch = new BatchInfo("Example: Playwright C# with the " + runnerName);

// Create a configuration for Applitools Eyes.
        Config = new Configuration();

// Set the Applitools API key so test results are uploaded to your account.
        // If you don't explicitly set the API key with this call,
        // then the SDK will automatically read the `APPLITOOLS_API_KEY` environment variable to fetch it.
        Config.SetApiKey(ApplitoolsApiKey);

// Set the batch for the config.
        Config.SetBatch(Batch);

// If running tests on the Ultrafast Grid, configure browsers.
        if (UseUltrafastGrid)
        {
            // Add 3 desktop browsers with different viewports for cross-browser testing in the Ultrafast Grid.
            // Other browsers are also available, like Edge and IE.
            Config.AddBrowser(800, 600, BrowserType.CHROME);
            Config.AddBrowser(1600, 1200, BrowserType.FIREFOX);
            Config.AddBrowser(1024, 768, BrowserType.SAFARI);

// Add 2 mobile emulation devices with different orientations for cross-browser testing in the Ultrafast Grid.
            // Other mobile devices are available, including iOS.
            Config.AddDeviceEmulation(DeviceName.Pixel_2, ScreenOrientation.Portrait);
            Config.AddDeviceEmulation(DeviceName.Nexus_10, ScreenOrientation.Landscape);
        }

// Start Playwright and launch the browser.
        Playwright = await Microsoft.Playwright.Playwright.CreateAsync();
        Browser = await Playwright.Chromium
            .LaunchAsync(new BrowserTypeLaunchOptions { Headless = Headless });
    }

[SetUp]
    public async Task OpenBrowserAndEyes()
    {
        // This method sets up each test with its own Page and Applitools Eyes objects.

// Get a new context from the browser
        Context = await Browser.NewContextAsync();

// Create a new page in the context.
        // Creating a new context is not mandatory and a new page can be created from the browser instance.
        // page_ = await browser_.NewPageAsync();
        Page = await Context.NewPageAsync();

// Create the Applitools Eyes object connected to the runner and set its configuration.
        Eyes = new Eyes(Runner);
        Eyes.SetConfiguration(Config);

// Open Eyes to start visual testing.
        // It is a recommended practice to set all four inputs:
        Eyes.Open(

// The page to "watch".
            Page,

// The name of the application under test.
            // All tests for the same app should share the same app name.
            // Set this name wisely: Applitools features rely on a shared app name across tests.
            "ACME Bank Web App",

// The name of the test case for the given application.
            // Additional unique characteristics of the test may also be specified as part of the test name,
            // such as localization information ("Home Page - EN") or different user permissions ("Login by admin").
            TestContext.CurrentContext.Test.Name,

// The viewport size for the local browser.
            // Eyes will resize the web browser to match the requested viewport size.
            // This parameter is optional but encouraged in order to produce consistent results.
            new RectangleSize(1200, 600));
    }

[Test]
    public async Task LogIntoBankAccount()
    {
        // This test covers login for the Applitools demo site, which is a dummy banking app.
        // The interactions use typical Selenium WebDriver calls,
        // but the verifications use one-line snapshot calls with Applitools Eyes.
        // If the page ever changes, then Applitools will detect the changes and highlight them in the Eyes Test Manager.
        // Traditional assertions that scrape the page for text values are not needed here.

// Load the login page.
        await Page.GotoAsync("https://demo.applitools.com");

// Verify the full login page loaded correctly.
        Eyes.Check(Target.Window().Fully().WithName("Login page"));

// Perform login.
        await Page.Locator("#username").FillAsync("andy");
        await Page.Locator("#password").FillAsync("i<3pandas");
        await Page.Locator("#log-in").ClickAsync();

// Verify the full main page loaded correctly.
        // This snapshot uses LAYOUT match level to avoid differences in closing time text.
        Eyes.Check(Target.Window().Fully().WithName("Main page").Layout());
    }

[TearDown]
    public async Task CleanUpTest()
    {
        // Close Eyes to tell the server it should display the results.
        Eyes.CloseAsync();

// Close the page.
        await Page.CloseAsync();

// Warning: `eyes_.CloseAsync()` will NOT wait for visual checkpoints to complete.
        // You will need to check the Eyes Test Manager for visual results per checkpoint.
        // Note that "unresolved" and "failed" visual checkpoints will not cause the JUnit test to fail.

// If you want the JUnit test to wait synchronously for all checkpoints to complete, then use `eyes_.Close()`.
        // If any checkpoints are unresolved or failed, then `eyes_.Close()` will make the JUnit test fail.
    }

[OneTimeTearDown]
    public static void PrintResults()
    {
        // Close the Playwright instance.
        Playwright.Dispose();

// Close the batch and report visual differences to the console.
        // Note that it forces JUnit to wait synchronously for all visual checkpoints to complete.
        TestResultsSummary allTestResults = Runner.GetAllTestResults();
        TestContext.Out.WriteLine(allTestResults);
    }
}

Step 3: Running your tests

Setting Applitools variables

Before running the visual test, you must find your Applitools API key and set it as an environment variable named APPLITOOLS_API_KEY. You may set it through your IDE (if applicable), or you may set it from the command line like this:

• macOS and Linux
• Windows

export APPLITOOLS_API_KEY=<your-api-key>

set APPLITOOLS_API_KEY=<your-api-key>

Having trouble with environment variables?

If you have trouble setting the APPLITOOLS_API_KEY environment variable, you can hard-code your API key like this:

// Change the following line:
// ApplitoolsApiKey = Environment.GetEnvironmentVariable("APPLITOOLS_API_KEY");
ApplitoolsApiKey = "<your-api-key>";

However, be warned: hard-coding secrets is poor practice. Do this only temporarily for debugging, and never commit hard-coded secrets to version control.

You may also need to set your Applitools Eyes server. By default, tests will use the public Applitools Eyes server at eyes.applitools.com. However, if your team is using a private Applitools Eyes server, you can target it by setting the APPLITOOLS_SERVER_URL environment variable. (If you are using a free Applitools account, then use the public server.)

Launching visual tests

To run tests from the command line (or Visual Studio Code's terminal), run:

dotnet build
pwsh Applitools.Example.Tests/bin/Debug/net7.0/playwright.ps1 install
dotnet test

If pwsh is not available, you must install PowerShell.

To run tests from Visual Studio, build the project and launch the tests from Test Explorer. To run it headlessly, set the HEADLESS environment variable to true.

After your tests run, you should see results in the Eyes Test Manager. You can log into the Test Manager at eyes.applitools.com or at the address for your private Applitools Eyes server.

When you run tests with the Applitools Ultrafast Grid, the tests will run one time on the local machine, and then they will upload snapshots to render on each target configuration in the cloud. The Test Manager will show a separate result for each rendering. When you run tests with the Applitools Classic runner, the Test Manager will show the one snapshot from your local machine.

You can also change the web page to inject visual bugs:

// await Page.GotoAsync("https://demo.applitools.com");
await Page.GotoAsync("https://demo.applitools.com/index_v2.html");

If you rerun the tests, then they should yield "unresolved" results for you to review. Visual differences will be highlighted in magenta. It's up to you to accept (👍) or reject (👎) the changes. Applitools will remember your decisions for future analysis.

Need extra help?

Again, it's okay.

If you get stuck on this example, don't suffer in silence! Please reach out to us to get things working. We can also help you get Applitools running in your own project.

Taking the next steps with Applitools

Congratulations on completing this quickstart! There's still so much to learn about visual testing with Applitools, but you're off to a great start.

Resources for next steps:

1. Setting match levels for visual checkpoints
2. Organizing Tests Into Batches
3. Run A/B Tests

You can also: Request a demo