Troubleshooting | Applitools Documentation

On this page

This topic addresses some of the common issues that you may encounter when using Autonomous.

Aborted Tests

A test is marked as Aborted if it did not complete successfully or was stopped by the user. If the test was aborted by the system, check that the tunnel is configured correctly. For details see Set Up a Tunnel to Test on Internal Networks.

If the tunnel is configured correctly, you should look at test details and may need to contact support. For details see Viewing or Copying Test Details.

CAPTCHA Challenges

Autonomous runs your application using an automated browser, which CAPTCHA and bot-detection services ( Google reCAPTCHA, Cloudflare, hCaptcha, Adobe Experience Manager, and similar) are designed to identify and block. When that happens, your test may fail to submit a form, render an interaction as "no action", or display a visible challenge.

How to Tell if CAPTCHA Is the Cause

Look for one of the following symptoms:

• A visible CAPTCHA challenge appears in the test result screenshots.
• A form click is registered, but the form is never submitted and the flow stops on the same page.
• A submission returns a server error (typically 400 Bad Request) with a response body that mentions captcha, RecaptchaToken, or CAPTCHA validation failed.
• The test passes when you run it manually in your own browser, but fails when run by Autonomous.

To confirm CAPTCHA is the cause, open the test in the interactive browser, open the browser developer tools (F12), and reproduce the step. Check the Network tab for the failing request and look at the response body. A response that references CAPTCHA validation confirms the diagnosis.

Solution 1: Allowlist the Applitools IP Addresses

This is the recommended first step in every environment.

Add the Applitools Autonomous egress IP addresses to your application's firewall, WAF, or CAPTCHA provider configuration so that traffic from those addresses is treated as trusted and is not challenged. The full list of IPs by region is in Whitelist IP Addresses.

Important notes:

• A simple "trusted IP" entry is sometimes not enough on its own. Several CAPTCHA and WAF providers (notably Cloudflare) fingerprint the browser in addition to the source IP. If you allowlist the IPs and still see challenges, see Provider-Specific Guidance below.
• If you use a tunnel, allowlist the tunnel IPs as well.

Solution 2: Lower or Disable the CAPTCHA in Non-Production

In development, QA, and staging environments, the simplest and most reliable approach is to lower the CAPTCHA's sensitivity or disable it entirely for the Applitools IPs.

For score-based Google reCAPTCHA (classic reCAPTCHA v3 and reCAPTCHA Enterprise), customers have successfully set the score threshold to 0 for the Applitools IPs, which effectively passes the check for traffic originating from those addresses without disabling CAPTCHA for end users.

In production, full CAPTCHA bypass is usually not acceptable. In that case, combine Solution 1 with Solution 3.

Solution 3: Provider-Specific Guidance

Different CAPTCHA and bot-detection providers require different configuration. The following notes describe what to change on your side.

Google reCAPTCHA

• Add the Applitools IPs to your site key's allowed configuration in the reCAPTCHA admin console.
• For score-based reCAPTCHA (classic v3 or reCAPTCHA Enterprise), lower the score threshold for the Applitools IPs, or skip the verification call server-side when the request comes from one of the Applitools IPs.

Cloudflare (Managed Challenge, Turnstile, Bot Fight Mode)

Allowlisting the IPs through Cloudflare's general "trusted IPs" list is often not sufficient because Cloudflare's bot detection also uses browser fingerprinting in addition to the source IP.

Instead, create an explicit WAF custom rule that:

• Matches requests from the Applitools IPs (use Cloudflare's ip.src in expression with the IPs from Whitelist IP Addresses).
• Sets the action to skip Managed Challenge, Bot Fight Mode, and any other security products you have enabled.

Apply the rule only in the hostname or path scope you want Autonomous to be able to test.

hCaptcha

• Add the Applitools IPs to your site's allowlist in the hCaptcha dashboard.
• For server-side verification, skip the hCaptcha check when the request originates from an Applitools IP.

Adobe Experience Manager (AEM)

AEM-hosted forms often perform their own CAPTCHA validation server-side before submitting. Allowlisting at the WAF layer is not enough on its own — the AEM form configuration itself needs to skip CAPTCHA validation for the Applitools IPs, or the CAPTCHA component needs to be disabled on the test environment.

Solution 4: Solve the CAPTCHA Inside the Test Flow

If your application uses a CAPTCHA that you control and that is deterministic (for example, a math problem CAPTCHA), you can solve it as part of the test flow using a JavaScript step in a custom flow.

This approach works when:

• You own the CAPTCHA implementation.
• The challenge is solvable from the page contents (a math expression, a known token, a fixed answer).

It does not work for third-party CAPTCHA services such as Google reCAPTCHA or Cloudflare, because those services are specifically designed to resist programmatic solving.

For details on adding JavaScript steps to a flow, see Authoring a Test.

When Autonomous Cannot Bypass a CAPTCHA

Autonomous is a bot, and CAPTCHA services on sites you do not control will block it. Examples include search engines, social media login pages, and other third-party services.

If your test depends on interacting with a CAPTCHA-protected site that your organization does not own, there is no general bypass available. Restructure the test to avoid the third-party step (for example, mock the dependency or test against a staging environment that does not include it).

Still Stuck? Contact Support

If you have applied the relevant solutions and your test is still being blocked, contact Applitools support and include the following so we can route to the right team quickly:

• The Plan ID or Test ID of a failing run.
• The CAPTCHA / bot-detection provider in use (Google reCAPTCHA, Cloudflare, hCaptcha, AEM, custom, or other).
• The environment (development, staging, production).
• Whether the Applitools IPs are currently allowlisted, and if so, at which layer (firewall, WAF, CAPTCHA provider, application).
• The response body of the failing request, captured from the browser developer tools as described in How to Tell if CAPTCHA Is the Cause.

Some enterprise WAF and CAPTCHA configurations (for example, those that fingerprint beyond the source IP and cannot be configured by rule) require a coordinated bypass that Applitools support arranges with your security team. Contact support to start that conversation.

Server Errors

Following are common server errors:

• 401 (Unauthorized) – Autonomous does not have permission to access this page. This can occur if the page is password-protected, requires credentials, or you need to accept cookies before the test opens. To resolve this issue, you can add basic authentication or a customized pre-test. For details, see Flow to Run Before Steps.
• Error 403 (Forbidden) – The server is blocking access. To resolve this issue, add the website to a whitelist (for assistance contact support), or access the website using a tunnel. For details, see Set Up a Tunnel to Test on Internal Networks.
• Error 404 (Not found) – the URL is not currently available, try again later. If this page no longer exists, you should remove it from the sitemap or the list of URLs.
• Error 429 (Too many requests) – The server is blocking access. To resolve this issue, add the website to a whitelist (for assistance contact support), or access the website using a tunnel. For details, see Set Up a Tunnel to Test on Internal Networks.
• 500 (Internal Server Error) – Autonomous is unable to establish a connection with the server. To confirm that the server is available, try accessing the URL with your browser.
• 503 (Service Unavailable) – Autonomous is unable to establish a connection with the server, the server may be temporarily overloaded or down for maintenance. To confirm that the server is available, try accessing the URL with your browser.

Full Website Test Errors

No sitemap file found

Problem:

Autonomous is unable to find the sitemap, or there is no sitemap file.

Solution:

By default, Autonomous looks for a sitemap file in the root directory, the file must be named sitemap.xml or listed in the robots.txt file. This error appears if Autonomous is unable to find a sitemap file. To resolve this problem try one of the following:

• Add a sitemap file named sitemap.xml to the root directory.
• If there is a sitemap file with a different name or in a different location, when creating the full website test, enter the full path to the sitemap file instead of the root directory.
• If there is no sitemap file and you are unable to create one, create a URL list test instead of a Full Website Test. See URL List Test

Test timed out

Problem:

The test timed out

Solution:

This could be caused by a network issue, try again later.

Test includes too many screens

Problem:

An error message appears that not all screens were included in the test or one or more directory was excluded from the test.

Solution:

There is a limit on the number of screens that can be included in a single test. The maximum number is based in your Autonomous plan. If your sitemap includes too many URLs, try one of the following:

• Manually select URLs that should be excluded from the test. For details, see Excluding or Adding URLs.
• Create a separate test for each sub-directory
• Contact Support to increase the number of URLs allowed in your plan

Sitemap processing failed

This error is caused because the server timed out before it was able to process the complete sitemap, this is normally because the sitemap is extremely large.

If the sitemap includes multiple nested sitemaps, you should create a separate full website test for each nested sitemap. When you create the test, instead of entering the URL of your website, enter the full path to a nested sitemap.

Unable to create test with sub-folder

Problem:

An "Unknown Error" occurred when creating a test for a sub-directory (for example http://www.mynewssite.com/sport)

Solution:

When creating a test for a sub-directory, Autonomous uses the sitemap in the root folder to identify screens in the sub-directory. If the sub-directory has a separate sitemap, when you create the test, enter the full path to the sitemap for the sub-directory.