SEO & AI Bot Behavior Testing Platform

About This Project

OpenSeoTest is an open-source platform for performing technical SEO testing. The goal is to provide a transparent laboratory to understand how Googlebot, search crawlers, and AI bots behave with different JavaScript content and techniques.

We believe that the SEO space needs more open and replicable experiments, not just opinions. On this platform, access logs are open for everyone, allowing you to verify your hypotheses immediately with real data.

We regularly publish our experiment results and deep-dive researches on the EdgeComet Blog.

Whether you are testing dynamic rendering implementations or researching bot capabilities, this project enables you to create new tests, deploy them, and observe the results.

Open Science Replicable experiments with open access logs

Bot Behavior Analyze how crawlers render JS & AJAX

Community Driven Create and deploy your own test cases

How It Works

The Debug Hash System

Every request generates a unique 8-character Debug Hash. This hash is injected into the URL, HTML content, and X-Debug-Hash headers. It acts as a "fingerprint" that allows us to distinguish between the initial server-side response and subsequent client-side executions.

Beacon Correlation

When a bot renders JavaScript, our beacon fires the same unique hash back to the server. This confirms that the JS was executed and allows us to verify exactly what content the bot was able to "see" and interact with.

Log Analysis

Our open dashboard lets you analyze debug hashes in real-time. Cross-reference server logs with JS beacon data to see the exact timeline of bot rendering behavior across the platform. Explore the logs.

Test Categories

Select a category to begin testing. Each test includes specific behaviors to verify bot rendering capabilities.

JavaScript Injection

Evaluates whether bots can execute JavaScript and how long they are willing to wait for content to appear after the initial page load. Crucial for client-side rendered apps.

DOMContentLoaded Injection 100ms Timeout 100ms 250ms Timeout 250ms 500ms Timeout 500ms 1000ms Timeout 1000ms 2000ms Timeout 2000ms

AJAX Delays

Measures the bot's ability to fetch external data asynchronously. Tests different server response times to determine the timeout thresholds of various crawlers.

500ms Delay 500ms 1000ms Delay 1000ms 2000ms Delay 2000ms 3000ms Delay 3000ms 4000ms Delay 4000ms 5000ms Delay 5000ms

AJAX Chain

Simulates complex loading dependencies where one request must finish before the next begins (Waterfall). Essential for testing Single Page Applications (SPAs) with nested data requirements.

3-Step Chain 3 steps 5-Step Chain 5 steps

JavaScript Errors

Checks the resilience of crawlers when encountering script errors. Does a syntax or runtime error in one script block prevent the indexing of subsequent content?

Syntax Error Before Injection Runtime Error Before Injection Error Between Injections

Realtime Updates

Observes how bots interact with content that changes in real-time. Do they take a static snapshot, or can they perceive dynamic updates over a short period?

Timer Display

HTTP Status Codes

Verifies standard protocol compliance. Ensures that bots correctly interpret redirects (301/302) and error states (404/500) rather than indexing error pages.

301 Permanent Redirect HTTP 301 302 Temporary Redirect HTTP 302 404 Not Found HTTP 404 500 Internal Server Error HTTP 500 503 Service Unavailable HTTP 503

Semantic HTML

Tests AI comprehension of semantic HTML vs utility CSS classes

Product Page (Semantic) Product Page (Semantic) Product Page (Tailwind) Product Page (JSON-LD) Product Page (JSON-LD in Head) Product Page (JSON-LD 2) Job Listing (Semantic) Job Listing (Tailwind)

Loading Time

Tests how long bots wait for server response. Server-side delays simulate slow page generation.

1 Second Delay 1000ms 2 Second Delay 2000ms 3 Second Delay 3000ms 4 Second Delay 4000ms 5 Second Delay 5000ms 6 Second Delay 6000ms 7 Second Delay 7000ms 8 Second Delay 8000ms 9 Second Delay 9000ms 10 Second Delay 10000ms

robots.txt Compliance

Tests whether AI bots respect robots.txt Disallow directives. A blocked page contains a secret code - if the AI quotes it, the bot ignores robots.txt.

Secret Page Test

Quick Start: Test AI Capabilities Yourself

You don't need to be a developer to see how AI bots handle the web. Try this simple experiment:

Pick a test: Right-click and copy the link for the AJAX Load (2s) test.
Open an AI Chat: Go to ChatGPT, Claude, or Gemini.
Ask a question: Paste the link and ask: "What is the price of the product on this page?"
Verify: If the AI answers with the correct price (which is generated via JavaScript after a delay), you know it successfully executed the code!

Popular Tests to Try

JS Injection (500ms) Content appears after half-second delay AJAX Load (2s) Product data fetched via AJAX Chained AJAX Sequential loading of multiple items

Open Source & Contributions

OpenSeoTest is a community-driven project. We encourage SEO researchers and developers to contribute new test cases, improve existing ones, or help with the platform's infrastructure.

View on GitHub