Visual Regression Testing with Selenium Shutterbug: Step-by-Step Setup & Screenshot Comparison

Visual Regression Testing with Maven, Selenium & Shutterbug

Introduction

Visual regression testing is essential for ensuring your website’s user interface remains consistent across updates, plugin changes, or code modifications. This technique compares screenshots from two environments—typically development/staging and live—spotting unwanted UI changes quickly.

In this guide, you’ll learn how to build a Maven-based Java project leveraging the Selenium Shutterbug library to capture full-page screenshots and do pixel-level comparisons automatically.

Step 1: Install Maven

If you are on Linux, the simplest way to install Maven is via Snap, which bundles Maven with Java, so a separate Java install isn’t necessary.

sudo snap install maven --classic

Then verify the installation:

mvn -v

You should see Maven’s version details and Java path in the output.

Step 2: Create a New Maven Project

Generate a new Maven Java project using this command:

mvn archetype:generate -DgroupId=com.visual.testing \
  -DartifactId=visual-regression-checker \
  -DarchetypeArtifactId=maven-archetype-quickstart \
  -DinteractiveMode=false

This will create a folder named visual-regression-checker containing the standard Maven project structure.

Step 3: Locate and Edit pom.xml

Navigate to the new project folder and open pom.xml located at the root.

Replace the <dependencies> section with the following to include Selenium, Shutterbug, WebDriverManager, and other supporting libraries:

<dependencies>
  <!-- JUnit for assertions -->
  <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.13.2</version>
    <scope>test</scope>
  </dependency>

  <!-- Selenium WebDriver -->
  <dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.21.0</version>
  </dependency>

  <!-- AShot for full-page screenshots -->
  <dependency>
    <groupId>ru.yandex.qatools.ashot</groupId>
    <artifactId>ashot</artifactId>
    <version>1.5.4</version>
  </dependency>

  <!-- Selenium Shutterbug for comparison and screenshot utilities -->
  <dependency>
    <groupId>com.assertthat</groupId>
    <artifactId>selenium-shutterbug</artifactId>
    <version>0.9</version>
  </dependency>

  <!-- WebDriverManager for automatic ChromeDriver setup -->
  <dependency>
    <groupId>io.github.bonigarcia</groupId>
    <artifactId>webdrivermanager</artifactId>
    <version>5.7.0</version>
  </dependency>

  <!-- Optional logging -->
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>2.0.13</version>
  </dependency>
</dependencies>

Also, ensure the <build> section includes Maven plugins for compilation and execution:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.8.1</version>
      <configuration>
        <source>11</source>
        <target>11</target>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>exec-maven-plugin</artifactId>
      <version>3.1.0</version>
      <configuration>
        <mainClass>com.visual.testing.ShutterbugScreenshotCompare</mainClass>
      </configuration>
    </plugin>
  </plugins>
</build>

Save and close pom.xml.

Step 4: Install Google Chrome and ChromeDriver

Install Google Chrome on your system if not already installed .

ChromeDriver Installation

Visit the ChromeDriver downloads page and download the version that matches your Chrome version.

For example, if you have Chrome version 114.0.5735.90, download ChromeDriver 114.x.

This setup ensures compatibility between Chrome browser and ChromeDriver used by Selenium.

Step 5: Write the Visual Regression Test Code

Create a Java class named ShutterbugScreenshotCompare.java inside:

visual-regression-checker/src/main/java/com/visual/

Replace its content with this code snippet, updating your dev and live URLs:

package com.visual;

import com.assertthat.selenium_shutterbug.core.Shutterbug;
import com.assertthat.selenium_shutterbug.utils.web.ScrollStrategy;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import com.assertthat.selenium_shutterbug.core.Snapshot;

public class ShutterbugScreenshotCompare {
    public static void main(String[] args) throws Exception {
        WebDriverManager.chromedriver().setup();
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        WebDriver driver = new ChromeDriver(options);

        try {
            String devUrl = "https://your-dev-site.com";
            String liveUrl = "https://your-live-site.com";

            driver.get(devUrl);
            Thread.sleep(4000);
            ((JavascriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight);");
            Thread.sleep(2000);

            Snapshot devSnapshot = Shutterbug.shootPage(driver, ScrollStrategy.BOTH_DIRECTIONS, 700, true).withName("dev");
            devSnapshot.save("screenshots");

            driver.get(liveUrl);
            Thread.sleep(4000);
            ((JavascriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight);");
            Thread.sleep(2000);

            Snapshot liveSnapshot = Shutterbug.shootPage(driver, ScrollStrategy.BOTH_DIRECTIONS, 700, true).withName("live");
            liveSnapshot.save("screenshots");

            boolean isSame = devSnapshot.equalsWithDiff(liveSnapshot.getImage(), "screenshots/diff.png", 0.1);

            if (isSame) {
                System.out.println("✅ DEV and LIVE screenshots match.");
            } else {
                System.out.println("❗ Screenshots differ. Check screenshots/diff.png for details.");
            }
        } finally {
            driver.quit();
        }
    }
}

Step 6: Compile and Run Your Visual Regression Tests

Navigate in terminal to the project directory:

cd visual-regression-checker

Compile the project:

mvn clean compile

Run the tests:

mvn exec:java -Dexec.mainClass="com.visual.ShutterbugScreenshotCompare"

Output

After running, your project folder will contain a new visual-regression-checker/screenshots/ folder with:

• dev.png: Dev/staging environment screenshot
• live.png: Live site screenshot
• diff.png: Visual difference overlay (only present if differences exceed tolerance)

Tips to Avoid Common Issues

• Make sure page layouts between dev and live are similar; otherwise, differences in length or dynamic content may cause false positives.
• Adjust Thread.sleep() duration to ensure pages fully load before screenshotting.
• Modify the tolerance parameter in equalsWithDiff for sensitivity (currently set to 0.1, meaning 10%).

Conclusion

This automated visual regression testing setup using Maven and Selenium Shutterbug allows you to easily detect subtle UI regressions between your staging and production environments. This approach helps you maintain a consistent and polished user experience.

Note: Although demonstrated here independently, this solution works not only for WordPress sites but also for any web platform, including Drupal, Next.js, React-based sites, and more. The method is universally applicable wherever you can access the URL of the site’s environment to capture screenshots.