Deploying Playwright on Vercel: Overcoming Challenges

Deploying Playwright on Vercel can present several challenges, especially due to the platform's strict 50MB function size limit. While Playwright’s Chromium binary typically exceeds 280MB, standard setups often fail in serverless environments that restrict large dependencies and lack built-in browser support. In this guide, discover a tested solution that leverages the @sparticuz/chromium library, a lightweight Chromium build optimized for AWS Lambda and Vercel.

Steps to Deploy Your Playwright Scraper on Vercel

1. Setting Up Your Environment
2. Configuring Playwright for Serverless
3. Writing the Scraping Script
4. Deploying to Vercel

This tutorial will guide you through these steps while building a simple Playwright web scraper that targets a sample e-commerce demo site.

1. Setting Up Your Environment

Before diving into coding, ensure your environment is equipped with the necessary tools:

• Node.js: A JavaScript runtime environment. Install the latest version if you haven't already.
• playwright-core: A Playwright variant sans browser binaries.
• @sparticuz/chromium: A lightweight, precompiled Chromium binary optimized for serverless environments.

Start by initializing a Node.js project and installing these required packages:

npm init -y
npm install @sparticuz/chromium playwright-core

Next, create an api directory and a scraper.js file inside the project root, along with a vercel.json file to configure the serverless functions.

2. Configuring Playwright for Serverless

Configuring Vercel for optimal serverless function performance is vital. Update your vercel.json file with the following:

{
    "functions": {
        "api/scraper.js": {
            "memory": 1024,
            "maxDuration": 10
        }
    },
    "routes": [
        {
            "src": "/api/scraper",
            "dest": "api/scraper.js"
        }
    ]
}

In this configuration:

• The functions section allocates 1GB of memory with a timeout of 10 seconds.
• The routes section defines how HTTP requests are mapped to your scraper API.

3. Writing the Basic Scraping Script

Writing effective scraping scripts involves ensuring compatibility both locally and remotely. Your scraper.js script will evolve from a basic functionality testing phase to a production-ready implementation:

const { chromium } = require('playwright-core');

const scraper = async (req, res) => {
    try {
        const executablePath = await chromiumBinary.executablePath();
        const browser = await chromium.launch({
            args: chromiumBinary.args,
            executablePath: executablePath,
            headless: true,
        });
        const context = await browser.newContext();
        const page = await context.newPage();
        await page.goto('https://www.scrapingcourse.com/ecommerce/');
        const htmlContent = await page.content();
        await browser.close();
        res.status(200).json({ htmlContent });
    } catch (error) {
        res.status(500).json({ error: 'Failed to scrape page' });
    }
};
module.exports = scraper;

This script efficiently manages request handling and returns the scraped HTML as a JSON response.

4. Deploying to Vercel

To deploy your project, first create an account on Vercel. Then install the Vercel CLI:

npm install -g vercel
vercel login

Deploy the project in preview mode using:

vercel

Follow the prompts to setup and complete the deployment.

Handling Common Deployment Issues

Deploying Playwright applications to Vercel can lead to issues stemming from memory limitations, timeout problems, or missing browser binaries. Here are solutions:

• Memory Issues: Monitor resource allocation and optimize your application by blocking unnecessary resources like images.
• Timeouts: Increase the timeout limit in your vercel.json file to accommodate web scraping tasks.
• Browser Binary Errors: Use the right binaries by implementing and correctly pointing to @sparticuz/chromium.

Furthermore, consider maintaining a persistent browser instance to avoid cold start delays and manage resources effectively. Implement periodic cleanup routines to keep your environment optimized throughout the scraping process.

Conclusion

With these straightforward steps, you can successfully host your Playwright scraper on Vercel, while the flexibility of the @sparticuz/chromium library ensures efficient resource usage. For large-scale projects, utilizing cloud-based solutions like the ZenRows Scraping Browser offers added benefits of scalability and automation.