Running headless browsers in production is one of the most deceptively complex problems in web development. What starts as a simple Puppeteer script quickly evolves into managing Chromium binaries, handling memory leaks, dealing with zombie processes, and scaling browser pools across servers.
A headless browser API abstracts all of that away. Instead of managing Chrome instances yourself, you make an HTTP request and get back a screenshot, PDF, scraped content, or extracted data. This guide covers both approaches: building your own headless browser setup and using an API to skip the infrastructure entirely.
Why Headless Browsers Are Hard to Self-Host
Before diving into code, it's worth understanding why teams eventually move away from self-hosted headless browsers:
- Memory leaks: Chromium consumes 200-500MB per instance. A single leaked browser context in a loop can OOM your server in minutes.
- Zombie processes: Crashed browser processes linger, consuming CPU and file descriptors until the server runs out.
- Font rendering: Headless Linux environments miss system fonts. Text renders as boxes or fallback glyphs without manual font installation.
- Anti-bot detection: Sites detect headless Chrome via navigator properties, WebGL fingerprints, and timing analysis. Stealth plugins help but break with every Chrome update.
- Scaling: Each concurrent capture needs its own browser context. 50 concurrent users means 50 Chromium instances — that's 10-25GB of RAM.
- Dependency hell: Chromium requires ~400MB of system libraries. Docker images bloat. Version mismatches cause silent failures.
Self-Hosted Headless Browser Setup
If you want to run your own headless browser infrastructure, here's what a production-ready setup looks like with Playwright:
Basic Browser Pool
import { chromium, Browser, BrowserContext, Page } from 'playwright';
interface BrowserInstance {
browser: Browser;
contexts: number;
createdAt: number;
}
class HeadlessBrowserPool {
private pool: BrowserInstance[] = [];
private maxBrowsers: number;
private maxContextsPerBrowser: number;
private maxBrowserAge: number;
constructor(options: {
maxBrowsers?: number;
maxContextsPerBrowser?: number;
maxBrowserAgeMs?: number;
} = {}) {
this.maxBrowsers = options.maxBrowsers ?? 5;
this.maxContextsPerBrowser = options.maxContextsPerBrowser ?? 10;
this.maxBrowserAge = options.maxBrowserAgeMs ?? 30 * 60 * 1000;
}
async acquire(): Promise<BrowserContext> {
// Recycle old browsers
const now = Date.now();
for (const instance of this.pool) {
if (now - instance.createdAt > this.maxBrowserAge) {
await instance.browser.close().catch(() => {});
this.pool = this.pool.filter(i => i !== instance);
}
}
// Find browser with capacity
const available = this.pool.find(
i => i.contexts < this.maxContextsPerBrowser
);
if (available) {
available.contexts++;
return available.browser.newContext();
}
// Launch new browser if pool not full
if (this.pool.length < this.maxBrowsers) {
const browser = await chromium.launch({
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-dev-shm-usage',
'--disable-gpu',
'--single-process',
],
});
const instance: BrowserInstance = {
browser,
contexts: 1,
createdAt: now,
};
this.pool.push(instance);
return browser.newContext();
}
throw new Error('Browser pool exhausted');
}
async release(context: BrowserContext): Promise<void> {
await context.close().catch(() => {});
const instance = this.pool.find(
i => i.contexts > 0
);
if (instance) instance.contexts--;
}
async shutdown(): Promise<void> {
await Promise.all(
this.pool.map(i => i.browser.close().catch(() => {}))
);
this.pool = [];
}
}
const pool = new HeadlessBrowserPool({
maxBrowsers: 5,
maxContextsPerBrowser: 10,
maxBrowserAgeMs: 30 * 60 * 1000,
});
Screenshot Capture Function
async function captureScreenshot(url: string, options: {
width?: number;
height?: number;
fullPage?: boolean;
format?: 'png' | 'jpeg' | 'webp';
quality?: number;
waitFor?: string;
blockAds?: boolean;
} = {}): Promise<Buffer> {
const context = await pool.acquire();
try {
const page = await context.newPage();
await page.setViewportSize({
width: options.width ?? 1280,
height: options.height ?? 720,
});
// Block unnecessary resources for speed
if (options.blockAds) {
await page.route('**/*', (route) => {
const type = route.request().resourceType();
if (['media', 'font'].includes(type)) {
return route.abort();
}
const url = route.request().url();
if (url.includes('googleads') || url.includes('doubleclick')) {
return route.abort();
}
return route.continue();
});
}
await page.goto(url, {
waitUntil: 'networkidle',
timeout: 30000,
});
// Wait for specific selector if provided
if (options.waitFor) {
await page.waitForSelector(options.waitFor, {
timeout: 10000,
});
}
// Extra settle time for animations
await page.waitForTimeout(500);
const screenshot = await page.screenshot({
fullPage: options.fullPage ?? false,
type: options.format ?? 'png',
quality: options.format === 'png'
? undefined
: (options.quality ?? 80),
});
return screenshot;
} finally {
await pool.release(context);
}
}
The API Approach — Skip the Infrastructure
A headless browser API handles all the complexity above. You send an HTTP request with a URL and options, and get back your result. No Chromium binaries, no memory management, no zombie processes.
Screenshot via API
// One API call replaces the entire browser pool above
const response = await fetch('https://api.snapapi.pics/v1/screenshot', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://example.com',
format: 'png',
width: 1280,
height: 720,
full_page: true,
block_ads: true,
wait_for: '.main-content',
}),
});
const imageBuffer = await response.arrayBuffer();
// Save or process the screenshot
Web Scraping via API
// Scrape any page — handles JS rendering, anti-bot, proxies
const response = await fetch('https://api.snapapi.pics/v1/scrape', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://news.ycombinator.com',
stealth: true,
formats: ['html', 'markdown', 'text'],
}),
});
const data = await response.json();
console.log(data.markdown); // Clean markdown content
console.log(data.text); // Plain text extraction
Structured Data Extraction
// Extract structured data — no CSS selectors needed
const response = await fetch('https://api.snapapi.pics/v1/extract', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://example.com/products',
schema: {
products: [{
name: 'string',
price: 'number',
rating: 'number',
in_stock: 'boolean',
}],
},
}),
});
const { extracted } = await response.json();
// extracted.products = [{ name: "...", price: 29.99, ... }]
PDF Generation
// Generate PDF from any URL or raw HTML
const response = await fetch('https://api.snapapi.pics/v1/pdf', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://example.com/invoice/123',
format: 'A4',
print_background: true,
margin: { top: '1cm', bottom: '1cm' },
}),
});
const pdfBuffer = await response.arrayBuffer();
// Save or email the PDF
Common Use Cases for Headless Browser APIs
Automated Testing & Visual Regression
Capture screenshots of your application at different breakpoints and compare them against baseline images. SnapAPI supports 30+ device presets — iPhone, iPad, Pixel, Galaxy, and more — making it easy to test responsive layouts without managing a device lab:
const devices = [
'iphone-15-pro',
'ipad-pro-12.9',
'pixel-8',
'macbook-pro-16',
];
const screenshots = await Promise.all(
devices.map(device =>
fetch('https://api.snapapi.pics/v1/screenshot', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://myapp.com/dashboard',
device,
}),
})
)
);
// Compare against baseline screenshots with pixelmatch
Social Media & OG Image Generation
Generate dynamic Open Graph images for blog posts, product pages, or social cards by screenshotting a template page with dynamic content:
// Generate OG image from template
const ogImage = await fetch('https://api.snapapi.pics/v1/screenshot', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: `https://myapp.com/og-template?title=${encodeURIComponent(title)}`,
width: 1200,
height: 630,
format: 'png',
}),
});
Competitive Monitoring
Track competitor pricing pages, feature updates, and design changes automatically:
import cron from 'node-cron';
// Check competitor pricing daily
cron.schedule('0 9 * * *', async () => {
const competitors = [
'https://competitor-a.com/pricing',
'https://competitor-b.com/pricing',
];
for (const url of competitors) {
// Screenshot for visual diff
const screenshot = await fetch('https://api.snapapi.pics/v1/screenshot', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({ url, full_page: true }),
});
// Extract pricing data
const pricing = await fetch('https://api.snapapi.pics/v1/extract', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url,
schema: {
plans: [{
name: 'string',
price: 'string',
features: ['string'],
}],
},
}),
});
// Store and compare with previous data
await saveAndAlert(url, pricing, screenshot);
}
});
AI-Powered Web Analysis
Use AI to analyze web pages without building any parsing logic — the LLM reads the page and answers your questions:
const analysis = await fetch('https://api.snapapi.pics/v1/analyze', {
method: 'POST',
headers: {
'X-Api-Key': 'sk_live_your_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://competitor.com',
prompt: 'Analyze this landing page. What is the value proposition? What pricing model do they use? What are the main CTAs?',
}),
});
const { result } = await analysis.json();
console.log(result);
// Structured analysis of the page content
MCP Integration — AI Agents + Headless Browser
The Model Context Protocol (MCP) lets AI agents use headless browser capabilities directly. SnapAPI's MCP server gives Claude, Cursor, VS Code, and other AI tools access to web capture without any code:
// Claude Desktop config (~/.claude/claude_desktop_config.json)
{
"mcpServers": {
"snapapi": {
"command": "npx",
"args": ["-y", "snapapi-mcp"],
"env": {
"SNAPAPI_API_KEY": "sk_live_your_key_here"
}
}
}
}
Once configured, AI agents can:
- Take screenshots of any URL during conversations
- Scrape and extract data from web pages in real time
- Generate PDFs from URLs or HTML content
- Record website videos for documentation
- Analyze web pages with AI-powered prompts
npx snapapi-mcp and it's available across all your AI tools.
SDK Examples — Every Language
SnapAPI provides official SDKs for 8 languages. Here's how screenshot capture looks across different stacks:
Python
import requests
response = requests.post(
'https://api.snapapi.pics/v1/screenshot',
headers={'X-Api-Key': 'sk_live_your_key_here'},
json={
'url': 'https://example.com',
'format': 'png',
'full_page': True,
'block_ads': True,
}
)
with open('screenshot.png', 'wb') as f:
f.write(response.content)
Go
package main
import (
"bytes"
"encoding/json"
"net/http"
"os"
)
func main() {
payload, _ := json.Marshal(map[string]interface{}{
"url": "https://example.com",
"format": "png",
"full_page": true,
})
req, _ := http.NewRequest("POST",
"https://api.snapapi.pics/v1/screenshot",
bytes.NewBuffer(payload))
req.Header.Set("X-Api-Key", "sk_live_your_key_here")
req.Header.Set("Content-Type", "application/json")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
f, _ := os.Create("screenshot.png")
defer f.Close()
f.ReadFrom(resp.Body)
}
PHP
$ch = curl_init('https://api.snapapi.pics/v1/screenshot');
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
'X-Api-Key: sk_live_your_key_here',
'Content-Type: application/json',
],
CURLOPT_POSTFIELDS => json_encode([
'url' => 'https://example.com',
'format' => 'png',
'full_page' => true,
]),
]);
$screenshot = curl_exec($ch);
curl_close($ch);
file_put_contents('screenshot.png', $screenshot);
Self-Hosted vs API — Decision Matrix
| Factor | Self-Hosted (Playwright/Puppeteer) | Headless Browser API (SnapAPI) |
|---|---|---|
| Setup time | Days to weeks (Docker, fonts, proxies) | Minutes (get API key, make request) |
| Infrastructure cost | $50-500+/mo servers + maintenance time | $0-79/mo depending on volume |
| Scaling | Manual (add servers, load balancers) | Automatic (API handles concurrency) |
| Anti-bot handling | Stealth plugins (break with updates) | Built-in stealth mode |
| Browser updates | Manual (test compatibility each time) | Managed (always latest stable) |
| Device emulation | Manual config per device | 30+ presets built-in |
| PDF generation | Custom code + print CSS | Single endpoint, A4/Letter/custom |
| Video recording | Complex (ffmpeg pipelines) | Single endpoint, MP4 output |
| AI analysis | Build your own LLM pipeline | Built-in /analyze endpoint |
| Data extraction | CSS selectors (fragile) | Schema-based + AI extraction |
| MCP support | Build your own server | Ready-to-use MCP server on npm |
| Best for | Full control, custom logic, low volume | Speed to market, scale, AI integration |
Getting Started with SnapAPI
SnapAPI gives you a complete headless browser API with zero infrastructure to manage:
- Sign up at snapapi.pics — free tier includes 200 requests/month
- Get your API key from the dashboard
- Make your first request — screenshot, scrape, extract, PDF, video, or analyze
- Install an SDK — JavaScript, Python, Go, PHP, Swift, Kotlin, and more
- Set up MCP — give your AI tools web capture capabilities with
npx snapapi-mcp
Stop Managing Browsers. Start Shipping.
200 free requests/month. SDKs in 8 languages. MCP server for AI agents. No credit card required.
Get Your Free API Key