Price Monitoring API

Price Monitoring with Screenshots and Scraping

Price monitoring requires two types of data: the visual proof of what a competitor's pricing page showed at a specific time, and the structured pricing data extracted from those pages for trend analysis and alerting. SnapAPI provides both with two endpoints: the screenshot endpoint for visual capture and the extract endpoint for structured pricing data extraction. A complete price monitoring pipeline calls both endpoints for each target URL on a schedule, stores the screenshot as visual evidence, and writes the extracted price data to a database for trend tracking and alert evaluation.

# Python: price monitoring with screenshot + extraction
import requests, boto3, psycopg2
from datetime import datetime

API_KEY = "YOUR_API_KEY"
S3 = boto3.client("s3")
DB = psycopg2.connect("postgresql://user:pass@localhost/prices")

headers = {"Authorization": f"Bearer {API_KEY}"}
COMPETITORS = [
    {"name": "CompetitorA", "url": "https://competitor-a.com/pricing"},
    {"name": "CompetitorB", "url": "https://competitor-b.com/pricing"},
]

for comp in COMPETITORS:
    url = comp["url"]
    encoded = requests.utils.quote(url, safe='')

    # Capture screenshot as visual evidence
    snap = requests.get(
        f"https://api.snapapi.pics/screenshot?url={encoded}&format=jpeg&full_page=true",
        headers=headers, timeout=60
    )
    ts = datetime.utcnow()
    s3_key = f"pricing/{comp['name']}/{ts:%Y%m%dT%H%M%S}.jpg"
    S3.put_object(Bucket="my-bucket", Key=s3_key, Body=snap.content)

    # Extract structured pricing data
    extract_resp = requests.post(
        "https://api.snapapi.pics/extract",
        json={
            "url": url,
            "schema": {
                "starter_price": "number",
                "growth_price": "number",
                "enterprise_price": "string",
                "starter_requests": "number",
                "currency": "string",
            }
        },
        headers={**headers, "Content-Type": "application/json"},
        timeout=60,
    )
    prices = extract_resp.json()

    # Store in database
    with DB.cursor() as cur:
        cur.execute(
            "INSERT INTO price_snapshots (competitor, captured_at, s3_key, data) VALUES (%s,%s,%s,%s)",
            (comp["name"], ts, s3_key, psycopg2.extras.Json(prices))
        )
    DB.commit()
    print(f"{comp['name']}: starter={prices.get('starter_price')}, screenshot={len(snap.content):,}b")

Detecting Price Changes and Triggering Alerts

Price change detection compares the latest extracted price data against the previous snapshot and triggers alerts when differences are found. Define a threshold for meaningful changes to avoid alerts on minor variations: a 1 percent price difference is likely a rounding artifact rather than a genuine price change, while a 10 percent or greater difference is a signal worth alerting on. Route price change alerts to Slack with a screenshot attachment showing what the page looked like at the time of the change, giving the team immediate visual context without needing to visit the competitor's site.

# Python: price change detection and Slack alert
import json, requests

def check_price_change(competitor, prev, current, screenshot_url):
    changes = []
    for field in ["starter_price", "growth_price"]:
        p = prev.get(field)
        c = current.get(field)
        if p and c and abs(c - p) / p > 0.05:  # 5% threshold
            direction = "decreased" if c < p else "increased"
            changes.append(f"{field}: {p} -> {c} ({direction})")

    if changes:
        msg = {
            "text": f"*Price change detected: {competitor}*
" + "
".join(changes),
            "attachments": [{"image_url": screenshot_url, "title": "Current pricing page"}]
        }
        requests.post(SLACK_WEBHOOK_URL, json=msg)
        print(f"ALERT: {competitor} - {changes}")
    else:
        print(f"No change: {competitor}")

Building a Price History Dashboard

Price history data becomes valuable when it is visualized as a time series showing how competitor pricing evolved over weeks and months. Store each price snapshot in a PostgreSQL database with a timestamp column and query it to build trend charts in your monitoring dashboard. A simple React dashboard using Recharts can display price trends for all monitored competitors on a single chart, making it easy to spot pricing patterns: when do competitors run promotions, how do price changes correlate with product announcements, and where is there pricing pressure on your own plans. The visual screenshot archive stored in S3 complements the numeric data by providing exact visual evidence for each data point, allowing stakeholders to see what the competitor's pricing page actually showed at any point in the history.

Legal Considerations for Price Monitoring

Monitoring publicly visible pricing pages for competitive intelligence purposes is generally considered lawful in most jurisdictions, as the pricing information is intentionally made public by competitors for potential customers to see. However, several specific practices can create legal risk. Scraping at very high frequency may be characterized as a denial-of-service attack if it generates server load. Bypassing technical access controls such as bot-blocking systems or login walls to access pricing that is not publicly visible may violate computer fraud laws. Collecting personal data incidentally encountered during scraping may create GDPR or CCPA obligations. Using scraped data in ways that harm the target business's core interests (such as republishing their content) carries legal risk beyond pure competitive intelligence. For most legitimate business use cases of monitoring publicly visible competitor pricing pages at reasonable frequencies (daily or weekly), the legal risk is low. Consult a lawyer if you are operating at significant scale or in regulated industries where competitive intelligence practices have additional scrutiny.

Price Monitoring at Scale

Large-scale price monitoring covering hundreds of competitor URLs and thousands of product pages requires a pipeline architecture that handles volume efficiently. Maintain a URL inventory database with capture frequency settings per URL, change detection history, and monitoring status. Use a job queue to distribute screenshot and extraction tasks across worker processes, with priority queuing for URLs that show high change rates or are marked as high priority. Implement adaptive frequency: capture frequently-changing product pages hourly, stable pricing pages daily, and archived comparison pages weekly. Deduplicate by comparing current extraction data against the previous snapshot before storing — if nothing changed, only store a metadata record noting that no change was detected, rather than storing a full duplicate screenshot. This adaptive approach reduces both API consumption and storage costs while maintaining comprehensive coverage of pages that matter. At 50,000 monthly requests on the Growth plan, you can monitor 1,600 URLs daily or 200 URLs every 3 hours — sufficient for thorough competitive intelligence in most markets.

Scaling Price Monitoring Across Thousands of SKUs

Production price monitoring at scale requires a queue-based architecture rather than a simple sequential loop. When you need to monitor thousands of product URLs, a single Python loop becomes a bottleneck: if each screenshot takes two seconds, monitoring ten thousand products takes more than five hours per cycle, making your price intelligence stale before the cycle even completes. The solution is a task queue with concurrent workers. Use Redis as the queue backend and spawn multiple worker processes that each pull URLs from the queue and call SnapAPI concurrently. With twenty workers each handling one URL every two seconds, the same ten thousand products complete in under nine minutes, giving you near-real-time price data across your entire catalog. Rate limit your SnapAPI calls to stay within your plan limits and add exponential backoff for any URLs that return errors so transient failures do not permanently stall individual products in the queue.

Parsing Price Data from Screenshot Extractions

SnapAPI's extract endpoint returns structured JSON fields from web pages, which is the preferred method for price extraction at scale because it gives you clean numerical data without parsing HTML. Configure the extract endpoint with a selector targeting the price element on each retailer's product page. Most e-commerce sites use consistent CSS class names for their price elements, such as price, product-price, or offer-price. For sites that render prices via JavaScript after the initial HTML load, add a delay parameter of one to two seconds and set the waitUntil option to networkidle to ensure the price is visible before extraction. When extraction returns a string like $24.99 or EUR 19,95, normalize it to a float using a cleaning function that strips currency symbols, thousand separators, and trailing whitespace, then converts decimal commas to decimal points for European price formats. Store the raw extracted string alongside the normalized float in your database so you can always trace a parsed value back to the original extracted text for debugging discrepancies.

Visual Price Verification with Screenshot Comparison

Numerical extraction works reliably for most product pages, but some retailers dynamically obfuscate their prices using CSS tricks, JavaScript rendering, or image-based price displays to prevent scraping. For these edge cases, SnapAPI's screenshot endpoint provides a fallback that captures a visual record of the page exactly as it appears to a browser. Crop the screenshot to the price region using the clip parameter with x, y, width, and height coordinates targeting the price element, then store the cropped image alongside the extracted price. When your extraction returns null or an implausible value like zero, review the corresponding screenshot to diagnose whether the page structure changed, a bot-detection overlay is blocking the content, or the product went out of stock. This visual fallback transforms price monitoring from a black box into a debuggable system where every anomalous reading has a corresponding screenshot you can examine. Many teams discover that their initial extraction failures were caused by cookie consent overlays that SnapAPI renders by default, which can be dismissed by setting the acceptCookies parameter.

Price Intelligence Dashboard Architecture

A price monitoring system without a visualization layer delivers raw data without actionable insight. Build a lightweight dashboard that connects to your price history database and renders trend charts showing each product's price over time. A simple React frontend with Chart.js or Recharts can display a thirty-day price history chart per product, a competitor comparison bar chart showing current prices across all tracked retailers, and a price alert feed showing which products changed price in the last monitoring cycle. On the backend, write SQL queries that aggregate minimum, maximum, and average price per product per day, compute the percentage change from the previous reading, and flag products whose current price deviates more than ten percent from the seven-day moving average as potential pricing anomalies worth investigating. Store screenshots in an S3 bucket or compatible object storage and include pre-signed URLs in the dashboard API response so the frontend can display the visual verification screenshot for any price reading the analyst wants to inspect.

Handling Anti-Scraping Measures in Price Monitoring

Retailers who are aware their prices are being monitored may implement anti-scraping measures ranging from simple user-agent checks to sophisticated behavioral analysis. SnapAPI's Chromium-based renderer presents a realistic browser fingerprint that passes most bot-detection systems, but high-frequency monitoring of the same URLs from the same IP can still trigger rate limiting or temporary blocks. Rotate your monitoring schedule to spread requests evenly across the day rather than running a single nightly batch, which would produce a suspicious spike in traffic from a single IP. Use SnapAPI's proxy parameter to route requests through residential proxies for retailers that implement IP-based blocking, paying particular attention to geographic targeting if the retailer shows different prices to different regions. Cache successful responses in Redis with a TTL matching your monitoring interval so that if a retailer temporarily blocks a request, your system returns the last known price rather than registering a false null reading. Log block events with the full URL and timestamp to identify which retailers are blocking your monitoring and calibrate your approach accordingly.