Scrapy Cloud runs Scrapy spiders you write in code. Crawley Cloud gives you the same jobs, scheduling, collections and API — but you build scrapers point-and-click, no Python required.
Scrapy Cloud (by Zyte) is the hosted home for spiders written with the Scrapy framework. If you already have a team of Python developers maintaining spiders, it is an excellent place to run them. Crawley Cloud is for the other case: you need the same jobs, schedules and item API, but nobody wants to write and maintain spider code.
Both platforms give you scheduled jobs, run logs, an items API and key/value collections. The split is in how a scraper gets built and what it costs to keep it working when the target site changes.
| Crawley Cloud | Scrapy Cloud | |
|---|---|---|
| Build without code | Yes — visual picker | No — write Scrapy spiders |
| Jobs, logs, items API | Yes | Yes |
| Collections (key/value store) | Yes | Yes |
| Scheduling & retention | Yes | Yes |
| Proxies & JS rendering | Yes, built in | Add-ons |
| Free tier | Yes, no credit card | Free units |
Scrapy is a full programming framework: custom middleware, arbitrary Python in every callback, complex crawl graphs, and a large ecosystem of extensions. If your extraction logic is genuinely unusual, or you want to version spiders in git with the rest of your codebase, code is the right tool and Scrapy Cloud runs it well.
Most scraping jobs are not unusual: a listing page, a detail page, a handful of fields, a schedule. Crawley builds those with a visual picker and a rules editor, so the person who needs the data can build and fix the scraper without a developer in the loop — and when a site changes its markup, fixing a rule takes minutes, not a deploy.
Proxies and JavaScript rendering are built in rather than add-ons, deliveries push results to S3, webhooks, Slack or email, and data-quality alerts (field triggers, drop-percentage checks, de-duplication across runs) come with every plan. Developers still get a full REST API, an OpenAPI spec and a CLI.
Scrapy Cloud bills by capacity units per month, and proxy rotation and browser rendering are separate Zyte products on top. The bill scales with how much you run and which add-ons you need; see their pricing page for current rates.
Crawley Cloud: Free (no credit card) · Starter $9/mo · Pro $29/mo · Scale $79/mo · Business $199/mo — flat monthly plans sized by concurrency units and retention, every feature in every plan.
Take one spider, open its target site in the Crawley picker, recreate the fields and pagination rules, and run it. Compare item counts with the last Scrapy job. Keep the items API consumers pointed at Scrapy Cloud until the Crawley run matches, then switch the schedule. Spiders with heavy custom logic can stay on Scrapy — the two coexist fine.
No — Crawley scrapers are built visually, not from Scrapy code. The practical path is to rebuild the common spiders and keep the exotic ones where they are.
Yes. Projects have key/value collections, and items are queryable through the REST API with the same job/run semantics.
Both are included in every plan; there is nothing to configure or buy separately.
Keep Scrapy Cloud for spiders that genuinely need code. Move the everyday scrapers to Crawley Cloud so the people who need the data can own them.
Get the platform power of Scrapy Cloud without writing spider code.
Start free →