Skip to content
Submit

Scrapling: adaptive scraping for agents

A scraper that survives page redesigns and ships an MCP/skill so agents pull trend data themselves.

Try it in 60 seconds

pip install scrapling
pip install "scrapling[fetchers]"
pip install "scrapling[ai]"

Install command copied from the project README. Check the README for the latest version.

What it does

A scraper that survives page redesigns and ships an MCP/skill so agents pull trend data themselves.

Selection methods · Fetchers · Spiders · Proxy Rotation · CLI · MCP

Scrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl.

Quoted from the README of D4Vinci/Scrapling.

Inside the repo

Key FeaturesSpiders - A Full Crawling FrameworkAdvanced Websites Fetching with Session SupportAdaptive ScrapingAI FeaturesHigh-Performance & battle-tested ArchitectureDeveloper/Web Scraper Friendly ExperienceGetting StartedBasic UsageSpidersAdvanced Parsing & NavigationAsync Session Management Examples

Step by step

  1. Open the GitHub repo and skim the README (install commands change often).
  2. Install it: copy the skill folder into ~/.claude/skills/ (Claude Code) or ~/.codex/skills/ (Codex), or use the plugin/marketplace command from the README.
  3. Restart Claude Code or Codex.
  4. Ask for the job in plain words; the skill loads when your request matches.

Docs, demos & materials

Sources & credit

All credit to the original authors. We rewrite in our own words and link back. Something wrong? Report it.

Related

SKILL

last30days: what people said this month, cited

Researches any topic across Reddit, X, YouTube and HN for the last 30 days and returns a cited brief.

Claude Code +1★ 64.3k
AGENT USE CASE

Let your agent scrape social, search and maps data

gives an agent thousands of ready scrapers (Instagram, TikTok, Google Maps, search) to pull research and lead data.

Codex +1★ 11.3k
SKILL

Social media research skills

Find outlier posts, mine comments, tear down competitors and read ad libraries from your agent.

Claude Code +1★ 3.7k
SKILL

Apify skills: scrape IG, TikTok, YouTube to CSV

Official Apify skills: the agent picks a scraper, runs it and returns CSVs from 130+ sources.

Claude Code +3★ 2.4k
SKILL

claude-ads: 190-check audit across 12 ad platforms

Free Claude Code skill: parallel subagents audit Google, Meta, TikTok, LinkedIn and more, then rank fixes.

Claude Code★ 10k
AGENT USE CASE1:21

Let sound design lead the scene

Building many scene variants with sound first, the creator says the result looks less like AI slop and renders fast.

Claude Code♥ 988

Coming soon

AgentsUses Weekly

A free weekly email with the best new skills and agent workflows is coming soon. Want a heads-up when the first issue goes out? Email hello@agentsuses.com and we will let you know.

Get notified by email