Web Scraping & Automation

Tools that automatically collect information from websites or automate repetitive browser tasks.

Web Scraping & Automation

beautifulsoup4

Untangles messy webpage code into something searchable, like alphabetizing a junk drawer.

Web Scraping & Automation

httpx

Requests' faster, more modern sibling, built for a Python that does several things at once.

Web Scraping & Automation

lxml

The quiet speed demon underneath a lot of Python's fastest scraping and data tools.

Web Scraping & Automation

MechanicalSoup

Fills out web forms and clicks links automatically, like a robot with reading glasses on.

Web Scraping & Automation

playwright

The browser automation tool built by engineers who got tired of their last one's limits.

Web Scraping & Automation

pyppeteer

An unofficial Python stunt double for Google's browser-automation tool, Puppeteer.

Web Scraping & Automation

requests

The library that made talking to the internet from Python feel like plain English.

Web Scraping & Automation

Scrapy

Built for scraping thousands of pages a minute, not just the one you're staring at.

Web Scraping & Automation

selenium

Automates a real web browser to click, type, and scroll exactly like a person would.

Web Scraping & Automation

urllib3

The unglamorous engine room that Requests, and much of the Python web, quietly runs on.