Craigslist Crawler

Craigslist Crawler originated as a simple application to check the local craigslist and send a notification if a particular deal was flagged. It evolved into a highly optimized, massively concurrent windows service that crawls every city on the site for the purpose of performing market research and analysis of human language. Currently pulls 100,000+ listings per night given my current hardware and network setup.

View Source on Github