I’ve spent a long time trying to get together a twitter scraper. I haven’t had more than twenty minutes a day, at most, free time to spend doing it. I finally spent some time ripping apart a quick script written in python by someone else, using tweepy. It’s really damned handy. I’ve set it to grab the max allowable tweets from any given user and take data I find important, then format it for storage in a comma-separated value flat file. I spent maybe four hours dicking around with the thing to get it working as seamlessly as possible. Once the “eureka” moment hit, I only spent about twenty minutes to turn data collection on an irritating, but practical, exercise in identity concealment.