5 Commits

Include renames

Author	SHA1	Message	Date
Matt Baer	da1d3d898c	Add spiders, scrapers, indexers; update bots	2015-08-17 15:21:43 -04:00
Matt Baer	b9838662f8	Merge branch 'master' of https://github.com/writeas/web-core	2015-07-07 19:57:56 -04:00
Matt Baer	d01e265587	Add bots documentation	2015-07-07 02:35:17 -04:00
Matt Baer	91771fc838	Add "crawler" and "voltron" to bot UA search	2015-07-01 17:07:59 -04:00
Matt Baer	d2454e8900	Add bots component ... Include generation script for extracting bots from server logs	2015-06-28 18:22:40 -04:00