 Matt Baer
|
c0e4c95cfb
|
Prevent findBots.sh running when no arg given
|
2018-01-25 17:30:41 -05:00 |
|
|
Matt Baer
|
ca03ff920a
|
Explain findBots.sh a little more
|
2018-01-25 17:29:58 -05:00 |
|
 Matt Baer
|
da1d3d898c
|
Add spiders, scrapers, indexers; update bots
|
2015-08-17 15:21:43 -04:00 |
|
|
Matt Baer
|
b9838662f8
|
Merge branch 'master' of https://github.com/writeas/web-core
|
2015-07-07 19:57:56 -04:00 |
|
|
Matt Baer
|
d01e265587
|
Add bots documentation
|
2015-07-07 02:35:17 -04:00 |
|
|
Matt Baer
|
91771fc838
|
Add "crawler" and "voltron" to bot UA search
|
2015-07-01 17:07:59 -04:00 |
|
|
Matt Baer
|
d2454e8900
|
Add bots component
Include generation script for extracting bots from server logs
|
2015-06-28 18:22:40 -04:00 |
|