# House.ofdoom.com robots.txt User-agent: * #Allowed for now - the new version of roxen should handle this #better #Internal graphics used by pages - stuff moves around at random #Disallow: /_internal/ #Mailcloak pages Disallow: /_internal/mailcloak!0/ #-------- #I have a couple of hundred thousand little Tiny JPEG files here, as a #archive from when I was playing with a web cam. There is some copyright #verification agent, which keeps downloading them all every couple of #days (I assume to see if I copied them from somewhere else). I got a #good laugh out of it for a while, but they keep reloading them faster #and faster, until at any given moment, they had 40-80 connections #open, and they would re-download the entire multi-gig set every day. # #They appear to ignore robots.txt, so I also blocked them at the firewall. Disallow: /~hungerf3/archive Disallow: /~hungerf3/old_archive