 El RIDO
|
878f13a643
adding an example to opt-out of the privatebin directory
|
há 6 anos atrás |
|
El RIDO
|
50881a75e6
ensure bots are prohibited completely from indexing
|
há 7 anos atrás |
 rugk
|
7875d7b0a3
Some rewording
|
há 10 anos atrás |
|
rugk
|
a5730b701c
Revert order change
|
há 10 anos atrás |
|
rugk
|
0567a720fd
Change order of allow rule in robots.txt
|
há 10 anos atrás |
 ic0nic
|
c918a9fe2e
Incorrect structure
|
há 12 anos atrás |
|
ic0nic
|
50d4fb9bf3
Allow discovery
|
há 13 anos atrás |
 Greg Knaddison
|
f37303d858
For server-load and privacy reasons, disallow robots.
|
há 14 anos atrás |
