ADDTODOC-PREPEND-BODY
Originally the wildcard wasn't supported by robots.txt but that has since changed. Both Google and Yahoo now recognize it: Regularly check what kind of pages Google is indexing on your site and look for patterns.
 
If there are a lot of PDF pages, or dozens of useless links from a particular component, you can act quickly to block them out with robots.txt. Use the site:mydomain.com search function or a tool such as  WebCEO.com. 
 

Among the most important things you can do is check your pages that are in Google's supplemental index. This is where you'll find lots of your low-quality pages, ripe for removal by robots.txt. If the pages don't contain useful information, dump them.

ADDTODOC-APPEND-BODY