Ever wondered why a google search returned search results that lead to sites that require a registration ? How the hell did googlebot index the site without a registration ? Many sites want their site indexed in google to recieve more hits and thus, make more money with advertisment on their site. Even more interesting is the following question: How do we use this to our advantage ?Its fairly easy, we have to disguise ourselfs as the google bot and voila, many sites let us in without registration. The parameter we have to change is called the User Agent. The user agent defines the browser and version that you use. For example it would show Internet Explorer as your browser if you are using this one.
You need to alter your settings to the following:
User Agent: Googlebot/2.1
Compatible: http://www.googlebot.com/bot.html
You can do so in Opera with ease. Firefox offers an extension which is downloadable from the official website.
For Internet Explorer you need to change registry entries.
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionInternet Settings5.0User Agent] @="Googlebot/2.1" "Compatible"="+http://www.googlebot.com/bot.html"
Save this as ua.reg and execute.
To revert the changes back, you need the following:
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionInternet Settings5.0User Agent] @="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
Save as oua.reg and execute.
