Web Security Testing : Automating Specific Tasks with cURL - Impersonating a Specific Kind of Web Browser or Device

1/31/2015 2:45:07 AM


Some web applications react to the User-Agent string that is passed from the web browser. The software actually selects different pages to display or different code to execute depending on what kind of browser it thinks it is talking to. cURL allows us to specify what our User-Agent string will be, thus allowing us to pretend to be any browser at all. This may allow you to simulate requests from mobile phones, Flash players, Java applets, or other non-browser software that makes HTTP requests.


# Internet Explorer on Windows Vista Ultimatecurl -o MSIE.html -A 'Mozilla/4.0 (compatible; MSIE 7.0;    Windows NT 6.0; SLCC1; .NET CLR 2.0.50727;    Media Center PC 5.0; .NET CLR 3.0.04506)' Firefox on MacOS Xcurl -o FFMac.html -A 'Mozilla/5.0 (Macintosh; U;    Intel Mac OS X; en-US; rv:    Gecko/20070309 Firefox/' "Blazer" web browser on PalmOS devicescurl -o Palm.html -A 'Mozilla/4.0 (compatible; MSIE 6.0; Windows 98;    PalmSource/hspr-H102; Blazer/4.0) 16;320x320'


There is no rhyme or reason to User-Agent strings, except the vestigial “Mozilla” at the beginning of the string—a reminder of the browser wars. There are many databases and websites that collect these strings, but as a tester, you want to gather them differently. You want to find out from the developers or from the source code itself which user agents the code responds to (if any). That way you can determine how many different kinds of tests you need to do. You may want to talk to operations staff to get some of your web server logs and look at what User-Agents you’re seeing in the wild.

If you want to browse around interactively, impersonating another device. By poking around interactively, you may discover that your application does react to the User-Agent, and, therefore, you need to make some test cases based on this recipe.

Providing customized content

Yahoo! is a major website that reacts to the User-Agent string. If you choose something it doesn’t recognize, it will send a very small web page (and one that has very little JavaScript and fewer advertisements). If your User-Agent is recognizable as Internet Explorer, Firefox, or another well-known browser, Yahoo! will deliver customized content—including JavaScript that is carefully tuned to execute correctly in your web browser. One of the reasons Yahoo! does this is to provide a good-looking interface to new devices that they have never heard of before. The first person to visit with a Nintendo Wii or an Apple iPhone got a generic page that probably rendered pretty well, but did not have all the features of Yahoo! when viewed in a browser. Eventually, as Yahoo! becomes aware of the capabilities of the Wii or the iPhone, they will change their site to react differently, based on the User-Agent.

Reacting to User-Agent is rare

Most web applications don’t react to browsers at all. You only need to consider this testing technique if you know for a fact that your application behaves this way. Note that many sites and applications that use complex cascading style sheets (CSS) or asynchronous JavaScript and XML (AJAX) will have a lot of complex JavaScript code that loads differently in the browser depending on which browser it is. This is not the same as the User-Agent string and having the server perform different operations based on what browser requests the page. Many sites send JavaScript that will be executed differently depending on the browser. Few look at the User-Agent string at run time.

Realize that, if you’re one of the lucky few who has software that responds differently to different User-Agents, this will increase your test matrix significantly. Tests for vulnerabilities like cross-site scripting (XSS), SQL injection, or session fixation will have to be done with representatives of various different kinds of browsers to be sure that all the code is tested.

  •  Web Security Testing : Automating Specific Tasks with cURL - Checking for Directory Traversal with cURL
  •  Web Security Testing : Automating Specific Tasks with cURL - Following Redirects Automatically, Checking for Cross-Site Scripting with cURL
  •  Web Security Testing : Automating Specific Tasks with cURL - Fetching Many Variations on a URL
  •  Web Security Testing : Automating Specific Tasks with cURL - Fetching a Page with cURL
  •  Sharepoint 2013 : SharePoint Publishing Infrastructure (part 6) - Check In/Out, Versioning, and Content Approval
  •  Sharepoint 2013 : SharePoint Publishing Infrastructure (part 5) - Content Management - Putting It All Together
  •  Sharepoint 2013 : SharePoint Publishing Infrastructure (part 4) - Content Management - Master Pages,Page Layouts
  •  Sharepoint 2013 : SharePoint Publishing Infrastructure (part 3) - Content Management - Site Columns, Content Types
  •  Sharepoint 2013 : SharePoint Publishing Infrastructure (part 2) - Publishing Features
  •  Sharepoint 2013 : SharePoint Publishing Infrastructure (part 1) - A Publishing Site
    Most View
    Using Services for UNIX to Integrate UNIX Systems with an Active Directory/Exchange Server 2007 Environment
    Constellation Virgo/Centaur: High-End Audio Superstars (Part 1)
    Windows Phone 8 Group Test – June 2013 (Part 3) : Nokia Lumia 820, Nokia Lumia 920
    Asus F2A85-V PRO Mainboard - A Socket FM2 Mainboard With Good Performance (Part 1)
    All About Battery Life (Part 3)
    SQL Server 2012 : Exploring SQL CLR - CLR Triggers
    Microsoft Dynamic AX 2009 : Configuration and Security - Security Framework (part 1)
    Ferrari 458 Speciale Versus McLaren 650S Versus Porsche 911 GT3 – Holy Trinity (Part 4)
    Ditch Your Laptop For Your Phone (Part 1)
    SFF Media PCs: What Hardware To Consider, And What To Ignore (Part 2)
    Popular Tags
    Microsoft Access Microsoft Excel Microsoft OneNote Microsoft PowerPoint Microsoft Project Microsoft Visio Microsoft Word Active Directory Biztalk Exchange Server Microsoft LynC Server Microsoft Dynamic Sharepoint Sql Server Windows Server 2008 Windows Server 2012 Windows 7 Windows 8 Adobe Indesign Adobe Flash Professional Dreamweaver Adobe Illustrator Adobe After Effects Adobe Photoshop Adobe Fireworks Adobe Flash Catalyst Corel Painter X CorelDRAW X5 CorelDraw 10 QuarkXPress 8 windows Phone 7 windows Phone 8 BlackBerry Android Ipad Iphone iOS
    Top 10
    Review : Acer Aspire R13
    Review : Microsoft Lumia 535
    Review : Olympus OM-D E-M5 Mark II
    TomTom Runner + MultiSport Cardio
    Timex Ironman Run Trainer 2.0
    Suunto Ambit3 Peak Sapphire HR
    Polar M400
    Garmin Forerunner 920XT
    Sharepoint 2013 : Content Model and Managed Metadata - Publishing, Un-publishing, and Republishing
    Sharepoint 2013 : Content Model and Managed Metadata - Content Type Hubs