Visual Studio Team System 2008 : Working with Test Results (part 2) - Build report and test result

7/25/2012 6:17:12 PM

Build report and test result

The VSTS takes the source code from the solution of the TFS and builds the projects, and reports it immediately. The report is also saved in TFS for future reference. Each and every step is reported in the build report. It consists of getting the source for the project, compiling the projects, compiling the test project, and running the test project, if it is set to run after the build. The report also includes the overall build status. The test run status is also reported and the test results are stored in the similar way, when the tests are run directly from Visual Studio. The following screenshot depicts the sample of the build report:

The above build succeeded without any errors in building the projects in the solution ClassLibrary1 and running the test. The test result is also shown under the section Result details for x86/Release. This is the result of tests in the test list included in the build for the solution. This is the same result shown in the test results window, when the tests are run using Test View or Test List Editor. This section also has the URL link for the test result folder where the test result .trx file and the other binaries are stored. The URL opens the test results folder as shown here:

Select a particular folder of the current test by the build under the TestResults folder and then clicking on OK, the results are imported to the Test Results window and the window shows up with the tests and the status.

Code coverage

The test run step in the build process can also include the code coverage for the current test. Code coverage is the process of finding out the percentage of code that is covered by the unit test. This helps us know better how much of the code remains to be tested and what percentage of code has been tested. The configuration file LocalTestRun.testrunconfig holds the settings required for enabling the code coverage for unit tests. Open the Solution Explorer and the configuration file, which in turn opens up the configuration utility.

Select the Code Coverage option in the list as shown in the left pane. It shows the assemblies from the solution. Apply the settings and check in the configuration file to TFS. When we start the build again, the test is run and the code coverage details are also shown in the build report.

The code coverage results are shown after the test results under the section, Result details for x86/Release. The following report shows the code coverage section with the total percentage of blocks covered and the percentage of code covered in each assembly. The percentage of blocks not covered by the code is also shown in the report.

Build report with failed test

Whatever we have seen so far is the positive side of testing. Now let us do some code changes to make the test fail. For example, one of the methods in the ClassLibrary1TestProject is modified with a different expected value from the test. Check the code and start the build process.

The TFS builds the solution and presents the build report with the status of each step during the build process. The following screenshot shows the build report:

As we had expected, the test failed in the build process while running the test. So the status of the build is partially completed. This is because the compilation of all the projects was successful. Also running the test was successful, but the test failed because the result did not match up with the expected result. Let us now look at the test result. Click on the test result URL link and import the trace file from the location.

The result window shows the methods that passed the test and the ones which failed. The error message is also shown as the expected result is different from the actual.

Right-click on the failed test in the Test Results window, and select the View Test Result Details option, which shows more details about the failed test. The details window shows the summary of the test such as the time the test started and ended, the duration, and the test name. The second section shows the error message from the test.

Stack trace

The Test Results window has the detailed section, which shows the Error Stack Trace of the failed test. This one clearly shows the line at which it failed in the class file. The section also has the URL link which will directly take us to the point where the test has failed. As shown in the above image, the URL opens the Class1Test.cs file and the cursor is placed on line 77.

Creating Work Item from the result

Work item is an item in the TFS, which refers to a unit of work. It could be just an item, which is a task or a defect or a work item of type Bug, or it could be an issue or a requirement item. The following screenshot shows the option to add a work item using the menu option in VSTS menu. The option lists the different types of work item.

The work item of type Bug is a defect raised against a failure or error occurred from the application. Now in our example above, the test result throws an error from the code saying the expected result does not match with the actual result. This code has to be fixed by the developer who wrote the code. Now we can add the corresponding bug work item to TFS using the menu option, or we can directly add that from the Test Results window.

The error is added as a Bug to the TFS work item. It also fills the details required for the bug. We can overwrite the details as per our requirement if we want. The Bug has different sections to it. We can overwrite or fill additional details to the Title and Description field for the Bug.

The second tab contains the history of the defect changes. Whenever we modify the properties of the Bug, it is tracked here in the History tab. The third one is the Links tab, which contains the link to the source from which this defect occurred. It is automatically filled by TFS when we create this defect. There are some options to the right of this Links tab. The Open option opens the Test Result and the Test Result details window.

The Edit... option in the Links tab opens the window that contains the link to the test result for the test that is the source for this defect. We can edit the comment if required. If we still do not have the test result in the TestResults folder, we can delete this link. If there are additional links to be added, or if the link is moved to a different place, we can add it.

The next tab in the bug work item is that of File Attachments, which has the trace file for the test attached to it. Select the trace file and click on Open which opens the test result window with the results loaded in the window.

This tab also has different options to add another attachment, save the attachment to a different location or to delete it.

Publish test results

Think about the different testers in different locations running the tests. Everyone will have the test results in his/her local machine. How do we consolidate and let other project teams or the other team members know about the test results? TFS maintains the operational store, which is a central database where all the details are stored for future use.

VSTS provides an option to publish the test results to the TFS operational store so that the other team at a different location can easily look at the test results that are published to a central location.

The Test Results window has the option to publish the results. Select the check box option against the test results in the window and click on the Publish button on the test results toolbar.

The publish option opens a new window, which lists all the test results in the current TestResults folder. Select the test to be published and select the Associated build number and the Build flavor. The code coverage data can be published along with the test result by selecting the option Include code coverage data for selected runs.

  •  Finance - Apple Versus Google
  •  Oracle Coherence 3.5 : Testing and debugging Coherence applications
  •  Oracle Coherence 3.5 : Accessing the data grid (part 6) - Using the Coherence API - Implementing CoherenceTarget, Testing the Cache loader
  •  Oracle Coherence 3.5 : Accessing the data grid (part 5) - Using the Coherence API - Loader design, Implementing CsvSource
  •  Oracle Coherence 3.5 : Accessing the data grid (part 4) - Using the Coherence API - The basics: NamedCache and CacheFactory
  •  Oracle Coherence 3.5 : Accessing the data grid (part 3) - Configuring Coherence
  •  Oracle Coherence 3.5 : Accessing the data grid (part 2) - Configuring the development environment
  •  Oracle Coherence 3.5 : Accessing the data grid (part 1) - Coherence console
  •  Oracle Coherence 3.5 : Installing Coherence, Starting up the Coherence cluster
  •  The Go-To Reference Design Map For The Cloud?
  •  Separating BPM and SOA Processes : SOA-Oriented Disputes with BEA
  •  Science! – Spintronics
  •  Linux - The Operating System With A Pure Heart (Part 2)
  •  Linux - The Operating System With A Pure Heart (Part 1)
  •  What's Your Strategy For Today’s Server Room Challenges?
  •  Truly Transparent Trinkets
  •  Gamma – Ray Lens …Made Possible
  •  Tech Chance - Revamps …Worsts Ever!
  •  Organic Computing (Part 2)
  •  Organic Computing (Part 1)
    Top 10
    SharePoint 2010 : Searching Through the API - The Search API
    SharePoint 2010 : Searching Through the API - Understanding the Query Expression Parsing Order
    Windows 7 : Designing a Client Hardware Platform (part 2) - Boot from VHD
    Windows 7 : Designing a Client Hardware Platform (part 1)
    Windows 7 : Designing and Managing a Licensing Strategy (part 1) - Volume Licensing Activation Methods
    GoPro Hero3 Black Edition Review - Bring The Quality Of Action Cameras To A Next Level (Part 2)
    GoPro Hero3 Black Edition Review - Bring The Quality Of Action Cameras To A Next Level (Part 1)
    How To Create A Lively Video Clip
    HDD, SSD and Hybrid Hard Drive Competition
    How To Transfer Data From SSD To HDD
    Most View
    Windows Server 2008 : Synchronizing Directory Information with Forefront Identity Manager (FIM)
    Algorithms for Compiler Design: PREDICTIVE PARSING ERROR RECOVERY
    AMD A6-3500 - Llano integrated-graphics processors
    Windows 7 : Developing Migration Files, Using USMT in Microsoft Deployment Toolkit
    How to Buy…A TV TUNER (Part 2)
    iPhone 5: A New Angle (Part 1)
    Nikon J2 With Metal Case And Cheaper Price Than J1
    SEH Computerteknik myUTN-150 - USB Deviceserver
    The Connected BlackBerry : Connection Paths (part 1) - Wireless Application Protocol Connection, Direct TCP Internet Connection, BlackBerry Internet Service Connection
    Windows Vista Security Guide
    Company Profiles: Twitter
    The Future Of Apple: Chip Off The Block (Part 8)
    20 Ice Cream Sandwich Secret Revealed! (Part 2)
    Communicate Between Two Machines on the Same Network (WCF)
    Designing a Windows Server 2008 R2 Active Directory : Understanding the Federated Forests Design Model
    Windows Phone 8 Operating System Review – Part1
    iPhone 3D Programming : Drawing an FPS Counter (part 2) - Rendering the FPS Text
    Nokia 808 Pureview - Best For Convergists
    Hands-On with the iPad Mini (Part 1)
    100 Windows Speed-Up Tips (Part 5) - Save space by dumping crash data