Build report and test result
The VSTS takes the source
code from the solution of the TFS and builds the projects, and reports
it immediately. The report is also saved in TFS for future reference.
Each and every step is reported in the build report. It consists of
getting the source for the project, compiling the projects, compiling
the test project, and running the test project, if it is set to run
after the build. The report also includes the overall build status. The
test run status is also reported and the test results are stored in the
similar way, when the tests are run directly from Visual Studio. The
following screenshot depicts the sample of the build report:
The above build succeeded without any errors in building the projects in the solution ClassLibrary1 and running the test. The test result is also shown under the section Result details for x86/Release.
This is the result of tests in the test list included in the build for
the solution. This is the same result shown in the test results window,
when the tests are run using Test View or Test List Editor. This section
also has the URL link for the test result folder where the test result .trx file and the other binaries are stored. The URL opens the test results folder as shown here:
Select a particular folder of the current test by the build under the TestResults folder and then clicking on OK, the results are imported to the Test Results window and the window shows up with the tests and the status.
Code coverage
The test
run step in the build process can also include the code coverage for
the current test. Code coverage is the process of finding out the
percentage of code that is covered by the unit test. This helps us know
better how much of the code remains to be tested and what percentage of
code has been tested. The configuration file LocalTestRun.testrunconfig
holds the settings required for enabling the code coverage for unit
tests. Open the Solution Explorer and the configuration file, which in
turn opens up the configuration utility.
Select the Code Coverage
option in the list as shown in the left pane. It shows the assemblies
from the solution. Apply the settings and check in the configuration
file to TFS. When we start the build again, the test is run and the code
coverage details are also shown in the build report.
The code coverage results are shown after the test results under the section, Result details for x86/Release.
The following report shows the code coverage section with the total
percentage of blocks covered and the percentage of code covered in each
assembly. The percentage of blocks not covered by the code is also shown
in the report.
Build report with failed test
Whatever we have seen so far is
the positive side of testing. Now let us do some code changes to make
the test fail. For example, one of the methods in the ClassLibrary1TestProject is modified with a different expected value from the test. Check the code and start the build process.
The TFS builds the
solution and presents the build report with the status of each step
during the build process. The following screenshot shows the build
report:
As we had
expected, the test failed in the build process while running the test.
So the status of the build is partially completed. This is because the
compilation of all the projects was successful. Also running the test
was successful, but the test failed because the result did not match up
with the expected result. Let us now look at the test result. Click on
the test result URL link and import the trace file from the location.
The result window shows the
methods that passed the test and the ones which failed. The error
message is also shown as the expected result is different from the
actual.
Right-click on the failed test in the Test Results window, and select the View Test Result Details
option, which shows more details about the failed test. The details
window shows the summary of the test such as the time the test started
and ended, the duration, and the test name. The second section shows the
error message from the test.
Stack trace
The Test Results window has the detailed section, which shows the Error Stack Trace
of the failed test. This one clearly shows the line at which it failed
in the class file. The section also has the URL link which will directly
take us to the point where the test has failed. As shown in the above
image, the URL opens the Class1Test.cs file and the cursor is placed on line 77.
Creating Work Item from the result
Work item is an item in the TFS,
which refers to a unit of work. It could be just an item, which is a
task or a defect or a work item of type Bug, or it could be an issue or a
requirement item. The following screenshot shows the option to add a
work item using the menu option in VSTS menu. The option lists the
different types of work item.
The work item of type Bug
is a defect raised against a failure or error occurred from the
application. Now in our example above, the test result throws an error
from the code saying the expected result does not match with the actual result.
This code has to be fixed by the developer who wrote the code. Now we
can add the corresponding bug work item to TFS using the menu option, or
we can directly add that from the Test Results window.
The error is added as a Bug
to the TFS work item. It also fills the details required for the bug.
We can overwrite the details as per our requirement if we want. The Bug
has different sections to it. We can overwrite or fill additional
details to the Title and Description field for the Bug.
The second tab contains
the history of the defect changes. Whenever we modify the properties of
the Bug, it is tracked here in the History tab. The third one is the Links
tab, which contains the link to the source from which this defect
occurred. It is automatically filled by TFS when we create this defect.
There are some options to the right of this Links tab. The Open option opens the Test Result and the Test Result details window.
The Edit... option in the Links
tab opens the window that contains the link to the test result for the
test that is the source for this defect. We can edit the comment if
required. If we still do not have the test result in the TestResults
folder, we can delete this link. If there are additional links to be
added, or if the link is moved to a different place, we can add it.
The next tab in the bug work item is that of File Attachments, which has the trace file for the test attached to it. Select the trace file and click on Open which opens the test result window with the results loaded in the window.
This tab also has different options to add another attachment, save the attachment to a different location or to delete it.
Publish test results
Think about the different testers
in different locations running the tests. Everyone will have the test
results in his/her local machine. How do we consolidate and let other
project teams or the other team members know about the test results? TFS
maintains the operational store, which is a central database where all
the details are stored for future use.
VSTS provides an option to
publish the test results to the TFS operational store so that the other
team at a different location can easily look at the test results that
are published to a central location.
The Test Results
window has the option to publish the results. Select the check box
option against the test results in the window and click on the Publish button on the test results toolbar.
The publish option opens a new window, which lists all the test results in the current TestResults folder. Select the test to be published and select the Associated build number and the Build flavor. The code coverage data can be published along with the test result by selecting the option Include code coverage data for selected runs.
