MSTest utility
This is the command line utility supported by Visual Studio. Using this MSTest tool, we can run the tests created in Visual Studio.
To access the MSTest tool, add the Visual Studio install directory to the path or access the Visual Studio command prompt from the Tools section of the Visual Studio group in the start menu.
After opening the command prompt, type MSTest.
The MSTest command
expects the parameter to be specified, which is the name of the test to
be run. To know the options of MSTest, just type MSTest/help or MSTest/? in the command prompt.
The help option lists the different parameter options that can be used with the MSTest and the description of each parameter and its usage.
Option
|
Description
|
---|
/help
|
Displays this usage message
<Short form: /? or /h>
|
/nologo
|
Does not display the startup banner and the copyright message
|
/testcontainer:[file name]
|
Loads a file that contains tests; you can specify this option more than once to load multiple test files
Examples:
/tescontainer:mytestproject.dll
/testcontainer:loadtest1.loadtest
|
/testmetadata:[file name]
|
Loads a metadata file
Example:
/testmetadata:testproject1.vsmdi
|
/runconfig:[file name]
|
Uses the specified run configuration file
Example:
/runconfig:mysettings.testrunconfig
|
/resultsfile:[file name]
|
Saves the test run results to the specified file
Example:
/resultsfile:c:\temp\myresults.trx
|
/testlist:[test list path]
|
The test list to run as specified in the metadata file; you can specify
this option multiple times to run more than one test list
Example:
/testlist:checkintests/clientteam
|
/test:[file name]
|
The name of a test to be run; you can specify this option multiple times to run more than one test
|
/unique
|
Runs a test only if one unique match is found for any given /test
|
/noisolation
|
Runs a test within the MSTest.exe process. This choice improves test run speed but increases risk to the MSTest process
|
/noresults
|
Does not save the test results in a TRX file; the choice improves test run speed but does not save the test run results
|
/detail:[property id]
|
The name of a property that you want to show values for, in addition to
the test outcome; please examine a test results file to see the
available properties.
Example:
/detail:errormessage
|
In addition to the above options, there are many other options which can be used with MSTest if team explorer is used.&;&;
Option
|
Description
|
---|
/publish:[server name]
|
Publishes results to the TFS
|
/publishbuild:[build name]
|
The build identifier to be used to publish test results
|
/publishresultsfile: [file name]
|
The name of the test results file to be published; if none is specified, use the file produced by the current test run
|
/teamproject: [team project name]
|
The name of the team project to which the build belongs; specify this when publishing test results
|
/platform:[platform]
|
The platform of the build against which to publish the test results
|
/flavor:[flavor]
|
The flavor of the build against which to publish test results
|
We will run and see some test for the above options using the MSTest command line tool.
Running a test from the command line
Let us see some of the options for MSTest
and see the results produced by these options. The MSTest utility does
not work for manual tests. It is applicable only for the automated
tests. Even if we apply the command for the manual test, the tool will
remove the non-automated test from the test run. For example, if we run
the test container command for the manual test, the reply would be
similar to the one shown below:
/testcontainer
T&;&;he testcontainer
option requires the file that contains information about the tests that
must be run. The test container file differs across tests. For example,
the unit test information is contained in the unit test project assembly
file. For the ordered test, it would be the<ordertestproject>.orderedtest file, which contains information about the ordered test.
If we have multiple test
projects created under the solution, each of these projects has its own
container for the tests within the project.
Let us consider an ordered test which contains four different tests such as generictest1.generictest, GetTotalPrice Unit test, and couple of manual tests. The name of the ordered test file is OrderedTest1.orderedtest. Now let us try using the MSTest
for running the tests in the ordered test file. You can see the output
of the test as shown in the following image. The automated tests removed
the manual non-automated tests from the list and ran only the automated
tests.
For Unit tests and Web tests, the test project file will be an assembly. For example, the following command will load the TestProjforTestingApp.dll assembly and run the test within that assembly.
First, the MSTest
will load all the tests within the ordered test and then start them
executing one by one. The result of each test run is also shown. The
results are not shown but the detailed test run is also stored in the
test trace file. We can load the trace file and see the details of the
test run.
/t&;estmetadata
The&; testmetadata
option is used for running the tests in multiple test projects under a
solution. This is based on the metadata file created under the solution.
The metadata file contains a list of all the tests added to the Test
List Editor. Using this Test Editor, we can create a new list of tests
from all the available tests under the solution irrespective of the
projects.
The testcontainer option is specific to a test project whereas testmetadata is for multiple test containers with the flexibility of choosing tests from each container.
For example, the following
image shows two different test lists which contain different tests added
to the list from all the available tests in the solution:
Any change or update made to
the test list editor, will be saved in the metadata file. Now, to run
only the tests added to the test lists in the editor, we use the option
metadata. To run a specific test list, we should use the /testlist option along with the testmetadata option as shown:
mstest /testmetadata:[file name] /testlist:[test list name]
The following command runs all the tests added to the list, TestListOne:
If there are no tests added to the test list, and if it is an empty list, the Mstest command will clearly display a message saying there are no tests to be run in the list.
/tes&;t
There&; are
situations where we might have to run a particular test from the
available list of tests. In that case, we can use the /test option with the testmetatadata option or the testcontainer option. For example, the following command runs only the CalculateTotalPriceTest test from the list:
The /test option can be used along with testmetadata or testcontainer, but not both. There are different usages for the /test option:
We can specify any number of tests using the /test multiple times against testmetadata or testcontainer option.
The name used against the /test
option is the search keyword of the fully qualified test names. For
example, if there are test names with fully qualified names such as:
TestProjforTestingApp.Class1Test.CalculateTotalPriceTest
TestProjforTestingApp.Service1Test.RegisterNewUserTest
TestProjforTestingApp.Service1Test.GetMyOrdersTest
And if the command contains the option, /test:TestingApp then all the above three tests will run as the name contains the Testingapp string in it. Even though we specify only the name to the /test option, the result displays the fully qualified name of the tests run in the results window.
/uniq&;ue
This o&;ption will
make sure that only one test, which matches the given name, is run. In
the above examples, we have three different tests with the string TestingApp in its fully qualified name. If we run the following command, all the above three tests will be executed:
mstest /testmetadata:c:\workspace\testingapp\TestingApp.vsmdi /test:
TestingApp
But if we specify the /unique option along with the above command, the MSTest
will return the message saying more than one test was found in the same
name. It means that the test will be successful only if a unique test
name is found with the name given.
The following command will execute successfully as there is only one test with the name Webtest3:
/noisolation
This opt&;ion runs tests with the MStest.exe process. This choice improves test run speed, but increases risk to the MSTest.exe process.
Usually, the tests are run in a separate process which takes its own memory. If we launch the MSTest.exe with the /noisolation option, the tests are run in a process which avoids more memory usage.
/runconfig
This is to specify to the test run to use a specific configuration file. If the configuration file is not specified, the MSTest uses the default configuration file. The following example forces the test to use the TestRunConfig2 configuration file:
/resultsfile
In all th&;e command executions, the MSTest stores the test results to a trace file. By default, the trace file name is assigned by the MSTest
with the login user ID, the machine name and the current date and time.
We can force the test tool to use a specific file to store the test
results. For example, the above test result is stored in the file User1_MyPerPC_2008-06-09 20_53_32_.trx at the default Visual Studio folder. We can use the resultsfile option to change the location and the filename of the results file.
The previous image shows the test results stored at the c:\temp location in the results file, StoreTestResult.trx.
/noresult&;s
This optio&;n is to inform the MSTest
application to not store the test results to the TRX file. This choice
does not store the results but increases the performance of the test
execution.
/nologo
This optio&;n is to inform the MSTest tool to not display the copyright information which is shown at the beginning of the test run.
/detail
This is th&;e option used
for getting the values of the properties of each test run result. Each
test result provides information about the test such as error messages,
start time, end time, test name, description, test type and many more
properties. Using this option we can get the property value after the
test run. For example, the following command shows the start and end
time of the test run and also the type of the test run.
The detailed option can be specified multiple times to get multiple properties value after the test run.