02: Management, preserving and organization of manual test cases within VSTS 2010.
- Part 00: Agile Testing with VSTS 2010 and TMap
- Part 01: User Stories
- [current post, removed the Agile Testing blablabla and part pre-fix]
This post belongs to the series of “Agile Testing with VSTS 2010 and TMap” although this topic has a wider scope, in used software delivery methodology you have to think about test case management.
A small introduction.
First a small introduction to give some direction on the needs of test case management.
The drawing below I made during a discussion with several TMap experts and explains how test cases are created and executed in a simple situation.
From top to bottom and left to right.
 | Teams start brainstorm over user needs, create stories and write these down on cards together with acceptance criteria.
Within VSTS 2010 this can be done in the work item type ‘user story’ [see previous post]. |
 | After prioritizing the stories, teams start the iterations by brainstorm on implementation tasks and test tasks.
Again, within VSTS 2010 this can be done in the work item type ‘user story’ [see previous post]. |
  | Next, the sources and test case are going to be created, developed. Within VSTS 2010 source files in the IDE and than added to version control. Test cases are added as work item to TFS. |
 | The source files are build, unit tested, labeled, packaged and finally deployed on a test environment where the manual test cases are executed against this deployment |
All the artifacts created by tasks are under version control, except test cases.
All the sources and corresponding automated tests like, unit tests, load tests, web tests and codedui tests are under version control. Getting the latest version of the sources also gets the latest version of these tests. But, also when using the more sophisticated capabilities of version control, like branching and merging, the tests also have this characteristics.
Manual test cases different, they are work items and aren’t under version control they have a history. Belong [often] to a user story. So, it gets interesting when making the above scenario more real live with sources and automated tests under version control with branching and merging. See drawing below…
- User stories, sources and test case are created [top left].
- [red line] A branch is made from the first version to start the development of an other version.
- User story 2 [center] is added to this branch [new/ added user needs], also sources are added to this branch and test cases are created.
- Finally version 2 is build and the test case of user story 2 are executed [bottom right].
Dotted red line; while there was a branch created from the sources, no branch was created from the test cases of user story 1 [top left], resulting in a version 2 with only partial tested functionality.
This branching scenario is very simple, there are some more complicated ones as you can read in Microsoft Team Foundation Server Branching. This 2006 MSDN story has also some nice branching and merging strategies.
The need for test case management within VSTS 2010.
This branching, merging and the decoupled characteristic of test cases is one of the reasons why we need test case management. We manually need to organize and manage our test cases in such a way that every iteration, version, service pack, hot fix and release is sufficient tested and to prove that its well tested.
As described in the paper Test Cube principle [pdf]:
A good test set (= collection of test cases) is of priceless value. During the realization of an application it is the basis for the reporting concerning test coverage and progress. In a maintenance situation it is a source of knowledge concerning the application and a source of ready-made test cases for regression tests.
In the next post we will have a look / deep dive in the capabilities of TFS 2010, MTLM [Test and Lab manager] how they fulfil the needs of test case management and in what way the TMap process template helps this process.
One part of the solution and even an agile one: automate the manual test cases as quick as possible and put them under version control. Next post more…
Enrich VSTA 2010 Use case diagram with SketchFlow Screens
A use case diagram visualizes the interaction of an external user with your system. So, you can say that in many situations [cases] that there is a need for an user interface, a screen.
Not that strange that RUP connects the user interface designer to use cases. [see image below at the bottom]
UML Type: Artifact
Something every diagram has in VSTS 2010 is the UML type ‘Artifact’. With this type we can attach physical pieces of information to or diagrams.
For example in this diagram below, I defined an artifact which points to a class file. [don’t think it very useful to associate C# class files with use cases.. but as an example]
Some nice functionality of this ‘artifact’ shapes gives us the capability to double click the shape and jumps to this file. More useful, what you also can do is point to a Word document with some additional information according to this use case diagram, double clicking will open the file in Word [or you’re preferred text editor].
Microsoft Expression Blend 3
SketchFlow is a visual tool for prototyping desktop and web applications [WPF/ Silverlight].You can download the trail here. The image below, is a prototype of an application made with SketchFlow. A bit sketching, playing with colors real designers work... :-)
Smashing Magazine had a nice overview of sketch techniques last week “35 Excellent Wireframing Resources” and I made some several weeks ago for this post Storyboard: Architectural Inspections - VSTA2010 – AppArchGuid.
Now, the interesting piece is that SketchFlow, Blend solutions have the same structure as Visual Studio solutions. So, we can open or sketch in Visual Studio. as you can see in the image below, it’s just a normal Silverlight 3.0 project. [installing Expression Studio gives VSTS the ability to open Silverlight 3 projects].
From this point it’s very easy to add the ‘XAML’ screens as an artifact link to our Use Case Diagrams.
The small challenge at this point is that double clicking on the artifact opens the XAML file of your prototype in ‘your preferred XAML viewer’ your internet browser, showing nothing. A small tweak [replace the full path property value with a batch file with one command “devenv.exe /edit filetoopen.xaml”, the /edit opens the file in the current open environment] makes it possible to open the XAML file in the currently open Visual Studio edition.
Finally the result is a Use Case Diagram with several artifacts referring to user interface screens. Which gives the user interface designer, the system specialist, the customer actually everybody who is involved in this software project an overview of the solution, which results in better understanding and the user interface designer still can work in SketchFlow.
final note: there are still a few integration problems, but I do think this is a valuable scenario….
Agile Testing with VSTS 2010 and TMap: Part 01-User stories
Previous post:
- Part 00: Agile Testing with VSTS 2010 and TMap
- … [current post]
User stories breakdown...
This post is about the work breakdown of user stories during the planning phase of an iteration.
[image from the documentation about VSTS 2010 on MSDN, this page]
During the planning phase of the project, also called iteration 0 [first blue piece], user stories are collected / brainstormed / defined /… in VSTS this information is collected in the new work item type ‘user story’ [image below].
During the planning of the iteration developers the team start to breakdown the user stories [which are selected for that iteration] in implementation tasks. Within VSTS this is done in the implementation tab. The new 2010 functionality of hierarchies between work items is used for this.
Another task also executed during this phase is the creation of test cases. Within the TMap methodology this is described in the planning phase. Where you create the test plan for that iteration, based on the risk class [see user story work item] and discussion with the customer the necessary test techniques are allocated for the user story and functional area. See the initial work items for iteration 1 in the image below, added during the unfolding of the TMap process template.
So, not only the implementation tasks needs to be determined also the test tasks needs to be allocated during the planning phase of an iteration. These test tasks are specific to testers like ‘create test cases for area …. based on the test technique <… decide based on risk and customer contact>’
List of different test techniques…
- Data combination test (DCoT)
- Data cycle test (DCyT)
- Decision table test (DTT)
- Elementary comparison test (ECT)
- Error guessing (EG)
- Error testing (ET)
- Process cycle test (PCT)
- Real life test (RLT)
- Semantic test (SEM)
- Syntactic test (SYN)
- Use case test (UCT)
So, just like the implementation tasks also the test tasks are a link type, have hierarchy with, the user story. [this is TMap process template specific]
We end up with a list of tasks for a user story, implementation tasks and test tasks… testers and developers are going to execute these tasks together in parallel, during the iteration we have: implemented sources for the user story and test cases which are ready for execution. This user story is finished when: every implementation task is fulfilled, all test cases are successful executed and … the tester hasn’t got any open tasks, so all test cases are created.
When giving the tester/ the team a place where they can record their testing tasks, testing is really going to be a first class citizen of the lifecycle. beside this benefit the connection between test activities, risk, user story and test cases gives a great opportunity for reports based on test effort, risks and implementation, later on more on this…
With this this lightweight addition to the user story work item we got a closer to several Agile Testing key characteristics, for example:
Customer involvement in writing tests. –> just like the developers, testers breakdown there work discusing it with the customer An iteration is ready when the tests succeed. –> and all tests are created..!
Copy-Past Modeling Diagrams and Modeling Elements within VSTS 2010
This is completely new for me… CTRL-A // CTRL-C
ALT-TAB // CTRL-V
It also works in other applications, even Outlook. This kind of functionality [copy past between models] was already written down in the DSL Tools Book [page 437].
Copy past between models is also possible, but than you have to look very careful what and how you do it. For example copy past within the same model create a model element with the same name with a number at the end.
It’s interesting to look at the Model Explorer, also in that one there are shapes added with the same name.
Now, copy pasting from one diagram to another diagram has the same behavior, modelelements are created with a number. [the number is added because they also get shown in the model explorer, two shapes with the same name isn’t possible]
If you want these shapes to be the same, and not a copy, you need to drag and drop the shape from the model explorer on design surface of the new diagram.
The benefit of this is that now these model shapes are connected, changing the name of one [on the diagrams or the model explorer ] will automatically synchronize the other elements.
Another thing to pay attention to is the copy-past behavior between different model projects…. actually you can see it immediately when pasting in the diagram of the other modeling project, no number added. So you could think that the behavior is the same as drag and drop from the model explorer. But that isn’t the case, while adding an other modeling project also a new root is created in the model explorer. So, those two model elements are two different things. Drag and drop isn’t possible between model projects.
If you do want to re-use mode elements in a different modeling project, you can us the copy past functionality, which works really nice. But, you also can copy past the content of the modeldefinition file to the other project. Don’t think that is the way to go… but it works, it also reminds the relationships between the elements. You could create a default package every project needs to us…
Soma on Architecture Tools…
Nice overview post on the Architecture Tools by Somarsegar. He uses some new bits for the images of the diagrams.
![y1pOvtjHPmcumaRA19YyvZNqNMPEGS3pYwfHuGvpDSmLZD0qqaJMpSF-VGysgRU-WIfWuR45LcB29n3I0V8bNJxAg[1]](http://www.clemensreijnen.nl/image.axd?picture=WindowsLiveWriter/SomaonArchitectureToolsandthelittledevil/66F49082/y1pOvtjHPmcumaRA19YyvZNqNMPEGS3pYwfHuGvpDSmLZD0qqaJMpSFVGysgRUWIfWuR45LcB29n3I0V8bNJxAg1_thumb.png "y1pOvtjHPmcumaRA19YyvZNqNMPEGS3pYwfHuGvpDSmLZD0qqaJMpSF-VGysgRU-WIfWuR45LcB29n3I0V8bNJxAg[1]")
The layer diagram has two more menu item groupings [I like the custom items] and strange numbers in the shapes, curious what that is.
Finally we have a nice overview how the Petshop application works :-)
Agile Testing with VSTS 2010 and TMap: Part 00
Part, because I’m planning several posts around this topic.
00, because this is the ‘setting the stage’ post…
While in the Agile manifesto is written: “Individuals and interactions over processes and tools” I will cover in the coming posts information how individuals and there interactions are supported by tools and processes.
Several characteristics of agile testing [in random order].
- Lightweight implementation of tools and processes.
- Find bugs as early as possible.
- Don’t do things twice, just enough testing.
- Short feedback loops to the team.
- Customer involvement in writing tests.
- An iteration is ready when the tests succeed.
- There needs to be space for exploratory testing.
- Focus on automation.
- Repeatability is a key success factor, frequent testing.
- Pair testing.
Tool and processes support should focus on these characteristics. At this moment the tool support with VSTS 2010 for the tester is getting really mature [see this post]. So, beside the agile support for the developer like TDD, refactoring, continuous builds, guidance and many other things [see this post and image below].
Also the tester wants / needs this kind of support, beside an easy connection with all the capabilities developers have he needs the capabilities to execute his work in an agile team…
So, what practices needs to be in place for the tester within a VSTS 2010 agile project? just randomly start using the offered tools isn’t the right direction. In the coming posts I will capture some thoughts, practices and lessons learned from around the world…
As a start some useful agile testing links:
MSDN VSTS2010 doc’s - Modeling User Requirements
Have been reading all the MSDN documentation about VSTS2010 last few days and I’m amazed by the quality of this. So if you don’t have the bits installed, just take a look at MSDN to get a deep dive in the new functionalities of VSTS2010.
This is a screenshot from the Team Architect doc’s about “Modeling User Requirements“…
and when you do have the bits installed, hit F1 sometimes… [I often forget that functionality]
TMap Process Template for VSTS2010 – Short Version
Got way too much comments on the length of the TMap process template ‘overview’ video, I know 38 minutes is long for listing to me, my wife even can hold it for 38 seconds, wen I start talking about my work ;-). So, I made a ‘community cut’, a shorter version with only the highlights of the extended version…
I’ll keep the YouTube constrains for other video’s, max 10 minutes. [post a comment if that's even too long]
just saw that the end is a bit strange/ abrupt, don’t know what went wrong, will change it on a latter time
TMap Process Template for VSTS2010 - Overview Video
This is an update of the work we have been doing regarding TMap [a widely adopted test approach] and VSTS2010. With the release of Beta1 of VSTS2010 we’ve started to build a TMap test process template. This 38 minutes video recording [silverlight] is an overview of this work in progress. Feel free to give us any kind of feedback, by using the comment form or by sending a mail directly at Clemens dot Reijnen at Sogeti dot nl and introduction story can be found here.
VSTS 2010 TMap Process Template 1,2 and 3
As probably everybody already know, Microsoft is going to deliver a toolset for the tester with Visual Studio Team System 2010. If you didn’t know this, you can start by reading several previous posts on this blog, on Rob’s blog or on Bing. It’s not only going to support development testing or system testing but the whole stack of the testing practice including acceptance testing [manual testing].
This support for all the different test tasks by VSTS 2010, combined with the power of TFS 2010 to guide processes and Sogeti’s method for structured software testing ‘TMap’ is a major win to the application lifecycle. To get this structured software testing method ‘TMap’ into TFS2010 we made a process template.
Now it’s a bit useless to deliver a process template only for the testing practice, although there are situations where it can be used. But, mostly the conditions are that testing practices are executed during the software development lifecycle. This is a big challenge when offering a method for structured software testing, needs to fit the software development lifecycle.
This challenge is also in place with the Secure Development Lifecycle Process template. The SDL team made template for secure software development, what is really great, but actually every software development process should be secure. So secure processes should be merged with existing templates as Brian Harry writes in his post “The Microsoft SDL Process Template and the Future”
I think everyone understands that we don’t just want “one template for secure software development”
The same counts for testing. There is one big problem in this situation… process templates don’t merge very well. So, what we did to deliver TMap to VSTS2010 is creating more several templates.
First a standalone template. I know it’s useless in most situations, but for example when a customer want to manually test the application disconnected from the development team another situation where you can use a standalone template is in our test lines and StaaS offering, both situations where we are testing [manually] the application disconnected from the development environment, but with a TFS connection.
The second template delivery we made is a combined one. TMap processes combined with the In-the-box MSF/ CMM templates. Now I know nobody uses these templates without editing them. But, by providing a combined version people / companies who do a small edit on the existing [in-the-box ] templates can use these.
The last delivery is a bit more challenging and we didn’t finalize this one yet. We are delivering the XML so you can manually merge the TMap processes with your own template.
- Standalone TMap process Template.
- Combined TMap processes with the MSF Agile and CMM process templates.
- TMap processes merged with any existing templates.
Which one would you like to use?
[next post, what's in it…]