The missing knowledge of testers in modern app development.
The game has changed for testers, they need to extend their basic testing skills when they want to stay valuable. As other team members, testers need to be a full member of the team with the current skill set this isn’t.
While supporting teams for a long time in delivering software I noticed I’m always teaching new ‘test’ team members the same skills. Skills a new team member actually already should have had before entering the team room.
A small list.
Knowledge about agile work in teams is the minimal a test team member should have.
When they fail a basic assessment like the Scrum.org open assessment ( https://www.scrum.org/Assessments/Open-Assessments/Scrum-Open-Assessment ) I silently start to cry.
Agile is that mainstream now a days that I even wonder why conferences like ‘The Agile Testingdays’ exists. The practice of agile test work should be crystal clear at this moment. So, mastering agile work is really the bear minimum.
Team Tools support.
Know your tools. Not only the common testing tool ‘Excel’ but more important the Application Lifecycle Management (ALM) tools the team uses.
ALM Tools like Visual Studio Online and other tools mentioned in the Gartner ALM report are important.
Magic Quadrant for Integrated Software Quality Suites
Gartner’s Magic Quadrant for Application Lifecycle Management
The tester isn’t alone anymore, testing isn’t the activity at the end of the waterfall what everybody skips, it is part of the delivery. Having knowledge how to use integrated team tools is a must to be valuable and not being a blocker of productivity.
Basic programing skills
Test automation is a must in the current fast delivery mindset of Continues Delivery. And this test automation must be, just like all the team work, integrated in the team process.
Standalone test automation tools are useless. Test automation connected with the team sprint tasks, the team build runs, the team delivery and the code base is a must for agile teams in being efficient with a focus on quality.
When test automation is in the same code base, in the same rhythm as the whole team, the code written for this test automation is logical the same programming language as the language the system is developed in, at least it is the most efficient.
Having basic programming skills as a tester is a must to be valuable for the team. As the image above shows, only just a small part is manual testing. So start to learn programming.
(Distributed) Version Control
Every artifact a team creates should be under version control. For the codebase it is normal to use a version control system. Either a centralized system like Visual Studio Online or a distributed system like GIT, everything is under version control.
All team members must understand the basics of version control. How to get or pull artifacts in or out and the pros and cons of the usages of branches.
Testers need to be capable to get the version they want to test from version control them self, they need to understand what artifacts are in the version and what features are delivered at what quality.
Knowledge of version control systems will make testers less depended on the other team members who do know what is in what version.
A good read: Version Control for Multiple Agile Teams
System under Test Deployment models.
As mentioned in the previous needed tester’s skill, testers should be less depended on other team members to get their system under test. Knowing how to deploy the software is a great added value for testers. This is for all kind of systems for example, knowing how to install a Store App on a tablet for testing will make the tester immediately more productive.
For more complex systems, knowing how PowerShell, Chocolaty, Puppet, Chef, Visual Studio Release Manager or other deployments mechanisms work would be great.
But at least understand how to start the deployment as a tester is the minimum. In the same category understand how to setup a test infrastructure for automated testing would be very useful.
Understanding the capabilities of current monitoring systems is a big benefit for testers. A tester who understands the behavior of the system under test is a invaluable benefit for the team. This tester helps to solve bugs as quick as possible by providing valuable usages statics, trace files or intellitrace files.
The last skill for a testers I want to talk about is Cloud. When a tester wants to stay in business, he or she needs to understand and usage the Cloud for testing.
Beside easy onboarding of new team members Dev & Test in the Cloud also can be used for your complete test infrastructure and with the new gained deployment knowledge also the provisioning of test environments should be easy for testers.
See this deck: Azure for Teams
I can imagine you think ‘as a tester’ this isn’t my job role, these are the activities and skills of other. But as James Whittaker mentions in this post
“For the vast majority of app development on this planet, software testing is an activity within the development process …. It’s time to bring quality into the 21st century where testing is such an integral part of software development that you’ll often forget you are doing because it has become so familiar.
Testing is integrated in modern development, a testers must evolve. To stay valuable for teams and keep their job the in this post mentioned skills must be added to the capabilities of the modern tester.
Not only these capabilities count for testers, all other team members should master them too. So there is also some lessons to learn for developers.
DECK: Azure for Teams
More business value with fast and flexible teams.
The deck I used (parts of it) at different conferences.
My Garmin Forerunner 405 died today.
I think it is the battery which didn’t charge properly any more.
The Forerunner 405, it was my first wearable device, connected with the internet. I also got the Heart Rate Monitor, the Foot Pod and the Speed/Cadence Bike Sensor sensors. Too much fun all these sensors when running.
All these hypes now days around wearable computing sounds a bit strange to me, it was already there five years ago. In what stone age have all these analysts lived these years. For sure they aren’t runners, they would have know better.
It is funny when I think back about a talk I had after I bought this Garmin device about logging my activities the guy was amazed I logged everything, he is now wearing a FitBit.
I had my Garmin for five years and I did run 5.390,06 km with it for 482:47:13 u:m:s. The device did cost about 300 euros (I think, not exactly), a simple math gives us that the device cost 5 cent per km, or 60 cent per hour. I can live with that.
I did two marathons with it:
I did many half marathons as training in the Amsterdamse Bos (my backyard)
I ran in Dubai, slowly due to the heat.
and in Seattle, with a beautiful view on Mnt Rainier
Did some hiking in Norway.. beautiful
Many times ski touring in the Swiss and Italian Mountains
And many many more, finally more than 500 activities have been logged with my 405 the past five years.
I sure want a new one, apps on a mobile device can’t replace it.
But, have to save some money first... I like the Fenix 2
Things to think of when using Team Member IDE on Azure, networks.
Azure for Teams
Having Team Member IDE’s on Azure provides create benefit for teams in moving fast and staying flexible.
But there are still some items you have to think of when start adopting it, networks being connected is an important one:
You never ever develop or work in isolation. Especially with the heterogenic systems we build now a days. So, being connected with the world from your Azure hosted Visual Studio IDE is a must.
For example, the SQL Server database you are consuming in your App is in the local data center. You need a connection to that datacenter from your Azure environment.
You can create a Site-to-Site VPN, for connecting datacenters to an Azure site or Point-to-Site VPN, when connecting a single machine to an Azure site.
· Configure a Site-to-Site VPN using the Management Portal Wizard http://msdn.microsoft.com/en-us/library/windowsazure/dn133795.aspx
· Configure a Point-to-Site VPN using the Management Portal Wizard http://msdn.microsoft.com/en-us/library/windowsazure/dn133792.aspx
This can be challenging, not from a technical perspective but more form an organizational perspective. From Azure you can setup a Side to Side network connection to the datacenter, but this also requires some changes in the company policy in connecting to Azure. Some IT-departments aren’t ready for that.
A solution can be, put the SQL Database beside the VM’s in the same Virtual Network on Azure.
Team Member IDE in the Cloud, more possibilities... Complete Team
Azure for Teams
Having a Team Member Desktop brings a team great benefits. Self-provisioning of environments for team members is one of these benefits, give the control to the Team responsible for the realization of the system. Helping to solve the team challenge of delivering fast and flexible new software.
Beside this single scenario which makes the team faster and flexible. There are many more possibilities for teams in using a Team Desktop on Azure, possibilities which bring new opportunities and benefits.
When you can create a Team Member Desktop in the Cloud for a single person, you also could create multiple for a complete team.
The start of a new project is an investment with a risk.
Most often after a few sprints it starts to get clear if the project is viable. The customer (product owner) gets the first drop and gives direction on the development. This direction could be, stop with the development, move to a different platform or some other mature change.
Changes in insight could result in losing the investment of project startup. Minimizing the investment will make it easier to move. Lowering the barrier to start a new project with less investment will give teams more possibilities.
Creating Team Desktops on Azure for a complete team, with multiple disciplines, will solve the team challenges of having Small Budgets and lowers the investment risk.
You could create every IDE in a separate personal MSDN subscription. Beside this is boring work it is also better to have team member connected in the same network and the environments managed via a project Azure subscription in an Azure Enterprise Subscription.
When a project has its own Azure subscription, they can manage their own ‘base’ images used to create the team member development environments, and other environments. Having all the project costs in one place.
This script is a good starting point, it creates four VM’s in an Azure network.
Faster realization of business value, some thoughts on Models and CodeGen.
Model driven development and code generation comes in many different forms, and all have the same promise: faster delivery of business value. Not every form has been even successful in realizing this promise. Where some have died silently in their own complexity, and some are standing still in innovation for years, there are other ‘generation’ solutions who are getting the momentum of success.
Learn about the MDD approach and how you can apply it to achieve the maximum business value and reduce the cost of solution development.
Article from 2006 - Implement model-driven development to increase the business value of your IT system- http://www.ibm.com/developerworks/library/ar-mdd1/
Model driven development is about the changing the abstraction level of the used programming languages to realize business value. When you make the language closer to the business solution than C#, C++, Java or any other programming/ machine language it is easier, and more accurate, to describe the solution. When you can run this, more abstract, language (or model) the same way as program languages are executed, than you have a model driven approach. More high level languages, describing the business can be used by the business to describe their own solutions. And, more low level abstraction languages and models can be used by engineers to describe a more technical solution. With these new languages it is easier to communicate about the specific domain, a domain specific language.
In Application Lifecycle Management * languages and models describes the different domains. A specific languages used to describe the business domain are often technology and platform agnostic. While specific languages used to describe the solution are often implementation and platform independent, which gives less re-use.
* Application Lifecycle Management is about the whole spectrum of software development, the processes that are used, the people who use it, make and maintain the applications and about the tools and technology that are used during the complete lifecycle. Application Lifecycle Management is not only about software development and a grown-up development environment but also about business, operations and the connection between these domains.
In both domains, the business and development, domain (often called the problem and solution domain) new possibilities, innovation and business opportunities are insight. But, for both domains making the decision to adopt model driven development need to be taken careful, a well tradeoffs process needs to take place.
With business domain specific languages the promises to gain benefit of fast realization of business value are the biggest.
It is a language used by the business / domain experts to describe the ‘problems’ [needs], the models can be used to generate code or can be executed on a platform.
Vendors who deliver technologies to create and execute Business DSL’s are for example MetaCase (http://www.metacase.com/) and Mendix (http://www.mendix.com), both mature in the solutions they provide. Microsoft has Domain Specific Languages capabilities in its development IDE (http://msdn.microsoft.com/en-us/library/ee943825.aspx), and Eclipse has one of the most used DSL tools addin (http://www.eclipse.org/modeling/ based on the open source project openArchitectureWare). MDA based on UML is another technology for MDD http://www.omg.org/mda/
There are a few adoption challenges with business domain specific languages. One challenge is the clear description (Meta language / the grammar) of the language. Now day’s business are moving and changing faster than the language can evolve. Microsoft changed to a Services and Devices Company, the language used to describe the business needs to change too. And, creating a specific business specific language isn’t that easy.
Another challenge is the execution of the models (languages) on the platforms. The language must be interpreted by the platform to run, or compile to executable code by the platform (see image above). Creating a compiler or an interpreter is very specialized and technical work. Maintaining these will need some effort.
The last challenge has to do with team work. Probably multiple people will work on the business models. Multiple versions of the models will live in different stages of the development process. Maintaining these model versions, work together on one model and find the differences between versions of models is a challenging task. Current available development and modeling environments aren’t mature enough to capture this.
The above mentioned technology vendors target these challenges, and while the innovation of platforms evolves they also need to evolve to execute the models on these new platforms. This will definitely help to lower the maintenance of software systems. Innovation goes slow and only for specific cases promise of faster business value is reached, when you have such a case it definitely pays off to investigate the possibilities.
Technical domain specific languages are used by the team to describe the solution domain. Where business domain languages have the benefit that the subject matter expert can describe the problem. With technical domain specific languages (and models) you loss this benefit. But there are already a lot of specific languages in place for the solution domain, which getting more and more mature.
There are more technical specific languages for one specific domain. For example SQL is a language for working with data. Another specific domain language is WiX for describing installer packages. On the other side there are also languages which are closer to the business. For example Smart User Cases ( http://www.smartusecase.com ), which uses specific User Case stereotypes to describe the business needs and generates with accelerators code, test cases or any other project artifact when there is an accelerator for. Code is generated and executed on the platform, so it is easy when solution is described to generate code for another platform.
Both most technical and business languages models are having the same adoption challenges. Hard to create the language, stay in sync with the platform innovations and use them in teams. But, technical languages are easier to adopt for multiple reasons.
One reason is that the created models and languages can be reused more often, they aren’t business specific, but can be used for multiple business domains. Which makes them more mature, robust and they are probably maintained by a technology vendor or open source group. And, innovation will move faster with more people using them.
For example Visual Studio LightSwitch is an addition to the Microsoft development platform to create business applications for the Desktop and the Cloud. Visual Studio LightSwitch generates the underlying infrastructure. The database, transaction mechanism and helps you craft the screens, which when you choose to build a HTML Client generates a responsive website which runs on all kind of devices.
Not only code generation (or model interpreters) have to be used for domain specific languages also specific frameworks for specific tasks are enablers for faster (and more reliable) realization of business value. Java is a kind of famous for all the different frameworks created for it. Examples are more technical Frameworks like the Spring Framework, but there also have been implementations like an Open Healthcare Framework.
In the same area as Frameworks you can find API’s, Application Programming Interfaces. API’s are the new enablers of fast business value. For almost every business problem there is an API available. The difference with Frameworks is that you don’t have to incorporate them in to your solution, you just use them.
The website ProgrammableWeb.com has a collection of 9920 APIs. With additions every day on all kind of categories, ready for business use with as an interface, a language to help with a specific technical or business domain problem. Use them in your solutions for fast enablers of specific functionality or stich several API’s together with additional business functionality and great a valuable mashup.
On the same technical side as API’s, as fast business value enablers, are Cloud Services. Cloud Services provide specific functionality for systems. The biggest difference with API’s is that Cloud Services provide more the technical infrastructure where API’s provide more business functionality. When your App needs to work with data, authentication or push than cloud service are a fast enabler.
A good example are the Microsoft Azure Mobile services and Media Services. Services that provide configurable functionality with generated code for your systems. It is possible to create it from scratch, but instead of hours it will take days to realize it.
In the area of Cloud Services innovation really moves fast. Services to enable faster business value are added almost on a daily base.
Other Cloud Services where innovation goes fast are services which helps development teams to do their job. Examples are Test and Development Platforms as a Service preconfigured and ready to use environments to quickly start working. Others examples are Load Testing from the Cloud and automated test execution on the Cloud.
Generate systems in the browser
One last innovation which takes off, are the web-based / do it yourself system generation sites. Where cloud services like Media Service from Azure provide generate code which you can incorporate in your code base. These generators are providing ready to use systems from the Web or Apps. A very successful is Windows Phone App Studio. Windows Phone App Studio is a website to generate a full working Windows Phone App in minutes, published to the store.
Another one from Microsoft is Microsoft “Project Siena”.
A very interesting innovation these kind of ready to use generators will available more and more, delivering the promise of code generation. But the generated solutions aren’t business ready, not unique enough, and need defiantly customization afterwards. I actually think it is create for trials, before doing the realy.
Insight on code generation innovation.
Code generation and model driven development are technologies with a promise to realize faster business value, but they are not always as flexible as the business need these days.
Specific to the situation it fulfills this promise. Where the generation from models (domain languages) on the business side are even more specific and should be analyzed careful be for used. But, when the business models are well implemented they will bring the most and fastest business value. Innovations on this area goes slow.
On the technical domain, code generation from models or specific languages (Lightswitch, Entity Framework) are well adopted by teams. The same for API’s, which add easy business knowledge to the systems. Both can benefit from and should use each other. Innovations goes faster in this technical domain, just due to the more frequent use than on the domain modeling side.
Fast innovation takes place in the Cloud Services stack, configurable services with ready to use code in your systems. More and more services will get available in an increasing pace to deliver that business value fast for your business users.
Found my Lost OneDrive (for business) files …
Got some scary moments after the upgrade to OneDrive, all my offline work was gone (which I’ve done during vacation, even worse :-).
When I upgraded a new folder was created named OneDrive and that one was synchronized with my O365 data, which didn’t had my local changes.
Lucky the old one was still there in C:\Users\Clemens, which isn’t a folder I open that often because I jut use the default Windows settings for maps. Copy pasting my offline work to the OneDrive folder did it.
Anyway, got to know.
Now it is still strange why my ‘new’ OneDrive doesn’t have the sync icon, but still is on the ‘old’ SkyDrive folder.
First meaningful activity.
Azure for Teams
When onboarding a developer the time between entering the team room and the first meaningful check-in, is waste.
In the same category the time between a testers enters the team room the first meaningful test specification or test execution is waste.
As everybody knows from Lean, we should aim to identify and eliminate waste in our projects. Mean Time to Valuable Activity is waiting time, time where a team member is idle and is not adding any value to the finished good or service. Creativity is lost, motivation flows away and talent isn’t used, what is another waste type in the 8 waste types defined in Lean.
Onboarding team members is hard. Multiple challenges needs to be handled: understanding the system, knowing the team work ethics, get the necessary tools, get access rights, etcetera.
Team Influence zone.
Most activities are in the teams influence zone. For example, time to understand the system been build can significantly be reduced by lowering the technical complexity. Another good reason to focus on Technical Dept as a team, the same for the methodology used. Activities which are in the hands of the team are adjustable.
Getting the necessary tools, equipment and access rights is often not an activity teams are allowed to handle. For usages rights the internal infrastructure department needs to give approval, for workstations and licenses some / multiple managers needs to sign documents. Sometimes it takes weeks before everything is agreed up on and in the hands of the new team member, ready to start the first meaningful activity.
Team Desktops on Azure
Time the new team member is waiting for equipment is waste, creativity is lost, motivation flows away and talent isn’t used. To remove this waste, is to give control to the team of every activity necessary to onboard a new team member. Technical Dept was already in their hands, getting the necessary tools is the one to get focused on.
With development and test environments in the Cloud (Team desktops on Azure). A team can get much more self-organized. Provisioning new environments for new team members on the fly without any interference of other internal departments, ie people outside the team.
A Team desktops on Azure is nothing more than a Visual Studio or Test Manager environment running on Azure VM’s (IaaS). With the Azure capability of creating your own discs teams can pre-configure the preferred IDE.
When having a MSDN subscription start using Team Desktops on Azure very easily can be done, most often within the boundaries of the subscription benefits.
When getting more mature in using Team Desktops on Azure, an Azure Enterprise Subscription is necessary. A good practice is to create a sub-subscription per project at start of the project. For sure with a team member as co-administrator of that subscription.
When teams can self-service the onboarding of new team members waste is minimized and the new team member is delivering faster more motivated business value to the project. Reducing the time to first meaningful check-in or test specification or test run is a good metric.
Visual Studio Online Quality Dashboard WIN8 App.
We just released to Store the Visual Studio Online Quality Dashboard WIN8 App. An App which helps you being informed how projects are doing.
The start screen loads all projects the user is member of and visualizes the last build run state and date.
In one overview you can see which projects needs your attention. With personalized project images you also can easily navigate to you specific project.
(setting the personalized project image can be done in the project detail screen via the appbar.)
The project detail pages provides three key quality metrics of your project; unsolved bugs, test cases and feedback items.
It can take a few seconds before the data loads, when opening a big project for the first time.
Scrolling to the left, shows graphs and more details for these quality metrics. An important part in the middle are the build definitions.
When scrolling even further to the left shows the tasks board of the project. Which is fully functional.
Drilling in to a build definition gives an overview of last build runs with the change set details and code changes.
Bugs, Test cases and Feedback
The Bug, test cases and feedback work items views for navigating and editing of specific work items.
On the project detail screen the Add TMap Tasks gives the user the capability to add predefined Testing Tasks to a Sprint.
Selecting a task gives some more details about the task.
After selecting TMap testing tasks and a sprint, you can upload them to you project in a single click.
The implementation is based on the TFS OData API https://tfsodata.visualstudio.com/
Read more about it:http://blogs.msdn.com/b/briankel/archive/2013/01/07/odata-service-for-team-foundation-server-v2.aspx
Telerik controls are used for the graphs: http://www.telerik.com/products/windows-8/overview.aspx
WinRT 8.1 Build Error … Task 'GenerateAppxSymbolPackage' failed. Could not find a part of the path …
Suddenly builds start to fail on a Win8.1 project, with the interesting message “Could not find a part of the path ”.
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\AppxPackage\Microsoft.AppXPackage.Targets (1512): Task 'GenerateAppxSymbolPackage' failed. Could not find a part of the path 'D:\Work\drops\projectName_18.104.22.168_Debug_Test'.
This messages is due to the fact that a team member runs the Package Creation Wizard from Visual Studio, sets the Bundle drop location and put the solution in the source repository.
The build server wants to use the same location as the value set by the developer. it is not saved in the Package mannifest, but in the csproj.
the value is saved in the csproj <AppxPackageDir> key, setting it during the build would be better. Setting it to a drop location known by the Build Server solves the error.