Free Test Conference Norway, 2010 - feedback

I have been speaking at Free Test conference in Trondheim, Norway last week. Even though we had not so many participants (~50 people vs. ~70 last year), which is probably due to the recession w/w, the atmosphere was good, and people really liked to get more knowledge. My talk was about Root Cause analysis - dealing with problems not symptoms. It dealt with an enhanced way for performing root cause analysis, using the 5ys and cause effect diagrams methods, enabling the outcome to include priorities in it, thus helping us pick the right improvement route to reduce chances for the problem to occur again (you may find it also on the testing experience magazine, volume 4, January 2009).

Other speakers I heard were Michael Bolton, Canada, who spoke about Median and the things be gain vs. the things we loose if we are over heating this median. Asha Jyothi Venkat Meruvu from India, spoke about the TestLink, open source tool for test management, and how it interacts with other tools (like Bugzilla, etc'). I also went to an interesting talk by Mike Scot, UK, who spoke of the Testify open source tool for quick infrastructure for test automation with Fit/Fitnesse, Java, etc'. The topic name was: 'Testify - One button test driven development tooling'. Mike presented the package he built quickly, and seems that it is easy to maintain and get things on ones test automation going fast.

Martin Gijsen, Netherlands, gave a nice view over the evolution of test automation, with clear examples and the right touch for business keyword driven approach. I guess KDT is here to stay...

Bernt Marius Johnsen, Norway, spoke of random generation of data, used with specific grammar of SQL sentences, to 'bomb' a few DB products (Like MySQL, Apache Derby and PostgreSQL) to find defects - which they do in his report.

Henri van de Scheur, who was the program chair, together with his dedicated organization team, worked a lot to make this conference a reality, and have succeeded in my eyes. Data, the organizer supporter deserve a lot of thank you on this, and I hope they will support the conference next year as well.

So, in general it was good choice of topics, and I had a lot of interest in those presentations, and it was fun as well! meeting interesting people with the same passion for test tooling, automation, and testing in general...

I am waiting for what next year is going to bring us...

Hiring New testers - How do they think?

As I was looking on other things, I stumbled on James A. Whittaker's blog on testers framework of thinking (http://googletesting.blogspot.com/), which I liked very much, also referred to Eric Jacobson (http://www.testthisblog.com/2010/01/test-this-light-switch.html) on the topic.

I have interviewed a few hundred testers myself in my professional life, and believe the insights are unique and gets your attention.

I usually ask the candidate to describe how she/he will test a commonly used device, thus not having to know specifications so much, and being able to focus on the approach, strategy, and scenarios. I like to see systematic thinking, and then on the other hand also unsystematic thinking. I believe that I have learned as an exploratory tester, that when you break you systematic thinking, bugs start to pop up.

Some good questions I have heard and used in past interviews are:
1) Which is more important positive or Negative testing?
analysis: in this question, I couldn't care less about what is being said, as they both are important, but I would like to hear the approach and how the candidate explains that to me.

2) If you had only half the time for testing what you have defined as in scope, how would you act?
analysis: in this case, I would like to hear the candidate find the way to identify, prioritize, improvise, be creative and approach this bad situation - which in many cases is a reality - and explain that to me. I accept questions and a few assumptions, so that the candidate would feel comfortable, but not too much.

3) If you had to provide quick feedback on the quality of a product, what will you do?
analysis: in this case, I try to put the candidate in pressure, as there is no time. I speak fast as well, and do not wait but only 3-5 seconds until I ask : well? and again, well? Creativity is important, improvisation as well, methods and good skills will get nice ideas come out from a guy who knows testing, and experienced challenging situations.

If we can identify testers thinking patterns and frameworks according to what we see in the interview, and even guess what approach they are going to take (after establishing the pattern), my question would be if we can identify and forecast their performance? and to what extent?

We all want to hire good potential testers, that would stay long at our company and fit our team profile, maybe further analysis will help us do that with a questioner or facilitated interview directed just at that?

We'll see...

In Agile Teams - A Plumber is a Plumber, and a Painter is a Painter

One of the things that Agile discipline encourage, and preach is that everybody should be equal on the Agile team.

Well, I guess I would not want my plumber to paint me a painting, nor do I want my painter to fix my pluming... I would not want the plumber to fix the roof of my house, or the roof builder to touch my pluming... etc'. a Plumber - even though knows how to fix a roof - will never be as good as someone who is a professional, with long years of experience, and methodological and theoretical knowledge in that area.

A developer - under this assumption - can test, but will not test as good as a professional tester on an Agile team. The same happens when a tester will develop software on the same Agile team. Yes, he can develop code, but no, not as good as a professional developer.

It is the same also on classical development models: testers are doing automation, but automation is writing code, testing it in low level as well (unit test). We - testers - are good and professional in testing. In Quality Control. Not in developing complex code.

So, the focus in Agile teams should be about communicating and bringing the best knowledge of each one of the team members on the Agile team to be visible to the other members. His/her knowledge can help the team go through the challenges that the company is facing, while others in the team can learn more about it.

Brussels Course: Adding Buisness Value Increasing ROI in Testing

Last Week I was at SWIFT, giving a course on Adding Business Value Increasing ROI in Testing. It was held in La-Hulpe area in Brussels, Belgium. I did this course with my German partner D&H.

The main course objective is to show and discuss how to optimize 3 major areas: the testing processes, the risk handling (risk based testing) and the effectiveness and efficiency of us managing our team of professionals.

We have discussed this and a lot of other good topics of discussion came like: how TPI(r) relates to Agile projects? Will some of the measurements and metrics work in an Agile world? etc'. In short, the idea behind improvement is not specific for a V model or Waterfall model, but has concepts that fit all models. But, nevertheless, one needs to adjust the tools to the right situation (as testing is of course context dependent).

We had 3 groups discussing: interfaces, Back Office and Network.

Also discussed dashboards and how they play a unique role in today's world both to find trends and identify risks (which can lead to improvement of our processes) and as management has a focus on testing and risks (see pic).

12th Year CONQUEST Nuremberg, Germany 2009

When many conferences around the world stop their operations due to the financial crisis, the Conquest (by iSQI) has not. With its 12th year in a row, the program was with the right mix, and still sponsors came. Conquest is the Quality Engineering in Software Technology, and is organized by iSQI/Germany.

iSQI organized it's 12th year Conquest testing conference and this time in Nuremberg. This year, India was the iSQI conference partner (last time it was Israel), and 5 companies were present with the delegations from India and came to present what they can offer. Meetings were held during all day Thur/Fri.

Tutorials were held on the topics: Test Management, Agile Software Development, Software Requirements, Load & Performance with high quality and international speakers and with live discussions.

Exhibitors that were present included also tsg, D&H, which presented unique courses - instructor led or via their virtual learning environment.

SIGIST Israel - Performance & Load Meeting - 26 May, Crown-Plaza

In the last SIGiST meeting on performance and load, there were ~50 professionals from 20 and more companies that came to hear about the unique methods exercised by experts. 3 speakers presented the topic.

On hte first presentation, we heard Ariel S from Comverse, that was given the task to establish the performance and Load department and expertise across the company. Ariel spoke about what is it performance and load profession, what challenges we are facing in this technical field of expertise.

Also, we had a discussion on the importance of this profession or area of expertise in the testing field, and which skills are needed from a professional exercising it.

Ariel finished saying that such a 'superman' does not exist, but the thing is that performance and load engineers must be heterogeneous in their knowledge and skills in order to perform a good job.

On the second presentation we heard Elad S. from NESS, who presented a performance and load project of monitoring and control system for the army. He spoke about the benefits that the performance and load test can create, what is the downside of NOT doing it, and not doing it in the right time - e.g. at the last time of the project.

Elad, stressed out the KPIs that are suggested to monitor and analyzed in order to isolate and identify the bottlenecks and problems of the system. Saving 60% of the HW of the project, making more than 97% of the transactions stand in the time standards and requirements, and making the system support in hundred of clients instead of less than 200 - all due to good performance and load test project that identified them.

Elad presented the difference a performance and load project can do to a product/project from the client perspective - the difference between working with the system to not working with it at all - money down the drain... and another statistics...

The last speaker, Nachum D, spoke of the business aspects of performance and load. How it is the business driver as well as other functional testing efforts. How we can identify areas of weak performance and load ahead of time, looking for them in the architecture and design documents already - very early in the SDLC.

"The search for the problems from the data collected, is the real challenge of the performance and load engineer", said Dimmer. He presented a few methods for collecting data using tools, and collecting thin data, and important data that will be used for analysis by the performance and load engineer. The aspects presented by Dimmer were for a Web application project, done with a client in the UK.

in short, the meeting was very professional, people asked good questions offline, and the community got a chance to say: we believe performance and load is important, and should be discussed more.

* SIGiST Israel is initiating a working group, to start reviewing performance and load methods and approaches, and will try to produce a white paper to be published on the SIGiST website (www.sigist.org.il)

* SIGiST is presenting a performance and load seminar a the SIGiST Israel 2009 Conference, 26-29 July 2009, with Mieke Gevers - Performance and Load in Action. More details in www.sigist.org.il/minisites/sigist.index.html

Test Management under Crisis

We are in a situation where the market is closing on us fast with a lot of demands on product quality, reducing costs, improving business value. Test managers will have to cope in various ways, and develop new strategies and tactics.

The main challenges we are faced with in my opinion are: managing our processes better, taking calculated risks (based on facts and trends), and managing the human factor better - we are dependant on good people in times like we have more than in regular times.

That can be achieved only via innovative and initiative effort by test managers, gaining knowledge in how to leverage their testing and also to add more value to the business.

Learning new ways to improve our test processes, is a good start. On that we may focus on TPI or TMM or TMMi, CTP or other methods that exist in the market, and are proactive proven.

On risk based testing, and managing our risks better, we should look for risk management knowledge - it is very much related and similar in guidelines and concepts.

The last topic, is maybe the most challenging one - people focus. How to drive good people to have high moral in times like we have, improve their technical abilities, their dedication, so they will be able to push our products quality froward.

There is no magic we can do, but to learn more about the motivation factors, and know more about the future technical challenges of our organization (product wise). Make sure we have been in at least 1 soft skills course, maybe take a consultant/coach to take us through some of the high bumps, and get on our way. It is a lot of trial and error, but errors are difficult since we are dealing with people.

High executives have a better position in making strong demands now, and test managers must come behind those demands, support them, and show what they can do.