Test Management under Crisis

We are in a situation where the market is closing on us fast with a lot of demands on product quality, reducing costs, improving business value. Test managers will have to cope in various ways, and develop new strategies and tactics.

The main challenges we are faced with in my opinion are: managing our processes better, taking calculated risks (based on facts and trends), and managing the human factor better - we are dependant on good people in times like we have more than in regular times.

That can be achieved only via innovative and initiative effort by test managers, gaining knowledge in how to leverage their testing and also to add more value to the business.

Learning new ways to improve our test processes, is a good start. On that we may focus on TPI or TMM or TMMi, CTP or other methods that exist in the market, and are proactive proven.

On risk based testing, and managing our risks better, we should look for risk management knowledge - it is very much related and similar in guidelines and concepts.

The last topic, is maybe the most challenging one - people focus. How to drive good people to have high moral in times like we have, improve their technical abilities, their dedication, so they will be able to push our products quality froward.

There is no magic we can do, but to learn more about the motivation factors, and know more about the future technical challenges of our organization (product wise). Make sure we have been in at least 1 soft skills course, maybe take a consultant/coach to take us through some of the high bumps, and get on our way. It is a lot of trial and error, but errors are difficult since we are dealing with people.

High executives have a better position in making strong demands now, and test managers must come behind those demands, support them, and show what they can do.

SIGiST Israel Year Conference 2008

Great show again.

For 4 days, SIGiST have put a testing show that exceeded last year by far. With more than 230 participants, 51 companies, 7 tutorials (1-2 days course), the Israeli forum for Testing - SIGiST Israel - have organized the biggest independent testing professional conference in Israel so far.

There were 2 days of professional courses, like performance & load in action, TPI, offshore testing risks and approaches, how to write test cases from requirements phase to end of test design, data base testing, and more. more than 85 people participated!

On the next 2 days, there were 3 parallel tracks on various topics: test management, test design techniques, test automation, test methodology.

Interesting sessions that made a unique appearance and interest in my eyes were of Debi Zylbermann, of Dr. Avi Ofer about metrics, and Dakar Shalom about BPT. you will be able to read about those on www.sigist.org.il in a few days probably.

Another improvement in this conference is that conference participants will be able to download any lecture - as they were audio recorded in sync with slides. Using a Webcast tool by SELA Group, a unique effort was done recording 28 hours of presentations - covering all track sessions. 28 hours of recording were done during the last 2 days, with 22-24 presentations to choose from.

Others, non participants, like SIGiST members and any person interested, will have to pay some amount. I was assured by SELA that SIGiST members will by asked to pay a symbolic amount only for that great service.

Now if you were in the conference, and missed the presentation you heard later on about good things, you can download and hear/see it with speakers voice and Q&A session at the end! What a great improvement for the conference services, and the testing community.

Awards: best presentation award was won by Danny Kovach, expert in Agile/Scrum for his presentation about the role of QA engineer and QA manager in an Agile world. Among the 3 best presentations were also Debi Zylbermann and Michael Stahl.

Sponsors were : QualiSystems, Aqua SW, Microsoft, RadView, iSQI, ITCB, SELA Group. Supporting organizations were: D&H, Testing Experience (magazine), Testing & Finance, Conquest, Gasq.

Report - Security Technical Meeting, At Sogeti, Netherlands

At the Security Technical Meeting of Sogeti in the Netherlands, I presented a topic: "The Main Challenges of Testing Today", and discussed 6 main challenges that needs to be addressed:

Technical Perspective:

1) Adding Value to the Business - we need to measure the right things, and aim with those measurements to fit business and management needs. We need also to speak the language of risk for the risk-takers to understand us, and to get clients and business involved with the testing group earlier in the life cycle.

2) Test Automation Huge systems and systems of systems pose a situation where we cannot cover a lot in our manual testing anymore. Will test automation need a boost? I believe so. What shall be the future of TA - will Model Based Testing catch up? Is TA going to be only a design issue?- Those questions were discussed.

3) Requirements Oriented Testing The testing world of today is moving into being more 'engineering' like profession. In order to do that, we shall be forced into becoming more proficient in requirements creating and analysis. How would that change our: skills? Tools? Estimation and budget? ROI? Systems quality? and maybe... lifestyle?

Profession Perspective:

1) Lack of unity and standard We should be more professional, reading more literature, learning more techniques and methods, adopting methods that have worked well in the field. We should certify and educate out test engineers, and invest in their knowledge.

2) Testing: is it a Profession Yet?We do not agree yet on the testing career path, or on the testing promotion levels both as a manager and as a professional. We should promote the TBOK, We should develop ourselves in: application domain, technical aspects, testing skills, and communication skills. Plus, as multidisciplinary test engineer is the tester of tomorrow, we must learn more domains in the engineering life cycle like: project management, product management, system architecture, etc'.

3) People as the Dominant factor of successful testing projects People have always been the dominant factor for our projects success. Since tomorrow's testers are going to be with much higher skills and knowledge (that is the demand of the market/business/clients), we should invest more in people: moral, education, certification, management - all these have to change in order to keep the knowledge (people...) in our companies. Outsourcing will also have a big weight and a factor in this investment.

Alon Linetzki
Best-Testing

Testing Experience Magazine

I would like to draw your attention to a new magazine for testers.It will be issued 4 per year (for free). Attached you will find a short introduction by José M. Díaz Delgado, Editor of the magazine "testing experience":

Dear ladies and gentlemen,We have for a long time hatched the idea of a magazine covering the daily experiences of testers and providing a platform for test professionals on the topics and trends around their fields of interest have. “testing experience” is intended to be both: a magazine and a platform for discussion.“testing experience” is the challenge of producing a high-quality magazine for professional testers made by and issued for people involved in testing.

In the first edition, we rely on a lot of experienced and well-known authors with attention-grabbing articles like Tom Glib, Patricia McQuaid, Erik van Veenendaal, Mike Smith, Alon Linetzki, Rex Black, Derk-Jan de Grood, Satoshi Masuda etc.

We hope to get the support of the global testing community by asking them to subscribe to the magazine and to make the magazine’s link known to their contacts.Please find the first issue on the link: www.testingexperience.com/testingexperience01_08.pdf If you’d like to get the “testing experience” as printed edition please subscribe on http://www.testingexperience.com/subscribe.php.

Please don’t hesitate to contact us, if you have any recommendation, article to send or just to unsubscribe! "

The first issue of the magazine had more than 13,000 downloads, and I congratulate Jose for his efforts, vision and motivation for creating this magazine. Good luck in the future!

Alon Linetzki
Best-Testing

Challenges of testing Down-scaling of systems

I once came across an unexplainable phenomena. We where in the middle of a POC, very tight schedule, and it was evening Thuresday. We decided to leave the server 'on', and not inject any transactions or events, and come back Saturday evening to continue working on it. When, after only a few hours later (11 or 12 I believe), we found out that the system carshed!

That was the fist time I saw a big system fall so hard, when not dealing with anything, just in idle state. regardless to say, that we have included from that day forward an 'Idle state test' in the regression of every release.

It brings us to the point of asking do we know to test downscaling systems? We always ask how to test up scaling ones, but the downscaling is a big issue as well. Systems are 'used' to high communication and high volume of events, and are exercising daemons, loggers, and other means to make sure everything is 'alive and kicking', but seldom do we see big systems testing trying to simulate small scale traffic.

What other issues are to take into consideration in downscaling testing?
- traffic
- buffers
- log mechanism
- memory
- synchronous things that should happen vs. asynchronous ones
- shooting 'by requests' processes and or quesries and or reports after long time

Add your own commenting this.

Alon Linetzki
Best-Testing

Testers Knowledge Accommulation & Tracking

We are putting too little effort in knowing what knowledge we have, and what knowledge we should have in the future to cope with emerging technologies.

As systems get more complex, and we are facing systems of systems, and as the knowledge we are required to know is growing (demand of the companies as a direct demand from the systems), we are about to witness a knowledge crises with testers if we do not respond. The future tester, as I wrote in one of my presentations to a European conference, is going to be more multidisciplinary, and know more of different topics (i.e. project management, product management, risk management, system architecture and design, etc'). So in order to be that person, we should educate ourselves, and be prepared.

Having said that, you have many courses existing today on all areas. It is a matter of budget and priorities for the the manager where to send his/her team to.

I use a simple matrix to track the knowledge of my team of testers, and it helps me also know where I have an "exposure" or a risk in the future test design, infrastructure administration or anything else I choose to track and direct the training and education efforts to:

The matrix helps me to identify the testers who learn faster, the testers that have little knowledge about something, and the exposure I have in dealing with complex test design and execution. It brings out the best out of people that can learn individually, and let them have further topics of knowledge to study on. They have to present what they do once in a while, and that brings more responsibility into their work, and of course satisfaction.

I can also identify problems using the matrix: personal gaps of knowledge (i.e. Ruth is no expert in any topic), and project gaps of abilities (i.e. no one has done test design more than once on Appl#2).

Professional knowledge is power, and we testers should realize that lack of that knowledge is our failure to face the future of our profession.

Alon Linetzki

Best-Testing

Legitimacy to Test An Application Domain

12 years ago when I have trained a group of testers for one of the banks, a guy stood up in the middle of my testing course, and told me: "Do you have an economics degree?". "No." I have answered, and he continued: "Do you have any formal education in Finance at all?". "No." I have answered, "not formal". Then he said:" Then why should I give you a job testing my financial systems?".

That was a legitimate question, and on the break I have answer that "I do not have a financial degree or any formal finance background, but I have tested such systems (as they have) in two different banks in the last 2 years. That answer satisfied him well.

My point is that we are asked to test complex application domain systems, and sometimes (or many times) without the domain knowledge. Is that legitimate from a company to hire only testers with domain knowledge? or it is not? Can we test well enough, while not possessing the domain knowledge?

Let's take as example. A health care machine like CT, that has an analysis module analyzing the brain slices (pictures) - is that OK to test without being a doctor? I mean the user is a doctor, and we always take the user point of view in our end to end testing.

And, it experience is accounted for, like in my example, than when can we have the first experience?

Life tought me that the simple dynamics is that within a team there is at least one person that knows alot on the application domain, and a few others that learn from him/her. That is the case in most projects I have encountered. Some companies would not hire a tester, but rather hire someone that possesses the skills of the specific applcation domain (i.g. flight engineer, pilot, etc') and train him/her in testing skills.

I guess to summarize the discussion - it depends on the application domain and on the complexity of the system or type of system we must test. The fact that lives are at stake (mission-critic systems) could matter as well for making a decision as a test manager of that company.