Session:Case Studies--Discussions

From WICSA Conference Wiki

Please use this page to capture ideas, thoughts, reactions, opinions and so on about the content of this session.

Attendees

Please add yourself to this list. Tell us something about your background. Add to the Discussions questions section below a few sentences about the session topic such as your position and questions you would like to see discussed.

• Frances Paulisch, Siemens, Session Chair
• Dan Paulish, Siemens Corporate Research (USA), General Chair
• Davide Falessi, PhD student, Univ. of TorVergata Rome, Presenting of "Do Software Architecture Design Methods Fulfil Architects Needs?", [[1]]

Key ideas

(key ideas, how papers relate to each other and support the session theme)

• key idea 1

Discussion questions

• question 1

One common theme between these two papers in the Case Studies section is the topic of "correlation" between important topics in the early phases of the lifecycle. The first paper on the correlation between requirements knowlege/experience and architecture and the second paper on the correlation betwseen task interdependencies and the people realizing those tasks. In both cases people play an important role in this correlation. Ideally, one would want to find hard statistical correlations but to what extent can one do that when people are involved -- despite the "variance" that people bring into the equation, one would like to be able to reach experience-based, experimentally based or even eventually statistically sound conclusions. How can one account for such important variance yet still reach conclusions that are valid for other persons/situations?

Discussions

The issue of “external validity” (the possibility to relate a result to another context) has been well discussed in field of empirical software engineering. In my opinion: 1) the level of external validity strongly depends on the context to be related, hence we cannot establish to a case study a “global external validity mark” however 2) the level of external validity can be objectively improved during the experiment (or case study) design, for example by using heterogeneous and a huge number of people (and objects). 3) The level of external validity is still far away to be automatically evaluated; the reader have to estimate such a level by reading the description of the case study (or experiment) in a natural language and then try to estimate the impact of the differences between the two contexts on the proposed result, hence 4) who wrote the paper should carefully and rigorously describe all the characteristics of the study (objects, subjects, results, etc.) for facilitating the reader to estimate how much such proposed result is valid in his context.