Session:Quality--Discussions
From WICSA Conference Wiki
Papers | Positions | Discussions | Results
Please use this page to capture ideas, thoughts, reactions, opinions and so on about the content of this working session.
Key ideas
(key ideas, how papers relate to each other and support the session theme)
- There is no universal quality model.
Discussion questions
- What constitutes a quality attribute? There are lists (ISO 9126, FURPS) but these lists are not exhaustive. What criteria makes something a quality attribute? For example, is cost a quality attribute, is buildability?
- Not all "ilities" are in the ISO (or other) taxonomies.
- More important than the QA name is having a specific quality attribute scenario.
- How to measure and observe quality attributes?
- This is about the system represented by software architecture, not the software architecture itself.
- What are the different kinds of relationships between quality attributes?
- What are the commanalities and variabilities/differences of quality attributes?
| Commonalities of QAs | Differences of QAs |
|---|---|
| It's a concern to some stakeholder(s). | Quantifiability |
| There are related measures/metrics. | Runtime quality vs. development time quality |
| Has to relate to something (some element or relation of the design), a context. | Locality (traceability) |
| Can be expressed as QA scenarios. Scenarios can capture stakeholders, systems, concern. | Some scenarios are structural, some scenarios are behavioral. |
| They all have variation points (see differences). | Different concepts at different levels of abstractions (same concept has different meanings)--e.g., power. |
| There is a "cost" to achieve it, and there is a value. | Some qualities are better realized at the architectural level and others at the implementation level. |
| All QAs are inter-related (they all relate to cost or function). There is no quality that is completely independent, orthogonal. |
- What does software architecture refactoring mean?
- That should be possible given that we have architectural patterns that promote qualities and applying them represents a refactoring of the design.
- Is there a relationship between structural transformations and quality changes?
- What's the difference between architecture refactoring and code refactoring?
- If one refactors a lot, one doesn't have a stable architecture. On the other hand, successful architectures go through 4, 5, ... different versions.
- Is stability an important quality of the architecture? Stability is the rate of change of the architecture.
- Is architecture stability related to good modifiability?
- What are the implications of stability? Good? Fragil? Dead?
- How to measure software architectures? Are there validated metrics for software architecture?
- This is about the software architecture, not the system represented by the software architecture.
- Is there V&V for software architecture? Who is responsible for it?
- How do we determine the portions of the architecture that determines specific QAs?
- Are there differences in quality attribute considerations in different domains? Does it make sense, for example, to focus on quality attributes for embedded systems, for e-commerce, etc. What benefit does this focus provide?
- How can trade offs among quality attributes be managed during design and during evaluation? In particular, how can the business value of particular aspects of particular quality attributes be ascertained?
- Are there known catalogs or repositories of tradeoffs that apply in general?
Pre-session Discussions
Session Minutes
Software Architecture Reliability Analysis (SARA) method
- Question: on slide 10, how do you derive scenarios from architecture and the taxonomy of fault/error/failure?
- Answer: Failure scenarios are defined at the component level and fault->error->failure chain associated with each scenario is specified. The specification of fault, error and failure taking part in a scenario is done according to their feature diagrams (i.e. taxonomies). So, the taxonomy is used for specifying the scenario together with the scenario template. Architectural components are units of interest (scope) for each scenario. Dependencies between components are also sources for new failure scenarios because a failure of a component turns out to be a fault for another one that depends on it.
- Example: There is a Image Coprocessor (IC) component, which is responsible for processing the video image before it is presented to the user. A failure sceario is that another component responsible for computing the scaling ratio ends up with a wrong result. This result is provided to IC for scaling of the image. So, this is a fault for IC because it will have a wrong scaling ratio value (error). In result, image will be scaled in a wrong way, as it will be perceived by the user (failure). Such a failure scenario is specified as follows:
- Answer: Failure scenarios are defined at the component level and fault->error->failure chain associated with each scenario is specified. The specification of fault, error and failure taking part in a scenario is done according to their feature diagrams (i.e. taxonomies). So, the taxonomy is used for specifying the scenario together with the scenario template. Architectural components are units of interest (scope) for each scenario. Dependencies between components are also sources for new failure scenarios because a failure of a component turns out to be a fault for another one that depends on it.
| FID | CID | Fault | Error | Failure |
|---|---|---|---|---|
| F8 | IC | description: Inaccurate scaling ratio information. source: DDI |
description: Video image can not be scaled appropriately. type: wrong value |
description: Provide distorted video image. type: presentation quality |
- Quantification based on user-perception.
- Failure analysis can be used for:
- Fault forecasting
- Fault removal
- Fault prevention
- Fault tolerance
- This is an enhanced FMEA at design time
Change Propagation for Assessing Design Quality of Software Architectures
- Uses 'change propagation probability matrix'
- Question: on slide 7, how does the CP matrix compare to [DSM]? Answer: DSMs don’t have the probability, but otherwise both matrices have the same structure.
- Question: can the probability be calculated automatically? Answer:Yes, it can. We had a demo in ICSM 2004 in chicago of a prototype tool of SACPT " Software Architecutre Change propoagation tool" as a proof of concept.
Quality-Driven Software Architecture Model Transformaton
- Uses transformations at the same abstraction level (within PIM), special kind of MDA transformation. PIM is considered as implementation language independent architectural model.
- The goal is to provide automation with two knowledge repositories: "stylebase" and "rulebase".
- Stylebase: what information is needed to describe architectural patterns uniformly for transformation? Name, reference, component types (data, computation etc.) and roles (client, controller etc.), connector types, pattern layout etc.
- Rulebase: rules (e.g. how to transform pattern A to pattern B?)
- Master's thesis on the first tool trial: [1]J. Merilinna, A Tool for Quality-Driven Architecture Model Transformation. VTT Publications 561, Espoo, 2005.
- Question: how do these transformations relate to what is usually referred to as "refactorings"?
- Answer:
- Both approaches try to change software structure without changing functionality
- Abstraction level of refactoring is (perhaps?) more close to implementation than in architectural model transformation
- In architectural transformation the functionality may change..
- variability of quality requirements causes indirect variability in functionality e.g. reliability requirement changes, new fault treatment mechanism is added
- functionality of the system remains the same but transformation changes functionality of components.
- .. but the driving force is the change in quality attributes/requirements
Encapsulating Quality Attribute Knowledge
- Precursor: driver-scientist story... is Paulo driver or scientist?
- SEI believes Architecture has significant impact on achieving QA requirements
- Reasoning framework: a vehicle for encapsulating the QA knowledge and the tools needed to analyze behavior of a system w.r.t to some QA
- Projects:
- PACC -- takes as input design,
- ArchE -- an architecture design expert system
- 6 elements
- problem description
- performance e.g., prediction of average latency
- analytic theory
- performance e.g., RMA, queueing theory
- analytic constraints -- RF can only reason about a constrained world
- performance e.g., constrainted inter-arrival rates for aperiodic streams, no I/O ops, components don't suspend themselves, fixed execution time
- problem description
- Projects:
- Current/future work
- RFs developed: performance (RMA, queueing theory), impact analysis, variability analysis, model checking
- RFs in progress: performance using RTQT, security, usability
- RFs implemented as Eclipse plug-ins in ArchE and PectIDE
- More info:
- Q: is there just one RF, or RFs per QA? how is quality encapsulated in the RF?
- The RFs encapsulate knowledge about QA.
- Q: how does the apparently independent RFs interact, if at all?
- Two ways:
- ?
- Deal with each QA individually with a meta QA to deal with the interaction
- Two ways:
Providing Support for Safe Software Architecture Transformations
- TranSAT
- Proposition
- Separation of different decisions during design stage
- Step-by-step integration
- Features
- Model (language?) for specifying transformation
- New reusable piece: software architecture pattern, with the following elements
- new plan
- join point mask
- transformation rule
- Issues
- No way to guarantee the consistency of a pattern
- Question (A. Metzger): Why can this not be guaranteed?
- No way to guarantee the correctness of architecture between steps
- Question (A. Metzger): Can the correctness of the arch. be guaranteed after the final step of a transformation (pattern application)?
- No way to guarantee the consistency of a pattern
- Proposition
- Various verification techniques to check consistency and correctness
- In particular, structural impact of transformation
- Primitive operators of structural transformation is defined, for description and reconfiguration
- For each op, there is defined semantics, simulation in virtual environment, and global evaluation of transformation
- In particular, structural impact of transformation
- Tool: compiler with a rule engine (DROOLS), separates operators semantics and compiler implementation
- Q: already languages proposed for transformation, have these been used?
- No, because the group needed to describe a new language to better constrain the transformation space for consistency verification
- Q: have different kinds of join point masks been considered?
