Session:Rationale--Discussions
From WICSA Conference Wiki
Please use this page to capture ideas, thoughts, reactions, opinions and so on about the content of this working session.
Key ideas
(key ideas, how papers relate to each other and support the session theme)
- In my experience, `Best Practice' is a dangerous concept, which is often misused to avoid having to supply rationale for architectural decisions. I suggest using the concept of `Best Fit Practice' instead, to signify that the practice referred to fits within the specific context, rather than being an `absolute' best. - --Eltjo 09:39, 3 Nov 2005 (EST)
Ground Rules
- Participation is critical to have a useful dialogue
- Discussion & conversation takes precedence over paper deliberation
- The group sets direction
- Everyone is accountable for helping to table tangent conversations
- Let the Group Expectations/Outcomes guide content
Group Interests
- Architecture Decisions are the cornerstone of the CapOne Architecture Process. Looking for new tools, techniques, and processes to incorporate into the CapOne methodology.
Group Workshop Expectations/Outcomes
- Distilling of key challenges in this area
- Prioritization of recommendations to researchers in this area
- Knowledge share amongst architects on best practices with respect to effective decision making
- Capturing the rationale in a usable form
- Two tracks
- Stakeholders
- Capturing rationale
- How to use the rationale for decisions
- Tracing design decisions. What are the ramifications?
- Classification/Taxonomy
- Type
- Order
- Inputs/Outputs for decisions
- Relevent stakeholders
- How to arrive at design decisions
- Interplay of requirements and design decisions
Expectations and Outcomes (Move to next section)
- Common techniques to take home
- How to get the tools, techniques, decision process in place
- Validation that we're using best practices
Key Challenges with respect to Architecture Decisions
- Lack of Tool Support for Architecture Decisions
- Lack of an accepted/standard metamodel for Architecture Decisions
- Lack of integration with other architecture representations
- How to order design decisions
- Just in time, other techniques
- People other than architects influence the design decisions. How to deal with this.
- Difficulty in communicating technical issues with other stakeholders
- Satisfying diverse stakeholder interests
- Management unaware of architectural issues
- Challenge is to translate into non-technical rationale
Papers
Presentation Links
Building up and Exploiting Architectural Knowledge
- Design rationale graphs are very complex
- Very good visualizations are necessary (See Tufte)
- Need the 'right' visualization
Q: What spatial aspect are you trying to solve?
A: Space shuttle example
- Use cases for Rationale, decisions, ... presented
- Currently carrying out case studies using the model
Q: How do you spot critical stakeholders
A: No algorithm, but this needs to be done
Q: Are you building a tool to visualize
A: A set of tools, also discover what type of knowledge to store
- A participating company performs audits for government projects
- Need to model their needs in terms of use cases
- Soft goal technique is useful for tradeoff analysis
- Ontology-based browser
Q: Why visualization? Graphical solutions don't scale.
A: Some kind of graphical visualization will also be needed [in addition to other artifacts]
- Tables don't show ordering well
- Can use work flow
- Graphs are interchangeable with other formats and allow you to see relationships -- can see decision dependency
- You don't look at thousands of decisions at once, there is a focus point
- Tables more useful for reporting and manipulation
- You need different kinds of views for this structure
- Graphs and tables overlap yet complement each other
- Technique described for splitting graphs by concerns
- Real challenge is to capture the information
- There is a limit on the amount of information that can be managed
Ontology-Based Architecture
- Extension of previous work
- Documented the practice of managing architecture through architectural decisions
- Decisions are the primary representation
- How do you manage decisions?
- How do you relate them to other architectural assests?
- How does it connect with the implementation?
- Ensure that implementers follow the architecture
- Still several problems with this approach
- Many decisions (hundreds)
- Need focus, precision, clarity, support for impact analysis...
- Difficulty in linking with views
- Solution
- Architecture meta-model
- Focus on important information rather than views ...
- Conceptual meta-models presented (Concerns -> Architecture Decision -> Roadmap, Architecture Decision -> Architecture Asset
- Ontology tool, "Protoge," is used. Free, extensible, plug-in support.
- Process for documenting presented (Populate model with concerns, existing architecture assets, ...)
Comments:
- Temporal issue is important. What does the architecture look like six months from now
- Takes time for the decision to be realized
- Need an automated way to show temporal views
Q: What are your overall objectives
A: Architect works with a model to see how artifacts connect
Q: Is this a recognized problem?
A: Yes.
- Forces architect to derive a decision from a concern.
- Guides the architect.
- Provides a level of precision
- Provides a vocabulary
- Saves time because you can automatically produce views and documents. Change the model and regenerate
- Satisfies stakeholders
- Queries help satisfy requests for information in particular formats
- Views can't easily be defined in advance
- Create just-in-time views
- Framing as a decision resonates well with management
Q: Understood that architecture was a blocks and connector form of documentation. This looks like an overlap with design. More like a framework (tool) that is about the process. Is this understanding correct?
A: Yes, this is rooted in those diciplines. This is an evolution now that we're at a different level of complexity
Q: How do you deal with higher granularity issues' effect on the architecture?
A: Constant re-prioritization is necessary
- Decisions can be driven by the implications of other decisions
Q: Process is similar to those of requirements engineering
A: It is similar, this isn't 'the' model, if you have a tool and understand the information to capture, focus on that. The views are representations of that information.
Q: Architects impact the requirements. Architects make compromises. You can't achieve all requirements...
A: Constraints, impact analysis tie decisions to requirements
Q: What is in the boxes?
- Names and descriptions? Also includes info describing conflicts
A: Yes, the model is annotated with this information
- E.g. this decision is an exception to an enterprise decision
Q: Difficult to detect conficts, inconsistencies
A: This is brain power at this point
Q: How to validate the meta-model?
A: Business practice validates. May have omissions.
- Not far along with automatic generation of views
- Experimenting with this approach as an agile tool
Towards an Operational Framework for Architectural Prototyping
- How to actually make the right decision decisions
- Experimental approaches to doing software architecture
- Build small prototypes, possibly a skeletal system that helps make a design decision
- Characteristics of an architectural prototype
- Constructed for exploration and learning
- The focus is on the construction phase
- Process
- Provide an operational framework for practitioners
- Define terminology and conceptual framework
- Activity-Based Computing (ABC) case study presented
- Captures state of applications, ability to suspend and resume
- Applications for nurses and doctors
- Transfering to another PC involves suspension and resuming on another PC
- State was serialized to an XML format
- An interface must be implemented by all ABC services
- This is tedious for service/application developers
- Results in some poor system qualities
- Proposal: Produce semi-automated state management support
- Isolate the problem in several executables
- Architectural process involved "Harvesting" and "Retrofitting"
- Summary
- Architectural protoyping process (Using executable systems as means for design exploration and evaluation)
- Start of a tentative framework
Q: Is this example from the lab
A: In a lab, ABC is a research project
- A problem with prototypes is that you can't stop it. It is challenging to scope. Need a specific question to answer
Q: How do you scope?
A: Have experience with estimation. Requires 53 hours of development time...
- Tell developers that the project will be thrown away. This avoids unneccesary work.
- In this case there is a problem to solve. If you knew there was a risk, you need to monitor the problem and resolve risks. Feedback loop can help determine when to stop a prototype project. Need different people with different viewpoints to quantify risks.
- Used an XP-stype approach to manage the prototype project.
Q: Does anyone have experience with a more abstract language? (e.g. ____)
A: Rules engine.
- People will build prototypes not in the deployment environment
- If you're solving a physical project, it needs to run in the deployment environment
Successful Architecture for a Short Message Service Center
- Practitioner's report
- A switch for mobile phone text messaging
- Part of the GSM standard for mobile communications
- CMG commissioned for realization in 1995
- Processes close to 50% of all worldwide text messages
- This project was successful 90% due to the architecture
- Treats architecture as a set of design decisions
- SMSC = Short Message Service Center
- Presented overview diagram of how SMSC relates to other components
- Simple main requirements: Pass messages, temporary storage
- Key design decisions
- Platform choice (IT platform vs. Telecom switch platform)
- Telecom lacked sufficient flexibility
- Selected OpenVMS instead of UNIX (Which was selected by competitors)
- Had experince with OpenVMS, UNIX streams thought to be less suitable
- Storage strategy
- RAM and proprietory file io vs. RDBMS
- Competitors selected RDBMS
- CMG thought RDBMS had too much overhead, built a proprietary system
- Interprocess communication
- Proprietary "lean" IPC vs. COTS
- Felt that there was too much functionality in COTS, developed their own
- Platform choice (IT platform vs. Telecom switch platform)
- Performance and reliability turned out to be the deciding success factors
- Beware of fashion in system design
- Often there is pressure from management to follow trends and fashions
- CBAM from SEI is useful for objectifying design decisions
- Term "best practice" often misused to avoid rationale. Use "best fit practice" instead
Q: Were three key decisions made explicitly or implicitly?
A: There were meetings between managers and architects where these decisions were made. It was known at the time that the decisions were made. They appear in the meeting minutes. The importance of the quality attributes were not known until later. Prioritize decision making by irreversability.
Q: Were the decisions unanimous, was it a long proces?
A: Made during the bid process
Q: If you expected a low quantity of messages, why was performance chosen as a quality attribute?
A: Requirements
- About one of the systems that this system replaced:
- A prototype was hacked together and ended up being sold (SMS system)
- Messages start to trickle in
- Huge problems
- An organizational unit attempts to improve, but not replace.
- It was scalable but expensive to scale?
- Business was so good that they never found the time to re-architect
- Finally, the demand was too high for the system
Q: Did you architect for a one node or multiple node solution?
A: Architected for a two box. Proprietary IPC helped here.
Q: Did you identify the critical decisions in hindsight?
A: Yes. Maybe.
- There were some irrational aspects taken into account. One of the very best programmers needed to work at home for personal reasons. Decided to make a project for him to do by himself. This became the proprietary IPC!
Q: Is this a matter of luck?
A: There was an element of luck. But you do need good architects with the right intuitions. Need managers that listen to the architect.
Q: How did the prospect of buying the technology back affect the decision making?
A: Had the luck that they didn't have to invest.
- After selling the system to 10 or 11 countries, the same management decided that Europe was saturated and reduced the size of the team by a third. Took years to repair this mistake.
Explicit Model
- Web services exist in a dynamic environment
- Unpredictability
- Partial solution: interfaces, ..
- Problem: Handling failures
- Presented an example diagram
- Discovery
- Bind
- Failure
- Reconfigure
- Approach: Monitoring and Management system
- Propose a generic and reusable method for configuring a system
- Keep the choices (design decisions) available at runtime
- This is a limited set of design decisions
- Runtime representation of design space
- Describe how the choices relate to qualities
- Open Questions
- How to populate the variability model
- Position: We need to conserve explicit runtime design decisions in our architecture
Q: Example of choices?
A: Which specific services to use. Need to describe which types of services are suitable (UDDI). Parallel vs. Sequential.
Q: What are the qualities?
A: Performance, availability, fault tolerance
- Recovery, roll-back procedures
Q: Assumes multiple instantiations of a service that provide the same behavior at different levels of quality.
A: Yes. Online photo services is an example.
- Service needs to become a commodity
Q: Individual services would address their own faults and provide better reliability than can be achieved externally
A: It may not be the case that a service is as reliable as claimed
Q: Predictable?
A: Should build the model partially at design time. Put in as much knowledge as possible at design time. Discover or detect at runtime. This is a tradeoff.
Q: How does this relate to rules engine?
A: This is an abstraction of the problem
Q: What if the decision is wrong?
A: Need to monitor, compare, evaluate whether the system is performing to determine when to reconfigure.
The Amigo Service Architecture for the Open Networked Home Environment
- Networked Home Environment
- Domains: Personal computing, mobile computing, consumer electronics, home automation
- Challenges: Dynamic configuration, resource constraints, heterogeneity
- Amigo: Ambient intelligence in the home by integrating the above four domains
- Objective: Interoperability
- Paradigm: Service-oriented architecture
- Drawback: imposes middleware on all devices
- Interaction between services is based on syntactic descriptions
- Want to enable interoperability between existing heterogeneous SOAs?
- Using ontologies, semantics of architectural entities are machine readable
- Several steps described explaining the interoperable SOA
- Currently working on
- A language for semantic services specification
- Interoperability mechanisms
Q: Related to Philips?
A: Yes, Philips is coordinator of the Amigo project. But this is a differnent approach because it retains existing technologies and architectures
Q: Examples of usage scenarios
A: Yes, movie follows you in the home is an example. But working on design now.
Q: How is this different from other groups' work?
A: The semantics-based interoperability approach. Model semantics of the whole architecture (All three layers: application, platform, middleware).
Q: Example?
A: Make web services interoperable with RMI. Two devices using different protocols can discover eachother. At the application layer, a printer can be found by a device looking for a "printing service." Requires common vocabulary.
Q: What are some of the key decisions you made?
A: This is at an abstract level of architecture. Currently deciding which technologies to use for prototypes.
Q: Who makes those decisions?
A: Presenter is in charge of the architecture. Process: Each partner proposes their technologies and we find the common ground. The presenter is responsible but decisions are made democratically.
A Dynamic Service Architecture for Scientific Computing
- Sensor network
- Set of conflicting requirements
- Took less than an hour to produce an architecture
- Had seen solutions to the problems before
- Need a catalog of best principles
- Can we package experiences from design decisions?
- Possible, Useful, what would it look like?
- Design patterns are distillation of design decisions
- Have been working on this for 10-15 years in software design
- Catalog the decisions to be made. Make a matrix that shows the patterns to look at.
- Disagreement on how design decisions relate with architecture decisions
- Architecture design decisions are much more heterogeneous
- Grady Booch's approach of gathering a collection of architecture is helpful
- By the time the decision is expressed to stakeholders, the organization needs to make a decision but architecturally, several designs have already been considered.
- There are more influences for architecture decisions than in software design
An Architecture and its Rationale
- Air traffic control system (CAATS)
- Wanted distributed OO system
- Problem: Avoid increase in application complexity due to the distributed nature
- Two fundamental decisions
- Object discovery
- Object interaction
- Interaction - Queries
- Uses time steps during which the universe is frozen to avoid inconsistent data
- Interaction - Updates
- Need to reconcile the decisions of agents operating on a "Flight." This is delegated to the server.
- Application code is ignorant of distribution issues
- Discovery
- Sometimes know only the properties of the desired object
- Sometimes a key value is known
- A catalog is maintained
- Publish/Subscribe-like mechanism also used. E.g. Allows computation of all flights in an airspace
- This architecture may appear in Grady Booch's book.
Q: Size
A: 50-150 workstations, 4 servers
- Disagreement over whether the desired properties can be achieved without a distributed system
Discussion questions
- What are some thoughts from the key ideas presented? To jump-start your session see the link above on Positions on the Use of Architecture Decisions.
- What tool support is available for capturing architecture decisions?
- There is a perceived lack of methodology for capturing/making architecture decisions? Is this true?
- Is there a tendency to only capture "positive rationale"?
- Is there a tendency to focus on concrete (FR, NFRs, constraints, assumptions) rationale versus comparative (cost, benefits, tradeoffs)?
- What are the factors that influence the use of documentation of design rationale?
- Under what situations would the use and documentation of design rationale provide a postive return on investment?
- How to provide direct benefits to the architect for capturig design rationale?
Pre-session Discussions
Session Minutes
Session minutes for the first session appear in the Papers section
Taxonomy Discussion
Ways to classify/prioritize
- qualities
- root cause
- risk
- reversability
- effect/implications
Q: What was interesting about the root cause chart? A: How do you find root causes and how much effort it takes, what are the requirements for automated support
- Bad decisions are rooted in requirements, due to shortage of time, poor prioritization.
- A lot of performance problems are due to poor coding techniques. Can be difficult to separate architectural performance and scalability issues from code issues.
- Decisions have several states: e.g. pilot/prototype, irreversable
- Has anyone tried classifying by effect (implication)?
- When there is a lot of uncertainty and risk, there is a need to begin prototyping
- Take inputs into a decision and quantify risks:
- Implementation
- Whether the decision meets its objective
- Cause and effect is a traceability problem
- There is a distinction between traceability and classification.
- Need to know the implications of the decision -- more important than the causes that lead to this decision
- Logging the situation under which you made a decision and whether the decision was right,
