November 8, 2009
In an earlier post, I outlined the four key ingredients for agile software reuse. One of which is the reuse roadmap. In this post I want to share an example roadmap and illustrate how they tie into other business and IT objectives. The idea is to start at the ad-hoc state (where there is little or no reuse) and move to truly utilizing systematic reuse across multiple projects.
Phase 1-Ad-hoc - In this initial phase there is little or no awareness of reusable assets, and there is very little customer integration expertise. Consumers have low awareness of what assets are available. Systematic reuse efforts not considered and there is a lack of alignment between project deliverables, reuse strategy, and business objectives. Applications within the same domain do not share assets i.e. there is a lot of duplicate code.
View Reuse Roadmap
Phase 2-Aware - In the second phase, legacy capabilities and processes are mined for building assets. Agile practices are minimally aligned to the reuse strategy. Consumers seek reusable assets for applications in the domain and rudimentary product lines take shape. Communication about reusable assets have improved and awareness is increasing.
Phase 3-Ready – In this phase, asset inventory expands task, entity, and utility capabilities. User stories are analyzed within a product line context and additional assets are built iteratively. Agile practices specifically code reviews, refactoring, iteration planning, and pair programming used to identify and refine assets in a continuous fashion. Consumers increasingly use reusable assets to build applications and start to co-create assets. Additionally, assets are used across multiple applications in the domain. A few business process flows start to leverage reusable assets.
Phase 4-Systematic – This is when your systematic reuse program starts to really reap rewards. Asset inventory expands to include additional capabilities in your domain. Assets are used to automate many more business processes and they provide variability mechanisms to meet business needs. User stories and iteration planning are well aligned with reuse strategy which facilitate iterative reusable asset development. Additionally, several assets built by refactoring legacy capabilities and communication about reusable assets are available. Finally, consumers are actively engaged in asset co-creation, retrieval, and consumption and supported by a mature integration function.
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | 
Planning, Reuse, agile | Tagged: agile, legacy, refactoring, roadmap, services, software reuse | 
Permalink
Posted by vijaynarayanan
November 8, 2009
Carlson Wagonlit Travel SOA vision charter states all our internal business units should contribute to delivering services. We should be able to support any external business or technical services. We designed an architectural blueprint for service categorization, to be used for the design of new systems as well as the refactoring of existing systems. The main advantages are:
- For the business decision maker, understanding the business value of a component and its commoditization level makes it easier to make build vs. buy vs. lease decisions, and may expose business opportunities for making a service available for others.
- For the architect, having a good grasp of the different categories assists in classifying existing or new services, as well as helps define the appropriate functionality to include in a particular service in order to promote composition and reuse.
This architectural blueprint for service categorization is presented in the following sections.
Technical/Infrastructure Domain Services
Technical / Infrastructure Services are common facilities that do not add any explicit business value, but rather are part of the required infrastructure for the implementation of any business process in an SOA. They are typically purchased or custom built and centrally managed. Let’s cite for example:
- Communication Services transport messages into, out of, and within the system without being concerned with the content of the messages.
- Utility Services provide generic, application-agnostic services that deal with aspects other than transporting application messages.
- Security Services provide required security-related capabilities like identity, privacy, confidentiality, non-repudiation, etc.
Business Domain Services
We decided to support three hierarchy levels for managing Business Domain services.
- Core Business Process Services tie together the data-centric and action-centric building blocks to implement the business processes of the organization. They are in general stateful services (manages Business Process state).
- Business Services implement the business-level capabilities of the organization. They are in general stateless services and represent the action-centric building blocks (or “atomic verbs”) which make up the organization’s business processes.
- Business Entity Services unlock and surface the business entities in the system. They are in general stateless services. They can be thought of as the data-centric components (“nouns”) of the business process: employee, customer, sales order, and so on.
Application Domain Services
Application Processes implement the specific business-level capabilities or some other action-centric business logic elements (“building blocks”) which are unique to a particular business application context. Those services are not managed by the SOA governance team, with stateless or statefull service support. Most of the time, the portal is consuming the Business domain services. We intend to test information mashup in the future.
About the author
William El Kaim is lead IT Architect at Carlson Wagonlit Travel for Global products. He spent more than 10 years working for consulting companies and client companies on enterprise architecture, solution architecture, SOA and Web services, and Identity and Access Management architecture. He had to implement several enterprise Architecture frameworks, processes, on different tools. Finally, he contributed to OMG ADTF work on UML 2 and Model Driven Architecture, and worked with SEI on software product line. You can follow him on twitter (welkaim) or read its blog (Blog: http://blog.resilient-it.com/).
Leave a Comment » | 
SOA | Tagged: Business Services, services taxonomy, SOA | 
Permalink
Posted by vijaynarayanan
November 7, 2009
In part 1 of this series, the rationale for a data services product line was introduced. In this part, I want to identify specific benefits in taking a service oriented approach to building a data services product line:
Abstraction: Enterprise data entities are complex to assemble and deliver. Core data entities might be physically stored in several repositories often requiring several dozen queries to assemble a customer or product. The enterprise data service needs to abstract the data service consumer from the physical implementation details of multiple sources and source–specific data access logic and semantics. From a consumer perspective, the service is a capability that they can access and interact with as though it is a single record. SOA can provide this abstraction in the form of one or more web services.
Federation: If all the repositories and services for a core enterprise data entity reside in a single department or division federation will be less attractive. But, given mergers and acquisitions as well as the need for getting access to enterprise data from several touch points exist federation is a real business need. The data service can federate across systems or repositories based data category or other criteria. Alternatively, the data service can be part of a composition of services that are used to support a larger enterprise-wide requirement. Either way, SOA provides the necessary machinery to federate data.
Interoperability-Data services need to be interoperable across several computing platforms and technologies. This is a fundamental requirement because the enterprise has several different technologies that aren’t going to suddenly become homogenous. SOA stacks with support for open web-based standards such as SOAP, HTTP, and XML are ideal to realize this need.
Integration-Enterprise data services need to be integrated with external consuming applications via two primary message exchange patterns namely on-demand (request/reply) and event driven (publish/subscribe). The service logic needs to be implemented by integrating across multiple data repositories and applications. The integration glue layer has connectivity components, error handling considerations, data transformations, data cleansing/lookup rules, and various data source specific considerations.
Reuse-Data services are meant to be reusable assets that get leveraged across the enterprise. Services can be reused across multiple physical transports, distribution channels, and business process orchestrations. The real benefit provided by SOA is the ability to reuse a service from a number of different computing platforms via open protocols. The ease of interoperability is a key enabler for service reuse.
Versioning-Multiple versions for services as a whole as well as fine grained service capabilities are essential for the product line. SOA provides mechanisms to host multiple service versions either as co-existing units of logic (e.g. using XML schema namespace-based versioning) or by having an adapter layer that can support multiple service versions (the adapter can translate the prior version request to a newer version and vice versa). This provides service consumers the flexibility to upgrade to newer versions of the service gracefully over time.
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | 
Reuse, SOA | Tagged: data services, Service-oriented architecture, SOA, software reuse | 
Permalink
Posted by vijaynarayanan
November 7, 2009
I want to expand on the idea of a data services product line – specifically data that is critical to several business processes in your firm. Why use SOA to build these reusable services? It makes sense due to several reasons:
The data services product line can include data services across customer, account, product, and document data domains providing several capabilities for several internal and external applications and business processes. The vision for this product line? To create a suite of services that will:
- act as the single point of entry into interacting with core enterprise data
- significantly increase reuse of core data assets across business processes
- decrease time to market for new applications and services
- decrease development, maintenance, and support costs across the data service domains
- be scalable and extensible as business usage and needs varied over time
Customer services, account services, & document services can be considered products in a data services product line.
There are typically several sources of variation within the product line that needed to be effectively managed. Listed below is a subset of these variations:
- Data Format
- Data Structure
- Data Source
- Event Trigger
- Data Visibility
- Error Handling
- Data Translations
- Physical Transports
- Workflow/Human approvals
Have you built data services at your organization?
Like this post? Subscribe to RSS feed or get blog updates via email.
1 Comment | Reuse, SOA | Tagged: business process, data services, enterprise data, MDM, SOA, software reuse | Permalink
Posted by vijaynarayanan
November 6, 2009
In an earlier post, I elaborated on agile principles that facilitate reuse. As a follow up, I want to emphasize how principles of service orientation do the same from a complimentary perspective.
| Principle |
How is this supported? |
Why is this supported? |
| Loose Coupling |
Service logic is built independent of specific product, technology, channel, and transport. Legacy services are wrapped to avoid direct coupling between providers and consumers. Both techniques are performed iteratively using refactoring. Decouple horizontal concerns such as logging, error handling, metrics, security from core logic. |
Lower the coupling, higher the opportunity for service reuse. Promotes agility by making less assumptions about upstream and downstream dependencies. Gives flexibility to service provider for performing continuous refactorings. Reuse horizontal capabilities for non-service components in the codebase. |
| Abstraction |
Code reviews and refactoring used to simplify and consolidate service contracts that aren’t well abstracted. Applies to both legacy and new services in the domain inventory. Abstract common service needs using multiple iterations and user stories. |
Abstraction clarifies domain intent and reduces needless coupling between providers and consumers. It eliminates point to point coupling between legacy services and consumers facilitating graceful migration. |
| Composability |
Refactor both new code and legacy modules within iterations. Identify opportunities for compositions when brainstorming implementation options for user stories. Use design patterns such as template method to design orchestrations. |
Composability is key for high levels of service reuse for implementing task services. |
| Standardized contract |
Code and design reviews will be used to ensure reuse of data types, error codes, schema definitions, and that the contracts don’t expose implementation/technology specific details. If contract changes late in an iteration, add needed refactorings to iteration backlog and fix in a future iteration. A service is not “done,done” unless the contract is reviewed. |
Helps align contracts with logical data models for enterprise data entities. Delivers consistent behavior when a service is reused across physical transports. Helps create loose coupling among legacy interfaces and consumers. |
| Autonomy |
By decoupling legacy modules from being tightly bound increasing autonomy. Host services in a standard runtime environment. |
Autonomous services are easier to reuse both as standalone entities as well as in service compositions. |
| Statelessness |
Reuse strategy used to identify stateless services. Code reviews will identify and remove state that inhibits reuse. |
Stateless services offer higher degree of reuse and are simpler to scale to accommodate increased traffic. |
| Reusability |
Building reusable services that can be mapped to user stories. Using legacy services to implement user stories and code reviews to remove duplication across the service inventory. Iteration planning allocates time for wrapping legacy services, unit testing, and working on refactoring backlog. Release planning is performed to ensure services are built within the context of an overall reuse strategy and a real consumer. Use product line techniques such as to allow service variations to accommodate varying consumer functional and quality of service characteristics. |
Minimize service development without a real consumer. Facilitates iterative and incremental manner for building reusable services. As user stories become better understood and code reviews have been done future iterations are continuously used to refactor the service. |
Do these resonate with how you approach SOA and service orientation while building reusable service capabilities?
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | Reuse | Tagged: abstraction, composition, loose coupling, reusable services, service orientation, SOA, software reuse | Permalink
Posted by vijaynarayanan
November 6, 2009
Tip #22 – Ensure Service Capabilities Stay Effectively Decoupled
The rationale for contract-first services (as opposed to code-first) is to effectively decouple service capabilities from needless implementation-specific, vendor-specific, and data-source specific realizations. You put the initial service capability out and ensured that the contract is decoupled – no legacy system specific details, no database-vendor or technology platform-specific attributes are present and your consumers are happy. So, are you done? Well, no…because we all know that the technology environment, business requirements, regulations, and continuous innovations are the realities of modern development. You have to be careful not to succumb to these pressures and tread towards needless coupling. It is very easy to add one more attribute or element to your service contract and not pay attention to tight coupling being introduced.
As you add new versions, introduce enhancements, and make bug fixes take care not to introduce needless coupling. You can tie in code reviews – focused specifically around service contracts – to ensure this happens as part of your overall governance strategy. I will post detailed examples for this quick tip in a follow up post.
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | Reuse, SOA, agile | Tagged: decoupling, governance, services, software reuse, tips | Permalink
Posted by vijaynarayanan
November 5, 2009
If you want to kill the potential for systematic reuse, all you have to do is think only about your silo and not about anything else. This silo could be many different things and in each case it inhibits reuse as well as the effectiveness of your team as a whole. Here are some specific examples:
- At the enterprise level, silo thinking can introduce very many inefficiencies and redundancies. Silo thinking will optimize solutions for a specific project at the expense of the overall firm. For example, say you have a reusable asset for calculating taxes and a new project re-implements the tax calculation business logic all over again. Or they ignore existing software infrastructure and introduce a new vendor product. This may help a specific project but adds cost and complexity to the enterprise.
- Silo thinking specific to an application means potentially reusable assets are not created and existing reusable assets are not leveraged. Think about the application’s needs to get reference data or connect with external systems or format data per a destination’s specific needs. Is your application using reusable assets? If not, is there an opportunity to create one by building or refactoring?
- Silo thinking specific to a team typically results in lack of reuse across projects that impact multiple teams. Do other teams in your department/division/organization know what reusable capabilities exist? do they know the overall reuse roadmap and how that aligns with business objectives? can they contribute towards growing the asset base?
- Silo thinking specific to a person typically results in solutions that don’t benefit other developers in the team. Maybe you have a neat utility that makes it easier to fetch correlated data across multiple data sources. Or you wrote a neat script that validates system variables or smoke tests key functionality. If they save you time and effort, why not share them with your team? Two immediate benefits will happen: other developers will start saving time as well increasing the team’s overall productivity. You will start getting constructive ideas on making these solutions better.
Have you noticed these? how do you encourage solutions that go beyond a specific silo’s needs?
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | Collaboration, Design, General, Reuse | Tagged: Collaboration, sharing, silo, software reuse | Permalink
Posted by vijaynarayanan
November 4, 2009
Here are a few points to consider while you refactor existing or new software assets for reuse:
- If your team is building reusable assets for multiple applications (as part of a product line) within your control you could use a library approach (a JAR or DLL). How you package and deploy the asset is dependent on the development platform. Similarly, if your team is building reusable assets for external applications outside your control, it is better to with a service oriented approach.
- Deploying reusable assets can be tricky! Spend some time thinking about how this asset will work within your environment. Is it going go be centrally deployed for all consumers? Or will each consumer have a copy of the asset in their application? If there are multiple copies, how will you track usage metrics?
- Make sure you have covered security requirements. Your shop might have special information/privacy standards and even legal ones to comply with. Ask around.
- Large shops might have enterprise standards that you might be forced to comply with. Work with the appropriate folks to avoid roadblocks when you deploy!
- A good rule of thumb is to seek input from your production support team (or your own team if you play that role) for understanding their needs. Think of production support input as part of your refactoring effort. Sure you can get away not doing this but why waste a precious opportunity to build a higher quality product while strengthening the partnership.
Don’t wait till the end of your iteration to consider these issues. You will have higher success with systematic reuse if you work these details out as well.
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | Design, Reuse, agile | Tagged: agile, legacy, refactoring, software reuse | Permalink
Posted by vijaynarayanan
November 3, 2009
There is saying that every speech is actually three speeches – the speech you planned to give, the one you gave, and the one you wish you gave. I think software development is very similar in spirit. Most software professionals want to produce high quality code. Yet,there is a difference between code that they write and what they wish they wrote.
There are a whole host of tactical code changes that creep into an application or product line which will have a negative impact for the long term. By tactical I mean short-sighted changes that corrupts the sanctity of the codebase – maybe it violates encapsulation, breaks the design principles that we meant to be followed, introduces needless coupling, worsens technical debt, and so on.
Why are these tactical changes being made? This question is extremely relevant to systematic reuse – whether you are pursuing reuse via objects, component based development, service orientation, or a combination of these. Here are some of my observations on why this happens:
- Time pressure – the need for meeting deadlines. Often senior developers are fully aware of the right place to add code yet proceed with tactical changes. If technical debt is consciously taken on, this isn’t too bad. But, are these always conscious decisions?
- Unwillingness to question key design assumptions (e.g. not considering ill effects of tight coupling between entities or systems) and not addressing support needs.
- Not considering the need to support multiple concurrent versions. Backward compatibility and client integration needs aren’t considered at development time.
- Lack of knowledge about existing capabilities in the codebase. It could also be inadequate knowledge about planned assets, or the roadmap for application transformation. This could be due to lack of communication and/or the developer being new to a team.
- Thinking about changes within the context of short-term deliverables only. This usually manifests itself in seemingly harmless code changes that mushroom into bigger problems later – adding a method in a convenient place rather than the appropriate one or cutting and pasting a variation of existing functionality.
- Lack of collective code ownership – if it isn’t my code or i didn’t write the first version why take the initiative to make it better?
- Inadequate knowledge of team’s coding and design standards and practices. Where should a new component be added? who will review the code to ensure that it is reusable? how do i let other developers know? how do i document assets?
Do these reasons resonate with your experience? What could be added to this list?
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | Design, General, Reuse | Tagged: deadlines, design assumptions, developers, Planning, software reuse, tactical | Permalink
Posted by vijaynarayanan
November 2, 2009
I wrote earlier about using agile software reuse and the key ingredients that are beneficial when pursuing such a strategy. In this post, I want to summarize how agile practices can be used to build reusable service capabilities.
| Practice |
How is this practice supported? |
Why is this practice supported? |
| User stories |
Examine user stories within the context of the overall reuse strategy. Differentiate functionality for the overall domain vs. for a single application. Match story with known service and identify development and refactoring tasks. |
User stories are the primary means for getting requirements for services. User stories can be used to recognize common functionality across services. It bridges business needs with reusable services. |
| Don’t Repeat Yourself |
Code reviews done to remove duplication across existing and new services. Review code at multiple levels of granularity – source code snippets, classes, services, and across the domain services inventory. Prefer building only domain services. |
Ensure services are reusable and do not have channel/technology/transport coupling.Keeps codebase light by removing unwanted code. Helps leverage internal and/or external solutions for functionality that is outside of your core domain. |
| Refactoring |
Heavily used for two primary purposes: iterative service development and legacy asset leverage. Refactoring used to identify and fix service limitations (functional and technical). Align user stories to known set of service refactorings. Refactoring also used to wrap legacy functionality and loosen coupling among legacy systems. |
Refactoring is the foundational technique to achieve agility and helps deliver functional services iteratively. Refactoring is also necessary since domain knowledge and business strategy gets refined over time. Legacy assets can quickly be put to use as part of the service inventory by refactoring them saving time and money. |
| Done, Done |
Unit and integration test the service, perform necessary refactorings, add suite of tests to the continuous integration process, and document service in the service catalog. |
Ensure reuse-friendly techniques are performed and quality isn’t comprised when adding/updating services to the domain inventory. |
| Release planning |
Only build services when there is a known consumer. Support backward compatibility using service wrappers and co-existence using versioning. Bundle enhancements across multiple services. |
Deliver business value early with service orientation. Maximize service reuse across initiatives by releasing early and often. Reduces burden on development team and allows graceful migration path for consumers. |
| Iteration planning |
Plan iteration scope using prioritized list of user stories. This list will in turn drive new service development and updates to existing ones. Allocate time for service development, decomposing legacy code, perform reviews, refactoring, integrate with continuous builds, as well as documentation. |
Capture assumptions and known refactorings to continuously deliver working software and address limitations over multiple releases. Iterations help build reusable services over time greatly minimizing schedule risk. |
This isn’t an exhaustive list of agile practices that facilitate reuse – consider these as a starting point.
Like this post? Subscribe to RSS feed or get blog updates via email.
Leave a Comment » | Reuse, SOA, agile | Tagged: agile, DRY, Planning, refactoring, software reuse, user stories | Permalink
Posted by vijaynarayanan
