Evaluating Existing Assets for Reuse Potential

April 25, 2010

One of the often-repeated reasons for not starting a reuse program is the upfront investment required for building the initial set of reusable assets. This is a legitimate impediment that can be addressed by taking advantage of existing code in your team. When you evaluate existing assets for their reuse potential, there are a lot of factors to consider. Here are a few salient ones:

  • Fit within product line: is the asset of relevance to one or more applications in your domain? You want to invest in assets that encapsulate business functionality that can be utilized in multiple business processes and applications. Similarly, if there are assets with business relevance but in a area that your organization intends to divest out of – you will be less inclined to use them.
  • Coupling: How tightly or loosely coupled are the pieces within an asset? sometimes the asset itself might be mixed in – tightly coupled – with a lot of extraneous code. Business logic might be mixed with data access, presentation, and even specific calls to external systems. Every one of these aspects introduces coupling that will need to be assessed thoroughly.
  • Integration: what is the effort required to integrate the reusable asset with applications? The asset might be very difficult to integrate because of platform specific requirements or because of language needs.
  • Business Assumptions: are there assumptions that the asset makes about business rules or business activities? if so, are these assumptions still relevant? will the rules need externalization/refactoring? Often, business assumptions are lost due to lack of alignment between business concepts and abstractions in code.

In a followup post, I will elaborate on robustness, scalability, deployment, documentation & samples. Without thinking through these aspects it is risky to introduce new assets for reuse. You will quickly discover that your consumers are unforgiving when it comes to asset quality. Additionally, a huge learning curve will drive away potential and existing developers and technical leads from evaluating assets for use with new applications.


5 Advantages of Using Interfaces for Designing Reusable Assets

April 24, 2010

Interfaces are fundamental to good design and more important when designing resuable assets. They provide several benefits:

  1. They model key behaviors that need to be supported - by designing interfaces, you are forced to partition a big chunk of functionality into a set of behaviors. The amount of detail in each interface varies but the very act to thinking through behaviors will provide a more loosely coupled design.
  2. Provides flexibility to change parts of a module/sub-system at anytime. This is specially relevant with projects where knowledge accumulates over time and domain understanding is unclear in during project inception. Interfaces help switch to a better implementation without adversely impacting the rest of the codebase (assuming, the contract stayed the same or only minimal changes were required).
  3. Explicitly support variability in an asset based on a defined set of behaviors. Interfaces are useful to encapsulate variations in a particular step (using Strategy), how different behaviors are combined (using pipe/filter) , or several steps in an overall algorithm (template method).
  4. With reusable assets, when you provide interfaces as contracts to your consumers – it gives you the freedom to change implementations and not break them. Note: this can be comforting but unless you ensure your consumers don’t instantiate concrete classes and don’t bypass integration-facing contracts there is no guarantee that they won’t break.
  5. Interfaces can potentially out live the implementation. This is related to #2 – as you learn new technologies, new ways to solving problems, interfaces give you the freedom to swap to a better implementation at any time. The interfaces that model domain concepts tend to be more stable as well – core business concepts don’t change overnight. However, business practices do change more frequently and the flexibility to combine and recombine concepts – conveniently captured using interfaces – is very powerful.

In a recently concluded project, I realized these benefits first-hand. The initial implementation of a search service was to go against the database directly. After a couple of iterations, we realized the need for a more effective and efficient solution – an indexed search engine. Even though the search module was used by several consumers the switch to a search engine based implementation was faster, more scalable, and most importantly, didn’t force consumers to make code changes.


Why do we fear Continuous Refactoring?

April 22, 2010

There are so many reasons why continuously refactoring code is a good idea – in fact, it is a sound investment for the overall health of your codebase. So, what could be some reasons why we fear doing refactorings? what could be the impediments to making changes?

In my experience, the #1 reason why developers fear refactoring is the lack of automated regression tests. This is why Martin Fowler emphasizes the need for tests prior to pursuing refactorings in his seminal book Refactoring. Without a comprehensive suite of tests, developers cannot be confident that changes to the code didn’t break existing functionality and it didn’t introduce new defects.

There are other reasons as well for fearing refactoring:

  • Not enough time during development – this is such an often repeated reason for a variety of issues :-) Here is the rub though: we learn most about the domain and the limitations of existing code with experience. This tends to happen at the “end” of an iteration or release cycle. Naturally, we don’t want to jeopardize our release to do refactorings. This is precisely why agile advocates the notion of continuous refactoring iteration after iteration. Your code reflects the state of the software (and the associated knowledge of the domain) at a snapshot in time. It is a work-in-progress – remaining that way till the product ceases to exist or be maintained.
  • Lack of disciplined effort to translate new domain knowledge into existing code. As a project proceeds, developers learn more about the domain and the gaps in their knowledge with respect to the domain. These gaps typically manifest themselves in various forms: needless abstractions, insufficient flexibility with known variations in the domain, and lack of domain-relevant concepts being modeled as first-class citizens.
  • Insufficient knowledge on using refactoring tools (e.g. using an IDE such as Eclipse, there are a plethora of refactorings can be implemented rapidly).
  • Lack of knowledge about refactoring tactics – moving methods, creating interfaces, replacing if..then logic etc. – that are necessary to keep a tidy, effective codebase
  • Unwillingness to improve the codebase on a continuous basis :-)

Reuse Core Components in Biz Processes and Services

April 19, 2010

There are a variety of components – that can encapsulate business logic (either simple algorithm/calculations or even complex orchestrations). These components can be invoked from business process orchestrations as well as stateless services.  These reusable components could be business rules integrated as decision services or legacy services wrapped using a more decoupled interface. Business events such as a new account being opened or a new security getting added to a portfolio could trigger a core piece of logic – e.g. get statement preferences – that can be used to fulfill both these needs.

For instance, in the diagram below that two business processes and a stateless service invoke a common component via a request dispatcher (or a router module).

If you tightly couple a piece of logic that is applicable across business processes or service capabilities you can refactor it to create a new resuable component. This is all the more reason why it is a good idea to go through a service mediation layer when leveraging legacy services from business processes. If you decide to reuse the legacy service in a new orchestration it will be straightforward to plugin a new consumer.


Refactor Reusable Assets With Domain Understanding

April 18, 2010

There is always an element of additional complexity when building resuable assets – whether it is because of externalizing configuration, adding additional interfaces/classes to provide flexibility, or introduction of additional layers of abstraction (to name a few). There is also the increased cost of design, testing, and integration with reusable assets. The question isn’t whether reusable assets are more or less complex – a more pragmatic question would be:

is the additional complexity absolutely essential and worth the cost?

This is a straightforward question to answer if you have visibility into what is coming up in the near future. If this isn’t the case – which is “most of the time” – the question becomes a trickier one. One way to address this issue is to introduce domain-relevant abstractions.

Instead of guessing what should and shouldn’t be an abstraction – work with domain experts and uncover true domain-relevant concepts. For example: Let’s say you decide to build a reusable library for providing customer data. Say this is used to populate a financial statement that shows a customer’s net worth (i.e. financial assets). Initially your software might have used the checking account balance to report this value. Why? Because, that was the only kind of account that was used to calculate this value. In the initial version the team might have thought ‘net worth’ was a data attribute – a number that is derived from the checking account balance. Over time, as domain understanding deepens, ‘net worth’ might become – not just a single account’s balance but a whole host of other things (e.g. accounts where the customer is a joint account holder including not only checking accounts but also savings, brokerage, etc.). ‘Net worth’ might still be a single number but it would have evolved to encompass a different meaning than just a single account’s balance. Not to mention business rules that have to be applied based on a plethora of account and client characteristics and quickly this would evolve into something more complex.

At the time of the initial release your team might not have this knowledge – and that is perfectly okay! Key is to evolve the codebase along with your increased understanding of the domain. Will net worth be it’s own class or interface in the initial version? If you don’t have even the preliminary understanding, why introduce needless complexity? wouldn’t it be better to refactor to reuse rather than build for reuse with an imperfect understanding?


Refactoring to Reuse #5

April 17, 2010

#5 – Replace Custom Implementations With Open Source Equivalents

I realize this is a very large category with possibly several dozen examples. In this post, I will highlight this strategy using the Apache Commons IO API. Many applications require the need for accessing file data – reading input business data or even temporary file data is a common need. Typically, you have some class that opens a file, reads data line by line, and either returns a custom java object or a string buffer.

FileReader fr = null;
 BufferedReader reader = null;
 try {
 fr = new FileReader(strFileName);
 reader = new BufferedReader(fr);
 String line = reader.readLine();
 while (line != null) {
 //do something with the line of data...
 line = reader.readLine();
 }
 } catch (IOException e) {
e.printStackTrace();
 } finally {
 try {
 if (reader != null) {
reader.close();
 }
 } catch (IOException e) {
e.printStackTrace();
 }
 }

How many times have you written this kind of logic? :-) When you look to simplify this or want to reduce repetition, consider the Apache Commons IO library. The FileUtils class offers just the method you need for accomplishing the often needed line by line data access to a file. It is trivial to use. For example, see below:

import org.apache.commons.io.FileUtils;
//other lines of code and class declaration omitted...
String strFileName = //path to file...
try {
   File f = new File(strFileName);
   List fileDataLines = FileUtils.readLines(f, "UTF-8");
} catch (IOException ioe) {
 ioe.printStackTrace();
}

That’s it – giving you the same functionality using a far simpler interface. There are several advantages to using these libraries – they are open-source, well documented and tested, and eliminate a lot of redundant code in your application. Travel light and only code stuff that you absolutely must. As I have highlighted before – reuse and agile go hand-in-hand. Less custom code you write, less maintenance for your team.


Checklist for Testing Service Capabilities in your SOA

April 17, 2010
checklist

Download Checklist

Here is a checklist for ensuring that your service capabilities are unit tested effectively.   These questions can come in handy when validating test coverage or when doing code reviews with fellow team members. I have used this checklist extensively in all my SOA development efforts and has helped with improving the quality of the services. The check list covers:

  • functional testing
  • error handling
  • data validation/formatting
  • performance testing
  • data binding/transport interfaces

Feel free to add/customize this checklist based on your team’s unique needs. I hope you find this resource useful!


Execute A Systematic Software Reuse Pilot

April 16, 2010
Before you go organization-wide with the systematic reuse strategy it is surely worthwhile to get a pilot project (or multiple pilots even) going. The objective of the pilot is to test how well your reuse strategy can be applied to a real-world application. In a nutshell, here is what a pilot could be:
  • Get engaged with 1 or 2 projects early in their lifecycle and identify reusable business and technical components based on their business needs. The key is identification – not necessarily implementation/realization. With the agile software reuse approach, we can refactor existing code to reuse or plan known assets as part of iterations on a as-needed basis.
  • Identify folks from the pilot projects who are receptive to reuse and work with them closely. You want reuse evangelists, like-minded developers and technical leads who share the possibilities of reuse. It helps if they are also aware of the pitfalls and challenges with reuse.
  • Collaborate with the project team to design new reusable components and integrate existing ones. Identify components that are : unique to an application, unique to a product line, unique to a domain, and relevant across multiple domains
  • Prioritize assets from above list based on project constraints as well as business need. Implement a subset of these assets and examine how the integration works with multiple assets being utilized by a business process, service capability, or an application.
  • Establish a environment for testing these reusable libraries outside the main application in order to facilitate reuse on subsequent projects. This is key for sustaining your future efforts – provide automated tests and an environment (a sandbox server to begin with will suffice) that allows developers to prototype using reusable assets.

Most likely, with a pilot you will learn a lot – a lot about how developers actually use reusable assets, what challenges come in the way in terms of project deadlines, lack of documentation, and unclear assumptions that the asset makes about the domain and even the operating environment. This exercise would also make it painfully obvious the technical debt various reusable assets are carrying – fixes, refactorings, and enhancements that are needed for reuse to be successful. This would be valuable feedback to use – work with your teams to flush out these issues and get to a working mode that allows for iterative and incremental delivery. Remember – your reusable asset doesn’t have to be perfect only constantly aligned to business needs.


12 Practices to Consider When Pursuing Systematic Software Reuse

April 15, 2010

Systematic reuse is hard because it isn’t a problem that technology alone can solve. It requires a various practices that have to be executed in a coordinated manner across teams and projects.  Here is a starter list of practices to consider for success with systematic reuse:

  • allocating resources including changes to budgeting practices for funding shared capabilities
  • setting up appropriate incentives for developers, technical leads, and management staff
  • creating a roadmap for increasing organizational agility – reduced time to market, increased flexibility, increased quality – across multiple processes and applications.
  • recognizing failure signs and course-correcting – not only for large initiatives but for individual asset development as well. Don’t forget the need to involve your consumers in addressing delivery and integration challenges.
  • utilizing product line management techniques for identifying, managing, capturing, and realizing variability in business capabilities.
  • explore reuse opportunities not only business functionality but also with IT processes such as bug tracking, problem management, incident management etc.
  • align reuse efforts closely with other initiatives such as service oriented architecture, business process management, master data management etc.
  • Provide training to developers, technical leads, and development managers on what is available, what is being planned, and how the resuable asset inventory is managed/evolved. Make your teams buy-into the effort.
  • Look for reuse opportunities throughout the entire development cycle- not just during implementation! You can reuse requirements, analysis patterns, domain models, business processes, as well as documentation.
  • Aligning reusable assets with business goals/objectives. Remember: the rationale for reuse is to reduce costs and increase revenue for your firm.
  • Communicate reusable assets within your immediate team, then your department, before taking it organization-wide. Without a support and maintenance strategy in place, your reuse efforts cannot be sustained.
  • Address Non Functional Requirements (scalability, reliability, etc.) when creating resuable assets.

Does this resonate with your experience? what is missing in this list?


Software Reuse Quick Tip #25

April 14, 2010

Tip #25 – Leverage Code Coverage Tools

Code coverage tools help with defect detection and prevention as part of continuous integration. I emphasized the importance of automated tests with regard to resuable assets – code coverage is no less important.

Code coverage reporting provides a window into the health of your codebase.

  • the depth of your automated tests – what percentage of code is being executed by the tests?
  • are there unused blocks of code – including functions, even classes – that are not executed?
  • they are invaluable in helping detect complex code – are some classes too bulky/ripe for refactoring? Code coverage will make these painfully obvious to the developer.

Code coverage thus not only helps you identify opportunities for better tests it also helps you eliminate dead/bloated code and rip apart code that is too complex. I wrote earlier post on developing reusable assets for use first – I recommend identifying extensions/flexibility on a as-needed basis for reusable asset evolution. Code coverage will point out areas for achieving this objective as well.

Getting started is very simply – ff you are using apache maven2 for instance in the java world, it is extremely simple to set these up and use them by adding a few entires to the pom.xml file. Below is the plugin declaration for Cobertura – for plugin’s full usage instructions go here.

<plugins>

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.3</version>
</plugin>
</plugins>

You can run code coverage by simply executing: mvn cobertura:cobertura from the project root folder- this will generate instrumented java classes, execute automated unit tests, and produce a code and branch coverage report.


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: