March 24, 2013
Refactoring is a way to improve code quality over time using incremental set of improvements – the idea is to increase the ability to make changes safer and faster and is not meant to deliver new functionality per se. For systematic reuse to succeed, refactoring has to happen often and on a continuous basis. Why? It provides several benefits from a systematic reuse perspective. For instance:
- You will learn which aspects of the code base have the most technical debt – what’s complicated to understand and extend/enhance? which parts of the codebase are difficult to verify via automated tests?
- See repetition more often and will eliminate redundant methods, classes, even entire chunks of functionality – over time, you will see the same capability being provided by a different library, or there is a new requirement in a project that can reuse the capability if changes were made to it etc. Finally – and this happens to me a lot – you ask yourself – “what was I thinking implementing it a certain way when there is a better approach?”
- Systematic reuse needs deep understanding of the domain – the team needs to tease apart different technical concerns, identify which ones are relevant for the business, and identify variations within the scope of candidate reusable components. Which brings us to the most important question – are the assumptions made earlier about the domain and the subsequent design still valid? This continuous validation and re-validation of the core underlying assumptions and design choices will ultimately decide the reuse effectiveness of the component
- Over time, doing this will provide the team with a valuable data on providing estimates – which parts of the codebase are tricky? which ones lack tests? which ones are bug infested? All these aspects weigh into an estimate and continuous refactoring will give the team very good insights.
So, please don’t wait for a project or a deliverable to arrive – start refactoring every day and across every project :)
February 17, 2013
Want to envision, design, and implement reusable software components that your development community will enthusiastically adopt? Co-create! Here are 5 tips to leverage co-creation when driving systematic reuse initiatives:
- Have an idea that applies to multiple projects? Get the project leads to co-ordinate and align resources and partner with developers from both teams to develop the design
- Share the source code of all your reusable components so every developer in the team can see under the hood how the component works and how it can be improved
- Work hands-on with developers when defining classes, external interfaces, etc. – not just via block diagrams but actual code. Pair program and show them how to think using abstractions, what aspects of the design to make extensible and the rationale for it, etc. – you’ll be surprised how effective this is and how much all parties learn from the process
- Share the big picture – every time and across every project – developers and development managers need to be convinced that their contribution aligns with the overall technical strategy. This should also highlight why a shared component’s test coverage and robustness needs to be high and continuously improved
- Use design reviews, code reviews, and retrospectives to continually look for ways to collaborate and leverage each other. See boiler plate code that can be better encapsulated, or missing tests, or a smarter algorithm – get your hands into the code and work with the concerned developer. They will appreciate why you want them to use a particular design pattern or think about a problem in a certain way. Added bonus – just like item #3 – every participant will learn from the exercise.
Finally, co-creating reusable components greatly reduces friction associated with having to implement a design that was mandated. You want passion and enthusiasm from the dev community – not compliance!
July 1, 2012
In the zeal to create a reusable component, you should be careful not to over-engineer i.e. pursue perfection. The functional needs have to be at the core of the reusable asset’s ability to solve a particular problem. When in doubt, remember that as time goes on, the team will learn the nuances of the problem domain and will be able to make better design decisions. Explore the domain with the team rather than force a solution. How exactly do we do that? We can:
- Practice creation of simple, domain-aligned abstractions
- Continuously validate the domain with functional experts in the team
- Consciously exclude aspects of the domain that aren’t relevant / well understood when defining reusable components
It is critical for the team to discover and stumble upon reusable assets – rather than big design up front efforts lasting for months. Keep it agile and continuously evolving and improving.
April 13, 2011
Wrote earlier about the importance of refactoring and continuous alignment within the context of systematic reuse effectiveness. Reducing technical debt is an integral aspect of refactoring. This post provides tips for reducing technical debt in your software assets:
- Minimize redundant definitions of key domain objects (i.e. competing, conflicting definitions of the same domain object across related applications)
- Minimize similar solutions for slightly varying business processes and instead create common process flows
- Loosen tightly coupled integration/transport logic with business logic
- Provide consistent strategies for implementing cross cutting concerns
- Replace tactical implementation for a problem that has a better open-source alternative
- Eliminate redundant approaches for managing configuration information
- Harmonize multiple, incompatible interfaces and make them more domain relevant
- Minimize excessive coupling with a particular technology/implementation stack
- last but not the least – create a comprehensive suite of automated tests
Are there similar themes in your development efforts? What steps are you taking to ensure technical debt is addressed?
November 26, 2010
Domain analysis is a foundational capability required for effective systematic reuse. Why? There are a lot of applications your teams are working on and the common theme among them most likely is the fact that they are in the same problem domain. In order to truly bring down cost of new applications and services, it is critical that the domain is understood and modeled appropriately. Here are some specific strategies to make this idea operational:
- Account for domain analysis and modeling in your iteration planning. Domain analysis is necessary to understand the nuances and variation points that an application/service/process needs to realize. Discovering the right variations requires time and interactions with business stakeholders and subject matter experts.
- Aspire for a core set of business object definitions that can be shared across business processes and service interfaces. Without appropriate data governance, domain knowledge will either be inaccurate/incomplete or worse duplicated in an inconsistent fashion. As the number of customer interfaces increase for your services, the domain inconsistencies will lead to greater point-to-point integrations and complexity.
- Align overall architecture with domain variations. Your domain is rich and complex but probably varies in a known set of ways. Additionally, what varies isn’t uniform and the rate of change across these variations aren’t identical. This is significant because the variations in the domain need to be captured in your overall architecture. Products/applications in the domain need to share a common architecture – only then can components integrate and inter-operate and systematic reuse will take hold. Constantly evaluate the need for a new version of a core business entity and associated classes to manage the entity.
- Refactor constantly to get to intention revealing design and code. As Eric Evans illustrates in Domain Driven Design, intention revealing code is easier to understand and evolve. It also makes it easier to address new business requirements – as the design/implementation are closely aligned with the business domain, the quality of communication (referred to as ubiquitous language) and the ability to change it both increase significantly.
This isn’t an exhaustive list – instead, it is meant to highlight the need for placing the domain in the middle of every design, architecture, and implementation decision your teams make.
November 6, 2010
Use Maven Assemblies for Packaging Configuration Artifacts
The maven assembly plugin can be used to generate separate artifacts for environment-specific configuration files. This is very useful to separate binaries (jar, war, etc.) from configuration (xml, properties files, etc.). This enables deployment of a binary across any environment. Reusable components are often used in several applications and processes and they often have configuration information. Your clients will want to customize the configuration for testing/altering behavior and it is critical that your build process separates binaries from configuration.
Here’s the snippet for adding the plugin in the POM file:
Here’s an example packaging a zip:
September 5, 2010
Received an email from a reader on the continuous alignment post and wanted additional examples. Practicing continuous alignment entails refactoring existing code to align assets better towards a reuse strategy.
Here are a few scenarios where refactoring opportunities can be utilized:
Horizontal logic mixed with app-specific logic: This is typically manifested in two flavors – cross-cutting functionality that is horizontal across several applications or domain-specific functionality that is applicable to a product line. Cross cutting functionality such as authentication, logging, error handling etc. can be refactored from app-specific to distinct components.
Redundant definition of business objects: maybe two or more of your projects are redefining a core concept from your problem domain and refactoring can help align them to a common definition. Once a common definition is used, add tasks into your iteration plans.
Manage application configuration: If several applications use ad-hoc set of practices to manage runtime configurations (e.g. xml configurations, property files etc.), refactoring them to use a consistent strategy would certainly make sense. This refactoring can be done one component at a time if time constraints make it harder to transform an entire suite of components in the app.
July 13, 2010
Some software assets don’t get reused – question is – is the reason well known? if so, what is being done to ensure that it doesn’t happen too often? In agile methodology, the retrospective serves as a vehicle to reflect on the iteration. Reflecting on the experience overall – what worked, what didn’t work, how can the team improve and so on.
In the same token, consider retrospective for reusable assets – did the asset get used the way it was intended to be used? if not, are new use cases providing opportunities that weren’t considered earlier? In the rush to perfection, there is always a danger is adding too much complexity to an asset. Are consumers even using all that complexity? remember there is a cost to complexity – having the right level of complexity with multiple instances of reuse is a happier place to be.
When you work with the internal development team and asset consumers – ask them specifically why something isn’t working. Is it the lack of adequate documentation and/or code samples? does the asset fail because it didn’t have domain relevance? are there better (cheaper, faster, more maintainable) ways to achieve the asset’s functional objectives? is it too slow and doesn’t meet performance SLA needs? are there too many configuration options? doesn’t work well in a particular runtime environment?
You get the idea: there are specific reasons why your software asset isn’t being reused. Find out why – continuously refactor and resolve root causes to get a higher return on your software investments.
April 24, 2010
Interfaces are fundamental to good design and more important when designing resuable assets. They provide several benefits:
- They model key behaviors that need to be supported - by designing interfaces, you are forced to partition a big chunk of functionality into a set of behaviors. The amount of detail in each interface varies but the very act to thinking through behaviors will provide a more loosely coupled design.
- Provides flexibility to change parts of a module/sub-system at anytime. This is specially relevant with projects where knowledge accumulates over time and domain understanding is unclear in during project inception. Interfaces help switch to a better implementation without adversely impacting the rest of the codebase (assuming, the contract stayed the same or only minimal changes were required).
- Explicitly support variability in an asset based on a defined set of behaviors. Interfaces are useful to encapsulate variations in a particular step (using Strategy), how different behaviors are combined (using pipe/filter) , or several steps in an overall algorithm (template method).
- With reusable assets, when you provide interfaces as contracts to your consumers – it gives you the freedom to change implementations and not break them. Note: this can be comforting but unless you ensure your consumers don’t instantiate concrete classes and don’t bypass integration-facing contracts there is no guarantee that they won’t break.
- Interfaces can potentially out live the implementation. This is related to #2 – as you learn new technologies, new ways to solving problems, interfaces give you the freedom to swap to a better implementation at any time. The interfaces that model domain concepts tend to be more stable as well – core business concepts don’t change overnight. However, business practices do change more frequently and the flexibility to combine and recombine concepts – conveniently captured using interfaces – is very powerful.
In a recently concluded project, I realized these benefits first-hand. The initial implementation of a search service was to go against the database directly. After a couple of iterations, we realized the need for a more effective and efficient solution – an indexed search engine. Even though the search module was used by several consumers the switch to a search engine based implementation was faster, more scalable, and most importantly, didn’t force consumers to make code changes.
April 22, 2010
There are so many reasons why continuously refactoring code is a good idea – in fact, it is a sound investment for the overall health of your codebase. So, what could be some reasons why we fear doing refactorings? what could be the impediments to making changes?
In my experience, the #1 reason why developers fear refactoring is the lack of automated regression tests. This is why Martin Fowler emphasizes the need for tests prior to pursuing refactorings in his seminal book Refactoring. Without a comprehensive suite of tests, developers cannot be confident that changes to the code didn’t break existing functionality and it didn’t introduce new defects.
There are other reasons as well for fearing refactoring:
- Not enough time during development – this is such an often repeated reason for a variety of issues :-) Here is the rub though: we learn most about the domain and the limitations of existing code with experience. This tends to happen at the “end” of an iteration or release cycle. Naturally, we don’t want to jeopardize our release to do refactorings. This is precisely why agile advocates the notion of continuous refactoring iteration after iteration. Your code reflects the state of the software (and the associated knowledge of the domain) at a snapshot in time. It is a work-in-progress – remaining that way till the product ceases to exist or be maintained.
- Lack of disciplined effort to translate new domain knowledge into existing code. As a project proceeds, developers learn more about the domain and the gaps in their knowledge with respect to the domain. These gaps typically manifest themselves in various forms: needless abstractions, insufficient flexibility with known variations in the domain, and lack of domain-relevant concepts being modeled as first-class citizens.
- Insufficient knowledge on using refactoring tools (e.g. using an IDE such as Eclipse, there are a plethora of refactorings can be implemented rapidly).
- Lack of knowledge about refactoring tactics – moving methods, creating interfaces, replacing if..then logic etc. – that are necessary to keep a tidy, effective codebase
- Unwillingness to improve the codebase on a continuous basis :-)