January 2, 2015
Systematic reuse takes conscious, disciplined effort – question is – where are the systematic reuse opportunities? how can we maximize these opportunities? – it may or may not surprise you that there is rarely a special, designated ‘project’ to achieve reuse or build a reusable API. Teams are busy and they want to solve tangible problems – they work on projects more often than reusable libraries and services. Some reuse advocates lament this situtation – they wish the organization supports systematic reuse with a protected budget, appropriate team members, and the organizational mandate to enforce reuse APIs and standards. I used to wish this too and I was wrong – way off the mark 🙂
Now for the good news – the great thing about the lack of a reuse budget is that, you can focus on the more important thing: achieving business objectives either by saving costs, creating new revenue lines, and reducing uncessary risk. Projects have the necessary business objectives baked into them – that’s why they are funded, resourced, executed, and tracked. Finally, they have an important constraint: time. Projects have deadlines. Schedule risk is a key one with reuse efforts and having a project deadline ensures the asset is going to be useful and relevant to the project at hand. Below are a few tips to get the maximum out of projects:
- Review the requirements – whether it is a set of tickets, a sprint plan, or a formal document – review and categorize requirements into ones that are specific to the project, common to the product line, and common across projects.
- Ensure you are engaged with the development team throughout the project lifecycle – reviewing and identifying opportunities for identifying, leveraging, and refactoring code. Very often, reviews are done long after development is complete with an impending deadline. This leaves little room for introspection, refactoring, or improvements to the codebase.
- Identify existing components and services that are potential reuse candidates – most importantly, identify assets that are readily reusable (even if they are part of an existing project codebase), ones that need minor refactoring, and ones that need substantial refactoring / development. Minor refactorings and enhancements which can be done in parallel while users are testing / verifying functionality is one of many possibilities.
- Insist on development teams using agreed upon interfaces for reuse-eligible functionality. Dev teams can either create a bespoke implementation, fork off an existing implementation, or leverage a reusable API. However, none of this is possible if the projects use bespoke classes / APIs. Interfaces give your team the freedom and pluggability to swap implementations, evolve asset maturity, and ultimately carve code off the project to seed and grow a reusable API. It doesn’t matter if the interface is implemented by a local component, a remote service, with or without persistence, etc. – the implementation will depend on performance and resiliency characteristics in addition to functional needs.
August 1, 2010
Here is a free e-book on getting started with systematic software reuse. It introduces software reuse, risks with reuse and how to address them. Also includes an example roadmap that can be used as a reference. Enjoy!
April 24, 2010
Interfaces are fundamental to good design and more important when designing resuable assets. They provide several benefits:
- They model key behaviors that need to be supported – by designing interfaces, you are forced to partition a big chunk of functionality into a set of behaviors. The amount of detail in each interface varies but the very act to thinking through behaviors will provide a more loosely coupled design.
- Provides flexibility to change parts of a module/sub-system at anytime. This is specially relevant with projects where knowledge accumulates over time and domain understanding is unclear in during project inception. Interfaces help switch to a better implementation without adversely impacting the rest of the codebase (assuming, the contract stayed the same or only minimal changes were required).
- Explicitly support variability in an asset based on a defined set of behaviors. Interfaces are useful to encapsulate variations in a particular step (using Strategy), how different behaviors are combined (using pipe/filter) , or several steps in an overall algorithm (template method).
- With reusable assets, when you provide interfaces as contracts to your consumers – it gives you the freedom to change implementations and not break them. Note: this can be comforting but unless you ensure your consumers don’t instantiate concrete classes and don’t bypass integration-facing contracts there is no guarantee that they won’t break.
- Interfaces can potentially out live the implementation. This is related to #2 – as you learn new technologies, new ways to solving problems, interfaces give you the freedom to swap to a better implementation at any time. The interfaces that model domain concepts tend to be more stable as well – core business concepts don’t change overnight. However, business practices do change more frequently and the flexibility to combine and recombine concepts – conveniently captured using interfaces – is very powerful.
In a recently concluded project, I realized these benefits first-hand. The initial implementation of a search service was to go against the database directly. After a couple of iterations, we realized the need for a more effective and efficient solution – an indexed search engine. Even though the search module was used by several consumers the switch to a search engine based implementation was faster, more scalable, and most importantly, didn’t force consumers to make code changes.
April 16, 2010
Before you go organization-wide with the systematic reuse strategy it is surely worthwhile to get a pilot project (or multiple pilots even) going. The objective of the pilot is to test how well your reuse strategy can be applied to a real-world application. In a nutshell, here is what a pilot could be:
- Get engaged with 1 or 2 projects early in their lifecycle and identify reusable business and technical components based on their business needs. The key is identification – not necessarily implementation/realization. With the agile software reuse approach, we can refactor existing code to reuse or plan known assets as part of iterations on a as-needed basis.
- Identify folks from the pilot projects who are receptive to reuse and work with them closely. You want reuse evangelists, like-minded developers and technical leads who share the possibilities of reuse. It helps if they are also aware of the pitfalls and challenges with reuse.
- Collaborate with the project team to design new reusable components and integrate existing ones. Identify components that are : unique to an application, unique to a product line, unique to a domain, and relevant across multiple domains
- Prioritize assets from above list based on project constraints as well as business need. Implement a subset of these assets and examine how the integration works with multiple assets being utilized by a business process, service capability, or an application.
- Establish a environment for testing these reusable libraries outside the main application in order to facilitate reuse on subsequent projects. This is key for sustaining your future efforts – provide automated tests and an environment (a sandbox server to begin with will suffice) that allows developers to prototype using reusable assets.
Most likely, with a pilot you will learn a lot – a lot about how developers actually use reusable assets, what challenges come in the way in terms of project deadlines, lack of documentation, and unclear assumptions that the asset makes about the domain and even the operating environment. This exercise would also make it painfully obvious the technical debt various reusable assets are carrying – fixes, refactorings, and enhancements that are needed for reuse to be successful. This would be valuable feedback to use – work with your teams to flush out these issues and get to a working mode that allows for iterative and incremental delivery. Remember – your reusable asset doesn’t have to be perfect only constantly aligned to business needs.
April 14, 2010
Here is a brief presentation on adopting agile practices based on an ignite-style talk I gave recently at a local meetup.
April 11, 2010
When building a reusable asset it is natural to anticipate future needs and provide the flexibility for various client usage scenarios. Before you jump in and code every single feature – pause. There is a reason why agile practices recommend iterative design and discourage BDUF (big design up front). With systematic reuse, it is more important to get aligned to multiple projects and applications rather than build the perfect asset. Obviously, if you have a window into your next iteration’s needs or another project’s requirements, you will want to accommodate them. By the same token, if you don’t have a good idea what the future is going to bring – don’t add code and bloat the codebase.
The good news with improving a reusable asset is that it will benefit not only your new client but also existing clients. With that in mind, strive to address the absolute must with a reusable asset. For example, if you are creating a new standardized service to provide customer data – consider the many aspects to vary and facilitate reuse. You can:
- Offer multiple flavors of the service’s data – a set of minimal fields, a “most common” list of fields, and a fully populated flavor that returns all data.
- Offer multiple versions of the different flavors – e.g a getClient_v1_0 and getClient_v1_1
- Vary the output format returning XML, JSON, serialized object, or even RSS feeds
- Provide localized data (formatting, replacing data per a geographic locale).
- Support a variety of transport interfaces – HTTP, JMS, FTP etc.
Given these options above, it is tempting to build that ultra-flexible reusable service meeting all the needs of today and tomorrow. However, projects have deadlines and business imperatives – it is critical that reusable assets don’t jeopardize business objectives. Build a sub-set of the above that is absolutely essential for your immediate deliverable. Be ruthless with code that is unused and unnecessary. Your project might only need to support XML response over HTTP – so build just that and nothing more. You can use design practices, patterns and refactoring techniques to decouple your code appropriately (e.g. separating business logic and formatting logic). This will position you for extending the codebase in the future.
November 19, 2009
One of the often-cited concerns with leveraging reusable software is design complexity. This is indeed a legitimate concern and as designers, we ought to ensure that it is managed appropriately. In this post, I want to provide some strategies for tackling integration complexity:
- Build Iteratively: this is undoubtedly an effective way to avoid over-engineered assets. Building assets iteratively means realizing functionality in small bites, over multiple releases even. Instead of trying to implement a perfect reusable asset, prefer building in increments. This has several benefits: reduced risk, increased relevance for your applications, early feedback on whether the asset has captured domain relationships appropriately, and opportunities to remove code or refactor behavior on a continuous basis.
- Capture natural variations in the domain: reusable assets that don’t reflect the natural variations in the problem domain run into lots of issues. If you keep scratching your head trying to infer what the asset is trying to accomplish – examine the consumer-facing interfaces and ask yourself, does the interface reflect domain variations or is it providing needless variations or worse, ignoring must-have ones?
- Prefer convention over configuration: This is one of the foundational principles behind why frameworks such as Ruby on Rails are so popular. You can use this idea and simplify assets in many ways! For example, instead of forcing configuration for files, maybe a standard location would suffice. This idea can be leveraged with scripts that setup developer environments, automated regression tests, and reading/saving program output etc. There might also be cases where input data is used to determine class instantiation or stylesheet selection. Again, if you come up with a simple convention, many lines of configuration can be eliminated.
- Loosely Couple Capabilities: Loosely coupled capabilities are easier to change and integrate. By creating reusable assets in a loosely coupled manner, you will also make it beneficial for consumers. Loose coupling provides another important benefit – making it easy to isolate assets and test them as individual components. If you are building service capabilities, explore the use of asynchronous message publications for real-time notifications to data/status updates.
- Strive for consistent naming and behavior: consistent naming reduces learning curve for developers as well as makes it easy for the asset provider to debug, integrate, and test reusable assets. Consistent behavior should go beyond simple method calls – you can extend this to services and business processes as well.
- Make Assumptions Explicit: A lot of design complexity can arise due to incorrect assumptions – for instance, there may be operating assumptions about security, data integration, tracking, and error handling. There are a lot of design assumptions that get made as a natural part of the development process (e.g. every customer will always have a address, or that every customer needs to get authenticated prior to instantiating a business process). Make sure these assumptions are put in the open and for everyone to validate. It often turns out that an asset doesn’t have to implement a feature or that it may be implement an existing feature incorrectly.
- Provide consumer-friendly interfaces: Start designing from the consumer’s standpoint and strive for simple yet functionally rich interfaces. This has several benefits: you won’t expose needless internal complexity associated with the asset to the consumer (i.e. achieve right level of encapsulation) and also make it simple for consumers to integrate with the asset. If you have 10 options for a reusable asset but most customers use 2 frequently, why not set the other parameters with sensible defaults? Consumer friendly interfaces also ensure that you build assets that have tangible business value.
- Avoid abstractions for the sake of technical elegance: not every abstraction is meaningful, especially with respect to your problem domain. I wrote earlier about the domain-specific nature of variations and why one set of abstractions isn’t always appropriate for your problem. Experiment and iterate to get the right abstractions – they will help establish a shared language within the team and reduce needless complexity because of overly generic interfaces.
- Minimize compile-time and runtime dependencies: Reducing the number of moving parts – both in terms of compile time libraries and runtime libraries, services, and systems will make it easier to manage design complexity. Always, ask yourself – is this dependency absolutely essential? Does it introduce a single point of failure in the overall architecture? Is there a way to cache frequently accessed data or return that isn’t 100% up to date?
- Provide Mock Interfaces: When possible provide mock data/objects that can help consumers integrate and test assets quickly. This is related to the earlier point about minimizing dependencies but is also useful for customers to get a flavor for the kind of data or business rules that get executed as part of the asset’s functionality. Mocking also helps with another key benefit: asset co-creation. If you are developing in parallel with a consumer, mocking is a great way to agree on interfaces and develop in parallel.
What is your view on these strategies? Can you share some of the ideas/approaches that you have pursued to tackle integration complexity?
Like this post? Subscribe to RSS feed or get blog updates via email.