June 1, 2016
How many times have you heard someone say – “we want to implement this once so we can reuse it over and over again…” – or some variation of this theme? The underlying assumption here is that it is better to get to the right implementation of a component so the team doesn’t have to touch it again. Let’s make it perfect, is the reasoning.
I have rarely seen this work in practice. In fact, it is very difficult to create a single perfect software implementation. After all, your team’s understanding of the nuances and subtleties of your domain grows with time and experience. That experience is earned using a combination of trying out abstractions, continuously validating functional assumptions, and ensuring that your software implementation is providing the right hooks to model and accommodate variations.
Instead of trying for perfection, focus on continuous alignment between your domain and the software abstractions. Instead of trying to write once and reuse many times, focus instead on anticipating change and continuous validation of requirements and associated assumptions. Instead of pursuing the one right implementation, enable easy pluggability of behavior and back it up with a robust set of automated tests. This way, you can ensure your team’s domain understanding is reflected appropriately in the software implementation.
You won’t write once – specially if your team lacks hard-won experience to create high quality abstractions. Embrace the idea that you will write something multiple times – not because it is desirable, more because it is inevitable. Deliver value to your business iteratively and deepen your understanding of both the problem and solution spaces. You will be pleasantly surprised with the results. Remember, pursuing reuse without continuous value is the wrong goal.
May 31, 2016
I wrote earlier about easing integration when providing reusable software assets. In this post, will elaborate on specific techniques for driving platform adoption & their rationale. Recognize that most developers who will evaluate your platform are trying to solve a specific business / functional problem. They want to quickly ascertain if what you are providing is a good fit or not. Instead of convincing them, help them arrive at a decision. Fast. How exactly do you do that? Here are a few ideas:
- Provide details on the kinds of use cases your platform is designed to address. Equally important is to be transparent about the use cases that you don’t support. Not now and never ever will.
- Create developer accelerators – e.g. a Maven Project Archetype or a sample project to try out common functionality
- Identify areas where developers can extend the platform functionality – how will they supply or override new behavior? How will you make it possible to inject, easy to test, and safe to execute? There are lots of techniques that you can use but first you have to decide to what extent you want to allow this in the first place.
- Make your platform available in “localhost” mode – i.e. conducive for use with the IDE toolset. This is more challenging than what it sounds – e.g. if your platform isn’t modular, making it work in local mode will be very challenging. Ditto if your platform relies on external services / connectivity / data stores, etc. that aren’t easily replaceable with in-memory / mock equivalents.
- Allow developers to discover your platform via multiple learning paths. Some might want to explore using a series of Kata lessons that tackle increasingly complex use cases. Others might be looking for answers to a specific problem. You need a user guide, code kata, examples, and more importantly, you need to make them easy to access.
- Identify which areas of the platform adoption curve are the most time consuming and figure out how to reduce if not eliminate them entirely. For instance – does your platform require an elaborate onboarding process? Are there steps that can be deferred till production deployment?
May 29, 2016
Managed platforms are a very effective and pragmatic way to drive systematic reuse across an organization. Managed platforms can provide a number of benefits ranging from simplified developer experience, out of the box productivity components and tooling, and most importantly, a whole host of non-functional concerns being addressed in an integrated fashion. Good managed platforms exhibit some common traits. They:
- Solve a specific problem really, really well
- Are easy to signup, develop in, and use via a developer SDK
- Provide a number of integrated components that address specific pain points
- Are extensible and provide clear and safe injection points via a public, published, well maintained API
- Free the developer from having to procure hardware or orchestrate deployment activities
- Make it easy to report bugs, fixes, and contribute enhancements
The best part of managed platforms? They can dramatically alter the productivity curve for your development teams. Every developer doesn’t have to worry about high availability, horizontal scaling, capacity management, public APIs, version management, backward compatibility, and ongoing care and feeding of core reusable components. The platform provides value and more specifically peace of mind via powerful abstractions.
This isn’t an exhaustive list but given the general push towards cloud based architectures, good platforms will give your teams much more than just reuse! I will elaborate on each of the above traits in follow up posts.
May 28, 2016
Although starting from scratch is simpler when building reusable assets, reality is that you are probably maintaining one or more legacy applications. Refactoring existing legacy assets has several benefits for the team. Here are a few:
- The refactoring effort will make you more knowledgeable about what you already own
- Will help you utilize these assets for making your systematic reuse strategy successful
- Saves valuable time and effort with upfront domain analysis (of course this is assuming what you own is relevant to your present domain)
- Make your legacy system less intimidating and more transparent.
- Provides the opportunity to iteratively make the legacy assets consistent with your new code
If you cannot readily identify which legacy module or process is reusable you have two places to get help – your customer and your internal subject matter experts. Your customer can help you with clarifying the role of a legacy process. Likewise, your team probably has members who understand the legacy system and have deep knowledge of the domain. Both of them can guide your refactoring efforts.
The act of examining a legacy module or process also has several benefits. You can understand the asset’s place in the overall system. The existing quality of documentation around this asset and usage patterns can be understood as well. Now, you can make an informed opinion of the current state of what you have and how you want to change it. Before making any changes though it helps to consider the next few moves ahead of you. Ask a few questions:
- Is the capability only available in the legacy application and not in any other system that you active maintain and develop?
- Is it available only to a particular user group, channel, or geography?
- Is the capability critical to your business sponsor or customer? If so, are they happy with the existing behavior?
- How is the capability consumed currently? Is it invoked as a service or via a batch process?
- How decoupled is the legacy capability from other modules in the application?
These questions will help you get clarity on the role of the legacy asset, its place in the overall application, and a high level sense of the effort involved in refactoring it to suit your requirements
January 6, 2015
Tip #32 – Fail the build when there is a test failure
Systematic reuse needs automated tests – lots of them. Ensure that your continuous builds don’t keep running when they encounter a failing test – this makes mistakes visible faster and easier to fix. If you are using Maven, you can use the surefire plugin configuration as shown below:
January 4, 2015
It has been a busy fortnight and have started reading a number of books:
…plus excited to take How to Learn: Powerful mental tools to help you master tough subjects via Coursera.
January 4, 2015
Wrote earlier about why interfaces are important and in this post want to elaborate on their advantages for building reusable services. Service interfaces contain only the operation or method definitions and have no implementations. They can be used in a variety of ways:
- Package service interfaces into a separate artifact to make it easy for client teams to integrate with services without pulling in bulky set of transitive dependencies
- Bind the interfaces to one or more transport / integration technology via standard Dependency Injection (DI). For example, service interfaces can be integrated with a REST-ful Resource or a EMS listener.
- Service interfaces can be backed by stub and/or mock implementation for automated unit and regression testing.
- Service interfaces can be decorated with common cross-cutting concerns to separate them from the implementation. This is the strategy implemented via the Java Dynamic Proxy example.
- Service interfaces can be implemented as a proxy to a remote implementation. For example, the client invokes the functionality via the interface but the runtime implementation makes a call to a server side API. This is useful if your teams need the flexibility to swap local / remote implementations depending on performance / dependency management related requirements
January 3, 2015
A lot of teams are building services for clients both internal and external to your organization. Typically, there is quite a bit of focus on succeeding from a functional sense – did we get the key requirements addressed? does it cover the plethora of rules across markets / jurisdictions? and so on. Some of the more experienced teams, consider the non-functional aspects as well – e.g. logging, auditing, exception handling, metrics, etc. and I talked about the value of service mediation for addressing these in an earlier post.
There is an expanded set of capabilities that are also necessary when addressing non-functional requirements – those that are very relevant specially when your service grows in popularity and usage. They fall under two categories: operational agility and fault-tolerance. Here are a few candidate capabilities in both these categories:
Operational Agility / Supportability:
- Ability to enable / disable both services and operations within a service
- Ability to provision additional service instances based on demand (elastic scaling)
- Maintenance APIs for managing resources (reset connection pool, individual connections, clear cached data, etc.)
- Ability to view Service APIs that are breaching and ones that are the risk of breaching operational SLAs
- Model and detect out of band behavior with respect to resource consumption, transaction volumes, usage trends during a time period etc.
- Failing fast when there is no point executing operations partially
- Ability to detect denial of service attacks from malicious clients
- Ability to gracefully handle unexpected usage spikes via load shedding, re-balancing, deferring non-critical tasks, etc.
- Detecting failures that impact individual operations as well as services as a whole
- Dealing with unavailable downstream dependencies
- Leveraging time outs when integrating with one or more dependencies
- Automatically recovering from component failures
In future posts, I will expand on each of these topics covering both design and implementation strategies. It is also important to point out that both these aspects are heavily interconnected and influence each other.
January 2, 2015
Systematic reuse takes conscious, disciplined effort – question is – where are the systematic reuse opportunities? how can we maximize these opportunities? – it may or may not surprise you that there is rarely a special, designated ‘project’ to achieve reuse or build a reusable API. Teams are busy and they want to solve tangible problems – they work on projects more often than reusable libraries and services. Some reuse advocates lament this situtation – they wish the organization supports systematic reuse with a protected budget, appropriate team members, and the organizational mandate to enforce reuse APIs and standards. I used to wish this too and I was wrong – way off the mark 🙂
Now for the good news – the great thing about the lack of a reuse budget is that, you can focus on the more important thing: achieving business objectives either by saving costs, creating new revenue lines, and reducing uncessary risk. Projects have the necessary business objectives baked into them – that’s why they are funded, resourced, executed, and tracked. Finally, they have an important constraint: time. Projects have deadlines. Schedule risk is a key one with reuse efforts and having a project deadline ensures the asset is going to be useful and relevant to the project at hand. Below are a few tips to get the maximum out of projects:
- Review the requirements – whether it is a set of tickets, a sprint plan, or a formal document – review and categorize requirements into ones that are specific to the project, common to the product line, and common across projects.
- Ensure you are engaged with the development team throughout the project lifecycle – reviewing and identifying opportunities for identifying, leveraging, and refactoring code. Very often, reviews are done long after development is complete with an impending deadline. This leaves little room for introspection, refactoring, or improvements to the codebase.
- Identify existing components and services that are potential reuse candidates – most importantly, identify assets that are readily reusable (even if they are part of an existing project codebase), ones that need minor refactoring, and ones that need substantial refactoring / development. Minor refactorings and enhancements which can be done in parallel while users are testing / verifying functionality is one of many possibilities.
- Insist on development teams using agreed upon interfaces for reuse-eligible functionality. Dev teams can either create a bespoke implementation, fork off an existing implementation, or leverage a reusable API. However, none of this is possible if the projects use bespoke classes / APIs. Interfaces give your team the freedom and pluggability to swap implementations, evolve asset maturity, and ultimately carve code off the project to seed and grow a reusable API. It doesn’t matter if the interface is implemented by a local component, a remote service, with or without persistence, etc. – the implementation will depend on performance and resiliency characteristics in addition to functional needs.
September 7, 2014
Systematic reuse initiatives don’t have to be big-bang events preceded by a lot of noise. It can be done quietly – project by project with a resolute focus on getting targeted wins. As I’ve blogged before, the key is discipline – not technology. The most fundamental question to ask your teams – do they have the basics in place? Specifically:
- How do they go from requirements to design – is there a set of known patterns and frameworks that will anchor the design? If so, ensure every project is aware of these and when appropriate leverages them in the design.
- Are there trust issues with the software being produced by the sister teams? Before you dismiss this as a ‘soft’ issue – remember, your developers and development leads are human and need healthy social relationships at work before they let others influence them. Influence translates to systematic reuse – not occasional but project after project.
- What happens in cases where a project develops a lot of potentially reusable code – who knows about their existence outside the immediate development team? who is going to be accountable for appropriate teams to leverage this work? If you don’t know the answers – don’t be surprised that your software solutions are siloed. No getting around Conway’s Law
- Do you send project updates and accomplishments like many development teams? Most of the time, the target audience is management and the intended message is to gloat how successful the delivery was. Celebrate and reward your teams but also take the time to reflect on two additional themes: did we best utilize the organization’s existing software assets – that includes prior requirements, component libraries, frameworks, services, and patterns? and did we contribute back to the organization’s repository of shared software assets? These aren’t tough questions to ask but you will be surprised by the answers!
- What are the biggest roadblocks to sharing software assets? Don’t assume it’s communication or organization structure or code quality or learning curve or integration ease (or all of them!). Go to the scene of action – spend time pair programming with developers to empathize their circumstances. Watch them struggle to get something to play well in their IDE or plain compile or execute with a myriad tweaks. Don’t assume – collect evidence and focus your improvement efforts on making their lives better.