Think Hard About Conway’s Law

August 24, 2014

Was reading Adrian Cockcroft’s interview on InfoQ recently – where he talks about MicroServices and DevOps, and whole range of topics. A particular point about changing the team’s culture directly applies to reuse. 

I quote from one of his answers: “…is it Conways law?… what’s the organization…? Basically the law that says the communication structure of an organization will be reflected in the code. So you set up the … you decide what structure you want your code to have and you make the organization look like that;”

I am sure you have been in situations where you wish: the code for a reuse candidate was in a common shared source control repository, the components were adopting compatible technologies, that there was clear separation of concerns, and most important, there was a true culture of sharing and systematic reuse across projects. 

This is so true that it got me thinking – how often do we want something to happen – be it collaboration, co-creation, systematic reuse – and yet we run into organizational roadblocks? Lack of core competencies – too many people solving the same problem in a sub-standard fashion, conflicting priorities, inadequate knowledge sharing of not just code but more fundamentally: key requirements, lack of human / monetary / infrastructure resources, etc.

Instead of negotiating constantly across multiple teams – all looking to meet several success criteria – have you thought of organizing them by the outcomes you seek? If you want shared software assets, you need to foster sharing organically, through the organizational interactions that happen every single day. Who reports to who, who is held accountable for what, – the nuts and bolts of responsibility and accountability have to be thought through for the shared software assets you aspire to create and reuse. 

It’s easy to get a single flash in the pan reuse success story – it’s much harder to institutionalize it across teams and projects. Have you got the responsibilities in the team defined in terms of the shared software assets you envision? More often than not – we tend to look for magic bullets in terms of tools, technologies, trends, fads, and yet – systematic reuse is fundamentally about the delicate dance of interactions, trade-offs, and decisions that development teams have to make in the midst of competitive pressures and harsh deadlines. Focus on the form and structure and reuse will follow. If you aren’t getting reuse, think hard about Conway’s Law – the state of your software won’t lie – it won’t hesitate to show up the healthy or not so healthy nature of team interactions, incentives, and priorities that are alive and thrive on the ground. 


Tools and Software Infrastructure for Systematic Reuse

August 24, 2012

A lot of readers have asked me to provide details on what tools and software infrastructure is needed to enable systematic reuse. That is a very good question and it is a critical enabler for building on early wins with reuse.  At a minimum you need to setup the following as your asset base grows:

Module Structure: how are the reusable components organized in your codebase? For starters, setup a separate multi-module build for segregating resuable components from application (or consumer) code. The reusable components should be versioned, managed, and governed separately using a proper release cycle outside of the application code. Each component will also need a comprehensive set of automated tests.

Dependency Management: Use ant/ivy or maven if you are using java for building reusable assets. Each module should have the absolute minimum list of internal and 3rd party library dependencies. Each module should make sure that dependency scope is marked appropriately – take care to mark dependencies in test, runtime, and provided scope as required. Finally, ensure the build is using a clean set of dependencies – remove conflicting/redundant dependencies, and remove dependencies that will conflict with consumer code if possible as well.

Code Quality: Reusable components have robust code quality tooling in place. Use checkstyle, FindBugs, and clover/cobertura etc. with the build script. If you are using maven, you can declare these as plugins and generate code quality reports alongside your build. If you want an integrated code quality dashboard and web based reporting, setup Sonar!

Continuous Integration: Automated tests and continuous integration are extremely critical to continuously refactor, enhance, and deploy resuable components. Setup a continuous integration server that kicks of automated builds on commits as well as when dependencies get updated. Setup a CI server such as Hudson to run unit, integration, and regression tests

Asset Repository: Once you build components, there needs to be a binary repository to host them and integrate them into other builds. Artifactory repo (provides open source and commerical versions) and can be used to house your reusable components (in fact any component really). Once the web GUI is setup, you can browse and search for artifacts as well as refer to them from internal build configurations.

Issue Tracking: Setup bugzilla or your favorite bug and issue tracking software so teams within your organization can submit bug fixes and enhancement requests. You will also need to publish and manage a release calendar to provide transparency into which reusable components are getting released when and how are they getting delivered in the context of real business deliverables.

What additional tools and practices do your teams follow?

Getting Started With Reuse By Harvesting Existing Code

August 23, 2012

Many teams want to incrementally build a portfolio of reusable components so multiple projects can leverage them. This post will provide a list of potential areas to look for achieving systematic reuse – chances are that you have a few components in these areas already. Assembling them into a standard set and making it available for easy consumption will dramatically increase adoption rates across projects. Here are the areas to invest and/or harvest existing code:

  • Configuration – specifically properties that vary across environments (e.g. DEV, QA, PROD). How are teams defining and managing this information today? If you are using Maven,  teams may be employing multiple strategies – profiles, assemblies, custom scripts, naming conventions, etc.  Why not consolidate practices?
  • Transformation APIs – many projects require data transformation between objects to JSON or objects to XML and vice versa. Teams might be using different transformation libraries, may be wiring them with Spring or other dependency injection frameworks in a bespoke manner. Why not adopt a consistent API that provides transformations via a common set of classes with pluggable format-specific providers?
  • Flexible Query Construction – lookup methods that require one or more parameters will end up having common boiler plate code that binds parameters using boolean conditions – why not provide a reusable API that allows the developer to wire parameters using a common API?
  • Integration with internal and/or external systems – retrying, alerting, message construction, metrics, and proactive monitoring are all cross cutting concerns that are common across projects. Teams must be realizing these capabilities using multiple APIs, injection hooks, and response loops – again a consistent and reusable API will not only cut development costs but also make application support simpler

Existing code and components do however carry several risks that have to be addressed to realize reuse potential. For example:

  1. How domain agnostic are the components? Does it have a well defined interface and ability to adapt to a family of use cases?
  2. What is the learning curve for developer adoption? Does it have well defined adoption patterns that are easy to learn and intuitive?
  3. How testable is the component? Does it have interfaces that capture and model varying behavior ?
  4. What is the support model for the component? How to submit bug fix or enhancement requests? is there enough expertise in the developer community?
  5. What will be the release cycle for reusable components ? This becomes critical as more projects start to leverage a common suite of components for a varying set of capabilities

Governance Enables Service Reuse – New Podcast Episode

December 27, 2011
Want to listen using iTunes?

Got iTunes?


New episode added to the Software Reuse Podcast Series on service governance covering design, implementation, testing, and provisioning and how they enable reuse.

Like this post? Subscribe to RSS feed or get blog updates via email.

5 Reasons for Building Tactical Services

October 7, 2010

Tactical services introduce higher than necessary coupling between service providers and consumers and have brittle contracts that forcing service implementation changes on consumers. They do not reuse standard schemas and datatypes increasing data transformation and integration costs and could tightly couple service business logic and transport-specific logic.

In short, tactical services inhibit reusability, increase maintenance costs and reduce the overall effectiveness of service oriented architecture (SOA) efforts.

So, why do teams end up with tactical services? Here are five reasons:

  1. Lack of time to design proper service interfaces – service contracts are rushed to clients exposing needless internal details, introducing redundant business object definitions, and providing inconsistent behavior
  2. Lack of a conceptual data model – if there isn’t a conceptual data model, capturing key domain concepts and their relationships – it is natural that multiple service operations start to define concepts in their unique way.
  3. Insufficient coordination between teams building service capabilities within the domain – when teams don’t talk to each other, many opportunities to reuse schemas, service semantics, behavior, and utilities are lost. As the number of teams increase, there is a greater need for service governance and alignment across projects.
  4. Lack of coherent strategy tying business process management, business events, and messaging within the context of service development. Business processes could be service enabled and standard business schemas can be used to notify interested consumers. However,without an overall strategy – teams will look at these independently thereby increasing implementation costs and missing opportunities for greater alignment.
  5. Insufficient technical leadership – when confronting multiple projects that are either occurring within a short time window or back to back, it is critical to demonstrate leadership. Why? there needs to be strong voice evangelizing use of business facing services, loosely coupled interfaces, and mediating service requests.

5 Signs Indicating Need for Service Governance

August 21, 2010

The word ‘governance’ seems to conjure up all sorts of negative images for IT folks – needless bureaucracy seems to top that list. However, without lightweight governance, SOA and systematic reuse efforts will fail to achieve their full potential. Can you spot signs that indicate need for governance? I believe so and here are five:

  1. Every project seems to reinvent business abstractions that are fundamental to your problem domain. No sharing or consistency of information models – this will be painfully evident when projects are happening back to back and your teams seem to be running into overlapping data modeling, data definition, and data validation issues.
  2. Directly linked to above – is service definitions seem to not reuse schemas – i.e. each service has a unique schema definition for a customer or product (or some key object from your domain) and your service consumers are forced to deal with multiple conflicting schemas.
  3. Legacy capabilities are leveraged as-is without mediation – increasing coupling between consumers and legacy systems. Tell tale sign here is if you see arcane naming and needless legacy data attributes sprinkled all over service interfaces.
  4. Services seem to have inconsistent runtime characteristics – new service capabilities are added without regard to performance requirements – issues tend to manifest in production where users or entire processes/applications get impacted to service behavior.
  5. Business processes bypass a service layer and directly access underlying data stores – if you have seen a business process invoking several stored procedures, doing FTP, publishing to multiple messaging destinations – all from a monolithic process definition that is a clear sign that governance is non existent.

These are few signs but key indicators that services are being built in a tactical fashion. In a follow up post, will expand on how governance can be leveraged appropriately to address these issues.

Think Capabilities Not Technologies

August 14, 2010

There is a reason why your team builds software – most likely, you are either building a product for external customers or supporting technology solutions for internal stakeholders within the organization. Systematic reuse cannot be achieved if we focus exclusively on technologies. Focus on capabilities, more specifically, business capabilities instead.

This is a simple but significantly useful perspective to adopt. Capabilities are tangible units of functionality – regardless of implementation technologies used or systems that they are part of. When capabilities are identified, you can make more informed decisions about whether or not it is a reuse candidate. This is also handy every time there is a discussion on which part of a functionality should reside in a server component or a presentation component etc. Capabilities, when aligned with business needs, gives you the right level of abstraction when considering refactoring efforts. Are we refactoring the right capabilities from a legacy codebase? is this a capability that is useful as-is or is it only relevant within the context of a specific business process? is the current implementation assuming (aka coupling) too much about a business capability?

These are extremely relevant questions – and notice how we haven’t talked about technology implementation decisions. That is on purpose – if we are not careful, impatient technology choices could adversely constrain business capabilities. Once you are clear on the capabilities – technology decisions and need for variability, agility, and other quality attributes can be made appropriately.

Getting Mindshare for Reusable Assets

August 1, 2010

Success with systematic reuse is part-technology, part-process, and part-communication. In fact, a LOT of communication. A critical factor influencing reuse effectiveness is the mindshare that you enjoy with the developer community. That community might be your immediate team or a department, or even the organization. Regardless of the scope, the following are relevant issues to think and plan for:

Awareness: are developers aware what reusable assets can and cannot do?  Can they understand the overall strategy behind the reuse effort? are they aware of specific assets, role for reuse in the development process, and the benefits?

Client experiences: think about the good and bad experiences. Every interaction matters, so does every medium – whether through mailing lists, bug reports, or phone calls to the reuse team.

Asset Quality: is the asset behaving the way it is supposed to? is it robust? does it allow for variability? Is it architecturally consistent with other assets? How does the potential consumer know about the quality?

Reputation: how strong are reusable assets referred to during design and implementation tasks? can clients trust the reuse team’s deliverables?

Documentation: are assets well documented? Is the document written with the reader in mind? Does it make sense – e.g. is it logically organized for a developer to follow and leverage?

Team member Behaviors: do they care about client projects like their own? Are they approachable by developers from various external teams? Ensure there is genuine empathy for business aligned projects – just like in the outside world – if you care enough and deliver, more reusable assets will be utilized in the future.

Emotional Connection: do reusable assets enhance learning and provide a sense of accomplishment for the developer/tech lead? Co-create assets whenever feasible.

Ease of use: how are assets set up? are they easily configurable? are bootstrap code generated for developers? Take care to highlight ease of integration and the multiple variability mechanisms to your audience.

This isn’t an exhaustive list but the intent was to provide some pointers when thinking about the overall reuse strategy and how that strategy needs to translate in the day to day actions that you pursue.

Evaluating Existing Assets for Reuse Potential

April 25, 2010

One of the often-repeated reasons for not starting a reuse program is the upfront investment required for building the initial set of reusable assets. This is a legitimate impediment that can be addressed by taking advantage of existing code in your team. When you evaluate existing assets for their reuse potential, there are a lot of factors to consider. Here are a few salient ones:

  • Fit within product line: is the asset of relevance to one or more applications in your domain? You want to invest in assets that encapsulate business functionality that can be utilized in multiple business processes and applications. Similarly, if there are assets with business relevance but in a area that your organization intends to divest out of – you will be less inclined to use them.
  • Coupling: How tightly or loosely coupled are the pieces within an asset? sometimes the asset itself might be mixed in – tightly coupled – with a lot of extraneous code. Business logic might be mixed with data access, presentation, and even specific calls to external systems. Every one of these aspects introduces coupling that will need to be assessed thoroughly.
  • Integration: what is the effort required to integrate the reusable asset with applications? The asset might be very difficult to integrate because of platform specific requirements or because of language needs.
  • Business Assumptions: are there assumptions that the asset makes about business rules or business activities? if so, are these assumptions still relevant? will the rules need externalization/refactoring? Often, business assumptions are lost due to lack of alignment between business concepts and abstractions in code.

In a followup post, I will elaborate on robustness, scalability, deployment, documentation & samples. Without thinking through these aspects it is risky to introduce new assets for reuse. You will quickly discover that your consumers are unforgiving when it comes to asset quality. Additionally, a huge learning curve will drive away potential and existing developers and technical leads from evaluating assets for use with new applications.

Execute A Systematic Software Reuse Pilot

April 16, 2010
Before you go organization-wide with the systematic reuse strategy it is surely worthwhile to get a pilot project (or multiple pilots even) going. The objective of the pilot is to test how well your reuse strategy can be applied to a real-world application. In a nutshell, here is what a pilot could be:
  • Get engaged with 1 or 2 projects early in their lifecycle and identify reusable business and technical components based on their business needs. The key is identification – not necessarily implementation/realization. With the agile software reuse approach, we can refactor existing code to reuse or plan known assets as part of iterations on a as-needed basis.
  • Identify folks from the pilot projects who are receptive to reuse and work with them closely. You want reuse evangelists, like-minded developers and technical leads who share the possibilities of reuse. It helps if they are also aware of the pitfalls and challenges with reuse.
  • Collaborate with the project team to design new reusable components and integrate existing ones. Identify components that are : unique to an application, unique to a product line, unique to a domain, and relevant across multiple domains
  • Prioritize assets from above list based on project constraints as well as business need. Implement a subset of these assets and examine how the integration works with multiple assets being utilized by a business process, service capability, or an application.
  • Establish a environment for testing these reusable libraries outside the main application in order to facilitate reuse on subsequent projects. This is key for sustaining your future efforts – provide automated tests and an environment (a sandbox server to begin with will suffice) that allows developers to prototype using reusable assets.

Most likely, with a pilot you will learn a lot – a lot about how developers actually use reusable assets, what challenges come in the way in terms of project deadlines, lack of documentation, and unclear assumptions that the asset makes about the domain and even the operating environment. This exercise would also make it painfully obvious the technical debt various reusable assets are carrying – fixes, refactorings, and enhancements that are needed for reuse to be successful. This would be valuable feedback to use – work with your teams to flush out these issues and get to a working mode that allows for iterative and incremental delivery. Remember – your reusable asset doesn’t have to be perfect only constantly aligned to business needs.

%d bloggers like this: