Make Platform Resources Machine Readable

June 9, 2016

When you build and evolve a managed platform, there are a variety of resources to think and reason about. These could be:

  1. Local dependencies – file system (local, network based mounts)
  2. External dependencies – REST services (proprietary APIs, public APIs)
  3. State stores – data stores (firm-hosted, cloud-hosted stores,…)
  4. Platform services – services that manage the platform functionality (provisioning, runtime management, operational tooling,…)
  5. Managed Resources – these are resources that your platform is managing on behalf of clients. These are the bread-and-butter abstraction over which you typically will have most control
  6. Desired States – this is the list of features that must be made available for regular processing, when platform is being upgraded, when platform is undergoing routine maintenance, etc.

This isn’t an exhaustive list by any means but my intention is to summarize resources a managed platform is made up of. There are numerous benefits in making your platform machine readable. You can now:

  • Think about the assumptions your team is making about non-functional requirements and characteristics of your internal services, dependencies, and managed resources. These would be things like response time, availability, etc. and can help you design and tune integration proxies.
  • Catalogue features that are important for your clients and under what conditions (e.g. ability to handle a provisioned resource, upgrading, etc.). More specifically, define the relationship between features and resource dependencies.
  • Define all these elements in a single, consistent, machine readable definition. This will allow your team to view resources and their state, visualize and report feature dependencies.
  • Design/ implement feature toggling – feature states can be derived using resources
  • Apply self-healing techniques – reset resource (e.g. close and re-initialize a corrupt connection pool, automatically start service instances in the event of a host crashing, etc.)

I will explore each of these points in much more detail in follow up posts. Needless to say, getting your platform in a machine readable state has several benefits.

Systematic Reuse Needs Extensibility

February 3, 2013

Accidental complexity is a real risk with trying to pursue systematic reuse . In trying to get reuse, you don’t want developers to jump through layers of indirection and complexity before they can write any useful code. Accounting for extensibility is a pragmatic approach when striving to design for systematic reuse.

Extensibility needs to be supported at various points in the design – it could be in user interface, integration, data contracts/definitions, system flows, services, and many other points. There are a plethora of choices and the question then becomes – how does one know what needs to be extensible? The answer lies in “finding what varies and encapsulating it”[1].

If a reusable component provides an extensibility hook – the consumer of the component can extend / replace default behaviour. Extensibility can be provided in a variety of ways – there ins’t a magic bullet here and each tactic needs to be considered in context. Here are various examples:

Spring Framework provides application context loader listeners that a developer can use to plugin custom logic. The listener doesn’t allow customization of how Spring constructs the beans or injects dependencies. However, it does provide a convenient mechanism for the developer to provide logic after container startup and before shutdown – place logic specific to a legacy/proprietary API that doesn’t support dependency injection. Alternatively, if there is a specific way to guarantee cleaning up resources prior to destroying context – that logic can be placed here as well.

Another extensibility strategy is from SLF4J – it is a logging facade and provides logging API methods that can be wired with concrete providers such as Log4J. Here, the idea is to free up individual classes from coupling with a specific provider. It provides much more points of extensibility than the above example.

Finally, the nature and number of extension points needs to be carefully weighed in the context of real applications. Not enough extension points hurts reusability. Similarly, too many increases learning curve, complexity, and possibly investment prior to making it usable in an application.

1. Alan Shalloway, James Trott, Design patterns explained: a new perspective on object-oriented design

Prefer Exploration Rather Than Perfection

July 1, 2012

In the zeal to create a reusable component, you should be careful not to over-engineer i.e. pursue perfection. The functional needs have to be at the core of the reusable asset’s ability to solve a particular problem. When in doubt, remember that as time goes on, the team will learn the nuances of the problem domain and will be able to make better design decisions. Explore the domain with the team rather than force a solution. How exactly do we do that? We can:

  • Practice creation of simple, domain-aligned abstractions
  • Continuously validate the domain with functional experts in the team
  • Consciously exclude aspects of the domain that aren’t relevant / well understood when defining reusable components

It is critical for the team to discover and stumble upon reusable assets – rather than big design up front efforts lasting for months. Keep it agile and continuously evolving and improving.

5 Advantages of Using Interfaces for Designing Reusable Assets

April 24, 2010

Interfaces are fundamental to good design and more important when designing resuable assets. They provide several benefits:

  1. They model key behaviors that need to be supported – by designing interfaces, you are forced to partition a big chunk of functionality into a set of behaviors. The amount of detail in each interface varies but the very act to thinking through behaviors will provide a more loosely coupled design.
  2. Provides flexibility to change parts of a module/sub-system at anytime. This is specially relevant with projects where knowledge accumulates over time and domain understanding is unclear in during project inception. Interfaces help switch to a better implementation without adversely impacting the rest of the codebase (assuming, the contract stayed the same or only minimal changes were required).
  3. Explicitly support variability in an asset based on a defined set of behaviors. Interfaces are useful to encapsulate variations in a particular step (using Strategy), how different behaviors are combined (using pipe/filter) , or several steps in an overall algorithm (template method).
  4. With reusable assets, when you provide interfaces as contracts to your consumers – it gives you the freedom to change implementations and not break them. Note: this can be comforting but unless you ensure your consumers don’t instantiate concrete classes and don’t bypass integration-facing contracts there is no guarantee that they won’t break.
  5. Interfaces can potentially out live the implementation. This is related to #2 – as you learn new technologies, new ways to solving problems, interfaces give you the freedom to swap to a better implementation at any time. The interfaces that model domain concepts tend to be more stable as well – core business concepts don’t change overnight. However, business practices do change more frequently and the flexibility to combine and recombine concepts – conveniently captured using interfaces – is very powerful.

In a recently concluded project, I realized these benefits first-hand. The initial implementation of a search service was to go against the database directly. After a couple of iterations, we realized the need for a more effective and efficient solution – an indexed search engine. Even though the search module was used by several consumers the switch to a search engine based implementation was faster, more scalable, and most importantly, didn’t force consumers to make code changes.

Addressing Integration Complexity with Reusable Assets

November 19, 2009

One of the often-cited concerns with leveraging reusable software is design complexity. This is indeed a legitimate concern and as designers, we ought to ensure that it is managed appropriately. In this post, I want to provide some strategies for tackling integration complexity:

  1. Build Iteratively: this is undoubtedly an effective way to avoid over-engineered assets. Building assets iteratively means realizing functionality in small bites, over multiple releases even. Instead of trying to implement a perfect reusable asset, prefer building in increments. This has several benefits: reduced risk, increased relevance for your applications, early feedback on whether the asset has captured domain relationships appropriately, and opportunities to remove code or refactor behavior on a continuous basis.
  2. Capture natural variations in the domain:  reusable assets that don’t reflect the natural variations in the problem domain run into lots of issues. If you keep scratching your head trying to infer what the asset is trying to accomplish – examine the consumer-facing interfaces and ask yourself, does the interface reflect domain variations or is it providing needless variations or worse, ignoring must-have ones?
  3. Prefer convention over configuration: This is one of the foundational principles behind why frameworks such as Ruby on Rails are so popular. You can use this idea and simplify assets in many ways! For example, instead of forcing configuration for files, maybe a standard location would suffice. This idea can be leveraged with scripts that setup developer environments, automated regression tests, and reading/saving program output etc.  There might also be cases where input data is used to determine class instantiation or stylesheet selection. Again, if you come up with a simple convention, many lines of configuration can be eliminated.
  4. Loosely Couple Capabilities: Loosely coupled capabilities are easier to change and integrate. By creating reusable assets in a loosely coupled manner, you will also make it beneficial for consumers. Loose coupling provides another important benefit – making it easy to isolate assets and test them as individual components. If you are building service capabilities, explore the use of asynchronous message publications for real-time notifications to data/status updates.
  5. Strive for consistent naming and behavior: consistent naming reduces learning curve for developers as well as makes it easy for the asset provider to debug, integrate, and test reusable assets. Consistent behavior should go beyond simple method calls – you can extend this to services and business processes as well.
  6. Make Assumptions Explicit: A lot of design complexity can arise due to incorrect assumptions – for instance, there may be operating assumptions about security, data integration, tracking, and error handling.  There are a lot of design assumptions that get made as a natural part of the development process (e.g. every customer will always have a address, or that every customer needs to get authenticated prior to instantiating a business process). Make sure these assumptions are put in the open and for everyone to validate. It often turns out that an asset doesn’t have to implement a feature or that it may be implement an existing feature incorrectly.
  7. Provide consumer-friendly interfaces: Start designing from the consumer’s standpoint and strive for simple yet functionally rich interfaces. This has several benefits: you won’t expose needless internal complexity associated with the asset to the consumer (i.e. achieve right level of encapsulation) and also make it simple for consumers to integrate with the asset. If you have 10 options for a reusable asset but most customers use 2 frequently, why not set the other parameters with sensible defaults? Consumer friendly interfaces also ensure that you build assets that have tangible business value.
  8. Avoid abstractions for the sake of technical elegance: not every abstraction is meaningful, especially with respect to your problem domain. I wrote earlier about the domain-specific nature of variations and why one set of abstractions isn’t always appropriate for your problem. Experiment and iterate to get the right abstractions – they will help establish a shared language within the team and reduce needless complexity because of overly generic interfaces.
  9. Minimize compile-time and runtime dependencies: Reducing the number of moving parts – both in terms of compile time libraries and runtime libraries, services, and systems will make it easier to manage design complexity. Always, ask yourself – is this dependency absolutely essential? Does it introduce a single point of failure in the overall architecture? Is there a way to cache frequently accessed data or return that isn’t 100% up to date?
  10. Provide Mock Interfaces: When possible provide mock data/objects that can help consumers integrate and test assets quickly. This is related to the earlier point about minimizing dependencies but is also useful for customers to get a flavor for the kind of data or business rules that get executed as part of the asset’s functionality. Mocking also helps with another key benefit: asset co-creation. If you are developing in parallel with a consumer, mocking is a great way to agree on interfaces and develop in parallel.

What is your view on these strategies? Can you share some of the ideas/approaches that you have pursued to tackle integration complexity?

Like this post? Subscribe to RSS feed or get blog updates via email.

tweet this add to post to facebook

List of Return Codes for SOA

November 18, 2009

List of Return Codes

I wrote earlier about the idea of using consistent error codes for reusable assets. As a follow up, here is a document with a list of reusable return codes that can be used when building service capabilities as part of SOA initiatives. They are categorized into:

  • request processing
  • data processing
  • dependency access
  • transactions

This isn’t an exhaustive list by any means but can get you started in terms of achieving consistency across services and projects.The service consumer can use the return code to understand the service provider’s response. This consistent, uniform categorization of return codes can help you reuse error handlers (e.g. handle a particular error in the same way regardless of which service capability raised it).  It will also help with production support and troubleshooting – less learning curve for support staff and developers to categorize errors at runtime.

Note: The return code derived based on errors need not necessarily be the return code sent back to the service consumer. It is entirely possible that you return a friendly error to the consumer and a detailed error to production support.

Are there additional ones to include in this list?

Like this post? Subscribe to RSS feed or get blog updates via email.

tweet this add to post to facebook

Design Review Checklist for Service Capabilities

November 9, 2009

Download Checklist

Here is a checklist for performing design reviews when building service capabilities that are part of Service Oriented Architecture (SOA) initiatives. I have found this checklist to be very useful and can serve as a key document as part of the service development lifecyle. This checklist contains close to 50 questions and covers the following areas:

  • Functionality
  • Design decomposition
  • Documentation
  • Coupling
  • Reuse
  • Consistency
  • Integration
  • Performance
  • Reliability
  • Deployment

Feel free to customize this checklist based on your needs.

Like this post? Subscribe to RSS feed or get blog updates via email.

tweet this add to post to facebook

%d bloggers like this: