Monday 30 November 2009

SOA Manifesto rambilings part 3 (the principles)

Principles

The principles are high level principles. They do not deal with design or organizational issues but may allude to them. They are part of the manifesto because they act as a guide in how one should approach SERVICE ORIENTATION and how one should apply the principles of SERVICE ORIENTATION. Following these principles will help to ensure that whatever gets delivered provides the necessary short and long term business value to justify its existence.


“Respect the social and power structure of the organization.”

At an enterprise level SERVICE ORIENTATION and a SERVICE ORIENTED ARCHITECTURE provide a more open mechanism for accessing capabilities through their SERVICE CONTRACTS. It makes it easier to reuse existing SERVICES and create new ones in line with business needs. What is very important in looking at this from an enterprise perspective is who owns the SERVICE, who has the budget and how do we cost account for the use of a SERVICE. Respecting the social and power structures of an organization is critical in understanding the economic effect of a SERVICE, how it is used and how it can continue to be used over time. Aligning budgets for SERVICES and aligning the use of SERVICES to some cost accounting mechanism requires that budgets are aligned to the power structures and that the SERVICE is cost accounted for, based on the social and economic power structures within an organization. Getting this wrong can result in SERVICE degradation over time as more people use it and budgets for its subsequent maintenance and operational use fail to reflect it use and importance.


“Recognize that SERVICE ORIENTED ARCHITECTURE ultimately demands change on many levels.”

This is a corollary of the previous principle. Adopting SERVICE ORIENTATION and implementing and using a SERVICE ORIENTED ARCHITECTURE requires alignment through the organization. SERVICES are used, SERVICES change, and SERVICES are composed, all of this can happen across the entire organization. Use happens from all parts of the organization, request to change comes from all parts of the organization and composition of new SERVICES may happen anywhere in the organization and may use SERVICES from other parts of the organization. This is why SERVICE ORIENTED ARCHITECTURE demands change at many levels. The capabilities that SERVICES expose become un-siloed and that makes them enterprise assets and not just assets for one business unit. This democratization of capability requires change on all levels of an organization as users express future desires driven by changing needs and as the implementation of a SERVICE ORIENTED ARCHITECTURE makes flexibility easier to achieve.


“The scope of SERVICE ORIENTED ARCHITECTURE adoption can vary. Keep efforts manageable and within meaningful boundaries.”

This is a principle derived from the Evolutionary value statement. It is far better to aspire to the benefits of a SERVICE ORIENTED ARCHITECTURE but realize them incrementally. Typically one might start with the portalisation (portal-led SOA) of some business unit and SERVICE ENABLE along the way. The additional cost for SERVICE ENABLEMENT is mitigated by a standards based approach to the integration of the underlying capabilities with the needs of the portalisation. What we see in Cognizant is that this approach is complemented by additional concurrent but offset projects that then seek to further SERVICE ENABLE along the way as the portalisation efforts move to another business unit. Portalisation is not the only way to start. One might have specific needs to reuse components or applications simply to make available data and process from several application but to do so in a more flexible way. SERVICE ORIENTED ARCHITECTURE adoption starts by making it manageable within meaningful boundaries and so can be effectively measured in terms of the business value that it yields. Never forget Return on Investment, it pushes and pulls development.


“Products and standards alone will neither give you SERVICE ORIENTED ARCHITECTURE nor apply the SERVICE ORIENTATION paradigm for you.”

Products and standards do not apply SERVICE ORIENTATION to deliver a SERVICE ORIENTED ARCHITECTURE but they may help. It is people who wield them, making sure you have the right skills to get the most out of a SERVICE ORIENTATION approach and get the most out of a SERVICE ORIENTED ARCHITECTURE is a key aspect of any SERVICE ORIENTED ARCHITECTURE program regardless of how it is constructed or the boundaries in which it fits. In Cognizant we specialize in matching the right resources to achieve a positive outcome and the role of the Enterprise Architect becomes a key to ensuring that all of the moving parts of the Software Development Lifecyle (SDLC) work together.


“SERVICE ORIENTED ARCHITECTURE can be realized through a variety of technologies and standards.”

A SERVICE ORIENTED ARCHITECTURE does not have to rely on an ESB, WSDL, BPEL or any other standard or technology. The key aspects of SERVICE ORIENTATION and SERVICE ORIENTED ARCHITECTURE are the descriptions that we call SERVICE CONTRACTS and the decoupling of them from the implementation of their capabilities at both design and runtime. An instance of an SERVICE ORIENTED ARCHITECTURE may use asynchronous messaging and pass documents between SERVICES. An instance may use WSDL invocations with some form of remote procedure call. An instance may use BPEL and it may use business rules. In all cases the de-coupling should be self evident. A yard stick for an instance of a SERVICE ORIENTED ARCHITECTURE is to locate a SERVICE REPOSITORY, because without one, de-coupling invocation on a SERVICE CONTRACT from a SERVICE IMPLEMENTATION becomes impossible to achieve in any uniform way.


“Establish a uniform set of enterprise standards and policies based on industry, de facto, and community standards.”

A bit of motherhood and apple pie but something that the can certainly catch you if you don’t do it. Establishing a uniform set of enterprise standards and policies can reduce the risk of mis-delivery of a solution. Where there are industry standards they often leverage XML as a means of encoding information and it is better to use them than not. It can cut down integration costs and reduce the time it takes to deliver an enterprise information model as well as the processes that manage that information. The enterprise standards should cover all aspects of governance and should support rapid change. The policies should guide security choices, transactions, and service levels. One key relationship between a SERVICE ORIENTED ARCHITECTURE and this principle is that many industry and de-facto standards are seeking to use the same SERVICE ORIENTED approach to describing the processes that underpin standards, this is true in International Standards Organization, the International Swaps and Derivatives Association, Healthcare Level 7 and many others. For those SERIVICES that are outward facing this may mean that the SERVICE CONTRACTS are already defined which can help reduce the cost of external connectivity and provide cross enterprise interoperability.


“Pursue uniformity on the outside while allowing diversity on the inside.”

This is a cry for SERVICE CONTRACTS and process uniformity on the boundaries of the enterprise and diversity as to how you achieve implementation on the inside. SERVICE CONTRACTS mandate a degree of uniformity. We can go further by publishing our external boundaries in terms of the SERVICE CONTRACTS we wish other to see and the ordering rules over those contracts that make up the processes that we wish to make visible. In so doing we ensure a higher degree of interoperability.


“Identify services through collaboration with business and technology stake-holders.”

From a technology platform perspective it is important to identify core TECHNOLOGY SERVICES. These will have the highest reuse rating and you can only do this by understanding the technical requirements from the technology stake-holders. Likewise the only way to gain reuse at the business level for BUSINESS SERVICES is to do it with the business stake-holders. The widest collaboration, across business units will yield the widest reuse, even if you have a single project, it is worth gathering requirements from a wider set of stockholders so that you can plan effective reuse and change management over time. Hence “Recognize that SERVICE ORIENTED ARCHITECTURE ultimately demands change on many levels” and “Identify services through collaboration with business and technology stake-holders” are complementary.


“Maximize SERVICE usage by considering the current and future scope of utilization.”

Wider collaboration for effective SERVICE IDENTIFICATION is part of the process of understanding the usage of both current and future SERVICES and their utilization. Don’t just look at the functional aspects because the scalability will be compromised as usage rises. Taking into account the scalability issues mitigates the risk of retrenchment and ultimately failure of large scale enterprise adoption.


“Verify that SERVICES satisfy business requirements and goals.”

This principle may well seem obvious, but it has profound consequences. One of the things that SERVICE ORIENTATION and SERVICE ORIENTED ARCHITECTURE instances provide is a clean separation of SERVICE CONTRACTS from SERVICE IMPLEMENTATION. It matters not if you use WSDL to do this or SOAML or Abstract BPEL, what is important is that before a SERVICE is constructed (implemented) we can, at worst, manually check that the collection of SERVICE CONTRACTS match the requirements that gave rise to them. If these requirements are themselves derived from higher level business requirements we can check the derivation. And if the higher level business requirements are derived from business goals we can check their alignment too.

When we do this manually, and most good SERVICE ORIENTED ARCHITECTURE practitioner do exactly this, we call it SERVICE ORIENTED ARCHITECTURE GOVERNANCE and it is part of Enterprise Architecture Governance. It is part of the enterprise alignment process that is done manually.

Given that SERVICE CONTRACTS can be properly described in a formal and structured way (i.e. WSDL , SOAML and Abstract BPEL) and given that we can encode the requirements that give rise to these SERVICE CONTRACTS, it would make a lot of sense to get computers to check the SERVICE REQUIREMENTS against SERVICE CONTRACTS and automatically show that the "SERVICES satisfy their business requirements". We are not suggesting at this point that we can do the "satisfiability" test through the different levels of requirements or indeed the business requirements against the business goals - although even this is possible. What we are suggesting is that the immediate requirements (SERVICE REQUIREMENTS) that give rise to the SERVICE CONTRACTS can be automatically checked to show that the SERVICES meet their SERVICE REQUIREMENTS.

This is what Testable Architecture is all about and the wider "satisfiability" is what Savara will deliver over time.


“Evolve services and their organization in response to real use.”

Accept that we live with imperfection but strive to attain it. Accept that things change. In this way we can focus on evolution to ensure that SERVICES meet their business requirements as those same requirements evolve over time. Equally we may choose to re-organize services for the same reasons. We may choose to uncouple a composite or to change it to reflect changes to real use. The best user interfaces are always tested by users and Apple have been very much the masters of getting it right. Doing the same for SERVICES from the top down, the middle out and the bottom up by evaluating their use in real situations will reveal improvements on design, improvements to requirements and manifest improvements with continual alignment to business requirements and goals.


“Separate the different aspects of a system that change at different rates.”

Engineering for change is not easy. It requires an understanding of what is needed today against what might be needed tomorrow. A key metric is how executive management and in turn their immediate management team steer the ship. What are the controls and the points of change that they need to do their job. Understanding industry influences and trends on an enterprise and understanding the vision provide a way of scoping what needs to be made flexible in the future. Understanding the mission and strategic goals provide a scoping of the changes needed in the near term. Having effective change management procedures in place and effective change request processes are imperative to continual improvement.

Another key aspect of rates of change is that processes change infrequently, but they do change, but the decision point, we might think of them as business rules, change far more frequently. Hence outboarding business rules and using a BUSINESS RULES SERVICE is an important architectural pattern which complements outboarding of processes through orchestration and workflow.

Don’t think that what you deliver will not change or is optimal. Let the users use and make suggestions. If you get flexibility engineered in from the start then the cost of suggested change will reduce and the ability of the enterprise to optimize how it does things increases.


“Reduce implicit dependencies and publish all external dependencies to increase robustness and reduce the impact of change.”

When we create COMPOSITE SERVICES, just like when we create views in a database, it is important to understand the SERVICES that they derive from. Without documenting and making these dependencies clear we potentially run into huge change management and governance problems. It should be standard best practice to publish the key dependencies both in terms of the services that may be used and the underlying technology dependencies too. Such dependencies should be documented and published as some form of attachment to services in the SERVICE REPOSITORY.


“At every level of abstraction, organize each service around a cohesive and manageable unit of functionality.”

Good design coalesces concepts into some form of cohesion. This is true for any piece of software that is constructed. SERVICES are no different in this regard and at each level of abstraction be it a BUSINESS SERVICE, a TECHNICAL SERVICE or a DATA SERVICE the principle of cohesion applies. Organizing and making things manageable takes effort and discipline but the rewards are often greater use and reuse because the focus of the SERVICE becomes clear and the concepts coalesce to deliver appropriate functionality for the level of abstraction.


Monday 16 November 2009

SOA Manifesto rambilings part 2 (the value statements)

The value statements reflect a desired prioritization of one value over another. It does not mean that the value that has a lower priority is not important it just means that the value with higher priority is seen as more important. The value statements are there to help prioritize key decisions. For example having a technical strategy with no business value makes little sense. But a technical strategy to support business value makes huge sense. Equally having project specific goals make sense but not at the cost of strategic goals. Should the need arise to support project specific goals over strategic goals then the decisions should be made with our eyes wide open.

Business value over technical strategy”

SO and SOA is not about technology. One may use technology to deliver it. The value is the value that the business identifies or agrees to. In the world of Enterprise Architecture and Solutions Architecture we measure the outcome of deliverables against agreed business expectations. The technical strategy that is employed is created based upon what we need to focus on technically to deliver business value. Many projects fail because they loose sight of the alignment with business imperative. Ensuring alignment to business value is the key to good governance.

“Strategic goals over project-specific benefits”

In the world of effective Enterprise Architecture applied to a business landscape, we seek to understand the vision of the enterprise and we seek to understand the mission statement. The vision is aspirational and forward looking, the mission is actionable and focused on the immediacy of the enterprise. The vision and mission are used to frame strategic goals for the enterprise. The strategic goals might be implemented by one or more programs of work within the IT space of an enterprise. These programs might incorporate many projects. One of the roles of an Enterprise Architect is to ensure that the programs align to the strategic goals and that the projects reflect them. Without the transparency and traceability of strategic goals to the vision and mission, projects cannot expect to deliver against them. Drift is a common concern and when siloed projects drift away from the strategic goals and focus on short term issues at the expense of the bigger picture. The result is a more costly solution as it has to be re-factored and/or re-built sooner than was anticipated because it failed to meet the strategic goals.

“Intrinsic interoperability over custom integration”

The easier and faster it is to compose new capabilities the more cost effective IT becomes in supporting the business and its changing needs. If we always have to place a mediator between services to reuse them then we are falling into the trap of custom integration and what this tells us is that the design of our services does not support intrinsic interoperability. From a principle perspective, moving to a more document centric approach encourages greater interoperability but compromises the ease of use or understanding of a service's capabilities. Probably no where else in designing services is it more important to get the functions and the types of the parameters correct and indeed to be unafraid of re-factoring over time to move towards greater intrinsic interoperability.

“Shared services over specific-purpose implementations”

The granularity of services in an implementation of an instance of a SOA is a common cause for concern. Reuse and therefore the sharing of capabilities through the embodiment of services and their contracts seems to be much harder than anticipated. Several methodologies have been espoused to help. What we all know instinctvely is it is never a good idea to re-develop the wheel. But of course we constantly re-develop the car. Cars are created through the composition of many existing components. The new Beetle makes use of the Golf chassis, but it feels like a very different car. Services as components for the construction of new capabilities is a key principle of SO but doing it is harder that saying it. What is needed is good component repositories with good documentation and reviews. Back in the 1980’s Shlaer and Mellor had the same idea but expressed it as a book of components, like those that electrical/electronic engineers might use, for things we use to create new capabilities. Sometimes a customer implementation is needed but it is not that often and measurements of customer implementation vs reuse should always be made on a regular basis to ensure that the benefits of shared services are being realised.

“Flexibility over optimization”

There is an old software proverb that rings true. Make it work before making it fast. Making things flexible, choosing where the points of flexibility need to be is not easy but by following some simple principles we can at least move much of what is needed outside of code making services configurable. Business rules can be used in place of coded logic, orchestration can be used in place of hardwired configuration. A heuristic approach that might be in a document for service implementers would be to ensure that business rules and processes stated in requirements are exposed to enable them to be changed.

“Evolutionary refinement over pursuit of initial perfection”

The old adage of don’t try to boil the ocean comes to mind. We always seek perfection but we always need to temper it with the realisation that perfection is a focus not a goal. Good software projects and programs are phased. Occasionally we have the big bang approach but they rarely succeed. So we need to be mindful that we cannot do it all in one go and so phase the approach. It is common for many SO projects and programs to be led by the user interface inwards. Service enabling as you go is a better approach than trying to service enable and then figure out how users interact with the service. This is the case because without the users we cannot be sure of the pain point the wishes and desires of those users and so cannot address their concerns. And it is the users (customers) that matter.


Thursday 5 November 2009

SOA Manifesto ramblings and observations PART 1

It has been a couple of week at least since we published the SOA Manifesto. Alas I have rather busy putting some of it into practice. But that is done now. So time to elaborate further.

I plan to break this blog into 3 parts to provide some clarifications, some observations and some personal opinions about the SOA Manifesto. In this blog I want to deal with three specific points in the SOA Manifesto. In the next two blog I will deal with much of the rest of the SOA Manifesto.

The three specific points relate to the following:

"Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation."

"Business value over technical strategy"

"Verify that services satisfy business requirements and goals."
"Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation."
I have, for a very long time, challenged the wisdom of the term Service Oriented Architecture. If you track back over the last 5-6 years the term SOA has been used and misused almost on a daily basis. Vendors have often hijacked the term claiming that they sell SOA.

The term Service Orientation is the antecedent to SOA. You cannot have anything being a SOA if you didn't do some sort of SO, just as you cannot have an Object Oriented Architecture or implementation of one without practicing OO.

Clearly we can use C++ in an non-OO way. We can use C in a non-OO way, but oddly enough we can also use it in an OO-way. X-Windows was a great exemplar of an OOA that was done by using OO within a C based implementation. If we constrain things and use a classic OO language, like Smalltalk, then by definition you end up with an OOA.

Thus the terms OOA and SOA are types of things not instances. There may be many instances of an OOA and many of a SOA. Therefore we can use the phrase "my implementation of a distributed system is an instance of a SOA", and we can also say that "my implementation was created by applying the principles of SO". Hence SO is something you do and an instance of a SOA is something that you might create through the application of SO principles.

Why is this important? Isn't it obvious? For those that are SO practitioners it is obvious, but we are faced with myth that has dogged the rise of SOA. I have many customers who ask how do I sell SOA? It is not something that you sell because you cannot buy it. So we need clarity and common language so that when we engage with customers we do not add to the confusion but rather remove it and provide clarity. So obvious to us practitioners yes, and important YES, so that we can effectively engage with businessES who have need of business benefits that a SO approach can deliver through Service Enablement to provide an instance of a SOA within a business context.

"Business value over technical strategy"
Providing a business context is all important. In the dot-com boom days many projects were technology led and the business benefits unclear. Most if not all failed as a result. Motherhood and apple pie comes in here. It is common sense that we understand the business value prior to any technical strategy. The technical strategy supports the business value. Technical strategy is there to support the vision, mission and strategic goals that a business sets for itself. The technical and application roadmaps are constructed to do exactly this. The funny thing about "common sense" is that it is not common, and that is why we needed to state this value proposition business value over technical strategy.

"Verify that services satisfy business requirements and goals."
This principle may well seem obvious too. But it has profound consequences. One of the things that SO and SOA instances provide is a clean separation of Service Capability (Contract) from Service Implementation. It matters not if you use WSDL to do this or SOAML or Abstract BPEL. The important thing here is that before a service is constructed (implemented) we can, at worst, manually check that the collection of Service Capabilities match the requirements that gave rise to them. If these requirements are themselves derived from higher level business requirements we can check the derivation. And if the higher level business requirements are derived from business goals we can check their alignment too.

When we do this manually, and most good SOA practitioner do exactly this, we call it SOA governance and we call it Architecture Governance (although governance is actually wider than this). It is part of the alignment process that is done manually.

Given that Service Capabilities can be properly described in a formal and structured way (WSDL , SOAML and Abstract BPEL all do this) and given that we can encode the requirements that give rise to these Service Capabilities - I shall call them Service Requirements - it would make a lot of sense to get computers to check service requirements against service capabilities and automatically show that the "service satisfy business requirements". I'm not suggesting at this point that we can do the "satisfiability" test through different levels of requirements or indeed the business requirements against the business goals - although even this is possible. What I am suggesting is that the immediate requirements that give rise to the Service Capabilities can be automatically checked to show that the Service Capabilities meet the Service Requirements.

This oddly enough is what Testable Architecture is all about and the wider "satisfiability" is what Savara will deliver over time.