SOA Manifesto rambilings part 4 (what else?)

The Manifesto does not tackle any relationship between PROCESS ORIENTATION and SERVICE ORIENTATION. In this blog I'll tackle this link and explain why they go hand in hand. The only reason this was not done in the context of the Manifesto was time and focus.

So here is my take on why it is in fact important and perhaps one of the most important aspects to SERVICE ORIENTATION and SERVICE ORIENTATED ARCHITECTURE.

PROCESS ORIENTATION is a paradigm that constrains a solution to focus on the underlying flows between components where a component maybe a person or an agent for a person (e.g. a portal) or a software component (which maybe embodied by a SERVICE CONTRACT).

BUSINESS PROCESS MANAGEMENT (BPM) is the management of PROCESSES that are formulated by applying PROCESS ORIENTATION. A BUSINESS PROCESS MANAGEMENT SYSTEM is a software platform that supports the construction of PROCESS DESCRIPTIONS, their simulation, execution and life-cycle management.

ORCHESTRATION (as embodied by WS-BPEL) is a mechanism for describing, simulating and executing COMPOSITE SERVICES which are “orchestrated” by means of a some flow between them to deliver a new SERVICE.

Today BPM and ORCHESTRATION are common bed-fellows. Many companies use the former to deliver the person based workflows and the latter for composing complementary SERVICES, which may then be consumed by a BPM layer. In the future these two bed-fellows will merge with the introduction of WS-HumanTask and Bpel4People. These two related standards will deliver the necessary descriptions to enable both BPM and ORCHESTRATION to be merged.

When we think of BPM and SERVICE ORIENTED ARCHITECTURE we need to understand where things are today and where they may be tomorrow and deal with the present and plan for the future. Typically a BPM SERVICE ORIENTED ARCHITECTURE approach focuses on the encoding of PROCESSES at the business level so that legacy applications involved in the PROCESS are SERVICE ENABLED along the way. The reason to do the SERVICE ENABLEMENT is that it is a cheaper and faster route to leveraging legacy application within an overall BPM context. This is the case because of the huge technology investment in tools to SERVICE ENABLE and in BPM to leveraging SERVICES.

Successful BPM and SERVICE ORIENTED ARCHITECTURE projects (together or separately) are all dominated by a clear view of business value to be delivered. They are often preceded by a Proof of Concept which is constrained and focuses on key integration aspects (for SERVICE ENABLEMENT) and key PROCESS aspects (which manifest business value).

Key to business value is the identification of value based on a clear understanding of the vision and mission of an enterprise coupled with strategic goals. This is the case because the strategic goals, which are derived from the vision and mission, provide clear guidance as to where the value lies. Identification of the strategic goals and the Key Performance Indicators that go with them enables a project to be framed that has clear business value and so when delivered can be measured to determine the value delivered. The same Key Performance Indicators and their derivatives may then be used to monitor and measure the running PROCESSES and their attendant SERVICES and the delivered flexibility provides the means by which the solution can be tuned and adjusted. This ensures that the run-time governance of the solution always aligns the solution to the business goals through their expression in the Key Performance Indicators.

SOA Manifesto rambilings part 3 (the principles)

Principles

The principles are high level principles. They do not deal with design or organizational issues but may allude to them. They are part of the manifesto because they act as a guide in how one should approach SERVICE ORIENTATION and how one should apply the principles of SERVICE ORIENTATION. Following these principles will help to ensure that whatever gets delivered provides the necessary short and long term business value to justify its existence.

“Respect the social and power structure of the organization.”

At an enterprise level SERVICE ORIENTATION and a SERVICE ORIENTED ARCHITECTURE provide a more open mechanism for accessing capabilities through their SERVICE CONTRACTS. It makes it easier to reuse existing SERVICES and create new ones in line with business needs. What is very important in looking at this from an enterprise perspective is who owns the SERVICE, who has the budget and how do we cost account for the use of a SERVICE. Respecting the social and power structures of an organization is critical in understanding the economic effect of a SERVICE, how it is used and how it can continue to be used over time. Aligning budgets for SERVICES and aligning the use of SERVICES to some cost accounting mechanism requires that budgets are aligned to the power structures and that the SERVICE is cost accounted for, based on the social and economic power structures within an organization. Getting this wrong can result in SERVICE degradation over time as more people use it and budgets for its subsequent maintenance and operational use fail to reflect it use and importance.

“Recognize that SERVICE ORIENTED ARCHITECTURE ultimately demands change on many levels.”

This is a corollary of the previous principle. Adopting SERVICE ORIENTATION and implementing and using a SERVICE ORIENTED ARCHITECTURE requires alignment through the organization. SERVICES are used, SERVICES change, and SERVICES are composed, all of this can happen across the entire organization. Use happens from all parts of the organization, request to change comes from all parts of the organization and composition of new SERVICES may happen anywhere in the organization and may use SERVICES from other parts of the organization. This is why SERVICE ORIENTED ARCHITECTURE demands change at many levels. The capabilities that SERVICES expose become un-siloed and that makes them enterprise assets and not just assets for one business unit. This democratization of capability requires change on all levels of an organization as users express future desires driven by changing needs and as the implementation of a SERVICE ORIENTED ARCHITECTURE makes flexibility easier to achieve.

“The scope of SERVICE ORIENTED ARCHITECTURE adoption can vary. Keep efforts manageable and within meaningful boundaries.”

This is a principle derived from the Evolutionary value statement. It is far better to aspire to the benefits of a SERVICE ORIENTED ARCHITECTURE but realize them incrementally. Typically one might start with the portalisation (portal-led SOA) of some business unit and SERVICE ENABLE along the way. The additional cost for SERVICE ENABLEMENT is mitigated by a standards based approach to the integration of the underlying capabilities with the needs of the portalisation. What we see in Cognizant is that this approach is complemented by additional concurrent but offset projects that then seek to further SERVICE ENABLE along the way as the portalisation efforts move to another business unit. Portalisation is not the only way to start. One might have specific needs to reuse components or applications simply to make available data and process from several application but to do so in a more flexible way. SERVICE ORIENTED ARCHITECTURE adoption starts by making it manageable within meaningful boundaries and so can be effectively measured in terms of the business value that it yields. Never forget Return on Investment, it pushes and pulls development.

“Products and standards alone will neither give you SERVICE ORIENTED ARCHITECTURE nor apply the SERVICE ORIENTATION paradigm for you.”

Products and standards do not apply SERVICE ORIENTATION to deliver a SERVICE ORIENTED ARCHITECTURE but they may help. It is people who wield them, making sure you have the right skills to get the most out of a SERVICE ORIENTATION approach and get the most out of a SERVICE ORIENTED ARCHITECTURE is a key aspect of any SERVICE ORIENTED ARCHITECTURE program regardless of how it is constructed or the boundaries in which it fits. In Cognizant we specialize in matching the right resources to achieve a positive outcome and the role of the Enterprise Architect becomes a key to ensuring that all of the moving parts of the Software Development Lifecyle (SDLC) work together.

“SERVICE ORIENTED ARCHITECTURE can be realized through a variety of technologies and standards.”

A SERVICE ORIENTED ARCHITECTURE does not have to rely on an ESB, WSDL, BPEL or any other standard or technology. The key aspects of SERVICE ORIENTATION and SERVICE ORIENTED ARCHITECTURE are the descriptions that we call SERVICE CONTRACTS and the decoupling of them from the implementation of their capabilities at both design and runtime. An instance of an SERVICE ORIENTED ARCHITECTURE may use asynchronous messaging and pass documents between SERVICES. An instance may use WSDL invocations with some form of remote procedure call. An instance may use BPEL and it may use business rules. In all cases the de-coupling should be self evident. A yard stick for an instance of a SERVICE ORIENTED ARCHITECTURE is to locate a SERVICE REPOSITORY, because without one, de-coupling invocation on a SERVICE CONTRACT from a SERVICE IMPLEMENTATION becomes impossible to achieve in any uniform way.

“Establish a uniform set of enterprise standards and policies based on industry, de facto, and community standards.”

A bit of motherhood and apple pie but something that the can certainly catch you if you don’t do it. Establishing a uniform set of enterprise standards and policies can reduce the risk of mis-delivery of a solution. Where there are industry standards they often leverage XML as a means of encoding information and it is better to use them than not. It can cut down integration costs and reduce the time it takes to deliver an enterprise information model as well as the processes that manage that information. The enterprise standards should cover all aspects of governance and should support rapid change. The policies should guide security choices, transactions, and service levels. One key relationship between a SERVICE ORIENTED ARCHITECTURE and this principle is that many industry and de-facto standards are seeking to use the same SERVICE ORIENTED approach to describing the processes that underpin standards, this is true in International Standards Organization, the International Swaps and Derivatives Association, Healthcare Level 7 and many others. For those SERIVICES that are outward facing this may mean that the SERVICE CONTRACTS are already defined which can help reduce the cost of external connectivity and provide cross enterprise interoperability.

“Pursue uniformity on the outside while allowing diversity on the inside.”

This is a cry for SERVICE CONTRACTS and process uniformity on the boundaries of the enterprise and diversity as to how you achieve implementation on the inside. SERVICE CONTRACTS mandate a degree of uniformity. We can go further by publishing our external boundaries in terms of the SERVICE CONTRACTS we wish other to see and the ordering rules over those contracts that make up the processes that we wish to make visible. In so doing we ensure a higher degree of interoperability.

“Identify services through collaboration with business and technology stake-holders.”

From a technology platform perspective it is important to identify core TECHNOLOGY SERVICES. These will have the highest reuse rating and you can only do this by understanding the technical requirements from the technology stake-holders. Likewise the only way to gain reuse at the business level for BUSINESS SERVICES is to do it with the business stake-holders. The widest collaboration, across business units will yield the widest reuse, even if you have a single project, it is worth gathering requirements from a wider set of stockholders so that you can plan effective reuse and change management over time. Hence “Recognize that SERVICE ORIENTED ARCHITECTURE ultimately demands change on many levels” and “Identify services through collaboration with business and technology stake-holders” are complementary.

“Maximize SERVICE usage by considering the current and future scope of utilization.”

Wider collaboration for effective SERVICE IDENTIFICATION is part of the process of understanding the usage of both current and future SERVICES and their utilization. Don’t just look at the functional aspects because the scalability will be compromised as usage rises. Taking into account the scalability issues mitigates the risk of retrenchment and ultimately failure of large scale enterprise adoption.

“Verify that SERVICES satisfy business requirements and goals.”

This principle may well seem obvious, but it has profound consequences. One of the things that SERVICE ORIENTATION and SERVICE ORIENTED ARCHITECTURE instances provide is a clean separation of SERVICE CONTRACTS from SERVICE IMPLEMENTATION. It matters not if you use WSDL to do this or SOAML or Abstract BPEL, what is important is that before a SERVICE is constructed (implemented) we can, at worst, manually check that the collection of SERVICE CONTRACTS match the requirements that gave rise to them. If these requirements are themselves derived from higher level business requirements we can check the derivation. And if the higher level business requirements are derived from business goals we can check their alignment too.

When we do this manually, and most good SERVICE ORIENTED ARCHITECTURE practitioner do exactly this, we call it SERVICE ORIENTED ARCHITECTURE GOVERNANCE and it is part of Enterprise Architecture Governance. It is part of the enterprise alignment process that is done manually.

Given that SERVICE CONTRACTS can be properly described in a formal and structured way (i.e. WSDL , SOAML and Abstract BPEL) and given that we can encode the requirements that give rise to these SERVICE CONTRACTS, it would make a lot of sense to get computers to check the SERVICE REQUIREMENTS against SERVICE CONTRACTS and automatically show that the "SERVICES satisfy their business requirements". We are not suggesting at this point that we can do the "satisfiability" test through the different levels of requirements or indeed the business requirements against the business goals - although even this is possible. What we are suggesting is that the immediate requirements (SERVICE REQUIREMENTS) that give rise to the SERVICE CONTRACTS can be automatically checked to show that the SERVICES meet their SERVICE REQUIREMENTS.

This is what Testable Architecture is all about and the wider "satisfiability" is what Savara will deliver over time.

“Evolve services and their organization in response to real use.”

Accept that we live with imperfection but strive to attain it. Accept that things change. In this way we can focus on evolution to ensure that SERVICES meet their business requirements as those same requirements evolve over time. Equally we may choose to re-organize services for the same reasons. We may choose to uncouple a composite or to change it to reflect changes to real use. The best user interfaces are always tested by users and Apple have been very much the masters of getting it right. Doing the same for SERVICES from the top down, the middle out and the bottom up by evaluating their use in real situations will reveal improvements on design, improvements to requirements and manifest improvements with continual alignment to business requirements and goals.

“Separate the different aspects of a system that change at different rates.”

Engineering for change is not easy. It requires an understanding of what is needed today against what might be needed tomorrow. A key metric is how executive management and in turn their immediate management team steer the ship. What are the controls and the points of change that they need to do their job. Understanding industry influences and trends on an enterprise and understanding the vision provide a way of scoping what needs to be made flexible in the future. Understanding the mission and strategic goals provide a scoping of the changes needed in the near term. Having effective change management procedures in place and effective change request processes are imperative to continual improvement.

Another key aspect of rates of change is that processes change infrequently, but they do change, but the decision point, we might think of them as business rules, change far more frequently. Hence outboarding business rules and using a BUSINESS RULES SERVICE is an important architectural pattern which complements outboarding of processes through orchestration and workflow.

Don’t think that what you deliver will not change or is optimal. Let the users use and make suggestions. If you get flexibility engineered in from the start then the cost of suggested change will reduce and the ability of the enterprise to optimize how it does things increases.

“Reduce implicit dependencies and publish all external dependencies to increase robustness and reduce the impact of change.”

When we create COMPOSITE SERVICES, just like when we create views in a database, it is important to understand the SERVICES that they derive from. Without documenting and making these dependencies clear we potentially run into huge change management and governance problems. It should be standard best practice to publish the key dependencies both in terms of the services that may be used and the underlying technology dependencies too. Such dependencies should be documented and published as some form of attachment to services in the SERVICE REPOSITORY.

“At every level of abstraction, organize each service around a cohesive and manageable unit of functionality.”

Good design coalesces concepts into some form of cohesion. This is true for any piece of software that is constructed. SERVICES are no different in this regard and at each level of abstraction be it a BUSINESS SERVICE, a TECHNICAL SERVICE or a DATA SERVICE the principle of cohesion applies. Organizing and making things manageable takes effort and discipline but the rewards are often greater use and reuse because the focus of the SERVICE becomes clear and the concepts coalesce to deliver appropriate functionality for the level of abstraction.

SOA Manifesto rambilings part 2 (the value statements)

The value statements reflect a desired prioritization of one value over another. It does not mean that the value that has a lower priority is not important it just means that the value with higher priority is seen as more important. The value statements are there to help prioritize key decisions. For example having a technical strategy with no business value makes little sense. But a technical strategy to support business value makes huge sense. Equally having project specific goals make sense but not at the cost of strategic goals. Should the need arise to support project specific goals over strategic goals then the decisions should be made with our eyes wide open.

“Business value over technical strategy”

SO and SOA is not about technology. One may use technology to deliver it. The value is the value that the business identifies or agrees to. In the world of Enterprise Architecture and Solutions Architecture we measure the outcome of deliverables against agreed business expectations. The technical strategy that is employed is created based upon what we need to focus on technically to deliver business value. Many projects fail because they loose sight of the alignment with business imperative. Ensuring alignment to business value is the key to good governance.

“Strategic goals over project-specific benefits”

In the world of effective Enterprise Architecture applied to a business landscape, we seek to understand the vision of the enterprise and we seek to understand the mission statement. The vision is aspirational and forward looking, the mission is actionable and focused on the immediacy of the enterprise. The vision and mission are used to frame strategic goals for the enterprise. The strategic goals might be implemented by one or more programs of work within the IT space of an enterprise. These programs might incorporate many projects. One of the roles of an Enterprise Architect is to ensure that the programs align to the strategic goals and that the projects reflect them. Without the transparency and traceability of strategic goals to the vision and mission, projects cannot expect to deliver against them. Drift is a common concern and when siloed projects drift away from the strategic goals and focus on short term issues at the expense of the bigger picture. The result is a more costly solution as it has to be re-factored and/or re-built sooner than was anticipated because it failed to meet the strategic goals.

“Intrinsic interoperability over custom integration”

The easier and faster it is to compose new capabilities the more cost effective IT becomes in supporting the business and its changing needs. If we always have to place a mediator between services to reuse them then we are falling into the trap of custom integration and what this tells us is that the design of our services does not support intrinsic interoperability. From a principle perspective, moving to a more document centric approach encourages greater interoperability but compromises the ease of use or understanding of a service's capabilities. Probably no where else in designing services is it more important to get the functions and the types of the parameters correct and indeed to be unafraid of re-factoring over time to move towards greater intrinsic interoperability.

“Shared services over specific-purpose implementations”

The granularity of services in an implementation of an instance of a SOA is a common cause for concern. Reuse and therefore the sharing of capabilities through the embodiment of services and their contracts seems to be much harder than anticipated. Several methodologies have been espoused to help. What we all know instinctvely is it is never a good idea to re-develop the wheel. But of course we constantly re-develop the car. Cars are created through the composition of many existing components. The new Beetle makes use of the Golf chassis, but it feels like a very different car. Services as components for the construction of new capabilities is a key principle of SO but doing it is harder that saying it. What is needed is good component repositories with good documentation and reviews. Back in the 1980’s Shlaer and Mellor had the same idea but expressed it as a book of components, like those that electrical/electronic engineers might use, for things we use to create new capabilities. Sometimes a customer implementation is needed but it is not that often and measurements of customer implementation vs reuse should always be made on a regular basis to ensure that the benefits of shared services are being realised.

“Flexibility over optimization”

There is an old software proverb that rings true. Make it work before making it fast. Making things flexible, choosing where the points of flexibility need to be is not easy but by following some simple principles we can at least move much of what is needed outside of code making services configurable. Business rules can be used in place of coded logic, orchestration can be used in place of hardwired configuration. A heuristic approach that might be in a document for service implementers would be to ensure that business rules and processes stated in requirements are exposed to enable them to be changed.

“Evolutionary refinement over pursuit of initial perfection”

The old adage of don’t try to boil the ocean comes to mind. We always seek perfection but we always need to temper it with the realisation that perfection is a focus not a goal. Good software projects and programs are phased. Occasionally we have the big bang approach but they rarely succeed. So we need to be mindful that we cannot do it all in one go and so phase the approach. It is common for many SO projects and programs to be led by the user interface inwards. Service enabling as you go is a better approach than trying to service enable and then figure out how users interact with the service. This is the case because without the users we cannot be sure of the pain point the wishes and desires of those users and so cannot address their concerns. And it is the users (customers) that matter.

---

SOA Manifesto ramblings and observations PART 1

It has been a couple of week at least since we published the SOA Manifesto. Alas I have rather busy putting some of it into practice. But that is done now. So time to elaborate further.

I plan to break this blog into 3 parts to provide some clarifications, some observations and some personal opinions about the SOA Manifesto. In this blog I want to deal with three specific points in the SOA Manifesto. In the next two blog I will deal with much of the rest of the SOA Manifesto.

The three specific points relate to the following:

"Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation."

"Business value over technical strategy"

"Verify that services satisfy business requirements and goals."

"Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation."
I have, for a very long time, challenged the wisdom of the term Service Oriented Architecture. If you track back over the last 5-6 years the term SOA has been used and misused almost on a daily basis. Vendors have often hijacked the term claiming that they sell SOA.

The term Service Orientation is the antecedent to SOA. You cannot have anything being a SOA if you didn't do some sort of SO, just as you cannot have an Object Oriented Architecture or implementation of one without practicing OO.

Clearly we can use C++ in an non-OO way. We can use C in a non-OO way, but oddly enough we can also use it in an OO-way. X-Windows was a great exemplar of an OOA that was done by using OO within a C based implementation. If we constrain things and use a classic OO language, like Smalltalk, then by definition you end up with an OOA.

Thus the terms OOA and SOA are types of things not instances. There may be many instances of an OOA and many of a SOA. Therefore we can use the phrase "my implementation of a distributed system is an instance of a SOA", and we can also say that "my implementation was created by applying the principles of SO". Hence SO is something you do and an instance of a SOA is something that you might create through the application of SO principles.

Why is this important? Isn't it obvious? For those that are SO practitioners it is obvious, but we are faced with myth that has dogged the rise of SOA. I have many customers who ask how do I sell SOA? It is not something that you sell because you cannot buy it. So we need clarity and common language so that when we engage with customers we do not add to the confusion but rather remove it and provide clarity. So obvious to us practitioners yes, and important YES, so that we can effectively engage with businessES who have need of business benefits that a SO approach can deliver through Service Enablement to provide an instance of a SOA within a business context.

"Business value over technical strategy"
Providing a business context is all important. In the dot-com boom days many projects were technology led and the business benefits unclear. Most if not all failed as a result. Motherhood and apple pie comes in here. It is common sense that we understand the business value prior to any technical strategy. The technical strategy supports the business value. Technical strategy is there to support the vision, mission and strategic goals that a business sets for itself. The technical and application roadmaps are constructed to do exactly this. The funny thing about "common sense" is that it is not common, and that is why we needed to state this value proposition business value over technical strategy.

"Verify that services satisfy business requirements and goals."
This principle may well seem obvious too. But it has profound consequences. One of the things that SO and SOA instances provide is a clean separation of Service Capability (Contract) from Service Implementation. It matters not if you use WSDL to do this or SOAML or Abstract BPEL. The important thing here is that before a service is constructed (implemented) we can, at worst, manually check that the collection of Service Capabilities match the requirements that gave rise to them. If these requirements are themselves derived from higher level business requirements we can check the derivation. And if the higher level business requirements are derived from business goals we can check their alignment too.

When we do this manually, and most good SOA practitioner do exactly this, we call it SOA governance and we call it Architecture Governance (although governance is actually wider than this). It is part of the alignment process that is done manually.

Given that Service Capabilities can be properly described in a formal and structured way (WSDL , SOAML and Abstract BPEL all do this) and given that we can encode the requirements that give rise to these Service Capabilities - I shall call them Service Requirements - it would make a lot of sense to get computers to check service requirements against service capabilities and automatically show that the "service satisfy business requirements". I'm not suggesting at this point that we can do the "satisfiability" test through different levels of requirements or indeed the business requirements against the business goals - although even this is possible. What I am suggesting is that the immediate requirements that give rise to the Service Capabilities can be automatically checked to show that the Service Capabilities meet the Service Requirements.

This oddly enough is what Testable Architecture is all about and the wider "satisfiability" is what Savara will deliver over time.

The SOA Manifesto

Well we did it. We sat down for the best part of 3 days and turned around an SOA Manifesto. I shall blog further, as will my fellow signatories, but for now here are all of the authors blogs to make it easier for you to see what they have to say over and above the SOA Manifesto and of course for you to contribute too.

It is very flattering that the press have used the term "Industry Visionaries" to collectively name us. But we are really no different to many of you practitioners out there. What will make the SOA Manifesto valuable is use and feedback to hone it over time. So take a look comment away and keep us all in the loop. We shall be looking.....

The movie of the announcement is also available.

Cheers

Steve T

In no particular order ....

Anne Thomas Manes
Stefan Tilkov
Dave Chappell
Brian Loesgen
Paul Brown
Mark Little
John deVadoss
Clemens Utschig-Utschig
Joe McKendrick Analyst
Ali Arsanjani


The others I shall add when I have their coordinated.

More on the SOA Manifesto

I have been thinking ever more about the SOA Manifesto. In part fuelled by the proximity of the WG meeting and in part by comments that have been made.

I take on board the comments on the need for "boldness" I have never been shy about dealing with that. But we need to ensure that the manifesto has use today that is practical and very business focused - something I have picked up from the many comments made at infoQ. And we need to map out what is missing and look deeper into a brave new world that builds upon our current available SOA platforms and methods of constructing solutions.

One thing stands out for me on the practical usage side and one on the future.

On the practical side:

• The use of Service Orientation is underpinned by delivering continuous business value. Business value MUST always be aligned to business goals and needs.

In the future:

• It MUST be possible to express the information model(s), the static service model and the dynamic model of service collaboration such that the first describes the data needs of services, the second describes the service usage across peers of a specific service and the third describes the common collaborative behavior of peers collectively. All of which MUST be founded upon one or more business requirements.

I think these two and refinements of them will stand us in good stead. The devil is always in the detail.

The former is really motherhood and apple pie. Easy to say but how do we bake it?

The latter is more of challenge because it requires a better understanding of what underpins a distributed system in terms of behavior and information that drives that behavior and these, at the end of the day, are complex mathematical topics. We must be careful and ensure that we do not go into too much detail but provide the right syntactic sugar that allows formalism to deliver to us in a way that we can all understand and use.

Let us not forget that without mathematics we would have no computing. In fact without it we would have no engineering. They are all based on mathematics but we rarely see the models that are used to underpin engineering but in the case of computing we use those models all the time. They have syntactic sugar we call them programming languages.

An Architecture Manifesto

Way back in 2004 a good friend of mine, Alexis Richardson, decided to convene a summit of architects. The summary of the event is below:

The Architects Summit is a one off event challenging a group of pragmatic and active technologists to formulate, document and publish a consensus view on requirements and best practice for software application platforms and development tools.

I cannot remember now how many turned up but it was well attended and amongst the participant Chris Swan and Hugh Grant were there, Rod Johnson was there, Matthew Rawlings, John Davies, Cameron Purdy and many others.

I was asked to co-chair with Alexis, presumably because chairing seems to be my thing amongst a diverse set of individuals and opinion.

The net result was certainly a stimulating debate but no manifesto was ever issued because we could not get agreement. In part this was because the camps divided into classic enterprise architects and guru-status technical architects. And never the twain can agree.

However, given the fashion for manifesto’s here is my own personal one which was done at that time and to which I still pretty well subscribe to.

The SOA Manifesto

I was recently invited to participate in the SOA Manifesto. It is huge honour and from the list of participants involved I would expect nothing less than a really focussed and helpful manifesto.

Manifesto's a a bit of craze these days. I noticed that there manifesto for cloud and for internet and so on. But oddly enough nothing for SOA.

As a participant I started to look into what is out there and I must admit I am really surprised at how little there is for SOA. Of course we have patterns and principles and even governance [1, 2] for SOA but as yet no one has written a manifesto personal or otherwise to share with the industry as a whole.

In arriving at some manifesto for SOA which will help focus people on the key aspects of what service-orientation means and what in means to have a service-oriented-architecture there are some key areas that need work.

First an foremost we need to ensure that executive sponsors across industries understand what it means to them. Inevitably they need to know what is the effect on the top line revenus and the bottom line costs. Equally they need to understand business model consideration in adopting a service-oriented approach. Whilst we would all like to offer thing free, offer it immediately and offer it so that it is perfect in every way we have to be realistic that software costs money, takes time and is rarely perfect. So ensuring that these apsects of the software development lifecycle (SDLC) are effectively articulated to business sponsors will help them make decisions.

Secondly we very much need to serve our own IT community. We need to arrive at some consensus that helps people understand what service-orientation really means, its guiding principles, and what consititutes a service-oriented-architecture. Now these may all seem obvious to some of you. But let me assure you that in my world I often have to deal with people who equate WS-* to SOA and equate ESB to SOA. Which of course is not the case. They may help and even constrain and SOA but they not make one.

So we have our work cut out. We also need to deal with the future and what that might entail and help to direct and give it impetus.

So watch this space. The manifesto is due out soon enough. And I for one am looking forward to it let alone having the honour of helping to fashion it.

Savara

JBoss Community Launches Savara Project.

I guess it has been coming for sometime given the blog entries from both myself and Mark Little. Finally we have launched Savara. It is an open source community project the aim of which is to deliver to the open source user community and other interested parties a set of tools for developing end to end service oriented solutions (in the first instance) and general distributed solutions (in the second). It will embody Testable Architecture but much more. The key aim is to develop not just tooling but also methodologies to support its use that will focus upon the testabilility of artefacts at different levels of refinement to ensure provable alignment or correctness.

We expect this initiative to change the way in which we gather requirements, fashion solutions and deliver them as well as manage changes over time. And we expect to move ever closer to our (unattainable aim) of deliver it now, deliver it free and deliver it correctly which will remain our mantra to keep us focused.

The vision is broad, the time scales potentially long but we do expect benefits to be provided early and on a continual basis.

I am sure with Gary Brown (Red Hat) and Bhavish Kumar (Cognizant) co-chairing the initiative we can expect great results.

Take a look when you can and get involved regardless of you affiliation.

Testable Architecture and Agile Development

I was watching and listening to Dan North and Michael Feathers on the web. It is worth a look. At about 12 minutes Dan says the following "You can TBD your way out of anything... There is balance between doing all up front design and doing no up front anything which is irresponsible." and he mentions that Richard Watt says "Enough up-front thinking" is what you need.

The tension between agile and "enough up front thinking" is what I want to explore. Those agile bigots that think you should not do anything up front miss the point. Hopefully there are not too many of them. Having deployed "testable architecture" in several sites now I do get asked how does this work with agile. And I think Dan and Michael have really nailed it.

What is enough?

The communication/interaction model between services in an SOA is probably a good place to start. The aim is to provide enough structure without getting in the face of developers who will operate in an agile process. If testable architecture, embodied today in the pi4soa tools, provides the necessary communication/interaction model along with the necessary minimum data structures (for example the business transaction identities needed to support the communication model and any asserted state thereafter) then we can (and we have in Cognizant) use it to provide "what is enough" and thereafter engage developers in an agile process to fill in the gaps. The structure is preserved and testable against requirements, as I have often said in my other posts. And in more recent posts that structure remains a valuable testable artefact for systems integration testing.

Defect Detection in Systems Integration Testing

Another short movie to illustrate a point. This time I want to move to the left brain side of the testable architecture methodology and deal with the issues that confront many of us on complex programmes.



The issue is one of defect detection, and I want to specifically highlight how we can use testable architecture in its current form (aka WS-CDL and the pi4soa tool-suite) to detect a system defect. The defect is categorised by a variation in the system wide (and so end-point specific) contract for the services. In this case we change the contract and update one of the services but not the other. It is a pretty typical problem that is hard to identify (although not impossible) during systems integration testing. What we do here is use the monitoring capabilities during the system integration testing to determine the root cause of the problem.

It is only but a short jump from here to full run time governance. And that is what Overlord is all about.

Enjoy.

Global Models

Perhaps the biggest contribution that the Choreography Working Group has made, along with our academic colleagues and colleagues from many vertical standards organisations, is the formulation of a global model. WS-CDL was the first attempt at codifying such a model.

BPMN2 when it is finally released will have done us a huge service if, as we expect, it combines both global and local behaviors. The lack of a local model for behavior has long been a gap in our thinking on global models and would add the refinement needed to go from very high level to executable artefacts in a controlled and refined way.

So it was great to see the blog at Overlord that welcomes BPMN2 to the world.

16 Minutes on Testable Architecture

Using formal methods to support an offshore global delivery method

I have blogged before on this topic but now I want to share some of the results with you.

I was recently involved in two insurance focused system integration initiatives. The fact that they were in insurance is more suggestive that the same approach can work anywhere as insurance tends to be fairly conservative when it comes to new technology.

The background, in both cases, was an in-flight SOA-road map that was being executed. The road map required integration of legacy applications and portal development to support a user experience. A classic ESB approach was used to manage integration and provide both mapping from and to a legacy message format and mediate between the new functionality and the old legacy system. For the most part this is straight forward, or at least so it seems. Estimates were given for the two initiatives to provide the necessary technical contract that would drive implementation offshore.

In the first instance the allotted time to develop a solution and provide the technical contracts was of the order of 15 days effort. This would involve gathering requirements, functional and non-functional, looking at message formats to support the data model, understanding the business transactions so that the solution could be effectively monitored, delivering service contracts for the mapper and mediator services, providing state diagrams to show the individual service behaviors needed and sequence diagrams to provide a collaboration context. The total number of services was 4 including the client, the mediator, mapper and legacy service.

In the second instance the allotted time to develop a solution was of the order of 60 days of effort. In this case there were many more services (I prefer to call them roles) that needed to be understood and many more interactions between those roles. The same level of technical contracts was required to be delivered over the 8 or so roles.

In both cases we used testable architecture. We gathered the requirements initially framed as use cases in UML like notation and then used the pi4soa collaboration tool suite to create sequence diagrams (in which each link referred to an example message in the scenario being modeled) to support each of the use cases. These were then used to drive the development of a CDL model and that model was tested against the sequence diagrams. This represents the left part of the method shown below:




In the first case it tool less than one day to gather the requirements and deliver the tested model. In the second case it took 4 days to do the same. The technical contracts, the WSDL, the BPEL, the BPMN, the state diagrams, sequence diagrams and much more took a few clicks as they were generated from the CDL model.

It may sound fantastical and mythical that we compressed the effort by some 80%. But the fact is that we did. Not only that we found design flaws early both in our use of BPEL and WSDL. We found competing requirements and corrected them and we produced technical contracts that were far more rigourous and complete and unambiguous than we had managed to achieve by hand.

In the early development of the pi4soa toosuite we developed a full FpML model of derivatives trading from indication of interest through to the end of economic life of a derivative. I don't claim that this model was correct but the process of doing it took 4 weeks to complete and in the process a far greater understanding of derivatives processing was gained by both those doing the model and those providing the expertese of derivative processing. Similarly with TWIST and payments and more recently with HL7.

Some question the value of descriptions. All I can say is that the proof is fairly evident that describing something well has the effect that one better understands what is required and design flaws tend to be highlighted. In the insurance enagagements and the much larger FpML engagement the use of testable architecture, the gathering of requirements as sequence diagrams with messages, the modeling of the entire set of requirements and the formal testing of the model against those requirements and finally the generation of technical contracts is without peer. I know of nothing similar that can do this.

Fortunately I work, these days, for a large systems integrator (Cognizant Technology Solutions). This has given me the chance of trying this stuff out and showing I have always believed to be true which is simply that the impact of description through a formal method will help to change the way in which we can effectively do more for less with higher quality and so move the entire process of creating solutions up one level of abstraction with no loss in our ability to express a solution.