Wednesday 30 January 2008

SOA Governance: WS-CDL meets Policy

Governance seems to be a pretty hot topic right now. Everyone seems to be focusing on it. IBM, Oracle and Microsoft have portals dedicated to it and offer solutions. I noticed that Mark Little of Redhat and JBoss fame has been blogging recently on it, "Tooling and Governance". All very interesting in particular Mark Little's blog.

The OMG has been firing up on the topic of governance too. Dr Said Tabet, co-founder of RuleML, gave a talk recently at the OMG conference entitled "Regulatory Compliance: Key to the success of BPM and SOA". This talk was interesting for a number of reasons. Back in 2004 Dr Tabet, Prof Chakravarthy, Dr Brown and I gave a position paper at a W3C workshop on policy "A generalized RuleML-based Declarative Policy specification language for Web Services,". In this paper, written before WS-CDL came into being, we postulated the idea of policy and WSDL and how they relate. Our thoughts have since clarified and the link is more meaningful, more valuable and more appropriate between WS-CDL and RuleML, more generally it is the link between a global description of an SOA solution and policy statements attached to that description.

All of this is great stuff but I feel a little bemused that few if any (none that I could find) have really gone back to the basics and understood and certainly explained what governance means. Instead we are left with market-speak such as "SOA governance is an extension of IT governance" and "".

What we need to do first of all is understand just what we might mean by governance? Then we might understand what it is we need rather than join the bandwagon and talk about SOA governance in either a purely technical way or in an abstract way that is not related to requirements and solutions.

It is surprisingly difficult to arrive at a consensus definition of what governance means. Online dictionary definitions do little to shed light on what we might mean. Having searched the web the one I liked most, which conveniently fits what I want to talk about is "The systems and processes in place for ensuring proper accountability and openness in the conduct of an organization's business.". I was very glad to see that the same problems related to understanding of what is meant by governance is not confined to me alone. One of the early movers in the space, Actional, thought so too.

I like this definition because as a serial entrepreneur I have been subjected to governance as it pertains to the running of a company and this pretty much describes one of the key responsibilities of a board of directors.

A system of processes would imply that there are documented procedures that one should adhere to. Accountability might be seen as a measure against those documented processes and openness might be seen as a means of conveying the levels of compliance (an interesting word given the talk Dr Tabet gave) to those processes.

How can we apply this to an SOA solution? Obviously for a people organisation records are kept and managers manage according to some rules. Should rules be broken documentation is provided to show where, how and why the rules were broken and if needed corrective action can be instigated. As people we are good at doing this sort of thing because we are flexible and can deal with missing information much better than computer systems.

Computer systems like clarity and ambiguity is not something that they deal with very well at all. So we need some way of describing unambiguously what the procedures and the rules might be against which we can govern. Without unambiguous procedures and rules we have no hope of governing anything computationally at all.

Governance and how it may be applied to an SOA solution for my mind must deal with some understanding of what the solution is supposed to do. It matters not a jot if we govern a solution that fails to meet its requirements because the governance will be abstract and have no grounding in the business to which it is being applied.

If a business fails to meet it's basic requirements, regardless of what they are, the board of directors would be failing in their duty of governance if they did not take action to steer the company in a direction in which it did meet it's requirement however that is be achieved.

It seems to me that it would be a good idea to be able to demonstrate that an SOA solution meets it's requirements.


Requirements applied to any system (SOA is irrelevant here) are often divided into functional and non-functional. Functional requirements tend to give rise to data models (i.e. a bid in an auction and all of it's attributes) and to behavior (i.e. the auction system accepts bids and notifies bidders and accepts items for auctions and notifies sellers). The former is often captured in a model using UML or XMLSchema and so on. The latter is often captured as a set of functional signatures (i.e. newBid(Bid), notify(LeadingBid),notify(LoosingBid)) and sequence diagrams showing how things plug together.

Non-functional requirements are constraints over the functional requirements. They might relate to the technology used (i.e. use JBossESB use WS-Transaction and use WS-RM and use WS-Security) or they might relate to performance (i.e. a business transaction will always complete within 2 minutes). In all cases we can think of these as policy statements about a solution. Those that relate to performance are dynamic policy statements and those that do not are static policy statements.

Good governance of an SOA solution is one that can show for any transaction across it that it complies to the requirements. That is it does what it is supposed to do. More specifically we can state that for any transaction across multiple services in an SOA solution that all messages flowing between services are valid, meaning that they are valid against some model of the data and that they are in the correct order (i.e. payment follows ordering). Furthermore that the flow of messages completes at the user within the agreed SLA for that user (i.e. it all took less than 2 minutes).


Figure 1: Example of a sequence diagram

Today's governance solutions lack a description of what is supposed to happen. They have no notion of correct order. This is probably why most solution are siloed and concentrate on individual services in an SOA solution rather than looking at the big picture.

If you imagine a set of services that make up an SOA solution each may have constraints upon it. The auction system may have a constraint that bidders are notified when outbid within 30 seconds. Payment processing completes within 1 minute after the auction finished and so on. The entire process however rarely is described and so the effect that individual policy statements on each service might have across the entire solution is lost. There is no where to say it and no description of the entire system against which to attach it.

One might suggest that BAM solves such problems. But is does not because it has no view of the underlying requirements of the system and so cannot determine correctness of behavior.

If we have a description of an SOA solution in terms of how services interact (the messages they exchange and the conditions under which exchange occur) and that description is unambiguous then we can start to see how governance can be achieved. Such a description is the basis for the procedures against which we measure governance. Governance is always measured, otherwise we cannot say we have good or bad governance. If such a description existed we can attach policy statements to the description. Some might be static and some dynamic. What it gives is an overall context to governance and enables us to say that a transaction was correct and importantly that another is not and examine why it is not.



Figure 2: Example of a description of an auction system (top level only)

This is why WS-CDL as a description is so powerful. It can be validated against sequence diagrams to ensure it models the functional requirements. It can be used to drive construction and with policy attachments can ensure that the right technologies are employed. During runtime it can be used to ensure that messages are correctly sequenced and that messages are flowing within allowable tolerances based on attached dynamic policies.



Figure 3: Example of policy attachment (to the bidding process)

Attaching policies to WS-CDL provides an overall governance framework in which we can be sure that exceptions are better identified and the solution as a whole steered in a more effective manner to achieve good governance across the board. Without it, or at least without an overall description against which policies are attached we are simply navigating without a compass or stars to guide us. Governance then becomes a matter of guesswork rather than something concrete that we can measure.

2 comments:

Unknown said...

Steve and I have been talking about this topic for years now, before SOA. The paper we published a few years ago dealing with services and policy attachments (referenced in Steve's post) was meant as a wake up call given the direction taken at the time by web services security and policy work.

Today, with the SOA buzz and Governance and Compliance sensitiveness, we still see a push for old solutions that never worked and will not work. Steve talked about the current frameworks, standards activities (most relevant ones but there are many others), and the issues, I mean the real issues, most companies face today. I still think some of the problems we face in Information Technology today are the same ones we faced 20, 30 years ago. One of them is related to the lack of communication and information management discipline, something that impacts the efficiency of requirements, delivery of solutions and services, and out of scope products.

When it comes to Risk, Governance, and Compliance, we are under even more pressure to deliver. Transparency and traceability are the two key elements here. Regulatory compliance represents a very challenging task for any IT group. Imagine how complex it gets when we are dealing with a company operating in multiple jurisdictions with often overlapping or conflicting regulatory requirements.

SOA Governance needs a declarative specification of services and WS-CDL does just that. Non functional requirements need to be de-coupled and attached separately as policies. The combination of those two things will enable us to finally get back on track and deliver a framework that works!

These are my initial comments. In an upcoming post, I will go into more detail on the standardization efforts, internal controls, and risks.

Said Tabet

Hediye said...

thanks good article