Almost certainly the will be the start of a stream of blog entires. I am still mulling over the plethora of techniques we are told about and still trying to see if any of them really help or if it is a case of good design principles rehashed yet again. Perhaps if I come up with something concrete we will call it YASIM (Yet Another Service IdentificationMethodology). Anyway here goes part 1.
One of the most difficult things to do in SOA-based projects is finding the right services. There is much fear uncertainly and doubt about this. To meet the challenge many vendors and integrators are seeking to create methodologies to support the process. Consequently there are lots of methodologies about logical designs and how to derive services from them. Some use data flow techniques other use process modeling techniques and all have some taxonomy of services that classify services (albeit with different names) as "business", "data", "technical" and "utility".
The data flow and modeling techniques are intended to tease out services that are business aligned by looking at the data (data flow) or the process. Services are derived from such views and then classified.
Data modeling apart - largely because it is not the way in which the prevailing winds blow - relies on some notion of activity grounded to a role. BPMN is a typical graphical language that is used. This technique is fine for the most part although in the case of BPMN scalability is an issue because it put roles at the front and obscures the process as a result, but nonetheless it works for the most part in deriving services.
Of course deriving what services are needed is only one step in the process of delivery. What is needed in support of this is to identify services that might fulfill need. In a greenfield site this is not a problem but as SOA adoption grows one of the benefits is supposed to be reuse. So how do we ensure reuse and what does it mean?
Finding a service that has both the correct method signatures and the correct behavior, the keystone of reuse, is for the most part an aspiration. It would be wonderful if we could ask a service repository for a service with methods "foo(FooXML)" and "bar(BarXML)" in which "foo" is always called before "bar" or in which order is irrelevant and for the repository to tell us what matches our query.
The reason why we cannot do this sort of search is in part because the behavior of a service is not captured in the repository and because the architecture description (data flow model or process model) does not support any formal linkage between it and the service descriptions in the repository. So we are left to figure it out and the best we can do is find that a service has a "foo" that takes a "FooXML" and a "bar" that takes a "BarXML" as input (similarly for the outputs).
I was at a recent Scribble meeting at Imperial College and I saw the future is now. The Pi4 Technologies Foundation along with Imperial College and Queen Mary College and a few others have been working on a language called Scribble. It is "son of" WS-CDL. A much cleaner curly brace notation for describing teh dynamic behavior of an architecture. Most importantly it has a behavioral type system. The demonstration showed that an encoding of WS-CDL can be rendered into Scribble and then the type system can be used to check for behavioral conformance of JBossESB actions. If you change the WS-CDL having generated the JBossESB (directly or by hand) you can see areas of conformance and non-conformance. Finally we have the linkage between the intent of the architect and the implementation contract of a service in a true SOA platform.
I am looking forward to playing with this when released as part of project Overlord and I expect it make reuse the norm rather than the exception and to do so at a fraction of the cost.
Friday, 12 September 2008
Monday, 21 July 2008
Mulling over architecture
I've been mulling in no specific direction today. Letting the wind take me to wherever it wishes and that often leads me to think more clearly.
I was talking to my dad over the weekend. He was involved in avionics most of his working life and was the examiner of the institute of Quality Assurance in the UK. We were talking about the Farnborough Airshow - which I attended on Friday - and about fly by wire. The European Fighter Aircraft (Typhoon) is a fly by wire as is the A 380. I don't know about anyone else but the standard of software development would make me concerned if I had to fly in one of those. I try to forget it all when I board an aircraft so I don't worry.
It turns out that much of what they do to increase the reliability of fly-by-wire is to use redundancy. Triple systems is one approach that is used. The components are developed independently and then monitored. When they diverge majority voting is used to determine the likely correct behavior. It is statistical because there is no guarantee that the behavior of the two that agree is correct. It is simply the case that two agreeing is likely to lead to correctness.
In high availability system of this nature they use multiple compliers too and they might choose to use different chip sets.
Given my stance on top-down it is heartening to see that project definition is what often guides these systems. No code is cut until after project definition which in turn provides the specification of the system. So what happens when they change things? They negotiate the change and provide a plan for the introduction of the change. It cannot be done on the fly because the complexity of the change often mitigates against a more agile approach. So they simulate they test and so on. What they have is very good governance processes which document the change, the impact and the resulting tests prior to productionisation. All similar to the approaches we use in commerce oriented solutions without simulation and without testing the architecture in any way.
The role of project definition is to provide the requirements against which testing can be measured and against which simulation can occur. The simulation provides a first step towards testable architecture ensuring that the overal design is commensurate with the requirements.
As an aside High Integrity Software development became a real vogue in the 1980's. One of the earliest proponents of formal methods which have underpinned High Integrity Software has been Tony Hoare (Elliot Brothers 1960 until 1968) who oddly enough was at Elliot Brothers during my dad's tenure (John Talbot 1961 until 1965 and then again 1968 until retirement in the 1990's).
So what does this mean to the world of software and commerce. Formal methods are valuable but not a panacea. They need to be introduced by stealth with their benefits laid out. They need to be employed early. As Anthony Hall elaborates:
"It is well known that the early activities in the lifecycle are the most important. According to the 1995 Standish Chaos report , half of all project failures were because of requirements problems. It follows that the most effective use of formal methods is at these early stages: requirements analysis, specification, high-level design. For example it is effective to write a specification formally rather than to write an informal specification then translate it .It is effective to analyse the formal specification as early as possible to detect inconsistency and incompleteness. Similarly, defining an architecture formally means that you can check early on that it satisfies key [functional and non-functional] requirements such as [message order and content], security [and performance SLA's]."
Formal method have been used in avionics for some time. Hence Tony Hoare's involvement in Elliot Brothers. They are for he most part hidden and become a function of tools that are used for design. The use of the Z notation in the early 1980's found popular acclaim in Ada based systems. The problem was it's cryptic nature and therefore lack of skills in using it. Which is why stealth is a good thing. Tie it up in a tool and make it easy (just like type systems in programming languages).
We have a much clearer formal understanding today of distributed computing through the work of Tony Hoare and Robin Milner. What is needed are tools to help us define architectures that remove the ambiguity of human translation and provide a mechanism for the analysis that is needed, henace the cry for formalism. The pi4soa tool suite is but a start. It can become more refined and integrated with other tools (such as Archimate and those specific tools that support Archimate). Architecture tooling and tooling for design is not the most popular of directions because of the lack of runtime scale for remuneration. But they are much needed as in the end they will enable solutions to be build faster, with higher quality and at lower cost whilst remaining suitable agile and aligned to the business and this is what formalism (suitably hidden) can provide.
I was talking to my dad over the weekend. He was involved in avionics most of his working life and was the examiner of the institute of Quality Assurance in the UK. We were talking about the Farnborough Airshow - which I attended on Friday - and about fly by wire. The European Fighter Aircraft (Typhoon) is a fly by wire as is the A 380. I don't know about anyone else but the standard of software development would make me concerned if I had to fly in one of those. I try to forget it all when I board an aircraft so I don't worry.
It turns out that much of what they do to increase the reliability of fly-by-wire is to use redundancy. Triple systems is one approach that is used. The components are developed independently and then monitored. When they diverge majority voting is used to determine the likely correct behavior. It is statistical because there is no guarantee that the behavior of the two that agree is correct. It is simply the case that two agreeing is likely to lead to correctness.
In high availability system of this nature they use multiple compliers too and they might choose to use different chip sets.
Given my stance on top-down it is heartening to see that project definition is what often guides these systems. No code is cut until after project definition which in turn provides the specification of the system. So what happens when they change things? They negotiate the change and provide a plan for the introduction of the change. It cannot be done on the fly because the complexity of the change often mitigates against a more agile approach. So they simulate they test and so on. What they have is very good governance processes which document the change, the impact and the resulting tests prior to productionisation. All similar to the approaches we use in commerce oriented solutions without simulation and without testing the architecture in any way.
The role of project definition is to provide the requirements against which testing can be measured and against which simulation can occur. The simulation provides a first step towards testable architecture ensuring that the overal design is commensurate with the requirements.
As an aside High Integrity Software development became a real vogue in the 1980's. One of the earliest proponents of formal methods which have underpinned High Integrity Software has been Tony Hoare (Elliot Brothers 1960 until 1968) who oddly enough was at Elliot Brothers during my dad's tenure (John Talbot 1961 until 1965 and then again 1968 until retirement in the 1990's).
So what does this mean to the world of software and commerce. Formal methods are valuable but not a panacea. They need to be introduced by stealth with their benefits laid out. They need to be employed early. As Anthony Hall elaborates:
"It is well known that the early activities in the lifecycle are the most important. According to the 1995 Standish Chaos report , half of all project failures were because of requirements problems. It follows that the most effective use of formal methods is at these early stages: requirements analysis, specification, high-level design. For example it is effective to write a specification formally rather than to write an informal specification then translate it .It is effective to analyse the formal specification as early as possible to detect inconsistency and incompleteness. Similarly, defining an architecture formally means that you can check early on that it satisfies key [functional and non-functional] requirements such as [message order and content], security [and performance SLA's]."
Formal method have been used in avionics for some time. Hence Tony Hoare's involvement in Elliot Brothers. They are for he most part hidden and become a function of tools that are used for design. The use of the Z notation in the early 1980's found popular acclaim in Ada based systems. The problem was it's cryptic nature and therefore lack of skills in using it. Which is why stealth is a good thing. Tie it up in a tool and make it easy (just like type systems in programming languages).
We have a much clearer formal understanding today of distributed computing through the work of Tony Hoare and Robin Milner. What is needed are tools to help us define architectures that remove the ambiguity of human translation and provide a mechanism for the analysis that is needed, henace the cry for formalism. The pi4soa tool suite is but a start. It can become more refined and integrated with other tools (such as Archimate and those specific tools that support Archimate). Architecture tooling and tooling for design is not the most popular of directions because of the lack of runtime scale for remuneration. But they are much needed as in the end they will enable solutions to be build faster, with higher quality and at lower cost whilst remaining suitable agile and aligned to the business and this is what formalism (suitably hidden) can provide.
Tuesday, 1 July 2008
The Industrialisation of IT
Possible the most important invention that gave rise to the industrial revolution was the micrometer. The inventor of the micrometer was William Gascoigne in the 17th century. It was directly responsible for the engineering discipline in constructing the steam engine and in constructing the Enfield rifle that was used during the civil war in the United States.
What the micrometer did was remove ambiguity. It gave rise to a language of design that enabled precision engineering which by extension gave rise to industrialisation with bullets being made in one place and gun barrels in another.
What has this got to do with CDL?
CDL is possibly as important in the industrialisation of IT. It gives us a language of precision of a system of services which in turn ensures that services are precise (by design) and so interoperate properly - just as the micrometer did for Enfield and Stevensons Rocket.
In classic engineering today simulation is used along with some formal mathematics to test out a design to ensure that it will work. In CDL the same principle is used to simulate and to test so that before a line of code is cut the CDL description is shown to be valid against the requirements and to be correct in computational terms (i.e. free from live locks, dead locks and race conditions).
Testable architecture along with a language of discourse that is precise removes the ambiguity between implementation and requirements. It enables industrialisation and facilitates off shoring of implementation in the same way that Enfield used the micrometer and the precision it gave to design to manufacture solutions in different locations and yet ensure that things work when they are put together.
What the micrometer did was remove ambiguity. It gave rise to a language of design that enabled precision engineering which by extension gave rise to industrialisation with bullets being made in one place and gun barrels in another.
What has this got to do with CDL?
CDL is possibly as important in the industrialisation of IT. It gives us a language of precision of a system of services which in turn ensures that services are precise (by design) and so interoperate properly - just as the micrometer did for Enfield and Stevensons Rocket.
In classic engineering today simulation is used along with some formal mathematics to test out a design to ensure that it will work. In CDL the same principle is used to simulate and to test so that before a line of code is cut the CDL description is shown to be valid against the requirements and to be correct in computational terms (i.e. free from live locks, dead locks and race conditions).
Testable architecture along with a language of discourse that is precise removes the ambiguity between implementation and requirements. It enables industrialisation and facilitates off shoring of implementation in the same way that Enfield used the micrometer and the precision it gave to design to manufacture solutions in different locations and yet ensure that things work when they are put together.
Monday, 23 June 2008
Response to Luis's comments in my last blog entry
Firstly thank you for the comment Luis.
It has been a little while since I last blogged. In that time the latest Pi4SOA release came out. Some of it's capabilities are listed below in the narrative.
To get the latest release follow the instructions at the end.
On to my response to Luis's comments.
Luis wrote:
Very good article indeed. It clearly describes the main doubts most people have regarding the real added value of CDL and how to addressed it. It is clear vendors are more worried about short term ROI other than best practices and real heterogeneous enterprise architecture; evidence of this is how vendors sell “SOA Suites” as a product / package and not as an architecture approach or practice as it should be (this is probably one of the reasons why many SOA projects end up in DOA –dead on arrival– instead of addressing the real business needs and technological value)
One suggestion; it would be nice to mention how and when CDL can be plug together with other technologies such as BPMN, SCA and BPEL on the different stages of a SOA project and the roll it has one the solution is in place (e.g. in the governance).
Here is my response:
The Pi4SOA tools suite currently supports BPMN. It is simply and export format in which CDL is exported to BPMN. However CDL is richer that BPMN so it is not free from loss of the expressions in CDL. However it may serve the role of review collateral.
BPEL is also supported. You can generate BPEL just as you can generate Java. The downside is that BPEL is more restrictive than CDL because CDL is wider than classic web services and provides message exchange patterns that BPEL cannot handle but which are applicable to SOA (i.e. first class notifications). The BPEL generation is still prototypical as it does not garner much demand. However I think that Overlord will change that.
As to SCA, no current bindings exist to SCA but I do not think that SCA in and of itself is really a target for CDL as it fits two level below rather than one level below. One might of course make the association of SCA to ESB in which case the latest release supports JBossESB out of the box. The problem with ESB's is that there is not real standard. Rather an ESB is a collection of integrated components (Message Bus, Orchestration, Registry, Business Rules and adapters).
Governance is all about control and traceability. This is exactly what CDL provides as an SOA blueprint for a dynamic model. It is the language of architecture in which Java, C# and BPEL provide business logic inside services and in which CDL describes the collaborative behavior of the services as peers. It provides traceability back to requirements (example messages and sequence diagrams) today and in the future will also deal with the non-functionals (SLA's WS-Policy statements and so on) and it provides total coherence to implementation through Java generation, UML (XMI model) generation, and BPEL generation as well as providing runtime checking of behavior against the description. So it provides top to bottom alignment from requirements to implementation. In this way it provides a governance platform that can help to manage the complexity of change over a large real-estate of services. Which is why Cognizant and Redhat are interested in CDL.
Downstream we might try to join it up with ArchiMate, which will provide the route to requirements from the business. But that is some time away and work has yet to start. If anyone is interested then contact me at at the Foundation (see below).
To use the latest release follow the steps below:
1. Go to www.pi4soa.org
2. Select "Download->Browse All Files
3. Press the green "Download" button
4. Select the appropriate platform and download (See below)
pi4soa2.0.0.CR1-eclipse3.3.2-linux.tar.gz
pi4soa2.0.0.CR1-eclipse3.3.2-macosx.zip
pi4soa2.0.0.CR1-eclipse3.3.2-win32.zip
pi4soa2.0.0.CR1-Release-Note.pdf
pi4soa2.0.0.CR1-withsrc.zip
pi4soa2.0.0.CR1.zip
pi4soa.sar
I also have lots of example which I am happy to share. To get them you will need to email me at pi4tech (steve@pi4tech.com)
It has been a little while since I last blogged. In that time the latest Pi4SOA release came out. Some of it's capabilities are listed below in the narrative.
To get the latest release follow the instructions at the end.
On to my response to Luis's comments.
Luis wrote:
Very good article indeed. It clearly describes the main doubts most people have regarding the real added value of CDL and how to addressed it. It is clear vendors are more worried about short term ROI other than best practices and real heterogeneous enterprise architecture; evidence of this is how vendors sell “SOA Suites” as a product / package and not as an architecture approach or practice as it should be (this is probably one of the reasons why many SOA projects end up in DOA –dead on arrival– instead of addressing the real business needs and technological value)
One suggestion; it would be nice to mention how and when CDL can be plug together with other technologies such as BPMN, SCA and BPEL on the different stages of a SOA project and the roll it has one the solution is in place (e.g. in the governance).
Here is my response:
The Pi4SOA tools suite currently supports BPMN. It is simply and export format in which CDL is exported to BPMN. However CDL is richer that BPMN so it is not free from loss of the expressions in CDL. However it may serve the role of review collateral.
BPEL is also supported. You can generate BPEL just as you can generate Java. The downside is that BPEL is more restrictive than CDL because CDL is wider than classic web services and provides message exchange patterns that BPEL cannot handle but which are applicable to SOA (i.e. first class notifications). The BPEL generation is still prototypical as it does not garner much demand. However I think that Overlord will change that.
As to SCA, no current bindings exist to SCA but I do not think that SCA in and of itself is really a target for CDL as it fits two level below rather than one level below. One might of course make the association of SCA to ESB in which case the latest release supports JBossESB out of the box. The problem with ESB's is that there is not real standard. Rather an ESB is a collection of integrated components (Message Bus, Orchestration, Registry, Business Rules and adapters).
Governance is all about control and traceability. This is exactly what CDL provides as an SOA blueprint for a dynamic model. It is the language of architecture in which Java, C# and BPEL provide business logic inside services and in which CDL describes the collaborative behavior of the services as peers. It provides traceability back to requirements (example messages and sequence diagrams) today and in the future will also deal with the non-functionals (SLA's WS-Policy statements and so on) and it provides total coherence to implementation through Java generation, UML (XMI model) generation, and BPEL generation as well as providing runtime checking of behavior against the description. So it provides top to bottom alignment from requirements to implementation. In this way it provides a governance platform that can help to manage the complexity of change over a large real-estate of services. Which is why Cognizant and Redhat are interested in CDL.
Downstream we might try to join it up with ArchiMate, which will provide the route to requirements from the business. But that is some time away and work has yet to start. If anyone is interested then contact me at at the Foundation (see below).
To use the latest release follow the steps below:
1. Go to www.pi4soa.org
2. Select "Download->Browse All Files
3. Press the green "Download" button
4. Select the appropriate platform and download (See below)
pi4soa2.0.0.CR1-eclipse3.3.2-linux.tar.gz
pi4soa2.0.0.CR1-eclipse3.3.2-macosx.zip
pi4soa2.0.0.CR1-eclipse3.3.2-win32.zip
pi4soa2.0.0.CR1-Release-Note.pdf
pi4soa2.0.0.CR1-withsrc.zip
pi4soa2.0.0.CR1.zip
pi4soa.sar
I also have lots of example which I am happy to share. To get them you will need to email me at pi4tech (steve@pi4tech.com)
Wednesday, 7 May 2008
Show me the money in CDL
Summary
The value of CDL is that it takes us from art to engineering by ensuring good governance that is supported by tools throughout. It gets you to a solution much faster and more accurately with higher quality. It does this through formal validation of CDL descriptions against requirements, through monitoring of services against CDL descriptions and directed guidance of services construction through generation based on a CDL description. The formal tie in of a description to requirements and the linkage directly to monitoring and generation is what provides quality and time to market coupled with the level at which CDL as a description languages operates. WS-CDL (CDL) has been around now for some time. The Pi4SOA tool suite has found great favour among vertical standards (ISDA, ISO, TWIST, HL7 and many others). Researchers continue to plough a furrow towards ever more interesting analysis of CDL as well as looking towards son-of-CDL. With Redhat's announcement of their SOA governance project (Overlord) and the central role played by CDL and the Pi4SOA tool suite one could be forgiven for assuming that SOA platforms vendors and tool vendors show interest in CDL and are actively looking at working on it and providing it to their communities.
So why is this not the case?
Some of it is still rooted in the adverse and misleading publicity that CDL received at the hands of some analysts and many large vendors. The deliberate confusion created in the minds of the population at large through the BPEL vs CDL debate caused much damage and was founded on many many fallacies and misunderstandings. But perhaps the fundamental reason for the lack of tools from vendors is rooted in the classic question "show me the money".
Despite some large corporates asking for CDL tooling, vendors have not really provided anything at all. And so we are left with a few tools largely open source from smaller players.
The SOA platform vendors make their money on run times not on design so they are focussed on execution and not description. This is why VC's often say you cannot make money from tools. It is why Eclipse is open source. So Java, BPEL J2EE Application Servers, ESB's and all of that stuff is all about execution and the scale that run time brings to revenue for vendors.
CDL is a description language not an executable language. At first glance it is not clear where there is any revenue. It has no runtime component. It's value at first glance is in clean concise accurate descriptions of System Architectures in terms of their observable behavior and message exchanges. So it is all about design and not execution.
One might argue that WSDL is also a description and yet has value. But that value is entirely driven by its use in execution. To have Web Services a WSDL description is needed. The attendant software to type check, marshal and un-marshall operations and data and to ensure correct message exchange patterns are adhered to is all part of the run time machinery needed to use WSDL.
WSDL has another value in that it is a language to describe the technical contract of a single service. It might not capture behaviour but it does capture the functional footprint and data types that a service needs to be useful at all. But this value, as a contract, is not why SOA Platform vendors invest in WSDL. The reason they invest is because they can clearly see the value in their run time environments and so the revenue that accrues.
So what of CDL? As with WSDL, CDL is a very good language for describing all of the contracts needed for a set of services. It describes the collaborative contract that is, in effect, a System Architecture description. But that, as far as SOA Platform vendors would be concerned, does not generate sufficient revenue to justify investment.
On the other hand there are many UML vendors who provide design time tools that users do buy. So there is a value. IBM have Rational, Magic have Magic Draw and so on. But it is not a big revenue ticket business. It is niche.
CDL can play a role in the execution phase and this is where the key revenue value can be clearly seen. If we have a System Architecture description that described the externally observable behavior of services then we can monitor the run time services against that description. This technique is like BAM but is really BAM with teeth as we can measure what we observe against what we expect. The result is monitoring, or governing, services such that we can determine where problems exist based on an artefact that also governs the change of the system as whole and may be used to drive evolution. This is why CDL is a key part of Redhat's SOA Governance project called Overlord.
From a user perspective, that is some entity wishing to deliver a solution, the value is more profound. On one side CDL provides possibly the very first formalised way of delivering an artefact, the System Architecture description, that can be validated against requirements and so yields testable architecture. This artefact represents to TO-BE state of a system. On the other side a CDL description can be used to describe, in much the same way, the AS-IS state of the system. In this case the artefact can be validated by automated observation (monitoring) to ensure that it truly represent the AS-IS state. When there is no variance in the monitoring we have a stable AS-IS description.
So CDL provides a means of having testable System Architectures for a TO-BE description and provides a means of defining an AS-IS description and means of validating that description against an existing IT real estate.
Clearly the difference between the AS-IS and TO-BE state represents the scope of work to be carried out. Because CDL is a formal description, analysis to identify the gap, between AS-IS and TO-BE, is much easier, faster and can be supported by tools to ensure that the gap is correctly identified. Furthermore the changes to the AS-IS state can be managed over the lifetime of a programme of work with continuous real time governance, through monitoring, to ensure that the solution delivered is correct at all times.
The value of CDL, in a nutshell, as has been blogged before, is that it takes us from art to engineering by ensuring good governance, that is supported by tools throughout. It gets you to a solution much faster and more accurately with higher quality.
As Redhat start to use the monitoring aspects of CDL SOA vendors will inevitably move towards it and I would anticipate SOA Governance and BAM to settle on CDL.
Sunday, 16 March 2008
The end of coding as we know it?
I was talking to a friend and colleague, who shall remain nameless, about the use of models as a principle means of deriving applications. Oddly enough, the day before, I was also talking to one of my new colleagues at Cognizant about something not dissimilar. In the former case there is at least one (and probably many) organisation who now seek to reduce the coding burden and have made efforts to turn their coding shops into testing shops with a little coding on the side. In the case of the latter we were talking about the real IP being in the process models and not the coding.
Clearly there is much work in MDA circles to solve these problems. After all there have been attempts at moving to executable UML Few if any have really succeeded. And those that come close tend to do so based on a more siloed view of an application. There are also initiatives within OMG to codify process models based on BPDM and relate them back to UML. This latter move is of considerable interest because on the one hand it recognises that UML today does not facilitate the encoding of business processes and it recognises the need for some description of the peered observable behavior of a set of roles which we might call a choreography description.
Can we really move towards a world in which models drive everything else and do so automatically? And if we can what do these models need to provide. What are the requirements?
I would contend that any such model, we might call it a dynamic blueprint for a SOA, needs to fulfill at least the following requirements:
Requirement 1 really states that any model must be able to describe the way in which services (which might be the embodiment of a role) exchange information and the ordering rules by which they do so. The type of information exchanged might be given as a static model (requirement 2). The ordering rules would be the conditional paths, loops, parallel paths that constitute the collaborative behavior of the model with respect to the peered roles.
Requirement 2 simply states that a static information or data model is required and that this can either be in place when creating a dynamic model or it may be created along side the dynamic model which would provide context for the information types. When we iterate between sequence diagrams and static data models today this is essentially what we do anyway. The difference is that the dynamic model is also complete unlike the sequence diagrams which provide context on the basis of the scenario that they represent.
Requirement 3 says that any dynamic model should not dictate any specific physical implementation, that is it should not require a solution to be hub and spoke, peered, hierarchical and so on. It should be capable of being implemented in a range of physical architectures which are independent of the dynamic model.
Requirements 4 to 6 say that a model must be subject to and support various forms of automatic verification just like programming languages are today when we compile them and the compiler picks up errors and so prevents the code from being made executable. In the case of a dynamic model we would want to ensure that the dynamic model meets a set of requirements for the domain that it represents. This might be achieved by validating a dynamic model against an agreed set of messages and an agreed set of sequence diagrams which collectively describe one or more use cases. On the other hand we would want to use a validated dynamic model, which as a result of validation we know meets our requirements, to verify that an implementation of that model conforms to the model. That is that there does not exist any observed execution of the implementation across all of it's constituent services any set of observable exchanges or conditions that cannot be directly mapped to the dynamic model. Putting it another way we want to use the dynamic model as input to some form of runtime governance applied to the behavior of our set of peered services. The requirements that mention liveness, live locks and so on are really not any different to saying that in any programming language it is illegal to access an array of 10 elements by writing x = array[11]. The difference is that we are looking to prevent badly formed and potentially disastrous problems arising in a distributed system and not in a single application as do compilers.. Model checking applied to a dynamic model for distributed systems is one way of ensuring that this does not happen in much the same way that type checking prevents errors at a localised application level. I mentioned something akin to this in my blog on the workshop I attended which was entitled "OO Languages with session types ".
Requirements 7 states that a model must be able to be simulated. What this means in practice is that if a dynamic model captures the collaborative behavior of a set of peered roles then we must be able to provide such a model with some input data and see the dynamic model activated. For example if a dynamic model starts with the offering of a product then we must be able to direct it to some product information and see the exchanges that then occur. Equally if we introduce a number of bidders in an auction system we need to be able to enact the choreography.
Requirement 8 is all about reviewability - if such a word exists. Simply stated it is the ability to generate or display a dynamic model in a way that reviewers can understand, comment and so sign off on.
If we had a language that we could use to describe such dynamic models, and of course I would contend that WS-CDL is a good starting point along with BPDM, and if you are interested in the future then look at Scribble too, then what does this means for for the software market as whole? In simple terms it changes the shape and size of delivery and has an impact on testing. It compresses things.
On the one hand we can view the dynamic model as UML artefacts empowering implementors. If we know that the dynamic model is correct with respect to the requirements, and we know that the dynamic model is correct with respect to any unintended consequences (aka liveness) then we can be sure that the implementors will have a precise and correct specification in UML of what they should write on a per service/role basis and so ensure that all the implemented services will not have any integration problems. It makes it much more efficient to outsource development because the dynamic modeling can be done close to the domain and the development can be done where it is most cost effective to do so - hooray for off shore development. Coupled with the ability to use the dynamic model as input for testing it also becomes possible to verify that a service is playing the correct role as it is being executed in testing.
On the other hand, if we can generate Java do we really need coders? And this is the dilemma. Can we really do without coders. If the high level dynamic model only deals with the externally observable behavior then somehow we still need the internal behavior (the business logic). If the internal behavior can be described fully in UML and code subsequently generated we can indeed generate everything. The dynamic model of the system as a whole plus the UML models for service business logic combine to provide a two step high level description of a system in which no code needs to be specified at all. So no coders? Of course it sounds too good to be true. Where does the code go? Where are the actions specified? In UML this could be done using an appropriate action language, something that has found it's way into UML2.0 but as yet not fully formed as a concrete language. Someone still has to write this stuff, so the coders just move up the stack and become more productive as they did with the onset of OO languages as a whole.
Is this the end of coding as we know it? One thing is for sure it is not the end right now. How far and how fast the growing wave towards modeling as opposed to coding will take us, for me at least I cannot see it is all the way. The action semantics still need to be coded or written textually and that is really coding by another name. I remember in the early 1990's much was made of visual programming languages. None of them made it.
What it is all about is structure and making the structure of things visible and so easier to manipulate and that is a huge leap forward because we simply have never had anything that enforces structure for a distributed system before.
In the grand scheme of things the very fact that we can articulate such possibilities (the end of coding as we know it) means that our industry as a whole is maturing and our understanding of the complexity inherent in distributed systems (SOA and the rest) is becoming clearer every day. It does not mean that we are there yet but because we can now think about the requirements of a language needed to describe such structure we are at least on the right path.
Clearly there is much work in MDA circles to solve these problems. After all there have been attempts at moving to executable UML Few if any have really succeeded. And those that come close tend to do so based on a more siloed view of an application. There are also initiatives within OMG to codify process models based on BPDM and relate them back to UML. This latter move is of considerable interest because on the one hand it recognises that UML today does not facilitate the encoding of business processes and it recognises the need for some description of the peered observable behavior of a set of roles which we might call a choreography description.
Can we really move towards a world in which models drive everything else and do so automatically? And if we can what do these models need to provide. What are the requirements?
I would contend that any such model, we might call it a dynamic blueprint for a SOA, needs to fulfill at least the following requirements:
- A dynamic model MUST be able to describe the common collaborative behaviors of a set of peered roles.
- A dynamic model MUST NOT dictate any one physical implementation.
- A dynamic model MUST but be verifiable against requirements.
- An implementation of a dynamic model MUST be verifiable against that dynamic model.
- A dynamic model MUST be verifiable against liveness properties (freedom from dead locks, live locks and race conditions).
- A dynamic model MAY be shown to be free from live locks deadlocks and race conditions.
- A dynamic model MUST be able to be simulated based on a set of input criteria.
- A dynamic model MUST enable generation of role based state behaviours to a range of targets including but not limited to UML activity diagrams, UML state charts, Abstract WS-BPEL, WS-BPEL, WSDL, Java and C#.
Requirement 1 really states that any model must be able to describe the way in which services (which might be the embodiment of a role) exchange information and the ordering rules by which they do so. The type of information exchanged might be given as a static model (requirement 2). The ordering rules would be the conditional paths, loops, parallel paths that constitute the collaborative behavior of the model with respect to the peered roles.
Requirement 2 simply states that a static information or data model is required and that this can either be in place when creating a dynamic model or it may be created along side the dynamic model which would provide context for the information types. When we iterate between sequence diagrams and static data models today this is essentially what we do anyway. The difference is that the dynamic model is also complete unlike the sequence diagrams which provide context on the basis of the scenario that they represent.
Requirement 3 says that any dynamic model should not dictate any specific physical implementation, that is it should not require a solution to be hub and spoke, peered, hierarchical and so on. It should be capable of being implemented in a range of physical architectures which are independent of the dynamic model.
Requirements 4 to 6 say that a model must be subject to and support various forms of automatic verification just like programming languages are today when we compile them and the compiler picks up errors and so prevents the code from being made executable. In the case of a dynamic model we would want to ensure that the dynamic model meets a set of requirements for the domain that it represents. This might be achieved by validating a dynamic model against an agreed set of messages and an agreed set of sequence diagrams which collectively describe one or more use cases. On the other hand we would want to use a validated dynamic model, which as a result of validation we know meets our requirements, to verify that an implementation of that model conforms to the model. That is that there does not exist any observed execution of the implementation across all of it's constituent services any set of observable exchanges or conditions that cannot be directly mapped to the dynamic model. Putting it another way we want to use the dynamic model as input to some form of runtime governance applied to the behavior of our set of peered services. The requirements that mention liveness, live locks and so on are really not any different to saying that in any programming language it is illegal to access an array of 10 elements by writing x = array[11]. The difference is that we are looking to prevent badly formed and potentially disastrous problems arising in a distributed system and not in a single application as do compilers.. Model checking applied to a dynamic model for distributed systems is one way of ensuring that this does not happen in much the same way that type checking prevents errors at a localised application level. I mentioned something akin to this in my blog on the workshop I attended which was entitled "OO Languages with session types ".
Requirements 7 states that a model must be able to be simulated. What this means in practice is that if a dynamic model captures the collaborative behavior of a set of peered roles then we must be able to provide such a model with some input data and see the dynamic model activated. For example if a dynamic model starts with the offering of a product then we must be able to direct it to some product information and see the exchanges that then occur. Equally if we introduce a number of bidders in an auction system we need to be able to enact the choreography.
Requirement 8 is all about reviewability - if such a word exists. Simply stated it is the ability to generate or display a dynamic model in a way that reviewers can understand, comment and so sign off on.
If we had a language that we could use to describe such dynamic models, and of course I would contend that WS-CDL is a good starting point along with BPDM, and if you are interested in the future then look at Scribble too, then what does this means for for the software market as whole? In simple terms it changes the shape and size of delivery and has an impact on testing. It compresses things.
On the one hand we can view the dynamic model as UML artefacts empowering implementors. If we know that the dynamic model is correct with respect to the requirements, and we know that the dynamic model is correct with respect to any unintended consequences (aka liveness) then we can be sure that the implementors will have a precise and correct specification in UML of what they should write on a per service/role basis and so ensure that all the implemented services will not have any integration problems. It makes it much more efficient to outsource development because the dynamic modeling can be done close to the domain and the development can be done where it is most cost effective to do so - hooray for off shore development. Coupled with the ability to use the dynamic model as input for testing it also becomes possible to verify that a service is playing the correct role as it is being executed in testing.
On the other hand, if we can generate Java do we really need coders? And this is the dilemma. Can we really do without coders. If the high level dynamic model only deals with the externally observable behavior then somehow we still need the internal behavior (the business logic). If the internal behavior can be described fully in UML and code subsequently generated we can indeed generate everything. The dynamic model of the system as a whole plus the UML models for service business logic combine to provide a two step high level description of a system in which no code needs to be specified at all. So no coders? Of course it sounds too good to be true. Where does the code go? Where are the actions specified? In UML this could be done using an appropriate action language, something that has found it's way into UML2.0 but as yet not fully formed as a concrete language. Someone still has to write this stuff, so the coders just move up the stack and become more productive as they did with the onset of OO languages as a whole.
Is this the end of coding as we know it? One thing is for sure it is not the end right now. How far and how fast the growing wave towards modeling as opposed to coding will take us, for me at least I cannot see it is all the way. The action semantics still need to be coded or written textually and that is really coding by another name. I remember in the early 1990's much was made of visual programming languages. None of them made it.
What it is all about is structure and making the structure of things visible and so easier to manipulate and that is a huge leap forward because we simply have never had anything that enforces structure for a distributed system before.
In the grand scheme of things the very fact that we can articulate such possibilities (the end of coding as we know it) means that our industry as a whole is maturing and our understanding of the complexity inherent in distributed systems (SOA and the rest) is becoming clearer every day. It does not mean that we are there yet but because we can now think about the requirements of a language needed to describe such structure we are at least on the right path.
Monday, 25 February 2008
General news
Not often I blog on something quite so general and to keep your interest I do intend to do a major piece on SOA for business and management rather than just a the usual techie dimension. It just takes a while to get my thoughts in order to do it.
On a more general topic I have changed jobs. Having been an entrepreneur for the last 10 years I have decided that I wish to devote time and energy to doing what I can in a larger organisation but one that is fast enough moving that I can use my entrepreneurial skills in. To this end I am now working for Cognizant as the lead architect for their Advanced Solutions Practice in Europe. All very exciting stuff because these guys are really growing fast and that gives me a whole lot of problems based on methodology and technology support for it to focus on.
I'm still very much Pi4 Tech and still associated with Hattrick (non-exec on the board).
I'm looking forward to bringing the experiences from my new role at Cognizant into the blogsphere.
On a more general topic I have changed jobs. Having been an entrepreneur for the last 10 years I have decided that I wish to devote time and energy to doing what I can in a larger organisation but one that is fast enough moving that I can use my entrepreneurial skills in. To this end I am now working for Cognizant as the lead architect for their Advanced Solutions Practice in Europe. All very exciting stuff because these guys are really growing fast and that gives me a whole lot of problems based on methodology and technology support for it to focus on.
I'm still very much Pi4 Tech and still associated with Hattrick (non-exec on the board).
I'm looking forward to bringing the experiences from my new role at Cognizant into the blogsphere.
Subscribe to:
Posts (Atom)