Monthly Archives: April 2005

Distracted by abstractions

Reading Craig’s post again I had to go back and re-read Joel’s Law on Abstractions .

It reminded me of how I’ve been thinking about BizTalk and the abstractions that it provides.   As you can probably tell from my level of interest (Shannon – my SO –  would call it an obsession or perserveration) and number of posts  I think BizTalk is a incredible product, so much so that I’ve dedicated my work life to it over any other technology (considering the size and complexity of BizTalk that might not have really been a choice ;-)).  That said – all products that create abstractions are inherently leaky (again read carefully – this doesn’t make BizTalk a bad product in any way). 

IMO the two key abstractions that are the “leakiest” in BizTalk are its integration with the CLR and its pub/sub architecture.  Be very clear about what I mean here – I do not mean that BizTalk’s integeration with the CLR is *bad*.  Far from it.  What I mean is that in order to be competent at using BizTalk you have to understand the CLR very well.  You have to know exactly how the CLR works, and then you have to understand how BizTalk uses and interacts with it.  Failing to do this will lead to hours of unproductive BizTalk development time.

The pub/sub architecture in BizTalk is the same.  It is a brilliant design IMO.  But you can start to develop in BizTalk without even knowing there is a pub/sub architecture (create a simple BizTalk application – nowhere does it ask you to set up a subscription) *until* you get your first “no matching subscription found for the message X” error message.

What does this all mean? To quote Joel – “there is no free lunch”.  Development is hard, BizTalk is hard, the CLR is hard, XML and Web Services are hard.  Gain power from knowledge.

More on Wikis

So I finally got my Wiki up (thanks to Craig again, since he helped me out again this morning with some formatting issues). Funny thing is – Aaron started one (I see that someone already cross-linked my Wiki – maybe I will like this Wiki stuff :)) on the same day. It’s an odd convergence. I wanted to also acknowledge AndrĂ© Dammeyer (a fellow newly awarded BizTalk MVP) – who has been wiki’ing on BizTalk for months (and who actually gave me the idea to store my BizTalk info in a Wiki). IMO there really can’t be enough information about BizTalk out there for people to take advantage of.

I've jumped on the Wiki bandwagon

Turns out blogs aren’t so great at organizing data.  I could have built a custom content management system for my site – but I decided to take a dive into the Wiki world.  Not sure how I feel about it yet – still can’t get use to not typing html tags – but so far I am pretty happy ( I am sure my happiness has something to do with the fact that I am using the FlexWiki project that my Minnesota buddy Craig Andera has worked so hard on).  Many of the students in the many BizTalk courses I have taught over the past year have asked me over and over to have one consolidated place to list all the samples and tools I post and a Wiki seemed like the place to do that.  We’ll see if it sticks with me :)

So anyway – here is the link to a list of my BizTalk tools and samples, links to BizTalk resources in general coming soon.

Yet another update to the ContextAdder Pipeline Component

I made this fix last week but am I just having time to post about it.  This was actually a pretty minor bug that turned out to be really really hard to fix.  You can continue to download bug fixed version from here

ContextAdder is a Pipeline Component that can be used at any pipeline stage (in either a receive or a send pipeline) to add context properties. Unlike most other situations where context properties are dynamic (i.e. they come from the message itself or the surrounding execution context) sometimes all you need is one property in the context for a particular scenerio in BizTalk to work.  So I built ContextAdder which is a generic pipeline component so I didn’t have to build a new pipeline component everytime I wanted to hard code some context property into a pipline. 

Anyway – the bug presented itself in the following scenerio – if you opened a pipeline file, changed the Context property information (either the name,value, namespace uri, or whether you wanted to promote the property or not)  and then did a build without explicitly saving, the pipeline designer and the ContextAdder lost the change.  If you saved manually before building everyting was fine.

It turns out that the issue was – the way the UITypeEditor works with the Property Grid in VS.NET didn’t cause the pipeline editor to think the document was “dirty” even though clearly it was.  The bug will present itself in any pipeline component that has a property which is a collection or had a UITypeEditor associated with it, it appears that the pipeline editor relies in general on the Property Grid to inform it when a pipeline has been modified and is “dirty”.

So I spend some time looking at the implementation of UITypeEditors for the built-in pipeline components (like the XML Dissasembler) and tried to mimic their functionality in my own custom UITypeEditor , which didn’t work.  Obviously I was missing the black magic (what they seem to do is to get an interface reference to a COM object and release it).  So I hacked it :).  I reached up through reflection inside of my UITypeEditor and manually set the pipeline document dirty flag to true.  If you download the sample you can see the code inside of the UITypeEditor derived class inside of the ContextAdderHelper assembly (I needed to add another assembly to the solution to make all the UITypeEditor goo to work correctly).  Anyway I assume I’ll submit this as a bug, there must be some sanctioned way to notify the pipeline editor that a document is dirty without resorting to reflection, if anyone knows what that is – please let me know!  Until then, enjoy the ContextAdder if it helpful to any of you.

New performance article and perf tip

If you are concerned about BizTalk and performance (well who isn’t ;-) – there is a great article here about managing performance and testing it as well.  One thing to keep in mind (that as far as I can see the article doesn’t mention) is that when you are testing performance (and when you are doing a production install) make sure to switch your build environment to “Deployment” instead of “Development”.  In a BizTalk project “Development” is the equivalent to a Debug build for regular .NET projects.  “Deployment” is the equivalent to a “Release” build.  You may see a small or large increase in perf when you switch to Deployment, but you should get better perf from that build, just like if you were doing regular .NET development.

The reason is that when Debug build is done, an attribute (DebuggableAttribute) is added at the assembly level to tell the JIT compiler how it can treat code from that assembly.  The DebuggableAttribute has two constructor arguments – IsJITOptimizerDisabled and IsJITTrackingEnabled, a C# assembly in a Debug build will have both of those properties set to true.  The BizTalk complier only sets IsJITTrackingEnabled to true so it isn’t slowing things down as much as it could be – but it is still slower than if the JIT’er doesn’t have to track.  IsJITOptimizerDisabled is the property that lowers performance the most – but remember if you are using .NET assemblies in your BizTalk apps, those assemblies will have JIT optimizations turned off for Debug builds.  So make sure to do a Release build before you do any perf testing or before you move to a production machine.  Just remember – when you test against CLR code and the JIT’er isn’t running at full speed  – you will not see accurate performance numbers.

Am I missing something?

Can anyone out there in BizTalk developer land who reads my blog help me out with a conceptual issue?

Many times I see Orchestration (from examples, clients etc) where the Orchestration is receiving a Message of a certain type and then at some point during the Orchestration sending out a Message of the same exact type.  The issue that is always perplexing to me is when I see the Send shape use a different Message variable than the Receive shape even though the Message being sent out is exactly the same as the Message that was received.  When the Send shape uses a different Message – the new Message has to be constructed (either via a Map or an expression via the Message Assignment shape).  Why create the seperate Message variable?  Why not just use the same Message variable?  Is there something I am missing that happens when you use a seperate Message variable?  To me it seems inefficient (not matter how trival the Construct Message shape might be) to have to create a new Message.

To illustrate – here is the kind of Orchestration I am talking about:


Notice that in this case even though Message_2 (which is being sent via the Send_1 Send shape) is the same *exact* message in terms of type and contents as Message_1 it is being constructed (it has to be of course since Message are immutable inside of an Orchestration unless they are being constructed).  Contrast that Orchestration with this one:


Notice how it only has one Message variable and no Construct Message shape but does exactly the same as the Orchestration that has two Message variables and the Construct Message shape.  Why do people use the Orchestration with the Construct Message shape instead of the one without?  Is there a compelling reason?

Update to BTSnapshot

I’ve made a few improvements to BTSnapshot over the weekend (yes this is how I spend my weekends – working on BizTalk :)). 

I’ve made two major improvements.  BTSnapshot now copies assemblies used from custom pipeline assemblies (this one was suggested from a student in my BizTalk course last week in Torrance – thanks Mir!).  The only thing to look out for here is that unlike all the other assemblies you use from BizTalk, pipeline component assemblies don’t go into the GAC, so there is a chance the assembly will be locked when you try use the “restore” command.

The second improvement is just speed.  I’ve been adding features, but was mostly just happy to have the functionality that the tool provided, but as with all code I needed to do some code improvements to speed things up.  Alot of the slowness was coming from one of the BizTalk types I was using – it was creating the COM+ (ServicedComponent) class that actually does the Undeploy/Deploy – was creating a seperate AppDomain everytime an undeploy or deploy or export was being done.  I just went right to the COM+ class instead and this sped things up alot.  I also consolidated some of the XmlSerializer code.  Still I’d like it to do a semi-complex BizTalk app in a second or so – but I cut down the time in half essentially – from 25 seconds or so for a semi-complex app to under 10.  So I am pretty happy about that.  Enjoy the tool.  If you’d like source, please email me directly and we’ll talk about it ;-).  Mostly I haven’t decided to give out the source partly because of the above discussion – that I knew the code worked – but wasn’t ready to expose it to the world.  The next feature I plan to add is a filter expression, so that if you just want to undeploy/deploy certain pieces of your current management database, you can name all the artifacts with a token (like “AppOne_Orchestration.dll” ) and the tool will only work on artifacts that match that token.

Do not use objects that implement IDisposable from a BizTalk Orchestration

Found this great article on IDisposable through Keith’s blog.  It reminded me of something I’ve been meaning to post about for a while – that you must never use an object that implements IDisposable from directly from an expression shape in an Orchestration.

In general expression shapes are pretty limited anyway, they aren’t actually C# which is a common misconception – they are actually a language called XLANGs.  There isn’t anyway to step through code you write in an expression.  They are really good for the code that is easier to write in an expression than in a C# or VB.NET assembly – like code that reads Context properties or initalizes messages.  Anything that isn’t BizTalk specific, and anything more complicated than a few lines of code is best done in a C# or VB.NET assembly where you get not only statement completion but also full debugging etc.

But one thing you should never do is use an object that implements IDisposable as a variable in an Orchestration.  The above linked article explains the ins and out better than I could of the correct code to use when using an object that implements IDisposable, but writing that code in expression shapes in an Orchestration just isn’t possible.  There are no finally blocks (exception blocks in a scope shape aren’t really implemented as catch blocks away exactly), so there is no place at all to safely call IDisposable.Dispose from.  This means directly connecting to a database by using a type like SqlConnection would just lead to potential badness.