Thursday, April 30, 2009

TDM Rerun #13: Shared Events, Part 2: Redesign

Now we can already guess where the general sluggishness of the shared event system comes from. The trouble lies in the constant XML loading and saving. Most of the shared event system tables are quite static, but that doesn’t hold for the Event Queue table, where new entries are inserted, modified and deleted all the time.

- Shared Events, Part 2: Redesign, The Delphi Magazine 102, February 2004

shared table memory snapshotMy second article on shared events architecture first addressed speed issues (original code was quite slow), then discussed internals of shared counters, shared linked lists and shared tables (all of which are used in the shared events system) and at the end returned to fine-tuning by fixing some remaining speed issues. As you can expect, the basis for the tuning was hard data from the profiler, not some wave-of-hand ideas about where the problem maybe lies.

Links: article (PDF, 99 KB), source code (ZIP, 1,9 MB), current version.

TDM Rerun #12: Shared Events

Shared event system, as I nicknamed this approach, is implemented as a shared set of in-memory tables, which are accessed and manipulated by producers and listeners. The important part is that there is no dedicated server: housekeeping is distributed between the producers and listeners.

- Shared Events, The Delphi Magazine 97, September 2003

imageShared events mechanism was definitely the most complicated architecture based on shared memory I ever put together. The system allowed multiple programs (running on the same computer) to cooperate using an event-based system. First program would publish an event (or more events) and others would subscribe to those events. First program would then broadcast the event, which would trigger notifications in all subscribed programs. The best trick was that there was no privileged part – no server, service or manager. Publishers and consumers shared all the work – tables were created as needed, housekeeping was done on both sides and so on.

Underlying architecture was largely redesigned after this article was published. Original source files are included only for historical reference.

Links: article (PDF, 496 KB), source code (ZIP, 1,9 MB), current version.

Thursday, April 02, 2009

Fluent XML [2]

Yesterday I described my approach to more fluent XML writing. Today I’ll describe the GpFluentXML unit where the ‘fluid’ implementation is stored. If you skipped yesterday’s post you’re strongly encourage to read it now.

Let’s start with the current version of the fluent XML builder interface, which is not completely identical to the yesterday’s version.

Interface

uses
OmniXML_Types,
OmniXML;

type
IGpFluentXmlBuilder = interface ['{91F596A3-F5E3-451C-A6B9-C5FF3F23ECCC}']
function GetXml: IXmlDocument;
//
function AddChild(const name: XmlString): IGpFluentXmlBuilder;
function AddComment(const comment: XmlString): IGpFluentXmlBuilder;
function AddProcessingInstruction(const target, data: XmlString): IGpFluentXmlBuilder;
function AddSibling(const name: XmlString): IGpFluentXmlBuilder;
function Anchor(var node: IXMLNode): IGpFluentXmlBuilder;
function Mark: IGpFluentXmlBuilder;
function Return: IGpFluentXmlBuilder;
function SetAttrib(const name, value: XmlString): IGpFluentXmlBuilder;
function Up: IGpFluentXmlBuilder;
property Attrib[const name, value: XmlString]: IGpFluentXmlBuilder
read SetAttrib; default;
property Xml: IXmlDocument read GetXml;
end; { IGpFluentXmlBuilder }

function CreateFluentXml: IGpFluentXmlBuilder;

The fluent XML builder is designed around the concept of the active node, which represents the point where changes are made. When you call the factory function CreateFluentXml, it creates a new IXMLDocument interface and sets active node to this interface (IXMLDocument is IXMLNode so that is not a problem). When you call other functions, active node may change or it may not, depending on a function.

AddProcessingInstruction and AddComment just create a processing instruction (<?xml … ?> line at the beginning of the XML document) and comment and don’t affect the active node.

AddChild creates a new XML node and makes it a child of the current active node.

Up sets active node to the parent of the active node. Unless, of course, if active node is already at the topmost level in which case it will raise an exception. In the yesterday post this method was called Parent.

AddSibling creates a new XML node and makes it a child of the current active node’s parent. In other words, AddSibling is a shorter version of Up followed by the AddChild.

SetAttrib or it’s shorthand, the default property Attrib, sets value of an attribute.

Mark and Return are always used in pairs. Mark pushes active node on the top of the internal (to the xml builder) stack. Return pops a node from the top of the stack and sets it as the active node. Yesterday this pair was named Here/Back.

Anchor copies the active node into its parameter. That allows you to generate the template code with the fluent xml and store few nodes for later use. Then you can use those nodes to insert programmatically generated XML at those points.

At the end, there’s the Xml property which returns the internal IXMLDocument interface, the one that was created in the CreateFluentXml.

And now let’s move to the implementation.

Implementation

Class TGpFluentXmlBuilder implements the IGpFluentXmlBuilder interface. In addition to the methods from this interface, it declares function ActiveNode and three fields – fxbActiveNode stores the active node, fxbMarkedNodes is a stack of nodes stored with the Mark method and fxbXmlDoc is the XML document.

type
TGpFluentXmlBuilder = class(TInterfacedObject, IGpFluentXmlBuilder)
strict private
fxbActiveNode : IXMLNode;
fxbMarkedNodes: IInterfaceList;
fxbXmlDoc : IXMLDocument;
strict protected
function ActiveNode: IXMLNode;
protected
function GetXml: IXmlDocument;
public
constructor Create;
destructor Destroy; override;
function AddChild(const name: XmlString): IGpFluentXmlBuilder;
function AddComment(const comment: XmlString): IGpFluentXmlBuilder;
function AddProcessingInstruction(const target, data: XmlString): IGpFluentXmlBuilder;
function AddSibling(const name: XmlString): IGpFluentXmlBuilder;
function Anchor(var node: IXMLNode): IGpFluentXmlBuilder;
function Mark: IGpFluentXmlBuilder;
function Return: IGpFluentXmlBuilder;
function SetAttrib(const name, value: XmlString): IGpFluentXmlBuilder;
function Up: IGpFluentXmlBuilder;
end; { TGpFluentXmlBuilder }

Some functions are pretty much trivial – one line to execute the action and another to return Self so another fluent XML action can be chained onto result of the function. Of course, some of those functions are simple because they use wrappers from the OmniXMLUtils unit, not from MS-compatible OmniXML.pas. [By the way, you can download OmniXML at www.omnixml.com.]

function TGpFluentXmlBuilder.AddChild(const name: XmlString): IGpFluentXmlBuilder;
begin
fxbActiveNode := AppendNode(ActiveNode, name);
Result := Self;
end; { TGpFluentXmlBuilder.AddChild }

function TGpFluentXmlBuilder.AddComment(const comment: XmlString): IGpFluentXmlBuilder;
begin
ActiveNode.AppendChild(fxbXmlDoc.CreateComment(comment));
Result := Self;
end; { TGpFluentXmlBuilder.AddComment }

function TGpFluentXmlBuilder.AddProcessingInstruction(const target, data: XmlString):
IGpFluentXmlBuilder;
begin
ActiveNode.AppendChild(fxbXmlDoc.CreateProcessingInstruction(target, data));
Result := Self;end; { TGpFluentXmlBuilder.AddProcessingInstruction }

function TGpFluentXmlBuilder.AddSibling(const name: XmlString): IGpFluentXmlBuilder;
begin
Result := Up;
fxbActiveNode := AppendNode(ActiveNode, name);
end; { TGpFluentXmlBuilder.AddSibling }

function TGpFluentXmlBuilder.GetXml: IXmlDocument;
begin
Result := fxbXmlDoc;
end; { TGpFluentXmlBuilder.GetXml }

function TGpFluentXmlBuilder.Mark: IGpFluentXmlBuilder;
begin
fxbMarkedNodes.Add(ActiveNode);
Result := Self;
end; { TGpFluentXmlBuilder.Mark }
function TGpFluentXmlBuilder.Return: IGpFluentXmlBuilder;
begin
fxbActiveNode := fxbMarkedNodes.Last as IXMLNode;
fxbMarkedNodes.Delete(fxbMarkedNodes.Count - 1);
Result := Self;
end; { TGpFluentXmlBuilder.Return }

function TGpFluentXmlBuilder.SetAttrib(const name, value: XmlString): IGpFluentXmlBuilder;
begin
SetNodeAttrStr(ActiveNode, name, value);
Result := Self;
end; { TGpFluentXmlBuilder.SetAttrib }

OK, Return has three lines, not two. That makes it medium complicated :)

In fact Up is also very simple, except that it checks validity of the active node before returning its parent.

function TGpFluentXmlBuilder.Up: IGpFluentXmlBuilder;
begin
if not assigned(fxbActiveNode) then
raise Exception.Create('Cannot access a parent at the root level')
else if fxbActiveNode = DocumentElement(fxbXmlDoc) then
raise Exception.Create('Cannot create a parent at the document element level')
else
fxbActiveNode := ActiveNode.ParentNode;
Result := Self;
end; { TGpFluentXmlBuilder.Up }

A little more trickstery is hidden inside the ActiveNode helper function. It returns active node when it is set; if not it returns XML document’s document element  or the XML doc itself if document element is not set. I don’t think the second option (document element) can ever occur. That part is just there to future-proof the code.

function TGpFluentXmlBuilder.ActiveNode: IXMLNode;
begin
if assigned(fxbActiveNode) then
Result := fxbActiveNode
else begin
Result := DocumentElement(fxbXmlDoc);
if not assigned(Result) then
Result := fxbXmlDoc;
end;
end; { TGpFluentXmlBuilder.ActiveNode }

Believe it or not, that’s all. The whole GpFluentXml unit with comments and everything is only 177 lines long.

Full GpFluentXML source is available at http://17slon.com/blogs/gabr/files/GpFluentXml.pas.

Wednesday, April 01, 2009

Fluent XML [1]

Few days ago I was writing a very boring piece of code that should generate some XML document. It was full of function calls that created nodes in the XML document and set attributes. Boooooring stuff. But even worse than that – the structure of the XML document was totally lost in the code. It was hard to tell which node is child of which and how it’s all structured.

Then I did what every programmer does when he/she should write some boring code – I wrote a tool to simplify the process. [That process usually takes more time than the original approach but at least it is interesting ;) .]

I started by writing the endcode. In other words, I started thinking about how I want to create this XML document at all. Quickly I decided on the fluent interface approach. I perused it in the OmniThreadLibrary where it proved to be quite useful.

That’s how the first draft looked (Actually, it was much longer but that’s the important part.):

xmlWsdl := CreateFluentXml
.AddProcessingInstruction('xml', 'version="1.0" encoding="UTF-8"')
.AddChild('definitions')
.SetAttr('xmlns', 'http://schemas.xmlsoap.org/wsdl/')
.SetAttr('xmlns:xs', 'http://www.w3.org/2001/XMLSchema')
.SetAttr('xmlns:soap', 'http://schemas.xmlsoap.org/wsdl/soap/')
.SetAttr('xmlns:soapenc', 'http://schemas.xmlsoap.org/soap/encoding/')
.SetAttr('xmlns:mime', 'http://schemas.xmlsoap.org/wsdl/mime/');

This short fragment looks quite nice but in the full version (about 50 lines) all those SetAttr calls visually merged together with AddChild calls and the result was still unreadable (although shorter than the original code with explicit calls to XML interface).

My first idea was to merge at least some SetAttr calls into the AddChild by introducing two versions – one which takes only a node name and another which takes node name, attribute name and attribute value – but that didn’t help the code at all. Even worse – it was hard to see which AddChild calls were setting attributes and which not :(

That got me started in a new direction. If the main problem is visual clutter, I had to do something to make setting attributes stand out. Briefly I considered a complicated scheme which would use smart records and operator overloading but I couldn’t imagine a XML creating code which would use operators and be more readable than this so I rejected this approach. [It may still be a valid approach – it’s just that I cannot make it work in my head.]

Then I thought about arrays. In “classical” code I could easily add array-like support to attributes so that I could write xmlNode[attrName] := ‘some value’, but how can I make this conforming my fluent architecture?

To get or not to get

In order to be able to chain anything after the [], the indexed property hiding behind must return Self, i.e. the same interface it is living in. And because I want to use attribute name/value pairs, this property has to have two indices.

property Attrib[const name, value: XmlString]: IGpFluentXmlBuilder 
read GetAttrib; default;

That would allow me to write such code:

.AddSibling('service')['name', serviceName]
.AddChild('port')
['name', portName]
['binding', 'fs:' + bindingName]
.AddChild('soap:address')['location', serviceLocation];

As you can see, attributes can be chained and I can write attribute assignment in the same line as node creation and it is still obvious which is which and who is who.

But … assignment? In a getter? Why not! You can do anything in the property getter. To make this more obvious, my code calls this ‘getter’ SetAttrib. As a nice side effect, SetAttrib is completely the same as it was defined in the first draft and can even be used insted of the [] approach.

I’ll end today’s instalment with the complete 'fluent xml builder’ interface and with sample code that uses this interface to build an XML document. Tomorrow I’ll wrap things up by describing the interface and its implementation in all boring detail.

type
IGpFluentXmlBuilder = interface ['{91F596A3-F5E3-451C-A6B9-C5FF3F23ECCC}']
function GetXml: IXmlDocument;
//
function Anchor(var node: IXMLNode): IGpFluentXmlBuilder;
function AddChild(const name: XmlString): IGpFluentXmlBuilder;
function AddComment(const comment: XmlString): IGpFluentXmlBuilder;
function AddSibling(const name: XmlString): IGpFluentXmlBuilder;
function AddProcessingInstruction(const target, data: XmlString): IGpFluentXmlBuilder;
function Back: IGpFluentXmlBuilder;
function Here: IGpFluentXmlBuilder;
function Parent: IGpFluentXmlBuilder;
function SetAttrib(const name, value: XmlString): IGpFluentXmlBuilder;
property Attrib[const name, value: XmlString]: IGpFluentXmlBuilder
read SetAttrib; default;
property Xml: IXmlDocument read GetXml;
end; { IGpFluentXmlBuilder }
 
  xmlWsdl := CreateFluentXml
.AddProcessingInstruction('xml', 'version="1.0" encoding="UTF-8"')
.AddChild('definitions')
['xmlns', 'http://schemas.xmlsoap.org/wsdl/']
['xmlns:xs', 'http://www.w3.org/2001/XMLSchema']
['xmlns:soap', 'http://schemas.xmlsoap.org/wsdl/soap/']
['xmlns:soapenc', 'http://schemas.xmlsoap.org/soap/encoding/']
['xmlns:mime', 'http://schemas.xmlsoap.org/wsdl/mime/']
['name', serviceName]
['xmlns:ns1', 'urn:' + intfName]
['xmlns:fs', 'http://fab-online.com/soap/']
['targetNamespace', 'http://fab-online.com/soap/']
.AddChild('message')['name', 'fs:' + baseName + 'Request'].Anchor(nodeRequest)
.AddSibling('message')['name', 'fs:' + baseName + 'Response'].Anchor(nodeResponse)
.AddSibling('portType')['name', baseName]
.Here
.AddChild('operation')['name', baseName]
.AddChild('input')['message', 'fs:' + baseName + 'Request']
.AddSibling('output')['message', 'fs:' + baseName + 'Response']
.Back
.AddSibling('binding')
.Here
['name', bindingName]
['type', 'fs:' + intfName]
.AddChild('soap:binding')
['style', 'rpc']
['transport', 'http://schemas.xmlsoap.og/soap/http']
.AddChild('operation')['name', baseName]
.AddChild('soap:operation')
['soapAction', 'urn:' + baseName]
['style', 'rpc']
.AddSibling('input')
.AddChild('soap:body')
['use', 'encoded']
['encodingStyle', 'http://schemas.xmlsoap.org/soap/encoding/']
['namespace', 'urn:' + intfName + '-' + baseName]
.Parent
.AddSibling('output')
.AddChild('soap:body')
['use', 'encoded']
['encodingStyle', 'http://schemas.xmlsoap.org/soap/encoding/']
['namespace', 'urn:' + intfName + '-' + baseName]
.Back
.AddSibling('service')['name', serviceName]
.AddChild('port')
['name', portName]
['binding', 'fs:' + bindingName]
.AddChild('soap:address')['location', serviceLocation];

What do you think? Does my approach make any sense?