Wednesday, December 30, 2009

A gift to all multithreaded Delphi programmers

A (very) prerelease version 1.05, available via SVN or as a ZIP archive.

I’ve managed to produce two interesting data structures:

  • TOmniQueue (existing class TOmniQueue was renamed to TOmniBoundedQueue) is a dynamically allocated, O(1) enqueue and dequeue, threadsafe,  microlocking queue. The emphasys is on dynamically allocated. In other words – it grows and shrinks!
  • TOmniBlockingCollection is a partial clone (with some enhancements) of .NET’s BlockingCollection.

Have fun and happy new year to all Delphi programmers!

Friday, December 18, 2009

OmniThreadLibrary 1.04b – It’s all Embarcadero’s fault

Delphi 2010 Update 2/3 broke OmniThreadLibrary, but as this update was revoked, I didn’t look into the problem at all.

Now that Update 4/5 is out and OTL is still broken I had no choice but to fix it. Luckily for me, ahwux did most of the work in detecting the problem and providing (at least partial) fix.

OTL is written without resorting to ugly hacks (at least whenever possible). So what could they do to break my code?

OTL uses RTTI information to implement ‘call by name’ mechanism. And that’s not the basic RTTI, implemented in TypInfo unit, but extended class-RTTI from ObjAuto. [In case you want to take a peek at the code – the relevant bits can be found in method TOmniTaskExecutor.GetMethodAddrAndSignature inside the OtlTaskControl unit.] The code checks the method signature (number of parameters, their types and the way they are passed to the method) to see if it matches one of three supported signatures.

For example, first parameter must be the Self object and the code checked this by testing (params^.Flags = []) and (paramType^.Kind = tkClass). This worked in Delphi 2007, 2009, and 2010 – but only in the original release and Update 1. Starting with the Update 2, params^.Flags equals [pfAddress] in this case.

Similarly, constant parameters had flags [pfVar] up to D2010 Update 1 while this changed to [pfConst, pfReference] in D2010 Update 2.

I’m not against those changes. After all, the RTTI parameter description is now much more accurate. But why do they have to make this change in an update!? [Yes, I’m screaming.]

The problem here is that I can’t detect during the compilation whether the Update 4 has been installed. I can easily check for Delphi 2010, but that’s all – there’s no way (I’m aware of) of detecting which update is installed. So now my code looks like this:

  function VerifyObjectFlags(flags, requiredFlags: TParamFlags): boolean;
begin
Result := ((flags * requiredFlags) = requiredFlags);
if not Result then
Exit;
flags := flags - requiredFlags;
{$IF CompilerVersion < 21}
Result := (flags = []);
{$ELSEIF CompilerVersion = 21}
// Delphi 2010 original and Update 1: []
// Delphi 2010 while Update 2 and 4: [pfAddress]
Result := (flags = []) or (flags = [pfAddress]);
{$ELSE} // best guess
Result := (flags = [pfAddress]);
{$IFEND}
end; { VerifyObjectFlags }

function VerifyConstFlags(flags: TParamFlags): boolean;
begin
{$IF CompilerVersion < 21}
Result := (flags = [pfVar]);
{$ELSEIF CompilerVersion = 21}
// Delphi 2010 original and Update 1: [pfVar]
// Delphi 2010 Update 2 and 4: [pfConst, pfReference]
Result := (flags = [pfVar]) or (flags = [pfConst, pfReference]);
{$ELSE} // best guess
Result := (flags = [pfConst, pfReference]);
{$IFEND}
end; { VerifyConstFlags }


Ugly!



If anybody from Embarcadero is reading this: Could you please refrain from doing such changes in IDE updates? Thanks in advance.



Oh, I almost forgot – OTL 1.04b is available on the Google Code.

Sunday, December 13, 2009

DsiWin31 1.53a

This release fixes nasty bug (introduced in release 1.51) which caused various TDSiRegistry function (and other DSi code using those functions) to fail on Delphi 2009/2010.

Other changes:

  • Implemented DSiDeleteRegistryValue.
  • Added parameter 'access' to the DSiKillRegistry.
  • [Mitja] Fixed allocation in DSiGetUserName.
  • [Mitja] Also catch 'error' output in DSiExecuteAndCapture.
  • DSiAddApplicationToFirewallExceptionList renamed to DSiAddApplicationToFirewallExceptionListXP.
  • Added DSiAddApplicationToFirewallExceptionListAdvanced which uses Advanced Firewall interface, available on Vista+.
  • DSiAddApplicationToFirewallExceptionList now calls either DSiAddApplicationToFirewallExceptionListXP or DSiAddApplicationToFirewallExceptionListAdvanced, depending on OS version.
  • Implemented functions to remove application from the firewall exception list: DSiRemoveApplicationFromFirewallExceptionList, DSiRemoveApplicationFromFirewallExceptionListAdvanced, DSiRemoveApplicationFromFirewallExceptionListXP.

OmniThreadLibrary 1.04a

This minor release was released mostly because of exception handling problems when thread pool was used in version 1.04. If you’re using thread pool feature and have OTL 1.04 installed, I’d strongly urge you to upgrade.

Besides code fix I sneaked in a small API upgrade. IOmniTask interface now defines methods RegisterWaitObject/UnregisterWaitObject which the task can use to wait on any waitable object when using TOmniWorker approach (no main thread loop). There’s also a new demo application 31_WaitableObjects which demonstrates the use of this feature.

Monday, November 30, 2009

OmniThreadLibrary patterns – Task controller needs an owner

Pop quiz. What’s wrong with this code?

CreateTask(MyWorker).Run;

Looks fine, but it doesn’t work. In most cases, running this code fragment would cause immediate access violation.

This is a common problem amongst new OTL users. Heck, even I have fallen into this trap!

The problem here is that CreateTask returns IOmniTaskControl interface, or task controller. This interface must be stored into some persistent location, or task controller would be destroyed immediately after Run is called (because the reference count would fall to 0).

A common solution is to just store the interface in some field.

FTaskControl := CreateTask(MyWorker).Run;

When you don’t need background worker anymore, you should terminate the task and free the task controller.

FTaskControl.Terminate;

FTaskControl := nil;

This works for background workers with long life span – for example if there’s a background thread running all the time the program itself is running. But what if you are starting a short-term background task? In this case you should monitor it with TOmniEventMonitor and cleanup task controller reference in OnTerminate event handler.

FTaskControl := CreateTask(MyWorker).MonitorWith(eventMonitor).Run;

In eventMonitor.OnTerminate:

FTaskControl := nil;

As it turns out, event monitor keeps task controller interface stored in its own list, which will also keep the task controller alive. That’s why the following code also works.

CreateTask(MyWorker).MonitorWith(eventMonitor).Run;

Since OTL v1.04 you have another possibility – write a method to free the task controller and pass it to the OnTerminated.

FTaskControl := CreateTask(MyWorker).OnTerminated(FreeTaskControl).Run;

procedure FreeTaskControl(const task: IOmniTaskControl);
begin
  FTaskControl := nil;
end;

If you’re using Delphi 2009 or 2010, you can put the cleanup code in anonymous method.

FTaskControl := CreateTask(MyWorker).OnTerminated(
procedure(const task: IOmniTaskControl) begin
  FTaskControl := nil;
end)
.Run;

OnTerminated does its magic by hooking task controller into internal event monitor. Therefore, you can get real tricky and just write “null” OnTerminated.

CreateTask(MyWorker).OnTerminated(DoNothing).Run;

procedure DoNothing(const task: IOmniTaskControl);
begin
end;

As that looks quite ugly, I’ve added method Unobserved just few days before version 1.04 was released. This method does essentially the same as the “null” OnTerminated approach, except that the code looks nicer and programmers intentions are more clearly expressed.

CreateTask(MyWorker).Unobserved.Run;

Monday, November 23, 2009

OmniThreadLibrary 1.04

Stable release is out! Get it while it’s still hot!

Click to download!

New since 1.04 alpha:

  • Bugfixes in the thread pool code.
  • Implemented IOmniTaskControl.Unobserved behaviour modifier.
  • D2010 designtime package fixed.
  • D2009 packages and test project group updated (thanks to mghie).

New since 1.03: read full list.

Tuesday, November 17, 2009

OmniThreadLibrary 1.04 now in beta

I’ve released OTL 1.04 beta, which is functionally the same as the alpha release but contains some bug fixes. You can download it from Google Code.

1.04 final will be released on 2009-11-23, i.e. next Monday.

Friday, November 13, 2009

OmniThreadLibrary 1.04 alpha

Not yet beta as I still have to fix few TODOs …

Get it here.

COMPATIBILITY ISSUES

  • Changed semantics in comm event notifications! When you get the 'new message' event, read all messages from the queue in a loop!
  • Message is passed to the TOmniEventMonitor.OnTaskMessage handler. There's no need to read from Comm queue in the handler.
  • Exceptions in tasks are now visible by default. To hide them, use IOmniTaskControl.SilentExceptions. Test 13_Exceptions was improved to demonstrate this behaviour.

Other changes

  • Works with Delphi 2010.
  • Default communication queue size reduced to 1000 messages.
  • Support for 'wait and send' in IOmniCommunicationEndpoint.SendWait.
  • Communication subsystem implements observer pattern.
  • WideStrings can be send over the communication channel.
  • New event TOmniEventMonitor.OnTaskUndeliveredMessage is called after the task is terminated for all messages still waiting in the message queue.
  • Implemented automatic event monitor with methods IOmniTaskControl.OnMessage and OnTerminated. Both support 'procedure of object' and 'reference to procedure' parameters.
  • New unit OtlSync contains (old) TOmniCS and IOmniCriticalSection together with (new) OmniMREW - very simple and extremely fast multi-reader-exclusive-writer - and atomic CompareAndSwap functions.
  • New unit OtlHooks contains API that can be used by external libraries to hook into OTL thread creation/destruction process and into exception chain.
  • All known bugs fixed.

New demos

  • 25_WaitableComm: Demo for ReceiveWait and SendWait.
  • 26_MultiEventMonitor: How to run multiple event monitors in parallel.
  • 27_RecursiveTree: Parallel tree processing.
  • 28_Hooks: Demo for the new hook system.
  • 29_ImplicitEventMonitor: Demo for OnMessage and OnTerminated, named method approach.
  • 30_AnonymousEventMonitor: Demo for OnMessage and OnTerminated, anonymous method approach.

A teaser from demo 30

procedure TfrmAnonymousEventMonitorDemo.btnHelloClick(Sender: TObject);
begin
btnHello.Enabled := false;
FAnonTask := CreateTask(
procedure (task: IOmniTask) begin
task.Comm.Send(0, Format('Hello, world! Reporting from thread %d',
[GetCurrentThreadID]));
end,
'HelloWorld')
.OnMessage(
procedure(const task: IOmniTaskControl; const msg: TOmniMessage) begin
lbLog.ItemIndex := lbLog.Items.Add(Format('%d:[%d/%s] %d|%s',
[GetCurrentThreadID, task.UniqueID, task.Name, msg.msgID,
msg.msgData.AsString]));
end)
.OnTerminated(
procedure(const task: IOmniTaskControl) begin
lbLog.ItemIndex := lbLog.Items.Add(Format('[%d/%s] Terminated',
[task.UniqueID, task.Name]));
btnHello.Enabled := true;
FAnonTask := nil;
end)
.Run;
end;

Friday, November 06, 2009

Do we need DelphiOverflow.com?

Today I was interviewed for the greatest Delphi podcast of them all and Jim asked me a question I didn’t know how to answer: “Do you think there should be Delphi equivalent of StackOverflow.com?” I’m afraid my answer was somewhere along: “Hmph. Yes. Very good question. Very good. Let’s talk about something else.”

And now I can’t get it out of my head. Should there be delphioverflow.com? What could we get out of it? I would be the first to admit that the StackOverflow model is greatest thing since Belgian waffles and that having Delphi questions and answers in such form would be very useful.

But wait – there already are Delphi questions on StackOverflow! Not that many as C# questions, but still enough that Delphi is seen on the front page and that other users can read about it and see that it is alive and well. Even more – there are enough knowledgeable Delphi programmers on SO and most questions get great answers in less than five minutes.

What other positive result could such site bring? Maybe Embarcadero people would be more eager to participate and answer questions on their own server? Maybe, but not sure. Delphi R&D team is very busy and sometimes they can’t even find time to answer newsgroup questions. And I’m pretty sure that - whatever such change would bring – newsgroups wouldn’t go away.

Let’s take a look from another perspective. What would be negative consequences? Less Delphi questions on StackOverflow. And that’s a Bad Thing because it lowers Delphi’s discoverability. We want to talk about Delphi in public places, not on some secluded server!

Now I know how to answer. No, I don’t think we need DelphiOverflow. We need more Delphi R&D people answering questions on StackOverflow.

(Your comments on the topic are very much welcome, as always!)

Wednesday, November 04, 2009

GpStuff 1.19 & GpLists 1.43

I’ll finish my short overview of changes in various Gp units with new GpStuff and GpLists.

Let’s deal with the latter first. There were only two changes. Firstly, Slice, Walk and WalkKV enumerators got the step parameter. Now Delphi is really as powerful as Basic!

Secondly, I’ve added method FreeObjects to the TStringList helper. It will walk the string list and free all associated objects – something that is not done automatically in the TStringList destructor. Very useful helper, if I can say so.

procedure TGpStringListHelper.FreeObjects;
var
iObject: integer;
begin
for iObject := 0 to Count - 1 do begin
Objects[iObject].Free;
Objects[iObject] := nil;
end;
end; { TGpStringListHelper.FreeObjects }

Changes in GpStuff were more significant.

There are new enumerator factories. EnumStrings allows you do do stuff like this:

for s in EnumStrings(['one', 'two', 'three']) do
// ...

EnumValues will do the same for integer arrays. EnumPairs is similar to EnumStrings but returns (key, value) pairs:

var
kv: TGpStringPair;

for kv in EnumPairs(['1', 'one', '2', 'two']) do
// k.key = '1', k.value = 'one'
// k.key = '2', k.value = 'two'

There is also EnumList, which enumerates lists of items (where the whole list itself is a string):

for s in EnumList('one,two,"one,two,three"', ',', '"') do
// s = 'one'
// s = 'two'
// s = 'one,two,three'

There were some changes in TGp4AlignedInt internals – now all values are integer, not cardinal (because underlying Windows implementation works with integers). There is also new function “Compare and Swap” (CAS) in TGp4AlignedInt and TGp8AlignedInt64 (which was previously called TGp8AlignedInt).

Finally, there are new interface and class - IGpTraceable and TGpTraceable.

type
IGpTraceable = interface(IInterface)
function GetTraceReferences: boolean; stdcall;
procedure SetTraceReferences(const value: boolean); stdcall;
function _AddRef: integer; stdcall;
function _Release: integer; stdcall;
function GetRefCount: integer; stdcall;
property TraceReferences: boolean read GetTraceReferences write SetTraceReferences;
end; { IGpTraceable }

TGpTraceable = class(TInterfacedObject, IGpTraceable)
private
gtTraceRef: boolean;
public
destructor Destroy; override;
function _AddRef: integer; stdcall;
function _Release: integer; stdcall;
function GetRefCount: integer; stdcall;
function GetTraceReferences: boolean; stdcall;
procedure SetTraceReferences(const value: boolean); stdcall;
property TraceReferences: boolean read GetTraceReferences write SetTraceReferences;
end; { TGpTraceable }

The TGpTraceable class helps me debug interface problems. It exposes GetRefCount function which returns reference count, and it can trigger debugger interrupt on each reference count change if TraceReferences property is set.

function TGpTraceable._AddRef: integer;
begin
Result := inherited _AddRef;
if gtTraceRef then
asm int 3; end;
end; { TGpTraceable._AddRef }

function TGpTraceable._Release: integer;
begin
if gtTraceRef then
asm int 3; end;
Result := inherited _Release;
end; { TGpTraceable._Release }
---Published under the Creative Commons Attribution 3.0 license

Monday, November 02, 2009

Read prefetch in GpHugeFile

There is only one big change in the latest GpHugeFile – read prefetch. Most people won’t need it at all and other will only need it occasionally, but for some people, sometimes, it will be a life saver.

The prefetch option is only useful when you read a file mostly sequentially from a relatively slow media. Useless? You never did that before? Did you ever played a video file from the network server or from the YouTube? Well, there you are!

Playing video files (especially HD) over network is not a trivial task. In some occasions (namely, slow networks or high bitrate files) the network speed is only slightly above the minimum required for the seamless video playout. Even more – the network speed is not constant because you share it with other users and at some times it may not be high enough to play the video without stuttering.

To solve this problem, video players use prefetch (or read-ahead) – they will read more data than required and use this buffer when the network slows down. Better said – video will always play from this buffer but the buffer size will vary depending on current network speed.

So how’s this typically done? One way is with a background thread that sequentially reads through the file and buffers the data and another is with asynchronous read operations. This very powerful approach is part of the standard ReadFileEx Win32 API and is relatively easy to use – you just start the read operation and some time later the system will notify you that the data is available. There are some problems, though, the biggest of them the requirement that your reading thread must be in a special alertable sleep state for this notification to occur.

The third option is not to use threads or asynch file ops, but to pass hfoPrefetch and hfoBuffered flags to the ResetEx. In the same call you can also set the number of prefetched buffers. As for the buffer size – it is also settable with a ResetEx parameters and will be rounded up to the next multiplier of the system page size (async file io requirement) or it will be set to 64 KB if you leave the parameter at 0.

When you se hfoPrefetch, TGpHugeFile will create background thread and this thread will issue asynchronous file io calls. Prefetched data is stored in a cache which is shared between the worker thread and the owner. Unfortunately for some, this option is only available in Delphi 2007 and newer because the worker object is implemented using OmniThreadLibrary.

Maybe you’ll wonder why the thread is not issuing normal synchronous reads? For two reasons – I didn’t want the thread to block reading data when owner executes a Seek (file repositioning will immediately tell the prefetcher that it should start reading from a different file offset) and I wanted to issue multiple read commands at the same time (namely 2).

Enough talk – if you want to learn more, look at the code. I’ll only give you the simplest possible demo:

program Project12;

{$APPTYPE CONSOLE}

uses
SysUtils,
GpHugeF;

var
hf : TGpHugeFile;
buf: array [1..65536] of byte;
bytesRead: cardinal;
bytesTotal: int64;

begin
hf := TGpHugeFile.Create(ParamStr(1));
try
if hf.ResetEx(1, 0, 0, 0, [hfoBuffered, hfoPrefetch]) <> hfOK then
Writeln('Fail!')
else begin
bytesTotal := 0;
repeat
hf.BlockRead(buf, SizeOf(buf), bytesRead);
Inc(bytesTotal, bytesRead);
until bytesRead = 0;
Writeln('Total bytes read: ', bytesTotal);
end;
finally FreeAndNil(hf); end;
Readln;
end.

Wednesday, October 28, 2009

GpHugeFile v6 and other updates

Today I finally published all my updated open source units. Some were recently modified and for others the update was long overdue. Some were only slightly modified, in others the changes were bigger. Read through the list below and you’ll see.

In future posts I intend to write some more about changes in GpHugeFile, GpStuff, and GpLists. If you want to know more about changes in other units, drop a comment.

GpHugeFile 6.01

  • Implemented read prefetch, activated by setting hfoPrefetch option flag.
  • Number of buffers to prefetch can be set with a ResetEx parameter.

Unfortunately for some, this option is only available in Delphi 2007 and newer because the worker object is implemented using OmniThreadLibrary.

GpVersion 2.04

  • Updated for Delphi 2009.
  • Extended IVersion interface with IsNotHigherThan, IsNotLowerThan and IsEqualTo.

GpTextFile 4.02

  • Compatible with Delphi 2009.

GpSync 1.22

  • Implemented TGpSWMR.AttachToThread.
  • Added internal check to ensure that TGpSWMR.WaitToRead/WaitToWrite/Done are called from one thread only.

GpStuff 1.19

  • Added EnumPairs string array enumerator.
  • Added EnumList string enumerator.
  • Added EnumStrings enumerator.
  • InterlockedIncrement/InterlockedDecrement deal with integers, therefore TGp4AlignedInt.Increment/Decrement must return integers. All other functions in TGp4AlignedInt also changed to work with integers.
  • Implemented function CAS (compare and swap) in TGp4AlignedInt and TGp8AlignedInt64 records.
  • TGp8AlignedInt renamed to TGp8AlignedInt64.
  • TGp8AlignedInt.Addr must be PInt64, not PCardinal.
  • Implemented IGpTraceable interface.

GpStructuredStorage 2.0a

  • [Erik Berry] Definition of fmCreate in Delphi 2010 has changed and code had to be adjusted.

GpStreams 1.25b

  • Safer TGpFixedMemoryStream.Read.
  • Added setter for TGpFixedMemoryStream.Position so that invalid positions raise exception.

GpSharedMemory 4.12

  • Compatible with Delphi 2009.

GpLists 1.43

  • Added parameter 'step' to various Slice(), Walk() and WalkKV() enumerators.
  • Added method FreeObjects to the TStringList helper.

Tuesday, October 27, 2009

And the award goes to …

image Daniel R. Wolf of the Delphi PRAXiS.

Not the official Spirit of Delphi award, but the Delphi Legends Community Award in the organization of Wings of Wind (who are those people???).

Anyway, somehow I got nominated and even scrapped together the fifth place, which is definitely more than I deserve, but still – thanks for the recognition, folks!

Such things make one work harder and publish more so you can definitely expect plenty of articles on this blog in the near future …

Sunday, October 25, 2009

TDM Rerun #14: A Portable XML

The code unit, OmniXML.pas, which contains the XML representation interfaces, parser and writer, was written by a single programmer, Miha Remec (he is also the guy behind the www.omnixml.com website). He started writing it in 2000, because he was missing a native Delphi DOM parser, one that would represent the DOM the same way as it was designed. The best Delphi parser around at that time was OpenXML, but it used classes to represent XML elements, not interfaces. OmniXML uses interfaces, derived from the IXMLNode (as specified by the DOM). That also makes it almost completely compatible with the MSXML parser, which uses the same approach.

- A Portable XML, The Delphi Magazine 105, May 2004

In the 2004 May issue I wrote about OmniXML, a native Delphi XML parser. I described the OmniXML approach and wrote few short pieces of code that demonstrated its use.

Today, five years later, OmniXML is still strong and I’m still using it, as you can see in my Fluent XML series.

Links: article (PDF, 45 KB), source code (ZIP, 795 KB).

Thursday, October 22, 2009

DSiWin32 1.51

It’s been some time since I’ve last updated my open source units… For example, DSiWin32, a collection of Win32 API helpers, was last updated in August 2008! Bad me!

Time to do some housecleaning, then. Let’s see what’s new in DSiWin32 1.51.

There are new dynamic forwarders: DSiWow64DisableWow64FsRedirection, DSiWow64RevertWow64FsRedirection, DSiGetTickCount64 and DSiGlobalMemoryStatusEx. They call appropriate API functions if they are available and return error on older Windows systems. [Click on the function name to see the API specification on the MSDN.]

Function DSiGetGlobalMemoryStatus is not much more than a wrapper to the GlobalMemoryStatusEx API and returns information about global memory status (paging, virtual memory etc). Information is returned in a TMemoryStatusEx record, which is also defined in the DSiWin32 unit.

We implemented six new function in the files section. DSiGetNetworkResource converts drive letter (created by mapping network location) back to the network path. DSiDisconnectFromNetworkResource disconnects drive letter from a network resource. [DSiConnectToNetworkResource was included in the previous public version 1.41.] DSiGetSubstDrive maps drive letter (created by using Subst command) to the associated folder and DSiGetSubstPath does similar for a file path that starts with a subst’ed letter. DSiDisableWow64FsRedirection disables file system redirection (mapping from \Windows\System32 to \Windows\SysWOW64 for 32-bit applications on 64-bit systems) for the current thread and DSiRevertWow64FsRedirection reverts this change.

There are also two new install functions. DSiAddApplicationToFirewallExceptionList adds application to the firewall exception list and DSiAddPortToFirewallExceptionList does the same for a TCP/IP port.

DSiGetCurrentThreadHandle and DSiGetCurrentProcessHandle return (true) handles for the current thread and process. [In contrast to GetCurrentThread and GetCurrentProcess APIs which return pseudo handle which cannot be used outside of the current thread/process context.]

DSiGetWindowsVersion was extended to detect Windows Server 2008, Windows 7 and Windows Server 2008 R2. DSiGetTrueWindowsVersion was also upgraded to return “Windows Server 2008 or Vista SP1” and “Windows 7 or Server 2008 R2”. It looks like it is not possible to discriminate between those operating systems on the API level :( Record TOSVersionInfoEx was also defined as it was used in the DSiGetWindowsVersion.

'Access' parameter was added to the DSiWriteRegistry methods so that user can request writing to the non-virtualized key when running on 64-bit system (KEY_WOW64_64KEY).

DSiExecuteAndCapture got much deserved workover. Now the caller can be informed of each line outputted by the child process.

Delphi 2009/2010 compatibility was fixed for DSiGetFolderLocation, DSiGetNetworkResource, DSiGetComputerName, DSiGetWindowsFolder, DSiExecuteAndCapture and bugs were fixed in DSiGetTempFileName and DSiGetUserName.

All in all, lots of things were changed and improved. If you’re already using DSiWin32 then this is a good time to upgrade. [And if not, start using it!]

Saturday, October 17, 2009

Open source Computer Vision library

Memo to self: When you play with computer vision next time, check the OpenCV library.

From the OpenCV www:

“OpenCV (Open Source Computer Vision) is a library of programming functions for real time computer vision. Example applications of the OpenCV library are Human-Computer Interaction (HCI); Object Identification, Segmentation and Recognition; Face Recognition; Gesture Recognition; Camera and Motion Tracking, Ego Motion, Motion Understanding; Structure From Motion (SFM); Stereo and Multi-Camera Calibration and Depth Computation; Mobile Robotics.”

Found via IPhone Sudoku Grab via Julian M Bucknall.

Apparently there is no Delphi interface (yet) but as the DLL has simple C interface, such interface could easily (or “easily”?) be implemented.

Tuesday, October 06, 2009

Fluent XML [3]

Few days ago I was answering some OmniXML related question on Slovenian Delphi forum and I tried using GpFluentXML to provide the answer. The emphasis is on “tried” – my unit was just not powerful enough to be useful in this case.

So what else could I do besides improving GpFluentXML?

Modifications were very small. I added two overloads (AddChild and AddSibling) which allow setting node value. Implementation is equally trivial as everything else inside the fluent XML unit.

function TGpFluentXmlBuilder.AddChild(const name: XmlString;
value: Variant): IGpFluentXmlBuilder;
begin
Result := AddChild(name);
SetTextChild(fxbActiveNode, XMLVariantToStr(value));
end; { TGpFluentXmlBuilder.AddChild }

(And similar for AddSibling.)

Now I (and you and everybody else) can write such code:

var
data : TData;
xmlBuilder: IGpFluentXmlBuilder;
begin
xmlBuilder := CreateFluentXml
.AddProcessingInstruction('xml', 'version="1.0" encoding="UTF-8"')
.AddChild('DataTransfer')
['xsi:noNamespaceSchemaLocation', 'http://www.tempuri/schema.xsd']
['xmlns:xsi', 'http://www.w3.org/2001/XMLSchema-instance']
.AddChild('Dokument');
for data in GetData do begin
xmlBuilder
.Mark
.AddChild('ID', data.ID)
.AddSibling(DocumentInfo')
.AddChild('GeneratedBy', data.Originator)
.AddSibling('Title', data.Title)
.Return;
end;
end;

A non-obvious trick – .Mark and .Return are used inside the loop to store/restore proper level at which the child nodes must be inserted.

Full GpFluentXML source is available at http://17slon.com/blogs/gabr/files/GpFluentXml.pas.

Tuesday, September 29, 2009

Unhibernating – with a T-shirt

Dear reader,

I’m fully aware that I was silent for a long long time.

That’s how life goes. Sometimes you have time to think and write and sometimes software has bugs, children are growing, house is telling you that it shouldn’t be in the “fixer-upper” mode anymore and the body starts warning that the guaranty has expired. Plus the days suddenly have only 24 hours and not 28 as it was customary for the last ten years.

In short – I had no time. Life was going on. And around. And over me. Especially over me.

Luckily, everything is turning out fine and I will again write about my Delphi adventures.

Today I visited the RAD Studio 2010 presentation organized by Embarcadero and Slovenian Delphi dealer, Marand. Mark Barrington and Pawel Glowacki were showing RAD Studio 2010, Embarcadero’s database tools and All-Access. Sadly, I only had time to attend the Delphi session which was very good and informative. (Thumbs up, Pawel!)

There were also T-shirts and I was lucky to get one.

P1090662

Nice T-shirt but it’s even better if you look closer at the code.

P1090665

Parallel Erathostenes sieve! That’s definitely the code I can appreciate!

[For the OTL fans – OTL is alive and well and I will publish next version sometime in October. And then I’ll write the documentation. Promise.]

Thursday, April 30, 2009

TDM Rerun #13: Shared Events, Part 2: Redesign

Now we can already guess where the general sluggishness of the shared event system comes from. The trouble lies in the constant XML loading and saving. Most of the shared event system tables are quite static, but that doesn’t hold for the Event Queue table, where new entries are inserted, modified and deleted all the time.

- Shared Events, Part 2: Redesign, The Delphi Magazine 102, February 2004

shared table memory snapshotMy second article on shared events architecture first addressed speed issues (original code was quite slow), then discussed internals of shared counters, shared linked lists and shared tables (all of which are used in the shared events system) and at the end returned to fine-tuning by fixing some remaining speed issues. As you can expect, the basis for the tuning was hard data from the profiler, not some wave-of-hand ideas about where the problem maybe lies.

Links: article (PDF, 99 KB), source code (ZIP, 1,9 MB), current version.

TDM Rerun #12: Shared Events

Shared event system, as I nicknamed this approach, is implemented as a shared set of in-memory tables, which are accessed and manipulated by producers and listeners. The important part is that there is no dedicated server: housekeeping is distributed between the producers and listeners.

- Shared Events, The Delphi Magazine 97, September 2003

imageShared events mechanism was definitely the most complicated architecture based on shared memory I ever put together. The system allowed multiple programs (running on the same computer) to cooperate using an event-based system. First program would publish an event (or more events) and others would subscribe to those events. First program would then broadcast the event, which would trigger notifications in all subscribed programs. The best trick was that there was no privileged part – no server, service or manager. Publishers and consumers shared all the work – tables were created as needed, housekeeping was done on both sides and so on.

Underlying architecture was largely redesigned after this article was published. Original source files are included only for historical reference.

Links: article (PDF, 496 KB), source code (ZIP, 1,9 MB), current version.