Happy New Year 2011

See you around in 2011 – with some luck maybe even in person.

[Picture of the living room wall taken by my trusty Panasonic Lumix DMC-LX5 and heavily processed in Adobe Photoshop Lightroom 3.]

Gp* Holiday Update

As always, all units are available on the Google Code.

DSiWin32 1.60

• When compiled with D2007 or newer, unit FileCtrl is not included.
• Call UniqueString before calling CreateProcessW.
• [Tommi Prami] Added types missing in Delphi 7.
• Added function DSiDisableStandby that will try to disable standby and hibernate on Windows XP SP 2 and newer.
• Define TStartupInfoW in Delphi 7 and earlier.
• DSiAddApplicationToFirewallExceptionList[Advanced|XP] got a new parameter
  TDSiFwResolveConflict (default rcDuplicate) where the caller can specify
  behaviour if the rule with the same name already exists.
  [rcDuplicate = add new rule with the same name, rcOverwrite = remove all rules
  with the same name and then add the new rule, rcSkip = leave existing rules
  intact and don't add the new rule]
• Implemented DSiFindApplicationInFirewallExceptionList[Advanced|XP].
• Bug fix in DSiAddApplicationToFirewallExceptionListAdvanced: setting rule.ServiceName to '' caused fwPolicy2.Rules.Add(rule) to raise exception.

You’ve got mail!

It was only three days ago that I released the new OTL and already I’ve got a gift – and it came from the other side of the world, from a place so far ago that Google Maps doesn’t know how to calculate directions to guide me there, from Australia!

No, I’m not kidding about the Google Maps.

Inside were books, books and more books. Chris, thanks a lot!

Out with the old … in with the two

OmniThreadLibrary 1.05 was a great success …

… but all good things must come to an end. Don’t worry, I’m not ending the project. It’s just that there’s a new player in town. Ladies and gentlemen, in the left corner I give you …

… OmniThreadLibrary 2.0! Get it while its hot!

OmniThreadLibrary 2.0 TODO

Only few things to do before the alpha release:

• Parallel.Pipeline must support cancellation;
• Parallel.ForEach would benefit from task-local state (maybe will be implemented after the 2.0 release);
• Cleanup various small TODOs;
• Add project files for Delphi XE;
• Retest everything.

OmniThreadLibrary Documentation

Thanks to the great people of DevJET software there may something (finally) happen on the very much neglected documentation front. They have donated a Documentation Insight license to the OmniThreadLibrary and I intend to use it to add documentation here and there while I’m working on the project.

A big Thank You to the DevJET!

The next part of the puzzle is how to export XmlDoc documentation into some useful form. I know Delphi XE can do html+javascript export but I don’t believe that it’s included in the Professional release which I have (but correct me if I’m wrong). It would be even better if it could be converted into the Wiki format used by the Google Code. Can you help me with suggestions?

Multistage processes with the OmniThreadLibrary

The OtlParallel unit in OmniThreadLibrary offers some high-level solutions that allow you to easily run some kinds of processes in parallel. Up to now it supported self-contained background calculations (Parallel.Future), independent parallel processes (Parallel.Join) and loop calculations where the background task is stateless (i.e. it only depends on the input – the loop value – and not on calculations done on other inputs – loop values; Parallel.ForEach). But lots of time processes don’t fall into any of those categories.

Mason Wheeler recently suggested adding support for multistage processes. Over the weekend I’ve implemented it in the OmniThreadLibrary and now this feature is ready for you to start testing. Great thanks to Mason for his suggestion and also for work on the implementation and design of newly introduced Parallel.Pipeline.

ITDevCon 2010

ITDevCon has finished almost a 24 hours ago but I was simply too tired to write anything about it until now. In short – great event!

There was close to 50 participants and sessions were in most cases nicely filled up. Three tracks were running in parallel and sometimes it was quite hard to choose the most interesting session. I was quite sad as I was not able to see any of Alexander Alexeev’s sessions on debugging (the first one overlapped with my own session and in the second case seeing Ray Konopka speaking of user interface design trumped all other options). Besides Ray session which was definitely the best on the ITDevCon (Ray told me to say so ;) ) I have to mention Paweł’s expose on DataSnap and Thierry’s multitouch session which were both very interesting.

Embarcadero tech support should pull its act together

For the past few days I have fought a small war with the Embarcadero registration system. For some unknown reason, Delphi XE installed fine but when I tried to register it, I got back

“No  valid  license information found for Embarcadero(R) Delphi(R) XE.
You  must provide a valid serial number in order to use Embarcadero(R)
Delphi(R) XE. Do you want to run the registration wizard again?”

The funny thing was that my registration count got incremented nevertheless! Of course I only noticed this after five failed installs when the sixth attempt told me that I’ve used my installation bonus and I can install no more Delphis :( Most probably there was some clash between the fresh Delphi XE installation, previous trial version and pre-previous [sorry I can’t tell you about it] version. Of course, I did try to fully remove all remnants of all installations but obviously I missed some important part.
At that point I did the right thing and opened a support case. A guy from Embarcadero support responded soon enough with a note that he bumped my registration count and I can retry. Of course I already knew that retrying won’t help so I asked him how to fully clean up my machine.
Then I waited 24 hours …
Next day I sent few more mails and finally got a pretty comprehensive clean-up list. Then only problem was that it was completely b0rken (as you’ll see in a second).
So I sent a response to that mail, requesting clarification of some items.
No reply.
Then the weekend occurred …
Nothing happened yesterday either. At last I sent another mail urging the support to please respond to my question and bump my registration count again (as I was already up to 10 registrations – only one of them really successful, and that one was on another machine).
Finally I got the reply. Actually, I received a mail stating that my registration count was bumped again. Nothing else. No reply to any of my questions. Nada, zip, zilch.
It looks like Embarcadero has tech support that can only do two things. 1. Bump registration count. 2. Send around bad instructions on how to remove remnants of Delphi from your system.

Embarcadero people, if you really want Delphi to be successful again, you should educate your tech support.
Luckily, I know enough about Delphi to clean my system even without answers to my questions so I was able to finally install XE. Hurrah!
I still have to prove that the cleanup list is really really bad, yes? Well, here it is, directly pasted from the support mail. My comments are in italic.
1. Uninstall the product via the "Embarcadero Rad Studio 2010" entry in "Uninstall a Program" in the Control Panel

I’m trying to uninstall Delphi XE so “Embarcadero RAD Studio XE” would be a better choice.

2. Remove the C:\Program Files\Codegear\Rad Studio\8.0 directory

Actually, folder is named \Embarcadero\, not \Codegear\.

3. Remove the C:\Users\Public\Public Documents\Rad Studio directory

That would trash the Delphi 2010 install. Only 8.0 subfolder has to be removed.

4. Remove the c:\ProgramData\CodeGear\Rad Studio\8.0 directory

Again, it’s \Embarcadero\.

5. Remove the HKEY_LOCAL_MACHINE\SOFTWARE\CodeGear\BDS\8.0 registry key

Same here.

6. Remove the HKEY_CURRENT_USER\SOFTWARE\CodeGear\BDS\8.0 registry key

And here.

7. Remove the
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{A2B58B18-5D04-4006-9713-B6945880746E} registry key
8. Remove the
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\CodeGear RAD Studio 2010 registry key

Embarcadero RAD Studio XE, not CodeGear RAD Studio 2010.

9. Remove folders with a GUID name in C:\Documents and Settings\All Users\Application Data\   

In item 3, C:\Users was used. Here, C:\Documents and Settings. On every computer, one of those will not be the correct choice.

9a. You can find the specific GUID folder names to remove by browsing to the subkeys of HKEY_LOCAL_MACHINE\Software\MimarSinan\InstallAware\Ident.Cache\
10. Remove the following files from Windows\System32:   
10a. *120.bpl   
10b. *120.jdbg   
10c. *120.xml   

In Delphi XE, those files end in 150, not 120. And on a 64-bit system they live in \Windows\SysWOW64.

10d. bdeadmin.*   
10e. cc32*.dll   
10f. midas.*
Dear reader, please tell me – am I overreacting? Or somebody should be fired here for not doing his/her work (and I definitely don’t mean the poor tech support guy I was talking to)?

Blaise Pascal Magazine Rerun #3: Real World Enumerators

This article was originally written for the Blaise Pascal Magazine and was published in Issue #5.

Relevant Delphi code is available at http://17slon.com/blogs/gabr/BPM/RealWorldEnumerators.zip

Blaise Pascal Magazine Rerun #2: Advanced Enumerators

This article was originally written for the Blaise Pascal Magazine and was published in Issue #4.

Relevant Delphi code is available at http://17slon.com/blogs/gabr/BPM/AdvancedEnumerators.zip.

Blaise Pascal Magazine Rerun #1: Introduction to Enumerators

This article was originally written for the Blaise Pascal Magazine and was published in Issue #3.

Too many what?

Delphi 2007.

Conditional symbols:

xFullDebugMode;xUseReservationList;xSimulateDVBMaster;xNoDongle;NoExcept;NoTrace;xLogDriverCalls;xNoDVBMaster;DEBUG;xDuplicateOutputToSimulator;xLogDelaying;TestSubtitleDelay;CatchUseOfFreedInterfaces;LockDriverCalls;NoDVBNowAdjustment;DetailedTooLateError

16 symbols.

257 characters in total.

Too many? Was Delphi 2007 built in DOS times?

Can somebody please check this with Delphi XE?

Ingenious!

Sometimes you find a code fragment that’s so great you only wish it was your idea …

const
  fm: array[Boolean] of Word = (fmCreate, fmOpenWrite);
fs := TFileStream.Create(fileName, fm[FileExists(fileName)]);

Obvious – in retrospective.

OmniThreadLibrary webinar – recording

While I was on vacation, Robert published a recording of my OmniThreadLibrary webinar. Happy downloading!

So terribly sorry …

Writing code ain’t that hard a job.

Writing correct code, well that’s a much harder venture.

And what can I say about writing correct multithreaded code? Only that it’s close to impossible.

That’s exactly why I started writing OmniThreadLibrary – I needed well-tested framework for multithreading processing.

But alas! there are bugs in OmniThreadLibrary too. Not many of them, true, but still they are there. Some are squashed soon, other remain hidden for a long time.

I found one such bug few days ago when I was searching for a reason why some code doesn’t parallelize well. In theory the speedup should be close to 8x (on a 8 core machine) but in practice the parallel code was only faster by a factor of 2 to 3.

At the end of a long day I found out a bug in the TOmniBlockingCollection that prevented all threads to be executing at once. Only two or three threads were really working and they did all the job – but only two to three times faster, of course.

The bug is now fixed in the trunk. Anybody can do a checkout and get a perfect (well, maybe not perfect but definitely a better-working) code.

Because this was quite an important fix I’ve also incorporated it into the 1.05 version of this unit. You can download it here. If you’re using 1.05 and TOmniBlockingCollection, you surely want to download the update.

I’m really sorry for letting this stupid bug slip through my testing. Won’t happen again. (Or maybe it will. Probably it will. Oh, heck, it surely will. I’ll just try to make such problems very rare. Promise.)

Scheduled OmniThreadLibrary presentations

On July 27th, I’ll be speaking for the Virtual Delphi Users Group. The topic will be “Parallel programming with OmniThreadLibrary”. Be aware that you have to register in advance if you want to participate (all questions will be answered!). The recording will be available some time after the presentation. [At the moment, the VDUG server has some occasional problems so be patient and/or retry later.]

On November 18 and 19, I'll be speaking at ITDev Con 2010. Two of my presentations will focus on OmniThreadLibrary and the third one on memory management with FastMM. All presentations will be given in English language.

TDM Rerun #16: Thread Pooling, The Practical Way

As we found out, the system thread pool in Windows 2000 (plus XP and 2003) is woefully inadequate for any serious use. It seems that its designer was only thinking about really trivial usage and expected everyone else to create their own thread pool. Luckily, it was possible to create a fully-fledged pooling layer based on the system thread pool and the application was saved.
- Thread Pooling, The Practial Way, The Delphi Magazine 112, December 2004

The December 2004 issue describes one of my first serious forays into the muddy waters of parallel processing. The article describes a work item pooling mechanism built in Windows (QueueUserWorkItem) and a management wrapper that I built around this API to make its use bearable.
The GpWinThreadPool unit (described in this article) has been later replaced with a TThread-based pool of my own design and that unit (GpThreadPool) was superseded by the thread pool built in the OmniThreadLibrary. The use of the code described in this TDM article is not really recommended (except maybe for the educational purposes).
Links: article (PDF, 116 KB), source code (ZIP, 2,6 MB).

--- Published under the Creative Commons Attribution 3.0 license

Gp* units now available via SVN

Because YOU asked for it … my units are now available on Google Code.

Important DSiWin32 and GpSecurity update

1) Lots of things fixed in DSiWin32 and great thanks to Christian Wimmer for pointing out the problems and suggesting some solutions.

2) By my mistake a very internal GpSecurity containing parts of JWA got included in some downloadable ZIP files. This was a direct violation of the JWA license and I apologize deeply for that. To fix this problem, a new GpSecurity was released which depends on the JWA.