Value Never Used – Or Is It?

Sometimes, dcc32 is just plain stupid …

I have this code in the OmniThreadLibrary:

function TOmniThreadPool.Cancel(taskID: int64): boolean;
var
  res: TOmniWaitableValue;
begin
  {$IF CompilerVersion >= 22}
  Result := false; // not really used
  {$IFEND}
  res := TOmniWaitableValue.Create;
  try
    otpWorkerTask.Invoke(@TOTPWorker.Cancel, [taskID, res]);
    res.WaitFor(INFINITE);
    Result := res.Value;
  finally FreeAndNil(res); end;
end;

The IFDEF part is there because otherwise compiler complains about result being potentially undefined.

[DCC Warning] OtlThreadPool.pas(1315): W1035 Return value of function 'TOmniThreadPool.Cancel' might be undefined

I’ve first observed this with the XE and it was fixed after the XE2 (can’t repeat this with XE8, but I didn’t check every compiler inbetween.)

New Home for OmniThreadLibrary

OmniThreadLibrary has been moved from Google Code to GitHub.

I still have to move existing issues as they were not moved automatically.

New Home for My Units

My collection of open-sourced units has been moved from Google Code to GitHub.

XE8 is out …

… and OmniThreadLibrary works fine with it. (That’s it. Just wanted to let you know.)

Runtime SQL Query Builder

I don’t want to use long SQL strings in the code.

Really.

Firstly, it is a pain to write long multiline strings in Object Pascal. (Embarcadero, are you listening? Can we please get multiline strings in Delphi? Please?)

Secondly, I’m very OCD when it comes to compiler watching my every step. I absolutely hate runtime errors and I want every problem to be detected during the compilation. That includes typos in database field names and changes attributed to the entropy (i.e. changes in the database structure).

On a tangent, that’s why we are creating and managing tables and databases in the code. We have all fields defined as constants, all SQL statements created in runtime and everything stored under version control.Upgrading a database to a version required by the program is therefore a very simple operation. (Downgrading is still a problem. Downgrading is hard.)

Thirdly, concatenating query strings together when you are dynamically building a query string based on the conditions in the program is a mess.

That’s why long years ago I created a simple runtime query builder. It was dormant (in the “good enough for me” state) for a long time but recently I required more functionality from it and I extended it further. It is now in a state which may be useful for somebody else but me so I’m releasing it as an open source under a BSD license.

For impatient: Get GpSQLBuilder on GitHub.

Unit Testing Recap & Downloads

The Unit Testing workshop was pretty much full.

It was good to see that many interested Delphi programmers in one room. Thanks for coming, everybody!

The downloads for the workshop are now online.

What’s New in Delphi XXXX

A great list of “What’s new” for Delphis from 3 to XE7 can be found on StackOverflow.

(just noticed and thought I’d better pass this on)

Unit Testing, Mocking, Inversion of Control

On March 19th (that’s next Thursday), I’ll be leading a workshop dedicated to unit testing Delphi programs. We’ll start with basics and and cover different tools (DUnit, DUnit2, DUnitX, TestInsight) and different programming methodologies.

Register here.

OmniThreadLibrary 3.04

OmniThreadLibrary 3.04 is now released. Get it here or check out the release-3.04 tag (or just follow the trunk, which is the best way to experience the OmniThreadLibrary).

Some other links you may need:

Home page: http://www.omnithreadlibrary.com/
Google+ community: https://plus.google.com/communities/112307748950248514961
Downloads: https://drive.google.com/folderview?id=0BwqVlLNTK4OxVEgzZnZGM1FNMGc&usp=sharing#list
Issue tracker: http://code.google.com/p/omnithreadlibrary/issues/list
SVN checkout instructions: http://code.google.com/p/omnithreadlibrary/source/checkout
Author's blog: http://thedelphigeek.com
Author's home page: http://primoz.gabrijelcic.org
Documentation wiki: http://otl.17slon.com/book/
Documentation book: http://leanpub.com/omnithreadlibrary

Changelog

New features

• Implemented simple and fast Parallel.&For which supports only integer ranges.
• Implemented Parallel.Map.
• Added overloaded Execute methods to IOmniParallelInitializedLoop and IOmniParallelInitializedLoop<T> so that IOmniTask parameter can be passed to the executor.
• Implemented Run overloads that internally call Invoke to start thread worker.
• Implemented TOmniValue.HasArrayItem.
• Implemented TWaitFor which supports waiting any/all and is not limited to 63 objects.
• IOmniTaskControl can wait on any number of comm handles and wait objects. That enables support for >60 tasks in the OtlThreadPool.
• Implemented TOmniBlockingCollection.ToArray<T>.
• NumTasks parameter (OtlParallel) can be negative. In that case, specified number of cores will be reserved for other purposes and all other will be used for processing.
• Example: If NumTasks(-2) is used when process has access to 8 cores, 6 of them (8 - 2) will be used to run the task. 
• Removed project groups.
• Tested with 2007-XE7.

Bug fixes

• Fixed race condition in IOmniPipeline termination code [tnx to Dean Hill].
• Creating parameters with the array overload (IOmniTaskControl.SetParameters(['From', 0, 'To', 99])) was not working.
• Fixed problems in demos 33_BlockingCollection and 47_TaskConfig.

New demos

• 55_ForEachProgress: Demonstrates progress bar updating from a ForEach loop.
• 56_RunInvoke: Simplified 'run & invoke' low-level API.
• 57_For: Simple and fast parallel for.
• 58_ForVsForEach: Speed comparison between Parallel.ForEach, Parallel.For, and TParallel.For (XE7+).
• 59_TWaitFor: Demo for the new TWaitFor class.
• 60_Map: Demonstrates the new Parallel.Map abstraction.
• 61_CollectionToArray: Demonstrates the new TOmniBlockingCollection.ToArray<T> method

OmniThreadLibrary 3.04 Release Candidate

RC for the new OmniThreadLibrary release is available here.

This version should work with Delphis from 2007 to XE7. Win32 and Win64 are supported for Console and VCL applications.

Converting Collection to an Array

Blocking collections (IOmniBlockingCollection) are basic elements for data storage and transfer in many high-level OmniThreadLibrary abstractions. They can, however, be somewhat clumsy when you want to read data from them, as there is no indexed access, just the basic “give me next element” enumeration.

To help a bit, SVN now includes three new class functions in the TOmniBlockingCollection class – ToArray<T>, ToArrayIntf<T>, and ToArrayRec<T>.

class function ToArray<T>(coll: IOmniBlockingCollection):
  TArray<T>;

class function ToArrayIntf<T: IInterface>(
  coll: IOmniBlockingCollection): TArray<T>;

class function ToArrayRec<T: record>(
  coll: IOmniBlockingCollection): TArray<T>;

Parallel Map

At my latest parallel programming presentation a participant suggested that I should extend the OmniThreadLibrary with a parallel mapping abstraction. Dear sir, here is a gift for you.

var
  numbers: TArray<integer>;
  odds   : TArray<string>;
begin
  //initialize the `numbers` array (not shown)
  odds := Parallel.Map<integer,string>(numbers,
    function (const source: integer; var dest: string): boolean
    begin
      Result := Odd(source);
      if Result then
        dest := IntTostr(source);
    end);
  //do something with the `odds` array (not shown)
end;

Implementing Record Assignment Operator [2]

Yesterday I hinted at having a working (and easy to use) solution which allows you to detect a record assignment (or copying, if you want) and to access both records (left and right part of the assignment operation). You can also modify record data when an assignment is detected. I also mentioned that my approach is very ugly, unstable, likely to cause problems in future Delphis and so on. You should read this article for the technical details, not to really use my code in an application. You have been warned!

Before I jump to hardcode Delphi stuff, I’d like to mention few different approaches that various smart people have already implemented. Most of them slipped below my radar in the past – and I’m guessing this may be the case for you too – so I’m listing them here in case you want to do more research on the topic.

Implementing Record Assignment Operator [1]

Following the yesterday’s hunch, I did some research and I came up with a way to detect when a record is copied (simple, supported, working everywhere) and even to access both records (source and target) during that operation (unsafe, unsupported and currently working only for Win32).

Now I can write such code:

Implementing Destructor for a Record

Smart records in Object Pascal are very nice, but they have a stupid limitation – you cannot implement a destructor for a record.

A solution for that is quite simple and can be found all over the internet – add an interface to this record and implement the cleanup in this interface. To make it even simpler, you can use the IGpAutoExecute interface from the completely free GpStuff unit.

Blast from the past: Turbo Pascal raytracer

Paweł’s recent post reminded me of a veeeeeery old raytracer I wrote in years 1985/89. (Yes, young ones, we had computers then!)

Header states:

1985-07-10:
  Computer: VAX-11/750
  Language: Pascal V2
  Graphics: VT100

1988-02-03:
  Computer: IBM-PC
  Language: Turbo Pascal 4.0
  Graphics: EGA 640 x 350 x 16

1989-07-03:
  Computer: IBM-PC/AT
  Language: Turbo Pascal 5.0
  Graphics: VGA 800 x 600 x 256

Quite a blast from the past – the 800 x 600 in 256 colors (out of a great palette of 262144 colors!) was more or less the greatest graphic card which you could get for PC at that time. And the first version was running on a VT100 terminal with an additional graphics board. Sadly, I don’t have that code anymore :(.

Attribute-based Command Line Parsing

Oh, command line parsing, the old enemy of mine!

In ye olden days, when I was learning Pascal, I did some programming on VAX/VMS systems, where you could (and should) leave the job of command line parsing to the DCL – the command line interpreter (something like CMD.EXE on Windows). You just wrote a definition file (see example on the right, found on the web) and DCL did the rest.

Since those days I hated that I have to parse command line programmatically. Sure, there were parsing libraries – and I wrote few of them myself – but I was never happy with them. There was always too much code to write.

Then I was looking for good examples on using attributes in Delphi and suddenly it occurred to me that I could use them for command line parsing, too! And that’s how GpCommandLineParser was born.

Before continuing, I should mention that I have based it (concept-wise, implementation is all my own) on the CommandParser unit, which comes with Delphi and can be found in Samples\Object Pascal\Database\dbExpress\Utils. This is a very nice but mostly unknown command line parser which I can definitely recommend.

OmniThreadLibrary 3.04 beta 1 – testers needed

I have (finally!) implemented support for more than 60 simultaneous tasks in the OTL thread pool and for more than 60 waitable events in the IOmniTaskControl. This also enables support for massively parallel execution in OtlParallel abstractions. IOW, Parallel.ForEach will run correctly on machines with 64 and more cores without needing to call NumTasks(60).

I’ve been testing this change and I think that it’s working correctly, but it is a big (and potentially dangerous) modification and so I’d like to see additional testers trying the new version in real programs before I submit it to the SVN.

So – if you like to live on the edge, if you’d like to help improve OTL and especially (but not required!) if you can test the code on a machine with 64 or more cores, please download the 3.04b1 from Dropbox and try it out. You can post reports here or send them to my email (my address is in the header of every OmniThreadLibrary file). In case of a problem, a short, repeatable, self-contained example will be of an immense help.

Thanks!

Parallel programming with XE7–reminder

A note for Slovenian readers.

Ne pozabite – jutri bom predstavil novo knjižnico System.Threading, ki olajša razvoj hitrih in odzivnih programov za vse platforme, ki jih podpira RAD Studio – Windows, OS X, iOS, Android – ter v obeh programskih jezikih – Delphi in C++. Ogledali si bomo tudi novosti v knjižnici OmniThreadLibrary.

Za več podatkov in prijavo kliknite tu!

Pohitrite programe z RAD Studiom XE7

This is an invitation for the XE7 workshop in Slovenia which is targeted at Slovenian programmers.

RAD Studio XE7 prinaša nov način paralelizacije programov. S kodo, zbrano v enoti System.Threading lahko na enostaven način pohitrite nekatere funkcije v programih, tako da se izvajajo sočasno z drugo kodo. Še lepše - System.Threading deluje na vseh podprtih platformah, torej poleg Windows tudi na OS X, iOS in Androidu. Na delavnici si bomo podrobno ogledali nove načine paralelizacije in pregledali druge knjižnice, ki podpirajo hkratno (paralelno) izvajanje v okolju Windows.

Predavanje bo v četrtek, 27. novembra, 2014.

Kliknite za prijavo