From time to time I got error reports about "error 1400" in OTL and my response is always the same: "You are using it wrong." I never wrote anything about that, though. My bad :(
This time, however, I received an excellent error report with a minimal reproducible example (thanks!), which gave me an excellent excuse for this blog post ;)
Let me start with a little demonstration...
TLThere were, however, other interesting results that my simple benchmark pointed out. More on that in a moment, but first…; DR Yes, allocating from the top is slower. No, the difference is not big and in most cases you’ll not even notice it.
A note for Slovenian readers.
Ne pozabite – jutri bom predstavil novo knjižnico System.Threading, ki olajša razvoj hitrih in odzivnih programov za vse platforme, ki jih podpira RAD Studio – Windows, OS X, iOS, Android – ter v obeh programskih jezikih – Delphi in C++. Ogledali si bomo tudi novosti v knjižnici OmniThreadLibrary.
Za več podatkov in prijavo kliknite tu!
This is an invitation for the XE7 workshop in Slovenia which is targeted at Slovenian programmers.
RAD Studio XE7 prinaša nov način paralelizacije programov. S kodo, zbrano v enoti System.Threading lahko na enostaven način pohitrite nekatere funkcije v programih, tako da se izvajajo sočasno z drugo kodo. Še lepše - System.Threading deluje na vseh podprtih platformah, torej poleg Windows tudi na OS X, iOS in Androidu. Na delavnici si bomo podrobno ogledali nove načine paralelizacije in pregledali druge knjižnice, ki podpirajo hkratno (paralelno) izvajanje v okolju Windows.
Predavanje bo v četrtek, 27. novembra, 2014.
A deceptively simple question – how do you update a progress bar from a ForEach loop – popped up on the Google+ OmniThreadLibrary community. The implementation turned out to be quite tricky so I created an example (55_ForEachProgress) which is now part of the OmniThreadLibrary SVN repository.
The starting point was a simple Parallel.ForEach loop which I further simplified in the demo.
Parallel
.ForEach(1, CNumLoop)
.Execute(
procedure (const task: IOmniTask; const i: integer)
begin
// do some work
Sleep(1);
// update the progress bar - how?
end
);We cannot simply update the progress bar from the ForEach executor as that code executes in a background thread and one must never ever access VCL GUI from a background thread! It is also no good to send “please update” Windows messages to main thread as Parallel.ForEach is by default blocking – it waits for all workers to stop working – and messages won’t be processed during ForEach execution.
Besides being a programmer, I like taking pretty pictures (and working with wood, but that’s another story). And from time to time I like to print some pictures and hang them on a wall.
Because framing pictures is expensive and, frankly, most of them don’t deserve it, I like to print a “virtual frame” – a thin black border followed by a wide white border – around each photo. Until recently, I was adding them in some stupid program I found on the Internet (it doesn’t deserve to be named), but I was always having problems because you had to specify border width in pixels – and if I wanted to set the same border (for example, a 1,5 mm black border followed by a 20 mm white border) for a set of pictures in a different resolutions, I had to calculate pixel sizes for each photo, which was a pain.
So I sat down and wrote a border-adding program. It allows you to specify border sizes in “real” measures, it can process one picture or a folder full of them and it can add a suffix to processed pictures (plus it will skip files containing this suffix if they are found in the source folder) and that’s about it. It is small, simple, solves a real problem and – what I found the most impressing – I wrote it in a single working day. Truth to be told, I needed about a month of work – an hour here, 15 minutes there – but all together I needed less than eight hours. That is the power of Delphi!
If you have a need for such tool, go ahead and download it from my Dropbox. You can also get the source, which is released to a public domain (i.e., do with it whatever you want). It depends on few external libraries, which are all released with an open source license: JCL, JVCL, GpDelphiUnits, OmniXML. At the moment the GUI is written in VCL, because I really didn’t want to spend too much time on this project and I know VCL by heart, but if somebody wants to go ahead and rewrite it in FireMonkey, I certainly won’t stand in his/her way.
This Tuesday (March 4th) Embarcadero and Marand are organizing a RAD Studio XE5 presentation in Ljubljana. We’ll be showing great features that were implemented in RAD Studio in the past few years – from 64-bit Windows compilation, Unicode support and styling to multiplatform development, LiveBindings and FireDAC. For the first time in Slovenia we’ll be presenting the new REST library and C++ support for iOS.
Presentation will be given in Slovenian language. Register here.
I’ll be presenting the full XE5 development powers – developing for Windows, OS X, iOS and Android in Ljubljana, on September 18th.
Long time ago, in August, JimVern asked on the OmniThreadLibrary forum:
I am creating a Windows Service to handle report requests for our clients. Each client has multiple users who may run a report. I want to serialize the report requests for one client so that only one report will run for a client at a time (FIFO), but reports can run simultaneously for different clients.
Since users can submit report requests anytime, I need to be able to append a new request to the end of an existing client FIFO queue without interrupting the execution of the current report. Or, if the client doesn't have a queue yet (e.g., it's the first report request), then I need to be able to add a client queue without disrupting any reports running for other clients. Once a client queue is empty, I want to destroy the client queue.
I am new to OTL, but it looks like it will easily do what I need. I'm just not sure where to focus my attention. Is there a way to create named task groups that can execute the top task of each group simultaneously, where tasks can be appended to an existing group, and where groups can auto-terminate when the tasks are all executed? Perhaps there an example project or a similar post that I can look at to get started? I'm still somewhat of a novice when it comes to threads, so any advise is also welcome.
My answer at that time was just a pointer in the correct direction:
Instead of running multiple tasks for one client, why don't you consider running one task per client and sending workload requests to that task?
You could then schedule each such task into a thread pool. The task would run in a loop and process requests. When it runs out of requests, it would shut down. The main program would be responsible for starting a new task if not already running. [There's a small race condition here - you would have to maintain a shared structure enumerating live tasks that would be modified from the task code and from the main program and it would have to be locked-accessed.]
I promised Jim a working solution but then I got lost in all sorts of other, more important projects and it was only few days ago that I was able to finish the job. Sorry, Jim :(
An example project is available in the OmniThreadLibrary repository in folder examples/report generator. This post describes the inner workings of that project.
procedure TForm125.Button1Click(Sender: TObject); var button: TButton; begin button := Sender as TButton; button.Caption := 'Working ...'; button.Enabled := false; Sleep(5000); button.Enabled := true; button.Caption := 'Done!'; end;
Now, your boss says, you have to make it parallel so the user can start three copies of it. (You also have to add two new buttons to the form to start those instances but that’s easy to do.)
This article was originally written for the Blaise Pascal Magazine and was published in Issue #12.
Relevant Delphi code is available at
http://17slon.com/blogs/gabr/BPM/Communication.zip.
Today I’m revisiting example from November 2011. This time I’ll solve it using the new Parallel.BackgroundWorker abstraction.
Part of the Zarko’s requirements (see the original post for full text) was a cancellation support.
At any time the "master" thread could be signaled to terminate (from the app's main thread) all child threads (and itself).
When I was originally implementing this using the Parallel.Pipeline abstraction I had to put some work into the cancellation support. The main reason for this was inappropriate abstraction – Parallel.Pipeline is designed around the data flow processing and supports only a basic cancellation of the “stop everything” type. To be fair, that would comply with the Zarko’s requirements, but I wanted nicer solution where you can stop processing and then continue with a new work item without rebuilding the background thread mechanism. [Pipeline solution inherently supports cancellation but you cannot recover from it – to continue processing one would have to destroy the pipeline and build a new one.]
When I wrote OmniThreadLibrary in Practice [2]–Background Worker and List Partitioning in November, I noticed that OmniThreadLibrary needed another high-level abstraction, a background worker. I published some ideas on how the background worker interface could look. Those ideas were later developed in a full-fledged high-level solution.
Background worker is designed around the concept of a work unit. You create a worker, which spawns one or more background threads, and then schedule work units to it. When they are processed, background worker notifies you so you can process the result. Work items are queued so you can schedule many work items at once and background thread will then process them one by one. [Actually, Parallel.Pipeline is used as a base for the implementation.]
type TOmniCS = record strict private ocsSync: IOmniCriticalSection; function GetSyncObj: TSynchroObject; public procedure Initialize; procedure Acquire; inline; procedure Release; inline; property SyncObj: TSynchroObject read GetSyncObj; end;As records don’t provide automatic initialization via parameterless constructor, the code is slightly tricky. The record contains a field (ocsSync) that contains the interface reference used to do real work. This interface is initialized in the Initialize method, which is in turn called from the Acquire and Release.
procedure TOmniCS.Acquire; begin Initialize; ocsSync.Acquire; end;
Here's my (simplified) task I would like to solve using OTL (Delphi XE) - my real-world task is somehow more complicated but can be described as:
input: a string. output: a TStringList containing characters (one per entry) of the input string.
Example: input: "delphi"
A background thread ("master") grabs the input string and splits it into several pieces (let's say 2): "del" and "phi". For each of the split strings a new thread ("child") is created that fills in the TStringList (output) with characters from the section of the string it receives.
At any time the "master" thread could be signaled to terminate (from the app's main thread) all child threads (and itself).
When everything is done the app's main thread processes the string list.
Preferably, the order of the characters should (when all ends) be 'd', 'e', 'l', 'p', 'h', 'i' (note that characters are actually items in the resulting string list).
From time to time I get a question on the OmniThreadLibrary forum that could be of interest to other OmniThreadLibrary users. As I believe that many of you don’t follow that forum I have decided to repost such questions on the blog (and provide answers, of course).
The first question in this series was asked by GoustiFruit:
I need to download a list of web pages, extract data on them and then store these data in a SQLite database. The downloading/extracting part will happen in multiple threads (I'm using Synapse), but querying the database needs to be done asynchronously as I can only have one concurrent access to it.
So I'd like to know how (conceptually) I could implement that ? My first idea is to run one single thread for querying the DB, run several threads for each Url to download/analyse and then exchange messages between these threads and the querying thread, with the extracted data as a parameter: does it make sense or am I totally wrong? I also read things about the connection pool concept but I'm not sure if it applies when only one connection is allowed at one time?
I installed Vista RTM immediately as it was available on the MSDN.
That was quite a surprise for me as I thought that many months would pass before my first Vista installation. But alas, I had to write an article on Vista for Slovenian leading computing magazine Monitor and I just got completely new Vista-ready laptop which I could use for playground. (It got Windows XP Home preinstalled - yuck!)
Anyway, I installed Vista (or, to be more precise, Windows Vista Ultimate) on that laptop. Everything worked fine (even though I had to scrap together few drivers from all over the internet). Everything except that I couldn't print. I have this nice colour laser printer (Samsung CLP-510, really smooth) in the office, and it is connected to the Windows 2003 Server. All other computers (running XP everyone of them) could use this shared printer but not the Vista, nooo sir.
It was quite an interesting malfunction, too - when I tried to connect to my shared printer, Vista started copying drivers (XP drivers, of course), and then reported that driver installation 'ran out of memory'. OK, I only have measly 2 gigs installed, but still...
I tried a different trick - installed printer driver locally (no problem here) and only after that tried to connect to the shared printer (in hope that currently installed driver will be used). But nooo, Vista reported same error again.
I gave up then saying: "Oh, well, it's only about two months until Vista is available for general public. Surely Samsung will have Vista drivers ready by then." But the problem was still nagging me and I spilled out my soul to my friends on Slovenian Delphi forum. Somebody has an interesting idea - why don't I connect to the Samsung as if it were a TCP/IP printer and then spool EMF content to the server. Still, he had no idea how to make this work in practice.
I searched for the solution occasionally but found out that Microsoft documentation on that topic is really hard to find if you don't know what you're searching for. I did'nt spend much time on that, though, as I could always use the printer from my main computer.
Today I returned to the problem and somehow managed to find the answer in Microsoft's documentation. It really is simple - when you already know how to do it :( .
For anybody trying to solve a similar problem - and for all of you that just want to learn more - here is the solution.
First, install Print Services for Unix on the Windows 2003 Server (Control Panel, Add or Remove Programs, Add/Remove Windows Components, Other Network File and Print Services, Details). Make sure that the TCP/IP Print Server service is running.
Next, install LPR Port Monitor feature on the Windows Vista (Control Panel, Programs and Features, Turn Windows features on or off, Print Services).
Create new port on the Windows Vista computer. Run Printers and clik Add Printer. Choose Add a local printer. Select Create a new port setting and set Type of port to LPR Port.
Click Next and enter address of the server providing Unix printing services and name of the printer.
Click OK and proceed as usual - select printer manufacturer etc.
Using this procedure I managed to get my Samsung working from the Vista.
I learned a lot and maybe I will be able to use this knowledge in the future.
Maybe it will help somebody out there with a similar (or completely different) problem.
It didn’t help me at all :(
You see, it is February already and Samsung published updated printer drivers two days ago.
