Blaise Pascal Magazine Rerun #2: Advanced Enumerators

This article was originally written for the Blaise Pascal Magazine and was published in Issue #4.

Relevant Delphi code is available at http://17slon.com/blogs/gabr/BPM/AdvancedEnumerators.zip.

In the previous issue you could learn some basic facts about enumerator basics. I described enumerator essentials, walked you through the compiler-provided execution framework and described how to write your own enumerator. Today I’ll show you how to implement more than one enumerator in a class and how to write enumerators that take parameters.

To start, let's write some test framework. Instead of creating a new class implementing an enumerator, we'll just inherit from the TStringList. That way, we get existing TStringList enumerator 'for free'.

type
  TBPStringList = class(TStringList)
  end;

To start testing, we’ll write a simple VCL application with one button and one listbox. In the form’s OnCreate event we’ll fill an instance of the TBPStringList class with initial contents and in the form’s OnDestroy event we’ll destroy this instance.

procedure TfrmBPAdvancedEnum.FormCreate(Sender: TObject);
var
  ch: char;
begin
  FslTest := TBPStringList.Create;
  for ch := 'a' to 'z' do
    FslTest.Add(ch);
end;

procedure TfrmBPAdvancedEnum.FormDestroy(Sender: TObject);
begin
  FreeAndNil(FslTest);
end;

To test the TStringList enumerator (and our application) we’ll use this small code fragment, assigned to a TButton’s OnClick.

procedure TfrmBPAdvancedEnum.btnStandardEnumClick(Sender: TObject);
var
  ln: string;
  s: string;
begin
  ln := '';
  for s in FslTest do
    ln := ln + s;
  lbLog.Items.Add('Standard enumerator: ' + ln);
end;

This code will show text “Standard enumerator: abcdefghijklmnopqrstuvwxyz” in the list box. Nothing exceptional, let’s move further.

Now that we have testing framework set up, we can start the real work. I promised to show the approach that would allow you to use multiple enumerators in a single class. To understand how that could possibly work we should return to the previous issue where I discussed the compiler support for enumerators. I mentioned the fact that a simple loop

for obj in list do
  DoSomething(obj);

is translated by the compiler into following pseudocode.

enumerator := list.GetEnumerator;
while enumerator.MoveNext do begin
  obj := enumerator.Current;
  DoSomething(obj);
end;
enumerator.Free;

The important (but not obvious) fact is that enumerator is created by calling GetEnumerator method of the object, record or interface used in the in..do part of the for loop. After the enumerator is generated, this parameter (list in the example above) is not needed anymore.

In the test framework we used a small for..in loop to list the contents of the TBPStringList.

for s in FslTest do
  ln := ln + s;

This for statement is compiled into an equivalent of the following Delphi code fragment.

enumerator := FslTest.GetEnumerator;
while enumerator.MoveNext do begin
  s := enumerator.Current;
  ln := ln + s;
end;
enumerator.Free;

If we want to use a different enumerator, we must trick the compiler to call another GetEnumerator method. As we cannot change the name of the method that is called, we must use a different object in the for..in loop. In other words, we must do.

for s in FslTest.SecondEnumerator do
  ln := ln + s;

In this example, SecondEnumerator is not an enumerator but an enumerator factory, that is, a class that is implementing GetEnumerator function. (Yes, this is complicated and mind-boggling but it’ll be much clearer once I start showing the actual code.)

Compiler will translate this code into following fragment.

enumerator := FslTest.SecondEnumerator.GetEnumerator;
while enumerator.MoveNext do begin
  s := enumerator.Current;
  ln := ln + s;
end;
enumerator.Free;

Reversing the list

Enough talking, let’s write some code and create something useful. As a useful addition to the TStringList class I’ve chosen a list reversing enumerator, an enumerator that returns elements in the reversed order, from last to first.

Let’s start with the actual enumerator.

type
  TStringsEnumReversed = class
  private
    FOwner: TStrings;
    FListIndex: integer;
  public
    constructor Create(owner: TStrings);
    function GetCurrent: string;
    function MoveNext: boolean;
    property Current: string read GetCurrent;
  end;

constructor TStringsEnumReversed.Create(owner: TStrings);
begin
  FOwner := owner;
  FListIndex := owner.Count;
end;

function TStringsEnumReversed.GetCurrent: string;
begin
  Result := FOwner[FListIndex];
end;

function TStringsEnumReversed.MoveNext: boolean;
begin
  Result := FListIndex > 0;
  if Result then
    Dec(FListIndex);
end;

There’s nothing fancy going on here. The code is almost identical to the TStringsEnumerator in Classes.pas, except that it starts at the last element and moved towards the first. You probably noticed that the reversing enumerator works on any TStrings data, not only on TBPStringList. This is intentional and will be explained later when I’ll introduce external enumerators.

Now let’s think about how this enumerator will be used.

procedure TfrmBPAdvancedEnum.btnReversedEnumClick(Sender: TObject);
var
  ln: string;
  s: string;
begin
  ln := '';
  for s in FslTest.Reversed do
    ln := ln + s;
  lbLog.Items.Add('Reversed enumerator: ' + ln);
end;

Almost identical to the standard enumerator test case, except that .Reversed is added. The mission, should we choose to accept it, is therefore to write function Reversed that returns a class which implements function GetEnumerator: TStringsEnumReversed. The question is – should it really return a class?

If we return back to the enumerator-implementation pattern (and I’m very sorry for repeating it the third time but it is very important) we’ll see something interesting.

enumerator := FslTest.Reversed.GetEnumerator;
while enumerator.MoveNext do begin
  s := enumerator.Current;
  ln := ln + s;
end;
enumerator.Free;

Let’s introduce a temporary variable to make the problem more obvious.

enumFactory := FslTest.Reversed;
enumerator := enumFactory.GetEnumerator;
while enumerator.MoveNext do
begin
  s := enumerator.Current;
  ln := ln + s;
end;
enumerator.Free;

The compiler takes care of destroying the enumerator, but who will destroy the enumFactory? One solution is to implement a enumerator factory singleton inside the TBPStringList object as I did in one of my first articles on that topic but much simpler approach is not to return a class but an interface. In that case the compiler itself will destroy the interface when it goes out of scope. Another option is to return a smart record (record that implements methods) – they are allocated from the stack and destroyed automatically. Both approaches have their downside. If we used interface-based factory then we have to write more boilerplate code (we have to define the interface). If we use record-based factory we cannot write a destructor for the factory. In this case we don’t need a destructor so we’ll use a record-based approach.

Back to the code, then. In the TBPStringList class we’ll add function Reversed which will return the factory record.

type
  TBPStringList = class(TStringList)
    function Reversed: TStringsEnumReversedFactory;
  end;

Implementation is equally simple.

function TBPStringList.Reversed: TStringsEnumReversedFactory;
begin
  Result := TStringsEnumReversedFactory.Create(Self);
end;

This will initialize new TStringsEnumReversedFactory record and pass the TBPStringList object to it. TStringsEnumReversedFactory must then store this object so it can pass it to the enumerator when GetEnumerator is called.

constructor TStringsEnumReversedFactory.Create(owner: TStrings);
begin
  FOwner := owner;
end;

function TStringsEnumReversedFactory.GetEnumerator: TStringsEnumReversed;
begin
  Result := TStringsEnumReversed.Create(FOwner);
end;

Compile, click on the button and program will log “Reversed enumerator: zyxwvutsrqponmlkjihgfedcba”.

I’ll admit again – this stuff is convoluted and hard to understand on the first reading. The best way to understand it is to load the code in the Delphi, put the breakpoint on the “for s in FslTest.Reversed do” line and run the program. When it stops on the breakpoint, use F7 to step through the code and you’ll see exactly the methods that are called and the order in which they are executed. Do it now!

Parameterized enumerators

I hope that you followed my advice and tried the Reversed enumerator in the debugger and that now you understand how enumerator factories work, because I’ll describe parameterized enumerators in much more flimsy way.

Parameterized enumerators are an extension of the concept described above. We saw that the first step in the creation of a secondary enumerator is a function that creates the enumerator factory. Again: the first step ... is a function. That means that we can pass it parameters (which it will pass to the factory which will then be passed to the enumerator) and that we can control enumerator behaviour using this parameters.

To demonstrate this approach we’ll write a slightly less useful enumerator that will return each N-th element in the list where N will be a parameter to the factory function. We can start by writing the test case.

procedure TfrmBPAdvancedEnum.btnSkipEnumClick(Sender: TObject);
var
  ln: string;
  s: string;
begin
  ln := '';
  for s in FslTest.Skip(3) do
    ln := ln + s;
  lbLog.Items.Add('Skip(3) enumerator: ' + ln);
end;

This code should return every 3^rd element from the list. Now we need the factory function. In addition to storing away owner in a private field (as in the Reversed example) it must also store the numeric parameter (with value 3 in this example).

type
  TBPStringList = class(TStringList)
    function Reversed: TStringsEnumReversedFactory;
    function Skip(every: integer): TStringsEnumSkipFactory;
  end;

function TBPStringList.Skip(every: integer): TStringsEnumSkipFactory;
begin
  Result := TStringsEnumSkipFactory.Create(Self, every);
end;

constructor TStringsEnumSkipFactory.Create(owner: TStrings; every: integer);
begin
  FOwner := owner;
  FEvery := every;
end;

function TStringsEnumSkipFactory.GetEnumerator: TStringsEnumSkip;
begin
  Result := TStringsEnumSkip.Create(FOwner, FEvery);
end;

The enumerator itself is very similar to the standard TStrings enumerator except that it advances the list index by the parameterized amount.

type
  TStringsEnumSkip = class
  private
    FOwner: TStrings;
    FListIndex: integer;
    FSkip: integer;
  public
    constructor Create(owner: TStrings; every: integer);
    function GetCurrent: string;
    function MoveNext: boolean;
    property Current: string read GetCurrent;
  end;

constructor TStringsEnumSkip.Create(owner: TStrings; every: integer);
begin
  FOwner := owner;
  FListIndex := -every;
  FEvery := every;
end;

function TStringsEnumSkip.GetCurrent: string;
begin
  Result := FOwner[FListIndex];
end;

function TStringsEnumSkip.MoveNext: boolean;
begin
  Inc(FListIndex, FEvery);
  Result := (FListIndex < FOwner.Count);
end;

The output from the test code is “Skip(3) enumerator: adgjmpsvy”.

Adding enumerators to VCL

I can hear you already – “All this is great, but what if I’d like to add a secondary enumerator to the TStringList itself? Or even worse, to the TListBox’s Items property (which is a TStrings object)?” No problem!

Remember the Reversed factory method from the first part of the article? It was a method, but we can easily change it into a global function.

function Reversed(sl: TStrings): TStringsEnumReversedFactory;
begin
  Result := TStringsEnumReversedFactory.Create(sl);
end;

Instead of passing Self to the enumerator factory it will simply pass the parameter. It can be any TStrings (or descendant) object. And yes, that’s exactly why all the enumerators in this article are working directly on TStrings and not on TBPStringList.

With this factory function we can now very simply reverse the order of lines in the listbox on the test form.

procedure TfrmBPAdvancedEnum.btnReverseLBGlobalClick(Sender: TObject);
var
  s: string;
  sl: TStringList;
begin
  sl := TStringList.Create;
  for s in Reversed(lbLog.Items) do
    sl.Add(s);
  lbLog.Items.Assign(sl);
  sl.Free;
end;

(Global functions are very useful when we want to write a generator enumerator which I’ll describe next time.)

There is another approach on enhancing existing classes with additional enumerators – class helpers. Quite obvious, if you know that class helpers are not really class methods but global methods that only appear to be implemented inside the existing objects using some heavy compiler trickstery. When you declare a class helper, you're not extending original virtual method table, you just get a new global function that takes a hidden parameter to 'Self' and uses it when calling other class helpers or methods from the original class. It's quite confusing and I don't want to dig much deeper in this direction. Suffice to say that class helpers allow you to create a very strong make-belief that new method was added to an existing class. (Some more information on class helpers can be found in Delphi help.)

We can reimplement our Reversed enumerator using class helper and inject it directly into the TStrings class (except that we are not injecting it, see the previous paragraph).

type
  TStringsEnumReversedHelper = class helper for TStrings
  public
    function Reversed: TStringsEnumReversedFactory;
  end;

function TStringsEnumReversedHelper.Reversed: TStringsEnumReversedFactory;
begin
  Result := TStringsEnumReversedFactory.Create(Self);
end;

We can now pretend that TStrings (and all descendant classes) implement the Reversed enumerator factory.

procedure TfrmBPAdvancedEnum.btnReverseLBClassHelperClick(Sender: TObject);
var
  s: string;
  sl: TStringList;
begin
  sl := TStringList.Create;
  for s in lbLog.Items.Reversed do
    sl.Add(s);
  lbLog.Items.Assign(sl);
  sl.Free;
end;

Class helpers are nice syntactic sugar but nothing more than that so in most cases you’re better off with a global function that generates the enumerator factory.

That’s all for today. See you next time when I’ll show some more examples of practical enumerator use and conclude the series. Bye!