Thursday, March 08, 2007

Fun with enumerators, part 4 - external enumerators

[Part 1 - Introduction, Part 2 - Additional enumerators, Part 3 - Parameterized enumerators.]

Welcome back, dear reader. If you were following this short series since Day 1, you now know how to create additional enumerators for a class. Today we'll do something even more interesting – we'll add an enumerator to a class that we cannot modify or derive from.

For example, in comments to Part 1 Renaud created enumerator for TDataSet by descending from it. His solution, however, is not practical when TDataSet is created somewhere deep in the class hierarchy.

Another example - one that we'll use today - is additional enumerator for TStrings. This class is used in various stock VCL components (ListBox.Items, Memo.Lines ...). It already provides an enumerator (iterating over all strings in the container), but just for the fun of it we'll add a reverse enumerator – one that will start with the last string and proceed toward beginning of the container.

Let's take another look at the pseudocode describing compiler-generated implementation of a generic for..in loop.

enumerator := list.GetEnumerator;
while enumerator.MoveNext do
//do something with enumerator.Current;
enumerator.Free;


In Part two we used enumerator := list.SomeProperty.GetEnumerator to access secondary enumerator and in Part three we used enumerator := list.SomeFunction(param).GetEnumerator to access parameterized enumerator. Delphi compiler is not picky when parsing parameters for the for..in loop. We can provide it with anything that implements GetEnumerator function. And nobody says that this 'anything' must come from a same class hierarchy as the enumerated targed. We can simply write a global function that will return some object with public GetEnumerator.



Let's repeat this: "There is no enforced connection between the factory that provides GetEnumerator and the structure we are enumerating." In fact, there may not be any structure at all. Delphi provider will be perfectly satisfied with this code generating Fibonacci numbers as long as we write correct factory and enumerator:

for i in Fibonacci(10) do
Writeln(i);


[I promise to show you the code to Fibonacci enumerator tomorrow. Or maybe the day after.]



By now it should be obvious what I'm leading at. We can write a global function taking TStrings parameter and returning an enumerator factory for this TStrings. We would then use it like this:

for s in ExternalEnumerator(stringList) do
// do something with s


ExternalEnumerator takes object we want to be enumerated, creates factory object for it and returns this factory object. Compiler then calls GetEnumerator on that factory object to get enumerator object. Then it uses enumerator object to enumerate stringList. At the end, compiler destroys enumerator object. But still the factory object exists and is not destroyed. How can we destroy it when it is no longer needed?



Simple, we will use helpful Delphi compiler. Instead of returning object reference from the ExternalEnumerator, we'll return an interface. Compiler will automatically manage its lifetime and will destroy it when it's no longer needed.



It looks like we have all parts ready now:





  • We will write interface defining GetEnumerator function and enumerator factory implementing this interface.

  • External enumerator function will create new enumerator factory object and return it.

  • Compiler will call GetEnumerator on the factory object, complete the for..in loop and destroy the enumerator object.

  • Some time later (maybe at the end of the for..in loop, maybe at the end of the method containing for..in loop - this is implementation specific and may differ between different versions of the compiler) enumerator factory interface will go out of scope and compiler will destroy the factory object.

  • And that's all folks.


First we need an interface defining GetEnumerator, enumerator factory class implementing this interface and enumerator function.

  IStringsEnumReversedFactory = interface
function GetEnumerator: TStringsEnumReversed;
end;

TStringsEnumReversedFactory = class(TInterfacedObject, IStringsEnumReversedFactory)
private
FOwner: TStrings;
public
constructor Create(owner: TStrings);
function GetEnumerator: TStringsEnumReversed;
end;

function EnumReversed(strings: TStrings): IStringsEnumReversedFactory;
begin
Result := TStringsEnumReversedFactory.Create(strings);
end;


Then we need an enumerator, but that's trivial especially as we've written many of them already.

  TStringsEnumReversed = class
private
FOwner: TStrings;
FListIndex: integer;
public
constructor Create(owner: TStrings);
function GetCurrent: string;
function MoveNext: boolean;
property Current: string read GetCurrent;
end;

constructor TStringsEnumReversed.Create(owner: TStrings);
begin
FOwner := owner;
FListIndex := owner.Count;
end;

function TStringsEnumReversed.GetCurrent: string;
begin
Result := FOwner[FListIndex];
end;

function TStringsEnumReversed.MoveNext: boolean;
begin
Result := FListIndex > 0;
if Result then
Dec(FListIndex);
end;


We can now write a simple tester:

procedure TfrmFunWithEnumerators.btnReverseClick(Sender: TObject);
var
ln: string;
s : string;
begin
ln := '';
for s in EnumReversed(FslTest) do
ln := ln + s;
lbLog.Items.Add('External enumerator: ' + ln);
end;


And here's a proof that EnumReversed really works.





It is maybe not obvious what really happens here, so let's take another look at this for..in loop - this time written as Delphi compiler implements it.

var
ln: string;
s: string;
enum: TStringsEnumReversed;
tmpIntf: IStringsEnumReversedFactory;
begin
ln := '';
tmpIntf := EnumReverse(FslTest);
enum := tmpIntf.GetEnumerator;
while enum.MoveNext do
ln := ln + enum.Current;
enum.Free;
lbLog.Items.Add('External enumerator: ' + ln);
tmpIntf := nil;
end;


As we've mentioned at the very beginning, this approach allows us to use enumerators on base classes (TStrings in our example), so here's a simple code that reverses items in the TListBox I'm using to display test results:

procedure TfrmFunWithEnumerators.btnReverseLogClick(Sender: TObject);
var
s : string;
sl: TStringList;
begin
sl := TStringList.Create;
for s in EnumReversed(lbLog.Items) do
sl.Add(s);
lbLog.Items.Assign(sl);
sl.Free;
end;


Result:



 



That's all for today. Tomorrow I'll show you another trick that will allow you to write for..in loop from the last example as for s in lbLog.Items.EnumReversed do.





Technorati tags: , ,

7 comments:

  1. Anonymous20:55

    For bonus points, you could make your external enumerator into a class helper.

    Then you could do something like

    TfrmFunWithEnumerators.btnReverseClick(Sender: TObject);
    var
    ln: string;
    s : string;
    begin
    ln := '';
    for s in FslTest.EnumReversed do
    ln := ln + s;
    lbLog.Items.Add('External enumerator: ' + ln);
    end;

    Sean

    ReplyDelete
  2. That's the Part 5 topic :)

    ReplyDelete
  3. Hi Grab,
    I tried it doing a function EnumDataSet( ADataSet: TDataSet): IDataSetEnumFactory;
    It works also (compared to TDataSetForIn in comment Part 1).
    Basically it allow to use a fonction instead of a class transtyping, as far as I understood.
    Using it in code is nearly the same:
    var DS: TDataSet;
    begin
    for DS in EnumDataSet( MyQuery) do...
    instead of:
    var DS: TDataSet;
    begin
    for DS in TDataSetForIn( MyQuery) do...
    But somehow the TDataSetForIn solution seems lighter and works also for any TDataSet descendant.

    May be I'm missing something, but what is the practical advantage of Part 4 solution over Part 1, in the case I juste want to replace the "while not EOF do..." loop on a TDataSet ?

    Also, MyQuery is already globally declared. Is there any way to avoid the LOCAL declaration of "var DS: TDataSet" (needed in both solutions) ?

    Renaud.

    ReplyDelete
  4. Your way is valid because you're not depending on TDataSetForIn being different from TDataSet inside GetEnumerator. Otherwise, your casting form TDataSet to TDataSetForIn would not work.

    Therefore, you can freely use the approach you like most.

    As for the second part of your question - you have to declare 'for' variables, there is no workaround for this.

    ReplyDelete
  5. Thanks Gabr for the answer.
    Anyhow, this "new" TDataSetForIn is going to save me hundreds lines of code.
    For sure, enumerator is fun !

    ReplyDelete
  6. Thanks for this article series! I use for-in loops quite often, and I do use very often IEnumerable<T> objects as a gateway from a data structure to the enumerator IEnumerator, but only recently I discovered that if such a gateway object is created in place it will only be freed at the end of the method or routine, whilst the enumerator object (IEnumerator) is freed right after the for-in loop.

    I'd rather expect that this:


    enum := EnumReverse(FslTest).GetEnumerator
    while enum.MoveNext do
    ...


    compiles as something like this:


    tmpIntf := EnumReverse(FslTest);
    enum := tmpIntf.GetEnumerator;
    tmpIntf := nil;
    while enum.MoveNext do
    ...


    I'd expect the reference count of tmpIntf to drop to 0 immediately after the enumerator is retrieved and thus being freed right away. Instead it seems that the compiler introduces additional local variables to hold these references up to the end of the method or routine.

    It doesn't even seem to get optimized in some way: I have a method with several sequential for-in loops, and all are constructed the same way like your Fibonacci example, and the reference count of all intermediate objects (IEnumerable) drops to 0 only at the end of the method!? No optimization here? Is there any explanation for this?

    ReplyDelete
    Replies
    1. This is how Delphi compiler works.

      Delete