Tuesday, January 17, 2012

Improved Painting

When I wrote about my first Smart Mobile Studio program I mentioned that it failed on iPad (and would probably fail on iPhone too but I couldn’t test that) because the Paint code took too long and the iOS simply aborted it. As far as I know there are two ways to fix this in JavaScript – you can either move the calculation to a background thread using a Web worker mechanism or split the Paint into multiple parts where each part only calculates and paints small part of the screen and then returns control to the browser. For the time being I went with the latter solution but that will not prevent me from testing the Web worker approach at some time.

Staggered Paint

The new Paint is significantly more complicated than the old one.

procedure TW3MandelbrotCanvas.Paint;
var
firstY : integer;
imageLine : TW3ImageData;
iterations: TIterationArray;
start : float;
y : integer;
begin
if not Visible then
Exit;
if not FInitialized then
MakeInitialWorld;
imageLine := Canvas.CreateImageData(FCoord.View.Width + 1, 1);
start := Now;
firstY := FResumePaint;
if firstY < 0 then begin
ClearView;
firstY := FCoord.View.Top;
end;
FResumePaint := -1;
for y := firstY to FCoord.View.Bottom do begin
iterations := FEngine.GetLine(FCoord.World.Left, FCoord.World.Right,
FCoord.MapToWorldY(y), FCoord.View.Width + 1);
MapIterationsToImageLine(iterations, imageLine);
Canvas.PutImageData(imageLine, FCoord.View.Left, UpDown(y));
if HasElapsedMs(start, CPaintTimeout_ms) and (y < FCoord.View.Bottom) then
begin

FResumePaint := y + 1;
if not assigned(FTimer) then
FTimer := TW3EventRepeater.Create(PaintTimer, 1);
Exit;
end;
end;
FTimer.Free;
FTimer := nil;
end;

After few initial checks and optional initialization, the code allocates one high off-screen image buffer (Canvas.CreateImageData). If this is a fresh paint (and not a continuation of a previous work), view is cleared (ClearView) and the paint loop starts from the top of the view (FCoord.View.Top). Otherwise, paint loop starts from the first unpainted line (FResumePaint).

The code then loops from the starting line to the end of the view (for loop). In each iteration it gets one line of data from the Mandelbrot engine (FEngine.GetLine), converts the data to pixels (MapIterationsToImageLine) and writes the data to the screen (Canvas.PutImageData). Then it checks the time. If more than CPaintTimeout_ms (one second in the current code) has elapsed, the code stores next line to be painted in the FResumePaint field and sets the timer which will call the Paint method as soon as possible.

function TW3MandelbrotCanvas.PaintTimer(sender: TObject): boolean;
begin
Paint;
end;

Then the story continues until the last line is painted and timer is destroyed. After that, new paint will be requested by the main form when the user action calls for it.

Painting Embellishments

Last time I mentioned painting lines and rectangles on the canvas to indicate zoom and pan operations.

image

Again, there are two ways to achieve nice smooth effect that is pleasing for the eye. As the user moves the line around, the rectangle/line must also move. The problem is not painting the new rectangle/line but restoring the Mandelbrot set image where the rectangle/line was painted previously.

One option is to keep Mandelbrot set image in the big off-screen bitmap and then always paint Mandelbrot set first (directly from this off-screen bitmap) and paint rectangle or line over it.

Another possibility is to store the bitmap under the rectangle/line into an off-screen buffer before it is painted and then restore it when the image must be restored. This is the way my program handles the job.

I’ll just show few examples of the code here. GrabArrow stores everything under the line (and that may be a large area as we can’t really store diagonal part of the image (or rather, it could be done but would be quite complicated) but must store a rectangle that contains the line.

procedure TEmbelishCanvas.GrabArrow(pFrom, pTo: TPoint);
var
rect: TRect;
begin
FArrow.Active := true;
FArrow.PFrom := pFrom;
FArrow.PTo := pTo;
rect := FixOrientation(w3_Rect(pFrom.X, pFrom.Y, pTo.X, pTo.Y));
FArrow.Image := FCanvas_ref.GetImageData(rect.Left - 1, rect.Top - 1,
rect.Right - rect.Left + 3, rect.Bottom - rect.Top + 3);
end;

[FixOrientation is a helper function that “orients” the rectangle (swaps corners if necessary) so that Left is not larger then Right and Top is not larger than Bottom.]

GrabMarquee stores the area under the rectangle and does it in an optimized manner – it only grabs four small rectangles lying directly below the rectangle sides.

procedure TEmbelishCanvas.GrabMarquee(rect: TRect);
var
width, height: integer;
begin
width := rect.Right - rect.Left;
height := rect.Bottom - rect.Top;
FMarquee.Active := true;
FMarquee.Rect := rect;
FMarquee.Top := FCanvas_ref.GetImageData(
rect.Left - 1, rect.Top - 1, width + 2, 2);
FMarquee.Bottom := FCanvas_ref.GetImageData(
rect.Left - 1, rect.Bottom - 1, width + 2, 2);
FMarquee.Left := FCanvas_ref.GetImageData(
rect.Left - 1, rect.Top - 1, 2, height + 2);
FMarquee.Right := FCanvas_ref.GetImageData(
rect.Right - 1, rect.Top - 1, 2, height);
end;

Hide does the reverse operation – restores the original bitmap.

procedure TEmbelishCanvas.Hide;
var
rect: TRect;
begin
if FArrow.Active then begin
rect := FixOrientation(w3_Rect(FArrow.PFrom.X, FArrow.PFrom.Y,
FArrow.PTo.X, FArrow.PTo.Y));
FCanvas_ref.PutImageData(FArrow.Image, rect.Left - 1, rect.Top - 1);
FArrow.Active := false;
end;
if FMarquee.Active then begin
FCanvas_ref.PutImageData(
FMarquee.Top, FMarquee.Rect.Left - 1, FMarquee.Rect.Top - 1);
FCanvas_ref.PutImageData(
FMarquee.Bottom, FMarquee.Rect.Left - 1, FMarquee.Rect.Bottom - 1);
FCanvas_ref.PutImageData(
FMarquee.Left, FMarquee.Rect.Left - 1, FMarquee.Rect.Top - 1);
FCanvas_ref.PutImageData(
FMarquee.Right, FMarquee.Rect.Right - 1, FMarquee.Rect.Top - 1);
FMarquee.Active := false;
end;
end;

The remaining code contains two paint methods (one draws a line, another a rectangle) and two public members ShowArrow and ShowMarquee which are called from the mouse handling events.

procedure TEmbelishCanvas.PaintArrow(pFrom, pTo: TPoint);
begin
FCanvas_ref.StrokeStyle := '#FFFFFF';
FCanvas_ref.LineWidth := 1;
FCanvas_ref.BeginPath;
FCanvas_ref.LineF(pFrom.X, pFrom.Y, pTo.X, pTo.Y);
FCanvas_ref.Stroke;
FCanvas_ref.ClosePath;
end;

procedure TEmbelishCanvas.PaintMarquee(rect: TRect);
begin
FCanvas_ref.StrokeStyle := '#FFFFFF';
FCanvas_ref.LineWidth := 1;
FCanvas_ref.StrokeRectF(rect.Left - 0.5, rect.Top - 0.5, rect.Right - rect.Left + 1, rect.Bottom - rect.Top + 1);
end;

procedure TEmbelishCanvas.ShowArrow(pFrom, pTo: TPoint);
begin
Hide;
GrabArrow(pFrom, pTo);
PaintArrow(pFrom, pTo);
end;

procedure TEmbelishCanvas.ShowMarquee(rect: TRect);
begin
Hide;
if (rect.Right = rect.Left) or (rect.Top = rect.Bottom) then
Exit;
GrabMarquee(rect);
PaintMarquee(rect);
end;

Image Cache

If you check the program in the desktop browser (Chrome and Safari work best) you’ll see that it also displays the current position and zoom factor in the top left corner (just two TW3Label components) and when you zoom into the image you’ll see the blue arrow buttons lighting up.

image

Those two buttons enable you to quickly browse through the last 10 images. Each image is stored in a off-screen buffer when it is calculated and clicking the arrow button just puts an image from this cache on the screen so this is blindingly fast. The mechanics of the implementation are exactly the same as for the embellishment drawing – GetImageData is used to grab pixels and PutImageData puts them back.

You’ll also see two disabled buttons (“L” and “T”) which are just a placeholders for future code and a Help button (“?”) which will display a short help.

I’ll return to the Help button implementation (it actually displays a new form) and to speed buttons later. Next time you can expect something way more interesting – I’ll teach the program to recognize touch.

5 comments:

  1. Anonymous10:19

    I don't use Chrome or Safari, but it works ok in Opera.

    ReplyDelete
  2. Not really (at least in my Opera) - coordinates (upper left corner) are truncated and buttons are ugly. Basic functionality works, though.

    ReplyDelete
  3. Nice! Will we get a better shading scheme for the next version? ;-)

    The ugly buttons in Opera/FireFox are because the CSS only specifies the -webkit attributes, so it's more a limitation of the current default mobile target (iOS & Android browsers are both webkit based).

    There is a small repaint "bug": if you go to the help form, and when you hit "back", the fractal is redrawn, rather than using the last cached image.

    ReplyDelete
  4. Yes, better shading schemes are planned, but first I have to include touch support.

    Thanks for the bug report!

    ReplyDelete
  5. Just want to mention that Smart Mobile Studio was designed for iPhone only, so im actually surpriced it works on these other browser at all. Support for more browsers will come as soon as v1.0 is ready - but in general you can also use chrome to run the code (the "emulator" in the IDE is chrome)

    ReplyDelete