Detecting Device Movement

For my next quest I decided to find out how accelerometer works in web applications. IOW, I wanted to control an object on my HTML page just by moving my iPad around.

I limited my quest to iOS because that’s my favorite toy. don’t know yet how to do it in Android – this StackOverflow post hinted that access from bare JavaScript may not be possible. PhoneGap somehow does that, but it’s quite possibly that they implement Java wrapper that exposes native accelerometer data to your JavaScript. But that’s just guessing.

Accelerometer in iOS

Accelerometer data is available to web application starting in iOS 4.2. Web documentation is helpful and I was able to write a Smart component that wraps the accelerometer access quite easily. Let’s see how I did it.

All functionality is wrapped in a component which you’ll be able to drop onto a form when Smart is mature enough. For now, you have to create it in the code.

  TGpW3MotionChangeEvent = procedure(Sender: TObject; info: TW3MotionData);

  TGpW3Motion = class(TW3Component)
  private
    FMotionData: TW3MotionData;
    FOnChange: TGpW3MotionChangeEvent;
  protected
    procedure CMMotion; virtual;
    procedure InitializeObject; override;
    procedure FinalizeObject; override;
  public
    property OnChange: TGpW3MotionChangeEvent read FOnChange write FOnChange;
  end;

Object initializer creates internal object holding the accelerometer data and subscribes to a devicemotion event. Unlike GUI-related events (touch, gesture), which are connected with the appropriate DOM element, devicemotion event is only triggered on the window object.

procedure TGpW3Motion.InitializeObject;
begin
  inherited;
  FMotionData := TW3MotionData.Create;
  w3_AddEvent(w3_getWindow,'devicemotion',CMMotion);
end;

In a similar manner, object finalizer unsubscribes from the devicemotion event.

procedure TGpW3Motion.FinalizeObject;
begin
  w3_removeEvent(w3_getWindow,'devicemotion',CMMotion);
  FMotionData.Free;
  inherited;
end;

The last part of the mystery, CMMotion method first calls event.preventDefault to prevent default activity associated with the event. [A event in this case, as far as I gathered, is a global JavaScript object associated with the currently active event. But again, I’m only guessing.] Then internal structure is updated (without passing any parameter to the Update call because it will access this global event object) and OnChange event is triggered.

procedure TGpW3Motion.CMMotion; 
begin
  asm
    event.preventDefault();
  end;
  FMotionData.Update;
  if assigned(FOnChange) then
    FOnChange(Self, FMotionData);
end;

Consuming Event Information

TW3MotionData.Update is a very simple forwarder which just grabs the global event object by calling a helper function w3_event.

procedure TW3MotionData.Update;
begin
  Consume(w3_event);
end;

The Consume method first grabs properties of the event (now called refObj). If w3_event was to return a Variant, I could simply use refObj.interval, refObj.rotationRate and so on, but as it is returning a TObject, I had to make a trip to JavaScript to grab the properties there.

procedure TW3MotionData.Consume(refObj: TObject);
var
  mAccel: Variant;
  mGrav: Variant;
  mRotat: Variant;
  x: integer;
begin
  (* consume ->into local variables *)
  asm
    @self.FInterval = (@refObj).interval;
    @mAccel = (@refObj).acceleration;
    @mGrav = (@refObj).accelerationIncludingGravity;
    @mRotat = (@refObj).rotationRate;
  end;
  if w3_isValidRef(mAccel) then
    FAcceleration.Consume(mAccel);
  if w3_isValidRef(mGrav) then
    FAccelInclGravity.Consume(mGrav);
  if w3_isValidRef(mRotat) then
    FRotationRate.Consume(mRotat);
  SetDisplayAcceleration;
end;

[Variants wrapping JavaScript objects were a recent addition to DWScript and the RTL is slightly lagging behind, but I presume w3_event will at some point be changed to return a Variant which would make coding such stuff much simpler.]

By declaring mAccel and other variables as Variant and not TObject I could make other Consume functions very simple.

procedure TW3Acceleration.Consume(refObj: Variant);
begin
  FX := refObj.x;
  FY := refObj.y;
  FZ := refObj.z;
end;

procedure TW3Rotation.Consume(refObj: Variant);
begin
  FAlpha := refObj.alpha;
  FBeta := refObj.beta;
  FGamma := refObj.gamma;
end;

Wait, Wait, What is Consumed Here?

Safari Developer Library explains it nicely:

• interval tells the number of seconds (documentation says milliseconds but that’s not true!) since the last motion event. On my iPad, this value is always 0.05 meaning that motion events are triggered 20 times per second.
• accelerationWithGravity contains acceleration data in all three axes (x, y and z) in m/s². This data comes from an accelerometer and is always available.
• acceleration contains similar information (also in m/s²) but coming from a gyroscope. As the original iPad doesn’t have a gyroscope, this property is always null on it. [As far as I could find out, iPad 2 contains a gyroscope and should return values in this property.]
• rotationRate contains data about rotation around all three axes. This data is also collected from the gyroscope and is not available on original iPad.

[If you want to read more, here is a nice article about gyroscopes.]

Display is Relative, Acceleration is Absolute

If you look closely at the Consume method above, you’ll notice a SetDisplayAcceleration call. To explain it, I have to step back and give some more consideration to the devicemotion properties.

When you work with Smart, you’ll notice that your application adapts to the display orientation changes. In other words, if you change device orientation from portrait to landscape, Width and Height of your application will change (and if you override a Resize method you can adapt to the new environment, for example by changing a component layout). Accelerometer and gyroscope data, however, are not affected by that – the data you get is the data that the measurement device returned and the measurement device doesn’t know that top has suddenly become right for your application. Because of that you have to know the current orientation (and boy had I a hard time finding this data) to correctly interpret the accelerometer information.

Let me explain this through an example.

If you hold your device in a normal portrait position (i.e. with the one and only button at the bottom) in a normal reading position (like this; image source: crunchbase), accelerometer will return a zero (or a value close to it) in X direction (X being left-to-right and device is not tilted in this direction) and a negative value in Y direction. But if you flip the device on its head (rotate it by 180 degrees) so that the button is on the top, it will return a positive value in Y direction (and X acceleration will still be nearly zero).

If you then rotate the device into a landscape position so that the button is on the right (like this; image source: crunchbase) you’ll get a negative value in X direction and (almost) zero in Y direction. And if you change the orientation to other landscape position (button on the left) you’ll get a positive value in X direction and (almost) zero in Y direction.

Because of that, you’ll code handling the accelerometer data becomes quite complicated:

  ax := info.AccelInclGravity.X * 5;
  ay := info.AccelInclGravity.y * 5;

  asm
    @orient = window.orientation;
  end;
  case orient of
    000: {portrait}        begin FvX := FvX + ax; FvY := FvY - ay; end;
    090: {landscape left}  begin FvX := FvX - ay; FvY := FvY - ax; end;
    -90: {landscape right} begin FvX := FvX + ay; FvY := FvY + ax; end;
    180: {flipped}         begin FvX := FvX - ax; FvY := FvY + ay; end;
  end;

I should rather say should become quite complicated as I moved this complication into the TW3Motion component so don’t have to think about it.

TW3MotionInfo class exposes display-relative acceleration, that is an acceleration relative to the current display orientation, in the DisplayAccel property. It exposes two fields – Horizontal (being the acceleration in left-right direction) and Vertical (acceleration in top-down direction). The example from few lines back can be therefore simplified to:

  FvX := FvX + info.DisplayAccel.Horizontal * 5;
  FvY := FvY + info.DisplayAccel.Vertical * 5;

The magic behind SetDisplayAcceleration is very simple – once you know how to do it. It took me quite some time experimenting to get everything just right.

procedure TW3MotionData.SetDisplayAcceleration;
var
  mOrient: integer;
begin
  asm
    @mOrient = window.orientation;
  end;
  case mOrient of
    000: //portrait
      begin
        FDisplayAccel.Horizontal := FAccelInclGravity.X;
        FDisplayAccel.Vertical := - FAccelInclGravity.Y;
      end;
    090: //landscape left
      begin
        FDisplayAccel.Horizontal := - FAccelInclGravity.Y;
        FDisplayAccel.Vertical := - FAccelInclGravity.X;
      end;
    -90: //landscape right
      begin
        FDisplayAccel.Horizontal := FAccelInclGravity.Y;
        FDisplayAccel.Vertical := FAccelInclGravity.X;
      end;
    180: //flipped
      begin
        FDisplayAccel.Horizontal := - FAccelInclGravity.X;
        FDisplayAccel.Vertical := FAccelInclGravity.Y;
      end;
  end;
end;

How About a Demo?

Next time. This post has become quite too long already. In the meantime, you can play with my demo provided you have an iPhone or iPad with iOS 4.2 or newer.