App Tethering with Delphi – A Remote Track-Pad App

Problem:

At home I have a PC based media centre with which I use a Logitech wireless keyboard with integrated track pad. However, often the case arises where I am lying on the couch and only need mouse functionality but the keyboard is on the other side of the room out of reach.

Solution:

I always have my phone within arms reach so figured a simple track-pad app would be ideal. More often than not I don’t require the full functionality of the keyboard, but just need the mouse functionality so this would be an ideal solution.

My approach was to try out App Tethering and attempt to make a client app for my iPhone, and a server app for the media centre pc. I used the FireMonkey Framework to create the client app and VCL for the server app.

With no App Tethering experience and very little FireMonkey experience this was a good learning opportunity, but as is often the case when learning new things it took longer than I originally had hoped. ūüôā

Result:

I now have a very basic app that provides basic mouse functionality on the media centre pc. There is plenty of room for improvement, but at this stage it is more of a proof of concept and a quick way to learn the basics while testing App Tethering and FireMonkey.

Some ideas for improvement include the following…

  • Create the server application as a service.
  • Provide additional¬†status information on the client.
  • Spend some time on¬†GUI¬†design on the client.
  • Add support for additional mouse events, i.e. the scroll wheel and additional buttons.

Details regarding the server and client applications are provided below for those interested in the code side of things.  Do take note that at the point of writing this I still have minimal experience with both FireMonkey and App Tethering so while I may have a functioning application there may be better ways of achieving the same result.

 


 

1 – The Server:

Remote Track Pad - Server

While App Tethering doesn’t necessarily describe the application relationship as client/server, for this project I do use the Client/Server analogy.

The Server is a very simple VCL application that will perform the mouse actions, it is made up of a single form with a TTetheringManager, TTetheringAppProfile and TLabel component.

Track Pad App Server - Main Form Design Time

   TfrmMain = class(TForm)
      labStatus: TLabel;
      TManager: TTetheringManager;
      TAppProfile: TTetheringAppProfile;
      procedure TAppProfileResourceReceived(const Sender: TObject; const AResource: TRemoteResource);
    private
      { Private declarations }
    public
      { Public declarations }

      function PointDeSerialiser(inPointStr : String) : TSJPoint; // Converts Serialised TSJPoint Object String back into a TSJPoint Object
    end;

 

Setting up the App Tethering server simply involves dropping the TTetheringManager and TTetheringAppProfile components on the form and then setting the Manager property of the TTetheringAppProfile component to point to the TTetheringManager component.

 

TrackPadApp_Server_dt_TTetheringAppProfile

 

1.1 – TAppProfileResourceReceived:

TAppProfileResourceReceived is the TAppProfile (TTetheringAppProfile) OnResourceReceived event and is where the mouse event procedures are handled.

Currently only three mouse events are handled, mouse move, left click and right click.

 

procedure TfrmMain.TAppProfileResourceReceived(const Sender: TObject; const AResource: TRemoteResource);
Var
   CursorOffSet : TSJPoint;
begin
     labStatus.Caption := 'Data Received -' + AResource.Hint;

     // Mouse Move
     if AResource.Hint = arDescMM Then Begin
        CursorOffSet := PointDeSerialiser(AResource.Value.AsString);
        labStatus.Caption := 'Data Received - Mouse Move - X=' + IntToStr(CursorOffSet.X) + ' Y=' + IntToStr(CursorOffSet.Y);
        mouse_event(MOUSEEVENTF_MOVE,CursorOffSet.X,CursorOffSet.Y,0,0);
     End;

     // Mouse Click Left
     if AResource.Hint = arDescMCL Then Begin
        labStatus.Caption := 'Data Received - Mouse Click Left';
        mouse_event(MOUSEEVENTF_LEFTDOWN,0,0,0,0);
        mouse_event(MOUSEEVENTF_LEFTUP,0,0,0,0);
     End;

     // Mouse Click Right
     if AResource.Hint = arDescMCR Then Begin
        labStatus.Caption := 'Data Received - Mouse Click Right';
        mouse_event(MOUSEEVENTF_RIGHTDOWN,0,0,0,0);
        mouse_event(MOUSEEVENTF_RIGHTUP,0,0,0,0);
     End;
end;

 

The mouse event that needs to occur is based on the string provided in the AResource.Hint property. Constants are used to standardise code and these are shared between the server application and the client application as follows…

 

Const
     // Resource Description Constants
     arDescMM  = 'Mouse Movement';
     arDescMCL = 'Mouse Click Left';
     arDescMCR = 'Mouse Click Right';

 

1.2 PointDeSerialiser:

PointDeSerialiser is used to convert a serialised TSJPoint Object (created by the Client Application and passed to the Server by App Tethering) back into a TSJPoint Object.

 

function TfrmMain.PointDeSerialiser(inPointStr: String): TSJPoint;
// 08/Dec/2014 - ADowling
// ---
// Quick Function to Serialize a TSJPoint, based on the code posted in the blog post
// http://www.danieleteti.it/2009/09/01/custom-marshallingunmarshalling-in-delphi-2010/
// ---
Var
   UnMar           : TJSONUnMarshal;  // DeSerialiser
   OutPoint        : TSJPoint;
begin
     UnMar := TJSONUnMarshal.Create;
     Try
        // Register a Reverter for the TSJPoint Object
        UnMar.RegisterReverter(TSJPoint,
                               function(Data : TListOfStrings) : TObject
                               var
                                  Point : TSJPoint;
                               begin
                                    Point := TSJPoint.Create;
                                    Point.X := StrToInt(Data[0]);
                                    Point.Y := StrToInt(Data[1]);
                                    Result := Point;
                               end
        );

        OutPoint := UnMar.Unmarshal(TJSONObject.ParseJSONValue(inPointStr)) as TSJPoint;
        Result := OutPoint;
     Finally
         UnMar.Free;
     End;
end;

 

TSJPoint is basically an Object definition to hold what the record TPoint would normally hold. We have defined a TSJPoint object so that it can be serialised as part of JSON Marshalling/UnMarshalling.

 

Type
   TSJPoint = class
     X : Integer;
     Y : Integer;
    end;

 


 

2 – The Client:

The Client is a simple FireMonkey application that will pass mouse actions to the server. A TPanel, and two TButtons are used to simulate a track-pad.  Similarly to the Server application, the client application requires a TTetheringManager and TTetheringAppProfile component. A TLabel component is also used to provide status information.

TrackPadApp_Client_dt

  TfrmMain = class(TForm)
    pnlTrackPad: TPanel;
    butMouseClickLeft: TButton;
    butMouseClickRight: TButton;
    labStaticDebugInfoStatus: TLabel;
    labStatus: TLabel;
    TManager: TTetheringManager;
    TAppProfile: TTetheringAppProfile;
    procedure FormCreate(Sender: TObject);
    procedure FormShow(Sender: TObject);
    procedure TManagerEndManagersDiscovery(const Sender: TObject; const ARemoteManagers: TTetheringManagerInfoList);
    procedure TManagerEndProfilesDiscovery(const Sender: TObject; const ARemoteProfiles: TTetheringProfileInfoList);
    procedure pnlTrackPadMouseDown(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single);
    procedure pnlTrackPadMouseUp(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single);
    procedure pnlTrackPadMouseMove(Sender: TObject; Shift: TShiftState; X, Y: Single);
    procedure butMouseClickLeftClick(Sender: TObject);
    procedure butMouseClickRightClick(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
    CursorIsMoving  : Boolean;                               // Is the Cursor moving
    CursorPrevPos   : TSJPoint;                              // Previous Cursor Position

    function PointSerialiser(inPoint : TSJPoint) : String;   // Serialises a TSJPoint Object into a String
  end;

 

Just like the Server application, the Manager Property of TTetheringAppProfile should point to the TTetheringManager component.

 

2.1 Application Startup:

When the application starts we need to unpair any managers already paired with the TTetheringManager (TManager) and then call the DiscoverManagers procedure. The DiscoverManagers procedure and related events that are called is where App Tethering does much of its magic and sets up the server connection.

 

procedure TfrmMain.FormShow(Sender: TObject);
var
   i : Integer;
begin
     CursorIsMoving := False;

     // Unpair any Managers already paired with the TTetheringManager
     for i := TManager.PairedManagers.Count -1 downto 0 do TManager.UnPairManager(TManager.PairedManagers[i]);
     TManager.DiscoverManagers;
end;

 

2.2 TTetheringManager Discovery Events:

The OnEndManagersDiscovery and OnEndProfilesDIscovery events of the TTetheringManager (TManager) component pair the Client application with the Server application and update the status label.

 

procedure TfrmMain.TManagerEndManagersDiscovery(const Sender: TObject; const ARemoteManagers: TTetheringManagerInfoList);
var
   I : Integer;
begin
     labStatus.Text := 'No receiver found';
     for I := 0 to ARemoteManagers.Count-1 do
         if (ARemoteManagers[I].ManagerText = 'RTPReceiverManager')  then begin
            TManager.PairManager(ARemoteManagers[I]);
            labStatus.Text := 'Receiver Found....';
            Break; // Break since we only want the first...
         end;
end;

procedure TfrmMain.TManagerEndProfilesDiscovery(const Sender: TObject; const ARemoteProfiles: TTetheringProfileInfoList);
var
   i : Integer;
begin
     labStatus.Text := 'No receiver found';
     for i := 0 to TManager.RemoteProfiles.Count-1 do
         if (TManager.RemoteProfiles[i].ProfileText = 'RTPReceiver') then begin
            if TAppProfile.Connect(TManager.RemoteProfiles[i]) then labStatus.Text := 'Receiver ready.'
         end;
end;

 

2.3 PointSerialiser:

PointSerialise is used to convert a TSJPoint Object into a JSON Object String that can be sent to the Tethering Server.

 

function TfrmMain.PointSerialiser(inPoint: TSJPoint): String;
// 08/Dec/2014 - ADowling
// ---
// Quick Function to Serialize a TSJPoint, based on the code posted in the blog post
// http://www.danieleteti.it/2009/09/01/custom-marshallingunmarshalling-in-delphi-2010/
// ---
Var
   Mar             : TJSONMarshal;    // Serialiser
   SerialisedPoint : TJSONObject;     // Serialised for of Object;
begin
     Mar := TJSONMarshal.Create(TJSONConverter.Create);

     // Register a Converter for the TSJPoint Object
     Mar.RegisterConverter(TSJPoint,
                           function(Data: TObject) : TListOfStrings
                           Var
                              Point : TSJPoint;
                           Begin
                                Point := TSJPoint(Data);
                                SetLength(Result, 2);
                                Result[0] := IntToStr(Point.X);
                                Result[1] := IntToStr(Point.Y);
                           End
     );

     try
        SerialisedPoint := Mar.Marshal(inPoint) As TJSONObject;
        Result := SerialisedPoint.ToString;
     finally
        Mar.Free;
     end;
end;

 

2.4 Mouse Events:

Mouse cursor movement is simulated by the TPanel (pnlTrackPad) MouseDown, MouseUp, and MouseMove Events.

 

2.4.1 MouseDown – MouseUp:

The MouseDown and MouseUp events are used to set the CursorIsMoving boolean value, which is evaluated by the MouseMove event and determines if a cursor co-ordinate offset needs to be sent to the server.

 

procedure TfrmMain.pnlTrackPadMouseDown(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single);
begin
     // ---
     // We only want to set CursorIsMoving to True if the Left Mouse Button has been clicked.
     //
     // At this stage we set the CursorPrevPos X/Y Values to the current X/Y values of the
     // cursor.
     // ---
     if (Button = TMouseButton.mbLeft)  then Begin
        CursorIsMoving := True;
        CursorPrevPos.X := Round(X);
        CursorPrevPos.Y := Round(Y);
     end;
end;

procedure TfrmMain.pnlTrackPadMouseUp(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single);
begin
     // ---
     // When the Left Mouse Button is released we set CursorIsMoving to False.
     // ---
     CursorIsMoving := False;
end;

 

2.4.2 MouseMove:

The MouseMove event checks the current cursor position and compares it to the previous cursor position (CursorPrevPos Field) and passes an offset value to the server. The cursor offset is passed in a TSJPoint Object which is first marshalled by the PointSerialiser function.

 

procedure TfrmMain.pnlTrackPadMouseMove(Sender: TObject; Shift: TShiftState; X, Y: Single);
Var
   CursorOffSet    : TSJPoint;
   CursorOffSetStr : String;
begin
     // Only perform processing if CursorIsMoving
     if CursorIsMoving then Begin

        CursorOffSet := TSJPoint.Create;
        Try

           // ---
           // Get the Cursor Movement OffSet.  Obtained by comparing previous cursor position with
           // X/Y parameters
           //
           // This Offset should only be +/- 1 in either direction
           // ---
           CursorOffSet.X := Round(X) - CursorPrevPos.X;
           CursorOffSet.Y := Round(Y) - CursorPrevPos.Y;

           // Convert the CursorOffSet Object to a String that can be passed to Tethering Server.
           CursorOffSetStr := PointSerialiser(CursorOffSet);

           // Set CursorPrevPos X/Y Values to the current X/Y parameter values.  This must be done to ensure
           // the CursorOffSet value does not end up being updated exponentially.
           CursorPrevPos.X := Round(X);
           CursorPrevPos.Y := Round(Y);

           // Send Serialised CursorOffSet Object to Tethering Server.
           TAppProfile.SendString(TManager.RemoteProfiles[0],         // Remote Profile to Receive the String
                                           arDescMM,                  // Description of the contents of the string
                                           CursorOffSetStr);          // String to be sent

        Finally
            FreeAndNil(CursorOffSet);
        End;

     end;
end;

 

2.4.3 Left and Right Click Buttons:

The Left and Right Click buttons send a simple command to the server to enact the matching mouse events on the server.

 

// ---
// Send a Left Click Command to Remote App
// ---
procedure TfrmMain.butMouseClickLeftClick(Sender: TObject);
begin
     // Send Left Mouse Click Command to Tethering Server.
     TAppProfile.SendString(TManager.RemoteProfiles[0],         // Remote Profile to Receive the String
                                     arDescMCL,                 // Description of the contents of the string
                                     arDescMCL);                // String to be sent
end;

// ---
// Send a Right Click Command to Remote App
// ---
procedure TfrmMain.butMouseClickRightClick(Sender: TObject);
begin
     // Send Left Mouse Click Command to Tethering Server.
     TAppProfile.SendString(TManager.RemoteProfiles[0],         // Remote Profile to Receive the String
                                     arDescMCR,                 // Description of the contents of the string
                                     arDescMCR);                // String to be sent
end;

 

About Anthony Dowling

Founder Sjones Limited, focusing on developing ID and Security Card Solutions. Long time Delphi Developer. @AntDowling
This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s