News
Photos
Articles
Components
Applications
Kleinkunst

.NET - UI automation with the White framework

Last weekend I was looking for a framework or library for .NET to automate Windows applications. On the CodePlex website I found the very interesting White project : http://www.codeplex.com/white

White is open-source, written in C# and it supports all rich client applications, which are Win32, WinForm, WPF and SWT (java). It provides a consistent object oriented API and it hides all the complexity of Microsoft's UI Automation library and Win32 Windows messages.

 

UISpy

Before using White to automate an application, you need a tool to identify all the necessary UI elements in a Windows application. Microsoft provides a quite useful tool called UISpy. It is a standalone executable which enables developers to view all UI elements and their details.

This tool is not available as a single download. You need to download the Windows SDK for Vista Update : http://www.microsoft.com/downloads/details.aspx?familyid=4377F86D-C913-4B5C-B87E-EF72E5B4E065&displaylang=en. After installation the EXE can be found in the C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin folder.

After starting UISpy a UI Automation tree will be displayed. The root element represents the current desktop and the child elements represent application windows. Each of these child elements contain UI elements such as menus, buttons, radiobuttons, textboxes, toolbars, listboxes, ...

UISpy also has a nice hover mode which you can use to select a UI item in the Windows applications by holding the CTRL key.

Detailed information about the UI element is displayed in the Properties panel. When you want to access a UI element via White, then you need to know the AutomationId, ControlType and Name.

More information about the UISpy tool can be found on the MSDN website : http://msdn.microsoft.com/en-us/library/ms727247.aspx

 

WinSpector & WinID

On the internet you can find several older but interesting spy tools which are also useful for finding the AutomationID (sometimes called ID) of controls. Just try them and use the tool you prefer.

 

Automate Windows Calculator

I created a small demo application to test all the features of the White framework. My application launches the Windows calculator, makes a sum and returns the decimal and binary result. It will click on buttons by searching them by AutomationId and Name, press function keys, send keystrokes, click on menu items, access the contents of a textbox, ... I have this tested this application with Windows Vista (Dutch) and Windows XP (English).

I started with a console application and added a reference to the Core.dll assembly. All other files (Bricks.dll, Bricks.RuntimeFramework.dll, Castle.Core.dll, Castle.DynamicProxy2.dll, log4net.config, log4net.dll, Xstream.Core.dll) should be copied to your project output folder.

Automation starts by calling the static Launch() method of the White Application class. GetWindows() has to be called to get a list of all opened windows. By using search criteria and the generic Get<>() method of a window it is quite easy to access buttons, textboxes, menus, ... The Keyboard singleton can be used to send keystrokes to the focused control or a window.

The source of my demo application is self-describing so I will not explain all the details. 

using System;
using Core;
using Core.InputDevices;
using Core.UIItems;
using Core.UIItems.Finders;
using Core.UIItems.MenuItems;
using Core.UIItems.WindowItems;
 
namespace ScipBe.Demo.White
{
  class Program
  {
    /* The STAThread attribute is needed to use the clipboard in a console application
     * It changes the apartment state of the current thread to be single threaded.  
     * This attribute ensures the communication mechanism between the current thread and 
     * other threads that may want to talk to it via COM. 
     */
    [STAThread]
    static void Main(string[] args)
    {
      // Launch the Windows Calculator (calc.exe)
      Application application = Application.Launch("calc.exe");
 
      // Get the main and only window
      Window window = application.GetWindows()[0];
 
      // Get a reference to the Edit TextBox (AutomationId = "403")
      TextBox textBox = (TextBox)window.Get(SearchCriteria.ByAutomationId("403"));
 
      // Get a reference to the RadioButton "Dec" (AutomationId = "307")
      RadioButton radioButtonDecimal = window.Get<RadioButton>(SearchCriteria.ByAutomationId("307")); 
      // If this RadioButton can not be found, the calculator uses the Standard view
      // Call the Scientfic/Wetenschappelijk menu to make more options visible
      if (radioButtonDecimal == null)
      {
        // Click on menu View/Beeld (AutomationItem = "Item 2")
        window.Get<Menu>(SearchCriteria.ByAutomationId("Item 2")).Click();
        // Click on menu Scientific/Wetenschappelijk (AutomationItem = "Item 304")
        window.Get<Menu>(SearchCriteria.ByAutomationId("Item 304")).Click();
        // The window layout changes. Apparently we need to get the main window
        // again to avoid exceptions when searching the buttons
        window = application.GetWindows()[0];
      }
 
      // Press F6 to make sure we are using the decimal numerical system
      window.Keyboard.PressSpecialKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.F6);
 
      // Focus the Edit TextBox and enter the text 54 
      textBox.Focus();
      Keyboard.Instance.Enter("54");
 
      // Press button "3" (AutomationId = "127") so the value will be 543
      window.Get<Button>(SearchCriteria.ByAutomationId("127")).Click(); 
 
      // Press the "+" button (AutomationId = "92")
      window.Get<Button>(SearchCriteria.ByAutomationId("92")).Click();
 
      // Enter the second value. Press button "6" (AutomationId = "130")
      window.Get<Button>(SearchCriteria.ByAutomationId("130")).Click(); 
      // Press button "7" (AutomationId = "131")
      window.Get<Button>(SearchCriteria.ByText("7")).Click();
 
      // Press the "=" button to calculate the result (AutomationId = "112")
      window.Get<Button>(SearchCriteria.ByText("=")).Click(); 
 
      // Focus the Edit TextBox again and copy the result to the clipboard
      // by sending the keyboard combination CTRL+C
      textBox.Focus();
      Keyboard.Instance.HoldKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.CONTROL);
      Keyboard.Instance.Enter("C");
      Keyboard.Instance.LeaveKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.CONTROL);
 
      // Press F8 to change the numerical system from decimal to binary
      window.Keyboard.PressSpecialKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.F8);
 
      // Show the decimal result via the text on the clipboard
      // and the corresponding binary value by accessing the text of the Edit TextBox
      Console.WriteLine("543 + 67 = {0} (dec)", System.Windows.Forms.Clipboard.GetText());
      Console.WriteLine("{0} (dec) = {1} (bin) ", System.Windows.Forms.Clipboard.GetText(), textBox.Text);
 
      // Close the Calculator application
      application.Kill();
 
      Console.ReadKey();
    }
  }
}

This demo application shows the most important features of White. Hopefully this is enough to start using White in your own applications. If you have any remarks or suggestions, feel free to send me an email.