Customizable On Screen Keyboard (OSK) for WPF (it works for WebBrowser control)

Summary

Recently my colleagues and I faced a very particular problem in WPF. We were creating a very stylish application for a kiosk, and the keyboard should look different from the default OSK in WPF. At the beginning we import our old OSK which concatenates characters to a string, but this approach won’t work on the WebBrowser control since this control does not exposes internal components such as textbox. The following post is about my experience creating a solution for this particular problem.

The Problem

• Customizable style for the on screen keyboard
• The OSK must be able to interact with the WebControl control.

The solution

The default OSK

At first the most logical solution is to use the default. The problem with the default OSK is that it’s not possible to customize. Moreover the default OSK is not embedded into the WPF, it works on top of any other windows application.

Default Windows On Screen keyboard (OSK)

Appending strings

The our team decided to import our old OSK from a previous project. This first version of the OSK was a custom control. This custom control has a lot buttons and one string. Each time a button is pressed, a new character  is append to the string, thus the user of the control only have to check the current string in order to know which characters has been typed. The problem with this other approach is that keys like “arrows”, “control” or “alt” don’t have a way to represented into a string. At this time we’ve created out QueryKeyboard Contol which defines a bunch of buttons and appends characters to a string.

Customizing  a WPF OSK

Later on, we realize that our application will have to interact with the WebBrowser. Thus I decide we can use the same approach that the Wosk: Flexible On Screen Keyboard using WPF (http://wosk.codeplex.com/SourceControl/list/changesets) was using. Basically, WOSK was inserting keyboard  codes into the keyboard buffer, so the the operative system will report the input the the current focused window and control.

Thus, the approach was to separate the logic of the virtual keyboard from the keyboard control. So, the virtual keyboard could work as independent a service or be part of a more complex control such as the QueryKeyboardControl.

I’ve publish a sample of the project in Git Hub at https://github.com/hmadrigal/playground-dotnet/tree/master/MsDesktop.OnScreenKeyboard. The following items are the most relevant into the project sample:

• Hmadrigal.Services.VirtualKeyboard: This projects holds the Service for a Virtual Keyboard.
  • NativeUser32.cs: Wrapper for Win32 low level calls
  • KeysEx.cs:  Known key codes. Set of known values to be inserted into the keyboard buffer.
  • VirtualKeyboardService.cs: Singleton class which exposes functionality in order to keep the state of the virtual keyboard.
• Hmadrigal.WpfToolkit: This projects holds the visual representation for a on screen keyboard control in WPF.
  • QuertyKeyboard.cs: Custom control which keeps logic for appending strings or utilizing the virtual keyboard service.
  • Generic.xaml: Contains the default style for the custom control
  • WeakEventHandling.cs: Weak event handling (see references)
• Hmadrigal.SampleKeyboard: WPF application using the custom keyboard with a web browser.

Querty Keyboard Control using the virtual keyboard service

Remarks

• The default style has set the attribute “Focusable” to “False” in all the buttons (or any other item which can get  the focus). This should be done for any custom style. Because of the virtual keyboard service does not known who is going to handle the reported input, then the app is responsible of setting the focus to the proper item.
• When debugging and using  keys like shift, alt, ctrl, those are kept pressed by the virtual keyboard service, but they will affect your debug execution because of the input is happening at the operative system level. So, be cautious about where the breakpoints are set.
• Multilingual by using IME. Particularly I’ve set up my PC to use IMEI Japanese using romanji, so  FYI it will continue working ;-).
• Because of the WebBrowser control is a Win32 interop control running on WPF, the default focus might no be on the WebBrowser contol. Sadly I don’t have a solution for this yet, just make sure that the user have an option to set the focus into the WebBrowser control once it has been loaded.

Code

The sample code for this application is on Git Hub at:
https://github.com/hmadrigal/playground-dotnet/tree/master/MsDesktop.OnScreenKeyboard

NOTE (01/29/2014) Even at this time, this post continues to be one of the most viewed in my (humble) blog. However, I had forgotten to update this post in particular, at https://polaris.codeplex.com is the latest version of this keyboard (which is more likely to have less bugs and more flexibility such as a Debug Mode, and adding custom keys with more ease). My advice is to get the code and export the control, since the polaris libraries are meant for being downloaded and work all together, and you will be more likely to use only the keyboard.

References

How to make the Windows OSK open when a TextBox gets focus, instead of requiring users to tap a keyboard icon?
http://stackoverflow.com/questions/7518074/how-to-make-the-windows-osk-open-when-a-textbox-gets-focus-instead-of-requiring/7529920#7529920

.NET Framework Developer Center / Creating custom InputDevice
http://social.msdn.microsoft.com/Forums/en-US/wpf/thread/a813a08a-7960-45fe-bc91-e81cdb75bd10

Wosk: Flexible On Screen Keyboard using WPF
http://wosk.codeplex.com/SourceControl/list/changesets

SendInput Example In C#
http://www.ownedcore.com/forums/mmo/warhammer-online/186390-sendinput-example-c.html

Simulating Keyboard with SendInput API in DirectInput applications
http://stackoverflow.com/questions/3644881/simulating-keyboard-with-sendinput-api-in-directinput-applications

Generic implementation for a weak event handling
http://puremsil.wordpress.com/2010/05/03/generic-weak-event-handlers/

From Joel Pobar’s CLR weblog Creating delegate types via Reflection.Emit
http://blogs.msdn.com/joelpob/archive/2004/02/15/73239.aspx