Customizable On Screen Keyboard (OSK) for WPF (it works for WebBrowser control)

Summary

Recently my colleagues and I faced a very particular problem in WPF. We were creating a very stylish application for a kiosk, and the keyboard should look different from the default OSK in WPF. At the beginning we import our old OSK which concatenates characters to a string, but this approach won’t work on the WebBrowser control since this control does not exposes internal components such as textbox. The following post is about my experience creating a solution for this particular problem.

The Problem

• Customizable style for the on screen keyboard
• The OSK must be able to interact with the WebControl control.

The solution

The default OSK

At first the most logical solution is to use the default. The problem with the default OSK is that it’s not possible to customize. Moreover the default OSK is not embedded into the WPF, it works on top of any other windows application.

Default Windows On Screen keyboard (OSK)

Appending strings

The our team decided to import our old OSK from a previous project. This first version of the OSK was a custom control. This custom control has a lot buttons and one string. Each time a button is pressed, a new character  is append to the string, thus the user of the control only have to check the current string in order to know which characters has been typed. The problem with this other approach is that keys like “arrows”, “control” or “alt” don’t have a way to represented into a string. At this time we’ve created out QueryKeyboard Contol which defines a bunch of buttons and appends characters to a string.

Customizing  a WPF OSK

Later on, we realize that our application will have to interact with the WebBrowser. Thus I decide we can use the same approach that the Wosk: Flexible On Screen Keyboard using WPF (http://wosk.codeplex.com/SourceControl/list/changesets) was using. Basically, WOSK was inserting keyboard  codes into the keyboard buffer, so the the operative system will report the input the the current focused window and control.

Thus, the approach was to separate the logic of the virtual keyboard from the keyboard control. So, the virtual keyboard could work as independent a service or be part of a more complex control such as the QueryKeyboardControl.

I’ve publish a sample of the project in Git Hub at https://github.com/hmadrigal/CodeSamples/tree/master/Hmadrigal.SampleKeyboard. The following items are the most relevant into the project sample:

• Hmadrigal.Services.VirtualKeyboard: This projects holds the Service for a Virtual Keyboard.
• NativeUser32.cs: Wrapper for Win32 low level calls
• KeysEx.cs:  Known key codes. Set of known values to be inserted into the keyboard buffer.
• VirtualKeyboardService.cs: Singleton class which exposes functionality in order to keep the state of the virtual keyboard.
• Hmadrigal.WpfToolkit: This projects holds the visual representation for a on screen keyboard control in WPF.
• QuertyKeyboard.cs: Custom control which keeps logic for appending strings or utilizing the virtual keyboard service.
• Generic.xaml: Contains the default style for the custom control
• WeakEventHandling.cs: Weak event handling (see references)
• Hmadrigal.SampleKeyboard: WPF application using the custom keyboard with a web browser.

Querty Keyboard Control using the virtual keyboard service

Remarks

• The default style has set the attribute “Focusable” to “False” in all the buttons (or any other item which can get  the focus). This should be done for any custom style. Because of the virtual keyboard service does not known who is going to handle the reported input, then the app is responsible of setting the focus to the proper item.
• When debugging and using  keys like shift, alt, ctrl, those are kept pressed by the virtual keyboard service, but they will affect your debug execution because of the input is happening at the operative system level. So, be cautious about where the breakpoints are set.
• Multilingual by using IME. Particularly I’ve set up my PC to use IMEI Japanese using romanji, so  FYI it will continue working .
• Because of the WebBrowser control is a Win32 interop control running on WPF, the default focus might no be on the WebBrowser contol. Sadly I don’t have a solution for this yet, just make sure that the user have an option to set the focus into the WebBrowser control once it has been loaded.

Code

The sample code for this application is on Git Hub at:
https://github.com/hmadrigal/CodeSamples/tree/master/Hmadrigal.SampleKeyboard

References

How to make the Windows OSK open when a TextBox gets focus, instead of requiring users to tap a keyboard icon?
http://stackoverflow.com/questions/7518074/how-to-make-the-windows-osk-open-when-a-textbox-gets-focus-instead-of-requiring/7529920#7529920

.NET Framework Developer Center / Creating custom InputDevice
http://social.msdn.microsoft.com/Forums/en-US/wpf/thread/a813a08a-7960-45fe-bc91-e81cdb75bd10

Wosk: Flexible On Screen Keyboard using WPF
http://wosk.codeplex.com/SourceControl/list/changesets

SendInput Example In C#
http://www.ownedcore.com/forums/mmo/warhammer-online/186390-sendinput-example-c.html

Simulating Keyboard with SendInput API in DirectInput applications
http://stackoverflow.com/questions/3644881/simulating-keyboard-with-sendinput-api-in-directinput-applications

Generic implementation for a weak event handling
http://puremsil.wordpress.com/2010/05/03/generic-weak-event-handlers/

From Joel Pobar’s CLR weblog Creating delegate types via Reflection.Emit
http://blogs.msdn.com/joelpob/archive/2004/02/15/73239.aspx

 

Xhader 1.1

Hello Fellas,

Today I’ve released a new version of Xhader.

http://xhader.codeplex.com/releases/view/77425

The major feature is that it should work on Blend 4. Additionally it should be using WPF 4.0 and SL 4.0 support.

Regards,
Herber

 

The Chaos Monkey and Working with AWS

Hello,

There has been awhile since my last post. I’d like to share a really good posts that I’ve read recently.

As usual, I want to share two articles:

5 Lessons We’ve Learned Using AWS
http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html

Working with the Chaos Monkey
http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html

The second is a post based on the first one, both are good to think about: “the best way to avoid failure is to fail constantly”

 

Setting up F# to bu used in Ubuntu with F#

Hello,

Because of I’m curious about technology, I’ve decide share some links and comments about it.
In general the process is simple:

1. Download the F# distribution from microsoft site or codeplex

http://msdn.microsoft.com/en-us/fsharp/cc835251.aspx

http://fsxplat.codeplex.com/

Uncompress and run the install script

2. You can also download the additional scripts.
Same procedure from http://fsxplat.codeplex.com/, the installation is similar uncompress and run the installation script. There is a chance where you can get the following error:

install-bonus.sh: 28: Syntax error: “(” unexpected (expecting “}”)

If so, change the first line from
#! /bin/sh -e
to
#!/bin/bash -e
source: http://askubuntu.com/questions/21100/problem-with-script-substitution-when-running-script

I’ll assume you’ve already MonoDevelop installed, so. Make sure you’re following all the pre requisites to make F# work on mono:

http://functional-variations.net/monodevelop/prereq.aspx

At last but not least, you can install the add-in for mono

A good amount of steps can be located at http://functional-variations.net/monodevelop/ , or you can try just by adding the repository and install it (repository http://functional-variations.net/addin )

and pretty much that it. if you want to see a video, there is one available at http://functional-variations.net/crossplatform/.

Microsoft Research has published a side dedicated to teach FSharp at http://www.tryfsharp.org/

Best regards,
Herber

 

Visual Studio re install individual packages

Hi,

I remember that sometime ago my Visual Studio installation didn’t install the package correctly. And if you’ve installed visual studio you might know that the standard installation might take 1 hour at least. If you decide to repair the visual studio then the process can take 2 hours. However by reinstalling the visual studio you still might have errors.

I resolved my problems by reinstalling only the broken package. So, I’ll explain how you can do this. I’ll take the images from my friend Anthony (arbot) which has the same issue, and he could solve it by installing only what he needed (see details at this post). So my example is based on Arbot’s problem with the “Microsoft SQL Server 2008 Management Objects“

Step 1: Prepare your bat-belt
Easy dude, you will need the Visual Studio Installation files. You are more likely to need admin rights to perform installation tasks.

Step 2: Identify your broken component
In the Arbot, the broken package was “Microsoft SQL Server 2008 Management Objects”. Unfortunately these kind of errors does not have a friendly message. Usually they might look like:

So don’t be afraid if you see:
“Could not load file or assembly
‘Microsoft.SqlServer.Management.SqlParser, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91′ or one of its dependencies. The system cannot find the file specified“
The thing here is that you should be able to recognize which part of your Visual Studio is failing. If you aren’t sure about the component, then check the list of packages /components that Visual Studio installs at
http://msdn.microsoft.com/en-us/library/ee225240.aspx

Step 3: Remove what it’s not working
Once you know what component is missed, go to the control panel and uninstall the specific component. In the case or Arbot he uninstalled all the products from the “Microsoft SQL Server 2008 Management Objects”, which are under the folder WCU\SMO see http://msdn.microsoft.com/en-us/library/ee225240.aspx to know which components are installed under the same folder.

Step 4: Make sure you’re ready to reinstall
Ok, It seems obvious, but before installing the Visual studio components you will have to make sure that your current installation has all the components you might need. So, take your time and check http://msdn.microsoft.com/en-us/library/ee225240.aspx and the prerequisites

Step 5: Install in order each package
open a command prompt in admin mode. (I’d recommend Visual Studio Command Prompt), and make sure MSIExec is in your path (http://technet.microsoft.com/library/cc759262(WS.10).aspx). So, we can run the installer from the command prompt.
Now In the command prompt, go to the installer folders of your broken package. In the case of Arbot, look for “WCU\SMO” folder. Review which platform you’re using because X86 platforms uses different commands than the X64 platforms. Additionally there are two kind of X64 platforms.
One you have identifies the packages that you have to install run the commands, in the case of Arbot he ran these commands:


C:\Windows\system32>MSIExec SQLSysClrTypes_x86_enu.msi /log "%TEMP%\dd_SQLSysClr
Types_x86_msi.txt"
C:\Windows\system32>MSIExec SharedManagementObjects_x86_enu.msi /log:"%TEMP%\dd_SharedManagementObjects_x86_MSI.txt"


Depending of the settings you’ve specified for MSIExec you might have to attend the installation and restart your computer.

Step 6: Try you reinstalled component on visual studio
If the installation worked properly it shouldn’t report any issue, and you always have the option to check the log file generated by the MSIExec.

Now you should be able to run Visual Studio and try once again to use the feature on Visual studio, and hopefully it should be working properly. My friend Anthony made a post about his problem if you need to see more pics

http://abakerp.blogspot.com/2011/02/cannot-load-file-or-assembly.html

Best regards,
Herber

 

XNAWP7-1 A introduction

Hello ,

I’ve been thinking about what should be my next post. I’d like to talk about WCF RIA, where I have had some encounters, but it’s like too soon to write about it. On the other hand, I won a raffle and the prize was a book about Windows Phone 7 Game Development (Thanks to 101FreeTechbook and additionally there is going to be a free workshop about XNA 4.0 development at gamedevelopedia (Starting at 17-Jan-2010). Thus, I’ve decided to write a set of tutorials about XNA and WP7 for those who wants to start as well as I want. If everything goes well the series will have the following tag XNAWP7, and I’ll try to post every week about the every new topic. So, lets get started.

WP7: Windows Phone 7 Development
Windows Phone 7‘s the platform Microsoft has release to create new experiences on Windows Phone development. This new platform is based on managed code, so it’s backed on .NET Compact Framework. At this time it’s possible to write application C#. There are two main framework available for WP7 development:

XNA: Xna is Not an Acronym
XNA is a high-performance game framework. It’s over DirectX and it allows to create application taking advantage of high performance on Graphics, sounds, networking and input devices. Additionally, it can take advantage of Microsoft Live to have a customized game experience. The only drawback of this platform is that you have to create most of the components almost from scratch or get a third party framework instead.

SL: Silverlight
Silverlight is framework build on .NET and it allows to create rich interactive applications by using a declarative language (XAML). Xaml uses XML to describe how the UI is drawn, and the C# to details on logic and interaction. This framework also supports game development, this framework is fairly enough for games which does not have high-performance graphics nor have tons of elements. Event though, SL4 is already in the market and SL5 is on feature request on Jan-2011, WP7 uses an special flavor of Silverlight 3. This special flavor of SL3 has removed not related feature of mobile such as System.Window.Browse namespace, and it has added specific new namespaces in order to take advantage of the platform specific items.

Windows Phone 7 and its hardware
Microsoft has defined very specific hardware in order to be able to run properly WP7 Operative System (WP7 OS).

• 1 Ghz CPU processor
• WVGA (480×800) or HVGA (480,320)
• Accelerometer sensor. This sensor helps to detect the orientation of the device, this means it allows to detect if the device is point up, down, left right, etc..
• GPS sensor This sensor helps to detect the position around the world by detecting the latitude and longitude

Development on WP7
At lats but not least, let see what it’s required to develop on WP7 platform. The SDK is free and it can be downloaded at AppHub If you have Visual Studio already installed the WP7 tools will integrate with VS automatically. If you don’t have Visual Studio, then you can get the Visual Studio Express version for free at Visual Studio Express.
If you wan to run you application in you phone, then you’ll need to get a XNA creators account to being able to deploy your application into your phone. Additionally by acquiring an account you’ll be able to publish your game in the Market place, at this time (Jan-2011) only US and Europe are available to publish content, it’s planned to open more countries during the next years.
Don’t get worried if you don’t have a Windows Phone 7, you can start coding using the emulator which is already included with WP7 SDK, however it’s strongly recommended to use a physical WP7 to perform final tests.

An initial application
As traditional, the hello world application. So, on this little sample you it’s possible to appreciate the basis to display text and sprites.

The Main Game loop
If haven’t worked with game frameworks before, let me point out a concept known as game loop. Traditional programing models are event based, this means that a certain functionality is executed by a given trigger. But the most common video game frameworks does not work on this scheme, instead they use the game loop which is a “infinite” cycle where the application performs an update and then draw the scene. Frameworks such as XNA have methods invoked before and after the game loop is executed to perform initialization tasks, or free resources. This main loop game is implemented by the Game class of XNA. BTW there is a particularity of using XNA on WP7. Normally an application has an entry point such as the main function, and you can clearly see this function on a WP7 template project. However WP7 loads looks for instances of Game in your projects and handles them such a contract to define the execution of your application.

A sprite
The word sprite is an term which hasn’t changed from its origin. During the age of 8-bits the very first images that were drawn on a screen were called sprites. So, an sprite a two dimensional rendered figure . Sprites have become a great resource because much of the game frameworks does not have a graphic interface library (GUI), thus sprites provides a good-alternative mechanism. On XNA an sprite is represented by the Texture2D class.

Sprite font
Particularly XNA has decide to draw text by using sprites. Which means that all the charset is backed by images. A font on XNA is associated to an spritefont. On XNA an sprite is represented by the SpriteFont class.

Drawing sprites
XNA defines a tool for drawing sprites (or spritefonts). XNA helps you to deal with the resources of you application by providing a content loader. This content loader is able to load the sprites, fonts, audio files that you application will consume.

some csharp code
With all the previous things said I’m attaching what it would be the first post (hopefully of many more posts).

Source code is at http://github.com/hmadrigal/xnawp7

 

Aspect Oriented Programming and Interceptor design pattern with Unity 2

Hello,

I’ve found a Microsoft article about Unity and Aspect Oriented programming, and it was really interesting. So, to be the theory in action I’ve decide to exemplify how Unity 2 uses the Interceptor design pattern to support Aspect Oriented Programming (aka AOP).  First of all, lets get the basis of each of these three main members of my sample:

• Microsoft Unity 2 is a Dependency Injection Framework which also supports which helps us to create loosely couple applications. See details at http://msdn.microsoft.com/en-us/library/dd203101.aspx
• Aspect Oriented Programming ( AOP ) is a development style where the main concerns are separated from the secondary concerts of the application. E.g. you can be able to separate your the logic used to log information from the main business logic of the application. See more info at http://en.wikipedia.org/wiki/Aspect-oriented_programming
• Interceptor design pattern is a software design pattern which allows to insert and configure logic when executing code in given methods. See more info at http://en.wikipedia.org/wiki/Interceptor_pattern

Once again, I’ve decide to use MonoDevelop to create the sample and the source code is available to be downloaded from github at https://github.com/hmadrigal/CodeSamples/tree/master/AspectOriented.Terminal

Sample structure

This example supposes that you have a proxy and want it to add logging information. So, we define a contract (interface) called IProxy to define the expected behavior of our proxy and let unity know which is the contract. Then we have in a separate project an concrete implementation Proxy which implements the interface. At last but not a least, we have a project which defines a Unity interceptor. All these components are united by the configuration file which lets unity know how these components should interact.

AspectOriented.Infrastructure’s project

This project contains all the common resources and the contracts (interfaces) to be used in the sample. Here is defined the IProxy interface:

using System;
namespace AspectOriented.Infrastructure.Services
{
  public interface IProxy
  {
    bool IsEnabled ();
    void Open ();
    void Close ();
  }
}

AspectOriented.UnityInterceptors’s project

The second project we’re going to talk is where the interceptor is defined. As we’ve already mentioned the interceptor will allow unity to perform tasks before and/or after the intercepted method is invoked. To define  which method is going to be intercepted, we can use two alternative: a configuration file or by writing some code.This will he explain in the AspectOriented.Terminal’s project.

using System;
using System.Collections.Generic;
using Microsoft.Practices.Unity.InterceptionExtension;

namespace AspectOriented.UnityInterceptors
{
	public class DiagnosticInterceptor : IInterceptionBehavior
	{
		public DiagnosticInterceptor ()
		{
		}

		#region IInterceptionBehavior implementation
		public IMethodReturn Invoke (IMethodInvocation input, GetNextInterceptionBehaviorDelegate getNext)
		{
			System.Console.WriteLine (String.Format ("[{0}:{1}]", this.GetType ().Name, "Invoke"));

			// BEFORE the target method execution
			System.Console.WriteLine (String.Format ("{0}", input.MethodBase.ToString ()));

			// Yield to the next module in the pipeline
			var methodReturn = getNext ().Invoke (input, getNext);

			// AFTER the target method execution
			if (methodReturn.Exception == null) {
				System.Console.WriteLine (String.Format ("Successfully finished {0}", input.MethodBase.ToString ()));
			} else {
				System.Console.WriteLine (String.Format ("Finished {0} with exception {1}: {2}", input.MethodBase.ToString (), methodReturn.Exception.GetType ().Name, methodReturn.Exception.Message));
			}

			return methodReturn;
		}

		public IEnumerable<Type> GetRequiredInterfaces ()
		{
			System.Console.WriteLine (String.Format ("[{0}:{1}]", this.GetType ().Name, "GetRequiredInterfaces"));
			return Type.EmptyTypes;
		}

		public bool WillExecute {
			get {
				System.Console.WriteLine (String.Format ("[{0}:{1}]", this.GetType ().Name, "WillExecute"));
				return true;
			}
		}
		#endregion
	}
}

As you might noticed the interceptor implements IInterceptionBehavior which will let Unity know that this class can act like an interceptor. To simplify things we only print message about when these calls are performed.

AspectOriented.Terminal’s project

This is the main project which generates the executable file. Here are two important things to point out.  Unity has being configured by using the configuration file, however you can use code instead. The main application performs calls on the instance that unity resolves, the interception process happens behind scenes and we only will see the result when we call the methods of the proxy.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<configSections>
		<section name="unity" type="Microsoft.Practices.Unity.Configuration.UnityConfigurationSection, Microsoft.Practices.Unity.Configuration" />
	</configSections>
	<unity xmlns="http://schemas.microsoft.com/practices/2010/unity">
		<!-- Interception is not part of the default Unity configuration schema. -->
		<!-- Before you can configure interception you must add the correct sectionExtension element to your configuration -->
		<!-- section in the configuration file.-->
		<sectionExtension type="Microsoft.Practices.Unity.InterceptionExtension.Configuration.InterceptionConfigurationExtension, Microsoft.Practices.Unity.Interception.Configuration" />
		<!-- Defines some aliast to easily manipulate the mappings -->
		<alias alias="IProxy" type="AspectOriented.Infrastructure.Services.IProxy, AspectOriented.Infrastructure" />
		<alias alias="Proxy" type="AspectOriented.Terminal.Services.Proxy, AspectOriented.Terminal" />
		<alias alias="DiagnosticInterceptor" type="AspectOriented.UnityInterceptors.DiagnosticInterceptor, AspectOriented.UnityInterceptors" />
		<!-- Default Container when creating the tree-chain of resolution-->
		<container>
			<!-- Loading the section extension only enables the interception configuration to be given in the configuration file. -->
			<!-- Interception itself will not work unless you also load the interception container extension in your Unity container instance.-->
			<extension type="Interception" />
			<register type="IProxy" mapTo="Proxy">
				<lifetime type="ContainerControlledLifetimeManager" />
				<interceptor type="InterfaceInterceptor" />
				<interceptionBehavior type="DiagnosticInterceptor" />
			</register>
		</container>
	</unity>
</configuration>

The configuration file defines two section which are mandatory to define interceptors, these are sectionExtension and extension. Then into the registration of a given type (or in our sample the IProxy Type) we can define the interceptors for this particular type resolution.

using System;
using Microsoft.Practices.Unity;
using Microsoft.Practices.Unity.Configuration;
using System.Configuration;
using AspectOriented.Infrastructure.Services;

namespace AspectOriented.Terminal
{
	class MainClass
	{
		public static void Main (string[] args)
		{
			// Loads the container
			IUnityContainer container = new UnityContainer ();
			container = Microsoft.Practices.Unity.Configuration.UnityContainerExtensions.LoadConfiguration (container);

			// Resolve the proxy-sample
			var proxy = Microsoft.Practices.Unity.UnityContainerExtensions.Resolve<IProxy> (container);
			if (proxy.IsEnabled ()) {
				proxy.Open ();
			}
			proxy.Close ();

			// End of the test
			Console.ReadKey ();
		}
	}
}

At last but not least we’ve our main code. It’s quite straightforward, it creates a Unity container, loads the container configuration from the configuration file, and then uses the container to resolve the given interface (contract). As I mentioned the magic happens behind-scenes, when you call the Proxy methods there is functionality which is going to be injected before and after each call of this instance. Take a look of the final screen-shot to see the all the terminal messages which were triggered during the method calls of the Proxy.

As you can notice the interception calls are injected before and after each call of the proxy instance. And this same functionality can be plugged to any other component (instance) by using the same mechanism.

Resources about this topic

• Aspect-Oriented Programming, Interception and Unity 2.0 http://msdn.microsoft.com/en-us/magazine/gg490353.aspx
• Specifying Types in the Configuration File http://msdn.microsoft.com/en-us/library/ff660933%28v=PandP.20%29.aspx
• Extending the Unity Configuration Schema http://msdn.microsoft.com/en-us/library/ff660880%28v=PandP.20%29.aspx
• Configuration Files for Interception http://msdn.microsoft.com/en-us/library/ff660932%28v=PandP.20%29.aspx
• Unity Interception Extension http://www.peternilsson.me/#post14
• Microsoft Unity – code to xml http://stackoverflow.com/questions/640806/microsoft-unity-code-to-xml

Once again, I’ve decide to use MonoDevelop to create the sample and the source code is available to be downloaded from github at https://github.com/hmadrigal/CodeSamples/tree/master/AspectOriented.Terminal

 

 

Free ebook “Free ebook: Moving to Microsoft Visual Studio 2010″

Hi,

Microsoft has a new free ebook. This is about Visual Studio 2010, you can get it at:

http://blogs.msdn.com/b/microsoft_press/archive/2010/09/13/free-ebook-moving-to-microsoft-visual-studio-2010.aspx

Best regards,

Herber

 

Free ebook: Programming Windows Phone 7, by Charles Petzold

Hello

Certainly a platform is as strong as the tools that you can use on it. Microsoft is giving a lot of resources for developers in order to start producing in Windows Phone 7 (WP7). This time Microsoft Press book is giving a free ebook Programming Windows Phone 7 by Charles Petzold.

 

You can download it at Free ebook: Programming Windows Phone 7, by Charles Petzold

Best regards,
Herber

 

Xhader 1.0 has been released!

Hello,

This is my first open source project that I’ve been published. It’s not based on my code, it’s based on a set of several tools and helpers I’ve been able to find in the web. So, the project itself is just a easy way to access all that resources in one single point by using your Microsoft Blend tool.

Xhader 1.0 Summary

Xaml is a markup language based on xml, which is used to create presentation layer in your Microsoft Application e.g. Silverlight Applications and WPF applications. There is a particular feature that is available for certain users which is to use the support to modify the rendere

See more information at:
Documentation

Download

Best regards,

Herber