I just finished up a Lightning Talk at the 2010 Linux Symposium where I chose the somewhat provocative title

Let go of your widgets! Image based embedded user interfaces

In such a short time frame it is hard to cover all of the different reasons why including a graphic designer as part of an embedded product development team has been a challenge … so I glossed over it and moved on.

Instead, I focused on looking at why Linux is a great platform for embedded products to start with.  In addition to its large developer community, Linux has near universal support from various silicon vendors.  Nearly every modern evaluation board that you will get from a vendor like Freescale or Renesas (among others) comes with Linux support available out of the box.

The problem arises as you move away from hardware/operating system choices up the application stack.

A common complaint for first time embedded Linux users is that there are simply to many choices and options available to them.  When you are getting started with a new technology, having more than one path forward is more of a hindrance than a benefit.

This is particularly challenging when it comes to choosing the rendering platform/technology that your user interface is going to be reliant on. Here are just a few of the choices available to you under Linux:

fbdev, directfb, Open GL/GL ES, OpenVG, x11/TinyX, QT, gtk/gtk+, cairo, skia, svg

… and the list goes on

Many of these technologies build on one another, but under Linux the graphics landscape is far from straightforward.

Unfortunately, with the way embedded user interfaces are developed today, a significant amount of time is spent translating the content and vision of a graphic designer to the actual screen content layout used in the product.  This work can’t start in earnest until a particular rendering technology is chosen.  Under Linux we’ve seen some of our customers completely paralyzed by this decision … so much that they’ve jumped ship to other OS technologies with a simpler graphics story.

This was bad news to deliver at a Linux conference

The good news is that we are working on a solution.  Storyboard Suite alleviates the need to select a particular graphics technology or operating system or hardware.  With to its flexible back end runtime technology you can continue to build your application while you make hardware and operating system selections, and the work can all be performed by a graphic designer who is ideally positioned to create and manage the product’s user interface.

An interesting case in point is our Beagleboard support.  We can easily leverage three different rendering technologies on this platform; fbdev, directfb and hardware accelerated OpenVG.  It is very interesting to see the small differences in general performance when you run the same application, but I don’t want to ruin the surprise so you’ll have to try it for yourself.

The good news for the Linux folks is that we run WinCE on this same board and Linux does run better out of the box!

Thomas

In my last layer post I discussed how you can use hardware layers to achieve animation effects using limited CPU.  The only problem with this is that most graphics toolits out there do not give you access to the layers.  Some call it a video overlay and let you use one layer but what if the hardware has more layers.  The Freescale i.MX line has 2 layers that can be used but some chips have up to 8 independent layers.  To my knowledge GTK and QT do not allow you access to multiple layers.  Other user interface applications, such as Adobe Flash and Browser based HMI’s render to a single framebuffer and therefore cannot take advantage of all the hardware features.

One system that has embraced the layer concept is the QNX Advanced Graphics framework, or the GF library.  This library was designed from the ground up to match the capabilities of the hardware and provide access to all features.  Being one of the original architects I feel it gives the most powerful API for layer control .   Using the GF library you can attach to layers and set features such as source and destination viewports, alpha blending and color keying.  For example to load a large image into a surface and pan around it you could do this:

// attach the image surface to the layer 

gf_layer_set_surfaces(layer, image_surface, 1);

// set destination viewport to the size of the display

gf_layer_set_dst_viewport(layer, 0, 0, xres, yres);

// show the top left content of the image surface

gf_layer_set_src_viewport(layer, 0, 0, width, height);

gf_layer_update(layer, 0);

Now you can pan the image using the display controller very easily by changing the viewport.

gf_layer_set_src_viewport(layer, x, y, x+width, y+height);

gf_layer_update(layer, 0);

To do this on other systems would require you to blit an area of the source image each time.  Doing this on a time can achieve a smooth animation such as an Ease-in-out.

Brian