When building a product with a user interface the trade off is always to get the best graphics performance with the lowest processor power.  Most of these processors have no graphics acceleration and possibly limited instruction sets.  Over the years I have come up with a list of simple things that can be done when designing the target user interface which will improve performance.  These are guidelines for designing the UI with should not really limit the designer very much.

Alpha blending is a killer on these systems.  If you have images which are not blended make sure you do not save them with an alpha channel in them.  Many times Photoshop and other tools will put an alpha channel into a PNG even if the image is fully opaque.  At runtime this can still slow some systems down as they have to check each pixels value.

Try to create opaque image images so that the color format matches your destination color format.  If the target framebuffer is in 16Bit color then create opaque images in a 16Bit format so that color conversion does not have to take place.

Scaling image data can be very costly on low end systems especially when filtering is used to make the images look better.  Try and scale the images on the host machine and put these scaled images on the target.

True Type fonts are generally drawn using alpha blending similar to images.  Bitmap fonts are generally simpler to draw and use much less CPU so if performance is key a nice bitmap font can be very helpful.  Also try and reuse fonts in the system as most UI packages can cache font metrics and image data.

Brian

I’ve been working on a base view in our tooling that would allow for drop down elements.  What I mean by this is that if there is an error in the view, then a label will drop down and let you know, or if you want to search the view for something, then hit a key combination and a search box will drop down.  There may be other stuff that I can put in at a later date, but for now I am focusing on those two.  I’m doing this because I think it’s a good way of getting rid of some of the dialogs that pop up in our tool.

The Eclipse layouts though, don’t seem to think that this is a good idea.  They want to layout my hidden area.  They want to lay it out in an unhidden sort of way.    Now to be fair, I haven’t told the layout that I don’t want hidden widget laid out.  The layout is just doing what it was written to do.  There are two ways that I can think of to get around this.  One is that I don’t create the hidden widget area until it is needed and then drop it down.  The other is that I create some layout data for the hidden widget and then tell that layout data that I want to exclude this widget from laying out.  Don’t worry, my widget has no issues with being excluded.  It has a very high self esteem.

The first solution makes for a cleaner API because the class doesn’t have to know anything about layouts.  However, once that widget is created, if a layout happens in it’s parent while it exists, I have to handle that somehow, or I am back to square one with the layout managing my widget.  I think that this will lead to a lot of suspect code as I try and listen for events and stop them from affecting my hidden area that is now shown.

The second solution will probably give me a better result as I will be using the Layout API and it will be doing what it thinks it is supposed to be doing.  However, my class will now have to figure out what the layout of the parent is, in order to create a layout data instance for my widget.  This means that I have to have a call to set the layout data, or I need to modify my creation function to take a layout data class in.  Also, because the layout data classes are not sub-classed off of a superclass, I have to handle each possible type of layout data independently, which is a bit cumbersome.

For some reason I am not too crazy about either solution.  It’s not that there is anything glaringly wrong with either solution, but this just feels as though it’s something that I shouldn’t have to do.  At least not in an object orientated language.  Ideally I think there should be a create data type call within the layout classes.  This call could return a layout data instance that is proper for their particular type of layout.  The layout data base class could contain all the generic stuff like width hint, height hint, and whether or not this particular widget should be excluded, and therefore I could at least manipulate some of the layout attributes on my widget without having to know specifically how the parent it going to request it’s children to layout.

I’m going to continue to research and think about this, but first I think I need to lay down. I may not be able to though, as my children are not hidden.

-Rodney

I just came across this article talking about the well known lack of flash on the iPhone. I felt it made many great points to explain why Apple wouldn’t want flash on their phone despite the demand from 80% their customer base. The iPhone’s user experience is one it’s greatest differentiators in the market and if they feel having flash on their device would compromise the performance and battery life then who can fault them.

Here is good quote from the author :

Anyway, my point is this: Apple want to tightly control the stuff people make for the iPhone because the product’s selling point isn’t the hardware, it’s the software — the user experience. Hardware can be copied — a user experience cannot.

It’s a good read and highlights some of the issues seen by many people who have tried using a technology not designed for the embedded market on their product.

I’ve been working on porting some open source code to Windows CE recently and ran into something which got me thinking about portable code, and thinking ahead when designing/implementing something.

Consider the Windows programming API. It provides both ANSI and Unicode versions of its API functions. The base API call is mapped to the correct version at compile time via preprocessor #defines. For example, CreateFile is mapped to CreateFileA and CreateFileW depending on if UNICODE is defined or not. The difference being, of course, that the filename argument varies in the size of it’s character array (char * vs. w_char *).

Now lets say some code directly references the ANSI version, because they aren’t concerned about Unicode and wide chars. This is fine and dandy until you try to port to a WINCE SDK that doesn’t include the ANSI versions in its core libraries. One or two instances of this aren’t that big of an issue, but chances are if a developer did this once, the practice has been reproduced throughout the code base. A portability nightmare.

The problem above is easily solved by calling the base API call, and creating a typedef for a base character array element which is dependent on the same preprocessor #define that Windows keys off of.

I would argue it is rare, especially when dealing with open source, to work on a piece of code that won’t be modified from it’s intended purpose or ported to a different platform by someone you will never interact with. An extra 30 minutes spent on thoughtful design creates much more portable code. This spares future developers working on the code base countless hours and headaches (which, hopefully, they pay forward through good design)!

–Chris K.

We have been hard at work enhancing Storyboard and taking advantage of new rendering technologies.  Recently I decided to add OpenVG into the mix. 

OpenVG is a vector graphics rendering API from the Khronos group.  There are hardware based accelerators from some vendors including Freescale for such boards as the i.MX35.  Storyboard can now render via the OpenVG API and take advantage of this hardware acceleration.  This also gives us acceleration for rendering effects such as gradients, rotation and scaling.  We can also provide the image effects such as convolution and blur via this API and custom render extensions.

The port was developed and verified using the OpenVG reference implementation provided by the Khronos group.  The reference implementation generally runs on windows using glut and there were a few changes required to build it under a linux system, which we would be happy to provide to anyone who wants them.

Stay tuned for the next renderer …

Brian

Happy New Year Everyone!

Well its great to be back in the Crank office after over indulging during what a very dear friend of mine calls “The Days of Plenty”. I hope all of you had a safe and relaxing holiday. I know I did. I had big plans on doing some work around the house and finishing up some work related stuff but ended up playing dinosaurs with my kids

So here we are at the beginning of 2010. A time to reflect on what we accomplished in the previous year and at the same time plan or set goals for the year ahead. Crank has achieved much in 2009 from releasing Storyboard 1.0 and then following up with 1.0.1. For 2010 we have a long list of exciting things we are working on. Everything from new features, functionality and supported platforms; all to be included in a near future Storyboard 1.1 release.

Things are already starting to heat up for 2010 in the electronics sector. Google is set to announce their GPhone (Nexus One). Apple is rumoured to finally talk about their tablet or iSlate. Freescale has already made an announcement about their tablet. CES is starting this week and funny how everyone is talking about smart phones and tablets. Sounds like a great time to speak to a company who is an innovator in embedded user interface solutions for consumer electronics, automotive head units, and industrial devices

Our very own Jason Clarke will be at CES  this week and he would love the opportunity to meet with you and discuss your experience with Storyboard or chat about possible areas where Storyboard might fit into your existing or future product releases. Jason can be reached at jason@cranksoftware.com to setup a meeting.

We here at Crank look forward to working with all of you and wish you a very happy and prosperous 2010!

What can I say, 2010 is going to be a very, very exciting year for Crank … a year of moving mountains!

-JamieV

Just in case you haven’t fully made up your mind on what you wanted for Christmas, here is one last minute idea that you can add to your list for Santa: Storyboard 1.0.1 … released just hours before Santa’s getting his sleigh ready for transit!

Storyboard 1.0.1 Update Site

I sat down to write out this entry shortly after writing about Storyboard’s initial success, but I found that it quickly became something that would likely never be read.

Instead of that monster post, I’ll just keep hitting the topic of widget free user interface development from different points of view in the next few posts.

Graphic Designers Don’t Draw Widgets

If you are building a user interface design tool that is focused on a target audience of graphic designers and artists, you have to integrate into not just their tools and work flow, but their creative approach. 

 Some may say that building user interfaces that don’t use traditional widget controls is a dangerous path to take so let me qualify.  I’m not suggesting that a user interface not have buttons, check boxes, scroll bars and the like.  I’m suggesting that if you ask someone artistically inclined to create a  button that there are very few constraints that they would encounter as part of their creative process.

It is the platforms that our application’s run on that impose constrains.  When the windowing system, and application framework or some widget toolkit library declares that your button must have a particular shape, beveled edges, rounded corners or some other arbitrary limit, you are going to have to make one of two decisions:

1. You compromise on creativity.  The stunning visual prototype of the application must be adjusted in order to fit in to the arbitrary constraints that are being imposed upon it externally.
2. You pay more to get what you want.  Every constraint can be removed if you are willing to pay the cost, potentially meaning investing significant software development effort just to get the look and feel you want.

Call me stubborn, but I want to have my cake and eat it too.  The problem is not a technical one, it is a function of software user interface design.  In a desktop environment a homogeneous look and feel is a desirable attribute.  However, in embedded single focus devices there is no such reason to impose arbitrary limits on the creativeness of an application’s user interface.

Next time, I’ll talk about the reality of how most creative user interface design is implemented in today’s embedded applications … think canvas‘ and label’s everywhere!

Thomas

Most embedded GUIs were not designed for use with animations. This is mainly due to the environment that they grew out of. The first GUIs in the embedded market were designed for the industrial market were functionality was their only concern. The market has changed and evolved since then and the requirement for a GUI to be only functional is a thing of  the past. Todays’ embedded GUIs are a key product differentiator, and this is not only true in the consumer market, every market is starting to see this trend including industrial. The problem with this trend is that frameworks and tools the embedded market is use to do not allow for simple concepts like an animation or a faded screen transition without a lot of custom work and extensions to the underlying framework.

There  are a few UI frameworks in the embedded market that handle these concepts natively but usually they provide you with no tooling solution and are tide tightly to a specific hardware platform or software stack. Many companies have tried turning to desktop solutions like  Adobe’s flash so they will have a tool suite to enable their UI designers and developers but soon realize that using the tool set provided makes their UI unfit for the embedded space and they are reduced to using a text editor as they edit their action scripts.

Here is a  quick look at how adding animations to your embedded UI doesn’t need to be a frustrating and tedious process when done with our Storyboard Suite.

The animation is a lot smoother than this video might show since I think I have to align my fps and the desktop capture software’s fps a little better. But I think you’ll get the idea.

Was passed along these 8 tips for designing user friendly applications the other day.  It is a quick read, and pretty good general advice … in particular about providing a consistent behaviour. 

The only item in the list that gave me a moment of pause was “Be creative, but not inventive“.  Why not be both I wonder?  How much does the risk of being too different compare to the reward of developing the next iPod type of user interface? 

Considering that the author works at SAP, and is likely targetting an audio of business tooling software developers, perhaps that is sage advice for that audience, in that context.  I think that in other markets, other user interface contexts, the advice is not as much a rule as a guideline. 

If your application is oriented at the PC desktop world, then you likely aren’t going to be revolutionizing the world with a new user interface any time soon, the bumptop desktop is a great example[1].  Non-rectangular UI elements, animated graphics, application re-skinning and task oriented UI changes is about as revolutionary as you’ll get.

Consumer devices tend to lead the way when it comes to pushing the envelope on user interface innovation.  Likely because their lifecycle is such that any success can be repeated and any failure can quickly be forgetten as ‘yet another dropped product’.  It may also be due to the fact that they are not limited (or empowered?) by the traditional input devices so new ways of connecting the user to the data they interacting with need to be created.

I’d like to see more creativity and more inventiveness in application user interfaces and we hope that that is something that Storyboard Suite is going to help our customers achieve.

Thomas   

 [1]: Don’t get me wrong, I love Bumptop.  I think that it is unique, cool and occasionally more usefull than my standard desktop.  Where it would be massively useful would be if I used a tablet interface the size of my coffee table, but alas I don’t have one of these