How to integrate a custom cover flow Flash widget in IT Mill Toolkit

We've seen different kinds of cover flows all around us, they are in our music players and even in some search engines on the web. It is a relatively new and fancy way of redesigning the user interfaces. This article will demonstrate how to implement a cover flow as a custom widget with IT Mill Toolkit. Below we can see a screenshot of our cover flow component.

What do we exactly want? We want to create a selection tool with a nice graphical user interface which can be used for example in an e-commerce site selling DVD-movies. In stead of having a list of movie titles, why not let the user browse through the covers? In our example application we've created a simple slide show viewer, where one can select the slide to be shown via our cover flow component.

The back-end uses IT Mill Toolkit 5.1.1 and for the Flash component we've decided to use Adobe Flex which is a nice open source framework for creating Flash movies. In the toolkit's manual we've seen an example on how to create a custom color picker GWT widget. Creating a custom Flash GWT widget is a bit different than creating any other component. The basics are the same, but now we have an additional problem – the communication between the Flash and GWT. This article will concentrate on demonstrating how Flash can be used in creation of IT Mill Toolkit user interface-components and how the communication problem between Flash and GWT has been solved. Even though this article covers some parts of the Flex development, this is not an article on developing application with Flex. The Flex source code of the cover flow is however included under the Apache License, Version 2.0.

The entire example can be found under the toolkit's demos-folder. To try it out for yourself you need to follow the steps mentioned in IT Mill Toolkit's manual chapter 8.7 “Creating a Widget Project in Eclipse”.

The server end

The server end of our component is pretty simple. Before we take a look at the code, we should first consider the functional requirements of our component. The user should be able to add images given as Resources from server-side. The component should be able to communicate with the server and notify which cover is currently being shown (which is selected). This sounds a lot like a select component, actually, a cover flow is a select component. We take advantage of the already existing components and build our cover flow so that it inherits the methods of AbstractSelect. This way we don't need to create all the features from scratch. The AbstractSelect doesn't contain all functions we need for our component as we want to be able to enable/disable the scrollbar in our cover flow and to change the Flash movie's background – both features must support run-time changes. For these two features we need to create custom methods.

Below is a rough sketch of the design we've used.

Now that we know what we want, we can take a look at the code. The server-side class contains only four methods:

• getTag() returns the unique UIDL tag name for this component
• setBackgroundColor() allows the user to define a background color for the cover flow. The function takes as input two RGB values, the start and end colors for the background gradient
• setScrollbarVisibility() allows users to define the visibility of the scrollbar
• paintContent() overrides the parent's method. The default select component's UIDL isn't enough, since we have three additional values (scrollbar visibility and background gradients) which needs to be passed. We add these variables as main tag attributes.

package com.itmill.incubator.ui.coverflow;

import com.itmill.toolkit.terminal.PaintException;
import com.itmill.toolkit.terminal.PaintTarget;
import com.itmill.toolkit.ui.AbstractSelect;

public class Coverflow extends AbstractSelect {

        private String backgroundGradientStart = "FFFFFF";
        private String backgroundGradientEnd = "EEEEEE";
        private boolean scrollbarVisibility = true;
        
    public String getTag() {
            return "cover";
    }


            
    /**
     * Paints the uidl
     * @param PaintTarget target
     * @throws PaintException
     */
    public void paintContent(PaintTarget target) throws PaintException {
        // Superclass writes any common attributes in the paint target.
        super.paintContent(target);
        
        target.addAttribute("backgroundGradientStart", backgroundGradientStart);
        target.addAttribute("backgroundGradientEnd", backgroundGradientEnd);
        target.addAttribute("scrollbarVisibility", scrollbarVisibility);
    }
    
    /**
     * The user can specify a background gradient for the coverflow. The input values
     * are RGB values for the start and end gradients.
     * @param int startR
     * @param int startG
     * @param int startB
     * @param int endR
     * @param int endG
     * @param int endB
     */
    public void setBackgroundColor(int startR, int startG, int startB, int endR, int endG, int endB) {
        backgroundGradientStart = "";
        backgroundGradientEnd = "";
        
        // Convert all integers to hexadecimal format and make sure they are two characters long (in
        // other words, add a zero infront if the value is less than 16 => 0x0F
        if(startR < 16)
                backgroundGradientStart += "0";         
        backgroundGradientStart = Integer.toHexString(Math.max(Math.min(startR,255),0));
        
        if(startG < 16)
                backgroundGradientStart += "0";
        backgroundGradientStart += Integer.toHexString(Math.max(Math.min(startG,255),0));
        
        if(startB < 16)
                backgroundGradientStart += "0";
        backgroundGradientStart += Integer.toHexString(Math.max(Math.min(startB,255),0));
        
        if(endR < 16)
                backgroundGradientEnd += "0"; 
        backgroundGradientEnd = Integer.toHexString(Math.max(Math.min(endR,255),0));
        
        if(endG < 16)
                backgroundGradientEnd += "0";
        backgroundGradientEnd += Integer.toHexString(Math.max(Math.min(endG,255),0));
        
        if(endB < 16)
                backgroundGradientEnd += "0";
        backgroundGradientEnd += Integer.toHexString(Math.max(Math.min(endB,255),0));
        
        this.requestRepaint();
        
    }
    
    /**
     * The user can toggle the visibility of the scrollbar
     * @param boolean visible
     */
    public void setScrollbarVisibility(boolean visible) {
        scrollbarVisibility = visible;
    }
        
}

Integrating the GWT widget

The next step is to create a custom GWT widget and integrate it with IT Mill Toolkit. In this article we won't discuss the finer details of the integration process as it is describe in the Toolkit's manual. What we do however discuss, is what special precautions we have to do when working with Flash.

The first problem is that there isn't a ready Flash component in the GWT which we could inherit. Therefore we're just going to extend the Composite class and build the Flash logic ourself. We start by creating an empty HTML object which we will later on fill with the Flash's object-tag and parameters. We add the empty HTML object with the initWidget()-method. The reason why we do not fill the object with the HTML tags in this point is that we want to wait for the updateFromUIDL()-method to be called first. The UIDL will contain a unique ID for this instance of the component, which we want to use in the object tag as the object's id-attribute. Once the updateFromUIDL()-method has been called for the first time, we will fill the HTML object with the appropriate HTML code.

   /**
     * Constructor
     */
    public ICoverflow() {
        flash = new HTML(); 
        
        initWidget(flash);        
    }

   .....

   private void setFlash() {
        String html = "<object id=\"fxcoverflow"
                + uidlId
                + "\" classid=\"clsid:d27cdb6e-ae6d-11cf-96b8-444553540000\" width=\"100%\""
                + " height=\"100%\" codebase=\"http://fpdownload.adobe.com/pub/shockwave/cabs/flash/swflash.cab#version=9,0,0,0\">"
                + "<param name=\"movie\" value=\""
                + GWT.getModuleBaseURL()
                + "coverflowflash.swf\">"
                + "<param name=\"quality\" value=\"high\">"
                + "<param name=\"flashVars\" value=\"pid="
                + uidlId
                + "&sbVis="
                + scrollbarVisibility
                + "&bgS=0x"
                + backgroundGradientStart
                + "&bgE=0x"
                + backgroundGradientEnd
                + "\" />"
                + "<embed name=\"fxcoverflow"
                + uidlId
                + "\" flashVars=\"pid="
                + uidlId
                + "&sbVis="
                + scrollbarVisibility
                + "&bgS=0x"
                + backgroundGradientStart
                + "&bgE=0x"
                + backgroundGradientEnd
                + "\" src=\""
                + GWT.getModuleBaseURL()
                + "coverflowflash.swf\" width=\"100%\" height=\"100%\" "
                + "quality=\"high\" "
                + "pluginspage=\"http://www.adobe.com/shockwave/download/index.cgi?P1_Prod_Version=ShockwaveFlash\">"
                + "</embed>" + "</object>";
        flash.setHTML(html);
    }

Note that we use “fxcoverflow” + uidlId (the unique identifier from the UIDL) as the flash object's id. We are also passing the background color and the scrollbar's visibility status as flashVars.

When developing the widget, we've used such folder structure that the GWT component is under the coverflow/gwt/ -folder. In the folder coverflow/gwt/public/ are all public files that should be accessible on the web server via a web browser, this folder includes the swf file of our Flash movie. Note that when we compile our GWT widget, all the files from the public folder will be copied to another folder, which then is accessible from the web. This folder is located under WebContent?/ITMILL/widgetsets/<package>/. This is also the path we should use in our HTML code which calls on the Flash movie. We've used the static GWT class's getModuleBaseURL()-method to dynamically create the path to the Flash movie.

Communication between GWT and Flash

At this point we should be able to launch a custom GWT Flash widget. What's left to do, is to establish a connection between Flash and GWT so that we can pass information from each component to the other. In Flex we can use the ExternalInterface to establish a communication interface between the Flash movie and JavaScript. We can call native JavaScript-functions directly from ActionScript and with JavaScript we can call on ActionScript functions. Sounds easy, well, it is. For JavaScript to be able to call on the ActionScript functions, we first need to make them available. This can be done with the ExternalInterface's addCallback-method. The method takes two input values, the first being the function name JavaScript should call on and the second is the actual function name where the call should be directed to. In our example, we need to publish four functions for JavaScript: addCover(), selectCover(), setBackgroundColor() and toggleScrollbarVisibility(). In our GWT component (ICoverflow.java) we need to make JSNI-methods (native JavaScript functions within Java) which then can call on the ActionScript functions. Below is an example of the addCover()-method which is used in the GWT component to add new covers to Flash.

public native void addCover(String id, String caption, String icon) /*-{    
    $doc["fxcoverflow" + id].addCover(caption.toString(), icon.toString());
}-*/;

This function takes three input values, the first being the uidlId which was used to create a unique identifier for our object and the two others are information on the cover. Note that even though the input values are parametrized to be strings, we're still doing a toString() conversion. This is because this method is native JavaScript code and the input parameters are actually objects, not strings. This method can then be normally called in our Java code, for example, as we've done in the getCovers()-method.

If you've looked at the updateFromUIDL()-method you probably noticed that we aren't directly calling on the addCover()-method but rather storing the covers in an ArrayList. This serves two purposes, the first being a way to detect duplicates as this is a way for us to check which covers have already been added to the cover flow. The second reason is that we simply cannot immediately add the covers. The reason is that when the updateFromUIDL()-method is called, we cannot be sure if the Flash movie has already loaded into the browser and made its functions available for us to use. Therefore we want to wait for the Flash movie to tell GWT that it is now ready to accept data and after that we can add the covers.

As mentioned earlier, Flash can directly call native JavaScript-functions with its ExternalInterface. The problem with GWT is that on compilation the source code is scrambled to an unreadable format, this includes the function names. We must know the function names in our compiled GWT component for Flash to be able to call the GWT component's function. This problem can be solved with another JSNI-method which publishes the required functions. Below is an example on the JSNI-method which published the getCovers() method for Flash.

public native void initializeMethods() /*-{
    var app = this;        
                
    $wnd.itmill.getCovers = function() {
        app.@com.itmill.incubator.ui.coverflow.gwt.client.ui.ICoverflow::getCovers()();
    };              
}-*/;

Flash can now call the native JavaScript-function getCovers(). It can be done with the ExternalInterface like in the example below.

ExternalInterface.call("itmill.getCovers");

One problem remains, what if we have several instances of the cover flow widget in our application? In a case where we have two instances of the cover flow widget, the first instance will publish the getCovers-method and the second one will override the same method. This is something we do not want to happen, as each instance (most likely) contains different data. What we need to do, is to dynamically name the functions in such a way, that both instances of the widget can publish independent methods. Instead of directly publishing a function, we'll rather publish an array object which contains a set of functions. Each function has an index in the array which is unique to each instance of the widget. The index consist of the function's name and of the widgets unique id separated with an underscore. Note that in the beginning we check if the array has already been published, after all, we do not want to override previously published functions.

public native void initializeMethods(String id) /*-{
    var app = this;
        
    if($wnd.itmill.coverflow == null)
        var coverflow = [];
    else
        var coverflow = $wnd.itmill.coverflow;
                
    coverflow['getCovers_' + id] = function() {
        app.@com.itmill.incubator.ui.coverflow.gwt.client.ui.ICoverflow::getCovers()();
    };   
        
    coverflow['setCurrent_' + id] = function(selected) {
        app.@com.itmill.incubator.ui.coverflow.gwt.client.ui.ICoverflow::setCover(Ljava/lang/String;)(selected);
    };
        
    $wnd.itmill.coverflow = coverflow;
}-*/;

The ActionScript code needs to be modified so that it matches the changed function names.

ExternalInterface.call("itmill.coverflow['getCovers_" + _pid + "']");

The pid-variable we see in this call is the same unique id we got from the UIDL. This variable is passed to the Flash movie as a flashVar upon its initialization.