Frequently Asked Questions (FAQ)

General Questions
Installation/Usage Questions
Programming Questions
Licensing Questions

General Questions

What is WBI?
Where can I get more information/documentation about WBI?
Where did WBI come from?
What is transcoding?
What is the relationship between WBI and IBM Websphere Transcoding Publisher?
How can I comment on/ask questions about WBI?

Answers

What is WBI?
WBI is a programmable HTTP proxy that was designed to make it easy to develop and deploy intermediary applications on the web. The goal was to produce a programming platform that can be used to implement all manner of intermediaries, from simple server functions to complex image and page editing and distributed applications. The WBI Development Kit, a full Java implementation of WBI, is freely available from alphaWorks, and provides all the APIs and documentation to get you started writing your own intermediary-based applications.
Where can I get more information/documentation about WBI?
The most up-to-date WBI information is available from the WBI home page.
Where did WBI come from?
WBI was developed in the USER Group in the Computer Science Department at IBM Almaden Research Center. See our little story.
What is transcoding?
Transforming information from one form to another is sometimes called transcoding. Intermediaries, such as WBI, make good platforms for developing and deploying transcoding applications.
What is the relationship between WBI and IBM WebSphere Transcoding Publisher?
Plugins developed with WBI Development Kit version 4.5 will run under IBM WebSphere Transcoding Publisher (WTP) version 1.1.2 with a few exceptions. Several features new to WBI 4.5 have not yet been incorporated into WTP, though they will be available in a future release of WTP. In particular,
- the new MegObject interface
- the HttpCookie and HttpSetCookie as well as HeaderField and some methods of MimeHeaderAscii
- RequestEvent.mayReject() and RequestEvent.wontReject()
are not available in WTP 1.1.
Plugins developed for WBI DK version 4.4 will run without qualification on WTP version 1.1. But note that plugins developed with older versions of the WBI DK (e.g., version 4.1 and 4.3) may not work with WTP.
How can I comment on/ask questions about WBI?
For technical/programming questions, write to wbi-developer@almaden.ibm.com. For business/licensing questions, write to wbi-license@almaden.ibm.com.

Installation/Usage Questions

What are WBI's system requirements?
Where can I get WBI?
What is the current version of WBI?
How do I use WBI?
WBI won't start. What's wrong?
WBI's GUI won't start. What's wrong?
How do I register/deregister a WBI plugin?
How can I interact with WBI?
Which files control WBI's current configuration?
What options are available when starting WBI?
I tried to register a plugin, and it didn't work. What's wrong?
How can I check which plugins are registered and enabled?
What are these two funny plugins that show up, "ibm/AdminHttpPlugin" and "ibm/NoGZipPlugin"?
I'm using WBI as a proxy, but now images are getting garbled or don't appear. Help.
Some web pages, such as Yahoo!, show up blank, garbled, or truncated. What's wrong?
I set my proxy settings, but Internet Explorer 5 doesn't use WBI as a proxy.
It appears as if meg output appears in the wrong loation in the WBI GUI's request processing view. Does this mean WBI does not work correctly?

Answers

What are WBI's system requirements?
WBI runs on any system for which Sun's Java 2 SDK is available. It also runs under older versions of Java, including JDKs 1.1.5 and 1.1.6, but the WBI Graphical User Interface (GUI) requires Java Swing, which must be installed separately on these older JDKs. We have tested WBI under Windows 95, Windows 98, Windows NT4, AIX, and Linux. We'd like to hear of your experiences running WBI on other systems.
Where can I get WBI?
WBI is available from the IBM alphaWorks Download Site.
What is the current version of WBI?
The current WBI version is WBI DK 4.5, available from the IBM alphaWorks Download Site. Please refer to the release notes for summaries of bug fixes and changes.
How do I use WBI?
Once you have downloaded WBI from alphaWorks, you need to unzip the files and configure your system. For more information on how to do that, see the Quick Start documentation.
WBI won't start. What's wrong?
This behavior can be caused by many things. Make sure that you are in the WBI home directory and that you used the command Run. If you tried to start WBI by using java Run, then you should make sure that your classpath is set correctly. (The classpath must include the WBI jar files.) Make sure that you have Sun's Java 2 SDK (or other Java runtime system) installed on your machine.
WBI's GUI won't start. What's wrong?
The WBI Graphical User Interface (GUI) requires Java Swing, which is part of Java 2 SDK, but must be installed separately for older versions of the Sun JDK. See a list of known problems related to the WBI GUI.
How do I register/deregister a WBI plugin?
To register/deregister a WBI plugin, use the WBI console. To register a plugin, use the command register followed by the path to the plugin's .reg file. You can also register the default plugin by using the command register default. To deregister a plugin, use the command deregister followed by the complete classname of the plugin (including the package name). To deregister all plugins, use the command deregister all. You can also use the GUI to do this.
How can I interact with WBI?
While WBI is running, you can interact with it two ways. Either via a command-line interface called the WBI console, or through the Graphical User Interface. Through the WBI console, you can register/deregister plugins, enable/disable plugins, find out which port WBI is listening to, and list registered plugins and MEGs. For more information, see the documentation on the WBI console. The GUI lets you do most things the console does, as well as see the flow of requests and responses through the various plugins that are registered. See the GUI documentation for details.
Which files control WBI's current configuration?
The configuration files in etc/config/wbi/ specify which port WBI listens to for HTTP requests and details how WBI should work with firewall machines, such as Socks servers and other proxies. For more information, see the documentation on WBI's configuration files.
What options are available when starting WBI?
When you start WBI with the Run command, you can specify whether WBI should display tracing and messaging information on the console. You can also specify which directory should be used as the WBI home directory. For more information, see the documentation on Run options.
I tried to register a plugin, and it didn't work. What's wrong?
This behavior can be caused by many things. First, make sure that you supplied a valid argument to the register command. The argument should be the path to the plugin's .reg file, relative to WBI's home directory. Next, make sure that the plugin has been compiled, or there will be no plugin classes to load. Finally, you may have to adjust your classpath to tell the Java VM where to find the plugin classes. If you are a developer having trouble registering a plugin you are working on, see the documentation on writing a .reg file.
How can I check which plugins are registered and enabled?
There are two ways to do this. The first way is to go to the WBI setup page. (Once you have installed WBI, you can access the setup page at http://_wbi/setup.) Plugins (other than the default plugin) that are registered will be listed in a table. A checkbox next to each plugin's name will indicate whether the plugin is enabled. Alternatively, you can type the command plugins into the WBI console . This will list which plugins are registered and also indicate whether they are enabled.
What are these two funny plugins that show up, "ibm/AdminHttpPlugin" and "ibm/NoGZipPlugin"?
These useful plugins are registered and enabled by default. The AdminHttpPlugin lets you configure various aspects of WBI's operation by visiting http://_wbi/setup. The NoGZipPlugin solves this problem.
I'm using WBI as a proxy, but now images are getting garbled or don't appear. Help.
We've seen this with some older versions of the JDK, say before 1.1.5. We've also seen it with some versions of the Java Just-In-Time compiler (JIT). If you're using JDK 1.1.6, make sure you have the JIT patch. Or try running without the JIT (try "set JAVA_COMPILER="). Please let us know if you run into this problem: wbi-developer@almaden.ibm.com.
Some web pages, such as Yahoo!, show up blank, garbled, or truncated. What's wrong?
Some newer versions of Netscape (4.7?) or possibly other browsers include a header Accept-encoding: gzip with the request sent to the server. This may cause some servers to return a compressed ("gzipped") version of the page. If a Document Editor tries to edit this "HTML" page without realizing it was gzipped, or removes the Encoding: gzip header the server added to response, things can go wrong. The solution is to make sure the "ibm/NoGZipPlugin" plugin is registered and enabled (however, it is registered and enabled by default). If it's not registered, the .reg file is etc/NoGZipPlugin.
I set my proxy settings, but Internet Explorer 5 doesn't use WBI as a proxy.
There are many places to set the HTTP proxy in Internet Explorer 5. If you are connected via a Local Area Network (LAN), the setting is in one place, and if you are connected via a phone dialer, the setting is in another. See our notes on configuring your browser to use WBI as a proxy.
It appears as if meg output appears in the wrong loation in the WBI GUI's request processing view.
This is purely a presentation error and does not affect WBI's behavior and performance. We are investigating this problem and will make a patch available as soon as possible. Please see details on this problem in the Known Problems section of the WBI GUI.

Programming Questions

Is there a WBI programming tutorial?
What is WBI's programming model?
Is there sample plugin code that I can look at?
Will plugins developed for WBI DK version 4.1 work with the new WBI DK version 4.5?
Can WBI plugins use native code just like other Java programs?
What are some of the more useful packages and classes in WBI?
Where can I get information about the WBI API?
How does the MEG chaining work?
How do I setup a MEG?
What kind of syntax do the condition rules use? Which variables do they test?
How can I debug a WBI plugin?
What kind of information is available (through WBI) from an HTTP request/response?
What is the convention for saving files created by a WBI plugin?
The MegInputStream is empty. What went wrong?
Does WBI work with databases?
Does WBI support multi-threading?
Does WBI support SSL?
My Meg keeps throwing a NotCharDataException. What's wrong?
When my Meg examines a form response, sometimes the parameters all look like "???". What's wrong?
I want to write a WBI plugin that adds applets to web pages, and I want these applets to communicate with my plugin. When I try to open a connection from my applet to the WBI proxy I get a security exception. What do I do?

Answers

Is there a WBI programming tutorial?
Yes. The best way to learn how to program with WBI is by reading the Programming section, which also contains links to sample plugins.
What is WBI's programming model?
A WBI "application" is called a plugin. A plugin consists of many modules, which are called MEGs. There are four different types of MEGs: Monitors, Editors (Request Editors and Document Editors), and Generators. Editors edit requests and responses, and Generators generate responses based on specific requests. Monitors examine requests and responses but do not change them. The MEGs in a plugin work together to achieve a particular functionality. For more information on WBI's programming model and data model, see the Architecture section.
Is there sample plugin code that I can look at?
Yes. The WBI DK includes several sample plugins. Each of these comes complete with source code, class files, and documentation. The source code and class files are located in the com/ibm/wbi/examples directory. The documentation is located in the doc/examples directory and can be reached from the Programming section.
Will plugins developed for WBI DK version 4.1 work with the new WBI DK version 4.5?
No. Plugins developed with the original version of the WBI Development Kit, version 4.1, cannot be used as is with the new DK download, version 4.5 (or the previous versions 4.4 and 4.3). However, it takes only a few minutes to port your plugins from 4.1 to 4.3 if you follow our notes on porting. Little porting should be needed to move from 4.3 to 4.4 or 4.4 to 4.5 --- only if you used the old Database API, which has been removed in favor of the new Persistence classes.
Can WBI plugins use native code like other Java programs?
Yes. Because WBI is written in Java, WBI plugins can use native code through the Java Native Interface (JNI). The example History plugin does this. Native code should reside in special subdirectories. We provide a method Proxy.getNativePath() that allows you to write platform independent Java code for loading native libraries. The idea is to separate native code into platform dependent subdirectories. The names of these directories reflect the operating systems and architectures that the native code was compiled for. The names can be determined by a Java program at runtime. See Proxy.getNativePath() for details.
What are some useful packages and classes in WBI?
The com.ibm.wbi.http.beans package contains many useful MEG classes that you can extend or use as is. The DocumentInfo class is helpful for getting information about HTTP requests and responses. DocumentInfo instances are obtained by calling getRequestInfo on a RequestEvent and casting the result to a DocumentInfo. When writing plugins and MEGs from scratch, you should extendHttpPlugin, HttpMonitor, HttpEditor, HttpRequestEditor, and HttpGenerator.
Where can I get information about the WBI API?
See the API overview section, which describes the packages in the WBI DK and has links to all the API documentation.
How does the MEG chaining work?
Many factors determine whether WBI should route a transaction through a particular Meg, including condition rules, priority numbers, and extra rule keys. For more information on MEG chaining, see the Architecture section.
How do I setup a MEG?
Each MEG has four properties: name, condition, priority, and enabled. The name is used for tracing/messaging information; the condition specifies when WBI should route the transaction through this MEG; the priority value orders MEGs when several match a given transaction; and the enabled property turns the MEG on and off. These properties can be set individually using setName, setCondition, setPriority, and setEnabled, or they can all be set at once using setup. For more information on setting up a MEG, see the Programming section.
What kind of syntax do the condition rules use? Which variables do they test?
Condition rules are simple statements combined with boolean operators. Some of the variables which can be used in the statements include host name, path, protocol, and response code. For more information, see the API documentation for MEG's setCondition method.
How can I debug a WBI plugin?
The WBI console can display all sorts of tracing and messaging information. WBI also logs transactions in two files stored in the log directory (off the WBI home directory). Of course, any Java debugger can be used, such as Sun's JDB or IBM's Jikes Debugger. And then there is good old System.out.println(...).
What kind of information is available (through WBI) from an HTTP request/response?
A DocumentInfo instance contains HTTP request and response header information for a specific transaction. This information includes client data (IP address, name, and port used), the URL that was requested, the response code, as well as the complete headers themselves.
What is the convention for saving files created by a WBI plugin?
In general, a WBI plugin can store files wherever and however it wants. However, we prefer to put them in the data directory, off the WBI home directory, to keep things organized. Because many plugins may be storing their own data, it helps to have one folder in the data directory for each plugin. Alternaitively, if your plugin needs merely to save settings and the like, have a look at our new Persistence classes, as these should simplify your job.
The MegInputStream is empty. What went wrong?
WBI represents HTTP request/response bodies as streams. When a MEG processes a document, it reads the data from a MegInputStream and writes data to a MegOutputStream. If the data that is read from a MegInputStream is supposed to be available in subsequent MEGs (i.e., ones that follow the current MEG in the processing chain), it must be explicitly written to the MegOutputStream by the MEG. If the data are not passed along, the next MEG in the processing chain will receive an empty MegInputStream.
Does WBI work with databases?
WBI comes with a rudimentary database-like structure called an Section, which can store key-value pairs in a backend systems, such as a filesystem. For more information see the persistence tutorial.
Does WBI support multi-threading?
Yes. One WBI instance can handle requests from multiple clients, such as multiple browser windows, at the same time. However, programmers must take the necessary actions to make their own MEGs thread-safe, for instance, using the synchronized mechanism in Java or by using local variables rather than class variables.
Does WBI support SSL?
Secure Sockets Layer (SSL) is a protocol that describes how authenticated and encrypted information can be sent between clients and servers. WBI plugins cannot monitor or modify documents (either requests or responses) that are sent via an SSL connection. As a proxy, WBI tunnels SSL transactions using the HTTP CONNECT method.
My Meg keeps throwing a NotCharDataException. What's wrong?
Several things could be wrong. When your Meg calls RequestEvent.getMegReader() or RequestEvent.getMegWriter(), steps are taken to determine whether the stream is character data or not, and if it is character data, what the encoding is. This information comes from the RequestInfo associated with the RequestEvent at the time of the call. This is likely to cause a problem in several scenarios:
- If your Meg is an Editor, the Content-Type may be incorrect, either because it was incorrect to begin with, or because some other Editor changed it. You may be able to determine the correct encoding yourself through some other means; if this is the case, you can call RequestEvent.getMegReader(String encoding).
- If your Meg is a Generator, it needs to set the Content-Type itself before calling getMegWriter(). The following code in a Generator's handleRequest() will throw a NotCharDataException, because the response header's content type does not identify the stream as character data:
```
public void handleRequest(RequestEvent e) 
    throws IOException {
  HttpResponseHeader h = 
    new HttpResponseHeader();
  e.setResponseHeader(h);
  e.getMegWriter();
}
```
  but the following code solves the problem:
```
public void handleRequest(RequestEvent e) 
    throws IOException {
  HttpResponseHeader h = 
    new HttpResponseHeader();
  h.set("Content-Type", "text/html");
  e.setResponseHeader(h);
  e.getMegWriter();
}
```
- If your Meg is a RequestEditor or a Generator, be aware that most requests have neither a Content-Type header nor a body to read. POST requests with a Content-Type of application/x-www-form-urlencoded are correctly identified as character data. POST requests of type multipart/form-data are not themselves character data, but they may contain character data. Use FormHelper.interpretMultipartFormData() to deal with this kind of post. Forms that are GET forms rather than POST forms do not have any request body to read; all the information is contained in the headers.
When my Meg examines a form response, sometimes the parameters all look like "???". What's wrong?
Sometimes a request URL may contain a string that is both encoded in a multibyte encoding and "UrlEncoded" (see RFC 2396, section 2.4 for a brief discussion of the escape syntax used). E.g., the URL "http://www.xyz.cn/cgi/search?key=%D0%C2%CE%C5" contains a search term ("%D0%C2%CE%C5") that is really some Chinese characters encoded into eight-bit bytes using the "GBK" encoding and then encoded into "%XX" sequences using "UrlEncoding". Unfortunately, there is no way to know that the GBK encoding was used; you just have to guess (see below). However, assuming you guess correctly, the following code should allow you to extract a String with the correct Unicode characters (of course, you should substitute the encoding actually used for "GBK"):
```
 
public void handleRequest(RequestEvent e) {
  Properties p = 
    FormHelper.interpretFormData(e); 
  // FormHelper has already automatically 
  // performed the step of UrlDecoding 
  String word8859_1 = 
    p.getProperty("_searchkey"); 
  byte wordBytes[] = 
    word8859_1.getBytes("iso-8859-1"); 
  String realWord = 
    new String(wordBytes, "GBK");
}
```
Why do you have to guess at the encoding used? To quote from RFC 2396, section 2.1:
For original character sequences that contain non-ASCII characters, however, the situation is more difficult. Internet protocols that transmit octet sequences intended to represent character sequences are expected to provide some way of identifying the charset used, if there might be more than one [RFC2277]. However, there is currently no provision within the generic URI syntax to accomplish this identification. An individual URI scheme may require a single charset, define a default charset, or provide a way to indicate the charset used.
It is expected that a systematic treatment of character encoding within URI will be developed as a future modification of this specification.
Since no such scheme has been developed for http, you might think using UTF-8 would make sense, especially since that is encouraged by RFC 2277, but that is often not the case in current practice. Rather, the server just uses an encoding that makes sense for the data it handles. E.g., a server in China might use GBK encoding, while a server in France might use ISO-8859-1.
I want to write a WBI plugin that adds applets to web pages, and I want these applets to communicate with my plugin. When I try to open a connection from my applet to the WBI proxy I get a security exception. What do I do?
For security reasons, the java implementation in browsers is restricted to make connections only to the host from which the code for the applet was served. To do its job, the security system must be able to tell that the applet was served from a particular IP address so that it can allow the applet to connect back only to that same address. Because the security system in the browser does not know that the WBI proxy might be intercepting and handling requests, it cannot treat applets served by WBI as coming from a a known IP address (afterall, WBI might have intercepted a request for a URL such as http://_wbi/myapplet.class and returned the class file), but the security system cannot resolve the IP address for the host _wbi through DNS. Thus, the security cannot allow the applt to connect to any host in this case.

The solution is essentially to trick the browser into thinking the applet was loaded from a some host on the internet that the security system can resolve through DNS. One approach is to use the CODEBASE parameter of the APPLET tag to tell the browser to load from some arbitrary URL with a DNS-resolvable hostname (e.g., http://www.ibm.com/_wbihack/classes/), and write a generator that looks for that URL ("host=www.ibm.com & path=/_wbihack/classes/") and returns the java class files. The applet can then open a URLCnnection to http://www.ibm.com/_wbihack/handler/, and your WBI plugin can intercept that connection as well and generate the appropriate response. You must use a URLConnection ratehr than a regular socket connection because the java will use the browser's proxy setting in that case rather than simply using the HTTP CONNECT method, which would actually send the request to www.ibm.com.

Licensing Questions

What kind of licensing restrictions does WBI have?
Is WBI supported?
May I have a copy of the WBI source code?
I am a university researcher. May I use WBI to support my research project?
I program the web for fun. May I use WBI for my next project?
I am starting a new company. May I use WBI as part of our product?
May I use WBI to build an intranet application that we will use within our company?

Answers

What kind of licensing restrictions does WBI have?
WBI is being made available on alphaWorks so that anyone can download it, use it for personal use, and develop their own WBI plugins. Use the WBI software at your risk. It can be used to develop your own WBI plugins, which can be distributed under whatever terms you specify. However, the license does not allow you to distribute WBI with your plugin. You can do one of two things:
(1) ask users to get their own copy of WBI from alphaWorks
(2) send email to wbi-license@almaden.ibm.com to find out about licensing an IBM product that is based on WBI and is plug-in compatible
For more information, see the License section.
Is WBI supported?
Technically, no. We cannot guarantee bug fixes or enhancements at this time. We do intend to continue improving and enhancing WBI, so we are very interested in your feedback. If you have questions, problems, or suggestions, please send email to wbi-developer@almaden.ibm.com. We cannot guarantee a satisfactory response, but we want WBI to useful to you! If your WBI application requires support, please send email to wbi-license@almaden.ibm.com to inquire about productized versions of the WBI Development Kit and WBI server.
May I have a copy of the WBI source code?
No. WBI is only being distributed in binary (class) form. We are considering an open-source distribution of WBI. If you think this is a good idea and would like us to do so, please send email to wbi-license@almaden.ibm.com.
I am a university researcher. May I use WBI to support my research project?
You may use WBI for your research, building innovative plugins that demonstrate your ideas. You may setup a WBI server that others can access to use your plugin. You may do this for research and non-commercial purposes only. If you wish to use WBI commercially, please send email to wbi-license@almaden.ibm.com.
I program the web for fun. May I use WBI for my next project?
You may build WBI plugins for your own personal use. You may also distribute plugins that you have built to your friends, or even sell them. However, you are not permitted to re-distribute WBI itself. Therefore, for others to use your plugin, they must download their own copy of WBI from alphaWorks.
I am starting a new company. May I use WBI as part of our product?
The current distribution is for non-commercial use only. We are very interested in enabling new companies that use WBI intermediary technology. Please send inquiries to wbi-license@almaden.ibm.com.
May I use WBI to build an intranet application that we will use within our company?
The current distribution is for non-commercial use only. An intranet application for a commercial enterprise is not allowed under this license. We are very interested in enabling new intranet applications with the WBI intermediary technology. Please send inquiries to wbi-license@almaden.ibm.com.