My first attempt at XMPP in Java App Engine

My first attempt at XMPP in Java App Engine

One of the things I love about App Engine is that the team keeps releasing new and exciting features. With the latest SDK, Google provided us with XMPP support, so let's give that a shot. During the course of this post, I am going to build a simple XMPP-chatbot version of ELIZA. You can give it a try by pointing your Google Talk client to j-eliza@appspot.com and inviting her to a chat. Naturally, you can also download the source code and try to run it yourself :-)

Eliza, please talk to me!

XMPP is one of the first features where I get to choose between Python and Java from the very beginning. Since my last experiment (on cron) was Python based, I am going to give Java a try this time.

The first step was to create a project in Eclipse and build a very simple echo-bot (a chat bot that always repeats whatever message I send to it). The following code (full source available here) accomplishes that:

  @Override
  public void doPost(HttpServletRequest req,
       HttpServletResponse resp) throws IOException {
    
    // Parse incoming message
     XMPPService xmpp = XMPPServiceFactory.getXMPPService();
     Message msg = xmpp.parseMessage(req);
     JID jid = msg.getFromJid();
    String body = msg.getBody();
     LOG.info(jid.getId() + " --> JEliza: " + body);
     
    // Get a response from Eliza
    String response = "echo: " + body;
     LOG.info(jid.getId() + " <-- JEliza: " + response);
     
    // Send out response
    msg = new MessageBuilder()
        .withRecipientJids(jid)
         .withBody(response)
        .build();
     xmpp.sendMessage(msg);

  }

Basically, all I do is to convert the incoming HttpServletRequest object into a Message, which contains a message body (whatever the sender typed into Google Talk) and the sender's id (JID). I then take that information and create a new Message using the MessageBuilder, and send it to the XMPPService

In addition to this Java code, there is also some additional setup required. I have to bind my servlet in the web.xml under a "magic" path,

 <servlet>
    <servlet-name>xmppreceiver</servlet-name>
     <servlet-class>com.appenginefan.xmpptest.XmpptestServlet</servlet-class>
  </servlet>
  <servlet-mapping>
     <servlet-name>xmppreceiver</servlet-name>
    <url-pattern>/_ah/xmpp/message/chat/</url-pattern>
   </servlet-mapping>

and I also need to enable messaging in the appengine-web.xml.

  <inbound-services>
     <service>xmpp_message</service>
  </inbound-services>

This was pretty much explained in the introduction, so I just had to be lazy and copy-and-paste that into the right files into my project. As easy as that sounds, it still requires a certain attention to detail. That lack of attention turned what should have been a 20 minute project into a one hour debugging session:

After uploading the first version, I connected to the chatbot and entered Hello World. Unfortunately, Eliza would not answer. Had I forgotten something? I re-read the article again, checked my configuration: everything was fine. I tried to fiddle with the settings, switched the path in my web.xml to wildcards -- alas, Eliza remained silent. In my despair, I pinged a couple of fellow App Engine enthusiasts. They also checked my files but did not find any problems. Had I run into a bug?

The next morning, I checked my inbox and found that somebody had tracked it down: xmpp servlets expect posts! While the introduction's sample code clearly uses doPost, the Eclipse template I had filled in used the doGet method. Had I started with an empty servlet and just followed the documentation, I would have never run into that problem. Oops :-(

Switching from doGet to doPost immediately did the trick. Whatever I chatted to my bot would immediately be chatted back. Eliza was finally talking to me.

Eliza, why don't you remember me?

Next step was to make Eliza "intelligent". A quick Google search for "Eliza" and "Java" pointed me to a simple open source version. Without going too much into detail, the API of the class looked something like this (visibility of inner fields changed by me):

public class ElizaParse {
  public String lastline;
   public boolean exit;
  public void PRINT(String s) {
     ...
  }
  public void handleLine(String s) {
     ...
  }
}

When a new eliza parser was instantiated, it would print a quick greeting (HI! I'M ELIZA...) and then wait for input. Using handleLine, a new line of input could be fed to the bot. The bot would then store that line in lastline and return an answer using PRINT. If the user wanted to end the conversation (by typing in something like "shut up"), the bot would set the exit field to true.

To make the parser work for our bot, we would need to simply feed it the incoming message and retrieve the answer by overwriting the PRINT method. The following code does that (full source is here):

  @Override
  public void doPost(HttpServletRequest req,
       HttpServletResponse resp) throws IOException {
    
    // Parse incoming message
     / ...
    
    // Get a response from Eliza
    final StringBuilder response = new StringBuilder();
     final ElizaParse parser = new ElizaParse() {
       
      @Override
      public void PRINT(String s) {
         
        // Skip the annoying intro
        if (s.startsWith("HI! I'M ELIZA")) {
           return;
        }
        
        // Write all output into a StringBuffer
        response.append(s);
         response.append('\n');
      }
    };
     parser.handleLine(body);
    body = response.toString();
     LOG.info(jid.getId() + " <-- JEliza: " + body);
     
    // Send out response
    // ...
  }

I uploaded the code and gave it a try. It worked pretty well, but wasn't perfect. Eliza has a little feature that detects when I type in the same line twice and prints "PLEASE DON'T REPEAT YOURSELF!". Also, while I was currently cutting out the greeting all the time, I did want it to appear at the beginning of a conversation.

Both issues came back to the fact that I was throwing away my parser at the end of the request. Eliza had no recollection of the last thing I had typed in, so it could not know if the conversation was new or whether I was repeating myself. In other words, I needed some persistence.

Making Eliza less forgetful

Storing the last line of a converstion could be done in many ways. I could just dump it in memcache, since I only need to remember it temporarily. Or I could create a JDO model class and use the datastore persistence, but that's a lot of code.

In the end, I chose something in the middle. Using the lower-level datastore APIs, I directly wrote entities into the store (using the chat's JID as key). The following code checks the datastore to see if we had a previous conversation:

    Key key = KeyFactory.createKey("chatData", ":"
         + jid.getId());
    Entity lastLineEntity = null;
     try {
      lastLineEntity = 
          DatastoreServiceFactory.getDatastoreService()
               .get(key);
                  
    } catch (EntityNotFoundException e) {
       lastLineEntity = new Entity(key);
      lastLineEntity.setProperty(LINE, "");
     }
    final String lastLine =
        (String) lastLineEntity.getProperty(LINE);
         
    // ...
    
    parser.lastline = lastLine;
 

At the end of my request, I can extract the new last line of conversation and write that back into the store (full source is here):

  if (parser.exit) {
      lastLineEntity.setProperty(LINE, "");
     } else {
      lastLineEntity.setProperty(LINE, parser.lastline);
     }
  DatastoreServiceFactory.getDatastoreService().put(
         lastLineEntity);

Final thoughts

Not counting some stupidity on my part during setup, using XMPPP in App Engine was super-simple to do. The API could not have been much easier, and I am very excited to see what interesting things people are going to do with it.

Posted by The App Engine Fan at 7:00 PM