2012-04-19

java

How To Use Cubeia Styx

The other day we release Styx to the public. It has served us well over the years as the main protocol generator for Firebase, so we thought we should be a bit more transparent with it. Speaking of “it”, here’s what “it” is:

  • A protocol format specified in an XML file.
  • Binary and JSON packaging for the above.
  • Automatic API generation in Java, C++, Flash, HTML5, etc…

This might still be a bit abstract, so in this post we’ll instead show you how to use it for yourself.

But first we’ll answer the universal question, “why?” A product like Firebase needs to talk to clients in different languages. For applications where latency isn’t a problem you can make do with text based protocols across the board but we needed a binary format as well, and that’s where it gets hairy.

By utilizing a code generation tool such as Styx you abstract away a part of your protocol, and for large projects this is a benefit in itself. Your protocol is a separate project which can be depended on by various components making your overall build more flexible.

Here’s the approximate steps in this tutorial:

  1. Create a maven project for the protocol. The Styx code generator is Maven only at the moment.
  2. Create a small protocol definition for our game. This is where we’ll define all protocol objects.
  3. Use Maven to generate protocol objects in Java and Flash.
  4. Use the above protocol objects in our games.

So, first we’ll create a Maven project for our protocol. Fortunately, this is pretty simple as there’s already a Maven archetype done. So we can cheat by executing this:

mvn archetype:generate 
  -DarchetypeGroupId=com.cubeia.firebase.tools 
  -DarchetypeArtifactId=firebase-protocol-archetype 
  -DarchetypeVersion=1.8.0

If you’ve done this before you recognize the pattern. You’ll be asked for a package name, a group and artifact ID. If in doubt, just use “test” for this tutorial. The project will compile the Java code and package in a JAR file, which is why you need this data.

Now we have a new project so go ahead, change directory to it’s root and compile it:

mvn clean package

If everything works out OK you have now created protocol objects in Flash, Java and javaScript. Have a look in this folder for the result:

target/jruby-protocol-plugin/generated-sources/

Now, the protocol isn’t very exciting yet. So let’s add some real protocol objects. You do this by editing your protocol definition file, which you can find here:

src/main/resources/protocol.xml

Open the above file in an editor. You’ll have some examples out-commented, feel free to play around with them, but we’ll do something else. For the sake of this tutorial we’ll add a deck of cards, and an action for dealing cards to a player. First, two enumerations for the card values:

<enum name="suit">
  <value>CLUBS</value>
  <value>DIAMONDS</value>
  <value>HEARTS</value>
  <value>SPADES</value>
 </enum>

 <enum name="rank">
  <value>TWO</value>
  <value>THREE</value>
  <value>FOUR</value>
  <value>FIVE</value>
  <value>SIX</value>
  <value>SEVEN</value>
  <value>EIGHT</value>
  <value>NINE</value>
  <value>TEN</value>
  <value>JACK</value>
  <value>QUEEN</value>
  <value>KING</value>
  <value>ACE</value>
</enum>

You’ll recognize these as the ordinary values for a standard deck of playing cards. Now let’s add an actual card by combining the two:

<struct name="card">
  <var name="suit" type="suit"/>
  <var name="rank" type="rank"/>
</struct>

Not too complex, eh? For both enums and struct, the “name” attribute on the main element will be used when the protocol objects are created, usually camel-cased. The “var” elements on the struct defines single references to either other structs, to the built-in types, or to enums. Now let’s do the “deal card” action:

<struct name="dealt_card_action">
  <list name="cards" type="card"/>
  <var name="playerId" type="int32"/>
</struct>

Here’s a new element, “list”: this will be treated differently depending on language but over the wire it will be treated as an array. We’ve included an INT field for the player ID to indicate what player received the cards.

If you haven’t, compile again, and this time you’ll notice that you’ve gotten a variety of objects matching what you defined above. For example, these are the produced JavaScript files:

Card.js
DealtCardAction.js
ProtocolObjectFactory.js
RandEnum.js
SuitEnum.js

That’s more or less what we expected. Except… “ProtocolObjectFactory”? This is a small file which given incoming data, knows how to manufacture protocol objects. So when we’ve gotten this far, let’s use it in Java. First off, you need to depend on your new protocol, but also on Styx itself where you want to use the protocol. In my test game, the dependencies looks like this:

<!-- This is the protocol -->
<dependency>
  <groupId>test</groupId>
  <artifactId>test</artifactId>
  <version>1.0-SNAPSHOT</version>
</dependency>
<!-- And we need the Styx library as well -->
<dependency>
  <groupId>com.cubeia.firebase</groupId>
  <artifactId>styx</artifactId>
  <version>1.7.0</version>
</dependency>

In this example we’ll use the binary Styx packaging, and read a protocol object from a byte array:

byte[] rawData = // this is the incoming action
StyxSerializer serializer = new StyxSerializer(new ProtocolObjectFactory());
DealCardAction action = (DealCardAction) serializer.unpack(ByteBuffer.wrap(rawData));

Now you have the object as a POJO and can go on and execute. The cast is becasue the “unpack” isn’t polymorphic. If you have many protocol objects, which you probably will have, you can either use the built in visitor pattern, or use “instanceof” to switch between them.

To go the other way, from protocol object to bytes is suitable simple:

DealCardAction action = // this is our action
StyxSerializer serializer = new StyxSerializer(new ProtocolObjectFactory());
byte[] rawData = serializer.pack(action).array();

Nice and symmetrical. Now if you want to use JSON instead of the binary format all you need to to is to use the “StyxJsonSerializer” instead and change the byte arrays for Strings, but I’ll leave that as an exercise.

Now, the code on the client side will vary a bit depending on language. But you won’t have the nice Styx serializer ready-made. But fortunately it is not difficult to do yourself. Here’s how packing and unpacking the binary protocol objects look in Flash:

public function unpack(inBuffer:ByteArray):ProtocolObject  {
  var payloadLength:int = inBuffer.readInt();
  // Styx by default uses length exclusive from the length header
  if( inBuffer.bytesAvailable < payloadLength-4) return null;
  var classId:int = inBuffer.readUnsignedByte();
  var po:ProtocolObject = factory.create(classId);
  po.load(inBuffer);
  return po;
}

public function pack(protocolObject:ProtocolObject):ByteArray {
  var packed:ByteArray = protocolObject.save();
  var buf:ByteArray = new ByteArray();
  buf.writeInt(1+4+packed.length);
  buf.writeByte(protocolObject.classId());
  packed.position = 0;
  buf.writeBytes(packed);
  buf.position = 0;
  return buf;
 }

Obviously, “factory” above is the created protocol object factory.

That’s it. This is how Firebase rolls. And maybe it’s how you’re going to roll soon as well. Enjoy!