Capturing and Visualizing Protocol Buffer Communication

Overview

As explained in a previous post, the Brinsop Voxtron modules use Protocol Buffer messages to communicate with each other. Since these messages use a binary format, they are not human-readable.

When modules receive or send messages, they automatically convert them to a human-readable text format (called Short Debug String) before writing them to the log file.

When viewing packets using a capturing tool such as Wireshark or RawCap, you only see the raw binary representation of the packages, and you need an extra tool in order to read the contents.

Afterwards it is also possible to create a graphical representation.

In this post we’ll create such an example trace.

Capturing Messages

It’s not possible to capture network traffic on the loopback device on Windows using Wireshark. Therefore we’re going to use RawCap, which can be found here.

After starting rawcap, we have to enter two settings:

  • Interface: 127.0.0.1
  • Output file: the default filename is fine (i.e. dumpfile.pcap)

Once RawCap is running and capturing network packets, we can test the Voxtron software by logging in, making test calls, etc…

Stopping RawCap

When the tests are done, we can stop RawCap. To do this, click on the RawCap window and hit CTRL + C.

RawCap will now have written all captured packets to the .pcap file.

Parsing the .pcap file

The .pcap file contains the binary Protocol Buffer message, so it is not readable yet. In order to create a readable representation of these messages, we’ll have to use the Voxtron pcap2txt tool, which can be found here: pcap2txt

Put the pcap2txt.exe in the same directory as the .pcap file and run it. For every .pcap file found, it will generate a new .txt file with the same name (e.g. dumpfile.pcap.txt).

Each line in that file contains the following elements:

  • Packet index in .pcap file, between brackets (e.g. [150])
  • Timestamp of capture (e.g. 12:17:18.213)
  • Sender and receiver (e.g. 127.0.0.1:46000 -> 127.0.0.1:49242)
  • Message number, between parentheses (e.g. (7655))
  • Protocol name (e.g. MONITORING)
  • Message parse check (VALID/INVALID)
  • Message Short Debug Script (same as in log files)

PcapTxt

 Mapping Ports to Modules

The packets only contain the IP address and port number of the sender and receiver (since all services were running on the same machine, the IP address is always 127.0.0.1).

If we want to know which module sent and received each packet, we need to map the port numbers to the modules. This can be done by using the following script: ListConnections When you run this script, it will generate a Connections.txt file, which describes which process was using which ports.

Visualizing Packets

Once we have the .pcap.txt and the Connections.txt file, we can generate a sequence diagram of the flow of messages between the modules.

For this we need:

  • The PbTraceFormatter.pl script: PbTraceFormatter
  • A perl interpreter (can be found here)
  • Mscgen, a sequence diagram generator (can be found here)
  • Graphviz, a general diagram drawing tool (can be found here)

After all this is installed, you can just run the PbTraceFormatter.pl script in the same directory as the .pcap.txt and Connections.txt files.

Afterwards you will get a .html file with a graphical representation of the messages.

If you open the .html file in a browser, you’ll see the following graphical representations:

Network Diagram

The network diagram at the top of the page shows all the modules for which packets have been captured. An arrow is drawn from one module to another, if at least one message has been sent from that module to the other module. Each module type is also shown in its own color, the arrows are shown in the color of the sending module.

Using this diagram it is easy to find out if some modules is not communicating with other modules. This can be caused by a problem in the configuration.

Output-1
Message Sequence Diagram

The message sequence diagram contains a vertical line for every module. Each message is shown as an arrow going from one module to another. The message type and packet number of the message are shown above each arrow.

The vertical lines have the same color as the corresponding module in the network diagram. The arrows are colored in the color of the sender.

If you want more information about a message, you can look it up in the messages table by looking for the corresponding message number. In most browsers you can just hit CTRL + F and type the message number between parentheses. (e.g. (388)).

Output-1B

 Message Table

The message table contains detailed information for each message.

It contains the following columns:

  • Message number, the background indicates VALID (green) or INVALID (red)
  • Timestamp of capture
  • Sender (background has same color as module in diagrams)
  • Receiver (background has same color as module in diagrams)
  • Message type
  • Message contents (each element on a single line)

Output-2

 Movie

Here you can see a screen recording of the complete process (capturing, testing, parsing and viewing the visualizations):

  • Run Time 1:06
  • Dimensions 1,600 × 900
  • File Name recording.mp4
  • File Size 1.88 MB
  • File Type MP4
  • Mime Type video/quicktime

Leave a Reply