Liras en la red: Developing a Bro analyzer as a plugin

Bro is an open source Unix based network monitoring framework that can be used mainly for collecting network measurements, and based on that conducting forensic investigations, traffic baselining, build a NIDSand many other tasks. It includes an event-driven programming language that allow to implement actions when an specific packet, message or event has happened.

The main building block of bro are the protocol analyzers and many protocol analyzer have been implemented and are included with the bro's core. However there are many other that are not implemented, and , in many cases, a company could be interested in implementing an analyzer for a proprietary protocol.

That was my task during the last month. My company had to implement a monitoring tool on a proprietary protocol and a plugin had to be developed to allow operators include this protocol into their bro installations.

This lack of information found has drive me to write this entry and to explain what I did , what problems I found how did manage to bypass them.

The main problems I have found in bro are the variety of different technologies necessary to develop an analyzer and the LACK of documentation. The last one is mainly the reason why I have written this post.

In my case I had a propietary protocol, that worked over TCP on port 302. The protocol message structure varies depending on the first byte (the function code) value. In this post I will only explain two different request messages, although the protocol had more than 20 different function codes:

Message 1

Function Code: 0x22
Message structure:
        CRC:                    32 bits;
        Value_Number:           8 bits;
        Var_Type_Code:          16 bits;
        u:                      16 bits; 
        u2:                     8 bits;
        Codified_Var_Numbers:   16 bits; 
        data:                   ;

Message 2

Function Code: 0x31
Message structure:
        id:                 16 bits;

Starting

As this was a proprietary protocol there was no other protocol I could base on. So I had to start the development from scratch. After reading the bro's documentation, mainly:

[1] - Adding analyzers
[2] - Writing bro plugins
[3] - Bro BinPAC Sample Analyzer
[4] - BinPAC
[5] - Extending Bro with Builtin Functions (BiF)
[6] - binpac: A yacc for Writing Application Protocol Parsers

...I got buffled by the amount of technologies to use.

Mainly, I had to write an anlyzer, which pretty well explained in the documentation, and I had to write a plugin, which is also well explained... if it's a simple plugin, but if you want to put it all together, there's no guide to follow.

To write a simple analyzer the easies way is to follow the instructions in [3], running the start.py script. It will basically create the following files for you:

CMakeLists.txt
events.bif
Plugin.cc (if you use the -plugin option which you need to)
MyProt.pac
MyProt-protocol.pac
MyProt-analyzer.pac
MyProt.cc
MyProt.h

That's OK, but it does not explain how to modify them to develop your analyzer. And the other hand and VERY IMPORTANT, it does NOT include the configure and MakeFile files.

Secondly, we could follow the instructions given in [2] and using the init-plugin script, it would create the following files and directories:

configure
Makefile
src directory
src/Plugin.cc
src/Plugin.
MyProt_plugin.bif
scripts directory
tests directory

Good enough, it includes a configure and Makefile files BUT, there's no explanation on how to modify these files to create an analyzer.

What did I do?

Join together all the files created by both scripts.

These are the steps followed to get a working framework:

Install bro with it's sources. The process is explained in "Installing bro". I would NOT have another bro installation in the box to avoid conflicts between them.
Create a work directory. In my case it was created in /tools/bro/bro/MyProt_plugin, while bro was installed in /tools/bro/bro
From this directory, run the start.py script with the -plugin option. Follow the instructions given in [3].
Run the init-plugin script, which is in /aux/bro-aux/plugin-support/init-plugin. The arguments a gave where "MYPROT MYPROT"
In my case I moved to the src directory the MyProt.cc and MyProt.h files generated by start.py. However this step is not necesary if later you do not modify some lines in other file.

The different technologies used

To continue with the explanation I will explain the different technologies involved in the analyzer development, which are C++, BIF, BinPAC and Bro.

At the end of the day, the analyzer must be compiled with a C++ compiler and built with Make. But to create these .cc and .h files, the bro guys use binpac and later use BIF. Finally to use the events generated using this analyzer, bro scripts must be written.

BinPac is a yacc parser that can be used to develop application protocol parsers. Its syntax is based on types and subtypes and I recommend to read the documents [4] and [6] to understand how to works. What BinPAC does is mainly, generate a .cc and .h files that when compiled and run, will be able to understand the protocol given.
BIF is an acronym of BuiltIn Functions and is a new layer that allows to interact the C/C++ event engine with the PAC Files. It has its own syntax and for an anlyzaer, it's used in the analyzer.pac file and the events.bif and types.bif files
The BRO scripting language allow us to take advantage of the bro framework and event infrastructure to write bro policies.
Finally BRO framework is written in C/C++ and while writing an analyzer there are parts that have to be written directly in C/C++.
One of the initial scripts is written in Python. This script (start.py) allow us to create the structure of C/C++ files for developing an anlyzer
The other script (init-plugin) is written in Bash Shell and will create the file structure for developing a plugin. Other Shell script was eventually written to automate the compilation process.

The files

The start.py script generated (among many other) two important files: MyProt.cc and MyProt.h. These are the starting point of an analyzer and its structure it always the same:

#include "MyProt.h"
#include "analyzer/protocol/tcp/TCP_Reassembler.h"
#include "events.bif.h"

using namespace analyzer::myprot;

MyProtTCP_Analyzer::MyProtTCP_Analyzer(Connection* c): TCP_ApplicationAnalyzer("MYPROT1234", c)
  {
   interp = new binpac::MyProtTCP::MyProtTCP_Conn(this);
  }

MyProtTCP_Analyzer::~MyProtTCP_Analyzer()
  {
   delete interp;
  }

void MyProtTCP_Analyzer::Done()
 {
         TCP_ApplicationAnalyzer::Done();
   interp->FlowEOF(true);
  interp->FlowEOF(false);
 }

void MyProtTCP_Analyzer::DeliverStream(int len, const u_char* data, bool orig)
 {
  TCP_ApplicationAnalyzer::DeliverStream(len, data, orig);
  interp->NewData(orig, data, data + len);
 }

void MyProtTCP_Analyzer::Undelivered(uint64 seq, int len, bool orig)
 {
  TCP_ApplicationAnalyzer::Undelivered(seq, len, orig);
  interp->NewGap(orig, len);
 }

void MyProtTCP_Analyzer::EndpointEOF(bool is_orig)
 {
  TCP_ApplicationAnalyzer::EndpointEOF(is_orig);
  interp->FlowEOF(is_orig);
 }

This file is the joining point between bro and the new Analyzer. With this code, the analyzer can be registered in bro, Connections will be created and allow the messages to be delivered to the Analyzer.

On the other hand we find the Plugin.cc and Plugin.h files which allow bro to create a plugin:

    #include "Plugin.h"

    namespace plugin {
    namespace MYPROT_MYPROT {

    Plugin plugin; } }

    using namespace plugin::MYPROT_MYPROT;

    plugin::Configuration Plugin::Configure()                                                         
        {
        plugin::Configuration config;
        config.name = "MYPROT::MYPROT";
        config.description = "MY protocol";
        config.version.major = 0;
        config.version.minor = 1;
        return config;
        }

It's a simple code that allow adding BIF functions into bif files and implementing them as a plugin. But this is not our case. We want an analyzer as a plugin. To get it we'll have to add the following to lines to this file. An include file will be added to the top of the file:

    #include "MyProt.h"

...and we'll add an analyzer component to the Plugin with the following line INSIDE the configure method:

AddComponent(new ::analyzer::Component("MYPROT1234", ::analyzer::myprot::MyProtTCP_Analyzer::Instantiate));

The "MYPROT1234" text is the Tag that allows bro to recognize the analyzer. It's also defined in the analyzer definiticion in MyProt.cc (see above).

For a Bro analyzer three different PAC files are generally used (MyProt.pac, MyProt-protocol.pac and MyProt-analyzer.pac). However this is not really necessary and everything could go in a single PAC File, but to make things easier I will explain it splitted in three files.

The main one (MyProt.pac) is always very similar (take a look at the comments added by me). It allows to declare what a Connection is, what a PDU is and call the other two PAC Files. This is MyProt.pac:

#
#THESE TWO FILES NEED TO BE COPIED FROM THE BRO SOURCE DIRECTORY TO OUR WORKING DIRECTORY TO MAKE IT WORK:
#
%include binpac.pac
%include bro.pac

###
#THESE LINES ARE NECESSARY BECAUSE LATER A events.bif.h file IS GENERATED, BUT IT'S NOT INCLUDED 
#ANYWHERE. WE MAKE THE INCLUDE HERE THAT WILL BE ADDED LATER IN THE .cc FILE
###
%extern{
#include "events.bif.h"
%}

###
#HERE WE DECLARE THE ANALYZER. IT IS ALWAYS MORE OR LESS THE SAME, IT HAS A CONNECTION AND TWO 
#FLOWS. OFICCIALLY A FLOW IS A SEQUENCE OF MESSAGES AND THE STATE BETWEEN MESSAGES
###
analyzer MyProtTCP withcontext {
 connection: MyProtTCP_Conn;
 flow:       MyProtTCP_Flow;
};

##
#...AND A CONNECTION IS AN UPLOAD FLOW AND A DOWNLOAD FLOW 
##
connection MyProtTCP_Conn(bro_analyzer: BroAnalyzer) {
 upflow = MyProtTCP_Flow(true);
 downflow = MyProtTCP_Flow(false);
};

##
#AS WE CAN SEE; AFTER DECLARING WHAT A CONNECTION AND A FLOW IS WE INCLUDE THE MYPROT-PROTOCOL.PAC 
#AND THE PROCESSING CONTINUES WITH IT
##
%include MyProt-protocol.pac

##
#A FLOW COMPONENTS ARE PDUs. IN MyProt-protocol.pac WE HAVE PREVIOUSLY DEFINED WHAT A MyProt PDU IS.
##
flow MyProtTCP_Flow(is_orig: bool) {
 flowunit = MyProtTCP_PDU(is_orig) withcontext (connection, this);
}

##
#FINALLY THE ANALYZER IS INCLUDED TO GENERATE THE EVENTS
##
%include MyProt-analyzer.pac

#
#MAINLY SPEAKING, IN THE PROTOCOL.PAC FILE WE EXPLAIN THE PROTOCOL AND IN THE ANALYZER-FILE.pac 
#WE WRITE THE FUNCTIONS TO UNDERSTAND THE PROTOCOL

OK, now we'll go through the MyProt-protocol.pac file (Again read the comments written by me). This file allow to define how is structure of the different messages of the protocol:

#HERE WE DEFINE WHAT IS A PDU FOR THIS PROTOCOL. AS WE SAW PREVIOUSLY, IT'S COMPOSED BY A ONE 
#BYTE HEADER AND A BODY
##
type MyProtTCP_PDU(is_orig: bool) = record {
 header: uint8;
 body: case is_orig of {
  true  -> request:  MyProtTCP_Request(header);   
  false -> response: MyProtTCP_Response(header);
 };
} &let {
 deliver: bool = $context.flow.deliver_MyProtTCP_PDU_FUNCTION(this)
};

##
#AS WE CAN SEE IN THE LET SECTION, WHEN RECEIVED, THE FUNCTION MyProtTCP_PDU_FUNCTION(MyProtTCP_PDU *f) 
#IS CALLED. THIS FUNCTION IS IMPLEMENTED IN THE MyProt-analyzer.pac FILE
##

##
#THE MyProtTCP_Request and Resopnse TYPES ARE DECLARED NEXT. AS WE CAN SEE THE FUNCTION CODES HAVE 
#SUBFUNCTION CODES NEXT AND IN THE RESPONSE PDUs THERE IS A RESPONSE STATUS VALUE 
##
type MyProtTCP_Request(header: uint8) = record {
       ufc:            uint16;
       body:           MyProt_Request(header, ufc);
} &let {
       deliver: bool = $context.flow.deliver_UniRequest(header, this);
} &byteorder=littleendian;


type MyProtTCP_Response(header: uint8) = record {
       urs:   uint16;
       body:       MyProt_Response(header, urs);
} &let {
       deliver: bool = $context.flow.deliver_UniResponse(header, this);
} &byteorder=littleendian;

##
#WHEN A REQUEST OR RESPONSE MESSAGE IS DETECTED, THE FUNCTION UNIREQUEST AND UNIRESPONSE ARE 
#CALLED. THESE FUNCTIONS ARE IMPLEMENTED IN THE ANALYZER.PAC FILE.
#
#THE MYPROT_REQUEST AND RESPONSE TYPES ARE DEFINED NEXT
##
type MyProt_Request(TCPheader: uint8, Uheader: uint16) = case Uheader of {
        0x22                -> readSystemVar:           readSystemVarRequest(TCPheader, Uheader);
        0x31                -> readId:              ReadIdRequest(TCPheader, Uheader);

(...)
};

type MyProt_Response(TCPheader: uint8, Uheader: uint16) = case Uheader of {
        0xFE                -> ok:            Response_OK(TCPheader, Uheader);
        0xFD    -> error:  Response_Error(TCPheader, Uheader); 
};

##
#IN THEM WE DEFINE THE DIFFERENT VALUES OF THE FUNCTION CODES AND WHAT FUNCTION WILL TREAT THEM. 
#IN THIS CASE WE WILL ONLY EXPLAIN THE ReadSystemVar REQUEST AND THE READID REQUEST
##

type ReadSystemVarRequest (TCPheader: uint8, Uheader: uint16) = record {
    data:       BinPAC_MyProt_ReadWriteSystemVarsRequest_Format;
} &let {
 deliver: bool = $context.flow.deliver_ReadSystemVarRequest(TCPheader, Uheader,this);
} &byteorder=littleendian;


type ReadIdRequest(TCPheader: uint8, Uheader: uint16) = record {
   Id: uint16;
} &let {
 deliver: bool = $context.flow.deliver_ReadIdRequest(TCPheader, Uheader,this);
} &byteorder=bigendian;

##
#FINALLY WE NEED TO DECLARE THE TYPE BinPAC_MyProt_ReadWriteSystemVarsRequest_Format THAT WE 
#USED IN THE ReadSystemVarRequest TYPE.
##
type BinPAC_MyProt_ReadWriteSystemVarsRequest_Format = record {
        CRC:            bytestring &length=4;
 Value_Number: uint8; 
 Var_Type_Code: uint16;
 u:   uint16;
 u2:   uint8; 
 Var_Numbers: uint16; 
 data:  bytestring &restofdata;
} &byteorder=littleendian;

##
#AS WE CAN SEE THIS IS THE STRUCTURE OF THE MESSAGE 1. WITH MESSAGE 2 WE DO NOT NEED A RECORD 
#BECAUSE IT ONLY HAS ONE VALUE, THE ID, WITH 16 BITS

And that's all, we have defined a simple protocol.

MyProt-Analyzer.pac

Now we need to implement the protocol and implement the events that bro will be generating when a a protocol message is detected. This will be don in the MyProt-Analyzer.pac file:

#THE ANALYZER.PAC FILE HAS THREE DIFFERENT SECTIONS. THE FIRST ONE IS A DECLARATION OF THE FUNCTIONS 
#IMPLEMENTED IN IT. IT STARTS WITH THE SYNTAX %header{ AND THE FUNCTIONS DECLARED ARE DEVELOPED IN 
#THE FORMER SECTION %code{ ... }. FINALLY IN THE %Init{ ... } SECTION WE WILL IMPLEMENT 
#THE "Deliver_*" FUNCTIONS MENTIONED IN THE PROTOCOL.PAC FILE. THESE THREE SECTIONS ARE ACTUALLY BIF CODE
#
#
#ALTHOUGH THE %Header and %Code SECTION MUST BE INCLUDED BEFORE THE %Init SECTION, I WILL EXPLAIN 
#THE %Init EARLIER AND THEN THE OTHER TWO:
##
%Header{
...
%}

%Code{
...
%}

%Init{

(...)
       function deliver_ReadIdRequest(TCPheader: uint8, Uheader: uint16, 
                message: const_bytestring): bool
               %{

               if ( ::myprot_readid_request )
                       {
                       BifEvent::generate_myprot_readid_request(connection()->bro_analyzer(),
                                    connection()->bro_analyzer()->Conn(),
                                    TCPheader
                                    , Uheader
                                    , bytestring_to_val(message)
                                    );
                       }

               return true;
                %}

##
#THERE ARE MANY THINGS TO EXPLAIN HERE: 
#
#::myprot_readid_request IS A GLOBAL HANDLER WHICH EXISTS ONLY IF A MESSAGE OF THIS TYPE 
#HAS BEEN DETECTED. ITS NAME IS THE SAME AS #THE generate_myprot_readid_request FUNCTION 
#(WITHOUT THE generate_ PART) AND THE SAME AS THE NAME OF THE EVENT DEFINED IN THE events.bif FILE.
#
#THE BifEvent::generate_myprot_readid_request IS A BIF FUNCTION THAT WILL GENERATE THE CODE THAT 
#WILL MANAGE THE EVENT CREATION WHEN THIS MESSAGE IS DETECTED. THE EVENT NAME WILL BE (IN THIS 
#CASE) myprot_readid_request. THIS SAME NAME WILL HAVE TO APPEAR IN THE events.bif FILE.
#
#THIS METHOD DECLARATION, IN THE EVENTS.BIF FILE, WILL HAVE ONLY 4 PARAMETERS (ALL EXCEPT THE 
#FIRST 'ANALYZER' ONE).
#
#IF IN THIS "deliver" FUNCTION WE NEED TO USE AN AUXILIARY FUNCTION, IT WILL HAVE TO DECLARED 
#IN THE %header SECTION AND IMPLEMENTED IN THE %code SECTION
#
#BIF HAS ITS OWN TYPES AND ANY INT8, INT16, INT32, FLOAT, DOUBLE CAN BE CONVERTED INTO ITS 
#TYPE "COUNT", WHICH IS A 64bit VALUE
##

function deliver_ReadSystemVarRequest(TCPheader: uint8, Uheader: uint16, 
         message: ReadSystemVarRequest(TCPheader, Uheader)): bool
               %{

               if ( ::MyProt_readsystemvar_request )
                       {
                       BifEvent::generate_MyProt_readsystemvar_request(connection()->bro_analyzer(),
                                   connection()->bro_analyzer()->Conn(),
                                   TCPheader
                                   , Uheader
                                   , MyProt_ReadWriteSystemVarsRequestFormat_ToBro(message->data())
                                   );
                       }

               return true;
                %}

##
#MANY THINGS TO MENTION HERE AS WELL:
#
#AS WE CAN SEE, IN THIS CASE THE THIRD PARAMETER OF THE "Deliver" FUNCTION IF A RECORD OF 
#TYPE ReadSystemVarRequest, DEFINED IN THE PROTOCOL.PAC FILE. AS WE CAN ALSO SEE, THIS RECORD 
#HAS ONLY TWO PARAMETERS WHILE IN ITS DECLARATION HAS THREE PARAMETERS... THE LAST ONE DISAPPEAR.
#
#IN THE CALL TO THE "Generate" FUNCTION, THE LAST PARAMETER USES SOMETHING 
#CALLED MyProt_ReadWriteSystemVarsRequestFormat_ToBro. THIS IS AN AUXILIARY FUNCTION, 
#IMPLEMENTED BY US IN THE %CODE SECTION
##

%Header{
          RecordVal* MyProt_ReadWriteSystemVarsRequestFormat_ToBro(BinPAC_MyProt_ReadWriteSystemVarsRequest_Format *rec);
%}

%Code{
          RecordVal* MyProt_ReadWriteSystemVarsRequestFormat_ToBro(BinPAC_MyProt_ReadWriteSystemVarsRequest_Format *rec)
                {
                RecordVal* myprot_format = new RecordVal(BifType::Record::MyProtReadWriteSystemVarsRequestFormat);
                myprot_format->Assign(0, bytestring_to_val(rec->shifted_CRC()));
                myprot_format->Assign(1, new Val(rec->Value_Number(), TYPE_COUNT));
                myprot_format->Assign(2, new Val(rec->Var_Type_Code(), TYPE_COUNT));
                myprot_format->Assign(3, new Val(rec->u(), TYPE_COUNT));
                myprot_format->Assign(4, new Val(rec->u2(), TYPE_COUNT));
                myprot_format->Assign(5, new Val(rec->Var_Numbers(), TYPE_COUNT));
                myprot_format->Assign(6, bytestring_to_val(rec->data()));
                return myprot_format;
                }
%}

##
#MANY THINGS TO MENTION HERE TOO:
#
#NOTICE THE BifType::Record::MyProtReadWriteSystemVarsRequestFormat TYPE. THIS IS A RECORD TYPE FOR 
#THE BIF AND BRO FRAMEWORKS. IT MUST BE DEFINED IN THREE DIFFERENT PLACES:
#             - src/types.bif
#             - /build/scripts/base/init-bare.bro
#             - /build/src/types.bif.netvar_h
#
#NOTICE THE "TYPE_COUNT" TYPES. THEY ARE DEFINED IN Val.h IN BRO. A COUNT TYPE IS A 64bit INTEGER.
#
#NOTICE THE bytestring_to_val(rec->data()) FUNCTIONS. THIS IS AN INTERNAL BRO/BIF FUNCTION THAT 
#ALLOW TO CONVERT A bytestream INTO A StringVal TYPE. IT'S NECESSARY WHEN SENDING STRINGS AS PARAMETERS

The events.bif file store the declaration of the different events the analyzer will be capturing. Its structure is the following:

(...)
event MyProt_readid_request%(c: connection, TCPheaders: MyProtTCPHeaders, 
            Uheader: URequestHeader, Data: string%);

event MyProt_writesystemvar_request%(c: connection, TCPheaders: UTCPHeaders, 
            Uheader: URequestHeader, UReadWriteSystemVarBody: MyProtReadWriteSystemVarsRequestFormat%);
(...)

These events will be re-written in our bro script files.

On the other hand if we are declaring new bro record, we need to define them in the types.bif file:

(...)
type MyProtRequestHeader: record;
type MyProtResponseHeader: record;

type MyProtReadWriteSystemVarsRequestFormat: record;
(...)

They will be specified in the init-bare.bro file:

(...)
type MyProtReadWriteSystemVarsRequestFormat: record {
        CRC:            string;
        Value_Number:   count;
        Var_Type_Code:  count;
        u               count;
        u2:             count;
        Vars_Numbers:   count;
        data:           count;
};
(...)

Finally to avoid error during compilation, for every new record it's necessary to include the following line in the /build/src/types.bif.netvar_h file:

(...)
    namespace BifType { namespace Record{  extern RecordType * MyProtReadWriteSystemVarsRequestFormat;  } } 
(...)

Compilation process

So, we have developed the BinPAC Parser. Everything's going to be easier now.... Stay quiet man, stay quiet. Things get worse now...

To "compile" this PAC files we'll use binpac:

binpac MyProt.pac

If there's no error, two new files will be generated:

MyProt_pac.cc
MyProt_pac.h

So, now we have three .cc files (Plugin.cc, MyProt.cc and MyProt_pac.cc) and three .h files (Plugin.h, MyProt.h and MyProt_pac.h). How the hell do we handle this??

Well, we need to include the MyProt_pac.h file in the MyProt.h file:

#include "myprot_pac.h"

Then we need to modify the CMakeFiles.txt to add all four files, together with the .bif files. So the CMakeFiles.txt will have to look like:


cmake_minimum_required(VERSION 2.8)

project(Plugin)

include(BroPlugin)

bro_plugin_begin(MYPROT MYPROT)
bro_plugin_bif(src/types.bif src/events.bif)
bro_plugin_cc(src/Plugin.cc src/MyProt.cc src/MyProt_pac.cc)
bro_plugin_dist_files(README CHANGES COPYING VERSION)
bro_plugin_end()

So, to resume, the steps that must be followed to make all this work are:

Prepare .pac files
launch the command: binpac Myprot.pac
If everything went OK, all MyProt.h, MyProt.cc, MyProt_pac.h, MyProt_pac.cc will have to be moved to the src directory (not actually necessary)
Modify Plugin.cc as mentioned earlier
If new records are used, the types.bif will has to be updated
If new events are defined, the events.bif has to be updated
Now we need to remove the build directory and run the compiling commands:

rm -rf build/
binpac myprot.pac 
mv myprot_pac.* src
./configure --bro-dist=..
make
make install

After building the analyzer

To test if bro is detecting the plugin we'll run:

bro -NN | grep -i myprot

Now we need to register the analyzer in the MyBroScript.bro script:

(...)
const ports = { 302/tcp };
(...)
event bro-init()
{
(...)
          Analyzer::register_for_ports(Analyzer::ANALYZER_MYPROT, ports);
(...)
event myprot_readid_request(c: connection, TCPheaders: MyProtTCPHeaders, MyProtheader: MyProtRequestHeader, Data: string)
{
      print("MyProt Read Id request Message detected");
}

event myprot_writesystemvar_request(c: connection, TCPheaders: MyProtTCPHeaders, MyProtheader: MyProtRequestHeader, 
              MyProtReadWriteSystemVarBody:MyProtReadWriteSystemVarsRequestFormat)
{ 
       print("MyProt 0x23 Message detected"); 
}

The hard part

My protocol has a request/response architectutrte, and the request/response message structure depends on a Function Code. This Function Code is sent in the request message, but NOT in the response message.

So, when I receive a response I don't know if it belongs to the request A or the request B. The only way to know that is checking the transport level ID.

So, I need to maintain any kind of array that relate transport id values and function codes. The question is how to do that. I need to read and write that global value in the MyProt-protocol.pac file.

The solution came by modifying the connection class and adding the array (in the MyProt.pac file):

(...)
refine connection UmasTCP_Conn += {
        %member{
                int previous_fcs[256];
        %}
};
(...)

Then added a RESPONSE type that call to a function to recover the FC data from the array:

(...)

type XXX{
(...)
} &let {
       (...)
       ufc: uint8 = $context.connection.get_Previous_FC(header.Transport_id);
};
(...)

And wrote two functions in the analyzer.pac file:

(...)
refine connection MyProtTCP_Conn += {
        (...)

        function get_Previous_FC(Transport_id: int): int
        %{
                return previous_fcs[tid%256];
        %}

        function SetTID_FC(transport_id:int, ufc:int): bool
        %{
                previous_fcs[transport_id%256]=ufc;

                return true;
        %}
(...)

The second one is called every time a message event is detected:

(...)
 function deliver_message(header: BinPAC_TCP_Header, MYPROTheader: BinPAC_MYPROT_header): bool
                %{
                if ( ::MyProt_message )
                   {
                           connection()->SetTID_FC(${header.transport_id}, ${MYPROTheader.myprot_fc});
(...)

Errors found

During compilation I found a wide variety of error, which very often I didn't know why they were happening. To avoid you the multiple hours spent looking for the root causes of these errors, I will explain the error message and the original cause of some of the errors found:

Error:

fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: cannot load plugin 
library /usr/local/bro/lib/bro/plugins/MYPROT_MYPROT_MYPROT//lib/MYPROT-MYPROT.linux-x86_64.so: 
/usr/local/bro/lib/bro/plugins/MYPROT_MYPROT//lib/MYPROT-MYPROT.linux-x86_64.so: undefined symbol:
_ZN7BifType6Record36MyProtReadWriteSystemVarsRequestFormatE

Explanation: This error appears after a successful compilation of the plugin, when running a bro script. In this case there is a Record that Bro is not able to find. The Plugin compiles fine BUT there is a declaration missing either in types.bif or in netvar_h. This could be caused as well for not including a types.bif file in the CMakelists.txt file

Error:

/tools/MyBro/bro/MyProt_plugin/src/myprot_pac.cc: In member function ‘bool binpac::MyProtTCP::MyProtTCP_Flow::deliver_ReadSystemVarRequest
(binpac::MyProtTCP::BinPAC_MyProtTCP_TransportHeader*, binpac::MyProtTCP::BinPAC_MyProt_Request_Header*, 
binpac::MyProtTCP::ReadSystemVarRequest*)’: /tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc:4300:71: error: 
‘MyProt_ReadWriteSystemVarsRequestFormat_ToBro’ was not declared in this scope
          , MyProt_ReadWriteSystemVarsRequestFormat_ToBro(message->data())

Explanation: There was a declaration missing in the %header section of the MyProt-analyzer.pac file. A similar error could be caused because the type name used in myprot-protocol.pac and in myprot-analyzer.pac are different. Check that.

Error:

myprot.pac:40: syntax error, at end of file (yychar=0)

Explanation: This error appears when running binpac and it's caused by a missing "}" that closes any function or if the final "}" is missing.

Error:

/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc: In function ‘RecordVal* 
binpac::MyProtTCP::MyProt_ReadWriteSystemVarsRequestFormat_ToBro
  (binpac::MyProtTCP::BinPAC_MyProt_ReadWriteSystemVarsRequest_Format*)’:
/tools/MyBro/bro/MyProt_plugin/src/MyProt_pac.cc:4902:42: error: 
‘MyProtReadWriteSystemVarsRequestFormat’ is not a member of ‘BifType::Record’
   RecordVal* MyProt_format = new RecordVal(BifType::Record::MyProtReadWriteSystemVarsRequestFormat);

Explanation: In this case, there a line missing in the %bro%/build/src/types.bif.netvar_h file. This could be caused as well if the name of the record is NOT correct when using it in the myprot-analyzer.pac file.

Error:

/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc: In function ‘RecordVal* 
binpac::myprotTCP::myprot_SetOwnerRequestFormat_ToBro(binpac::myprotTCP::BinPAC_myprot_SetOwnerRequest_Format*)’:
/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc:4894:62: error: expected ‘;’ before ‘)’ token
   myprot_format->Assign(0, new Val(rec->unknown(), TYPE_COUNT)));

Explanation: As explained in the error, there is an extra ')' in the indicated line.

Error:

/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc: 
In member function ‘bool binpac::myprotTCP::myprotTCP_Flow::deliver_InitCommRequest
(binpac::myprotTCP::BinPAC_MyProtTCP_TransportHeader*, binpac::myprotTCP::BinPAC_myprot_Request_Header*, 
const const_bytestring&)’: /tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc:3987:10: error: too many 
arguments to function ‘void BifEvent::generate_myprot_initcomm_request
(analyzer::Analyzer*, Connection*, Val*, Val*)’
          );
          ^

Explanation: In this case, the number of arguments of an event in its declaration in events.bif and in the "generate_myprot_XXX_request" call in the myprot-analyzer.pac file.

Error:

In file included from /tools/pruebas_bro/MyBro/bro/umas_plugin/src/Umas.h:5:0,
                 from /tools/pruebas_bro/MyBro/bro/umas_plugin/src/Plugin.cc:3:
/tools/pruebas_bro/MyBro/bro/umas_plugin/src/umas_pac.h:3:0: error: unterminated #ifndef
 #ifndef umas_pac_h
 ^

Explanation: In this case, you have to run binpac myprot.pac and check the error given:

root@kali:/tools/MyBro/bro/myprot_plugin# binpac myprot.pac 
switching to file ./binpac.pac
switching to file ./bro.pac
switching to file ./myprot-protocol.pac
switching to file ./myprot-analyzer.pac
./myprot-protocol.pac:427: error : `BinPAC_MyProt_ReadWriteRequest_Format' undeclared

Error:

/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc:4927:3: error: expected unqualified-id before ‘{’ token
   {
   ^

Explanation: There is an extra ';' in the %code section of the myprot-analyzer.pac. Of course you can always go to line 4927 of myprot_pac.cc file and look for an error.

Error:

error in /usr/local/bro/lib/bro/plugins/MYPROT_MYPROT/lib/bif/./events.bif.bro, line 85: identifier not defined: MyProtReadWriteRequestFormat
error in /usr/local/bro/lib/bro/plugins/MYPROT_MYPROT/lib/bif/./events.bif.bro, line 90: identifier not defined: MyProtReadWriteRequestFormat
internal error in /usr/local/bro/lib/bro/plugins/MYPROT_MYPROT/lib/bif/./events.bif.bro, line 156: internal type MyProtReadWriteRequestFormat missing
Aborted

Explanation: This error appear when running our bro script because there is no entry in the init-bare file for the Record indicated.

Error:

error in /usr/local/bro/lib/bro/plugins/MYPROT_MYPROT/lib/bif/./events.bif.bro, line 125 and ./test2.bro, line 201: incompatible 
types (event(c:connection; TCPheaders:count; Uheader:URequestHeader; function:count;) and event(c:connection; TCPheaders:count; 
Uheader:URequestHeader;))

Explanation: This is NOT an error in our analyzer but in the bro script. We are not writing correctly all parameters of an event call. As you can see, there is an argument left.

Error:

/tools/MyBro/bro/MyProt_plugin/src/MyProt_pac.h:10:2: error: expected initializer before ‘RecordVal’
CMakeFiles/UMAS-UMAS.linux-x86_64.dir/build.make:242: fallo en las instrucciones para el objetivo 'CMakeFiles/MYPROT-MYPROT.linux-x86_64.dir/src/Plugin.cc.o'

Explanation: In this case it's a CMake error (it is showed in spanish). It was caused by a '#' in the header section of the analyzer.pac file...

Error:

/tools/MyBro/bro/MyProt_plugin/src/Myprot_pac.h:2669:2: error: expected initializer before ‘RecordVal’
  RecordVal* MyProot_SetOwnerRequestFormat_ToBro(BinPAC_MyProt_SetOwnerRequest_Format *rec);

Explanation: There was a ';' left at theend of function declaration in the analyzer.pac file.

Error:

/tools/MyBro/bro/Myprot_plugin/src/MyProt_pac.cc: In member function ‘bool binpac::MyProtTCP::MyProtTCP_Flow::deliver_InitCommResponse(binpac::MyProtTCP::BinPAC_MyProtTCP_Header*, binpac::MyProtTCP::BinPAC_MyProt_Response_Header*, binpac::MyProtTCP::InitCommResponse*)’:

/tools/MyBro/bro/MyProt_plugin/src/MyProt_pac.cc:4268:10: error: too many arguments to function ‘void BifEvent::generate_myprot_initcomm_response(analyzer::Analyzer*, Connection*, Val*, Val*)’         );

Explanation: The declaration of an event in the events.bif an in the analyzer.pac file was different.

Error:

/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc:2698:96: error: no matching function for call to ‘binpac::MyProtTCP::MyProtTCP_Flow::deliver_ReadInfoResponse0(binpac::MyProtTCP::BinPAC_MyProtTCP_Header*, binpac::MyProtTCP::BinPAC_MyProt_Response_Header*, binpac::MyProtTCP::ReadInfoResponse0*)’
  deliver_ = t_context->flow()->deliver_ReadInfoResponse0(TCPheader(), MYPROTheader(), this);

Explanation: In this case the 'deliver' function used in the protocol.pac an in the analyzer.pac files were different..

Error:

/tools/MyBro/bro/myprot_plugin/src/myprot_pac.cc:1894:59: error: invalid use of non-static member function
           message->parameter_header()->Parameter_Type());

Explanation: The first problem here is that the issue is NOT in the line shown by the compiler. In my case I had the following record in my myprot-analyzer.pac:

                    BifEvent::generate_comm_request(connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), 
                                                                        TCPHeaderToBro(header), message->msg_type,
                                                                         message->parameter_header()->Parameter_Type());

...and the problem it's there, but it's not related with the last line! it comes with the msg_type variable, it need a parentheses at the end:

message->msg_type(),

...and that's all, working!

Of course there are many other error you will find while compiling but it's impossible to add them all here.

I hope this post will be helpful to anybody. This is what I learned during two week battling against bro and its friends!.

2 comentarios:

Unknown29 de agosto de 2017, 9:26
you have idea about how to write NIS protocol analyser ?
ResponderEliminar
Respuestas
Anónimo30 de marzo de 2022, 11:04
Liras En La Red: Developing A Bro Analyzer As A Plugin >>>>> Download Now

>>>>> Download Full

Liras En La Red: Developing A Bro Analyzer As A Plugin >>>>> Download LINK

>>>>> Download Now

Liras En La Red: Developing A Bro Analyzer As A Plugin >>>>> Download Full

>>>>> Download LINK
ResponderEliminar
Respuestas

Añadir comentario

Liras en la red

Redes Sociales

jueves, 28 de abril de 2016

Developing a Bro analyzer as a plugin

Starting

The hard part

Errors found

2 comentarios: