I updated my SlackBuilds and installed the new versions on my server. I waited a bit before releasing the packages as there had been some reports on a new “Unknown record type” in Barnyard2, this time with record type 110. Read my previous posts here and here to learn more details about this problem.

For the “110″ record type the workaround is disabling the “inspect_gzip” option in the snort.conf flie, while the authors of Barnyard2 work on a new release.

For daq-0.3 and snort-2.9.0.1 I submitted the new SlackBuilds and pre-built packages for Slackware, Slackware64 and ARMedslack are available on my site.

ERROR: Unknown record type read: 110

I had not seen this one myself, but did some quick investigating and found out that it is a new record type in Snort 2.9.0
This new record type is called “Unified2_Extra_Data” and is used for gzip and XFF data.

Barnyard2 will need to be adapted to recognize these new types and this will hopefully happen soon…

ERROR: Unknown record type read: 104
Fatal Error, Quitting..

I started looking for information on the internet (Google is our friend…), and found out that at least I was not alone

Similar problems

Basically, I found two types of reports:

1. some with record types like 2148576734, 1697542248, 3569595041, etc.
2. others more like the error I received, with the unknown record type being either 104 (like mine) or 105

Error type 1 (apparently random record types)

These errors are caused by reading a log from Snort that is not in the “unified2″ format. Barnyard2 only uses the newer unified2 log format from Snort as input.
The solution is to check your Snort configuration file and change it to use the newer format.

Error type 2 (record types 104 and 105)

The first “solution” I found was removing two parameters in the Snort configuration file for the unified2 log format, mpls_event_types and vlan_event_types.
The default line in the Snort 2.9.0 configuration reads:

output unified2: filename merged.log, limit 128, nostamp, mpls_event_types, vlan_event_types

For Barnyard I already had to remove the “nostamp” parameter, as it prefers to work with the time-stamp extensions.
I do not use the mpls option in Snort, but the vlan ID can be useful…

To get Snort working, I removed the two parameters, but was not satisfied.

Finding a real solution – Start examining the source code!

After reading through the source code of Barnyard2 and of the new u2spewfoo utility that comes with Snort 2.9.0, I found out that these record types are from version 2 of the Unified2 record format.
In the Barnyard2 source file unified2.h the definitions can be found:

#define UNIFIED2_EVENT 1
#define UNIFIED2_PACKET 2
#define UNIFIED2_IDS_EVENT 7
#define UNIFIED2_EVENT_EXTENDED 66
#define UNIFIED2_PERFORMANCE 67
#define UNIFIED2_PORTSCAN 68
#define UNIFIED2_IDS_EVENT_IPV6 72
#define UNIFIED2_IDS_EVENT_MPLS 99
#define UNIFIED2_IDS_EVENT_IPV6_MPLS 100
//version 2
#define UNIFIED2_IDS_EVENT_V2 104
#define UNIFIED2_IDS_EVENT_IPV6_V2 105

So, if Barnyard2 knows about the record types, why can’t it process them?
Well, I found that in spi_unified2.c there was this piece of code:

int Unified2ReadRecord(void *sph)
{
 DEBUG_WRAP(DebugMessage(DEBUG_LOG,"Reading record type=%u (%u bytes)\n", ntohl(u2hdr.type), ntohl(u2hdr.length)););
 switch (ntohl(u2hdr.type))
 {
 case UNIFIED2_IDS_EVENT:
 return Unified2ReadEventRecord(sph);
 break;
 case UNIFIED2_IDS_EVENT_IPV6:
 return Unified2ReadEvent6Record(sph);
 break;
 case UNIFIED2_PACKET:
 return Unified2ReadPacketRecord(sph);
 break;
 default:
 FatalError("Unknown record type read: %u\n", ntohl(u2hdr.type));
 break;
 }
 return -1;
}

This means that it will simply show an error message Unknown record type read: ... when it encounters anything different from type 7, 72 or 2. And that’s exactly what it does in our case

This gave me something to start working on…
I first wrote a quick patch to simply recognize and handle record type 104 and sent it to the development team to check if this was the way to go. They answered me that it was, so I completed the patch with record type 105 (which is for ip6) and included the necessary routines for the debugging mode.

This is the resulting patch:

--- barnyard2-1.8_orig/src/input-plugins/spi_unified2.c 2010-03-03 08:03:12.000000000 -0300
+++ barnyard2-1.8/src/input-plugins/spi_unified2.c 2010-10-09 23:31:17.000000000 -0300
@@ -62,11 +62,15 @@
 int Unified2ReadEventRecord(void *);
 int Unified2ReadEvent6Record(void *);
+int Unified2ReadEventRecordV2(void *);
+int Unified2ReadEvent6RecordV2(void *);
 int Unified2ReadPacketRecord(void *);
 void Unified2PrintCommonRecord(Unified2EventCommon *evt);
 void Unified2PrintEventRecord(Unified2Event *);
 void Unified2PrintEvent6Record(Unified2Event6 *evt);
+void Unified2PrintEventRecordV2(Unified2Event_v2 *);
+void Unified2PrintEvent6RecordV2(Unified2Event6_v2 *evt);
 void Unified2PrintPacketRecord(Unified2Packet *);
 /* restart/shutdown functions */
@@ -163,9 +167,15 @@
 case UNIFIED2_IDS_EVENT:
 return Unified2ReadEventRecord(sph);
 break;
+ case UNIFIED2_IDS_EVENT_V2:
+ return Unified2ReadEventRecordV2(sph);
+ break;
 case UNIFIED2_IDS_EVENT_IPV6:
 return Unified2ReadEvent6Record(sph);
 break;
+ case UNIFIED2_IDS_EVENT_IPV6_V2:
+ return Unified2ReadEvent6RecordV2(sph);
+ break;
 case UNIFIED2_PACKET:
 return Unified2ReadPacketRecord(sph);
 break;
@@ -221,6 +231,50 @@
 return -1;
 }
+int Unified2ReadEventRecordV2(void *sph)
+{
+ ssize_t bytes_read;
+ int record_size;
+ Spooler *spooler = (Spooler *)sph;
+
+ record_size = sizeof(Unified2Event_v2);
+
+ if(!spooler->record.data)
+ {
+ // SnortAlloc will FatalError if memory can't be assigned.
+ spooler->record.data=SnortAlloc(record_size);
+ }
+
+ if (spooler->offset < record_size)
+ {
+ /* in case we don't have it already */
+ bytes_read = read(spooler->fd, spooler->record.data + spooler->offset,
+ record_size - spooler->offset);
+
+ if(bytes_read == -1)
+ {
+ LogMessage("ERROR: read error: %s\n", strerror(errno));
+ return BARNYARD2_FILE_ERROR;
+ }
+
+ if(bytes_read + spooler->offset != record_size)
+ {
+ spooler->offset += bytes_read;
+ return BARNYARD2_READ_PARTIAL;
+ }
+
+#ifdef DEBUG
+ Unified2PrintEventRecordV2((Unified2Event_v2 *)spooler->record.data);
+#endif
+
+ spooler->offset = 0;
+
+ return 0;
+ }
+
+ return -1;
+}
+
 int Unified2ReadEvent6Record(void *sph)
 {
 ssize_t bytes_read;
@@ -265,6 +319,50 @@
 return -1;
 }
+int Unified2ReadEvent6RecordV2(void *sph)
+{
+ ssize_t bytes_read;
+ int record_size;
+ Spooler *spooler = (Spooler *)sph;
+
+ record_size = sizeof(Unified2Event6_v2);
+
+ if(!spooler->record.data)
+ {
+ /* SnortAlloc will FatalError if memory can't be assigned */
+ spooler->record.data=SnortAlloc(record_size);
+ }
+
+ if (spooler->offset < record_size)
+ {
+ /* in case we don't have it already */
+ bytes_read = read(spooler->fd, spooler->record.data + spooler->offset,
+ record_size - spooler->offset);
+
+ if(bytes_read == -1)
+ {
+ LogMessage("ERROR: read error: %s\n", strerror(errno));
+ return BARNYARD2_FILE_ERROR;
+ }
+
+ if(bytes_read + spooler->offset != record_size)
+ {
+ spooler->offset += bytes_read;
+ return BARNYARD2_READ_PARTIAL;
+ }
+
+#ifdef DEBUG
+ Unified2PrintEvent6RecordV2((Unified2Event6_v2 *)spooler->record.data);
+#endif
+
+ spooler->offset = 0;
+
+ return 0;
+ }
+
+ return -1;
+}
+
 int Unified2ReadPacketRecord(void *sph)
 {
 ssize_t bytes_read;
@@ -366,10 +464,66 @@
 " packet_action = %d\n", evt->packet_action););
 }
+void Unified2PrintEventRecordV2(Unified2Event_v2 *evt)
+{
+ char sip4[INET_ADDRSTRLEN];
+ char dip4[INET_ADDRSTRLEN];
+
+ if(evt == NULL)
+ return;
+
+ Unified2PrintEventCommonRecord((Unified2EventCommon *)evt);
+
+ inet_ntop(AF_INET, &(evt->ip_source), sip4, INET_ADDRSTRLEN);
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " ip_source = %s\n", sip4););
+
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " sport_itype = %d\n", ntohs(evt->sport_itype)););
+ inet_ntop(AF_INET, &(evt->ip_destination), dip4, INET_ADDRSTRLEN);
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " ip_destination = %s\n", dip4););
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " dport_icode = %d\n", ntohs(evt->dport_icode)););
+
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " ip_protocol = %d\n", evt->protocol););
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " packet_action = %d\n", evt->packet_action););
+}
+
 void Unified2PrintEvent6Record(Unified2Event6 *evt)
 {
 char sip6[INET6_ADDRSTRLEN];
 char dip6[INET6_ADDRSTRLEN];
+
+ if(evt == NULL)
+ return;
+
+ Unified2PrintEventCommonRecord((Unified2EventCommon *)evt);
+
+ inet_ntop(AF_INET6, &(evt->ip_source), sip6, INET6_ADDRSTRLEN);
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " ip_source = %s\n", sip6););
+
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " sport_itype = %d\n", ntohs(evt->sport_itype)););
+ inet_ntop(AF_INET6, &(evt->ip_destination), dip6, INET6_ADDRSTRLEN);
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " ip_destination = %s\n", dip6););
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " dport_icode = %d\n", ntohs(evt->dport_icode)););
+
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " ip_protocol = %d\n", evt->protocol););
+ DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+ " packet_action = %d\n", evt->packet_action););
+}
+
+void Unified2PrintEvent6RecordV2(Unified2Event6_v2 *evt)
+{
+ char sip6[INET6_ADDRSTRLEN];
+ char dip6[INET6_ADDRSTRLEN];
 if(evt == NULL)
 return;

The patch can also be downloaded here.

The code could probably be optimized since there are many duplicate lines now, but I did not want to change too much.
I sent the patch to the Barnyard2 developers and they may commit it in their git repository.
It looks like this patch should work on the 1.9-beta version as well, but I have not tested this… YMMV!

I have been running the patched Barnyard2 for several hours now on my Snort server without problems, using the vlan_event_types parameter.