Setting up splunk for HP printers

Splunk is one of the most well-known SIEMs. Many companies use it to gather security information all around the company/corporation to determine possible attacks and/or security breaches.

HP Printers, although with an increasing level of security are still weak enough to be a substancial target for adversaries willing to break into the company's network.

Configuring Splunk to gather Syslog information from HP printers is not as straight-forward task as expected. Although HP printers can set up syslog through their embedded web server, syslog messages do NOT have the standard syslog structure. HP printers syslog format is based on a code and then a message, removing other standard information like timestamp, device or severity.

Actually all of them goes encoded in the HP syslog messages but they need to be decoded in order any SIEM to understand these syslog messages.

The standard syslog message has the following format:

%timestamp% %source ip% %severity level%: %device%: %message%

...or any glavour of this format. See https://tools.ietf.org/html/rfc5424

However the HP printers syslog message has the following format:

%< internal_code >% %device%: %message% %optional_fields%

As it can be seen some standard information (timestamp, source_ip) is missing. This missing information is optionally sent through the optional fields. In this way, the message continue with fields like "time=20-02-2019..." or "ip=12.23.34.45".

However standard SIEMs are NOT prepared for this syslog format and need some preprocessing before gathering them.

The HP printer Splunk App

Splunk provides an app (or plugin) that gathers HP printer's information. Its name is “HP Printer Security” and it can be found at https://splunkbase.splunk.com/app/3588/.

To obtain this app it's necessary to register in the splunkbase website and log in.Once done:

1. Log into Splunk Enterprise.
2. On the Apps menu, click Manage Apps.
3. Click Install app from file.
4. In the Upload app window, click Choose File.
5. Locate the .tar.gz file you just downloaded, and then click Open or Choose.
6. Click Upload.
7. Click Restart Splunk, and then confirm that you want to restart.

The new app appears in the left hand side of the web interface.

A short read to this plugin's README file let's us understand it's NOT an official plugin. It instead was developed by a group of students.

Reading a little bit more, we find that the plgin actually reads a syslog file (In /etc/apps/data/hp_gen_log.log) and looks for lines with the standard syslog format:

%timestamp% %source ip% %severity level%: %device%: %message%

...like:

e.g. 2016-12-06T17:54:25.324722+08:00 172.20.134.251 LPR.INFO: printer: peripheral low-power state.

This expected format has nothing to do with the standard HP printers syslog format.

Fixing the problem

So we need to make splunk understand that the HP printer logs comes with a different format. There are two ways to accomplish this. You can either install a syslog server so that syslog messages are copied into a local file with the appropriate format. Or you can configure splunk to convert syslog message and introduce them in the splunk database directly with the appropriate format.

The first workaround means installing syslog-ng and configuring it so that the syslog messages are save to file with a different format. This is NOT explained in this post.

The second workaround means modifying the HP printer plugin & splunk configuration to accept syslog message directly from the network interface.

Setting up splunk

To make this app work with LFP printers, it’s necessary to modify three files in the /etc/system/local directory. First we’ll need to identify the new fields to be added to the printersec database. We’ll have to modify %splunk%/etc/system/local/fields.conf adding the following lines:

[code]
INDEXED=TRUE
[severity]
INDEXED=TRUE
[severity_val]
INDEXED=TRUE
[device]
INDEXED=TRUE

Then, in file %splunk%/etc/system/local/transforms.conf , we’ll calculate the values of these fields. Append the following lines into this field:

[eval1]
INGEST_EVAL=code=replace(substr(_raw,1,4),"([<>])","")

[eval2]
INGEST_EVAL=severity_val=code%8

[eval3]
INGEST_EVAL=severity=case(severity_val == "1", "LPR.ALERT", severity_val == "2", "LPR.CRITICAL", severity_val == "3", "LPR.ERROR", severity_val == "4", "LPR.WARNING", severity_val == "5", "LPR.NOTICE", severity_val == "6", "LPR.INFORMATIONAL")

[eval4]
INGEST_EVAL=device=mvindex(split(mvindex(split(_raw,":"),0)," "),1)

[eval5]
INGEST_EVAL=sourcetype="hp_printer_syslog"

Note: eval5 section could not be necessary if you were able to configure sourcetype as “hp_printer_syslog”.

The third file to modify is %splunk%/etc/system/local/props.conf. Here we tell splunk “please, use the sections I added in transforms.conf file”. To do this append the following lines to props.conf:

[linux_messages_syslog]
TRANSFORMS= eval1, eval2, eval3, eval4, eval5

Now it’s necessary to restart splunk in order to charge the file modifications. This is different in every case. In linux:

 
%splunk%/bin/splunk restart

In windows go to Control Panel --> Administrative tools --> Services --> Locate splunk service --> Restart

In order to avoid splunk reading the test values from a file, go to %splunk%/etc/apps/printersec/default/inputs.conf and remove the sections related to file hp_print_gen.log.

Next you need to clean all test data included with the HP Printer plugin. To do this open splunk and go to Settings --> Monitoring Console and, in the Search box write:

index=”printersec” | delete

Now you can start gathering syslog information from your HP printers.

The following will be how the HP Printer plugin for Splunk will show:

About UEFI and BIOS

UEFI is a much needed replacement for legacy BIOS setups. As for how it works – that’s going to take a bit longer to explain. There are quite a few advantages that UEFI has over BIOS setups, as well as some potential problems to consider.

To understand UEFI it’s helpful to have a solid grasp on BIOS. We’ll begin with a brief look at how a BIOS functions, and then move into UEFI.

BIOS>

When we turn on a computer we hit the power button, the BIOS program, stored in non-volatile (traditionally read only) memory is accessed. When a computer powers on, it begins by executing at memory address 0xffff0 – the end of the BIOS. Its first responsibility is the POST – Power On Self Test. This is where the BIOS accesses things like memory, I/O devices (think keyboard, monitor, mouse) and checks that everything works properly. It also performs a memory test to determine the RAM available, sets memory and disk drive parameters, configures any plug and play devices (PCIe, USB) and assigns interrupt requests and direct memory addressing. Last but not least it identifies a boot device, one of potentially several partitions flagged as bootable, looks at the boot sector (MBR) and loads it into memory.

Now the BIOS proceeds to check each device in it’s specified order for a bootable device. If one if found it provides a jump instruction to 0x7C00. The MBR contains our first stage boot loader, disk signature, and partition table. The BIOS transfers control to the start of MBR, which then scans for a Volume Boot Record on the specified partition. Keeping it simple, this loads code that loads a second stage boot loader (like NTLDR) which then finally loads the operating system.

To make it simple, BIOS checks for essential hardware, looks for bootable devices, loads a bootstrapper into memory, which calls on a boot loader to start the operating system. What’s the deal with all these steps? Well our BIOS setup lacked the ability to load any more than that first 512 MB block. So we needed this multi-stage process to get things up and running. Kind of a mess right? Well UEFI solved that.

Some of the limitations we faced with BIOS/MBR setups were a maximum of 4 primary partitions, and a 2.2 TB size limit on partitions. Of course we can have plenty of logical partitions if we choose, but remember primary partitions are the only bootable partitions as far as Windows is concerned. Though Linux is perfectly happy booting from one (so long as it’s not a dynamic disk anyways). The partition size limit didn’t matter as much in the past, but now we can pick up a 6 TB Hard Disk Drive with prices dropping over time.

We know how BIOS works now and see its shortcomings. Now let’s look at UEFI.

UEFI Explained

UEFI stands for Unified Extensible Firmware Interface, and it is the replacement for BIOS. Along with UEFI comes GPT partitioning to replace MBR setup. Together they added quite a few features, but before we get into that, let’s look at how a computer works with UEFI.

UEFI does not rely on a boot sector like our old friend BIOS. In most cases the old MBR sector does still exist for backwards compatibility. You may hear talk of “disabling UEFI” to use older discs/OS’s – this is incorrect. UEFI is in fact always enabled (as it is your firmware) and all that’s happening is UEFI is supporting a legacy boot method. So, on to the boot process!

When the UEFI computer is turned on, it uses what’s called a boot manager to look at the current boot configuration. In other words, it checks to see which operating system to boot first. It then loads that operating system and executes the necessary code to get it started up. Much simpler. We skip the extra steps of our multi-stage boot loaders and just boot right from the firmware!

UEFI Advantages

There are quite a few advantages to our new UEFI setups, so let’s take a look. MBR type partitioning limits you to 4 primary partitions. GPT can handle a theoretically infinite amount. Though Windows “limits” you to 128. On top of that we can now have much larger partitions. Instead of being limited to 2.2 TB, we can now have up to 9.4 ZB!

With UEFI you also sometimes get a fancy graphical interface for a firmware menu. Gone are the days of the 1990’s looking computer screen (well…in some of them anyways). Now we might be looking at a stylish point and click user interface. This can be good since it makes life easier for the end user. This is bad because it makes changing things you shouldn’t a little easier. End users might be more inclined to explore their systems BIOS (I feel for you, help desk folks).

UEFI presents a simplified boot process that makes for shorter OS load times. With Windows 8+, if you enable fast boot it loads so quickly you can’t get to your firmware configuration screen the traditional way unless you disable fast boot (Shift + restart will take you there if need be though).

UEFI also stores a copy of the partition table in a secondary location. If your partition table becomes corrupted, you aren’t going to flip your desk over and denounce computers. There are a few other advantages like the demise of 16-bit real mode, but that requires diving in deeper than we need.

UEFI Disadvantages

UEFI made pretty great changes, but it’s not all sunshine and blue skies. We did not address secure boot in the advantages section. Secure boot stops any operating system from booting that is not signed by a key which is embedded into the UEFI firmware. This can prevent things such as root kits. That’s fantastic. Malicious boot loaders will now have a much much harder time getting onto systems. However, there’s a problem. OEM manufacturers have the possibility of locking customers out of this system and thereby preventing them from installing another operating system.

Secure boot is a fantastic idea and a potential advantage, so long as it is left in the hands of the customer.You can’t boot via anything that you have not given your blessing while secure boot is on. The only reason for placing secure boot under disadvantages is because of the potential for abuse.

Another downside is UEFI’s use of the FAT32 standard for EFI partitions. This adds a lot of overhead to a system that doesn’t really need it. We’ve got 32 bit pointer-sectors for partitions that only need to load an operating system. That seems a bit excessive to me.

Lastly, UEFI still doesn’t fix one of the problems of our old BIOS/MBR setups. We still have to re-probe for devices once the operating system loads. It would be nice if there was a way to pass that information from the POST onto the Kernel to skip a step and move straight to device initialization.

Conclusions in the UEFI vs Legacy BIOS Boot Debate

UEFI isn’t quite too complicated. It has its ups and downs, but from my perspective it seems like there is more to love than hate. As long as OEMs don’t take away the user’s control over secure boot, I’m perfectly content with this newer system. UEFI seems like it’s here to stay. Love or hate it, but if you’re a computer person, hopefully you understand how it works now.

For more information check out this post on UEFI and Legacy BIOS from our intern program. It gets into the differences in coding for each system, hardware compatibility, and more.