Insufficient Logging and Monitoring

Logging and monitoring are often an afterthought when something has already gone wrong, but really, failure to ensure there is proper logging and monitoring can be very costly. 

On one extreme, when an incident occurs (be it security-related or not), having few or no logs at all makes it impossible to figure out what’s actually happened. On the other extreme, logging too much data can lead to privacy issues which can then lead to issues with regulators.

Below, we’ll go through some best practices that can help guide you to better logging and monitoring. 

Best practices

Let’s take a look at some best practices for good logging and monitoring.

Audit logging for sensitive functions

It's important to create audit logs for sensitive events like login attempts (whether the attempts were successful or not), user account changes, accessing/changing sensitive data, and other similar instances. This is true for both access by general users as well as any internal/administrative users. 

Logging these things become extremely relevant in cases where unauthorized access is believed to have occurred, regardless of whether it's an insider or outsider. 

When such an event happens, it's critical to be able to account for what data was accessed by whom, in order to understand the scale and scope of the event.

Error logging

Having error and warning logs aren’t just a general engineering best practice to monitor the health of the application; they can also act as a warning flag. 

When an attacker is in the process of discovering a vulnerability, they’ll quite often generate a large volume of errors and warnings that can give you an indication that an attacker may have found a vulnerability in your application.

Storing logs in a centralized location

Logs should always be transmitted and stored in a centralized place in real-time. This is both to ensure that the logs are immediately available for analysis by a SIEM, and to prevent an attacker who may have compromised a server from being able to modify, or delete logs. 

Retain logs for a defined amount of time

The unfortunate reality is that most breaches take days to detect and, in fact, 20% of breaches take months to detect. When the time from breach to detection takes that long, logs can often be the only source of data at one's disposal to determine what may have happened.

As such, storing logs for a well-defined period of time is extra-important. Standards like PCI mandate that logs are accessible with little to no delay, going back 3 months, while logs going back 12 months must be able to be restored on request. 

This approach strikes a good balance between being able to easily start investigating a potential incident and managing costs through cold storage of logs. 

Regularly audit logs for PII

Logs are great for investigating potential incidents, but the reality is that the logs themselves can also cause an incident. 

There's a fine balance between including the information needed to be able to investigate incidents, and logging too much. It's quite easy to accidentally include PII in logs which can cause problems with privacy regulations.

It's important to regularly audit logs for PII, and ensure it gets removed.

‍