Algorithmic systems analysis

Challenges with GDPR and log data

Unomaly automatically reduces data to assure GDPR compliance in regards to log data by protecting privacy in logs and allowing engineers to detect breaches in real time.

The Challenge

The purpose of the GDPR is to enforce a high standard of data protection on data that references individuals and could be directly or indirectly tied to the user. Indirect referencing happens a lot in log data as software systems produce log messages that include users’ IP addresses, User IDs and Session IDs, putting log data within the scope of the GDPR.

Log data is the ground truth of what's happening in your systems and, as such, can tell you about breaches and incidents on a root cause level. However, log data is difficult to analyze given the sheer amount of it and the GDPR limitation saying that you only may store data that is adequate, relevant, and limited to what is necessary for the purpose.

Here, then, is the dilemma: you have to analyze data to detect breaches and report them but you can't store all data. How are you supposed to comply with this contradiction?

The Solution

To comply with the GDPR, there are two ways of going about the handling of log data. One way is the traditional strategy of store and search; keeping your log data in a pile and either performing searches in it or manually setting up rules to find issues. The problem with this method is threefold:

• It's not GDPR compliant since you're storing data that you can't motivate why you need.

• It's expensive as you'll constantly spend time and money over the next 5-10 years implementing protections and preparing audits.

• It doesn’t work because you can't manually search the vast amount of log data to discover incidents or write rules that will predict every possible future scenario.

Instead, the reasonable path is the one recommended by the GDPR: minimizing data through solutions designed with privacy in mind. The reason that this works is that you don't need to protect data that doesn't exist. Machine learning can implement the principles of privacy by design and organizations can operate on real-time insights without having to manually ask questions, risk user privacy or maintain protection of data that isn’t valuable.

Unomaly

Unomaly uses machine learning to automatically analyze 100% of log data in your entire infrastructure and detects first signals of breaches and operations issues in real time. IT teams are interested in abnormal log data as it's abnormal activity that indicates an incident. Because of this, storing normal log data after it's been analyzed is useless and a liability. The solution to becoming GDPR compliant when it comes to log data is to remove the repetitive log data that contains no valuable information.

Unomaly analyzes all your log data to learn normal log structures, which are stored in the Unomaly baseline without storing personally identifiable information. Every incoming line of log data is matched against the baseline to determine whether it's normal or abnormal. If abnormal, the log is signalled as an anomaly. This results in the ability to detect intrusions and operations issues in real time, allowing you to solve and report incidents, while reducing log data with over 99.99999%, removing privacy data that you don't need.

Unomaly automatically reduces data to assure GDPR compliance in regards to log data by protecting privacy in logs and allowing engineers to detect breaches in real time.

We're excited to show you the value of Unomaly. This is why we offer an introduction to Unomaly as well as a trial. Find out more about how Unomaly can start providing value to you within an hour and make your organization more GDPR compliant.