Intrusion detection and intrusion prevention are part of everyday life for admins. Solutions for these issues are basically very straightforward beasts that analyze data streams and use sophisticated heuristic methods to identify patterns that point to an imminent or ongoing attack. If any suspicious activity is noticed, an alarm helps you launch countermeasures at an early stage.

The developers at Coroot [1] go beyond the usual intrusion detection systems (IDS) on the market today and offer a kind of artificial intelligence (AI) IDS as a service, revealing several interesting technical details that are worth a closer look.

Dumb Systems

The countermeasures of solutions typically found on the market today can be anything from simple packet filters to sophisticated firewall constructs. Firewalls, for example, leverage connection states as a parameter to decide whether or not to deliver packets. Today's IDS or intrusion prevention systems (IPS) no longer simply police intrusions, they often offer options that automatically identify and warn you about distributed denial of service (DDoS) attacks so that you can immediately block the attack at the network level.

Although these solutions are anything but easy to use because they require a high level of skill and familiarity, these systems are in fact "dumb" in and of themselves. At the end of the day, they just use predefined patterns to determine whether or not an attack is taking place. The principle is similar to that of antivirus software, which only detects malware if it finds a matching signature in its database; it needs a description of the files and checksums that comes with the virus. If this signature is missing for some reason, the program is blind to the threat; thus, a lucrative malware market thrives on the darknet, where previously unknown exploits are sold over the virtual counter for large mounts of money.

AI Improvements

From today's perspective, classic anti-spam tools can be regarded as a first tentative approach toward automatic pattern-based prevention of malicious activities on the Internet. SpamAssassin [2] and Rspamd [3], to name just two examples, rely on heuristic methods to analyze various factors under the hood before deciding whether a message is spam.

These methods improve with user training and regular corrective intervention as follows: If email from a Nigerian prince with an unlikely offer of $20 million is spam, then the same conclusion probably applies to an email from a bank in Guinea with the same offer. A number of factors play a role, including, for example, the probability of such an offer from a stranger reaching the recipient by email completely out of the blue. No matter how you look at it, these systems have nothing to do with AI; they work purely on the basis of manual training with a few automated features.

In the wake of the AI wave currently sweeping the industry, resourceful companies are promising better spam filters and better AI-based intrusion IDS and IPS. As a reminder, the vast majority of AI incarnations are primarily based on large language models (LLMs; i.e., very sophisticated statistics). The principle always remains the same and is very similar to the heuristics of spam filters: From an arbitrary amount of training data, the system is taught to respond to certain events in a certain way.

The main difference between AI models and tools such as SpamAssassin is the use of unsupervised learning that ultimately aims to train the models without human intervention. Unsurprisingly, IT now has a widespread desire to use AI to detect attacks and intrusions, which has been covered several times in this magazine in the past, one example being the various AI plugins for Prometheus [4]. These plugins detect specific patterns in metrics data automatically with the use of AI libraries, which in turn is what enables Prometheus to trigger alarms.

Coroot

Up front, Coroot is a centrally hosted service in the cloud with a subscription-based business model. Although the API always needs to run in the local setup, the AI capabilities are stored in the cloud.

European customers can see potential trouble: Coroot intervenes deeply with the data flow of the systems it is intended to monitor, with no alternative to doing things any other way. After all, Coroot's central task is to identify threats by the traffic flowing through, which means that the data automatically leaves the European Union's General Data Protection Regulation (EU GDPR) area. Apart from the possibility that organizations might be opposed to their data being directed in this way, the compliance issue between the US Clarifying Lawful Overseas Use of Data (CLOUD) Act and the GDPR also plays a role. If this situation is not going to cause a compliance problem for you, Coroot is a very interesting tool.

Coroot is primarily aimed at operators of state-of-the-art systems that rely on container orchestration, which makes Kubernetes [5] a permanent fixture in this universe, and for good reason. Practically the entire Coroot model for tapping data relies on Kubernetes under the hood. What's more, Coroot is designed to collect the corresponding user data from containers only, which does not rule out anything; today, more or less any application can be containerized with relative ease. That said, administrators will want at least to take a closer look at the deployment model if they are considering Coroot.

The first step is to familiarize yourself with the solution's key components. Coroot itself acts as the platform's linchpin, on the one hand acting as an API and, on the other, handling communication with the provider's cloud services (i.e., the part that delivers the AI functions) in the background. Coroot relies on two services to maintain and manage its data, storing the metrics data in the well-known Prometheus time series database. Once again Prometheus proves to be the go-to tool when it comes to metrics data storage. Coroot stores logs, traces, and recognized profiles in ClickHouse [6].

ClickHouse is not a proprietary Coroot development but a column-based database that focuses on analyzing incoming data. A SQL dialect lets you automatically analyze the stored data on the basis of various parameters. If you are thinking of a kind of database for business intelligence, you are on the right track. Prometheus and ClickHouse can also be used as data sources, besides storing data, and are therefore a genuine bidirectional exchange.

From Coroot's point of view, the question of how to store data is at least as important as how to acquire the data to be analyzed. The developers have expanded their solution to use telemetry data in line with the OpenTelemetry standard [7] as the basis for its analyses. This approach primarily means two types of data: logfiles and traces. This data can be linked to existing applications in two ways: either with the OpenTelemetry SDK [8], which ensures deep integration at an application's API level, or by teaming the application with OpenTelemetry Collector, which writes logs and traces.