Threat Intelligence and Threat Indicators
Threat Intelligence is becoming more and more important in cyber security. Its main purpose is to help organizations identify and assess potential threats, understand the motivations and tactics of threat actors, and make informed decisions about how to protect their assets and systems. Threat intelligence is often used in conjunction with security technologies such as firewalls, intrusion detection and prevention systems, and security information and event management (SIEM) systems to improve organizations’ ability to detect, prevent, and respond to security incidents.
Moreover, integration of strong threat indicators from multiple sources has also become a fundamental pillar of modern threat prevention, that enables to protect web applications and APIs. There are various kinds of threat indicators, like source IP addresses known to be used by malicious threat actors or traditional, static threat signatures, as commonly used in IPS security layers or traditional WAF solutions. Strong and up-to-date threat indicators with a high confidence level, play an important role in proactively blocking known threats and future attacks. In case of IP-based threat indicators, those provide precise identification of the attacker’s location, making it easy to block all traffic from suspicious sources. As unique identifiers for traffic sources, they can provide high accuracy in comparison to other threat indicator types. Therefore, the usage of IP-based threat indicators has often become a requirement as part of regulatory compliance frameworks.
About open-appsec and CrowdSec
open-appsec, is an open-source Web Application & API Security solution which provides preemptive, zero-day threat prevention based on contextual machine learning, IPS, and more. You will find more details about open-appsec further below. In this article, we will present open-appsec’s new integration with CrowdSec Threat Intelligence (CTI).
CrowdSec provides a large, open-source, community-fueled collection of Threat Intelligence (Providing indicators based on IP addresses and networks). The primary source of this intelligence comes from open-source and lightweight software, the "CrowdSec security engine", which analyzes local security logs from the services you want to protect as well as 3rd part security solutions (like open-appsec), using suitable parsers and malicious behaviors detection rules to identify, block and then share offending IP addresses on the community network. In exchange for those reports of malicious IPs, users will then benefit from the real-time community blocklist and can make it available to the "remediation components" of their choice (also known as Bouncers): many provided on CrowdSec hub and others provided by security solutions (like open-appsec).
This new integration allows open-appsec to connect to the CrowdSec local API to consume the CrowdSec Threat Intelligence and function itself as a CrowdSec “Bouncer” preventing traffic originating from known malicious IPs and IP networks from accessing web applications and APIs which are protected by open-appsec. open-appsec can block or detect traffic based on these indicators in addition to its own comprehensive security mechanisms - like its Contextual Machine Learning engine, which provides preemptive, signature-less Zero-day and OWASP-Top-10 prevention, as well as its IPS engine, its Threat Cloud indicator enforcement, schema enforcement, and more.
In return, a parser that can be added to the CrowdSec agent is now available, allowing it to also parse open-appsec’s security logs and share the collected indicators.
open-appsec and CrowdSec both support deployments on all relevant platforms: K8s, Linux and Docker. open-appsec itself currently supports deployments integrated with NGINX, Ingress NGINX or Kong API Gateway functioning as reverse proxies, and more integrations will be added soon.
open-appsec’s Background
open-appsec is an open-source Web Application & API Security solution, available at GitHub, which provides automatic security using machine learning. It has proven multiple times to effectively protect against zero-day and OWASP-Top-10 attacks, as it is not signature-dependent but uses contextual machine learning instead. open-appsec reduces the administrative effort as well as the amount of false positives significantly while providing strong protection even for yet-unknown, zero day attacks. open-appsec supports all typical deployment platforms like VMs, Kubernetes, and Docker and integrates with NGINX, NGINX Ingress and Kong, and more integrations will be added soon.
As signatures for new attacks, by design, can only be created after new attacks have been published, a WAF solution that relies solely on signatures will never protect preemptively (in advance) against zero-day attacks. This is especially important as a vulnerability usually exists for a long time within an affected code of a software or a library, before the first public disclosure of a corresponding CVE record describing it.
The following timeline visualizes the three relevant phases related to the ‘Vulnerability Window’:
open-appsec's Machine Learning-based approach can solve this challenge and provide true preemptive protection against zero-day attacks while functioning independently of any signature updates and keeping false positives to a minimum level.
open-appsec’s automatic machine learning engine continuously analyzes HTTP/S requests to Websites or APIs. Incoming HTTP requests are evaluated against two machine-learning models:
a supervised model that was trained offline with millions of malicious and benign requests.
a non-supervised model that is built in real-time in the protected environment and is specific to its traffic patterns.
Contextual analysis includes the application’s structure and how users interact with its content, in order to automatically stop and block malicious requests and bad actors.
In parallel to the ML engine, open-appsec provides additional security capabilities. To name a few, IPS engine with Snort support, behavioral AntiBot protection (in Premium edition), and a huge collection of cloud IOCs for blocking malicious IPs, Anonymizers, and Tor (in Enterprise edition).
open-appsec & CrowdSec Integration
The following diagram visualizes the integration of open-appsec and CrowdSec:
Introducing the open-appsec “CrowdSec Bouncer” Functionality
With the newly added “CrowdSec Bouncer” capability, open-appsec is now able to connect and interact with CrowdSec security engine by connecting to the LAPI (Local API). From LAPI's "/decisions/stream" endpoint it will be able to retrieve the decisions from the "Shoot-in-sight" community blocklist, which contains a list of tens of thousands of known high-confidence, malicious source IPs and IP networks.
When connected to CrowdSec’s LAPI, open-appsec receives also local decisions (added by the CrowdSec agent based on local scenarios and log parsing) as well as those from the community blocklist (as shared by the community of CrowdSec users).
open-appsec can then be configured to use these indicators either for preventing or for detecting traffic originating from the IP addresses/networks which are part of the ingested CrowdSec indicators. This adds another effective security layer of prevention capabilities to open-appsec, in addition to its already existing security layers.
Resulting Set of Security Layers in open-appsec
Contextual Machine Learning for Web Apps and Web APIs with Preemptive, 0-day and OWASP Top 10 attack prevention
Snort signature support
IPS (Premium edition)
Behavioural AntiBot for Web Applications (Premium edition)
Malicious IPs, Anonymizers and Tor Prevention (Threat Cloud-Based, Premium edition)
API schema enforcement (Premium edition)
NEW: CrowdSec Threat Indicator Prevention (enforcing CrowdSec’s “community blocklist” as part of new L7 access control feature)
How to Deploy
For deployment of open-appsec in Kubernetes, please use the latest available Helm chart for open-appsec. This will allow you to configure the following newly added values specifically relevant for the CrowdSec “Bouncer” integration:
For more details, please have a look at:
· open-appsec Documentation
· CrowdSec Documentation
CrowdSec Log Parser and Scenario for open-appsec
You can also add an open-appsec scenario and open-appsec parser to your CrowdSec installation, allowing you to gather and add intelligence to CrowdSec based on open-appsec’s Security Logs. As open-appsec has very unique ML-based Threat detection capabilities this can provide strong additional value to your Threat Intelligence and the CrowdSec community.
For more details, please refer to open-appsec docs.
How Can I Test This Integration Easily Myself?
There’s no need to create a lab environment of your own. You can use the open-appsec Playground which now also features a ready-to-use hands-on CrowdSec scenario, where you can experience a demo attack, deploy open-appsec with CrowSec and then view how this integration can block the same attack.
Check out our constantly growing list of open-appsec Labs.
Alternatively, just follow the ‘Getting Started’ guides available in our Documentation for the primary open-appsec deployment method of your choice and also check out the ‘Integrations’ section for the specific configuration details regarding the open-appsec CrowdSec integration options.
For more information please visit:
header.all-comments