Having a solid understanding of Kafka security is the Rosetta Stone to the event streaming landscape. With that knowledge, you can connect any event-driven application to dozens of Kafka brokers with relatively few changes.
Encryption is a powerful feature that requires some trade-offs in terms of performance. However, the benefits far outweigh the costs.
Authentication
Authentication is an essential element in securing Kafka. Authentication ensures that only authenticated users can write to and read from your Kafka cluster. By default, Kafka is the wild west without security, and if your company moves towards shared tenancy or enables multiple teams to use the same group for real-time data integration, authentication, authorization, and encryption are must-haves.
Kafka is designed as a multi-user event broker. While various means of safeguarding data transmission exist, many organizations still require improvement in implementing an effective authentication system across the enterprise. Using an open-source Kafka security solution, you can easily integrate your existing authentication services into a single unified platform and secure all aspects of your event broker.
Solace supports various SASL mechanisms, including GSSAPI (Kerberos), SCRAM, and PLAIN. These have a performance cost associated with them as the broker will need to encrypt and decrypt each message sent to and from simple topics on the brokers. The good news is that the performance impact is minimal and only affects ongoing events. It doesn’t affect events that have already been saved to disk so that anyone with read access to your filesystem can access them. That being said, it’s crucial to prioritize establishing a secure Kafka environment for your organization.
Authorization
Authentication aims to identify and verify who a system or individual is. Authorization, conversely, determines if the person or system has permission to use a resource or access a file.
Apache Kafka on Kubernetes is often set up to create a scalable event-streaming platform. It allows applications to easily publish and consume data streams stored as records in a topic. It’s an excellent solution for distributed systems that require fast data sharing between different parts of the system or various scenarios.
While Kafka is a highly scalable platform, some security challenges must be considered. For example, when using Kafka with replication, it’s essential to consider the implications of a broker or cluster failure. It is why it’s often a good idea to use a Kafka deployment with both replication and mirroring to ensure fault tolerance and protection of your data.
Securing Kafka with an ACL-based approach requires significant complexity to maintain and update configurations for proxies and brokers. It requires additional 3rd party tools and introduces the risk of potential privacy breaches. Lastly, a Kafka solution needs support for blocking authentication on brokers and clients previously trusted through revocation.
Encryption
Kafka is typically deployed in a cluster and is designed to support high availability with replicated partitions. This is great for scalability and fault tolerance. Data is stored in plain text and sent across the network, creating a vulnerability immediately—Encrypt communication between your Kafka broker and your applications.
To encrypt your Kafka communication, you need to use a physical layer security (TLS) or Secure Socket Layer (SSL) implementation that encrypts the data at rest and in motion between the brokers and your applications—setting up a secure critical management infrastructure to manage and store your encryption keys.
Once you have your encryption and authentication setup correctly, you can ensure that only authenticated users can read and write data in the cluster. It helps prevent data pollution and fraudulent activities.
When a consumer requests data from a topic, the Kafka authorizer checks the policies to determine whether or not to allow the request. The policy check takes session information (used to extract the client identity), resources (topic, consumer group, cluster), and operations. It looks up the corresponding ACLs in the policy store using a general format of (actor, resource, and operation) params.
Access Control
You need some form of authorization to control access to the data streams in your Apache Kafka cluster. It enables you to prevent malicious attacks such as unauthorized data transfer or tampering.
It is accomplished through Access Control Lists (ACLs) that specify which users or client applications can read and write to different Kafka resources. The ACLs can be configured at the topic, cluster, or group level. ACLs are enforced by the Kafka server plugin known as an authorizer, which is also used to manage authentication and encryption in the system.
Suppose you roll out your Kafka environment’s TLS (Transport Layer Security) support. In that case, it is vital to implement the proper ACLs so that only authenticated clients can access the data. To learn how to do this, check out the “encryption with TLS” section of the confluent documentation.
Using ACLs to manage access to your Kafka data is one of the best ways to protect your production environment, especially when dealing with sensitive information. The ACLs in your Kafka system are designed to grant permission based on who you are and what roles or groups you belong to. If an ACL allows your request, it will proceed, but if there’s a deny ACL in the system that matches your demands, that will be applied instead (deny permanently trumps allow).
0 Comments