In this Apache Kafka administration training course, you will learn to build and manage Kafka clusters using industry best practices developed by the world’s foremost Apache Kafka experts. You will learn how Kafka and the Confluent Platform work, their main subsystems, their functions, how they interact, and how to set up, manage, and tune your cluster. It covers the fundamentals of Kafka architecture, Distributed systems, and the roles of Producers, consumers, and brokers within the ecosystem. The course emphasizes hands-on experience with Kafka's immutable log, Topic partitions, and the critical role of Apache Zookeeper in Cluster coordination. By delving into managing, configuring, and Optimizing Kafka for performance, learners will understand the intricacies of scaling, monitoring, and maintaining High availability and Fault tolerance. The course also addresses Kafka security measures, including Authentication, Authorization, and Encryption practices.

Course Objectives:

In this course, you will learn to:

• Comprehend Apache Kafka's architecture, including its components like Producers, Consumers, and Brokers, and their primary functions.
• Understand the principles of distributed systems, covering scalability, fault tolerance, and high availability within Kafka.
• Recognize the role and essential services provided by Apache Zookeeper in a Kafka ecosystem.
• Configure and manage Kafka clusters for optimal performance, and learn how to handle common issues such as data imbalance and broker failure.
• Analyze the impact of partitions and message sizes on Kafka's performance and make informed decisions based on trade-offs.
• Implement Kafka security measures, including authentication, authorization, and encryption both in-flight and at rest.
• Troubleshoot common Kafka security issues and effectively manage Access Control Lists (ACLs).
• Design robust Kafka systems considering factors like CPU, RAM, network, and storage, along with rack awareness and disaster recovery strategies.
• Explore Kafka Connect for integrating systems with source and sink connectors, ensuring scalability and high availability.
• Master the concepts of immutability in Kafka logs, understand the meaning of "committed" in the context of messages, and ensure exactly-once message delivery semantics.

Course content

Introduction to Kafka

• What is Kafka?
• Kafka Features and Terminologies
• High-Level Kafka Architecture
• Real-Life Kafka Case Studies

Kafka Components – Theory

• Broker
• Controller
• Topics
• Partition
• Replication
• Producer
• Consumer

Confluent Kafka Single-Node Installation (as a system unit in Kraft mode)

• Linux Environment Setup and Pre-requisites
• Code Download (Confluent Community)
• Single Broker Installation
• Kafka Broker and Controller Configuration and Setup

Kafka Administration (Using Command Line Interface – CLI)

• Replication Factor
• Partitions
• Console Producer
• Console Consumer
• Console Consumer Groups
• Reset the Offsets of a Topic for a Consumer Group

Kafka Administration (Using GUI Tools)

• OffSet Explorer (Kafka Tools)

Kafka Topic Partition Reassignment

• Setup a Two-Broker Kafka Cluster
• Topic Partition Re-assignment

Kafka Configurations

• Broker-Level Configurations:
• hours
• bytes
• topics.enable
• Changing a Topic Configuration
• Retention Time
• Retention Bytes
• Log Cleanup Policies
• Delete
• Compact

Kafka – Connect

• Use of Kafka – Connect Schema Registry
• Building a Connector File with a Simple JSON File
• File Stream Source Connector Setup in Distribution Mode
• Kafka Connector Status: Start, Stop, Pause, and Resume Scenarios

Schema – Registry

• Use of Schema Registry in Kafka
• Building a Schema in Avro Using an AVRC File
• Kafka-Avro-Console-Producer Use Case
• Kafka-Avro-Console-Consumer Use Case

Kafka REST API

• Understanding REST API
• Configuring and Spinning Up a REST API
• Kafka Topic Endpoints with REST API

Kafka SSL Security

• Encryption – SSL/TLS Setup
• Authentication
• Authorization (ACL)

To see the full course content Download now