Building Data Pipelines with Apache Kafka Training Course
Apache Kafka serves as a distributed streaming platform and has become the de facto standard for constructing data pipelines. It addresses a wide array of data processing scenarios, functioning as a message queue, a distributed log, a stream processor, and more.
The course begins with an exploration of the theoretical foundations of data pipelines, followed by an in-depth look at Kafka's core concepts. We will also examine essential components such as Kafka Streams and Kafka Connect.
This course is available as onsite live training in Italy or online live training.Course Outline
- Data pipelines 101: ingestion, storage, processing
- Kafka fundamentals: topics, partitions, brokers, replication, etc.
- Producer and Consumer APIs
- Kafka Streams as a processing layer
- Kafka Connect for integrating with external systems
- Kafka best practices and tuning
Requirements
A foundational understanding of Java 8 or Scala is recommended. To execute the examples locally, please ensure that Docker and Docker Compose are installed.
Open Training Courses require 5+ participants.
Building Data Pipelines with Apache Kafka Training Course - Booking
Building Data Pipelines with Apache Kafka Training Course - Enquiry
Building Data Pipelines with Apache Kafka - Consultancy Enquiry
Testimonials (2)
Possibility to perform independent exercises in the training environment.
Tomasz - PKO Zycie Towarzystwo Ubezpieczen S.A.
Course - Kafka for Administrators
The trainer tried to make the most complicated topics , explain it in simpler way
Calvin Raj Antony - SICPA SA
Course - Administration of Kafka Message Queue
Upcoming Courses
Related Courses
Administration of Confluent Apache Kafka
21 HoursConfluent Apache Kafka is a distributed event streaming platform built for high-throughput, fault-tolerant data pipelines and real-time analytics.
This instructor-led live training (available online or onsite) targets intermediate-level system administrators and DevOps professionals looking to install, configure, monitor, and troubleshoot Confluent Apache Kafka clusters.
Upon completion of this course, participants will be able to:
- Grasp the components and architecture of Confluent Kafka.
- Deploy and manage Kafka brokers, Zookeeper quorums, and essential services.
- Configure advanced features such as security, replication, and performance tuning.
- Utilize management tools to monitor and maintain Kafka clusters.
Course Format
- Interactive lecture and discussion.
- Extensive exercises and practice sessions.
- Hands-on implementation in a live-lab environment.
Customization Options
- To request customized training for this course, please contact us to arrange.
Apache Kafka Connect
7 HoursThis instructor-led, live training in Italy (online or onsite) is designed for developers seeking to integrate Apache Kafka with existing databases and applications for purposes such as processing and analysis.
Upon completion of this training, participants will be able to:
- Utilize Kafka Connect to ingest large volumes of data from a database into Kafka topics.
- Channel log data generated by application servers into Kafka topics.
- Render collected data available for stream processing.
- Export data from Kafka topics into secondary systems for storage and analysis.
Confluent Apache Kafka: Cluster Operations and Configuration
16 HoursConfluent Apache Kafka is an enterprise-grade distributed event streaming platform built on Apache Kafka. It supports high-throughput, fault-tolerant data pipelines and real-time streaming applications.
This instructor-led, live training (online or onsite) is aimed at intermediate-level engineers and administrators who wish to deploy, configure, and optimize Confluent Kafka clusters in production environments.
By the end of this training, participants will be able to:
- Install, configure, and operate Confluent Kafka clusters with multiple brokers.
- Design high-availability setups using Zookeeper and replication techniques.
- Tune performance, monitor metrics, and apply recovery strategies.
- Secure, scale, and integrate Kafka with enterprise environments.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Building Kafka Solutions with Confluent
14 HoursThis instructor-led live training, available either online or on-site, is designed for engineers who want to leverage Confluent (a Kafka distribution) to create and manage a real-time data processing platform for their applications.
Upon completion of this training, participants will be able to:
- Install and configure the Confluent Platform.
- Utilize Confluent’s management tools and services to simplify Kafka operations.
- Store and process incoming stream data.
- Optimize and manage Kafka clusters.
- Secure data streams.
Course Format
- Interactive lectures and discussions.
- Extensive exercises and practical sessions.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- This course is based on the open-source version of Confluent: Confluent Open Source.
- To request customized training for this course, please contact us to arrange.
A Practical Introduction to Stream Processing
21 HoursDuring this instructor-led, live training in Italy (onsite or remote), participants will learn how to set up and integrate various Stream Processing frameworks with existing big data storage systems, as well as related software applications and microservices.
Upon completing this training, participants will be able to:
- Install and configure various Stream Processing frameworks, including Spark Streaming and Kafka Streaming.
- Understand and choose the most suitable framework for specific requirements.
- Process data continuously, concurrently, and on a record-by-record basis.
- Integrate Stream Processing solutions with existing databases, data warehouses, data lakes, and other systems.
- Integrate the most appropriate stream processing library with enterprise applications and microservices.
Distributed Messaging with Apache Kafka
14 HoursDesigned for enterprise architects, developers, and system administrators, this course enables participants to master the implementation and management of high-throughput distributed messaging systems. The curriculum can be customized to address specific professional needs, such as focusing exclusively on system administration tasks.
Kafka for Administrators
21 HoursThis instructor-led, live training in Italy (online or on-site) is aimed at beginner-level, intermediate-level, and advanced-level system administrators and operations engineers who wish to use Apache Kafka to deploy, secure, monitor, and troubleshoot Kafka clusters.
By the end of this training, participants will be able to explain Kafka architecture and KRaft mode, operate and secure Kafka clusters, monitor performance and reliability, and resolve common production issues.
Apache Kafka for Developers
21 HoursThis instructor-led, live training in Italy (online or onsite) is aimed at intermediate-level developers who wish to develop big data applications with Apache Kafka.
By the end of this training, participants will be able to:
- Develop Kafka producers and consumers to send and read data from Kafka.
- Integrate Kafka with external systems using Kafka Connect.
- Write streaming applications with Kafka Streams & ksqlDB.
- Integrate a Kafka client application with Confluent Cloud for cloud-based Kafka deployments.
- Gain practical experience through hands-on exercises and real-world use cases.
Apache Kafka for Python Programmers
7 HoursThis instructor-led live training in Italy (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Apache Kafka features in data streaming with Python.
By the end of this training, participants will be able to use Apache Kafka to monitor and manage conditions in continuous data streams using Python programming.
Kafka Fundamentals for Java Developers
14 HoursThis instructor-led, live training in Italy (online or onsite) targets intermediate-level Java developers who wish to integrate Apache Kafka into their applications for reliable, scalable, and high-throughput messaging.
Upon completion of this training, participants will be able to:
- Comprehend the architecture and core components of Kafka.
- Set up and configure a Kafka cluster.
- Produce and consume messages using Java.
- Implement Kafka Streams for real-time data processing.
- Ensure fault tolerance and scalability within Kafka applications.
Administration of Kafka Message Queue
14 HoursThis instructor-led live session, available Italy (online or onsite), targets intermediate-level system administrators aiming to effectively utilize Kafka's message queuing features.
By the conclusion of this training, participants will be able to:
- Comprehend the architecture and capabilities of Kafka's message queuing.
- Configure Kafka topics tailored for message queuing scenarios.
- Produce and consume messages using Kafka.
- Monitor and manage Kafka when used as a message queue.
Security for Apache Kafka
7 HoursThis instructor-led, live training in Italy (online or onsite) is aimed at software testers who wish to implement network security measures into an Apache Kafka application.
By the end of this training, participants will be able to:
- Deploy Apache Kafka onto a cloud based server.
- Implement SSL encryption to prevent attacks.
- Add ACL authentication to track and control user access.
- Ensure credible clients have access to Kafka clusters with SSL and SASL authentication.
Apache Kafka and Spring Boot
7 HoursThis instructor-led, live training in Italy (online or onsite) is aimed at intermediate-level developers who wish to learn the fundamentals of Kafka and integrate it with Spring Boot.
By the end of this training, participants will be able to:
- Understand Kafka and its architecture.
- Learn how to install, configure, and set up a basic Kafka environment.
- Integrate Kafka with Spring Boot.
Stream Processing with Kafka Streams
7 HoursKafka Streams is a client-side library designed for building applications and microservices that exchange data with a Kafka messaging system. Traditionally, Apache Kafka has depended on Apache Spark or Apache Storm to process data between message producers and consumers. By leveraging the Kafka Streams API within an application, data can be processed directly inside Kafka, eliminating the need to route it to a separate cluster for processing.
In this instructor-led live training, participants will learn how to integrate Kafka Streams into a set of sample Java applications that exchange data with Apache Kafka for stream processing.
By the end of this training, participants will be able to:
- Understand the features and advantages of Kafka Streams compared to other stream processing frameworks
- Process stream data directly within a Kafka cluster
- Develop a Java or Scala application or microservice that integrates with Kafka and Kafka Streams
- Write concise code that transforms input Kafka topics into output Kafka topics
- Build, package, and deploy the application
Audience
- Developers
Format of the course
- Part lecture, part discussion, exercises, and heavy hands-on practice
Notes
- To request a customized training for this course, please contact us to arrange it
Administration of Kafka Topic
14 HoursThis instructor-led, live training in Italy (online or onsite) is designed for system administrators at the beginner to intermediate level who wish to learn how to effectively manage Kafka topics for efficient data streaming and processing.
By the end of this training, participants will be able to:
- Understand Kafka topic fundamentals and architecture.
- Create, configure, and manage Kafka topics.
- Monitor Kafka topics for health, performance, and availability.
- Implement security measures for Kafka topics.