Introduction

Theory:

1.Topics:

--> Topics is nothing but a particular stream of data like a table in a database.we can create as many topics we want.Each topic can be identified by its name.

Partition

-->Each Topics are split in partitions and each partition is ordered.Each message within a partitions gets an Incremental Id.

Ex:

Brokers & Topics:

-->Topics are hosted in a server called brokers.

->Each Topic partition will be spread across brokers there is no sequence like which partition shoud go which broker.

Topic Replication Factor:

-->Kafka is distributed system.So replication should be there for each topic partition.

-->In this replication factor leader election is handled by zookeeper.when one of the broker went down and come back and it will conduct the election and one of them become the leader after syncing the data.

Producers:

-->Producers will send data to topics.By default when there is no key when sending the data it uses round robin technique.Producers will automatically load balance the data while it i sending.

-->The meaning of producers and consumers can be learnt in a video:

https://www.youtube.com/watch?v=udnX21__SuU

Consumers:

-->Consumers will read data from the topic.It will read the topic in multiple partitions parallel and each partiton reading time it will read the offsets in sequence.

ConsumerOffsets:

-->ConsumerKafka Broker:

Zookeeper:

-->Zookeeper stores the metadata of all the brokers.

-->zookeeper leader will perform write operations form kafka and read cluster members of zookeeper are used for read operations.

Kafka Topics Overview: