kafka Interview Questions and Answers

Prepare in advance for your Kafka interview with the best possible Apache Kafka interview questions and answers compiled by our experts that will help you crack your Kafka interview and land a good job as an Apache Kafka Developer, Big Data Developer, etc. The following Apache Kafka interview questions discuss the key features of Kafka, how it differs from other messaging frameworks, partitions, broker and its usage, etc. Prepare well and crack your interview with ease and confidence!

1. What is the retention policy for Kafka records in a Kafka cluster?

Ans:

Kafka cluster retains all data records using a configurable retention period. The data records are retained even if they have been consumed by the consumers. For example, if the retention period is set as one week, then the data records are stored for one week after their creation before they are deleted. So consumers can access this data for one week after its creation.

2. What are the core APIs provided in Kafka platform?

Ans:

Kafka provides the following core APIs:

• Producer API – An application uses the Kafka producer API to publish a stream of records to one or more Kafka topics.
• Consumer API – An application uses the Kafka consumer API to subscribe to one or more Kafka topics and consume streams of records.
• Streams API – An application uses the Kafka Streams API to consume input streams from one or more Kafka topics, process and transform the input data, and produce output streams to one or more Kafka topics.
• Connect API – An application uses the Kafka connect API to create producers and consumers that connect Kafka topics to existing applications or data systems.

3. Compare: RabbitMQ vs Apache Kafka 

Ans:

One of Apache Kafka’s alternatives is RabbitMQ. So, let’s compare both:

i. Features:

Apache Kafka– Kafka is distributed, durable and highly available, here the data is shared as well as replicated.

RabbitMQ– There are no such features in RabbitMQ.

ii. Performance rate:

Apache Kafka– To the tune of 100,000 messages/second.

RabbitMQ- In case of RabbitMQ, the performance rate is around 20,000 messages/second.

4. Justify the offset in writer information integration tool?

Ans:

Messages square measure kept in partitions and assigned a distinctive ID to every of them for fast and straightforward access. That distinctive range is known as because the offset that’s accountable to spot every of the messages within the partition.

5. What is the difference between Apache Kafka and Apache Storm?

Ans:

• Apache Kafka: It is a distributed and robust messaging system that can handle huge amounts of data and allows passage of messages from one end-point to another.
• Apache Storm: It is a real time message processing system, and you can edit or manipulate data in real time. Apache storm pulls the data from Kafka and applies some required manipulation.

6. What do you know about a partition key?

Ans:

A partition key is used to point to the aimed division of communication in Kafka producer. Usually, a hash-oriented divider concludes the division ID with the input, and also people use modified divisions.

7. Explain the role of Streams API?

Ans:

An API which permits an application to act as a stream processor, and also consuming an input stream from one or more topics and producing an output stream to one or more output topics, moreover, transforming the input streams to output streams effectively, is what we call Streams API.

8. What is a way to balance masses in writing once one server fails?

Ans:

Every partition in the writer has one main server that plays the role of a pacesetter and one or additional non-connected servers that square measure named because the followers. Here, the leading server sets the permission and remainder of the servers simply follow him consequently. In case, the leading server fails then followers take the responsibility of the most server.

9. Within the producer, when will a “queue fullness” situation come into play?

Ans:

Queue fullness occurs when there are not enough Followers servers currently added on for load balancing.

10. Explain the term “Log Anatomy”?

Ans:

We view log as the partitions. Basically, a data source writes messages to the log. One of the advantages is, at any time one or more consumers read from the log they select.

11. What is multi-tenancy?

Ans:

This is the most asked Kafka Interview Questions in an interview. Kafka can be deployed easily as a multi-tenant solution. The configuration for different topics on which data is to be produced or consumed this feature is enabled. With all this, it also provides operational support for different quotas.

12. What do you mean by Stream Processing in Kafka?

Ans:

The type of processing of data continuously, real-time, concurrently, and in a record-by-record fashion is what we call Kafka Stream processing.

13. If the replica stays out of the ISR for a very long time, then what does it tell us?

Ans:

If the replica stays out of the ISR for a very long time, or the replica is not in sync with the ISR then it means that the follower server is not able to grasp data as fast the leader is doing. So basically the follower is not able to come up with the leader’s activities.

14. Do you know how to improve the throughput of the remote consumer?

Ans:

Well, it is an interesting and advanced concept in Kafka. If the consumer is located in the distant location then you need to optimize the socket buffer size to tune the overall throughput of a remote consumer.

15. When do you call the cleanup method?

Ans:

The cleanup method is called when a Bolt is being shutdown and should cleanup any resources that were opened. There’s no guarantee that this method will be called on the cluster: For instance, if the machine the task is running on blows up, there’s no way to invoke the method. The cleanup method is intended when you run topologies in local mode (where a Storm cluster is simulated in process), and you want to be able to run and kill many topologies without suffering any resource leaks.

16. Why do you think the replications are dangerous in Kafka?

Ans:

Duplication assures that the issued messages available are absorbed in the case of any appliance mistake, plan fault, or recurrent software promotions.

17. State Disadvantages of Apache Kafka?

Ans:

Limitations of Kafka are:

• No Complete Set of Monitoring Tools.
• Issues with Message Tweaking.
• Not support wildcard topic selection.
• Lack of Pace.

18. How to balance loads in Kafka when one server fails?

Ans:

Every partition in Kafka has one main server that plays the role of a leader and one or more non-connected servers that are named as the followers. Here, the leading server sets the permission and rest of the servers just follow him accordingly. In case, the leading server fails then followers take the responsibility of the main server.

19. How to start a Kafka server?

Ans:

Given that Kafka exercises Zookeeper, we have to start the Zookeeper’s server. One can use the script packaged with Kafka to get a crude but effective single node Zookeeper instance> bin/zookeeper-server-start.sh config/zookeeper.properties. Now the Kafka server can start> bin/Kafka-server-start.sh config/server.properties.

20. What ensures load balancing of the server in Kafka?

Ans:

As the main role of the Leader is to perform the task of all read and write requests for the partition, whereas Followers passively replicate the leader. Hence, at the time of the Leader failing, one of the Followers takes over the role of the Leader. Basically, this entire process ensures load balancing of the servers.

21. What roles do Replicas and the ISR play?

Ans:

Basically, a list of nodes that replicate the log is Replicas. Especially, for a particular partition. However, they are irrespective of whether they play the role of the Leader. In addition, ISR refers to In-Sync Replicas. On defining ISR, it is a set of message replicas that are synced to the leaders.

22. What is the way to send large messages with Kafka?

Ans:

In order to send large messages using Kafka, you must adjust a few properties. By making these changes you will not face any exceptions and will be able to send all messages successfully. Below are the properties which require a few changes:

• At the Consumer end – fetch.message.max.bytes
• At the Broker, end to create replica– replica.fetch.max.bytes
• At the Broker, the end to create a message – message.max.bytes
• At the Broker end for every topic – max.message.bytes

23. How is Kafka used as a stream processing?

Ans:

Kafka can be used to consume continuous streams of live data from input Kafka topics, perform processing on this live data, and then output the continuous stream of processed data to output Kafka topics. For performing complex transformations on the live data, Kafka provides a fully integrated Streams API.

24. What are the benefits of using Kafka more than other messaging services like JMS, RabbitMQ, and others?

Ans:

Now a days kafka is a key messaging framework, not because of its features even for reliable transmission of messages from sender to receiver, however, below are the key points which should consider:

Reliability − Kafka provides a reliable delivery from publisher to a subscriber with zero message loss..

Scalability −Kafka achieve this ability by using clustering along with the zookeeper coordination server

Durability −By using distributed log, the messages can persist on disk.

Performance − Kafka provides high throughput and low latency across the publish and subscribe application.

Considering the above features Kafka is one of the best options to use in Big Data Technologies to handle the large volume of messages for a smooth delivery.

25. Where does the meta information about Topics stored in a Kafka Cluster?

Ans:

Zookeeper stores the information about Topics. The information it stores is : number of partitions in a Topic; which node is the master of which partition, which node has the replica of the partition, etc.

26. Describe scalability in the context of Apache Kafka. 

Ans:

Apache Kafka has the ability to be scaled out without causing any semblance of downtime by tacking on nodes.

27. What is the main difference between Kafka and Flume?

Ans:

Even though both are used for real-time processing, Kafka is scalable and ensures message durability.

28. Would it be possible to use Kafka without the zookeeper?

Ans:

No, it is not possible to use Kafka without the zookeeper. The user will not be able to connect directly to the Kafka server in the absence of a zookeeper. For some reason, if the zookeeper is down then the individual will not be able to access any of the client requests.

29. Is message duplication necessary or unnecessary in Apache Kafka?

Ans:

Duplicating or replicating messages in Apache Kafka is actually a great practice. It ensures that all messages will never be lost, even if the main or producer server suffers a failure.

30. What are Kafka Topics?

Ans:

Kafka Topics are categories or feeds to which data streams or data records are published to. Kafka producers publish data records to the Kafka topics and Kafka consumers consume the data records from the Kafka topics.

31. Describe high-throughput in the context of Apache Kafka.

Ans:

There is no need for substantially large hardware in Apache Kafka. This is because Apache Kafka is capable of taking on very high-velocity and very high-volume data. It can also take care of message throughput of thousands of messages per second. In summary, Apache Kafka is very fast and efficient.

32. Explain the functionality of the Connector API in Kafka?

Ans:

The Connector API is responsible where it allows the application to stay connected and keep a track of all the changes that happen within the system. For this to happen, we will be using reusable producers and consumers which stays connected to the Kafka topics.

33. What is the real-world use case of Kafka, which makes it different from other messaging frameworks?

Ans:

There are a plethora of use cases, where Kafka fits into the real work application, however I listed below are the real work use cases which are frequently used.

Metrics: Use for monitoring operation data, which can use for analysis or doing statistical operation on gather the data from distributed system

Log Aggregation solution: can be used across an organization to collect logs from multiple services, which are consumed by consumer services to perform the analytical operation.

Stream Processing: Kafka’s strong durability is also very useful in the context of stream processing.

Asynchronous communication: In microservices, keeping this huge system synchronous is not desirable, because it can render the entire application unresponsive. Also, it can defeat the whole purpose of dividing into microservices in the first place. Hence, having Kafka at that time makes the whole data flow easier. Because it is distributed, highly fault-tolerant and it has constant monitoring of broker nodes through services like Zookeeper. So, it makes it efficient to work.

Chat bots: Chat bots is one of the popular use cases when we require reliable messaging services for a smooth delivery.

Multi-tenant solution: Multi-tenancy is enabled by configuring which topics can produce or consume data. There are also operations support for quotas

Above are the use cases where predominately require a Kafka framework, apart from that there are other cases which depend upon the requirement and design.

34. What square measures the most options of writers that build it appropriate for information integration and processing in real-time?

Ans:

Some of the foremost lightness options of writers that build it well-liked worldwide include – information partitioning, quantifiability, low-latency, high throughputs etc. These options square measure the rationale why writers had become the foremost appropriate selection for information integration and processing within the period of time.

35. Explain what geo-replication is within Apache Kafka.

Ans:

For the Apache Kafka cluster, Apache Kafka MirrorMaker allows for geo-replication. Through this, messages are duplicated across various data centers or cloud regions. Geo-replication can be used in active or passive scenarios for the purpose of backup and recovery. It is also used to get data closer to users and support data locality needs.

36. Explain the term “Topic Replication Factor”.

Ans:

It is very important to factor in topic replication while designing a Kafka system. Hence, if in any case, a broker goes down its topics’ replicas from another broker can solve the crisis.

37. What are the three main system tools within Apache Kafka?

Ans:

The three main system tools in Apache Kafka include Apache Kafka Migration Tool, Consumer Offset Checker, and Mirror Maker. Apache Kafka Migration Tool is used to move a broker from a specific version to another version. Consumer Offset Checker is used to show topics, partitions, and owners within a specific set of topics or consumer group. Mirror maker is used to mirror an Apache Kafka cluster to another Apache Kafka cluster.

38. What is the maximum message size that can be handled and received by Apache Kafka?

Ans:

The maximum message size that Apache Kafka can receive and process is approximately one million bytes, or one megabyte.

39. What does it indicate if a replica stays out of ISR for a long time?

Ans:

If a replica remains out of ISR for an extended time, it indicates that the follower is unable to fetch data as fast as data accumulated at the leader.

40.Within the producer can you explain when will you experience QueueFullException occur?

Ans:

Well, if the producer is sending more messages to the broker and if it cannot handle this in the flow of the messages then we will experience QueueFullException. The producers don’t have any limitation so it doesn’t know when to stop the overflow of the messages. So to overcome this problem one should add multiple brokers so that the flow of the messages can be handled perfectly and we won’t fall into this exception again.

41. What are the key components of Kafka?

Ans:

Kafka consists of the following key components:

Kafka Cluster – Kafka cluster contains one or more Kafka brokers (servers) and balances the load across these brokers.

Kafka Broker – Kafka broker contains one or more Kafka topics. Kafka brokers are stateless and can handle TBs of messages and, thousands of reads and writes without impacting performance.

Kafka Topics – Kafka topics are categories or feeds to which streams of messages are published to. Every topic has an associated log on disk where the message streams are stored.

Kafka Partitions – A Kafka topic can be split into multiple partitions. Kafka partitions enable the scaling of topics to multiple servers. Kafka partitions also enable parallel consumption of messages from a topic.

Kafka Offsets – Messages in Kafka partitions are assigned a sequential id number called the offset. The offset identifies each record location within the partition. Messages can be retrieved from a partition based on its offset.

Kafka Producers – Kafka producers are client applications or programs that post messages to a Kafka topic.

Kafka Consumers – Kafka consumers are client applications or programs that read messages from a Kafka topic.

42. When does the queue full exception emerge inside the manufacturer?

Ans:

Queue Full Exception naturally happens when the manufacturer tries to propel communications at a speed which Broker can’t grip. Consumers need to insert sufficient brokers to collectively grip the amplified load since the Producer doesn’t block.

43. In the Producer, when does QueueFullException occur?

Ans:

Whenever the Kafka Producer attempts to send messages at a pace that the Broker 

cannot handle at that time QueueFullException typically occurs. However, to collaboratively handle the increased load, users will need to add enough brokers, since the Producer doesn’t block.

44. When not to use Apache Kafka?

Ans:

• Kafka doesn’t number the messages. It has a notion of “offset” inside the log which identifies the messages.
• Consumers consume the data from topics but Kafka does not keep track of the message consumption. Kafka does not know which consumer consumed which message from the topic. The consumer or consumer group has to keep a track of the consumption.
• There are no random reads from Kafka. Consumer has to mention the offset for the topic and Kafka starts serving the messages in order from the given offset.
• Kafka does not offer the ability to delete. The message stays via logs in Kafka till it expires (until the retention time defined).

45. What is the role of the ZooKeeper in Kafka?

Ans:

Apache Kafka is a distributed system built to use Zookeeper. Zookeeper’s main role here is to build coordination between different nodes in a cluster. However, we also use Zookeeper to recover from previously committed offset if any node fails because it works as a periodically committed offset.

46. Explain the role of the offset?

Ans:

There is a sequential ID number given to the messages in the partitions of what we call an offset. So, to identify each message in the partition uniquely, we use these offsets.

47. Describe durability in the context of Apache Kafka.

Ans:

 Messages are essentially immortal because Apache Kafka duplicates its messages.

48. Describe low latency in the context of Apache Kafka. 

Ans:

Apache Kafka is able to take on all these messages with very low latency, usually in the range of milliseconds.

49. Explain the role of the Kafka Producer API?

Ans:

The role of Kafka’s Producer API is to wrap the two producers – kafka.producer.SyncProducer and the kafka.producer.async.AsyncProducer. The goal is to expose all the producer functionality through a single API to the client.

50. Is apache Kafka a distributed streaming platform?If yes, what you can do with it?

Ans:

Yes, Apache Kafka is a streaming platform. A streaming platform contains the vital three capabilities, they are as follows: – It will help you to push records easily – It will help you store a lot of records without giving any storage problems – It will help you to process the records as they come in

51. Is replication critical or simply a waste of time in Kafka?

Ans:

Replicating messages could be a smart follow in writer that assures that messages can ne’er lose though the most server fails.

52. Which components are used for stream flow of data?

Ans:

Bolt:- Bolts represent the processing logic unit in Storm. One can utilize bolts to do any kind of processing such as filtering, aggregating, joining, interacting with data stores, talking to external systems etc. Bolts can also emit tuples (data messages) for the subsequent bolts to process. Additionally, bolts are responsible to acknowledge the processing of tuples after they are done processing.

Spout:- Spouts represent the source of data in Storm. You can write spouts to read data from data sources such as databases, distributed file systems, messaging frameworks etc. Spouts can broadly be classified into following –

Reliable:- These spouts have the capability to replay the tuples (a unit of data in data stream). This helps applications achieve ‘at least once message processing’ semantics as in case of failures, tuples can be replayed and processed again. Spouts for fetching the data from messaging frameworks are generally reliable as these frameworks provide the mechanism to replay the messages.

Unreliable:- These spouts don’t have the capability to replay the tuples. Once a tuple is emitted, it cannot be replayed irrespective of whether it was processed successfully or not. This type of spouts follow ‘at most once message processing’ semantics.

Tuple:- The tuple is the main data structure in Storm. A tuple is a named list of values, where each value can be any type. Tuples are dynamically typed — the types of the fields do not need to be declared. Tuples have helper methods like getInteger and getString to get field values without having to cast the result. Storm needs to know how to serialize all the values in a tuple. By default, Storm knows how to serialize the primitive types, strings, and byte arrays. If you want to use another type, you’ll need to implement and register a serializer for that type.

53. How are Kafka Topic partitions distributed in a Kafka cluster?

Ans:

Partitions of the Kafka Topic logs are distributed over multiple servers in the Kafka cluster. Each partition is replicated across a configurable number of servers for fault tolerance.

Every partition has one server that acts as the ‘leader’ and zero or more servers that act as ‘followers’. The leader handles the reads and writes to a partition, and the followers passively replicate the data from the leader.

If the leader fails, then one of the followers automatically takes the role as the ‘leader’.

54. Describe fault-tolerance in the context of Apache Kafka. 

Ans:

Probably one of the biggest benefits of Apache Kafka that make the platform so attractive to tech companies is its ability to keep data safe in the event of a total system failure, major update, or component malfunction. This is known as fault-tolerance. Apache Kafka is fault-tolerant because it replicates every message within the system to store in case of malfunction.

55. Elaborate the architecture of Kafka. 

Ans:

In Kafka, a cluster contains multiple brokers since it is a distributed system. Topic in the system will get divided into multiple partitions, and each broker stores one or more of those partitions so that multiple producers and consumers can publish and retrieve messages at the same time.

56.What are the key benefits of using storms for real time processing?

Ans:

• Easy to operate: Operating storm is quiet easy
• Real fast: It can process 100 messages per second per node
• Fault Tolerant: It detects the fault automatically and restarts the functional attributes
• Reliable: It guarantees that each unit of data will be executed at least once or exactly once
• Scalable: It runs across a cluster of machines.

57. How is Kafka used as a storage system?

Ans:

Kafka has the following data storage capabilities which makes it a good distributed data storage system:

Replication – Data written to Kafka topics are by design partitioned and replicated across servers for fault-tolerance.

Guaranteed – Kafka sends acknowledgment to Kafka producers after data is fully replicated across all the servers, hence guaranteeing that the data is persisted to the servers.

Scalability – The way Kafka uses disk structures enables them to scale well. Kafka performs the same irrespective of the size of the persistent data on the server.

Flexible reads – Kafka enables different consumers to read from different positions on the Kafka topics, hence making Kafka a high-performance, low-latency distributed file system.

58. What is Broker and how does Kafka utilize brokers for communication?

Ans:

• Brokers are the system which is responsible for maintaining the published data.
• Each broker may have one or more than one partition.
• Kafka contains multiple brokers to maintain the load balancer.
• Kafka broker are stateless
• eg: Let’s say there are N partitions in a topic and there is N broker, then each broker has 1 partition.

59. What Is ZeroMQ?

Ans:

ZeroMQ is “a library which extends the standard socket interfaces with features traditionally provided by specialized messaging middleware products”. Storm relies on ZeroMQ primarily for task-to-task communication in running Storm topologies.

60. How do you send messages to a Kafka topic using Kafka command line client?

Ans:

Kafka comes with a command line client and a producer script kafka-console-producer.sh that can be used to take messages from standard input on console and post them as messages to a Kafka queue.

61. How are the messages consumed by a consumer in Kafka?

Ans:

By making use of send file API transfer of messages is done in Kafka. Using this file the transfer of bytes takes place from the socket to disk through the kernel space-saving copies and the calls between kernel users and back to the kernel.

62. Explain how you can reduce churn in ISR?Ans: When does a broker leave the ISR?

Ans:

ISR is a set of message replicas that are completely synced up with the leaders, in other words ISR has all messages that are committed. ISR should always include all replicas until there is a real failure. A replica will be dropped out of ISR if it deviates from the leader.

63. What happens if the preferred replica is not in the ISR?

Ans:

If the preferred replica is not in the ISR, the controller will fail to move leadership to the preferred replica.

64. How can you justify the writer’s architecture?

Ans:

Kafka products rely on a distributed style wherever one cluster has multiple brokers/servers related to it. The ‘Topic’ is going to be divided into lots of partitions to store the messages and there’s one client cluster to fetch the messages from brokers.

65. What’s a client cluster in Kafka?

Ans:

A client cluster is formed of one or additional shoppers that along take the various topics and fetch information from the brokers.

66. What is the replica?What does it do?

Ans:

A replica can be defined as a list of essential nodes that are responsible to log for a particular partition, and it doesn’t matter whether they actually play the role of a leader or not.

67. Explain the concept of Leader and Follower. 

Ans:

Every partition in Kafka has one server which plays the role of a Leader, and none or more servers that act as Followers. The Leader performs the task of all read and write requests for the partition, while the role of the Followers is to passively replicate the leader. In the event of the Leader failing, one of the Followers will take on the role of the Leader. This ensures load balancing of the server.

68. How can you get exactly once messaging from Kafka during data production?

Ans:

During data production to get exactly once messaging from Kafka you have to follow two things: avoiding duplicates during data consumption and avoiding duplication during data production. Here are the two ways to get exactly one semantics while data production: – Avail a single writer per partition, every time you get a network error checks the last message in that partition to see if your last write succeeded – In the message include a primary key (UUID or something) and de-duplicate on the consumer

69.Why is Kafka preferred over traditional message transfer techniques?

Ans:

Kafka products are more scalable, faster, robust and distributed by design.

70. What is Apache Kafka?

Ans:

Apache Kafka is a publish-subscribe open source message broker application. This messaging application was coded in “Scala”. Basically, this project was started by Apache software. Kafka’s design pattern is mainly based on the transactional logs design.

71. Enlist the several components in Kafka.

Ans:

The most important elements of Kafka are:

Topic –Kafka Topic is the bunch or a collection of messages.

Producer –In Kafka, Producers issue communications as well as publish messages to a Kafka topic.

Consumer –Kafka Consumers subscribes to a topic(s) and also reads and processes messages from the topic(s).

Brokers –While it comes to manage storage of messages in the topic(s) we use Kafka Brokers.

72. What is a Consumer Group?

Ans:

The concept of Consumer Groups is exclusive to Apache Kafka. Basically, every Kafka consumer group consists of one or more consumers that jointly consume a set of subscribed topics.

73. Is it possible to use Kafka without ZooKeeper?

Ans:

It is impossible to bypass Zookeeper and connect directly to the Kafka server, so the answer is no. If somehow, ZooKeeper is down, then it is impossible to service any client request.

74. What do you know about Partition in Kafka?

Ans:

In every Kafka broker, there are few partitions available. And, here each partition in Kafka can be either a leader or a replica of a topic.

75. Why is Kafka technology significant to use?

Ans:

There are some advantages of Kafka, which makes it significant to use:

High-throughput

We do not need any large hardware in Kafka, because it is capable of handling high-velocity and high-volume data. Moreover, it can also support message throughput of thousands of messages per second.

Low Latency

Kafka can easily handle these messages with the very low latency of the range of milliseconds, demanded by most of the new use cases.

Fault-Tolerant

Kafka is resistant to node/machine failure within a cluster.

Durability

As Kafka supports message replication, so,  messages are never lost. It is one of the reasons behind durability.

Scalability

Kafka can be scaled-out, without incurring any downtime on the fly by adding additional nodes.

76. What are the main APIs of Kafka?

Ans:

Apache Kafka has 4 main APIs:

• Producer API
• Consumer API
• Streams API
• Connector API

77. What are consumers or users?

Ans:

Mainly, Kafka Consumer subscribes to a topic(s), and also reads and processes messages from the topic(s). Moreover, with a consumer group name, Consumers label themselves. In other words, within each subscribing consumer group, each record published to a topic is delivered to one consumer instance. Make sure it is possible that Consumer instances can be in separate processes or on separate machines.

78. Why are Replications critical in Kafka?

Ans:

Because of Replication, we can be sure that published messages are not lost and can be consumed in the event of any machine error, program error or frequent software upgrades.

79. If a Replica stays out of the ISR for a long time, what does it signify?

Ans:

Simply, it implies that the Follower cannot fetch data as fast as data accumulated by the Leader.

80. What is the process for starting a Kafka server?

Ans:

• It is the very important step to initialize the ZooKeeper server because Kafka uses ZooKeeper.So, the process for starting a Kafka server is:
• In order to start the ZooKeeper server: > bin/zookeeper-server-start.sh config/zookeeper.properties
• Next, to start the Kafka server: > bin/kafka-server-start.sh config/server.properties

81. Is Apache Kafka a distributed streaming platform? If yes, what can you do with it?

Ans:

Undoubtedly, Kafka is a streaming platform. It can help:

• To push records easily
• Also, can store a lot of records without giving any storage problems
• Moreover, it can process the records as they come in

82. What can you do with Kafka?

Ans:

It can perform in several ways, such as:

In order to transmit data between two systems, we can build a real-time stream of data pipelines with it.

Also, we can build a real-time streaming platform with Kafka that can actually react to the data.

83. What is the purpose of the retention period in the Kafka cluster?

Ans:

However, retention period retains all the published records within the Kafka cluster. It doesn’t check whether they have been consumed or not. Moreover, the records can be discarded by using a configuration setting for the retention period. And, it results as it can free up some space.

84. Explain the maximum size of a message that can be received by the Kafka?

Ans:

The maximum size of a message that can be received by the Kafka is approx. 1000000 bytes.

85.What are the types of traditional methods of message transfer?

Ans:

Basically, there are two methods of the traditional message transfer method, such as:

•  Queuing: It is a method in which a pool of consumers may read a message from the server and each message goes to one of them.
•  Publish-Subscribe: Whereas in Publish-Subscribe, messages are broadcasted to all consumers.

86. What does ISR stand for in the Kafka environment?

Ans:

ISR refers to In sync replicas. These are generally classified as a set of message replicas which are synced to be leaders.

87. What is Geo-Replication in Kafka?

Ans:

For our cluster, Kafka MirrorMaker offers geo-replication. Basically, messages are replicated across multiple data centers or cloud regions, with MirrorMaker. So, it can be used in active/passive scenarios for backup and recovery; or also to place data closer to our users, or support data locality requirements.

88. Explain Multi-tenancy?

Ans:

We can easily deploy Kafka as a multi-tenant solution. However, by configuring which topics can produce or consume data, Multi-tenancy is enabled. Also, it provides operations support for quotas.

89. What is the role of Consumer API?

Ans:

An API which permits an application to subscribe to one or more topics and also to process the stream of records produced to them is what we call Consumer API.

90. What is the role of Connector API?

Ans:

An API which permits to run as well as build the reusable producers or consumers which connect Kafka topics to existing applications or data systems is what we call the Connector API.