Видео

Thank you so much for this video. Summarizes the general concepts brilliantly.

Damn. Finally a good video on Kafka. Thank you!

I just needed to watch it to become clear. Thanks a lot.

Y gregorio?

great video ! love this animation style and the simple explanation

This is genius, thank you so much!

It was very well presented, in a simple and effective way. This put me on the page about Kafka technology.

Idk how many videos I went over but not a single one would say that partition in Kafka is JUST A QUEUE. They would unnecessarily make it complicated with more technical jargon. Thank You good Sir🧔‍♂👋🎩

Great Video! I love it!

extremely helpful! Thanks James!

Well explained 🙏🏻

Please continue with the Golang content! It's so good!

Beautiful, thank you!!

bless you sir

promotion ka tareeka thoda cazual hai😂

To add Kekfa helps to achieve 2 main goals 1. Use Ques for async communication 2. Achieve a pub-sub model

best explanation ever!

simply beautiful, thank you

Great intro to the fundamentals! Great use of example too

thank you for this clear video

A great starter video. 😊

Simple

Love the brevity of the video !

I love this animation style and want to use one similar for a course's extra credit project! What software did you use to create these if you don't mind me asking?

Could there be figures that do not change locations? For example, "A,B,C,D,E" could A ever be the first post-randomiztion?

Great vid!

thanks!!

sub: Apache Kafka is the answer to the problems faced by the distribution and the scaling of messaging systems. let me try to illustrate this by an example imagine we were to design a system that listens to various basketball game updates from various sources such updates might include game scoring participants and timing information it then displays the games status on various channels such as mobile devices and computer browsers in our architecture we have a process that reads these updates and writes them in a queue we call this process a producer since it's producing these updates onto the queue at the head of this queue a number of downstream processes consume these updates to display them on the various channels we call these processes consumers over time we decide to expand and start following more and more games the problem is that our servers are now struggling to handle the load this is mainly because the queue is hosted on one server which is running out of memory and processing capacity our consumers are also struggling in a similar fashion so now we start thinking of how we can add more computing power by distributing our architecture but how do we go about distributing AQ data structure by its nature the items in a queue follow a specific ordering we could try to randomly distribute the contents of the queue onto multiple ones if we do this our consumers might consume the updates in the wrong order this would result in inconsistencies for example the wrong scoring being displayed across the channels one solution is to let the application specify the way to distribute the items in the queue in our example we could distribute the items using the match name meaning that the updates coming from the same match would be on the same queue this strategy would maintain an ordering per basketball match this is the basic fundamental difference of Kafka from other messaging systems that is item sent and received Kafka require a distribution strategy let's have a look at some more detail and terminology used in Kafka each one of these queues is called the partition and the total number of partitions is called a partition count each server holding one or more of these partitions is called a broker and each item in a partition is called a record the field used decide which partition the record should be stored in it's called the partition key it's up to the application to decide which field to use as the partition key if no key is specified Kafka simply assigns a random partition a grouping of partitions handling the same type of data is called a topic in order to identify each record uniquely Kafka provides a sequential number to each record this is called an offset essentially a recording that topic is identified by a partition number and an offset in our application since we have now distributed our data in the topic using the name as the partition key we can now also parallelize our consumer applications having one consumer per partition guarantees ordering per game consumers can live on one machine or distributed amongst multiple ones one important concept in Kafka is that consumers are very lightweight and we can create many of them without affecting performance this is mainly because Kafka only needs to maintain the latest offsets read by each consumer typically consumers read one record after the other and resume from where they left after a restart however in Kafka it's up to the consumer implementation to decide on how to consume records it's quite common to have consumers to read all the records from the beginning on startup or to read the record in different orders such as reading back to front for example in Kafka each consumer belonging to the same consumer group do not share partitions this means that each consumer would read different records from the other consumers multiple consumer groups are useful when you have different applications reading the same contents in our example we could have a consumer group called mobile and another consumer group called computer these groups will read the same records but update different channels each consumer in these groups will have separate offset pointers to keep track which latest record was read if consumers can read using custom ordering how can Kafka determine that the record has been consumed and it can safely delete that record so it can free up space the answer is that tough comp provides various policies that allow it to do a record cleanup for example using irritation policy you can provide a record age limits say 24 hours after which the records are automatically deleted using this policy if your consumer application is never down for more than this age limit no messages are lost another capability of Kafka is to store records in a fault tolerant and durable way each record is stored on persistent storage so that if a broker goes down it can recover when it comes back up additionally Kafka replicates partitions so that when a broker goes down a backup partition takes over and processing can resume this replication is configured using a replication factor for example a replication factor of three leads to three copies of a partition one leader and two backups this means that we can tolerate up to two brokers going down at the same time Kafka can be a solution to your scalability and redundancy problems if the problem is well stated and the technologies are well understood there are of course a lot more technical and implementation details which can be found on kafkas documentation I hope that this short video has been helpful at providing an introduction but the fundamental concepts in Kafka if you like it please give it a thumbs up and subscribe.

Thank you sir very simple and informative explanation

Thank you, not many info or vídeos about divolte and avron :)

Best intro to Kafka video I've seen. Thank you.

what software do you use for illustration?

Top stuff! Very clear and simple explanation. Thanks!

Nice and clear explanation - thank you.

thank you for sharing

how to scale queue? each queue need it's own distribution strategy replication factor

Only a person who has deep knowledge on a subject can explain anything with clarity in a short video. Looking forward to more videos from you!

view a lot of videos about kafka, and you 're video is the best one

Simple clear and concise. Thank you James

Its crazy how good this video is

Excellent explanation and solution

thank you for all the hard work. this is vary informative. ps. could you please tell me what is the application/s used to create the infographics images

This is the best and simplistic explanation I've seen on Muxes so far all over RUclips. Thanks a bunch for this great resource

Thanks a million for this one, from West Africa🎉

Thanks!

great video but, why do you talk like that it was hard to hear what you where talking about, while you stop and start a sentence at random points.

Love how you break it down into simple concepts, animations on point as usual, thanks James!

Wow! Amazing content in your channels. Much Love ❤️

Excellend explanation, thanks!

Nice video, but why so few views?