Step 1–Set up Ubuntu 16.04 Virtual Machine
First, follow steps 1 through 3 in my previous tutorial. NOTE: in your Vagrantfile you may need to use /usr/vagrant/api as the directory instead of /usr/ubuntu/api. I’m not sure why, but some installs use vagrant as the root user, and others use ubuntu. I’m sure there’s some reason why, but I haven’t figured it out.
Step 2–Install the latest JDK
Second, log into your box by typing vagrant ssh on the command line and follow these instructions for installing the latest JDK. (Or these.)
Step 3–Install Zookeeper
Third, install Apache Zookeeper with the following command:
$ sudo apt-get install zookeeperd
It should automatically start up on port 2181. You can verify this with the following command:
$ netstat -ant | grep :2181
Which should give you output like the following:
tcp6 0 0 :::2181 :::* LISTEN
Zookeeper is a small, fast “coordination service” that is maintained by Apache. Rather than having everybody (like Hadoop, Kafta, Drill, etc.) develop and include their own coordination service into their distributed software package, Apache decided to build a really high quality, lightweight coordinator (Zookeeper) that can be used by all of these other distributed systems to manage configuration, assure message ordering, etc. for your entire cluster. Here’s an overview, if you’d like to know more.
Step 4
Fourth, go to the Kafka downloads page. Click on the first binary version available, next to where it says Scala 2.11 (currently version 1.0.1). At the top of the page that comes up there will be a link after the words: “We suggest the following mirror site for your download:” Right click and copy the link address for your download. Then in your terminal, type wget and then a space and then paste the URL you just copied, and then hit Enter. For example:
$ wget http://mirror.cogentco.com/pub/apache/kafka/1.0.1/kafka_2.11-1.0.1.tgz
Once it’s done, if you view the contents of the directory, it should look something like this:
vagrant@ubuntu-xenial:~$ ls -la total 84444 drwxr-xr-x 6 vagrant vagrant 4096 Mar 11 05:17 . drwxr-xr-x 4 root root 4096 Mar 11 04:59 .. -rw-r--r-- 1 vagrant vagrant 220 Mar 9 08:35 .bash_logout -rw-r--r-- 1 vagrant vagrant 3771 Mar 9 08:35 .bashrc drwx------ 2 vagrant vagrant 4096 Mar 11 05:00 .cache drwxr-xr-x 1 vagrant vagrant 68 Mar 11 04:31 kafka -rw-rw-r-- 1 vagrant vagrant 49766096 Mar 5 19:00 kafka_2.11-1.0.1.tgz drwxrwxr-x 2 vagrant vagrant 4096 Mar 11 05:17 .oracle_jre_usage -rw-r--r-- 1 vagrant vagrant 655 Mar 9 08:35 .profile drwx------ 2 vagrant vagrant 4096 Mar 11 05:00 .ssh
Unzip the Kafka download as follows (using the filename for the version of Kafka that YOU downloaded–it might be different than the one listed in this tutorial):
$ tar -xzf kafka_2.11-1.0.1.tgz
This will unzip the contents of the file into a directory named kafka_2.11-1.0.1. cd into that directory and then you can start up a Kafka server instance as follows:
$ cd kafka_2.11-1.0.1 $ bin/kafka-server-start.sh config/server.properties &
Congratulations! Assuming all has gone well, you should now have a Kafka server up and running. If you’d like to do something basic with it, pick up at Step 3 of the Quickstart on the Kafka website.
I’m still learning about what Kafka is and what can be done with it. When I find out, I’ll blog about it here!
Photo Credit:
TED-Ed “What makes something ‘Kafkaesque’?” by Noah Tavlin on YouTube