in Uncategorized

Installing Kafka and Zookeeper on Vagrant Ubuntu 16.04

Step 1–Set up Ubuntu 16.04 Virtual Machine

First, follow steps 1 through 3 in my previous tutorial. NOTE: in your Vagrantfile you may need to use /usr/vagrant/api as the directory instead of /usr/ubuntu/api. I’m not sure why, but some installs use vagrant as the root user, and others use ubuntu. I’m sure there’s some reason why, but I haven’t figured it out.

Step 2–Install the latest JDK

Second, log into your box by typing vagrant ssh on the command line and follow these instructions for installing the latest JDK. (Or these.)

Step 3–Install Zookeeper

Third, install Apache Zookeeper with the following command:

It should automatically start up on port 2181. You can verify this with the following command:

Which should give you output like the following:

Zookeeper is a small, fast “coordination service” that is maintained by Apache. Rather than having everybody (like Hadoop, Kafta, Drill, etc.) develop and include their own coordination service into their distributed software package, Apache decided to build a really high quality, lightweight coordinator (Zookeeper) that can be used by all of these other distributed systems to manage configuration, assure message ordering, etc. for your entire cluster. Here’s an overview, if you’d like to know more.

Step 4

Fourth, go to the Kafka downloads page. Click on the first binary version available, next to where it says Scala 2.11 (currently version 1.0.1). At the top of the page that comes up there will be a link after the words: “We suggest the following mirror site for your download:” Right click and copy the link address for your download. Then in your terminal, type wget and then a space and then paste the URL you just copied, and then hit Enter. For example:

Once it’s done, if you view the contents of the directory, it should look something like this:

Unzip the Kafka download as follows (using the filename for the version of Kafka that YOU downloaded–it might be different than the one listed in this tutorial):

This will unzip the contents of the file into a directory named kafka_2.11-1.0.1. cd into that directory and then you can start up a Kafka server instance as follows:

Congratulations! Assuming all has gone well, you should now have a Kafka server up and running. If you’d like to do something basic with it, pick up at Step 3 of the Quickstart on the Kafka website.

I’m still learning about what Kafka is and what can be done with it. When I find out, I’ll blog about it here!

Photo Credit:

TED-Ed “What makes something ‘Kafkaesque’?” by Noah Tavlin on YouTube

Write a Comment

Comment