KAFKA

1. Type virtual machines in the search.
2. Under Services, select Virtual machines.
3. In the Virtual machines page, select Add. The Create a virtual machine page opens.
4. In the Basics tab, under Project details, make sure the correct subscription is selected and then choose to Create new resource group. Type myResourceGroup for the name.*.
5. Under Instance details, type myVM for the Virtual machine name, choose East US for your Region, and choose Ubuntu 18.04 LTS for your Image. Leave the other defaults.
6. Under Administrator account, select SSH public key, type your user name, then paste in your public key. Remove any leading or trailing white space in your public key.
7. Under Inbound port rules > Public inbound ports, choose Allow selected ports and then select SSH (22) and HTTP (80) from the drop-down.
8. Leave the remaining defaults and then select the Review + create button at the bottom of the page.
9. On the Create a virtual machine page, you can see the details about the VM you are about to create. When you are ready, select Create.

It will take a few minutes for your VM to be deployed. When the deployment is finished, move on to the next section.

Connect to virtual machine

Create an SSH connection with the VM.

1. Select the Connect button on the overview page for your VM.
2. In the Connect to virtual machine page, keep the default options to connect by IP address over port 22. In Login using VM local account a connection command is shown. Select the button to copy the command. The following example shows what the SSH connection command looks like:

bashCopy

ssh azureuser@10.111.12.123

1. Using the same bash shell you used to create your SSH key pair (you can reopen the Cloud Shell by selecting >_ again or going to https://shell.azure.com/bash), paste the SSH connection command into the shell to create an SSH session.

Usage/Deployment Instruction

Step 1: Access Kafka from Azure Marketplace and click ON Get it now button.

Click on Continue then on Create,

Step 2:Now to create a virtual machine, enter or select appropriate values for zone, machine type, resource group and so on as per your choice.

Click on Create;

Usage and Deployment Instruction:

Step 1: Open Putty for SSH

Step 2: Open Putty and Type <instance public IP> at “Host Name”

Step 3: Open Connection->SSH->Auth tab from Left Side Area 

Step 4: Click on browse button and select ppk file for Instance and then click on Open

Step 5: Type “ubuntu” as user name Password auto taken from PPK file

Step 5.1: If you get any update option from Ubuntu, then you have to follow the following steps:

After then follow the following commands

$ apt-get update

$ apt-get upgrade

$ sudo su

$ cd kafka_2.13-2.8.0

Step 6: Use following Linux command to Start Kafka and Zookeeper

STEP 7: START THE KAFKA ENVIRONMENT

#Start the ZooKeeper service

zookeeper-server-start.sh config/zookeeper.properties

Open another terminal session and run:

#Start the kafka broker service.

kafka-server-start.sh config/server.properties

Once all services have successfully launched, you will have a basic Kafka environment running and ready to use.

STEP 8: CREATE A TOPIC TO STORE YOUR EVENTS

So before you can write your first events, you must create a topic. Open another terminal session and run:

bin/kafka-topics.sh –create –topic quickstart-events –bootstrap-server localhost:9092

STEP 9: WRITE SOME EVENTS INTO THE TOPIC

bin/kafka-console-producer.sh –topic quickstart-events –bootstrap-server localhost:9092

STEP 10: READ THE EVENTS

Open another terminal session and run the console consumer client to read the events you just created:

bin/kafka-console-consumer.sh –topic quickstart-events –from-beginning –bootstrap-server localhost:9092

If you write a new message using the Producer API, it should also be displayed instantly on the Consumer side as shown in the screenshot below.