Why Java Developers Need Java Messaging System?

This post shares info on Java Messaging System. Java developers are using it in their practice for several reasons. You can read the post and find the reasons.

Why messaging system?

In the electronics world, data is the actual term used for all information. For which data, “Message” term can be used? Let’s take simple scenario, if you are registering your details in the web portal, it takes all your information and store entire data in relational database which will be called as a Data. Once you registered successfully, an email functionality triggers and send an email to you saying that account is successfully created. This email event is called as messaging.

In traditional system, both registering new user and sending email are tightly coupled i.e. both are in same application. This design is extremely responsible for the system’s performance. Imagine 100s of user registering in 15 minutes, the same application should focus creating new user and sending email. This process is too costly. To avoid this issue, messaging systems are come into picture.

 It runs independently and just looks up for the message and doesn’t care about where messages are coming from. If the message arrives, it consumes the message and sends it to destination. This is called as loosely coupled and both applications run independently.

The responsibility of Web Portal is to create new user and to send email to user is messaging system’s responsibility. Messaging system is not only used for sending email. There are many scenarios where we need Messaging system such as notification, news feed etc.

There are many messaging system in the market. Rabbit MQ is one of such popular messaging systems in Java, which includes three roles – publisher, subscriber and queue.

Publisher is the one who sends the message. The sent message will store in temporally in a buffer called queue. Subscriber is the one who receives the message. The Rabbit MQ is having all these players that are required for messaging service.

Why we need Apache Kafka?

Since there are more matured messaging systems available in the world, why we need Apache Kafka? That’s a question that triggers developer’s mind.

The other messaging systems handle only low volume data. But Rabbit MQ handles 20K messages per second. Apache kafka handle 100K messages per second. Kafka provides easy way to create many kafka clusters which is called as broker. This is for maintaining load balance. If one broker gets down, the other automatically takes care.

Apache kafka persists the messages in hard disk and java developers can set life time of message.

Following players are included in the Apache Kafka in messaging system-

  • Topics
  • Producer
  • Brokers
  • Consumer

Topics

Topics hold the stream of messages. For example Topic “A” can hold the message “Hello World”. If we create a topic, kafka creates by default one partition. Partition contains sequence of index in which messages are get stored. Based on our requirement we can create many partitions in Topics as in the diagram.

Topic “A”

Partition 1
0
1
2
3
4
5
Partition 2
0
1
2
Partition 3
0
1

Producer

Producer uses the topics to send the messages to broker.

Consumer

Consumer receives the messages from broker.

Broker

Broker acts like a middleware component for producer and consumer. If producer sends messages, those messages are maintained in broker. Consumers consume messages from broker.

Using kafka cluster we can create multiple brokers which maintain the messages and load balance. The beauty of Kafka broker is that it can handle many read/writes manipulation without any performance tuning.

The entire Kafka cluster runs in Zookeeper.

What is zookeeper?

Zookeeper is an environment that maintains all Kafka broker and its state. If any Kafka broker is down or not available, Zookeeper is liable to notify Consumer/Producer, so Consumer/Producer will start looking into another available Broker.

System diagram for Kafka messaging system:

Kafka messaging system environment

Kafka messaging system environment

Who are Leaders and Followers?

I have already told that we can create many Kafka brokers in which one Kafka broker is primary broker that handles all manipulation between Producer and Consumer. This Kafka is called as Leader and other Kafka brokers are called as followers. Due to some situation the leader Kafka is down or shut down for any reason, zookeeper selects a new leader from available follower, and notifies to Consumer/Producer about the new Kafka leader. Consumer/Producer starts looking into correct Kafka broker.

Kafka messaging system setup

Download zookeeper from following Apache link

http://www.apache.org/dyn/closer.cgi/zookeeper/

After download go to download folder and open conf folder

zookeeper-3.4.6\conf

You can see zoo_sample.cfg

Copy the this file as zoo.cfg

Open zoo.cfg

Change following property with correct path.

Data Dir=<Your path>

Zookeeper store all data in this path

You have done with Zookeeper configuration and this is a time start zookeeper server.

3

Double click highlighted file(zkServer) to start the zookeeper. You can see following console after starting the Zookeeper successfully.

4

Now we have configured Zookeeper for your Kafka server.

Download Kafka from following apache link

https://www.apache.org/dyn/closer.cgi?path=/kafka/0.9.0.0/kafka_2.11-0.9.0.0.tgz

Extract the Kafka download folder as in below screenshot.

5

Go to bin folder and apply following command to start Kafka server.

kafka-server-start F:\Work\kafka\kafka\kafka_2.11-0.9.0.0\config\server.properties

This command starts your Kafka server.

If you want to stop kafka server, go to kafka_2.11-0.9.0.0\bin\windows and click kafka-server-stop

If you are using Linux, go to kafka_2.11-0.9.0.0\bin\

Now we have kafka server that is ready to start. Once you started the kafka server, you can see following console

6

Kafka Producer

publicclass MyProducer

{

                publicstaticvoid main(String[] args) throws Exception{

                                Properties props = new Properties();

                                props.put("bootstrap.servers", "localhost:9092");

                                props.put("acks", "all");

                                props.put("retries", 0);

                                props.put("batch.size", 16384);

                                props.put("buffer.memory", 33554432);

                                props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");

                                props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

                                 Producer<String, String>producer = new KafkaProducer(props);

                                for(inti = 0; i< 5; i++) {

                                                producer.send(new ProducerRecord<String, String>("test-topic", "Test-Key " + (i+1) ,"Test-Value" + (i+1)));

                                                }

                                producer.close();

    }

}

The above code sets necessary configuration for Kafka producer and creates a new topic called “test-topic” which carries the five different messages. This topic message is sent by producer using producer.send method.

Kafka Consumer

publicstaticvoid main(String[] args) {

                //Kafka Consumer configuration

 Properties props = new Properties();

 props.put("bootstrap.servers", "localhost:9092");

 props.put("group.id", "test");

 props.put("enable.auto.commit", "true");

 props.put("auto.commit.interval.ms", "1000");

 props.put("session.timeout.ms", "30000");

 props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

                props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

                      KafkaConsumer<String, String>consumer = new KafkaConsumer(props);

                consumer.subscribe(Arrays.asList("test-topic"));

while(true){

         ConsumerRecords<String, String>records = consumer.poll(100);

                                for (ConsumerRecord<String, String>record : records)

                                                  System.out.println("Partition Offset = " + record.offset() + " Key = " +  record.key() + " Value = " + record.value());

                                  }

}

In the above code sets, the necessary configuration for kafka Consumer and receiving messages from a producer. The Producer carries the messages via topic called “test-topic”.

Boostrap.servers

It will start up all available brokers in the kafka

Group.id

We can set group name for multiple consumer. In our case our consumer belongs to test group.

enable.auto.commit

It will commit the offset value based on auto.commit.interval.ms

session.timeout.ms

Using this timeout we can find out any failures in Kafka server.

Key.deserializer and value.deserializer

This is for internal string serialization process for message data.

Acks

This property is having three possible value which are 0, 1 and all. If set 0, producer doesn’t wait for acknowledgement. If set 1, producer accepts ack from leader, if set all, producer accepts ack from all replica i.e. if your kafka server is having more than 1 broker, it accepts the ack from all (Both leader and follower).

Retries

Number of try for sending messages to Consumer

batch.size

Buffer size for batch size. Let say producer is sending multiple messages to a partition, he will use the batch.size.

buffer.memory

The buffer.memory used by producer for sending messages.

Hope you have understood what Java Messaging System is and why java developers use it. If you have any question, ask from experts and get answered.

Advertisements

How To Protect Your Intellectual Property While Outsourcing Java Development Projects

Intellectual assets or property of a person or a company hold many intangible things that are usually not visible through eyes, but valuable to someone. It includes trade secrets, designs, artistic work, trademarks, copyrights, concepts, company’s data, etc.

Companies encounter heavy competition and thus, they all need to protect their secret data from competitors and other third parties. This is why it is crucial to protect all your intellectual property while planning for outsourcing development project.

The requirement for outsourcing web development is rising with time and especially .net, PHP, and java development outsourcing have emerged as the significant and often outsourced web technologies. Protecting intellectual property is the concern for companies and businesses.

Many times businesses have caught unethical companies that have been misusing the technologies and intellectual property of clients. How the enterprises can protect their secret data and keep it safe forever?

Intellectual property

Here Are Few Tips That Will Help You Out

  • Well documented information security management policy

You must enquire about the information security management policy of a vendor prior providing your project details to them. There is a specific set of rules and regulations that every professional and vendor follows while handling the secret data of their clients. In case you find that the service provider is not taking the policy seriously or there is no policy, then you must look for another IT development firm.

Genuine vendor will provide project development policy and almost every vendor has a system in which every member of the company needs to sign a confidential agreement. You can search such vendor for your project.

  • Never explain the whole project, provide them required information only

This is one of the best ways to lower the risk of confidential property theft. You can set a limit on information you share with company regarding your project.

  • Source code and software ownership

It is important to clarify the ownership of the source code before they start with the project. There are companies and developers who agree to provide complete ownership of the source code and software, but then there are vendors exploit situations where clients don’t make first move towards discussion for the topic.

It hardly matters if your requirement is large or small; you should understand that safety of your data is critical thing that should not be compromised. When you think of hiring java development outsourcing vendor, you should select the one who meets all your requirements and above tips.

“Hire Java Developers, Superheroes In Their Profession”

Corporate segments these days are plagiarizing themselves in extracting hidden talent even more. Whether you are trying to construct anything, if I talk about apps, websites and many other functions, every requirement has a code.

java superheroOnce an engineer stated that ‘We believe in constructing codes through brilliant minds’, so all I can say is that coding has now become the part and parcel of our lives. Today software engineers can write series of programs and generate heavy salaries per month. This is what has inspired them to live create innovation in the way coding is done. Yes, although it requires talents to generate codes, but I will say that ‘Now coding has given a sip of coffee to drink’. Least you can go through the online job portals you’ll surely find 10 openings only for the Java developers as compared to any other job postings.

Well, these days, hiring Java developers has become a prime requirement of every leading organization because they require talents to give desired outcomes. And engineers too are technological savvy because all they do think is that they pick challenging jobs to settle abroad and live a technologically-oriented luxurious life. There are several rules that need to be considered while hiring Java developers and they are as follows:

  • Hire the top graded candidates: Yes, this rule states that you can hire the well experienced professionals who are passionate about creating codes in particular format.
  • Java developers are superheroes in their profession: Yes, Java developers are the superheroes of their programming firms. They can create a team of superheroes that would again help in gaining results from the extremely talented minds.
  • Hire only talented individuals: Java professionals seeking interviews should hire only talented individuals who are passionate about coding. Well, with the growing population of engineers in India, a half of the job postings will be reserved only for the Java developers as compared to any other job openings.
  • Be your own small organization: You can seek advantage of this, because large Multinational organizations seek to outsource work to the small firms who have a passion to create codes and who have talented professionals who can be multitaskers too.
  • It’s more on work: Anything that you do requires passion and the abilities to stand firmly on your toes. Yes, it’s a bit difficult but your only passion for your work can lead you to create rainbows.
  • Open source platform invites opportunities: Yes, open source platform invites new talents and also draws new opportunities and also you get overwhelming responses from the applicants. Once, you start posting your job requirements, this is the stage when you start receiving pool of applications from the interested peers. Well, this is also an opportunity for you to be recognized and also to be known by all.

There are many places in which individuals can apply for their passionate jobs. But, this field of hiring Java developers occupies a very special position in the heart of all the passed out engineers.

Save