How MapReduce Has Become YARN In Hadoop?

MapReduce has been changed completely in last few months and now it has become YARN (Yet Another Resource Navigator). So what should be used for your business either MapReduce or YARN? Which is better platform to manage Hadoop application software?

Hadoop Application Software

Resource Manager is one of the very important components available in YARN assures profits for your business. Further Resource Manager is divided into two parts – Scheduler and Application Manager.

When applications are in running state, the job of scheduler is to allocate resources in proper way. The just schedules resources, it does not monitors or tracks the Hadoop Application status. Also it does not make sure that tasks will be restarted once they are failed. The reason for failure of app may be hardware limitation, software problems or other resource allocation issues.

The scheduler generally completes its task based on application requirements and ignores other components usually that include CUP, memory, network, disk usage etc.  Next comes to Application Manager that manages task like monitoring progress, reporting status etc. Further resource manager is also clubbed up with high availability feature to make it more suitable for the job.

Resource Manager RM restart

Resource Manager Restart is a feature that enhances functionalities of resource manager to keep them active all the time. It is further divided into two phases – P1 and P2. P1 or phase 1 restores previous state where resource manager stopped working. It synchronizes overall process of Hadoop application or Hadoop software.

Phase 2 or P2 reconstructs running state of resource manager. It means application will not lose its productivity when it will be restarted.

  • Phase 1 Restart

The overall concept is that RM restart will resume metadata of Hadoop application and also stores final state of an application. Besides the final state, it also stores application credential like security key, code etc. It picks up metadata and resubmit the application. Simple resource manager cannot resubmit the application.

  • Phase 2 Restart

In Phase 2 also, RM restart will resume metadata of Hadoop application and also stores final state of an application. Besides the final state, it also stores application credential like security key, code etc. It picks up metadata and resubmit the application. Simple resource manager cannot resubmit the application. After this, it has the capability of preserving the previous state so application work will not lose even if it will be restarted later.

At Aegis, Our Hadoop developers and senior custom software development team has wonderful exposure on latest Hadoop techniques and frameworks. We have completed many successful projects in Hadoop that makes our Hadoop developers most suitable Hadoop development.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s