hadoop - YARN-Cgroups : Failed to initialize container executor in non-secure cluster -

-

i trying use cgroups yarn 2.6.0 in non-secure mode. works fine if use defaultcontainerexecutor. but, error when try use linuxcontainerexecutor.

now, when execute --> $ yarn nodemanager, fails

exitcodeexception exitcode=24: file /home/hduser2/hadoop/hadoop-2.6.0/etc/hadoop must owned root, owned 1001      @ org.apache.hadoop.util.shell.runcommand(shell.java:538)     @ org.apache.hadoop.util.shell.run(shell.java:455)     @ org.apache.hadoop.util.shell$shellcommandexecutor.execute(shell.java:715)     @ org.apache.hadoop.yarn.server.nodemanager.linuxcontainerexecutor.init(linuxcontainerexecutor.java:181)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.serviceinit(nodemanager.java:209)     @ org.apache.hadoop.service.abstractservice.init(abstractservice.java:163)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.initandstartnodemanager(nodemanager.java:462)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.main(nodemanager.java:509) 15/08/08 23:07:39 info nodemanager.containerexecutor:  15/08/08 23:07:39 info service.abstractservice: service nodemanager failed in state inited; cause: org.apache.hadoop.yarn.exceptions.yarnruntimeexception: failed initialize container executor org.apache.hadoop.yarn.exceptions.yarnruntimeexception: failed initialize container executor     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.serviceinit(nodemanager.java:211)     @ org.apache.hadoop.service.abstractservice.init(abstractservice.java:163)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.initandstartnodemanager(nodemanager.java:462)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.main(nodemanager.java:509) caused by: java.io.ioexception: linux container executor not configured (error=24)     @ org.apache.hadoop.yarn.server.nodemanager.linuxcontainerexecutor.init(linuxcontainerexecutor.java:187)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.serviceinit(nodemanager.java:209)     ... 3 more caused by: exitcodeexception exitcode=24: file /home/hduser2/hadoop/hadoop-2.6.0/etc/hadoop must owned root, owned 1001      @ org.apache.hadoop.util.shell.runcommand(shell.java:538)     @ org.apache.hadoop.util.shell.run(shell.java:455)     @ org.apache.hadoop.util.shell$shellcommandexecutor.execute(shell.java:715)     @ org.apache.hadoop.yarn.server.nodemanager.linuxcontainerexecutor.init(linuxcontainerexecutor.java:181)     ... 4 more 15/08/08 23:07:39 warn service.abstractservice: when stopping service nodemanager : java.lang.nullpointerexception java.lang.nullpointerexception     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.stoprecoverystore(nodemanager.java:161)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.servicestop(nodemanager.java:273)     @ org.apache.hadoop.service.abstractservice.stop(abstractservice.java:221)     @ org.apache.hadoop.service.serviceoperations.stop(serviceoperations.java:52)     @ org.apache.hadoop.service.serviceoperations.stopquietly(serviceoperations.java:80)     @ org.apache.hadoop.service.abstractservice.init(abstractservice.java:171)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.initandstartnodemanager(nodemanager.java:462)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.main(nodemanager.java:509) 15/08/08 23:07:39 fatal nodemanager.nodemanager: error starting nodemanager org.apache.hadoop.yarn.exceptions.yarnruntimeexception: failed initialize container executor     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.serviceinit(nodemanager.java:211)     @ org.apache.hadoop.service.abstractservice.init(abstractservice.java:163)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.initandstartnodemanager(nodemanager.java:462)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.main(nodemanager.java:509) caused by: java.io.ioexception: linux container executor not configured (error=24)     @ org.apache.hadoop.yarn.server.nodemanager.linuxcontainerexecutor.init(linuxcontainerexecutor.java:187)     @ org.apache.hadoop.yarn.server.nodemanager.nodemanager.serviceinit(nodemanager.java:209)     ... 3 more caused by: exitcodeexception exitcode=24: file /home/hduser2/hadoop/hadoop-2.6.0/etc/hadoop must owned root, owned 1001      @ org.apache.hadoop.util.shell.runcommand(shell.java:538)     @ org.apache.hadoop.util.shell.run(shell.java:455)     @ org.apache.hadoop.util.shell$shellcommandexecutor.execute(shell.java:715)     @ org.apache.hadoop.yarn.server.nodemanager.linuxcontainerexecutor.init(linuxcontainerexecutor.java:181)     ... 4 more 15/08/08 23:07:39 info nodemanager.nodemanager: shutdown_msg:

site specific yarn configuration properties :

<property>         <name>yarn.nodemanager.container-executor.class</name>         <value>org.apache.hadoop.yarn.server.nodemanager.linuxcontainerexecutor</value>  </property> <property>         <name>yarn.nodemanager.linux-container-executor.resources-handler.class</name>         <value>org.apache.hadoop.yarn.server.nodemanager.util.cgroupslceresourceshandler</value>  </property> <property>         <name>yarn.nodemanager.linux-container-executor.cgroups.hierarchy</name>         <value>/hadoop-yarn</value>  </property> <property>         <name>yarn.nodemanager.linux-container-executor.cgroups.mount</name>         <value>true</value>  </property> <property>         <name>yarn.nodemanager.linux-container-executor.cgroups.mount-path</name>         <value>/cgroup</value>  </property> <property>         <name>yarn.nodemanager.linux-container-executor.group</name>         <value>hadoop</value>  </property> <property>         <name>yarn.nodemanager.resource.percentage-physical-cpu-limit</name>         <value>95</value> </property>      <property>             <name>yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage</name>             <value>true</value>     </property>

and container-executor.cfg is:

yarn.nodemanager.linux-container-executor.group=hadoop min.user.id=1000

it great if can please me figure out what’s wrong setup.

your issue can explained in stack-trace here: caused by: exitcodeexception exitcode=24: file /home/hduser2/hadoop/hadoop-2.6.0/etc/hadoop must owned root, owned 1001

the entire path containing container-executor.cfg must owned , writable root.

likely problem can solved chown root:root /home/hduser2/hadoop/hadoop-2.6.0/etc/hadoop though may need continue path changing ownership.


Comments

  1. Unknown15 October 2019 at 03:32

    Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating hadoop training

    ReplyDelete

Post a Comment

Popular posts from this blog

asp.net - 'System.Web.HttpContext' does not contain a definition for 'GetOwinContext' Mystery -

-
i realize question might seem trivial some, it's these types of things find myself fighting quite bit , want make sense of despite seeming losing battle in .net (for me anyway). so, if following: using system.web; ... applicationuser user = system.web.httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); that produces error in title , red getowincontext() , error cannot resolve symbol 'getowincontext()' however, if following (remove system.web in front of httpcontext ), works expected (or @ least no errors): using system.web; ... applicationuser user = httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); however, if (same line that's working using system.web commented out): //using system.web; ... applicationuser user = httpcontext.getowincontext().getusermana...
Read more

yii2 - Yii 2 Running a Cron in the basic template -

-
i have seen instructions on running cron in advanced template, cant figure out how on basic template. have following controller in basic controllers folder. <?php namespace app\controllers; use yii\console\controller; /** * cron controller */ class croncontroller extends controller { public function actionindex() { echo "cron service runnning"; } public function actionmail($to) { echo "sending mail " . $to; } } i have navigated root of application , tried these comands yii cron php yii cron im getting unknown comand "cron" i know old question failed find actual answer this. if @ code yii exec file you'll notice requires /common/config/aliases.php file console/config/main.php. can naturally change these or copy on these advanced template. hope solves problem wanting same thing.
Read more

android - Crash when clicking button with custom theme -

-
i creating custom theme button , using android:onclick event of button xml handle click of button. due reason crashing below exception java.lang.illegalstateexception: not find method myonclick(view) in activity class android.view.contextthemewrapper onclick handler on view class android.widget.button id 'button1' and working fine if remove theme attribute button, below theme button <style name="buttontheme" parent="@android:style/widget.button"> <item name="android:textcolor">#ff0000</item> <item name="android:shadowcolor">#ff000000</item> </style> and button defined in xml below, <button android:id="@+id/button1" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_below="@+id/textview1" android:layout_margin="20dp" android...
Read more