What is Nutch 1.10 crawl command for elasticsearch -

-

using nutch 1.10 (newbie), trying learn how crawl using nutch 1.10 , using elasticsearch indexer. not sure why, can not crawl command work:

bin/crawl -i --elastic -d elastic.server.url=http://localhost:9200/elastic/ urls elastictestcrawl 1

update: used

bin/crawl -i -d elastic.server.url=http://localhost:9200/elastic/ urls/ elastictestcrawl/  2

--almost succesfully, received following error when came indexing part of command:

error running:   /home/david/apache-nutch-1.10/bin/nutch clean -delastic.server.url=http://localhost:9200/elastic/ elastictestcrawl//crawldb failed exit value 255.

what exit value 255 nutch 1.x? , why space deleted between "-d , elastic..."

i have these elasticsearch properties here in nutch-site.xml file:

if can point error of ways, great!

update posted own answer below, second one. had accepted first answer months ago when got working. answer more clear , concise make easier (and quicker) started nutch.

unfortunately can't tell you're going wrong i'm in same boat although can see running nutch , elastic on same box i've split across two.

i've not got work according guide found on integrating nutch 1.7 elastic should 

bin/crawl urls/ testcrawl -depth 3 -topn 5

it may isn't working me because i've added complication of networking.

i assume have created index called elastictestindex in elastic instance , launched on box before trying run crawl?

should of guide got command is

https://www.mind-it.info/integrating-nutch-1-7-elasticsearch/

update:

i'm not sure i'm quite there yet using update i've got further had.

you putting in port 9200 web administartion port need use port 9300 interact service change port 9300

i'm not sure thing portion after slash refers index in example make sure have "elastic" set index. or change

blah (low rep score can't put in many urls) blah localhost:9300/[index name]/

so uses , index have created. if haven't created 1 can putty following command.

curl -xput 'http://localhost:9200/[index name]/'

using command supplied alternative port did run although i've yet extract crawl data elastic.

supplemental update:

it's dumping data crawled nutch elastic me , having put different index in on command line can tell ignores , uses ever in nutch-site.xml


Comments

Post a Comment

Popular posts from this blog

yii2 - Yii 2 Running a Cron in the basic template -

-
i have seen instructions on running cron in advanced template, cant figure out how on basic template. have following controller in basic controllers folder. <?php namespace app\controllers; use yii\console\controller; /** * cron controller */ class croncontroller extends controller { public function actionindex() { echo "cron service runnning"; } public function actionmail($to) { echo "sending mail " . $to; } } i have navigated root of application , tried these comands yii cron php yii cron im getting unknown comand "cron" i know old question failed find actual answer this. if @ code yii exec file you'll notice requires /common/config/aliases.php file console/config/main.php. can naturally change these or copy on these advanced template. hope solves problem wanting same thing.
Read more

asp.net - 'System.Web.HttpContext' does not contain a definition for 'GetOwinContext' Mystery -

-
i realize question might seem trivial some, it's these types of things find myself fighting quite bit , want make sense of despite seeming losing battle in .net (for me anyway). so, if following: using system.web; ... applicationuser user = system.web.httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); that produces error in title , red getowincontext() , error cannot resolve symbol 'getowincontext()' however, if following (remove system.web in front of httpcontext ), works expected (or @ least no errors): using system.web; ... applicationuser user = httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); however, if (same line that's working using system.web commented out): //using system.web; ... applicationuser user = httpcontext.getowincontext().getusermana
Read more

php - How do you embed a video into a custom theme on WordPress? -

-
i'm using wordpress 4.2.3 , created custom theme use. when try embed youtube video (pasting http:// link content area), video shows in edit content area, when view site, shows text url instead. i tried sorts of things clear formatting, removing link, unchecking settings in writing area, etc. temporarily switched default wp theme , video shows up, i'm missing in template code, , can't figure out what. here code used outputting content in template. index.php <div class="video"> <?php $post_id = 26; $queried_post = get_post($post_id); echo $queried_post->post_content; ?> </div> thanks - appreciate it!
Read more