Hadoop fJgĂяoărbOf[^

1. Tv

{txt`̃t@CǂݍŁAo
gppxJEgA悭gĂꂩ\[g
\B
󕶌ɂɓo^Ă鏬Ȃǂ̒̑̕Ŝ
łB


2. t@Cꗗ

bocchan.txt		Ėڟ΁uVvi󕶌ɂj
get_httpstring.dec	lbg[N͂̕擾
outputs.str		f[^
Readme.txt		̃t@C
teniwoha.dec		`ԑf̓vO(map)
teniwoha.sh		HadoopgȂsVF
teniwoha-hadoop.sh	Hadoopgs
teniwoha-http.sh	lbg[N͂̕𕪐͂鏈
teniwoha-http-hadoop.sh lbg[N͂̕hadoopŕ͂鏈
tmpfile.str		f[^ꎞt@C
wsort-reduce.dec	`ԑf͌ʂWvvO(reduce)


2. HadoopgȂ

̂悤ɁAɕt@Cw肵ĎsB

$ sh ./teniwoha.sh bocchan.txt

ʂoutputs.strt@CɕۑB

sɂ͎gpΏەt@Cɂ邪A\ȏォB


3. Hadoopg

HadooṕAOɎ̊ɃCXg[ĂB

teniwoha-hadoop.sh̏ɂϐ̒lÅ
킹ĕύXB

export JAVA_HOME=/usr
export HADOOP_HOME=${HOME}/work/hadoop.d
export HADOOP_STREAMING=${HADOOP_HOME}/hadoop-1.0.3/contrib/streaming/hadoop-streaming-1.0.3.jar

JAVA_HOMÉAJAVÃCXg[ĂfBNg
HADOOP_HOMÉAHadoop̃CXg[ĂfBNg
HADOOP_STREAMINǴAHadoop̒hadoop-streaming-*.jar̃t@C

̂悤ɁAɕt@Cw肵ĎsB

$ sh ./teniwoha-hadoop.sh bocchan.txt

ʂoutputs.strt@CɕۑB

sԂ́AHadoop̃NX^(m[hAm[h̐\)
ςB


4. Network(WWW)͂̕𕪐͂鏈

̂悤ɁAɕt@Cw肵ĎsB

$ sh ./teniwoha-http.sh http://news.google.co.jp/

ʂoutputs.strt@CɕۑB

5. Network(WWW)͂̕hadoopgĕ͂鏈

̂悤ɁAɕt@Cw肵ĎsB

$ sh ./teniwoha-http-hadoop.sh http://news.google.co.jp/

ʂoutputs.strt@CɕۑB

ȏ


