如何將hadoop編譯在windows
A. 鎬庝箞鍦╳indows涓婃惌寤篽adoop鐜澧
1銆佷笅杞藉畨瑁匤DK錛屼互鍙奅clipse
鍏蜂綋鐪嬫ら摼鎺ワ細http://blog.csdn.net/weixuehao/article/details/17715793
2銆佹柊寤篔AVA Project
3銆佸幓cloudera緗戠珯涓嬭澆hadoop鐨凜DH3u6鐗堟湰銆傚皢璧瘋В鍘嬪埌win7鐨勬煇涓鐩褰
4銆侀変腑OperateHDFS錛屽彸鍑伙紝閫夋嫨鈥減roperty鈥濓紝寮瑰嚭鍙充晶鐨勭獥鍙ｏ紱
閫夋嫨鈥淛ava Build Path鈥 鈥斺 鈥淟ibraries鈥 鈥斺 Add External Jars
5銆佸埌涓婇潰鐨勭敾闈㈠悗錛屼笅闈㈠氨鏄閫夋嫨寮鍙慼adoop紼嬪簭鏃訛紝鐢ㄥ埌鐨勪竴浜沯ar鍖
灝唋ib鏂囦歡澶逛笅鐨勬墍鏈夌殑jar鍖咃紝娣誨姞榪涘幓錛涘皢綰㈡嗗唴鐨刯ar鍖咃紝瀹夎呰繘鍘匯
6銆佸壋寤洪厤緗鏂囦歡澶
7銆佸皢core-site.xml,hdfs-site.xml,mapred-site.xml鏀懼叆鍒版ゆ枃浠跺す涓銆
8銆佸彸閿->灞炴 榪涜屽備笅鎿嶄綔錛屾坊鍔犲垰鍒氬壋寤篶onf鏂囦歡澶
閰嶇疆鎼炲畾錛
鍐欎唬鐮佺殑榪囩▼涓瑕佸啓鍏ョ殑namenode鍜宩obtracker鍦板潃(淇鏀逛負浣犺嚜宸辯殑)
鍦ㄤ綘鐨勪唬鐮佷腑錛屽姞鍏ュ備笅浠ｇ爜灝卞彲浠ヤ簡
Configuration conf = new Configuration()錛
conf.set("fs.default.name","192.1.1.2:9000");
conf.set("mapred.job.tracker","192.1.1.2:9001");
B. 鍦╓indows涓婅繍琛孒adoop錛岄厤緗濂絡ava錛宑ygwin錛宧adoop涔嬪悗錛岃佹庝箞鍋
鐜澧冨強杞浠跺噯澶囷細
win7錛64浣嶏級
cygwin 1.7.9-1
jdk-6u25-windows-x64.zip
hadoop-0.20.2.tar.gz
1.瀹夎卝dk錛屽苟緗甹ava鐜澧冨彉閲忓寘鎷錛欽AVA_HOME錛孭ATH錛孋LASSPATH
2.瀹夎匟adoop錛岀増鏈涓0.20.2錛屾垜鏄鐩存帴鏀懼埌/home鐩褰曚笅錛屽苟瑙ｅ帇
tar 鈥搝xvf
hadoop-0.20.2.tar.gz
3.閰嶇疆Hadoop錛岄渶瑕佷慨鏀筯adoop鐨勯厤緗鏂囦歡錛屽畠浠浣嶄簬conf瀛愮洰褰曚笅錛屽垎鍒鏄痟adoop-env.sh銆乧ore-site.xml銆乭dfs-site.xml
鍜宮apred-site.xml
(1) 淇鏀筯adoop-env.sh錛
鍙闇瑕佸皢java_HOME 淇鏀規垚JDK 鐨勫畨瑁呯洰褰曞嵆鍙
export
JAVA_HOME=/cygdrive/d/java/jdk1.6.0_25
錛堟敞鎰忥細璺寰勪笉鑳芥槸windows 椋庢牸鐨勭洰褰昫:\java\jdk1.6.0_25錛岃屾槸LINUX
椋庢牸/cygdrive/d/java/jdk1.6.0_25錛
(2) 淇鏀筩ore-site.xml錛氾紙鎸囧畾namenode錛
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
(3)淇鏀筯dfs-site.xml錛堟寚瀹氬壇鏈涓1錛
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
(4) 淇鏀筸apred-site.xml 錛堟寚瀹歫obtracker錛
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
4.楠岃瘉瀹夎呮槸鍚︽垚鍔燂紝騫惰繍琛孒adoop
(1) 楠岃瘉瀹夎
$ bin/hadoop
Usage: hadoop [--config confdir] COMMAND
where COMMAND is one of:
namenode -format format the DFS filesystem
secondarynamenode run the DFS secondary namenode
namenode run the DFS namenode
datanode run a DFS datanode
dfsadmin run a DFS admin client
mradmin run a Map-Rece admin client
fsck run a DFS filesystem checking utility
fs run a generic filesystem user client
balancer run a cluster balancing utility
jobtracker run the MapRece job Tracker node
pipes run a Pipes job
tasktracker run a MapRece task Tracker node
job manipulate MapRece jobs
queue get information regarding JobQueues
version print the version
jar <jar> run a jar file
distcp <srcurl> <desturl> file or directories recursively
archive -archiveName NAME <src>* <dest> create a hadoop archive
daemonlog get/set the log level for each daemon
or
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.
(2) 鏍煎紡鍖栧苟鍚鍔℉adoop
bi
C. 如何在windows下安裝Eclipse Hadoop插件
在windows下安裝Eclipse Hadoop插件主要安裝編譯Eclipse-hadoop插件即可。
1、配置環境變數：把下載好的hadoop-2.5.2解壓到D盤的根目錄下，然後計算機==》屬性==》高級系統設置==》環境變數
