如何将hadoop编译在windows
A. 镐庝箞鍦╳indows涓婃惌寤篽adoop鐜澧
1銆佷笅杞藉畨瑁匤DK锛屼互鍙奅clipse
鍏蜂綋鐪嬫ら摼鎺ワ细http://blog.csdn.net/weixuehao/article/details/17715793
2銆佹柊寤筼AVA Project
3銆佸幓cloudera缃戠珯涓嬭浇hadoop镄凛DH3u6鐗堟湰銆傚皢璧疯В铡嫔埌win7镄勬煇涓鐩褰
4銆侀変腑OperateHDFS锛屽彸鍑伙纴阃夋嫨钬减roperty钬濓纴寮瑰嚭鍙充晶镄勭獥鍙ｏ绂
阃夋嫨钬淛ava Build Path钬 钬斺 钬淟ibraries钬 钬斺 Add External Jars
5銆佸埌涓婇溃镄勭敾闱㈠悗锛屼笅闱㈠氨鏄阃夋嫨寮鍙慼adoop绋嫔簭镞讹纴鐢ㄥ埌镄勪竴浜沯ar鍖
灏唋ib鏂囦欢澶逛笅镄勬墍链夌殑jar鍖咃纴娣诲姞杩涘幓锛涘皢绾㈡嗗唴镄刯ar鍖咃纴瀹夎呰繘铡汇
6銆佸垱寤洪厤缃鏂囦欢澶
7銆佸皢core-site.xml,hdfs-site.xml,mapred-site.xml鏀惧叆鍒版ゆ枃浠跺す涓銆
8銆佸彸阌->灞炴 杩涜屽备笅镎崭綔锛屾坊锷犲垰鍒氩垱寤篶onf鏂囦欢澶
閰岖疆鎼炲畾锛
鍐欎唬镰佺殑杩囩▼涓瑕佸啓鍏ョ殑namenode鍜宩obtracker鍦板潃(淇鏀逛负浣犺嚜宸辩殑)
鍦ㄤ綘镄勪唬镰佷腑锛屽姞鍏ュ备笅浠ｇ爜灏卞彲浠ヤ简
Configuration conf = new Configuration()锛
conf.set("fs.default.name","192.1.1.2:9000");
conf.set("mapred.job.tracker","192.1.1.2:9001");
B. 鍦╓indows涓婅繍琛孒adoop锛岄厤缃濂络ava锛宑ygwin锛宧adoop涔嫔悗锛岃佹庝箞锅
鐜澧冨强杞浠跺嗳澶囷细
win7锛64浣嶏级
cygwin 1.7.9-1
jdk-6u25-windows-x64.zip
hadoop-0.20.2.tar.gz
1.瀹夎卝dk锛屽苟缃甹ava鐜澧冨彉閲忓寘𨰾锛钦AVA_HOME锛孭ATH锛孋LASSPATH
2.瀹夎匟adoop锛岀増链涓0.20.2锛屾垜鏄鐩存帴鏀惧埌/home鐩褰曚笅锛屽苟瑙ｅ帇
tar 钬搝xvf
hadoop-0.20.2.tar.gz
3.閰岖疆Hadoop锛岄渶瑕佷慨鏀筯adoop镄勯厤缃鏂囦欢锛屽畠浠浣崭簬conf瀛愮洰褰曚笅锛屽垎鍒鏄痟adoop-env.sh銆乧ore-site.xml銆乭dfs-site.xml
鍜宫apred-site.xml
(1) 淇鏀筯adoop-env.sh锛
鍙闇瑕佸皢java_HOME 淇鏀规垚JDK 镄勫畨瑁呯洰褰曞嵆鍙
export
JAVA_HOME=/cygdrive/d/java/jdk1.6.0_25
锛堟敞镒忥细璺寰勪笉鑳芥槸windows 椋庢牸镄勭洰褰昫:\java\jdk1.6.0_25锛岃屾槸LINUX
椋庢牸/cygdrive/d/java/jdk1.6.0_25锛
(2) 淇鏀筩ore-site.xml锛氾纸鎸囧畾namenode锛
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
(3)淇鏀筯dfs-site.xml锛堟寚瀹氩坛链涓1锛
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
(4) 淇鏀筸apred-site.xml 锛堟寚瀹歫obtracker锛
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
4.楠岃瘉瀹夎呮槸钖︽垚锷燂纴骞惰繍琛孒adoop
(1) 楠岃瘉瀹夎
$ bin/hadoop
Usage: hadoop [--config confdir] COMMAND
where COMMAND is one of:
namenode -format format the DFS filesystem
secondarynamenode run the DFS secondary namenode
namenode run the DFS namenode
datanode run a DFS datanode
dfsadmin run a DFS admin client
mradmin run a Map-Rece admin client
fsck run a DFS filesystem checking utility
fs run a generic filesystem user client
balancer run a cluster balancing utility
jobtracker run the MapRece job Tracker node
pipes run a Pipes job
tasktracker run a MapRece task Tracker node
job manipulate MapRece jobs
queue get information regarding JobQueues
version print the version
jar <jar> run a jar file
distcp <srcurl> <desturl> file or directories recursively
archive -archiveName NAME <src>* <dest> create a hadoop archive
daemonlog get/set the log level for each daemon
or
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.
(2) 镙煎纺鍖栧苟钖锷℉adoop
bi
C. 如何在windows下安装Eclipse Hadoop插件
在windows下安装Eclipse Hadoop插件主要安装编译Eclipse-hadoop插件即可。
1、配置环境变量：把下载好的hadoop-2.5.2解压到D盘的根目录下，然后计算机==》属性==》高级系统设置==》环境变量
