Zookeeper集群的安装和配置,Zookeeper集群配置

在自己成功了Hadoop集群配置今后(新普金娱乐,http://www.linuxidc.com/Linux/2013-06/86347.htm 卡塔 尔(阿拉伯语:قطر‎就要求安装zookeeper
各样Computer做同样的操作

极度提醒:自个儿博客部分有参照网络其余博客,但均是本人亲手工编织写过并表达通过。如开采博客有荒诞,请立即提议以防误导其余人,感谢!应接转载,但记得标记小说出处:

1. Zookeeper介绍:

1、下载Zookeeper

安装
cd /usr/local/src
tar zxvf zookeeper-3.4.5.tar.gz
mv zookeeper-3.4.5 /usr/local/zookeeper
chown -R hadoop:hadoop /usr/local/zookeeper

原理

ZooKeeper是以法斯特 Paxos算法为根基的,Paxos
算法存在活锁的主题素材,即当有多少个proposer交错提交时,有非常大希望相互排挤引致未有一个proposer能提交成功,而法斯特Paxos作了有的优化,通过大选发生一个leader
(领导者),唯有leader才干交到proposer,具体算法可以预知FastPaxos。因而,要想弄懂ZooKeeper首先得对法斯特 Paxos有所理解。
ZooKeeper的主导运作流程:
1、选举Leader。
2、同步数据。
3、选举Leader进程中算法有不少,但要到达的推选正式是雷同的。
4、Leader要负有最高的施行ID,雷同root权限。
5、集群中山学院部的机器获得响应并follow选出的Leader。

正文将zookeeper在同风华正茂台服务器上做二个伪集群布署(实际临盆条件不会这样安顿卡塔 尔(英语:State of Qatar),zookeeper有个特点正是节点日常是布局单数(3,5,7,9。。。)具体访谈量来衡量,节点down掉四分之二才算zookeeper服务down掉。

1.主干介绍:

Zookeeper:
为分布式应用提供遍及式合作(协调卡塔尔服务。使用相符文件系统的树形结构。目标是遍布式服务不再需求由于同盟冲突而除此以外完成同盟服务。

Zookeeper’s Hierarchical Namespace

Zookeeper集群的安装和配置,Zookeeper集群配置。貌似景况下Zookeeper集群的服务器数量是奇数个,2N+1,
N代表允许宕机的个数,下边小说中介绍使用3台主机实行尝试。

Zookeeper集群统生机勃勃为别的布满式应用(也是集群,比方HDFS集群、Yarn集群等等卡塔尔国集群提供经济同盟服务

2、解压

创制数量目录
mkdir /tmp/zookeeper
chown -R hadoop:hadoop /tmp/zookeeper

1、下载地址

2. Zookeeper角色。

Zookeeper剧中人物不是手工业配置的,而是集群自动大选发生的:

(1卡塔 尔(英语:State of Qatar)Leader:做决定(做哪些决定跟Zookeeper达成的效能有关卡塔尔,更新系统状态

(2卡塔 尔(英语:State of Qatar)Follower:接入客户端。接受顾客端乞求,并重临响应结果给顾客端。

(3)Observer:

(4卡塔 尔(阿拉伯语:قطر‎Client:使用合营服务的布满式应用,比方HDFS集群。

Zookeeper Service

3、步向conf目录,创建文件

校勘配置文件
始建布局文件(注意:以下操作在hadoop顾客中张开卡塔尔国
su – hadoop
cd zookeeper/conf
cp zoo_sample.cfg zoo.cfg

2、解压安装

1、使用cd命令步入/usr/local/application/zookeeper目录,并上传下载好的zookeeper-3.4.6.tar.gz至安装目录,分别解压移动至zookeeper2181、zookeeper2182、zookeeper2183。

tar -zxvf zookeeper-3.4.6.tar.gz
mv zookeeper-3.4.6 zookeeper2181

tar -zxvf zookeeper-3.4.6.tar.gz
mv zookeeper-3.4.6 zookeeper2182

tar -zxvf zookeeper-3.4.6.tar.gz
mv zookeeper-3.4.6 zookeeper2183

rm -rf zookeeper-3.4.6.tar.gz

3.Zookeeper的头名应用项景:

(1卡塔尔统一命名服务(Name Service卡塔 尔(阿拉伯语:قطر‎

(2卡塔尔国配置管理(Configuration Management卡塔尔国

(3卡塔尔国集群处理(Group Membership卡塔 尔(阿拉伯语:قطر‎

(4)共享锁(Locks)/ 同步锁

  zoo.cfg

编排配置文件

3、改善配置

1、修改zookeeper2181/conf/zoo_sample.cfg文件名称叫zoo.cfg,并改善铜绿配置:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.

 
dataDir=/usr/local/application/zookeeper/zookeeper2181/data
 
dataLogDir=/usr/local/application/zookeeper/zookeeper2181/logs

# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=192.168.1.11:4000:5000
server.2=192.168.1.11:4000:5000
server.3=192.168.1.11:4000:5000

2、修改zookeeper2182/conf/zoo_sample.cfg文件名叫zoo.cfg,并改良青绿配置:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/usr/local/application/zookeeper/zookeeper2182/data
dataLogDir=/usr/local/application/zookeeper/zookeeper2182/logs
# the port at which the clients will connect
clientPort=2182
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=192.168.1.11:4000:5000
server.2=192.168.1.11:4000:5000
server.3=192.168.1.11:4000:5000

3、修改zookeeper2183/conf/zoo_sample.cfg文件名字为zoo.cfg,并改善米红配置:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.

 
dataDir=/usr/local/application/zookeeper/zookeeper2181/data
 dataLogDir=/usr/local/application/zookeeper/zookeeper2181/logs

# the port at which the clients will connect
clientPort=2183
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=192.168.1.11:4000:5000
server.2=192.168.1.11:4000:5000
server.3=192.168.1.11:4000:5000

4、校订zookeeper2181配置,依次实施下边施命发号

cd zookeeper2181
mkdir data logs
cd data
vi myid(首先输入字母i进入编辑模式,修改myid内容1,这个1与zoo.cfg文件里的server.1对应(具体服务器是多少就输入多少))

5、校勘zookeeper2182配置,依次实践下边施命发号

cd zookeeper2182
mkdir data logs
cd data
vi myid(首先输入字母i进入编辑模式,修改myid内容2,这个2与zoo.cfg文件里的server.2对应(具体服务器是多少就输入多少))

6、匡正zookeeper2183配置,依次推行上面发号布令

cd zookeeper2183
mkdir data logs
cd data
vi myid(首先输入字母i进入编辑模式,修改myid内容3,这个3与zoo.cfg文件里的server.3对应(具体服务器是多少就输入多少))

2. Zookeeper集群情况搭建和运转:

要是有单独的集群规划,可以在独立的机械上设置Zookeeper集群,我利用的是尝试虚构机情况,就向来在hadoop-senior01、hadoop-senior02、hadoop-senior03安装,与Hadoop共存,没十分。

4、修正文件zoo.cfg

vi zoo.cfg
# 修正数据目录dataDir为以下值
dataDir=/tmp/zookeeper
 
# 增多以下段落
server.1=penn1-VirtualBox:2887:3887
server.2=penn2-VirtualBox:2888:3888

4、运营服务

1、运维zookeeper2181服务,步入bin目录,输入指令:./zkServer.sh start

[root@localhost zookeeper2181]# cd bin
[root@localhost bin]# ./zkServer.sh start
JMX enabled by default
Using config: /usr/local/application/zookeeper/zookeeper2181/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost bin]# 

2、运行zookeeper2182服务,步入bin目录,输入指令:./zkServer.sh start

[root@localhost zookeeper2182]# cd bin
[root@localhost bin]# ./zkServer.sh start
JMX enabled by default
Using config: /usr/local/application/zookeeper/zookeeper2182/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost bin]# 

3、运维zookeeper2183服务,步向bin目录,输入指令:./zkServer.sh start

[root@localhost zookeeper2183]# cd bin
[root@localhost bin]# ./zkServer.sh start
JMX enabled by default
Using config: /usr/local/application/zookeeper/zookeeper2183/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost bin]# 

4、查看服务,步向bin目录使用命令:./zkServer.sh
status,发掘自动大选了leader。

[root@localhost bin]# ./zkServer.sh status
JMX enabled by default
Using config:
/usr/local/application/zookeeper/zookeeper2183/bin/../conf/zoo.cfg
Mode: leader
[root@localhost bin]# cd
/usr/local/application/zookeeper/zookeeper2181/bin
[root@localhost bin]# ./zkServer.sh status
JMX enabled by default
Using config:
/usr/local/application/zookeeper/zookeeper2181/bin/../conf/zoo.cfg
Mode: follower
[root@localhost bin]# cd
/usr/local/application/zookeeper/zookeeper2182/bin
[root@localhost bin]# ./zkServer.sh status
JMX enabled by default
Using config:
/usr/local/application/zookeeper/zookeeper2182/bin/../conf/zoo.cfg
Mode: follower

1. 下载Zookeeper安装包:

行使版本:zookeeper-3.4.5.tar.gz

(1卡塔 尔(阿拉伯语:قطر‎java景况设置:

见前边小说。

(2卡塔 尔(英语:State of Qatar)上传解压安装包:

$ tar -zxf zookeeper-3.4.5.tar.gz -C /opt/modules/

  dataDir=/home/wyh/hadoop/zookeeper-3.4.9/zookeeper/data
  dataLogDir=/home/wyh/hadoop/zookeeper-3.4.9/zookeeper/logs
  clientPort=4180
  server.43=192.168.181.130:2888:3888
  server.47=192.168.181.131:2888:3888
  server.48=192.168.181.132:2888:3888

server.id=host:port:port标记差别的ZooKeeper服务器
 
亟需在种种服务器的多少目录中(那边为/tmp/zookeeper卡塔 尔(阿拉伯语:قطر‎创造myid文件,文件的剧情为上述对应的server.id中的id

5、常用命令

运转服务:./zkServer.sh start (必要步向bin目录卡塔 尔(英语:State of Qatar)

悬停服务:./zkServer.sh stop (必要步入bin目录卡塔尔国

重启服务:./zkServer.sh restart (需求步入bin目录卡塔尔国

查看境况:./zkServer.sh status (供给步向bin目录卡塔 尔(英语:State of Qatar)

2. 配置Zookeeper:

(1卡塔尔国复制配置文件:

Zookeeper的主目录的conf文件夹下,有二个实例的安排文件zoo_sample.cfg,大家复制大器晚成份出来:

$ cd /opt/modules/zookeeper-3.4.5/conf

$ cp -a zoo_sample.cfg zoo.cfg

$ vi zoo.cfg

$ mkdir data

下边通过编写制定这一个文件来产生Zookeeper的布署:

(2卡塔 尔(阿拉伯语:قطر‎配置数据存款和储蓄目录:

根据支付标准,我们平时在zookeeper的安装目录下,新建一个目录data,将以此目录作为zookeeper的数据存款和储蓄目录。

dataDir=/opt/modules/zookeeper-3.4.5/data

zoo.cfg重要参数介绍:

tickTime:zookeeper服务器与服务器之间,可能服务器与客商端之间保持心跳的时日间距。单位:飞秒。

clientPort:客商端连接zookeeper服务器的端口。暗许2181

dataDir:Zookeeper保存数据的目录。

(3卡塔 尔(阿拉伯语:قطر‎在zoo.cfg中增添服务器音信:

格式: server.A=B:C:D  。 在zoo.cfg中添加如下内容:

server.1=192.168.8.128:2888:3888

server.2=192.168.8.129:2888:3888

server.3=192.168.8.130:2888:3888

(4卡塔 尔(阿拉伯语:قطر‎在每台服务器的$ZOOKEEPE奥迪Q7_HOME/data/目录创制myid文件。

在hadoop-senior01服务器(192.168.8.128)上,cd
到$ZOOKEEPER_HOME/data/目录,创立myid文件,myid文件的源委是1

在hadoop-senior02服务器(192.168.8.129)上,cd
到$ZOOKEEPER_HOME/data/目录,创制myid文件,myid文件的开始和结果是2

在hadoop-senior03服务器(192.168.8.130)上,cd
到$ZOOKEEPER_HOME/data/目录,创设myid文件,myid文件的开始和结果是3

注意:myid文件在linux下用vi创建。

在后生可畏台机器上布署好,拷贝到别的服务器:

$ scp -r zookeeper-3.4.5/
natty@hadoop-senior02.pmpa.com:/opt/modules/

  192.168.181.130、192.168.181.131、192.168.181.132

vi /tmp/zookeeper/myid
# 值为种种服务器对应的server.id中的id
id

3.启动Zookeeper:

(1)启动Zookeeper:

$ bin/zkServer.sh start

(2卡塔尔查看意况

$ bin/zkServer.sh status

$ jps

稳重:运转操作须要在三台服务器上都执行

  是三台集群机器

相关文章