高可用分布式clickhouse集群搭建

in 编程
关注公众号【好便宜】( ID:haopianyi222 ),领红包啦~
阿里云,国内最大的云服务商,注册就送数千元优惠券:https://t.cn/AiQe5A0g
腾讯云,良心云,价格优惠: https://t.cn/AieHwwKl
搬瓦工,CN2 GIA 优质线路,搭梯子、海外建站推荐: https://t.cn/AieHwfX9

介绍clickhouse单机和高可用分布式搭建

    docker单机搭建方法

#使用yandex的服务端镜像
docker pull yandex/clickhouse-server
#使用yandex的客户端镜像
docker pull yandex/clickhouse-client

yandex/clickhosue-server的Dockerfile如下,可参考进行自定义镜像搭建

FROM ubuntu:16.04

ARG repository="deb http://repo.yandex.ru/clickhouse/deb/stable/ main/"
ARG version=\*

RUN apt-get update && \
    apt-get install -y apt-transport-https && \
    mkdir -p /etc/apt/sources.list.d && \
    echo $repository | tee /etc/apt/sources.list.d/clickhouse.list && \
    apt-get update && \
    apt-get install --allow-unauthenticated -y clickhouse-server-common=$version clickhouse-server-base=$version && \
    rm -rf /var/lib/apt/lists/* /var/cache/debconf && \
    apt-get clean

COPY docker_related_config.xml /etc/clickhouse-server/config.d/
RUN chown -R clickhouse /etc/clickhouse-server/

USER clickhouse
EXPOSE 9000 8123 9009
VOLUME /var/lib/clickhouse

ENV CLICKHOUSE_CONFIG /etc/clickhouse-server/config.xml

ENTRYPOINT exec /usr/bin/clickhouse-server --config=${CLICKHOUSE_CONFIG}

运行clickhosue服务镜像 yandex/clickhouse-server

$ docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 yandex/clickhouse-server

运行clickhouse客户端镜像 yandex/clickhouse-client

$ docker run -it --rm --link some-clickhouse-server:clickhouse-server yandex/clickhouse-client --host clickhouse-server

这里推荐一个可视化的连接工具DBeaver https://dbeaver.jkiss.org/download/

可以直接连接到clickhosue  用户名默认为 default

    高可用集群搭建

        我这里准备三台机器 系统都是ubuntu server 16.04

            192.168.43.225 yun0

            192.168.43.226 yun1

            192.168.43.227 yun2

    在机器上安装clickhouse数据库(如果你是虚拟机的话,可以先装一台 ,再复制俩台)

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E0C56BD4    # optional
sudo apt-add-repository "deb http://repo.yandex.ru/clickhouse/deb/stable/ main/"
sudo apt-get update
sudo apt-get install clickhouse-server-common clickhouse-client -y

如果提示没有app-add-repository,你需要安装下面俩个

apt-get install python-software-properties
apt-get install software-properties-common

装完后进行clickhouse集群的配置

配置/etc/clickhouse-server/config.xml

在60行到70行之间,有一个监听主机(配置远程连接)

一般把下面这三行注释去掉就行了

<listen_host>::</listen_host>                                            
<listen_host>::1</listen_host>                                           
<listen_host>127.0.0.1</listen_host>

同时在这个xml文件里面可以看到这样一段注释

<!-- If element has 'incl' attribute, then for it's value will be used corresponding substi
    tution from another file.
155          By default, path to file with substitutions is /etc/metrika.xml. It could be changed i
    n config in 'include_from' element.
156          Values for substitutions are specified in /yandex/name_of_substitution elements in tha
    t file.
157       -->

metrika.xml就是分布式配置文件

在etc目录下新建metrika.xml文件

可直接在下面的实例中修改

<yandex>
<clickhouse_remote_servers>
    <perftest_3shards_1replicas>
        <shard>
             <internal_replication>true</internal_replication>
            <replica>
                <host>yun0</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <replica>
                <internal_replication>true</internal_replication>
                <host>yun1</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>yun2</host>
                <port>9000</port>
            </replica>
        </shard>
    </perftest_3shards_1replicas>
</clickhouse_remote_servers>


<zookeeper-servers>
  <node index="1">
    <host>yun0</host>
    <port>2181</port>
  </node>

  <node index="2">
    <host>yun1</host>
    <port>2181</port>
  </node>
  <node index="3">
    <host>yun2</host>
    <port>2181</port>
  </node>
</zookeeper-servers>

<macros>
    <replica>yun0</replica>
</macros>


<networks>
   <ip>::/0</ip>
</networks>


<clickhouse_compression>
<case>
  <min_part_size>10000000000</min_part_size>
                                             
  <min_part_size_ratio>0.01</min_part_size_ratio>                                                                                                                                       
  <method>lz4</method>
</case>
</clickhouse_compression>

</yandex>
            

这个配置主要修改三个地方 

1.配置节点

<clickhouse_remote_servers>
    <perftest_3shards_1replicas>
        <shard>
             <internal_replication>true</internal_replication>
            <replica>
                <host>yun0</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <replica>
                <internal_replication>true</internal_replication>
                <host>yun1</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>yun2</host>
                <port>9000</port>
            </replica>
        </shard>
    </perftest_3shards_1replicas>
</clickhouse_remote_servers>

2.配置zookeeper 

<zookeeper-servers>
  <node index="1">
    <host>yun0</host>
    <port>2181</port>
  </node>

  <node index="2">
    <host>yun1</host>
    <port>2181</port>
  </node>
  <node index="3">
    <host>yun2</host>
    <port>2181</port>
  </node>
</zookeeper-servers>

3.前面俩个都没啥好说的,下面这个注意每个节点填当前节点的ip

<macros>
    <replica>yun0</replica>
</macros>

后面就是安装jdk,zookeeper.这俩个就不介绍了。

装完后重启  启动zookerper,clickhouse-server服务。

关注公众号【好便宜】( ID:haopianyi222 ),领红包啦~
阿里云,国内最大的云服务商,注册就送数千元优惠券:https://t.cn/AiQe5A0g
腾讯云,良心云,价格优惠: https://t.cn/AieHwwKl
搬瓦工,CN2 GIA 优质线路,搭梯子、海外建站推荐: https://t.cn/AieHwfX9
扫一扫关注公众号添加购物返利助手,领红包
Comments are closed.

推荐使用阿里云服务器

超多优惠券

服务器最低一折,一年不到100!

朕已阅去看看