网络工具所有节点安装 net-tools epel-release httpd wget
Hosts 所有节点配置
防火墙所有节点关闭 firewalld selinux
透明大页面所有节点 defra/enable false
Swappiness 所有节点设置为 0
主节点安装 httpd yum源 createrepo
所有节点配置yum源服务
主节点配置 mysql jdbc jdk
Cloudera manager 主节点安装server 从节点安装agent
Cloudera manager 连接mysql 主节点
启动cloudera manager
添加supergroup 有端口韦7180
版本选择为select Edition / cloudera express

Vbox虚拟机安装

启动两个网卡第一个网卡 NAT 第二个网卡 Host-Only

打开网络配置并设置host-only为静态服务

网络规划

192.168.1.10 nodeserver

192.168.1.11 node1

192.168.1.12 node1

Master配置

安装工具

所有节点安装 net-tools epel-release httpd wget chrony

yum install -y httpd

chrony时间配置

yum install chrony

配置chrony

Vi /etc/chrony.conf

主节点配置云服务时间

ntp.aliyun.com 阿里云
ntp.tencent.com 腾讯云

添加server ntp.aliyun.com iburst

allow all

local stratum 10

从节点配置主节点时间

Vi /etc/chrony.conf

server 192.168.1.10 iburst

关闭防火墙

关闭firewalld

systemctl stop firewalld.service 关闭

systemctl enable firewalld.service 开机自启

关闭selinux

临时关闭 setenforce 0 打开 1

永久关闭

vi /etc/selinux/config

的修改SELINUX=disable

修改swap交换空间与透明大页面

Swap执行命令

mkswap /swapfile 创建

swapoff 关闭

swapon 启用

cat /proc/sys/vm/swappiness

echo vm.swappiness=0 >> /etc/sysctl.conf

sysctl –p

sysctl vm.swappiness=10

关闭transparent_hugepage

/sys/kernel/mm/transparent_hugepage/

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

永久关闭transparent_hugepage

Vi /etc/rc.d/rc.local

chmod +x /etc/rc.d/rc.local

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

拷贝多个系统

拷贝多个克隆系统

修改系统的hostname

修改ip

修改从节点配置主节点时间

Vi /etc/chrony.conf

server 192.168.1.10 iburst

修改hosts

vi /etc/hosts

192.168.1.10 nodeserver

192.168.1.11node1

192.168.1.12 node2

Hosts分发

scp /etc/hosts 192.168.1.11:/etc/hosts

scp /etc/hosts 192.168.1.12:/etc/hosts

从节点关闭防火墙

配置ssh免密协议

每个节点

ssh-keygen

主节点分发ssh公钥

Ssh-copy-id 192.168.1.11

从节点将公钥加入到authorized_keys

# cat id_rsa.pub >> authorized_keys

将 authorized_keys复制到下一个节点

scp authorized_keys node2:/root/.ssh/

下一个节点重复上一步，并讲整合的所有公钥分发给每一个节点

scp authorized_keys nodeserver:/root/.ssh/

scp authorized_keys node1:/root/.ssh/

主节点yum源配置

创建http文档 cdh_local_repo 与 packages

mkdir -p /var/www/html/cdh_local_repo

mkdir -p /var/www/html/packages

在cdh_local_repo中上传cdh相关rpm与jdk文档

Yum源自定义配置

/etc/yum.repo.d/ yum源路径

Cd /etc/yum.repo.d/

创建.repo格式文件

cp CentOS-Base.repo CentOS-Loac.repo

修改CentOS-Loacl.repo

[base-local]

name=CentOS-Local

baseurl=http://192.168.1.10/cdh_local_repo

gpgcheck=0

enabled=1

拷贝到从节点

scp CentOS-Loacl.repo node1:/etc/yum.repos.d/

scp CentOS-Loacl.repo node2:/etc/yum.repos.d/

启动httpd服务

systemctl start httpd

systemctl enable httpd

制作repo 下载createrepo

yum install createrepo

Createrepo制作源

createrepo /var/www/html/cdh_local_repo/

yum makecache 服务器同步到yum缓存

yum clean all 清理当前yum缓存

yum repolist 缓存并列出可用源

安装jdk mysql cloudera

每个节点安装jdk

查看

yum list | grep jdk

yum install jdk1.8.x86_64

安装mysql

节点安装jdbc

创建文件夹

Mkdir -p /usr/share/java

下载jdbc

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz

解压jdbc

tar -xvf mysql-connector-java-5.1.46.tar.gz

修改文件名

cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

分发给从节点

从节点创建文件夹

Mkdir -p /usr/share/java

scp mysql-connector-java.jar node1:/usr/share/java/

scp mysql-connector-java.jar node2:/usr/share/java/

主节点安装mysql

下载mysql.rpm

wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

解压

rpm -ivh mysql-community-release-el7-5.noarch.rpm

安装

yum install -y mysql-server

主节点安装配置mysql

启动mysql

systemctl start mysqld

systemctl enable mysqld

主节点的mysql授权

grant all privileges on *.* to root@’nodeserver’ identified by ‘123456’ with grant option;

grant all privileges on *.* to root@’localhost’ identified by ‘123456’ with grant option;

grant all privileges on *.* to root@’%’ identified by ‘123456’ with grant option;

创建数据库

create database scm default character set utf8;

create database metastore default character set utf8;

create database hue default character set utf8;

安装cloudera manager

主环境装 server agent

yum install -y cloudera-manager*

从环境装 agent

yum install -y cloudera-manager-agent.x86_64

cloudera连接mysql

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root 123456

修改agent的config

vi /etc/cloudera-scm-agent/config.ini

[General]

# Hostname of the CM server.

server_host=192.168.1.2

给HDFS一个授权用户主从都要

groupadd supergroup

usermod -a -G supergroup root

启动cloudera

systemctl start cloudera-scm-server

systemctl start cloudera-scm-agent

查看启动情况端口号为7180

netstat -lnp