如何在Redhat7.7中安装CDP DC7.0.3

一：关于CDP DC 7.0.3 概述

二：系统环境初始化

三：构建CDP DC 7.0.3

一：关于CDP DC 7.0.3 概述

1.1 CDP dc 7.0.3

CDP DC7.0.3是Cloudera与Hortonworks合并后，第一个融合CDH和HDP全部组件的on-premise版本，CDP Data Center主要由Cloudera Runtime构成，Cloudera Runtime由超过35个开源项目组成，固然CDP Data Center还包括其它功能如管理功能Cloudera Manager，Key Management，专业支持等，以下图所示：

1.2 Cloudera Runtime的主要组件版本以下图所示：

二：系统环境初始化

2.1 系统环境介绍

OS: rhel7.7x64 

cat /etc/hosts 
---
192.168.100.21  rhel01.flyfish
192.168.100.22  rhel02.flyfish
192.168.100.23  rhel03.flyfish
192.168.100.24  rhel04.flyfish
192.168.100.25  rhel05.flyfish
192.168.100.26  rhel06.flyfish
---

本次采用 前四台 机器

2.2 无密钥登陆配置

作root用户无密钥认证

 ssh-keygen ---一直敲回车到最后

 cat id_rsa.pub >> authorized_keys 

 将全部的公钥导入authorized_keys  分发到 全部的 机器的.ssh/ 下面

 而后测试

2.3关闭 firewalld，iptables 与 selinux

全部节点都要配置：

yum install -y firewall* iptable* 

  (1) 关闭firewalld

 systemctl stop firewalld.service
 systemctl disable firewalld.service
 firewall-cmd --state

 (2) 关闭iptables 
 iptables -F
 systemctl stop iptables.service 
 service iptables save
 systemctl disable iptables.service 

 (3) 禁用 selinux 

  sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config  
  setenforce 0
  getenforce 0
  sestatus

  重启机器

  reboot

2.4 全部机器更改RHEL7.7X64的文件句柄数

vim /etc/security/limits.conf
---

*               soft    nofile          65535
*               hard    nofile          1029345
*               soft    nproc           unlimited
*               hard    nproc           unlimited
*               soft    memlock         unlimited
*               hard    memlock         unlimited

---

ansible cdh7 -m copy -a "src=/etc/security/limits.conf dest=/etc/security/"

2.5 时间同步：

全部机器 yum install chrony*
1. rhel01.flyfish:

vim /etc/chrony.conf 增长
----
注释掉相关默认的server
server ntp1.aliyun.com iburst

allow 192.168.100.0/24

service chronyd start
chkconfig chronyd on 

2. rhel02.flyfish----rhel04.flyfish 同步rhel01.flyfish

vim /etc/chrony.conf
----
注释掉默认的server
server 192.168.100.21 iburst
----

service chronyd stop 
service chronyd start 
chkconfig chronyd on

2.7 cdp dc 7.0.3 安装所需文件下载

CM 的rpm包：

---
cloudera-manager-agent-7.0.3-1635136.el7.x86_64.rpm
cloudera-manager-daemons-7.0.3-1635136.el7.x86_64.rpm
cloudera-manager-server-7.0.3-1635136.el7.x86_64.rpm
cloudera-manager-server-db-2-7.0.3-1635136.el7.x86_64.rpm
enterprise-debuginfo-7.0.3-1635136.el7.x86_64.rpm
openjdk8-8.0+232_9-cloudera.x86_64.rpm
allkeys.asc

下载地址：
http://archive.cloudera.com/cm7/7.0.3/redhat7/yum/RPMS/x86_64/
http://archive.cloudera.com/cm7/7.0.3/allkeys.asc

----

cdh7.0.3 的parcel：

--
CDH-7.0.3-1.cdh7.0.3.p0.1635019-el7.parcel
CDH-7.0.3-1.cdh7.0.3.p0.1635019-el7.parcel.sha256
manifest.json

下载地址：
http://archive.cloudera.com/cdh7/7.0.3.0/parcels/

---

2.8 设置swap

将全部的swap为1，并即时生效

ansible cdh7 -m shell -a "echo '1' > /proc/sys/vm/swappiness"

ansible cdh7 -m shell -a "sysctl -a | grep vm.swappiness"

vim /etc/sysctl.conf 
----
最后加上：
vm.swappiness = 1 
----
sysctl -p

2.8 设置最大透明数

ansible cdh7 -m shell -a "echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

ansible cdh7 -m shell -a "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled"

ansible cdh7 -m shell -a "cat /sys/kernel/mm/transparent_hugepage/enabled"
ansible cdh7 -m shell -a "cat /sys/kernel/mm/transparent_hugepage/defrag"

修改全部节点的/etc/rc.d/rc.local文件的权限以实现开机执行
ansible cdh7 -m shell -a "chmod +x /etc/rc.d/rc.local"

vim /etc/rc.d/rc.local
---
最后加上：

if test -f /sys/kernel/mm/transparent_hugepage/enabled; 

then

echo never > /sys/kernel/mm/transparent_hugepage/enabled

fi

if test -f /sys/kernel/mm/transparent_hugepage/defrag; 
then

echo never > /sys/kernel/mm/transparent_hugepage/defrag

fi

---

ansible cdh7 -m copy -a "src=/etc/rc.d/rc.local dest=/etc/rc.d/"

ansible cdh7 -m shell -a "tail -f /etc/rc.d/rc.local"

2.9 配置httpd 服务器

ansible rhel01.flyfish -m yum -a "state=present name=httpd"
ansible rhel01.flyfish -m service -a "name=httpd state=started"
ansible rhel01.flyfish -m shell -a "ps -ef |grep httpd"

修改 /etc/httpd/conf/httpd.conf 在<IfModule mime_module>中修改如下内容

vim /etc/httpd/conf/httpd.conf

---
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz .parcel
---

service httpd restart

2.10 安装maridb 数据库

yum install mariadb* 

service mariadb start 

chkonfig mariadb on 

mysql_secure_installation

2.11 建立CM，Hive等须要的表

vim cm.sql
----
create database metastore default character set utf8;
CREATE USER 'hive'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%';
FLUSH PRIVILEGES;
create database cm default character set utf8;
CREATE USER 'cm'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%';
FLUSH PRIVILEGES;
create database rm default character set utf8;
CREATE USER 'rm'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON rm. * TO 'rm'@'%';
FLUSH PRIVILEGES;
create database hue default character set utf8;
CREATE USER 'hue'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%';
FLUSH PRIVILEGES;
create database oozie default character set utf8;
CREATE USER 'oozie'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';
FLUSH PRIVILEGES;
create database ranger default character set utf8;
CREATE USER 'rangeradmin'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON ranger. * TO 'rangeradmin'@'%';
FLUSH PRIVILEGES;
----

mysql -uroot -pflyfish225

source cm.sql

2.12 安装JDBC驱动

mkdir -p /usr/share/java/
tar -zxvf mysql-connector-java-5.1.44.tar.gz

mv mysql-connector-java-5.1.44-bin.jar /usr/share/java

cd /usr/share/java/

mv mysql-connector-java-5.1.44-bin.jar mysql-connector-java.jar 

chmod 777 mysql-connector-java.jar

三：构建CDP DC 7.0.3

3.1 安装 jdk

全部节点 卸载系统自带jdk

rpm -qa |grep jdk 
rpm -e java-1.8.0-openjdk-1.8.0.222.b03-1.el7.x86_64 copy-jdk-configs-3.3-10.el7_5.noarch java-1.8.0-openjdk-headless-1.8.0.222.b03-1.el7.x86_64 --nodeps 

yum -y install openjdk8-8.0+232_9-cloudera

vim /etc/profile
---
到最后加上jdk 的环境变量

export JAVA_HOME=/usr/java/jdk1.8.0_232-cloudera
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin

----
source /etc/profile

java -version

3.2 配置cdh7.0.3 的源

mv cdh7.0.3 cdh7
cd cdh7 

createrepo .

cd ..

mv cdh7 /var/www/html/

制做Cloudera Manager的repo源

cd /etc/yum.reos.d/

vim cdh.repo
---
[cmrepo]
name = cm_repo
baseurl = http://192.168.100.21/cdh7/
enable = true
gpgcheck = false

---

ansible cdh7 -m copy -a "src=/etc/repos.d/cdh7.repo dest=/etc/repos.d/"
yum repolist

3.3 安装Cloudera Manager Server

yum -y install cloudera-manager-server

初始化数据库

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql cm cm password

启动Cloudera Manager Server

systemctl start cloudera-scm-server
systemctl status cloudera-scm-server

cd /var/log/cloudera-scm-server
tail -100f cloudera-scm-server.log

netstat -nultp |grep 7180

3.4 经过http://192.168.100.21:7180/cmf/login访问CM

用户名：admin
密码： admin

登陆到CM，选择60天试用，点击继续

集群安装：

集群名字

识别集群的主机

Parcel Repository & Network Settings

安装 cloudera-scm-agent

分配parcel包

点击Inspect Network Performance和Inspect Hosts，检查主机

定义服务：

点击“继续”，进入下一步，测试数据库链接

首先输入CM的用户名和密码admin/admin

如下4个密码，都须要包含至少一个字母与一个数字，为了安装方便都输入admin@123

安装服务初始化

如何在Redhat7.7中安装CDP DC7.0.3

一： 关于CDP DC 7.0.3 概述

1.1 CDP dc 7.0.3

1.2 Cloudera Runtime的主要组件版本以下图所示：

二： 系统环境初始化

2.1 系统环境介绍

2.2 无密钥登陆配置

2.3关闭 firewalld，iptables 与 selinux

2.4 全部机器更改RHEL7.7X64的文件句柄数

2.5 时间同步：

2.7 cdp dc 7.0.3 安装所需文件下载

2.8 设置swap

2.8 设置最大透明数

2.9 配置httpd 服务器

2.10 安装maridb 数据库

2.11 建立CM，Hive等须要的表

2.12 安装JDBC驱动

三： 构建CDP DC 7.0.3

3.1 安装 jdk

3.2 配置cdh7.0.3 的源

3.3 安装Cloudera Manager Server

3.4 经过http://192.168.100.21:7180/cmf/login访问CM

Trending Articles

一：关于CDP DC 7.0.3 概述

二：系统环境初始化

三：构建CDP DC 7.0.3