December 2019
M T W T F S S
« Nov    
 1
2345678
9101112131415
16171819202122
23242526272829
3031  

Categories

WordPress Quotes

A successful man is one who can lay a firm foundation with the bricks others have thrown at him.
David Brinkley
December 2019
M T W T F S S
« Nov    
 1
2345678
9101112131415
16171819202122
23242526272829
3031  

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (268)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
EXCHANGE (3)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (1)
horoscope (23)
Hyper-V (10)
IIS (15)
IPTABLES (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
SCALEIO (1)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Ubuntu (1)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)
ZIMBRA (17)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

24 visitors online now
5 guests, 19 bots, 0 members

Hit Counter provided by dental implants orange county

expect,sshpass,pash

expect,sshpass,pash
rpm -qa expect

yum install expect -y

useradd mohan

echo 123456|passwd –stdin mohan

su – mohan

ssh-keygen -t dsa -P ” -f ~/.ssh/id_dsa >/dev/null 2>&1

yum install lrzsz -y

cat mohan_sshkey.exp
#!/usr/bin/expect
if { $argc !=2 } {
send_user “usage: expect mohan_sshkey.exp file host\n”
exit
}
#define var
set file [lindex $argv 0]
set host [lindex $argv 1]
set password “123456”
#spawn scp /etc/hosts root@10.0.0.142:/etc/hosts
#spawn scp -P52113 $file testserver@$host:$dir
spawn ssh-copy-id -i $file “-p 22 oldgirl888@$host”
expect {
“yes/no” {send “yes\r” ;exp_continue}
“password” {send “$password\r”}
}
expect cof
exit -onexit {
send_user “testserver say good bye to you!\n”
}
#script usage
#expect testserver-6.exp file host dir
#example
#expect mohan_sshkey.exp file host dir
#expect mohan_sshkey.exp ~/etc/hosts 10.0.0.41:~

expect mohan_sshkey.exp .ssh/id_dsa.pub 192.168.1.21

#!/bin/sh
. /etc/init.d/functions
for ip in 8 31 41
do
expect mohan_sshkey.exp ~/.ssh/id_dsa.pub 192.168.1.$ip >/dev/null 2>&1
if [ $? -eq 0 ];then
action “$ip” /bin/true
else
action “$ip” /bin/false
fi
done

oracle backup

#!/bin/bash
#oracle_backup.sh version1.0 chmod 700
#00 00 * * * /usr/local/scripts/oracle_backup.sh
#chmod 700 /usr/local/scripts/oracle_backup.sh
#You must first import the environment variables in the Oracle user’s bash_profile file
#Must be used to define the DBA account first dpdata1 path and authorized directory to read and write, modify the dpdata1 group

export ORACLE_BASE=/usr/local/u01/oracle
export ORACLE_HOME=/usr/local/u01/oracle/product/11.2.0/dbhome_1
export ORACLE_SID=oracle11
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$PATH
datetime=$(date +”%Y%m%d”)
dpdata1_dir=”/data/backup/oracle_backup”
oracle_u01=”u01″
oracle_u02=”u02″
oracle_password1=”u01_password”
oracle_password2=”u02_password”

expdp ${oracle_u01}/${oracle_password1} directory=dpdata1 dumpfile=${oracle_u01}_${datetime} logfile=${oracle_u01}_${datetime}.log

if [ $? -ne 0 ];then
echo “$(date +”%Y-%m-%d_%H:%M:%S”)oracle_${oracle_u01}_backup_file!” > ${dpdata1_dir}/${datetime}_err.log
fi

expdp ${oracle_u02}/${oracle_password2} directory=dpdata1 dumpfile=${oracle_u02}_${datetime} logfile=${oracle_u02}_${datetime}.log

if [ $? -ne 0 ];then
echo “$(date +”%Y-%m-%d_%H:%M:%S”)oracle_${oracle_u02}_backup_file!” >> ${dpdata1_dir}/${datetime}_err.log
fi

/usr/bin/bzip2 -z ${dpdata1_dir}/*${datetime}*
find $dpdata1_dir -type f -ctime +30 -name “*.bz2” -exec rm -vf {} \;

Kubernetes

What is Kubernetes?
Kubernetes is a Google open source container cluster management system, based on Docker build containers, the use of Kubernetes can manage a number of Docker host in the container.
The main functions are as follows:
1) abstracts multiple Docker hosts into one resource to manage containers in a clustered manner, including task scheduling, resource management, resilient scaling, rolling upgrades, and more.
2) Use the layout system (YAML File) to quickly build container clusters, provide load balancing, solve container direct correlation and communication problems
3) automatically manage and repair the container, simply say, such as creating a cluster, there are ten containers, if a container is closed, then, will try to restart or re-allocation of containers, always ensure that there will be ten containers in operation, Kill the excess.

What is Kubernetes?
Kubernetes is a Google open source container cluster management system, based on Docker build containers, the use of Kubernetes can manage a number of Docker host in the container.
The main functions are as follows:
1) abstracts multiple Docker hosts into one resource to manage containers in a clustered manner, including task scheduling, resource management, resilient scaling, rolling upgrades, and more.
2) Use the layout system (YAML File) to quickly build container clusters, provide load balancing, solve container direct correlation and communication problems
3) automatically manage and repair the container, simply say, such as creating a cluster, there are ten containers, if a container is closed, then, will try to restart or re-allocation of containers, always ensure that there will be ten containers in operation, Kill the excess.
Kubernetes role composition:
1) Pod
Pod is the smallest unit of kubernetes, a pod can be composed of one or more containers;
The same Pod can only run on the same host, share the same volumes, network, namespace;
2) ReplicationController (RC)
RC is used to manage the pod. An RC can consist of one or more pods. After the RC is created, the system creates the number of pods based on the number of copies defined. In the course of running, if the number of Pod is less than defined, it will restart the stop or reassign Pod, otherwise kill the excess. Of course, you can also dynamically stretch the size of the Pods or familiar.
RC is associated with the corresponding Pods through the label, and in the rolling upgrade, the RC uses a Pod that replaces the entire Pods to be updated.
3) Service
Service defines the abstract resources of a Pod logical set, and the containers in the Pod collection provide the same functionality. The collection is created according to the defined Label and selector. When a Service is created, a Cluster IP is assigned. This IP provides a unified access interface with the defined port and implements load balancing.
4) Label
Label is used to distinguish between Pod, Service, RC key / value key pairs;
Pod, Service, RC can have multiple labels, but each label’s key can only correspond to one;
Mainly to the Service request through the lable forwarding to the backend to provide services Pod collection;
Kubernetes components:
1) kubectl
The client command line tool formats the accepted command and sends it to the kube-apiserver as the operating entry for the entire system.
2) kube-apiserver
As a control entry for the entire system, the REST API service provides an interface.
3) kube-controller-manager
Used to perform the background tasks in the entire system, including node status, Pod number, Pods and Service associations.
4) kube-scheduler
Responsible for node resource management, accepting pods tasks from kube-apiserver, and assigning them to a node.
5) etcd
Responsible for service discovery and configuration sharing between nodes.
6) kube-proxy
Run on each compute node, responsible for Pod network proxy. Timing from the etcd to obtain the service information to do the appropriate strategy.
7) kubelet
Run on each compute node, as an agent, accept the Pods task assigned to the node and manage the container, periodically get the container status, and feed it back to kube-apiserver.
8) DNS
An optional DNS service for creating DNS records for each Service object so that all Pods can access the service through DNS.

redis 3

CentOS Redis 3 and 6.8 installed on the startup script, strong Redis is not to say, and directly see the following article explain.

Step 1: Download, compile and install

cd /opt
wget http://download.redis.io/releases/redis-3.2.5.tar.gz

tar zxvf redis-3.2.5.tar.gz

cd redis-3.2.5

yum install -y gcc*
yum install -y tcl
make MALLOC=libc
make
make test

cd deps
make hiredis jemalloc linenoise lua geohash-int
cd ..
make install

[root@cluster1 redis-3.2.5]# cd utils/
[root@cluster1 utils]# ls
build-static-symbols.tcl corrupt_rdb.c generate-command-help.rb hyperloglog lru redis_init_script redis-sha1.rb speed-regression.tcl
cluster_fail_time.tcl create-cluster hashtable install_server.sh redis-copy.rb redis_init_script.tpl releasetools whatisdoing.sh
[root@cluster1 utils]# chmod +x install_server.sh
[root@cluster1 utils]# ./install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default – /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default – /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default – /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service…
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server…
Installation successful!

daemonize no
port 6379
logfile “”
pidfile /var/run/redis.pid
# requirepass foobared
dir ./

daemonize yes
port 6379
logfile “/var/log/redis/6379.log”
pidfile /var/run/redis_6379.pid
requirepass 9k3NgZq%gO
dir /data/redis

grep vm.overcommit_memory /etc/sysctl.conf
echo “vm.overcommit_memory = 1” >> /etc/sysctl.conf
sed -i “s/vm.overcommit_memory = 0/vm.overcommit_memory = 1/g” /etc/sysctl.conf
sysctl -p

Redis
chkconfig redis on
chkconfig –list redis
service redis start
service redis stop

update Redis on CentOS 6.5

First, make sure the following repos, EPEL and REMI, are installed:
sudo rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
sudo rpm -Uvh http://rpms.remirepo.net/enterprise/remi-release-6.rpm
Check the version of Redis in REMI repo: (As of June 2015, the version is 2.8.13)
yum –enablerepo=remi info redis
Then install related dependency (jemalloc) from EPEL repo:
sudo yum –enablerepo=epel install jemalloc
Before installation, you should stop the old Redis daemon:
sudo service redis stop
Then install the newer version of Redis:
sudo yum –enablerepo=remi install redis
Edit Redis configuration file if needed:
sudo vi /etc/redis.conf
Restart Redis daemon, and make it auto-start on reboot:
sudo service redis start
sudo chkconfig redis on
Finally, check the version of currently installed Redis:
redis-cli info | grep redis_version

redis

Centos 6.8

cd /opt
wget http://download.redis.io/releases/redis-3.0.5.tar.gz
tar zxvf redis-3.0.5.tar.gz
cd redis-3.0.5
make && make install

mkdir /etc/redis
mkdir /var/log/redis
mkdir -p /data/redis
cd /opt/redis-3.0.5
cp redis.conf /etc/redis/7963.conf
vi /etc/redis/7963.conf

daemonize no
port 6379
logfile “”
pidfile /var/run/redis.pid
# requirepass foobared
dir ./
??
daemonize yes
port 7963
logfile “/var/log/redis/7963.log”
pidfile /var/run/redis_7963.pid
requirepass 9k3NgZq%gO!W7x-0y=LI
dir /data/redis

sed -i “s/daemonize no/daemonize yes/g” /etc/redis/7963.conf
sed -i “s/port 6379/port 7963/g” /etc/redis/7963.conf
sed -i “s/pidfile \/var\/run\/redis.pid/pidfile \/var\/run\/redis_7963.pid/g” /etc/redis/7963.conf
sed -i “s/logfile \”\”/logfile \”\/var\/log\/redis\/7963.log\”/g” /etc/redis/7963.conf
sed -i “s/# requirepass foobared/requirepass 9k3NgZq%gO!W7x-0y=LI/g” /etc/redis/7963.conf
sed -i “s/dir .\//dir \/data\/redis/g” /etc/redis/7963.conf

grep vm.overcommit_memory /etc/sysctl.conf

echo “vm.overcommit_memory = 1” >> /etc/sysctl.conf

sed -i “s/vm.overcommit_memory = 0/vm.overcommit_memory = 1/g” /etc/sysctl.conf

sysctl -p

cd /opt/redis-3.0.5
cp utils/redis_init_script /etc/init.d/redis
vi /etc/init.d/redis
vi /etc/init.d/redis
———————————–

# chkconfig: 2345 90 10
# description: Simple Redis init.d scrip
REDISPORT=6379
REDISPORT=7963

CONF=”/etc/redis/${REDISPORT}.conf”
PASS=`grep “requirepass ” $CONF | awk {print’$2′}`
$CLIEXEC -p $REDISPORT shutdown
if [ -z “$PASS” ]
then
$CLIEXEC -p $REDISPORT shutdown
else
$CLIEXEC -p $REDISPORT -a $PASS shutdown
fi

sed -i “1 a # chkconfig: 2345 90 10” /etc/init.d/redis
sed -i “2 a # description: Simple Redis init.d scrip” /etc/init.d/redis
sed -i “s/REDISPORT=6379/REDISPORT=7963/g” /etc/init.d/redis
sed -i $’13 a PASS=`grep “requirepass ” $CONF | awk {print\’$2\’}`’ /etc/init.d/redis
sed -i “s/PORT shutdown/PORT -a \$PASS shutdown/g” /etc/init.d/redis

Redis
chkconfig redis on
chkconfig –list redis
service redis start
service redis stop

CentOS 7 Redis

## RHEL/CentOS 7 64-Bit ##
wget http://dl.Fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
rpm -ivh epel-release-7-5.noarch.rpm

yum repolist

yum -y update
yum install redis php-pecl-redis
systemctl start redis-server.service
systemctl enable redis-server.service
systemctl is-active redis-server.service

phpRedisAdmin RedisWeb Redis

git clone https://github.com/ErikDubbelboer/phpRedisAdmin.git
cd phpRedisAdmin/includes
cp config.sample.inc.php config.inc.php

# nano config.inc.php
RedisAdmin???????Apache

### nano /etc/httpd/conf.d/redisadmin.conf
### Now add the following ###
#
# Web Interface for RedisAdmin
#


Order Deny,Allow
Deny from all
Allow from 127.0.0.1
Allow from

Alias /redisAdmin /downloads/phpRedisAdmin
Alias /redisadmin /downloads/phpRedisAdmin

### nano /scripts/redis-check.sh
#!/bin/bash
PS=$(which ps)
GREP=$(which grep)
WHEN=$(date +”%Y-%m-%d-%H:%M:%S”)
if ! $PS aux | $GREP “redis.conf” | $GREP -v grep 2>&1 > /dev/null; then
/etc/init.d/redis restart
echo ‘Restarted Redis @’ $WHEN
fi
#Check Second instance
if ! $PS aux | $GREP “redis2.conf” | $GREP -v grep 2>&1 > /dev/null; then
/etc/init.d/redis2 restart
echo ‘Restarted Redis2 @’ $WHEN
fi

chmod +x /scripts/redis-check.sh

### nano /var/spool/cron/root
*/3 * * * * /bin/bash /script/redis-check.sh >> /var/log/redis-check.log

http://www.linuxidc.com/Linux/2016-06/132687.htm

tar xf redis-3.0.7.tar.gz -C /usr/local
mv redis-3.0.7 redis
cd /usr/local
make
make install
cp utils/redis_init_script /etc/init.d/redis

vim /etc/init.d/redis
# chkconfig: 2345 88 45
chmod +x /etc/init.d/redis
chkconfig –add redis

vim /usr/local/redis/redis.conf
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
port 7001
logfile “redis_7001.log”
dir “/data/redis/7001”
appendonly yes #????aof

mkdir /etc/redis/{7001,7002} -pv
mkdir /data/redis/{7001,7002} -pv
mkdir /etc/redis/{7003,7004} -pv
mkdir /data/redis/{7003,7004} -pv
mkdir /etc/redis/{7005,7006} -pv
mkdir /data/redis/{7005,7006} -pv
cd /etc/redis

sed ‘s/6379/7001/g’ /usr/local/redis/redis.conf > /etc/redis/7001/redis.conf
sed ‘s/6379/7002/g’ /usr/local/redis/redis.conf > /etc/redis/7002/redis.conf
sed ‘s/6379/7003/g’ /usr/local/redis/redis.conf > /etc/redis/7003/redis.conf
sed ‘s/6379/7004/g’ /usr/local/redis/redis.conf > /etc/redis/7004/redis.conf
sed ‘s/6379/7005/g’ /usr/local/redis/redis.conf > /etc/redis/7005/redis.conf
sed ‘s/6379/7006/g’ /usr/local/redis/redis.conf > /etc/redis/7006/redis.conf

/usr/local/redis/src/redis-server /etc/redis/700[1-6]/redis.conf

yum -y install ruby rubygems

gem install redis

cp /usr/local/redis/src/redis-trib.rb /usr/bin/redis-trib

redis-trib create –replicas 1 192.168.130.242:7001 192.168.130.242:7002
192.168.130.243:7003 192.168.130.243:7004 192.168.130.244:7005 192.168.130.244:7006

Centos 6.8

cd /opt
wget http://download.redis.io/releases/redis-3.0.5.tar.gz
tar zxvf redis-3.0.5.tar.gz
cd redis-3.0.5
make && make install

mkdir /etc/redis
mkdir /var/log/redis
mkdir -p /data/redis
cd /opt/redis-3.0.5
cp redis.conf /etc/redis/7963.conf
vi /etc/redis/7963.conf

daemonize no
port 6379
logfile “”
pidfile /var/run/redis.pid
# requirepass foobared
dir ./
??
daemonize yes
port 7963
logfile “/var/log/redis/7963.log”
pidfile /var/run/redis_7963.pid
requirepass 9k3NgZq%gO!W7x-0y=LI
dir /data/redis

sed -i “s/daemonize no/daemonize yes/g” /etc/redis/7963.conf
sed -i “s/port 6379/port 7963/g” /etc/redis/7963.conf
sed -i “s/pidfile \/var\/run\/redis.pid/pidfile \/var\/run\/redis_7963.pid/g” /etc/redis/7963.conf
sed -i “s/logfile \”\”/logfile \”\/var\/log\/redis\/7963.log\”/g” /etc/redis/7963.conf
sed -i “s/# requirepass foobared/requirepass 9k3NgZq%gO!W7x-0y=LI/g” /etc/redis/7963.conf
sed -i “s/dir .\//dir \/data\/redis/g” /etc/redis/7963.conf

grep vm.overcommit_memory /etc/sysctl.conf

echo “vm.overcommit_memory = 1” >> /etc/sysctl.conf

sed -i “s/vm.overcommit_memory = 0/vm.overcommit_memory = 1/g” /etc/sysctl.conf

sysctl -p

cd /opt/redis-3.0.5
cp utils/redis_init_script /etc/init.d/redis
vi /etc/init.d/redis
vi /etc/init.d/redis
———————————–

# chkconfig: 2345 90 10
# description: Simple Redis init.d scrip
REDISPORT=6379
REDISPORT=7963

CONF=”/etc/redis/${REDISPORT}.conf”
PASS=`grep “requirepass ” $CONF | awk {print’$2′}`
$CLIEXEC -p $REDISPORT shutdown
if [ -z “$PASS” ]
then
$CLIEXEC -p $REDISPORT shutdown
else
$CLIEXEC -p $REDISPORT -a $PASS shutdown
fi

sed -i “1 a # chkconfig: 2345 90 10” /etc/init.d/redis
sed -i “2 a # description: Simple Redis init.d scrip” /etc/init.d/redis
sed -i “s/REDISPORT=6379/REDISPORT=7963/g” /etc/init.d/redis
sed -i $’13 a PASS=`grep “requirepass ” $CONF | awk {print\’$2\’}`’ /etc/init.d/redis
sed -i “s/PORT shutdown/PORT -a \$PASS shutdown/g” /etc/init.d/redis

Redis
chkconfig redis on
chkconfig –list redis
service redis start
service redis stop

CentOS 7 Redis

## RHEL/CentOS 7 64-Bit ##
wget http://dl.Fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
rpm -ivh epel-release-7-5.noarch.rpm

yum repolist

yum -y update
yum install redis php-pecl-redis
systemctl start redis-server.service
systemctl enable redis-server.service
systemctl is-active redis-server.service

phpRedisAdmin RedisWeb Redis

git clone https://github.com/ErikDubbelboer/phpRedisAdmin.git
cd phpRedisAdmin/includes
cp config.sample.inc.php config.inc.php

# nano config.inc.php
RedisAdmin???????Apache

### nano /etc/httpd/conf.d/redisadmin.conf
### Now add the following ###
#
# Web Interface for RedisAdmin
#


Order Deny,Allow
Deny from all
Allow from 127.0.0.1
Allow from

Alias /redisAdmin /downloads/phpRedisAdmin
Alias /redisadmin /downloads/phpRedisAdmin

### nano /scripts/redis-check.sh
#!/bin/bash
PS=$(which ps)
GREP=$(which grep)
WHEN=$(date +”%Y-%m-%d-%H:%M:%S”)
if ! $PS aux | $GREP “redis.conf” | $GREP -v grep 2>&1 > /dev/null; then
/etc/init.d/redis restart
echo ‘Restarted Redis @’ $WHEN
fi
#Check Second instance
if ! $PS aux | $GREP “redis2.conf” | $GREP -v grep 2>&1 > /dev/null; then
/etc/init.d/redis2 restart
echo ‘Restarted Redis2 @’ $WHEN
fi

chmod +x /scripts/redis-check.sh

### nano /var/spool/cron/root
*/3 * * * * /bin/bash /script/redis-check.sh >> /var/log/redis-check.log

http://www.linuxidc.com/Linux/2016-06/132687.htm

tar xf redis-3.0.7.tar.gz -C /usr/local
mv redis-3.0.7 redis
cd /usr/local
make
make install
cp utils/redis_init_script /etc/init.d/redis

vim /etc/init.d/redis
# chkconfig: 2345 88 45
chmod +x /etc/init.d/redis
chkconfig –add redis

vim /usr/local/redis/redis.conf
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
port 7001
logfile “redis_7001.log”
dir “/data/redis/7001”
appendonly yes #????aof

mkdir /etc/redis/{7001,7002} -pv
mkdir /data/redis/{7001,7002} -pv
mkdir /etc/redis/{7003,7004} -pv
mkdir /data/redis/{7003,7004} -pv
mkdir /etc/redis/{7005,7006} -pv
mkdir /data/redis/{7005,7006} -pv
cd /etc/redis

sed ‘s/6379/7001/g’ /usr/local/redis/redis.conf > /etc/redis/7001/redis.conf
sed ‘s/6379/7002/g’ /usr/local/redis/redis.conf > /etc/redis/7002/redis.conf
sed ‘s/6379/7003/g’ /usr/local/redis/redis.conf > /etc/redis/7003/redis.conf
sed ‘s/6379/7004/g’ /usr/local/redis/redis.conf > /etc/redis/7004/redis.conf
sed ‘s/6379/7005/g’ /usr/local/redis/redis.conf > /etc/redis/7005/redis.conf
sed ‘s/6379/7006/g’ /usr/local/redis/redis.conf > /etc/redis/7006/redis.conf

/usr/local/redis/src/redis-server /etc/redis/700[1-6]/redis.conf

yum -y install ruby rubygems

gem install redis

cp /usr/local/redis/src/redis-trib.rb /usr/bin/redis-trib

redis-trib create –replicas 1 192.168.130.242:7001 192.168.130.242:7002
192.168.130.243:7003 192.168.130.243:7004 192.168.130.244:7005 192.168.130.244:7006

Mysql

wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
rpm -ivh mysql-community-release-el7-5.noarch.rpm
yum update
yum install mysql-server

systemctl start mysqld

[root@clusterserver1 ~]# systemctl start mysqld
[root@clusterserver1 ~]# systemctl enable mysqld
[root@clusterserver1 ~]# systemctl status mysqld

[root@clusterserver1 ~]# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MySQL to secure it, we’ll need the current
password for the root user. If you’ve just installed MySQL, and
you haven’t set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on…

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

Set root password? [Y/n] y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
… Success!

By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] y
… Success!

Normally, root should only be allowed to connect from ‘localhost’. This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y
… Success!

By default, MySQL comes with a database named ‘test’ that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] y
– Dropping test database…
ERROR 1008 (HY000) at line 1: Can’t drop database ‘test’; database doesn’t exist
… Failed! Not critical, keep moving…
– Removing privileges on test database…
… Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y
… Success!

All done! If you’ve completed all of the above steps, your MySQL
installation should now be secure.

Thanks for using MySQL!

Cleaning up…
[root@clusterserver1 ~]# systemctl restart mysqld

[root@clusterserver1 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.33 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql>

create database testdb;
create user ‘testuser’@’localhost’ identified by ‘password’;
grant all on testdb.* to ‘testuser’ identified by ‘password’;

mysql> create database testdb;
Query OK, 1 row affected (0.00 sec)

mysql> create user ‘testuser’@’localhost’ identified by ‘password’;
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on testdb.* to ‘testuser’ identified by ‘password’;
Query OK, 0 rows affected (0.00 sec)

mysql>

create database testdb;
grant all on testdb.* to ‘testuser’ identified by ‘password’;

mysql -u testuser -p

use testdb;
create table customers (customer_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, first_name TEXT, last_name TEXT);

[root@clusterserver1 ~]# mysql -u testuser -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.6.33 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql> use testdb;
Database changed
mysql> create table customers (customer_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, first_name TEXT, last_name TEXT);
Query OK, 0 rows affected (0.01 sec)

mysql>

Reset the MySQL Root Password

If you forget your root MySQL password, it can be reset.

Stop the current MySQL server instance, then restart it with an option to not ask for a password.

systemctl stop mysqld
mysqld_safe –skip-grant-tables &

Reconnect to the MySQL server with the MySQL root account.

mysql -u root

Use the following commands to reset root’s password. Replace password with a strong password.

use mysql;
update user SET PASSWORD=PASSWORD(“password”) WHERE USER=’root’;
flush privileges;
exit

Then restart MySQL.

systemctl start mysqld

Tune MySQL

MySQL Tuner is a Perl script that connects to a running instance of MySQL and provides configuration recommendations based on workload. Ideally, the MySQL instance should have been operating for at least 24 hours before running the tuner. The longer the instance has been running, the better advice MySQL Tuner will give.

Download MySQL Tuner to your home directory.

wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl

To run it:

perl ./mysqltuner.pl

You will be asked for the MySQL root user’s name and password. The output will show two areas of interest: General recommendations and Variables to adjust.

MySQL Tuner is an excellent starting point to optimize a MySQL server but it would be prudent to perform additional research for configurations tailored to the application(s) utilizing MySQL on your Linode.

docker cmd

Basic Instructions

Download the image file

$ docker pull image_name

Activate or deactivate the container
$ docker [start|stop] container_name

Download the image file

$ docker pull image_name

Activate or deactivate the container

$ docker [start|stop] container_name
Build -> Start -> Execute command ( -ti parameter)

$ docker run -ti –name container_name image_name command
Build -> Start -> Execute command -> Delete Container ( -rm parameters)

$ docker run –rm -ti image_name command
file system and the corresponding port ( -v and -p parameter)

$ docker run -ti –rm -p 80:80 -v /your_path:/container_path -e PASSWORD=1234 image_name
Docker Clear (cleanup) instruction
Delete all container being executed

$ docker kill $(docker ps -q)

Remove dangling image file

docker rmi $(docker images -q -f dangling=true)

Remove all stopped container

docker rm $(docker ps -a -q)
Docker machine instruction

Start machine

$ docker-machine start machine_name

Specifies the machine to set the docker

$ eval “$(docker-machine env machine_name)”
Interactive instruction with container

Instruction in the container

$ docker exec -ti container_name command

Online Watch container logs

$ docker logs -ft container_name

Storage container being implemented into image files

$ docker commit -m “message” -a “author” container_name username/image_name:tag

docker compose format

docker-compose.yml following format

version: “2”
service:
container_name: “hello-world”
image: golang
command: “go run hello-world.go”
ports
– “80:8080”
volumes:
– /hello-world:/root/hello-world
redis:
image: redis

Build -> Execute container

$ docker-compose up
These are commonly used docker instruction, be sure to learn for beginners.

CentOS 7 Change Network Interface Name From Eno* To Eth0

CentOS 7 Change Network Interface Name From Eno* To Eth0

After you installed CentOS 7.0 or RHEL 7.0 system, you maybe notice that the default name of network interface was changed from “Eth*” to “Eno*”. How do I change network interface name to “Eth0? in CentOS 7.0? This post will show you the ways to change network interface.
CentOS 7 Change Network Interface Name
Step1# issue the ifconfig command to check the current network interface information, type:
# yum install net-tools

[root@localhost Desktop]$sudo ifconfig
eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.42.129 netmask 255.255.255.0 broadcast 192.168.42.255
inet6 fe80::20c:29ff:fec7:25ae prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:c7:25:ae txqueuelen 1000 (Ethernet)
RX packets 200948 bytes 253071365 (241.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 56043 bytes 3420351 (3.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Step2# edit “/etc/sysconfig/grub” configuration file using “vim” command:
before:

[root@localhost Desktop]$sudo vim /etc/sysconfig/grub
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT=”console”
GRUB_CMDLINE_LINUX=”rd.lvm.lv=centos/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos/root crashkernel=auto vconsole.keymap=us rhgb quiet”
GRUB_DISABLE_RECOVERY=”true”
then searching for “GRUB_CMDLINE_LINUX” line and appending the following content “net.ifnames=0 biosdevname=0“, just like the below:
1
GRUB_CMDLINE_LINUX=”rd.lvm.lv=centos/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos/root crashkernel=auto vconsole.keymap=us rhgb quiet net.ifnames=0 biosdevname=0″
Step3# Using “grub2-mkconfig” command to re-generate a new grub configuration file, type:

[root@localhost Desktop]$sudo grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file …
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Warning: Please don’t use old title `CentOS Linux, with Linux 3.10.0-123.el7.x86_64′ for GRUB_DEFAULT, use `Advanced options for CentOS Linux CentOS Linux, with Linux 3.10.0-123.el7.x86_64′ (for versions before 2.00) or `gnulinux-advanced-dbedd8fa-5d86-4ea0-8551-8444a48cd44f nulinux-3.10.0-123.el7.x86_64-advanced-dbedd8fa-5d86-4ea0-8551-8444a48cd44f’ (for 2.00 or later)
Found linux image: /boot/vmlinuz-0-rescue-3303e35a730a41e3b4e99b544acea205
Found initrd image: /boot/initramfs-0-rescue-3303e35a730a41e3b4e99b544acea205.img
done
Step4# Rename “Eno” network file using ”mv” command, type:

$sudo mv /etc/sysconfig/network-scripts/ifcfg-eno16777736 /etc/sysconfig/network-scripts/ifcfg-eth0

Step5# Edit “/etc/sysconfig/network-scripts/ifcfg-eth0 ” configuration file and set the value of “Name” field to “eth0?.

# vim /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=eth0
UUID=5ae10208-855b-41af-99e7-0673d3792d15
ONBOOT=yes
HWADDR=00:0C:29:C7:25:AE
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
?
Step6# reboot system, after rebooting system, using “ifconfig” command check network interface information again.

[root@localhost Desktop]# ifconfig
eth0: flags=4163 UP,BROADCAST,RUNNING,MULTICAST mtu 1500
inet 192.168.42.129 netmask 255.255.255.0 broadcast 192.168.42.255
inet6 fe80::20c:29ff:fec7:25ae prefixlen 64 scopeid 0x20
ether 00:0c:29:c7:25:ae txqueuelen 1000 (Ethernet)
RX packets 49 bytes 5285 (5.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 76 bytes 8540 (8.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
done….

GlusterFS servers on RHEL 7 and Centos 7

GlusterFS servers on RHEL 7 and Centos 7

vi /etc/hosts

192.168.1.71 apache1.rmohan.com apache1
192.168.1.72 apache2.rmohan.com apache2
192.168.1.74 apache3.rmohan.com apache3

For this lab, we disabled both firewall and SELinux

vi /etc/selinux/config
egrep “^SELINUX=” /etc/sysconfig/selinux
SELINUX=disabled
systemctl is-enabled firewalld
disabled

systemctl disable firewalld
yum update
yum -y install wget openssh* net-tools ntp
ntpdate -u 0.centos.pool.ntp.org

Before proceeding with the installation, we need to enable both EPEL (Exta Packages for Enterprise Linux) and GlusterFS repositories on both the servers to resolve the external dependencies at the time of installation. If you have enabled only GlusterFS repository and not enabled EPEL repository then it is likely that you will hit the following error message while installing glusterfs-server:
–> Finished Dependency Resolution
Error: Package: glusterfs-server-3.7.11-1.el7.x86_64 (glusterfs-epel)
Requires: liburcu-cds.so.1()(64bit)
Error: Package: glusterfs-server-3.7.11-1.el7.x86_64 (glusterfs-epel)
Requires: liburcu-bp.so.1()(64bit)
You could try using –skip-broken to work around the problem
You could try running: rpm -Va –nofiles –nodigest

EPEL installation
Retrieve EPEL repository : all 3 servers
wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-6.noarch.rpm

Gluster installation :
Retrieve GlusterFS repository : all 3 servers
wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/RHEL/glusterfs-epel.repo

Download GlusterFS packages : all 3 servers
yum -y install glusterfs-server

Now, start the service and keep it started across reboot :
[root@apache3 ~]# systemctl start glusterd && systemctl enable glusterd
Created symlink from /etc/systemd/system/multi-user.target.wants/glusterd.service to /usr/lib/systemd/system/glusterd.service.
[root@apache3 ~]#

Update “/etc/hosts” :
192.168.1.71 apache1.rmohan.com apache1
192.168.1.72 apache2.rmohan.com apache2
192.168.1.74 apache3.rmohan.com apache3

Check the 2 GlusterFS servers

[root@apache1 ~]# gluster peer probe apache2
peer probe: success.
[root@apache1 ~]# gluster peer probe apache3
peer probe: success.
[root@apache1 ~]#

[root@apache2 ~]# gluster peer probe apache3
peer probe: success. Host apache3 port 24007 already in peer list
[root@apache2 ~]# gluster peer probe apache1
peer probe: success. Host apache1 port 24007 already in peer list
[root@apache2 ~]#

[root@apache3 ~]# gluster peer probe apache1
peer probe: success. Host apache1 port 24007 already in peer list
[root@apache3 ~]# gluster peer probe apache2
peer probe: success. Host apache2 port 24007 already in peer list
[root@apache3 ~]#

Check the Peers :
root@apache1 ~]# gluster peer status
Number of Peers: 2

Hostname: apache2
Uuid: 09db776a-ab96-47a3-ad77-6bc77906566b
State: Peer in Cluster (Connected)

Hostname: apache3
Uuid: 14db2875-378d-481d-968c-552a10f2fd7e
State: Peer in Cluster (Connected)
[root@apache1 ~]#

[root@apache2 ~]# gluster peer status
Number of Peers: 2

Hostname: apache1.rmohan.com
Uuid: 16612fd4-8b94-4006-8ed7-24ac33c564e0
State: Peer in Cluster (Connected)
Other names:
apache1

Hostname: apache3
Uuid: 14db2875-378d-481d-968c-552a10f2fd7e
State: Peer in Cluster (Connected)

[root@apache3 ~]# gluster peer status
Number of Peers: 2

Hostname: apache1.rmohan.com
Uuid: 16612fd4-8b94-4006-8ed7-24ac33c564e0
State: Peer in Cluster (Connected)
Other names:
apache1

Hostname: apache2
Uuid: 09db776a-ab96-47a3-ad77-6bc77906566b
State: Peer in Cluster (Connected)

Create GlusterFS volume
We add 2 new drives from VMware ESXi (in fact, the 3 GlusterFS servers are 3 virtual machines) : do the below commands on all 3 servers

[root@apache1 ~]# for HOST in $(ls /sys/class/scsi_host) ; do echo ‘- – -‘>”/sys/class/scsi_host/$HOST/scan” ; done
[root@apache1 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
??sda1 xfs be8c0a37-1ed9-4743-b4ef-03d59ba3d06d /boot
??sda2 LVM2_member YXOIJV-EPlD-dXwg-ePQX-D7av-jPdr-Grb4rp
??centos-root xfs d2c46dc9-6365-4345-9b60-021aabfd0525 /
??centos-swap swap 74c70324-6875-40e6-9d8f-591d3325c06e [SWAP]
??centos-home xfs bd8e9210-bc95-4af9-be87-a33da31f8dbb /home
sdb
sr0 iso9660 CentOS 7 x86_64 2016-04-01-13-21-01-00
[root@apache1 ~]#

[root@apache1 ~]# parted -s /dev/sdb mklabel gpt && parted -s /dev/sdb mkpart primary xfs 0% 100% && mkfs.xfs -f /dev/sdb
meta-data=/dev/sdb isize=256 agcount=4, agsize=1310720 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0
data = bsize=4096 blocks=5242880, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[root@apache1 ~]#

Create gluster and mount on all 3 nodes

[root@apache1 ~]# mkdir /gluster
[root@apache1 ~]# vi /etc/fstab
[root@apache1 ~]# grep -i gluster /etc/fstab
/dev/sdb /gluster xfs defaults 0 0
[root@apache1 ~]#

[root@apache1 ~]# mount -a
[root@apache1 ~]# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/centos-root xfs 38G 1.2G 37G 3% /
devtmpfs devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs tmpfs 1.9G 8.6M 1.9G 1% /run
tmpfs tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 xfs 497M 186M 312M 38% /boot
/dev/mapper/centos-home xfs 19G 33M 19G 1% /home
tmpfs tmpfs 371M 0 371M 0% /run/user/0
/dev/sdb xfs 20G 33M 20G 1% /gluster
[root@apache1 ~]#

Create a directory within the file system on the servers :
mkdir -p /gluster/briks

Create the GlusterFS volume based on those 3 replicas :

gluster volume create appgluster_vol replica 3 apache1:/gluster/briks apache2:/gluster/briks apache3:/gluster/briks

[root@apache1 ~]# gluster volume create appgluster_vol replica 3 apache1:/gluster/briks apache2:/gluster/briks apache3:/gluster/briks
volume create: appgluster_vol: success: please start the volume to access data
[root@apache1 ~]#

Then, start this volume :

[root@apache1 ~]# gluster volume start appgluster_vol
volume start: appgluster_vol: success
[root@apache1 ~]#

[root@apache1 ~]# gluster volume info

Volume Name: appgluster_vol
Type: Replicate
Volume ID: 856f1052-9d19-49ee-b0c1-1e53d32b86f4
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: apache1:/gluster/briks
Brick2: apache2:/gluster/briks
Brick3: apache3:/gluster/briks
Options Reconfigured:
performance.readdir-ahead: on
[root@apache1 ~]#

[root@apache1 ~]# gluster volume status
Status of volume: appgluster_vol
Gluster process TCP Port RDMA Port Online Pid
——————————————————————————
Brick apache1:/gluster/briks 49152 0 Y 3108
Brick apache2:/gluster/briks 49152 0 Y 2923
Brick apache3:/gluster/briks 49152 0 Y 2892
NFS Server on localhost 2049 0 Y 3128
Self-heal Daemon on localhost N/A N/A Y 3136
NFS Server on apache2 2049 0 Y 2943
Self-heal Daemon on apache2 N/A N/A Y 2951
NFS Server on apache3 2049 0 Y 2912
Self-heal Daemon on apache3 N/A N/A Y 2920

Task Status of Volume appgluster_vol
——————————————————————————
There are no active volume tasks

[root@apache1 ~]#

Client side
Both firewall and SELinux are disabled :

192.168.1.21 clusterserver2.rmohan.com clusterserver2
192.168.1.26 clusterserver3.rmohan.com clusterserver3

one the clinet side add the host file
192.168.1.21 clusterserver2.rmohan.com clusterserver2
192.168.1.26 clusterserver3.rmohan.com clusterserver3
192.168.1.71 apache1.rmohan.com apache1
192.168.1.72 apache2.rmohan.com apache2
192.168.1.74 apache3.rmohan.com apache3

[root@clusterserver3 ~]# mount.glusterfs apache1:/appgluster_vol /mnt/
[root@clusterserver3 ~]# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/centos-root xfs 50G 3.6G 47G 8% /
devtmpfs devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs tmpfs 1.9G 16K 1.9G 1% /dev/shm
tmpfs tmpfs 1.9G 17M 1.8G 1% /run
tmpfs tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/loop0 ext4 1.9G 6.1M 1.7G 1% /srv/node/swiftloopback
/dev/mapper/centos-home xfs 46G 33M 46G 1% /home
/dev/sda1 xfs 497M 213M 285M 43% /boot
tmpfs tmpfs 371M 0 371M 0% /run/user/0
apache1:/appgluster_vol fuse.glusterfs 20G 33M 20G 1% /mnt
[root@clusterserver3 ~]#

[root@clusterserver3 ~]# echo “here is a GlusterFS file” > /mnt/file

[root@apache1 ~]# cat /gluster/briks/file
here is a GlusterFS file

[root@apache2 ~]# cat /gluster/briks/file
here is a GlusterFS file
[root@apache2 ~]#

[root@apache3 ~]# cat /gluster/briks/file
here is a GlusterFS file
[root@apache3 ~]#

Replication Features in MySQL 8.0.1

MySQL 8.0.1 has just been released with a wonderful set of features across the board. Specifically in replication, there is plenty of new and exciting functionality as well as performance improvements.  In this post I am going to summarize the replication features of this development milestone release.

  • The Group Replication plugin. You can now use Group Replication together with MySQL 8.0.1. Group Replication had been released as a server plugin in MySQL 5.7.17 and it is now released together with MySQL 8.0.1 as well.
  • Transaction SAVEPOINT support in Group Replication. Transaction savepoints are now supported in Group Replication.
  • Commit timestamps in the binary log. The binary log generated by a MySQL 8.0.1 master contains two new timestamps referring to the points in time when the transaction was committed on the original master and on the immediate master. This is interesting in chained replication scenarios, as transactions may have committed several hops up the chain. As such, these timestamps are useful for tracking how much a slave is behind the immediate and original master, or for instance, to compute more precisely how much delay the slave applier should introduce while handling a transaction.
  • Improved Delayed Replication. By relying on the extra timestamp information in the binary log, immediate commit and original commit timestamps, the binary log applier is capable of better calculating the delay it needs to introduce. This work makes the delay calculation more reliable in chained replication scenarios.
  • More instrumentation for the replication applier. New fields were added to the existing performance schema tables. The user can now see at which time the last transaction in the applier was queued in the relay log, when it was processed and when it was applied. In addition, the newly introduced immediate and original commit timestamps are also exposed. This feature will greatly help users to profile the applier execution and conduct analysis when they find that for some reason there is replication lag.
  • Channel specific filters in multi-source replication. This allows user to specify different replication filter rules for different replication channels. Therefore, the slave will be able to filter out (or transform) the execution of selected replicated changes on a per-source basis.
  • Additional Metadata in the binary log. The binary log from a MySQL 8.0.1 server shall contain additional table metadata information into the ROW-based changes. For instance, whether a numeric column is SIGNED or UNSIGNED, additional information about column character set, or even the name of the columns. The user is also able to control the amount of metadata logged.
  • RESET MASTER command extensions. The RESET MASTER command was extended. The user can now specify the binary log file index number to start from once the reset happens, instead of starting from .000001. The bulk of this patch was contributed by Daniël van Eeden (see BUG#77482). Thank you Daniël!
  • Synchronized DDL and binary log commits. With the new data dictionary work done in MySQL 8, there are several DDL statements that were made atomic. As such, DDLs can be synchronized with the binary log commit operation, and as a consequence both are kept in sync. In the event of the server crashing while the DDL changes are being written to the binary log, then on recovery, the DDL will either be committed or rolled back, depending whether the DDL changes were fully written to the binary log or not, respectively. In summary, several DDL statements are now as crash-safe as regular DML statements with respect to replication operations.
  • More efficient replication threads. By improving the synchronization between receiver (I/O) and applier (SQL) threads, the slave applier throughput goes up considerably. This improvement alone can, in some workloads, get a throughput increase of up to 24% on Sysbench RW and up to 64% on Sysbench Update Index.
  • Write-set based multi-threaded applier scheduling policy.  Group replication uses write-sets to first detect conflicting transactions and also to establish dependencies between transactions, deeming which can and cannot be applied in parallel. In MySQL 8.0.1 this functionality was extended to asynchronous replication by making the master also track dependencies between transactions by looking into their write-sets. As such, it can calculate the optimal parallelization window and encode that in the binary log. Then the applier on the slave can use this information to apply as many transactions in parallel as possible. This approach is quite interesting, since this enhancement means that the slave can even apply in parallel a workload that is single threaded on the master! Typically, this will exhibit a large throughput improvement, even when compared to the commit order parallelization scheme introduced in MySQL 5.7 which was already a huge leap forward back then.
  • Configuration variable to expire binary logs after N seconds. A new system variable, binlog_expire_logs_seconds, was introduced. It allows the user to instruct the server to automatically expire a log that is older than the number of seconds configured.

MySQL 8.0.1 is an interesting release. In the replication context alone, it exhibits quite a compelling list of new features. These range from more instrumentation and monitoring extensions, to additional metadata and information in the binary log, to better performance and efficiency and to improved configurability and overall management of the replication framework. But, there is no need to repeat myself. And as a matter of fact, there shall be additional blog posts detailing the new features, so stay tuned! Meanwhile, you can download the new MySQL 8.0.1 DMR and check it out. As usual, lets us know your feedback, e.g., through bug reportsfeature requests, the replication mailing list or even through comments on this or other blogs on this website. Mind you that MySQL 8.0.1 is a development milestone release (DMR), thence not declared generally available yet. Use it at your own risk.

Enjoy MySQL 8.0.1.