September 2019
M T W T F S S
« Aug    
 1
2345678
9101112131415
16171819202122
23242526272829
30  

Categories

WordPress Quotes

Nothing can stop the man with the right mental attitude from achieving his goal; nothing on earth can help the man with the wrong mental attitude.
Thomas Jefferson
September 2019
M T W T F S S
« Aug    
 1
2345678
9101112131415
16171819202122
23242526272829
30  

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (268)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
EXCHANGE (3)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (1)
horoscope (23)
Hyper-V (10)
IIS (15)
IPTABLES (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
SCALEIO (1)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)
ZIMBRA (17)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

33 visitors online now
5 guests, 28 bots, 0 members

Hit Counter provided by dental implants orange county

pyenv + pyenv-virtualenv (CentOS 7)

Overview
I summarized the installation on CentOS.
Although it is little different from OSX, it is organized because the premise environment is different.

environment
I am trying in the next environment.

$ uname -r
3.10.0-229.14.1.el7.x86_64
$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.1.1503 (Core)
Release: 7.1.1503
Codename: Core

Installing packages

Installation of packages required for CentOS

yum -y install git
yum -y groupinstall “Development Tools”
yum -y install readline-devel zlib-devel bzip2-devel sqlite-devel openssl-devel

Installation of pyenv

git clone https://github.com/yyuu/pyenv.git ~/.pyenv
echo ‘export PYENV_ROOT=”$HOME/.pyenv”‘ >> ~/.bash_profile
echo ‘export PATH=”$PYENV_ROOT/bin:$PATH”‘ >> ~/.bash_profile
echo ‘eval “$(pyenv init -)”‘ >> ~/.bash_profile
source ~/.bash_profile
exec $SHELL -l

pyenv-virtualenv

git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv
echo ‘eval “$(pyenv virtualenv-init -)”‘ >> ~/.bash_profile
exec $SHELL -l

How to use pyenv

List of installable distributions, versions

pyenv install 2.7.10
pyenv install 3.5.0

Installation

Installation of the version you want to use

$ pyenv versions

  • system (set by /home/saitou/.pyenv/version)
    2.7.10
    3.5.0

Changing the version to be used by default

Do not make changes to Python on the system side.
In the following, 2.7.10 which was additionally installed is changed to the default of the user environment.

pyenv global 2.7.10
pip install – U pip
If you want to use Python on the system side, specify system.

$ python -V
Python 2.7.5
$ pyenv global 2.7.10
$ python -V
Python 2.7.10
$ pyenv global system
$ python -V
Python 2.7.5

How to use pyenv + pyenv-virtualenv

pyenv virtualenv

I want to use different versions for each project.
In this way you can allocate specific directories to specific versions and use them separately.

$ pyenv virtualenv 3.5.0 new_env
$ mkdir -p work/new_project && work/new_project/
$ pyenv versions

  • system (set by /home/saitou/.pyenv/version)
    2.7.10
    3.5.0
    new_env
    $ pyenv local new_env
    pyenv-virtualenv: activate new_env
    $ python -V
    Python 3.5.0
    $ cd ..
    pyenv-virtualenv: deactivate new_env
    $ python -V
    Python 2.7.5
    $ cd new_project/
    pyenv-virtualenv: activate new_env
    $ python -V
    Python 3.5.0

Yellowdog Update Modified

Yum (Yellowdog Update Modified) is an RPM package management tool used on CentOS and RedHat systems. The yum history command allows the system administrator to roll back the system to the previous state, but due to some limitations, rollback is not possible in all cases Success, sometimes yum command may do nothing, and sometimes may delete some other packages.

I suggest that you still have to do a complete system backup before you upgrade, and yum history can not be used to replace the system backup. System backup allows you to restore the system to an arbitrary node status.

In some cases, what should I do if the installed application does not work or has some errors after it has been patched (possibly due to library incompatibilities or package upgrades)?

Talk to the application development team and find out where the problem is with libraries and packages, then use the yum history command to roll back.

Server patching is one of the important task of Linux system administrator to make the system more stable and better performance. All the vendors used to release security/vulnerabilities patches very often, the affected package must be updated in order to limit any potential security risks.

Yum (Yellowdog Update Modified) is RPM Package Management utility for CentOS and Red Hat systems, Yum history command allows administrator to rollback the system to a previous state but due to some limitations, rollbacks do not work in all situations, or The yum command may simply do nothing, or it may remove packages you do not expect.

I advise you to take a full system backup prior to performing any update/upgrade is always recommended, and yum history is NOT meant to replace systems backups. This will help you to restore the system to previous state at any point of time.

yum update
Loaded plugins: fastestmirror, security
Setting up Update Process
Loading mirror speeds from cached hostfile
epel/metalink | 12 kB 00:00

  • epel: mirror.csclub.uwaterloo.ca
    base | 3.7 kB 00:00
    dockerrepo | 2.9 kB 00:00
    draios | 2.9 kB 00:00
    draios/primary_db | 13 kB 00:00
    epel | 4.3 kB 00:00
    epel/primary_db | 5.9 MB 00:00
    extras | 3.4 kB 00:00
    updates | 3.4 kB 00:00
    updates/primary_db | 2.5 MB 00:00
    Resolving Dependencies
    –> Running transaction check
    —> Package git.x86_64 0:1.7.1-8.el6 will be updated
    —> Package git.x86_64 0:1.7.1-9.el6_9 will be an update
    —> Package httpd.x86_64 0:2.2.15-60.el6.centos.4 will be updated
    —> Package httpd.x86_64 0:2.2.15-60.el6.centos.5 will be an update
    —> Package httpd-tools.x86_64 0:2.2.15-60.el6.centos.4 will be updated
    —> Package httpd-tools.x86_64 0:2.2.15-60.el6.centos.5 will be an update
    —> Package perl-Git.noarch 0:1.7.1-8.el6 will be updated
    —> Package perl-Git.noarch 0:1.7.1-9.el6_9 will be an update
    –> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================

Package Arch Version Repository Size

Updating:
git x86_64 1.7.1-9.el6_9 updates 4.6 M
httpd x86_64 2.2.15-60.el6.centos.5 updates 836 k
httpd-tools x86_64 2.2.15-60.el6.centos.5 updates 80 k
perl-Git noarch 1.7.1-9.el6_9 updates 29 k

Transaction Summary

Upgrade 4 Package(s)

Total download size: 5.5 M
Is this ok [y/N]: n

As you can see in the above output git package update is available, so we are going to take that. Run the following command to know the version information about the package (current installed version and available update version).

yum list git
Loaded plugins: fastestmirror, security
Setting up Update Process
Loading mirror speeds from cached hostfile

  • epel: mirror.csclub.uwaterloo.ca
    Installed Packages
    git.x86_64 1.7.1-8.el6 @base
    Available Packages
    git.x86_64

Run the following command to update git package from 1.7.1-8 to 1.7.1-9.

yum update git

Loaded plugins: fastestmirror, presto
Setting up Update Process
Loading mirror speeds from cached hostfile

  • base: repos.lax.quadranet.com
  • epel: fedora.mirrors.pair.com
  • extras: mirrors.seas.harvard.edu
  • updates: mirror.sesp.northwestern.edu
    Resolving Dependencies
    –> Running transaction check
    —> Package git.x86_64 0:1.7.1-8.el6 will be updated
    –> Processing Dependency: git = 1.7.1-8.el6 for package: perl-Git-1.7.1-8.el6.noarch
    —> Package git.x86_64 0:1.7.1-9.el6_9 will be an update
    –> Running transaction check
    —> Package perl-Git.noarch 0:1.7.1-8.el6 will be updated
    —> Package perl-Git.noarch 0:1.7.1-9.el6_9 will be an update
    –> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================

Package Arch Version Repository Size

Updating:
git x86_64 1.7.1-9.el6_9 updates 4.6 M
Updating for dependencies:
perl-Git noarch 1.7.1-9.el6_9 updates 29 k

Transaction Summary

Upgrade 2 Package(s)

Total download size: 4.6 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 4.6 M
(1/2): git-1.7.1-9.el6_9.x86_64.rpm | 4.6 MB 00:00

(2/2): perl-Git-1.7.1-9.el6_9.noarch.rpm | 29 kB 00:00

Total 5.8 MB/s | 4.6 MB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : perl-Git-1.7.1-9.el6_9.noarch 1/4
Updating : git-1.7.1-9.el6_9.x86_64 2/4
Cleanup : perl-Git-1.7.1-8.el6.noarch 3/4
Cleanup : git-1.7.1-8.el6.x86_64 4/4
Verifying : git-1.7.1-9.el6_9.x86_64 1/4
Verifying : perl-Git-1.7.1-9.el6_9.noarch 2/4
Verifying : git-1.7.1-8.el6.x86_64 3/4
Verifying : perl-Git-1.7.1-8.el6.noarch 4/4

Updated:
git.x86_64 0:1.7.1-9.el6_9

Dependency Updated:
perl-Git.noarch 0:1.7.1-9.el6_9

Complete!

Verify updated version of git package.

yum list git

Installed Packages
git.x86_64 1.7.1-9.el6_9 @updates

or

rpm -q git

git-1.7.1-9.el6_9.x86_64

As of now, we have successfully completed package update and got a package for rollback. Just follow below steps for rollback mechanism.

First get the yum transaction id using following command. The below output clearly shows all the required information such transaction id, who done the transaction (i mean username), date and time, Actions (Install or update), how many packages altered in this transaction.

yum history

or

yum history list all

Loaded plugins: fastestmirror, presto

ID | Login user | Date and time | Action(s) | Altered

13 | root               | 2017-08-18 13:30 | Update         |    2
12 | root               | 2017-08-10 07:46 | Install        |    1
11 | root               | 2017-07-28 17:10 | E, I, U        |   28 EE
10 | root               | 2017-04-21 09:16 | E, I, U        |  162 EE
 9 | root               | 2017-02-09 17:09 | E, I, U        |   20 EE
 8 | root               | 2017-02-02 10:45 | Install        |    1
 7 | root               | 2016-12-15 06:48 | Update         |    1
 6 | root               | 2016-12-15 06:43 | Install        |    1
 5 | root               | 2016-12-02 10:28 | E, I, U        |   23 EE
 4 | root               | 2016-10-28 05:37 | E, I, U        |   13 EE
 3 | root               | 2016-10-18 12:53 | Install        |    1
 2 | root               | 2016-09-30 10:28 | E, I, U        |   31 EE
 1 | root               | 2016-07-26 11:40 | E, I, U        |  160 EE

The above command shows two packages has been altered because git updated it’s dependence package too perl-Git. Run the following command to view detailed information about the transaction.

yum history info 13

Loaded plugins: fastestmirror, presto
Transaction ID : 13
Begin time : Fri Aug 18 13:30:52 2017
Begin rpmdb : 420:f5c5f9184f44cf317de64d3a35199e894ad71188
End time : 13:30:54 2017 (2 seconds)
End rpmdb : 420:d04a95c25d4526ef87598f0dcaec66d3f99b98d4
User : root
Return-Code : Success
Command Line : update git
Transaction performed with:
Installed rpm-4.8.0-55.el6.x86_64 @base
Installed yum-3.2.29-81.el6.centos.noarch @base
Installed yum-plugin-fastestmirror-1.1.30-40.el6.noarch @base
Installed yum-presto-0.6.2-1.el6.noarch @anaconda-CentOS-201207061011.x86_64/6.3
Packages Altered:
Updated git-1.7.1-8.el6.x86_64 @base
Update 1.7.1-9.el6_9.x86_64 @updates
Updated perl-Git-1.7.1-8.el6.noarch @base
Update 1.7.1-9.el6_9.noarch @updates
history info

Fire the following command to Rollback the git package to the previous version.

yum history undo 13

Loaded plugins: fastestmirror, presto
Undoing transaction 53, from Fri Aug 18 13:30:52 2017
Updated git-1.7.1-8.el6.x86_64 @base
Update 1.7.1-9.el6_9.x86_64 @updates
Updated perl-Git-1.7.1-8.el6.noarch @base
Update 1.7.1-9.el6_9.noarch @updates
Loading mirror speeds from cached hostfile

  • base: repos.lax.quadranet.com
  • epel: fedora.mirrors.pair.com
  • extras: repo1.dal.innoscale.net
  • updates: mirror.vtti.vt.edu
    Resolving Dependencies
    –> Running transaction check
    —> Package git.x86_64 0:1.7.1-8.el6 will be a downgrade
    —> Package git.x86_64 0:1.7.1-9.el6_9 will be erased
    —> Package perl-Git.noarch 0:1.7.1-8.el6 will be a downgrade
    —> Package perl-Git.noarch 0:1.7.1-9.el6_9 will be erased
    –> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================

Package Arch Version Repository Size

Downgrading:
git x86_64 1.7.1-8.el6 base 4.6 M
perl-Git noarch 1.7.1-8.el6 base 29 k

Transaction Summary

Downgrade 2 Package(s)

Total download size: 4.6 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 4.6 M
(1/2): git-1.7.1-8.el6.x86_64.rpm | 4.6 MB 00:00

(2/2): perl-Git-1.7.1-8.el6.noarch.rpm | 29 kB 00:00

Total 3.4 MB/s | 4.6 MB 00:01
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : perl-Git-1.7.1-8.el6.noarch 1/4
Installing : git-1.7.1-8.el6.x86_64 2/4
Cleanup : perl-Git-1.7.1-9.el6_9.noarch 3/4
Cleanup : git-1.7.1-9.el6_9.x86_64 4/4
Verifying : git-1.7.1-8.el6.x86_64 1/4
Verifying : perl-Git-1.7.1-8.el6.noarch 2/4
Verifying : git-1.7.1-9.el6_9.x86_64 3/4
Verifying : perl-Git-1.7.1-9.el6_9.noarch 4/4

Removed:
git.x86_64 0:1.7.1-9.el6_9 perl-Git.noarch 0:1.7.1-9.el6_9

Installed:
git.x86_64 0:1.7.1-8.el6 perl-Git.noarch 0:1.7.1-8.el6

Complete!
After rollback, use the following command to re-check the downgraded package version.

Rollback Updates using YUM downgrade command
Alternatively we can rollback an updates using YUM downgrade command.

yum downgrade git-1.7.1-8.el6 perl-Git-1.7.1-8.el6

Loaded plugins: search-disabled-repos, security, ulninfo
Setting up Downgrade Process
Resolving Dependencies
–> Running transaction check
—> Package git.x86_64 0:1.7.1-8.el6 will be a downgrade
—> Package git.x86_64 0:1.7.1-9.el6_9 will be erased
—> Package perl-Git.noarch 0:1.7.1-8.el6 will be a downgrade
—> Package perl-Git.noarch 0:1.7.1-9.el6_9 will be erased
–> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================

Package Arch Version Repository Size

Downgrading:
git x86_64 1.7.1-8.el6 base 4.6 M
perl-Git noarch 1.7.1-8.el6 base 29 k

Transaction Summary

Downgrade 2 Package(s)

Total download size: 4.6 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): git-1.7.1-8.el6.x86_64.rpm | 4.6 MB 00:00

(2/2): perl-Git-1.7.1-8.el6.noarch.rpm | 28 kB 00:00

Total 3.7 MB/s | 4.6 MB 00:01
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : perl-Git-1.7.1-8.el6.noarch 1/4
Installing : git-1.7.1-8.el6.x86_64 2/4
Cleanup : perl-Git-1.7.1-9.el6_9.noarch 3/4
Cleanup : git-1.7.1-9.el6_9.x86_64 4/4
Verifying : git-1.7.1-8.el6.x86_64 1/4
Verifying : perl-Git-1.7.1-8.el6.noarch 2/4
Verifying : git-1.7.1-9.el6_9.x86_64 3/4
Verifying : perl-Git-1.7.1-9.el6_9.noarch 4/4

Removed:
git.x86_64 0:1.7.1-9.el6_9 perl-Git.noarch 0:1.7.1-9.el6_9

Installed:
git.x86_64 0:1.7.1-8.el6 perl-Git.noarch 0:1.7.1-8.el6

Complete!
Note : You have to downgrade a dependence packages too, otherwise this will remove the current version of dependency packages instead of downgrade because the downgrade command cannot satisfy the dependency.

For Fedora Users
Use the same above commands and change the package manager command to DNF instead of YUM.

dnf list git

dnf history

dnf history info

dnf history undo

dnf list git

dnf downgrade git-1.7.1-8.el6 perl-Git-1.7.1-8.el6

dnf listgit

dnf history

dnf history info

dnf history undo

dnf listgit

dnf downgrade git-1.7.1-8.el6 perl-Git-1.7.1-8.el6

How to Backup and Restore RabbitMQ Data & Configurations

In this post, I’ll like us to look at how to perform a backup for RabbitMQ configurations and data. This will also include information on restoring a RabbitMQ backup into a new deployment.

Get Cluster Status

$ rabbitmqctl cluster_status
Cluster status of node rabbit@computingforgeeks-centos7 …
[{nodes,[{disc,[‘rabbit@computingforgeeks-centos7’]}]},
{running_nodes,[‘rabbit@computingforgeeks-centos7’]},
{cluster_name,<<“rabbit@computingforgeeks-centos7”>>},
{partitions,[]},
{alarms,[{‘rabbit@computingforgeeks-centos7’,[]}]}]
How to Backup RabbitMQ Configurations
Please note this backup doesn’t include Messages since they are stored in a separate message store. It will only backup RabbitMQ users, vhosts, queues, exchanges, and bindings. The backup file is a JSON representation of RabbitMQ metadata. We will do a backup using rabbitmqadmincommand line tool.

The management plugin ships with a command line tool rabbitmqadmin. You need to enable the management plugin:

rabbitmq-plugins enable rabbitmq_management
This plugin is used to perform some of the same actions as the Web-based UI, and which may be more convenient for automation tasks.

Download rabbitmqadmin
Once you enable the management plugin, download rabbitmqadmin Python command line tool that interacts with the HTTP API. It can be downloaded from any RabbitMQ node that has the management plugin enabled at

http://{node-hostname}:15672/cli/
Once downloaded, make the file executable and move it to /usr/local/bin directory:

chmod +x rabbitmqadmin
sudo mv rabbitmqadmin /usr/local/bin
To backup RabbitMQ configurations, use the command:

rabbitmqadmin export
Example:

$ rabbitmqadmin export rabbitmq-backup-config.json
Exported definitions for localhost to “rabbitmq-backup-config.json”
The export will be written to filerabbitmq-backup-config.json.

How to Restore RabbitMQ Configurations backup
If you ever want to restore your RabbitMQ configurations from a backup, use the command:

rabbitmqadmin import
Example

$ rabbitmqadmin import rabbitmq-backup.json
Imported definitions for localhost from “rabbitmq-backup.json”
How to Backup RabbitMQ Data
RabbitMQ Definitions and Messages are stored in an internal database located in the node’s data directory. To get the directory path, run the following command against a running RabbitMQ node:

rabbitmqctl eval ‘rabbit_mnesia:dir().’
Sample output:

“/var/lib/rabbitmq/mnesia/rabbit@computingforgeeks-server1”
This directory contains many files:

ls /var/lib/rabbitmq/mnesia/rabbit@computingforgeeks-centos7

cluster_nodes.config nodes_running_at_shutdown rabbit_durable_route.DCD rabbit_user.DCD schema.DAT
DECISION_TAB.LOG rabbit_durable_exchange.DCD rabbit_runtime_parameters.DCD rabbit_user_permission.DCD schema_version
LATEST.LOG rabbit_durable_exchange.DCL rabbit_serial rabbit_vhost.DCD
msg_stores rabbit_durable_queue.DCD rabbit_topic_permission.DCD rabbit_vhost.DCL
In RabbitMQ versions starting with 3.7.0 all messages data is combined in the msg_stores/vhosts directory and stored in a subdirectory per vhost. Each vhost directory is named with a hash and contains a .vhost file with the vhost name, so a specific vhost’s message set can be backed up separately.

To do RabbitMQ definitions and messages data backup, copy or archive this directory and its contents. But first, you need to stop RabbitMQ service

sudo systemctl stop rabbitmq-server.service
The example below will create an archive:

tar cvf rabbitmq-backup.tgz /var/lib/rabbitmq/mnesia/rabbit@computingforgeeks-centos7
How to Restore RabbitMQ Data
To restore from Backup, extract the files from backup to the data directory.

Internal node database stores node’s name in certain records. Should node name change, the database must first be updated to reflect the change using the following rabbitmqctl command:

rabbitmqctl rename_cluster_node
When a new node starts with a backed up directory and a matching node name, it should perform the upgrade steps as needed and proceed to boot.

Installing RabbitMQ on CentOS 7

I’ll take you through the installation of RabbitMQ on CentOS 7 / Fedora 29 / Fedora 28.

RabbitMQ is an open source message broker software that implements the Advanced Message Queuing Protocol (AMQP).

It receives messages from publishers (applications that publish them) and routes them to consumers (applications that process them).

Follow the steps below to install RabbitMQ on Fedora 29 / Fedora 28.

Step 1: Install Erlang
Before installing RabbitMQ, you must install a supported version of Erlang/OTP. The version of Erlang package available on EPEL repository should be sufficient.

sudo dnf -y install erlang
Confirm installation by running the erlcommand:

$ erl
Erlang/OTP 20 [erts-9.3.3.3] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.3.3.3 (abort with ^G)
1>
Step 2: Add PackageCloud Yum Repository
A Yum repository with RabbitMQ packages is available from PackageCloud.

Create a new Repository file for RabbitMQ.

sudo vim /etc/yum.repos.d/rabbitmq_rabbitmq-server.repo
Add:

[rabbitmq_rabbitmq-server]

name=rabbitmq_rabbitmq-server
baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/7/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

[rabbitmq_rabbitmq-server-source]

name=rabbitmq_rabbitmq-server-source
baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/7/SRPMS
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300
Step 3: Install RabbitMQ on Fedora 29 / Fedora 28
The last step is the actual installation of RabbitMQ:

sudo dnf makecache -y –disablerepo=’*’ –enablerepo=’rabbitmq_rabbitmq-server’
sudo dnf -y install rabbitmq-server
Confirm version of RabbitMQ installed:

$ rpm -qi rabbitmq-server
Name : rabbitmq-server
Version : 3.7.8
Release : 1.el7
Architecture: noarch
Install Date: Thu 15 Nov 2018 01:32:16 PM UTC
Group : Development/Libraries
Size : 10858832
License : MPLv1.1 and MIT and ASL 2.0 and BSD
Signature : RSA/SHA1, Thu 20 Sep 2018 03:32:57 PM UTC, Key ID 6b73a36e6026dfca
Source RPM : rabbitmq-server-3.7.8-1.el7.src.rpm
Build Date : Thu 20 Sep 2018 03:32:56 PM UTC
Build Host : 17dd9d9d-9199-4429-59e6-dc265f3581e9
Relocations : (not relocatable)
URL : http://www.rabbitmq.com/
Summary : The RabbitMQ server
Description :
RabbitMQ is an open source multi-protocol messaging broker.
Step 4: Start RabbitMQ Service
Now that you have RabbitMQ installed on your Fedora, start and enable the service to start on system boot.

sudo systemctl start rabbitmq-server
sudo systemctl enable rabbitmq-server

Step 5: Enable the RabbitMQ Management Dashboard (Optional)
You can optionally enable the RabbitMQ Management Web dashboard for easy management.

sudo rabbitmq-plugins enable rabbitmq_management
The Web service should be listening on TCP port 15672

ss -tunelp | grep 15672

tcp LISTEN 0 128 0.0.0.0:15672 0.0.0.0:* users:((“beam.smp”,pid=9525,fd=71)) uid:111 ino:39934 sk:9 <->
If you have an active Firewalld service, allow ports 5672 and 15672

sudo firewall-cmd –add-port={5672,15672}/tcp –permanent

By default, the guest user exists and can connect only from localhost. You can log in with this user locally with the password “guest”

To be able to login on the network, create an admin user like below:

rabbitmqctl add_user admin StrongPassword
rabbitmqctl set_user_tags admin administrator
Login with this admin username and the password assigned.

RabbitMQ User Management Commands
Delete User:

rabbitmqctl delete_user user
Change User Password:

rabbitmqctl change_password user strongpassword
Create new Virtualhost:

rabbitmqctl add_vhost /my_vhost
List available Virtualhosts:

rabbitmqctl list_vhosts
Delete a virtualhost:

rabbitmqctl delete_vhost /myvhost
Grant user permissions for vhost:

rabbitmqctl set_permissions -p /myvhost user “.” “.” “.*”
List vhost permissions:

rabbitmqctl list_permissions -p /myvhost
To list user permissions:

rabbitmqctl list_user_permissions user
Delete user permissions:

rabbitmqctl clear_permissions -p /myvhost user
The next article to read is:

sudo firewall-cmd –reload
Access it by opening the URL http://[server IP|Hostname]:15672

Ansibile yaml file

autocmd FileType yank setlocal ai ts=2 sw=2 et
vim set cursorcolumn color

iptables tips and tricks

Tip #1: Take a backup of your iptables configuration before you start working on it.

Back up your configuration with the command:

/sbin/iptables-save > /root/iptables-works

Tip #2: Even better, include a timestamp in the filename.

Add the timestamp with the command:

/sbin/iptables-save > /root/iptables-works-`date +%F`

You get a file with a name like:

/root/iptables-works-2018-09-11

If you do something that prevents your system from working, you can quickly restore it:

/sbin/iptables-restore < /root/iptables-works-2018-09-11
ln –s /root/iptables-works-`date +%F` /root/iptables-works-latest

Tip #4: Put specific rules at the top of the policy and generic rules at the bottom.

Avoid generic rules like this at the top of the policy rules:

iptables -A INPUT -p tcp --dport 22 -j DROP

The more criteria you specify in the rule, the less chance you will have of locking yourself out. Instead of the very generic rule above, use something like this:

iptables -A INPUT -p tcp --dport 22 –s 10.0.0.0/8 –d 192.168.100.101 -j DROP

This rule appends (-A) to the INPUT chain a rule that will DROP any packets originating from the CIDR block 10.0.0.0/8 on TCP (-p tcp) port 22 (–dport 22) destined for IP address 192.168.100.101 (-d 192.168.100.101).

There are plenty of ways you can be more specific. For example, using -i eth0 will limit the processing to a single NIC in your server. This way, the filtering actions will not apply the rule to eth1.

Tip #5: Whitelist your IP address at the top of your policy rules.

This is a very effective method of not locking yourself out. Everybody else, not so much.

iptables -I INPUT -s <your IP> -j ACCEPT

You need to put this as the first rule for it to work properly. Remember, -I inserts it as the first rule; -A appends it to the end of the list.

Tip #6: Know and understand all the rules in your current policy.

Not making a mistake in the first place is half the battle. If you understand the inner workings behind your iptables policy, it will make your life easier. Draw a flowchart if you must. Also remember: What the policy does and what it is supposed to do can be two different things.

Set up a workstation firewall policy

Scenario: You want to set up a workstation with a restrictive firewall policy.

Tip #1: Set the default policy as DROP.

# Set a default policy of DROP
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]

Tip #2: Allow users the minimum amount of services needed to get their work done.

The iptables rules need to allow the workstation to get an IP address, netmask, and other important information via DHCP (-p udp –dport 67:68 –sport 67:68). For remote management, the rules need to allow inbound SSH (–dport 22), outbound mail (–dport 25), DNS (–dport 53), outbound ping (-p icmp), Network Time Protocol (–dport 123 –sport 123), and outbound HTTP (–dport 80) and HTTPS (–dport 443).

# Set a default policy of DROP
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]

# Accept any related or established connections
-I INPUT  1 -m state –state RELATED,ESTABLISHED -j ACCEPT
-I OUTPUT 1 -m state –state RELATED,ESTABLISHED -j ACCEPT

# Allow all traffic on the loopback interface
-A INPUT -i lo -j ACCEPT
-A OUTPUT -o lo -j ACCEPT

# Allow outbound DHCP request
-A OUTPUT –o eth0 -p udp –dport 67:68 –sport 67:68 -j ACCEPT

# Allow inbound SSH
-A INPUT -i eth0 -p tcp -m tcp –dport 22 -m state –state NEW  -j ACCEPT

# Allow outbound email
-A OUTPUT -i eth0 -p tcp -m tcp –dport 25 -m state –state NEW  -j ACCEPT

# Outbound DNS lookups
-A OUTPUT -o eth0 -p udp -m udp –dport 53 -j ACCEPT

# Outbound PING requests
-A OUTPUT –o eth0 -p icmp -j ACCEPT

# Outbound Network Time Protocol (NTP) requests
-A OUTPUT –o eth0 -p udp –dport 123 –sport 123 -j ACCEPT

# Outbound HTTP
-A OUTPUT -o eth0 -p tcp -m tcp –dport 80 -m state –state NEW -j ACCEPT
-A OUTPUT -o eth0 -p tcp -m tcp –dport 443 -m state –state NEW -j ACCEPT

COMMIT

Restrict an IP address range

Scenario: The CEO of your company thinks the employees are spending too much time on Facebook and not getting any work done. The CEO tells the CIO to do something about the employees wasting time on Facebook. The CIO tells the CISO to do something about employees wasting time on Facebook. Eventually, you are told the employees are wasting too much time on Facebook, and you have to do something about it. You decide to block all access to Facebook. First, find out Facebook’s IP address by using the host and whois commands.

host -t a www.facebook.com
www.facebook.com is an alias for star.c10r.facebook.com.
star.c10r.facebook.com has address 31.13.65.17
whois 31.13.65.17 | grep inetnum
inetnum:        31.13.64.0 – 31.13.127.255

Then convert that range to CIDR notation by using the CIDR to IPv4 Conversion page. You get 31.13.64.0/18. To prevent outgoing access to www.facebook.com, enter:

iptables -A OUTPUT -p tcp -i eth0 –o eth1 –d 31.13.64.0/18 -j DROP

Regulate by time

Scenario: The backlash from the company’s employees over denying access to Facebook access causes the CEO to relent a little (that and his administrative assistant’s reminding him that she keeps HIS Facebook page up-to-date). The CEO decides to allow access to Facebook.com only at lunchtime (12PM to 1PM). Assuming the default policy is DROP, use iptables’ time features to open up access.

iptables –A OUTPUT -p tcp -m multiport –dport http,https -i eth0 -o eth1 -m time –timestart 12:00 –timestart 12:00 –timestop 13:00 –d
31.13.64.0/18  -j ACCEPT

This command sets the policy to allow (-j ACCEPT) http and https (-m multiport –dport http,https) between noon (–timestart 12:00) and 13PM (–timestop 13:00) to Facebook.com (–d 31.13.64.0/18).

Regulate by time—Take 2

Scenario: During planned downtime for system maintenance, you need to deny all TCP and UDP traffic between the hours of 2AM and 3AM so maintenance tasks won’t be disrupted by incoming traffic. This will take two iptables rules:

iptables -A INPUT -p tcp -m time –timestart 02:00 –timestop 03:00 -j DROP
iptables -A INPUT -p udp -m time –timestart 02:00 –timestop 03:00 -j DROP

With these rules, TCP and UDP traffic (-p tcp and -p udp ) are denied (-j DROP) between the hours of 2AM (–timestart 02:00) and 3AM (–timestop 03:00) on input (-A INPUT).

Limit connections with iptables

Scenario: Your internet-connected web servers are under attack by bad actors from around the world attempting to DoS (Denial of Service) them. To mitigate these attacks, you restrict the number of connections a single IP address can have to your web server:

iptables –A INPUT –p tcp –syn -m multiport -–dport http,https –m connlimit -–connlimit-above 20 –j REJECT -–reject-with-tcp-reset

Let’s look at what this rule does. If a host makes more than 20 (-–connlimit-above 20) new connections (–p tcp –syn) in a minute to the web servers (-–dport http,https), reject the new connection (–j REJECT) and tell the connecting host you are rejecting the connection (-–reject-with-tcp-reset).

Monitor iptables rules

Scenario: Since iptables operates on a “first match wins” basis as packets traverse the rules in a chain, frequently matched rules should be near the top of the policy and less frequently matched rules should be near the bottom. How do you know which rules are traversed the most or the least so they can be ordered nearer the top or the bottom?

Tip #1: See how many times each rule has been hit.

Use this command:

iptables -L -v -n –line-numbers

The command will list all the rules in the chain (-L). Since no chain was specified, all the chains will be listed with verbose output (-v) showing packet and byte counters in numeric format (-n) with line numbers at the beginning of each rule corresponding to that rule’s position in the chain.

Using the packet and bytes counts, you can order the most frequently traversed rules to the top and the least frequently traversed rules towards the bottom.

Tip #2: Remove unnecessary rules.

Which rules aren’t getting any matches at all? These would be good candidates for removal from the policy. You can find that out with this command:

iptables -nvL | grep -v "0     0"

Note: that’s not a tab between the zeros; there are five spaces between the zeros.

Tip #3: Monitor what’s going on.

You would like to monitor what’s going on with iptables in real time, like with top. Use this command to monitor the activity of iptables activity dynamically and show only the rules that are actively being traversed:

watch --interval=5 'iptables -nvL | grep -v "0     0"'

watch runs ‘iptables -nvL | grep -v “0     0″‘ every five seconds and displays the first screen of its output. This allows you to watch the packet and byte counts change over time.

Report on iptables

Scenario: Your manager thinks this iptables firewall stuff is just great, but a daily activity report would be even better. Sometimes it’s more important to write a report than to do the work.

Use the packet filter/firewall/IDS log analyzer FWLogwatch to create reports based on the iptables firewall logs. FWLogwatch supports many log formats and offers many analysis options. It generates daily and monthly summaries of the log files, allowing the security administrator to free up substantial time, maintain better control over network security, and reduce unnoticed attacks.

SSL and TLS 1.3 on Nginx

I have heard that there is TLS1.3,

I have been tickle, I want to toss and try. In the past, there were not many browsers supported,
and there were not many people on the Internet who tried it. There are some large website sites that have already got TLS1.3,
and many bloggers have upgraded their blogs to TLS1.3, leaving valuable experience. I can’t help it anymore.
Let’s take a look at it today. Openssl 1.1.1 LTS has been released, update the official version of TLS1.3.

Software version
?Nginx: nginx-1.15.4
?OpenSSL: openssl-1.1.1(LTS)

Tutorial

Installation dependency

Sudo apt update
sudo apt install -y build-essential libpcre3 libpcre3-dev zlib1g-dev liblua5.1-dev libluajit-5.1-dev libgeoip-dev google-perftools libgoogle-perftools-dev

Download and unzip the required software

Wget https://nginx.org/download/nginx-1.15.4.tar.gz
tar zxf nginx-1.15.4.tar.gz
wget https://www.openssl.org/source/openssl-1.1.1. tar.gz
tar zxf OpenSSL-1.1.1.tar.gz

OpenSSL patching

Pushd openssl-1.1.1 #?TLS1.3 Draft 23, 26, 28, Final patch
curl https://raw.githubusercontent.com/hakasenyang/openssl-patch/master/openssl-equal-1.1.1_ciphers.patch | patch -p1
#?ign Strict-SNI log patch
curl https://raw.githubusercontent.com/hakasenyang/openssl-patch/master/openssl-ignore_log_strict-sni.patch | patch -p1
popd

Nginx patch

Pushd nginx-1.15.4
#?SPDY, HTTP2 HPACK, Dynamic TLS Record, Fix Http2 Push Error, PRIORITIZE_CHACHA patch
curl https://raw.githubusercontent.com/kn007/patch/43f2d869b209756b442cfbfa861d653d993f16fe/nginx.patch | patch -p1
curl https ://raw.githubusercontent.com/kn007/patch/c59592bc1269ba666b3bb471243c5212b50fd608/nginx_auto_using_PRIORITIZE_CHACHA.patch | patch -p1
#? Strict-SNI patch
curl https://raw.githubusercontent.com/hakasenyang/openssl-patch/master/nginx_strict-sni .patch | patch -p1
popd

Compile and install Nginx

If you have compiled and installed Nginx, you can enter nginx -V to view the previous configure configuration. Compile with the required parameters later.

Key parameters:
? Add –with-openssl=../openssl-1.1.1 to specify the OpenSSL path
?HTTP2 HPACK needs to add the –with-http_v2_hpack_enc parameter.
?SPDY needs to be added –with-http_spdy_module

Note that the –with-openssl parameter is changed to its own OpenSSL folder address.

My full configure command is as follows, please analogy.

Cd nginx-1.15.4

./configure \
–user=www \
–group=www \
–prefix=/usr/local/nginx \
–with-http_stub_status_module \
–with-threads \
–with-file-aio \
–with -pcre-jit \
–with-http_ssl_module \
–with-http_v2_module \
–with-http_gzip_static_module \
–with-http_sub_module \
–with-http_flv_module \
–with-http_mp4_module \
–with-http_gunzip_module \
–with -http_realip_module \
–with-http_addition_module \
–with-stream \
–with-stream_ssl_module \
–with-stream_ssl_preread_module \
–with-stream_realip_module \
–with-http_slice_module \
–with-http_geoip_module \
–with-google_perftools_module \
–with-openssl=../openssl-1.1.1 \
–with-http_v2_hpack_enc \
–with-http_spdy_module

After configure is complete, enter the following statement to start compiling.

Make

After the compilation is completed, if no error is reported, enter the following to install.

Make install

Configuring Nginx Web Hosting

Add the following to the appropriate location in your conf file to replace the original content. I removed TLS1 and TLS1.1 due to security upgrade considerations. In addition, the new cipher suite for TLS 1.3 can only be used in TLS 1.3, and the old cipher suite cannot be used for TLS 1.3. It seems that all virtual hosts must be configured to use TLS1.3.

Ssl_early_data on;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers [TLS13+AESGCM+AES128|TLS13+AESGCM+AES256|TLS13+CHACHA20]:[EECDH+ECDSA+AESGCM+AES128|EECDH+ECDSA+CHACHA20]:EECDH+ECDSA+ AESGCM+AES256:EECDH+ECDSA+AES128+SHA:EECDH+ECDSA+AES256+SHA:[EECDH+aRSA+AESGCM+AES128|EECDH+aRSA+CHACHA20]:EECDH+aRSA+AESGCM+AES256:EECDH+aRSA+AES128+ SHA:EECDH+aRSA+AES256+SHA:RSA+AES128+SHA:RSA+AES256+SHA:RSA+3DES;
ssl_ecdh_curve X25519:P-256:P-384;
ssl_prefer_server_ciphers on;

Finally, use nginx -t to test the correctness of the nginx configuration.

success

Restart Nginx and you will find that your website is already connected to TLS1.3.

rminal window and follow these steps:

1. Generate the private key using the command sudo openssl genrsa -out ca.key 2048

2. Generate a CSR using the command sudo openssl req -new -key ca.key -out ca.csr

3. Use the command sudo openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt to generate a self-signed key

Now we need to copy the newly generated file to the correct location with the following command:

sudo cp ca.crt /etc/ssl/certs/
sudo cp ca.key /etc/ssl/private/
sudo cp ca.csr /etc/ssl/private/

Create an Nginx configuration

Remember, we want to enable SSL via TLS support. To do this, we must create a new Nginx configuration file with the following command:

Sudo nano /etc/nginx/conf.d/ssl.conf

In the file, paste the following:

Server {

Location / {
root /usr/share/nginx/html;
index index.html index.htm;
}

Listen 443 ssl;
server_name www.example.com;
ssl_certificate /etc/ssl/certs/ca.crt;
ssl_certificate_key /etc/ssl/private/ca.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers TLS- CHACHA20-POLY1305-SHA256: TLS-AES-256-GCM-SHA384: TLS-AES-128-GCM-SHA256: HIGH: !aNULL:!MD5;

}

Note: Be sure to change the root location to reflect your Nginx installation. However, if you follow the steps to build a Nginx that supports TLS, the above configuration should work.

Save and close the file. Test the new Nginx configuration file with the following command:

Sudo nginx -t

You should see the test passed.

Restart and test

Now we need to restart NGINX. Use the following command to do this:

Sudo systemctl restart nginx

Point your browser to https://SERVER_IP and you should see the NGINX welcome screen.
To ensure that your site is delivered with TLS 1.3 enabled, you can use the browser’s built-in tools.
For example, in Firefox, open the page and click the security button (the lock icon to the left of the address bar).
Click the right arrow associated with the page, then click More Info.
In the results window (Figure B), you should see the connection using TLS 1.3 encryption.

This is all about enabling SSL and TLS on the Nginx website.
Remember that you should use an SSL certificate from a reputable certification authority.
However, it is always a good idea to use a self-signed certificate for testing purposes.
Once you have confidence in this process, please purchase a certificate and deploy it to your Nginx site.

tomcat tuning

Sync sync disk

echo 3 > /proc/sys/vm/drop_caches # Clean up useless memory space

Tomcat8 final configuration
1.${tomcat}/bin/catalina.sh Join
1.${tomcat}/bin/catalina.sh
JAVA_OPTS=”-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1G -Xmx1G -Xss256k -XX:NewSize=1G -XX:MaxNewSize=1G
-XX:PermSize=128m -XX:MaxPermSize=128m -XX:+DisableExplicitGC”

2.
2.
JAVA_OPTS=”$JAVA_OPTS -server -Xms3G -Xmx3G -Xss256k -XX:PermSize=128m -XX:MaxPermSize=128m -XX:+UseParallelOldGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/aaa/dump -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/usr/tomcat/dump/heap_trace.txt -XX:NewSize=1G -XX:MaxNewSize=1G”

2.${tomcat}/conf/server.xml
Open commented out

<Executor name=”tomcatThreadPool” namePrefix=”catalina-exec-”
maxThreads=”300″ minSpareThreads=”50″/>

Add options not found in Connector

<Connector port=”80″ protocol=”org.apache.coyote.http11.Http11NioProtocol”
executor=”tomcatThreadPool”
minSpareThreads=”50″
maxSpareThreads=”500″
enableLookups=”false”
acceptCount=”500″
debug=”0″
connectionTimeout=”10000″
redirectPort=”8443″
compression=”on”
compressableMimeType=”text/html,text/xml,text/plain,text/javascript,text/css”
disableUploadTimeout=”true”
URIEncoding=”UTF-8″
useBodyEncodingForURI=”true”
/>

Detailed explanation of each parameter:

-Xms: Set the JVM initial memory size (default is 1/64 of physical memory)

-Xmx: Set the maximum memory that the JVM can use (default is 1/4 of physical memory, recommended: 80% of physical memory)

-Xmn: Set the minimum memory of the JVM (128-256m is enough, generally not set)

The default free heap memory is less than
At 40%, the JVM will increase the heap until the maximum limit of -Xmx; when the free heap memory is greater than 70%, the JVM will reduce the heap to the minimum limit of -Xms. So the server is generally set to -Xms,
-Xmx is equal to avoid resizing the heap after each GC.

In larger applications, the default memory is not enough and may cause the system to fail. A common problem is to report a Tomcat memory overflow error “java.lang.OutOfMemoryError:
Java heap space”, causing the client to display 500 errors.

-XX:PermSize : Perm memory size when starting the JVM

-XX:MaxPermSize : is the maximum available Perm memory size (default is 32M)

-XX:MaxNewSize, default is 16M

The full name of PermGen space is Permanent Generation
Space, refers to the permanent storage area of ??memory, this memory is mainly stored by the JVM Class and Meta information, Class will be placed in PermGen when it is Loader
In space, it is different from the Heap area that stores the instance (Instance), GC (Garbage
Collection) will not be in the main program runtime against PermGen
Space is cleaned up, so if your application has a very CLASS, it is likely to appear “java.lang.OutOfMemoryError:
PermGen space” error.

For WEB projects, when jvm loads a class, the objects in the permanent domain increase sharply, so that jvm constantly adjusts the size of the permanent domain. To avoid adjustments, you can use more parameter configuration. If your WEB
APP uses a large number of third-party jars, the size of which exceeds the default size of jvm, then this error message will be generated.

Other parameters:

-XX:NewSize: The default is 2M. This value is set to a large adjustable new object area, reducing Full.
GC times

-XX:NewRatio : Change the proportion of new and old space, which means that the size of the new space is 1/8 of the old space (default is 8)

-XX:SurvivorRatio: Change the size ratio of the Eden object space and the remaining space, meaning that the Eden object is empty.

The size between the two is greater than the survivor space by 2 times survivorRatio (default is 10)

-XX:userParNewGC can be used to set parallel collection [multiple CPU]

-XX:ParallelGCThreads can be used to increase parallelism [multiple CPU]

-XXUseParallelGC can be set to use parallel clear collector [multi-CPU]

maxThreads
The maximum number of request processing threads to be created by this Connector, which therefore determines the maximum number of simultaneous requests that can be handled.If not specified, this attribute is set to 200. If an executor is associated with this connector, this attribute is ignored as the connector will execute tasks using the executor rather than an internal thread pool.
300
minSpareThreads
The minimum number of threads always kept running. If not specified, the default of 10 is used.
50
connectionTimeout
The number of milliseconds this Connector will wait, after accepting a connection, for the request URI line to be presented. Use a value of -1 to indicate no (i.e. infinite) timeout. The default value is 60000 (i.e. 60 seconds) but note that the standard server.xml that ships with Tomcat sets this to 20000 (i.e. 20 seconds). Unless disableUploadTimeout is set to false, this timeout will also be used when reading the request body (if any).
tcpNoDelay
If set to true, the TCP_NO_DELAY option will be set on the server socket, which improves performance under most circumstances. This is set to true by default.
socketBuffer
The size (in bytes) of the buffer to be provided for socket output buffering. -1 can be specified to disable the use of a buffer. By default, a buffers of 9000 bytes will be used.
server
Overrides the Server header for the http response. If set, the value for this attribute overrides the Tomcat default and any Server header set by a web application. If not set, any value specified by the application is used. If the application does not specify a value then Apache-Coyote/1.1 is used. Unless you are paranoid, you won’t need this feature.
maxHttpHeaderSize
The maximum size of the request and response HTTP header, specified in bytes. If not specified, this attribute is set to 8192 (8 KB).
maxKeepAliveRequests
The maximum number of HTTP requests which can be pipelined until the connection is closed by the server. Setting this attribute to 1 will disable HTTP/1.0 keep-alive, as well as HTTP/1.1 keep-alive and pipelining. Setting this to -1 will allow an unlimited amount of pipelined or keep-alive HTTP requests. If not specified, this attribute is set to 100.
maxConnections
For BIO the default is the value of maxThreads unless an Executor is used in which case the default will be the value of maxThreads from the executor. For NIO the default is 10000. For APR/native, the default is 8192.
keepAliveTimeout
The number of milliseconds this Connector will wait for another HTTP request before closing the connection. The default value is to use the value that has been set for the connectionTimeout attribute. Use a value of -1 to indicate no (i.e. infinite) timeout.

Database Pool Configuration

<Resource name=”jdbc/productdb” auth=”Container” type=”javax.sql.DataSource”
maxTotal=”10″ maxIdle=”30″ maxWaitMillis=”10000″ logAbandoned=”true”
username=”root” password=”admin” driverClassName=”com.mysql.jdbc.Driver”
url=”jdbc:mysql://localhost:3306/products”/>
</Context>

JVM Settings
We have set the minimum and maximum heap size to 1GB respectively as below:

export CATALINA_OPTS=”-Xms1024m -Xmx1024m”

-Xms – Specifies the initial heap memory
-Xmx – Specifies the maximum heap memory

AJP Connector configuration
The AJP connector configuration below is configured so that there are two threads allocated to accept new connections.
This should be configured to the number of processors on the machine however two should be suffice here.
We have also allocated 400 threads to process requests, the default value is 200.
The “acceptCount” is set to 100 which denotes the maximum queue length to be used for incoming connections.
The default value is 10. Lastly we have set the minimum threads to 20 so that there are always 20 threads running in the pool to service requests:

<Connector port=”8009″ protocol=”AJP/1.3″ redirectPort=”8443″ acceptorThreadCount=”2″ maxThreads=”400″ acceptCount=”200″ minSpareThreads=”20″/>

Database Pool Configuration
We have modified the maximum number of pooled connections to 200 so that there are ample connections in the pool to service requests.

<Context>
<Resource name=”jdbc/productdb” auth=”Container” type=”javax.sql.DataSource”
maxTotal=”200″ maxIdle=”30″ maxWaitMillis=”10000″ logAbandoned=”true”
username=”xxxx” password=”xxxx” driverClassName=”com.mysql.jdbc.Driver”
url=”jdbc:mysql://localhost:3306/products”/>
</Context>

JVM Settings
Since we have increased the maximum number of pooled connections and AJP connector thread thresholds above,
we should increase the heap size appropriately. We have set the minimum and maximum heap size to 2GB respectively as below:

export CATALINA_OPTS=”-Xms2048m -Xmx2048m”

JVM Heap Monitoring and Tuning

Specifying appropriate JVM heap parameters to service your deployed applications on Tomcat is paramount to application performance.
There are a number of different ways which we can monitor JVM heap usage including using JDK hotspot tools such as jstat, JConsole etc. –
however to gather detailed data on when and how garbage collection is being performed, it is useful to turn on GC logging on the Tomcat instance.
We can turn on GC logging by modifying the catalina start up script with the following command:

JAVA_OPTS=”$JAVA_OPTS -verbose:gc -Xloggc:gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps”

We can set the minimum and maximum heap size,
the size of the young generation and the maximum amount of memory to be allocated to the permanent generation used to store application class metadata by specifying the setting the CATALINA_OPTS parameter by executing this command:

export CATALINA_OPTS=”-Xms1024m -Xmx2048m -XX:MaxNewSize=512m -XX:MaxPermSize=256m”

This configuration is optimized for REST/HTTP API call. And it doesn’t use any reverse proxy like Apache, NginX etc. We will reside simple L4 switch infront of tomcat groups.

In addition we will not use Tomcat Clustering, Session etc. So the clustering configuration is omitted.

Listener Setting
<Listener className=”org.apache.catalina.security.SecurityListener” checkedOsUsers=”root” />

checkedOsUser setting means Unix system user “root” cannot start Tomcat. If user starts tomcat as a root user it makes log file as a root user permission. In that case tomcat user cannot delete the log file.

<Listener className=”org.apache.catalina.core.JreMemoryLeakPreventionListener” />

This makes detect memory leak.

Connector Setting
protocol=”org.apache.coyote.http11.Http11Protocol”

It makes tomcat use BIO. Tomcat has options for IO (BIO,NIO,APR). APR is fastest IO setting. It uses Apache web server IO module, so it is fastest. But it uses C code (JNI call), it can have a risk to kill tomcat instance. (with core dump). APR is more faster about 10% than BIO. But BIO is more stable. Use BIO. (Default is BIO)

acceptCount=”10?

It specifies server request queue length. If message is queued in the request queue, it means server cannot handle incoming message (it is overloaded). It will wait for idle thead and the request message will be pending. This setting reduce total size of request queue to 10. If the queue has been overflowed, client will get a error. It can protect server from high overload and let system manager to know the server has been overloaded.

enableLookups=”false”

In Java Servlet Code, user can look up request message origin (IP or URL).

For example user in yahoo.com send request to server, and Tomcat try to resolve incoming request IP address.
“enableLooksups” option enables return DNS name not a IP address. During this processing Tomcat look up DNS.
It brings performance degradation. This option removes DNS look up stage and increase performance.

compression=”off”

We are using REST protocol not a normal web contents like HTML,Image etc.
This options allows to compress HTTP message. It consumes computing power but it can reduce network payload.
In our environment compression is not required. It is better to save computing power. And in some particular Telco network, compression is not supported.

connectionTimeout=”10000?

It is HTTP Connection time out (client to server). It is milliseconds. (10,000 = 10 sec).

If server cannot make a connection from client til 10 sec. It will throw HTTP time out error.
In normal situation, our API response time is under 5 sec. So 10 sec means, server has been overloaded.
The reason why I increased the time up to 10 sec is, depends on network condition, connection time will be deferred.

maxConnections=”8192?

The maximum number of connection, tomcat can handle. It means tomcat can handle maximum 8192 socket connection in a time. This value is restricted by Unix system parameter “ulimit –f” (You can check up in unix console)

maxKeepAliveRequests=”1?

As I mentioned above, this configuration is optimized to REST API request not a common web system. It means client will send REST API call only. It sends the request and get a response. Client will not send request in a short time. It means we cannot reuse the connection from the client. So this setting turn of HTTP Keep Alive. (After response the request from client, tomcat disconnect the connection immediately)

maxThreads=”100?

This defines total number of thread in Tomcat. It represents max number of active user at that time. Usually 50~500 is good for performance. And 100~200 is best (it is different depends on use case scenario).

Please test with 100 and 200 values and find value for performance. This parameter also get a impact from DB connection pool setting, even if we have a lot of thread , and the total number of db connection is not enough, the thread will wait to acquire the connection.

tcpNoDelay=”true”

This allows us to use TCP_NO_DELAY in tcp/ip layer. It makes send small packet without delay. In TCP, to reduce small package congestion, it gathers small packet to tcp buffer until it has been filled and send the packet. TCP_NO_DELAY option makes send small packet immediately even though TCP buffer is not full.

JVM Tuning
Java Virtual Machine tuning is also very important factor to run Tomcat

The focus of JVM tuning is reducing Full GC time.

-server

This option makes JVM to optimize server application. It tunes HotSpot compiler etc internally. This option is very important and mandatory in server side application

-Xmx1024m –Xms1024m -XX:MaxNewSize=384m -XX:MaxPermSize=128m

This memory tuning options, our infrastructure is using c1.mediuem amazon instance, so the available memory is about 1.7 gb total. Heap size is 1G and let them to have fixed size. It defines max 1Gb, min 1Gb heap size. The NewSize is 384mb (1/3 size of total heap size). 1/3 New Size is best performance usually. Perm size is defines area of memory to load class. 64mb is enough. But we will use 128m first time and tune based on gc log analysis later.

Total physical memory consumption is 1G heap + 128mb perm = 1.128 GB and JVM internally uses memory to run JVM itself. It consumes about 350~500mb. So total estimated required memory is about 1.128GB+500m = 1.5 GB.

As I mentioned, c1.mediuem size has only 1.7GB physical memory. If consumed memory exceeds actual physical memory, it makes disk swapping. If JVM memory is swapped out to disk, the performance is significantly degraded. Please take care swapping is not occurred.

-XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./java_pid<pid>.hprof

These options are for trouble shooting “OOM (Java Out Of Memory Error”. If out of memory error has been occurred. The memory layout will be dumped to disk. The location of dumpfile is specified by “-XX:HeapDumpPath” option

-XX:ParallelGCThreads=2 -XX:-UseConcMarkSweepGC

These options specify GC strategy. It uses ParallelGC for Minor collection and 2 threads will be used for the Minor GC. And for Old area, concurrent gc will be used. It will reduce Full gc time

-XX:-PrintGC -XX:-PrintGCDetails -XX:-PrintGCTimeStamps -XX:-TraceClassUnloading -XX:-TraceClassLoading

These option specifies GC logging. It logs the GC log detail to stderr (console output). It shows usage trend os Java Heap memory, time stamp etc. (it contains old,new & perm area usage).

Especially, ClassLoading & UnLoading option show what class is loaded and unloaded to memory. It helps us to trace Perm Out of memory error.

CentOS 7.3 compile and install Nginx 1.12.2

CentOS 7.3 compile and install Nginx 1.12.2

1. Introduction to
Nginx Nginx (pronounced [engine x]) was developed for performance optimization. Its best known advantages are its stability and low system resource consumption, as well as high processing power for concurrent connections (single physical server available) Supporting 30,000 to 50,000 concurrent connections), is a high-performance HTTP and reverse proxy server, and an IMAP/POP3/SMTP proxy service.

Linux system: CentOS 7.3

2. Installation preparation
2.1 gcc installation

To install nginx, you need to compile the source code downloaded from the official website first, and compile it depends on the gcc environment. If there is no gcc environment, you need to install it:

[root@nginx ~]# yum -y install gcc-c++

2.2 pcre installation

PCRE (Perl Compatible Regular Expressions) is a Perl library that includes a perl-compatible regular expression library. Nginx’s http module uses pcre to parse regular expressions, so you need to install the pcre library on linux, a secondary development library developed with pcre. Nginx also needs this library.

[root@nginx ~]# yum -y install pcre pcre-devel

2.3 zlib installation

The zlib library provides a variety of ways to compress and decompress. nginx uses zlib to gzip the contents of the http package, so you need to install the zlib library on Centos.

[root@nginx ~]# yum -y install zlib zlib-devel

2.4 OpenSSL installation

OpenSSL is a powerful Secure Sockets Layer cryptography library that includes major cryptographic algorithms, common key and certificate encapsulation management functions, and SSL protocols, and provides a rich set of applications for testing or other purposes.
Nginx supports not only the http protocol, but also https (that is, http on the ssl protocol), so you need to install the OpenSSL library in Centos.

[root@nginx ~]# yum -y install openssl openssl-devel

3. Nginx installation

3.1 Nginx version

Download URL: https://nginx.org/en/download.html

Select the latest stable version of nginx-1.12.2
release notes:

Mainline version: Mainline is the version that Nginx is currently working on. It can be said that the development version of
Stable version: the latest stable version, the recommended version of the production environment
Legacy versions: the legacy version of the legacy version

3.2 Nginx Download

Use the wget command to download

[root@nginx ~]# wget -c https://nginx.org/download/nginx-1.12.2.tar.gz

Install without the wget command:

[root@nginx ~]# yum -y install wget

3.3 Decompression

[root@nginx ~]# tar -zxvf nginx-1.12.2.tar.gz

3.4 Installation and Configuration

3.4.1 Creating a New nginx User and Group

[root@nginx include]# groupadd nginx
[root@nginx include]# useradd -g nginx -d /home/nginx nginx
[root@nginx include]# passwd nginx

3.4.2 Third-party module installation

This article uses the third-party module sticky as an example, the version is 1., 2.5, download address:

You can download it from the Linux Community Resource Station:

——————————————Dividing line—— ————————————

The free download address is at http://linux.linuxidc.com/

Username and password are both www.linuxidc.com

The specific download directory is compiled and installed in the /2000 data/September/27/CentOS 7.3 installation Nginx 1.12.2/

The download method can be found at http://www.linuxidc.com/Linux/2013-07/87684.htm

——————————————Dividing line—— ————————————

Upload and unzip:

[root@nginx ~]# tar -zxvf nginx-goodies-nginx-sticky-module-ng-08a395c66e42..gz
[root@nginx ~]# mv nginx-goodies-nginx-sticky-module-ng-08a395c66e42 nginx-sticky -1.2.5

3.4.3 Installation

[root@nginx ~]# cd nginx-1.12.2
[root@nginx nginx-1.12.2]# ./configure –add-module=/root/nginx-sticky-1.2.5

Specify user, path, and module configuration (optional):

./configure \
–user=nginx –group=nginx \ #Installed user group
–prefix=/usr/local/nginx \
#Specify the installation path –with-http_stub_status_module \ #Monitor nginx state, need to be in nginx.
Conp configuration –with-http_ssl_module \ #Support HTTPS
–with-http_sub_module \ #Support URL redirection
–with-http_gzip_static_module #static compression–
add-module=/root/nginx-sticky-1.2.5 #Install sticky module

3.5 compilation

[root@nginx nginx-1.12.2]# make && make install

Error:

/root/nginx-sticky-1.2.5//ngx_http_sticky_misc.c: In the function ‘ngx_http_sticky_misc_sha1’:
/root/nginx-sticky-1.2.5//ngx_http_sticky_misc.c:176:15: Error: ‘SHA_DIGEST_LENGTH’ is not declared (first used in this function)
u_char hash[SHA_DIGEST_LENGTH];
^
/root/nginx-sticky-1.2.5//ngx_http_sticky_misc.c:176:15: Note: Every undeclared identifier appears in it Only one time is reported in the function
/root/nginx-sticky-1.2.5//ngx_http_sticky_misc.c:176:10: Error: Unused variable ‘hash’ [-Werror=unused-variable]
u_char hash[SHA_DIGEST_LENGTH];
^
/ Root/nginx-sticky-1.2.5//ngx_http_sticky_misc.c: In the function ‘ngx_http_sticky_misc_hmac_sha1’:
/root/nginx-sticky-1.2.5//ngx_http_sticky_misc.c:242:15: Error: ‘SHA_DIGEST_LENGTH’ is not declared ( Used for the first time in this function)
u_char hash[SHA_DIGEST_LENGTH];

Solution:

Modify the ngx_http_sticky_misc.c file to add #include <openssl/sha.h> and #include <openssl/md5.h> modules

[root@nginx nginx-1.12.2]# sed -i ’12a #include <openssl/sha.h>’ /root/nginx-sticky-1.2.5/ngx_http_sticky_misc.c
[root@nginx nginx-1.12.2] # sed -i ’12a #include <openssl/md5.h>’ /root/nginx-sticky-1.2.5/ngx_http_sticky_misc.c

Recompile:

[root@nginx nginx-1.12.2]# make && make install

3.6 nginx command global execution settings

[root@nginx bin]# cd /usr/local/nginx/sbin/
[root@nginx sbin]# ln -s /usr/local/nginx/sbin/nginx /usr/local/bin/nginx

4. Nginx related commands

4.1 version view

[root@nginx ~]# nginx -v
nginx version: nginx/1.12.2

4.2 Viewing Loaded Modules

[root@nginx ~]# nginx -V
nginx version: nginx/1.12.2
built by gcc 4.8.5 20150623 ( Red Hat 4.8.5-28) (GCC)
configure arguments: –add-module=/root/nginx -sticky-1.2.5/

4.3 Start and stop command

4.3.1 Starting

[root@nginx nginx-1.12.2]# nginx

4.3.2 Stop

[root@nginx nginx-1.12.2]# nginx -s stop
[root@nginx nginx-1.12.2]# nginx -s quit

4.3.3 Dynamic loading

[root@nginx nginx-1.12.2]# ngins -s reload

4.3.4 Testing the correctness of the configuration file nginx.conf

[root@nginx ~]# nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

Nginx -s quit: This method stops the process until the nginx process finishes processing the task and stops.
Nginx -s stop: This method is equivalent to first detecting the nginx process id and then using the kill command to force the process to be killed.

Nginx -s reload: Dynamic loading. When the configuration file nginx.conf changes, the command is dynamically loaded.

4.4 Boot from boot

Edit the /etc/rc.d/rc.local file and add a line /usr/local/nginx/sbin/nginx

[root@nginx rc.d]# cd /etc/rc.d
[root@nginx rc.d]# sed -i ’13a /usr/local/nginx/sbin/nginx’ /etc/rc.d/rc. Local
[root@nginx rc.d]# chmod u+x rc.local

5. Change the default port

Edit the configuration file /usr/local/nginx/conf/nginx.conf and change the default port 80 to 81:

[root@nginx ~]# view /usr/local/nginx/conf/nginx.conf

Load configuration:

[root@nginx ~]# nginx -s reload

6. Visit Nginx

6.1 Turn off the firewall

[root@nginx ~]# firewall-cmd –state
running
[root@nginx ~]# systemctl stop firewalld.service
[root@nginx ~]# firewall-cmd –state
not running

6.2 Accessing Nginx

Http://localhost:81

CentOS 7 deploys rsync backup server

1.1 rsync (official address http://wwww.samba.org/ftp/rsync/rsync.html)

A remote data synchronization tool that quickly synchronizes files between multiple hosts over a LAN/WAN. Rsync uses the so-called “rsync algorithm” to synchronize files between two local and remote hosts. This algorithm only transfers different parts of two files, rather than transmitting them all at once, so the speed is quite fast.

1.2rsync backup mode

1) Local data backup method

Rsync parameter The data to be backed up where the backup data is saved

2) Remote backup mode

Pull:rsync [OPTION…] [USER@]HOST:SRC… [DEST]
What is the rsync parameter to pull data from the corresponding host to pull data to save the local path
Push:rsync [OPTION…] SRC … [USER@]HOST:DEST
rsync where the local data is pushed by the local host data

3) Daemon process

Pull:rsync [OPTION…] [USER@]HOST::SRC… [DEST]
rsync parameter authenticates the user to pull data from the corresponding host. Pull data to save the local path
Push:rsync [OPTION…] SRC … [USER@]HOST::DEST
rsync parameter authenticates the location where the user will push the local host data for push data

2. Environmental preparation

[root@backup ~]# cat /etc/RedHat-release
CentOS Linux release 7.2.1511 (Core)
[root@backup ~]# uname -r
3.10.0-327.el7.x86_64
[root@backup ~]# getenforce
Disabled
[root@backup ~]# systemctl status firewalld.service
? firewalld.service – firewalld – dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
[root@backup ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.0.41 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::20c:29ff:fe40:1a4e prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:40:1a:4e txqueuelen 1000 (Ethernet)
RX packets 1607 bytes 355312 (346.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 358 bytes 47574 (46.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.1.41 netmask 255.255.255.0 broadcast 172.16.1.255
inet6 fe80::20c:29ff:fe40:1a58 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:40:1a:58 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 23 bytes 1698 (1.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 03.??rsync???????

3.1 Check if rsync is installed

[root@backup ~]# rpm -qa rsync

rsync-3.0.9-17.el7.x86_64

3.2 Writing rsync configuration files

[root@backup ~]# cat /etc/rsyncd.conf
#rsync_config
#created by fengyu 2018-3-16
uid = rsync Operator
gid = rsync User group
use chroot = no Related security
max connections = 200 Maximum number of connections
timeout = 300 Timeout
pid file = /var/run/rsyncd.pid The process number file corresponding to the process
lock file = /var/run/rsyncd.log lock file
log file = /var/log/rsyncd.log log file
[backup] module Name
path = /backup module location
ignore errors ignore error program
read only = false read only
list = false list of
hosts allowed = 172.16.1.01/24 network segment allowed accesses
deny = 0.0.0.0/32 network forbidden to access segment
Auth users = rsync_backup User that does not exist, only used for authentication
secrets file = /etc/rsync.password There is no key file when the user authenticates

3.3 Create an administrative user

[root@backup ~]# useradd -s /sbin/nologin -M rsync

3.4 Creating an Authentication User Password File

[root@backup ~]# echo “rsync_backup:123456” > /etc/rsync.password
[root@backup ~]# chmod 600 /etc/rsync.password

3.5 Create a backup directory

[root@backup ~]# mkdir /backup
[root@backup ~]# chown -R rsync.rsync /backup/

3.6 start daemon

[root@backup ~]# rsync –daemon
[root@backup ~]# netstat -lntup | grep rsync
tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN 3286/rsync
tcp6 0 0 :::873 :::* LISTEN 3286/rsync

4. Configure the rsync daemon client (here, the NFS storage server is used as an example. In the work, the rsync server and the NFS server are matched with each other)

4.1 Creating a Password Authentication File

[root@nfs01 ~]# echo “123456” > /etc/rsync.password

[root@nfs01 ~]# chmod 600 /etc/rsync.password

4.2 Writing real-time monitoring push scripts

[root@nfs01 backup]# cat /server/scripts/inotify.sh
#!bin/bash
inotifywait -mrq –format “%w%f” -e create,close_write,delete,moved_to /data/|\
while read fy
do
rsync -az /data/ –delete rsync_backup@172.16.1.41::backup –password-file=/etc/rsync.password
done

4.3 Put the script execution command into the /etc/rc.local directory (under the CentOS 7 system, you need to execute the permissions in the /etc/rc.local directory)

[root@nfs01 ~]# echo “/usr/bin/sh /server/scripts/inotify.sh” >> /etc/rc.local