July 2020
M T W T F S S
« Mar    
 12345
6789101112
13141516171819
20212223242526
2728293031  

Categories

WordPress Quotes

I try to learn from the past, but I plan for the future by focusing exclusively on the present. That's were the fun is.
Donald Trump
July 2020
M T W T F S S
« Mar    
 12345
6789101112
13141516171819
20212223242526
2728293031  

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (270)
centos8 (3)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
EXCHANGE (3)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (2)
horoscope (23)
Hyper-V (10)
IIS (15)
IPTABLES (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
SCALEIO (1)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Ubuntu (1)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)
ZIMBRA (17)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

0 visitors online now
0 guests, 0 bots, 0 members

Hit Counter provided by dental implants orange county

How to Install Docker CE and Docker-Compose on CentOS 8

Docker is a set of Platform as a Service (PaaS) products that uses operating system level virtualizations to deliver software in the form of containers. Docker CE (Community Edition) is the strip down version of Docker EE (Enterprise Edition). Docker CE is free and open source and distributed under Apache License 2.0.

In Red Hat Enterprise Linux (RHEL) 8 / CentOS 8, Support of Docker has been removed by the vendor. Whereas a new containerization platform libpod (Podman’s Container Management Library) has been introduced inplace of Docker.

However, we can still install Docker and it’s dependencies on CentOS 8 / RHEL 8 from third party yum repositories.

In this article, we are installing Docker CE and docker-compose on CentOS 8.

Prerequisites

  • You must have Alibaba Cloud Elastic Compute Service (ECS) activated and verified your valid payment method. If you are a new user, you can get $450 – $1300 worth in Alibaba Cloud credits for your new account. If you don’t know how to setup your ECS instance, you can refer to this tutorial or quick-start guide.
  • domain name registered from Alibaba Cloud. If you have already registered a domain from Alibaba Cloud or any other host, you can update its domain nameserver records.
  • Domain name must be pointed to your Alibaba Cloud ECS’s IP address
  • Access to VNC console in your Alibaba Cloud or SSH client installed in your PC
  • Set up your server’s hostname and create a user with root privileges.

Environment Specification

We have configured a CentOS 8 virtual machine with following specifications.

  • CPU – 3.4 Ghz (2 cores)
  • Memory – 2 GB
  • Storage – 40 GB
  • Operating System – CentOS 8.0
  • Hostname – docker-01.example.com
  • IP Address – 192.168.116.6/24

Adding Docker CE yum Repository on CentOS 8:

Connect with docker-01.example.com using ssh as root user.

Docker CE is available to download from Docker’s Official Website However, we can also install it from Docker CE yum repository.

Add Docker CE yum repository using dnf command.

[root@docker-01 ~]# dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
Adding repo from: https://download.docker.com/linux/centos/docker-ce.repo

Build cache for Docker yum repository.

[root@docker-01 ~]# dnf makecache
CentOS-8 - AppStream                            7.0 kB/s | 4.3 kB     00:00
CentOS-8 - Base                                 2.2 kB/s | 3.9 kB     00:01
CentOS-8 - Extras                               1.7 kB/s | 1.5 kB     00:00
Docker CE Stable - x86_64                       6.5 kB/s |  21 kB     00:03
Metadata cache created.

Installing Docker CE on CentOS 8:

After addition of Docker CE yum repository, we can now easily install Docker CE on CentOS 8 by using a dnf command.

Docker CE requires containerd.io-1.2.2-3 (or later) package, which is blocked in CentOS 8. Therefore, we have to use an earlier version of containerd.io package.

Install docker-ce with an earlier version of containerd.io using following command.

[root@docker-01 ~]# dnf -y install --nobest docker-ce
Last metadata expiration check: 0:21:14 ago on Wed 25 Dec 2019 10:25:37 PM PKT.
Dependencies resolved.

 Problem: package docker-ce-3:19.03.5-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed
  - cannot install the best candidate for the job
  - package containerd.io-1.2.10-3.2.el7.x86_64 is excluded
  - package containerd.io-1.2.2-3.3.el7.x86_64 is excluded
  - package containerd.io-1.2.2-3.el7.x86_64 is excluded
  - package containerd.io-1.2.4-3.1.el7.x86_64 is excluded
  - package containerd.io-1.2.5-3.1.el7.x86_64 is excluded
  - package containerd.io-1.2.6-3.3.el7.x86_64 is excluded
================================================================================
 Package                      Arch   Version             Repository        Size
================================================================================
Installing:
 docker-ce                    x86_64 3:18.09.1-3.el7     docker-ce-stable  19 M
Installing dependencies:
 container-selinux            noarch 2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7
                                                         AppStream         43 k
 checkpolicy                  x86_64 2.8-2.el8           BaseOS           338 k
 libcgroup                    x86_64 0.41-19.el8         BaseOS            70 k
 policycoreutils-python-utils noarch 2.8-16.1.el8        BaseOS           228 k
 python3-audit                x86_64 3.0-0.10.20180831git0047a6c.el8
                                                         BaseOS            85 k
 python3-libsemanage          x86_64 2.8-5.el8           BaseOS           127 k
 python3-policycoreutils      noarch 2.8-16.1.el8        BaseOS           2.2 M
 python3-setools              x86_64 4.2.0-2.el8         BaseOS           598 k
 containerd.io                x86_64 1.2.0-3.el7         docker-ce-stable  22 M
 docker-ce-cli                x86_64 1:19.03.5-3.el7     docker-ce-stable  39 M
Enabling module streams:
 container-tools                     rhel8
Skipping packages with broken dependencies:
 docker-ce                    x86_64 3:19.03.5-3.el7     docker-ce-stable  24 M

Transaction Summary
================================================================================
Install  11 Packages
Skip      1 Package

Total download size: 84 M
Installed size: 348 M
Downloading Packages:
(1/11): libcgroup-0.41-19.el8.x86_64.rpm        182 kB/s |  70 kB     00:00
(2/11): container-selinux-2.94-1.git1e99f1d.mod 108 kB/s |  43 kB     00:00
(3/11): python3-audit-3.0-0.10.20180831git0047a 102 kB/s |  85 kB     00:00
(4/11): policycoreutils-python-utils-2.8-16.1.e 132 kB/s | 228 kB     00:01
(5/11): python3-libsemanage-2.8-5.el8.x86_64.rp 106 kB/s | 127 kB     00:01
(6/11): checkpolicy-2.8-2.el8.x86_64.rpm        126 kB/s | 338 kB     00:02
(7/11): python3-setools-4.2.0-2.el8.x86_64.rpm  113 kB/s | 598 kB     00:05
(8/11): python3-policycoreutils-2.8-16.1.el8.no 109 kB/s | 2.2 MB     00:20
(9/11): docker-ce-18.09.1-3.el7.x86_64.rpm       75 kB/s |  19 MB     04:16
(10/11): containerd.io-1.2.0-3.el7.x86_64.rpm    80 kB/s |  22 MB     04:41
(11/11): docker-ce-cli-19.03.5-3.el7.x86_64.rpm 122 kB/s |  39 MB     05:31
--------------------------------------------------------------------------------
Total                                           240 kB/s |  84 MB     05:58
warning: /var/cache/dnf/docker-ce-stable-091d8a9c23201250/packages/containerd.io-1.2.0-3.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
Docker CE Stable - x86_64                       1.5 kB/s | 1.6 kB     00:01
Importing GPG key 0x621E9F35:
 Userid     : "Docker Release (CE rpm) <docker@docker.com>"
 Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
 From       : https://download.docker.com/linux/centos/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1
  Installing       : docker-ce-cli-1:19.03.5-3.el7.x86_64                  1/11
  Running scriptlet: docker-ce-cli-1:19.03.5-3.el7.x86_64                  1/11
  Installing       : containerd.io-1.2.0-3.el7.x86_64                      2/11
  Running scriptlet: containerd.io-1.2.0-3.el7.x86_64                      2/11
  Installing       : python3-setools-4.2.0-2.el8.x86_64                    3/11
  Installing       : python3-libsemanage-2.8-5.el8.x86_64                  4/11
  Installing       : python3-audit-3.0-0.10.20180831git0047a6c.el8.x86_    5/11
  Running scriptlet: libcgroup-0.41-19.el8.x86_64                          6/11
  Installing       : libcgroup-0.41-19.el8.x86_64                          6/11
  Running scriptlet: libcgroup-0.41-19.el8.x86_64                          6/11
  Installing       : checkpolicy-2.8-2.el8.x86_64                          7/11
  Installing       : python3-policycoreutils-2.8-16.1.el8.noarch           8/11
  Installing       : policycoreutils-python-utils-2.8-16.1.el8.noarch      9/11
  Installing       : container-selinux-2:2.94-1.git1e99f1d.module_el8.0   10/11
  Running scriptlet: container-selinux-2:2.94-1.git1e99f1d.module_el8.0   10/11
  Running scriptlet: docker-ce-3:18.09.1-3.el7.x86_64                     11/11
  Installing       : docker-ce-3:18.09.1-3.el7.x86_64                     11/11
  Running scriptlet: docker-ce-3:18.09.1-3.el7.x86_64                     11/11
  Verifying        : container-selinux-2:2.94-1.git1e99f1d.module_el8.0    1/11
  Verifying        : checkpolicy-2.8-2.el8.x86_64                          2/11
  Verifying        : libcgroup-0.41-19.el8.x86_64                          3/11
  Verifying        : policycoreutils-python-utils-2.8-16.1.el8.noarch      4/11
  Verifying        : python3-audit-3.0-0.10.20180831git0047a6c.el8.x86_    5/11
  Verifying        : python3-libsemanage-2.8-5.el8.x86_64                  6/11
  Verifying        : python3-policycoreutils-2.8-16.1.el8.noarch           7/11
  Verifying        : python3-setools-4.2.0-2.el8.x86_64                    8/11
  Verifying        : containerd.io-1.2.0-3.el7.x86_64                      9/11
  Verifying        : docker-ce-3:18.09.1-3.el7.x86_64                     10/11
  Verifying        : docker-ce-cli-1:19.03.5-3.el7.x86_64                 11/11

Installed:
  docker-ce-3:18.09.1-3.el7.x86_64
  container-selinux-2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7.noarch
  checkpolicy-2.8-2.el8.x86_64
  libcgroup-0.41-19.el8.x86_64
  policycoreutils-python-utils-2.8-16.1.el8.noarch
  python3-audit-3.0-0.10.20180831git0047a6c.el8.x86_64
  python3-libsemanage-2.8-5.el8.x86_64
  python3-policycoreutils-2.8-16.1.el8.noarch
  python3-setools-4.2.0-2.el8.x86_64
  containerd.io-1.2.0-3.el7.x86_64
  docker-ce-cli-1:19.03.5-3.el7.x86_64

Skipped:
  docker-ce-3:19.03.5-3.el7.x86_64

Complete!

Enable and start Docker service.

[root@docker-01 ~]# systemctl enable --now docker.service
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service â /usr/lib/systemd/system/docker.service.

Check status of Docker service.

[root@docker-01 ~]# systemctl status docker.service
â docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor pres>
   Active: active (running) since Wed 2019-12-25 22:56:45 PKT; 30s ago
     Docs: https://docs.docker.com
 Main PID: 3139 (dockerd)
    Tasks: 17
   Memory: 66.9M
   CGroup: /system.slice/docker.service
           ââ3139 /usr/bin/dockerd -H fd://
           ââ3148 containerd --config /var/run/docker/containerd/containerd.tom>

Dec 25 22:56:43 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:43.>
Dec 25 22:56:43 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:43.>
Dec 25 22:56:43 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:43.>
Dec 25 22:56:43 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:43.>
Dec 25 22:56:44 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:44.>
Dec 25 22:56:44 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:44.>
Dec 25 22:56:45 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:45.>
Dec 25 22:56:45 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:45.>
Dec 25 22:56:45 docker-01.recipes.com dockerd[3139]: time="2019-12-25T22:56:45.>
Dec 25 22:56:45 docker-01.recipes.com systemd[1]: Started Docker Application Co>

Check Docker version.

[root@docker-01 ~]# docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.39 (downgraded from 1.40)
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:25:41 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.1
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.6
  Git commit:       4c52b90
  Built:            Wed Jan  9 19:06:30 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Docker CE has been installed on CentOS 8.

Create a Container using Docker in CentOS 8:

Let’s put Docker into action by creating a simple container.

For this purpose, we are using official image of Alpine Linux from Docker Hub.

[root@docker-01 ~]# docker search alpine --filter is-official=true
NAME                DESCRIPTION                                     STARS               OFFICIAL            AUTOMATED
alpine              A minimal Docker image based on Alpine Linux⦠  5945                [OK]

Pull Alpine Linux image from Docker Hub.

[root@docker-01 ~]# docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
e6b0cf9c0882: Pull complete
Digest: sha256:2171658620155679240babee0a7714f6509fae66898db422ad803b951257db78
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latest

List locally available docker images.

[root@docker-01 ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
alpine              latest              cc0abc535e36        23 hours ago        5.59MB

Create and run a container using Alpine Linux image.

[root@docker-01 ~]# docker run -it --rm alpine /bin/sh
/ # cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.11.2
PRETTY_NAME="Alpine Linux v3.11"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
/ # uname -a
Linux c0089c037e24 4.18.0-80.11.2.el8_0.x86_64 #1 SMP Tue Sep 24 11:32:19 UTC 2019 x86_64 Linux
/ # exit

Installing Docker-compose on CentOS 8:

Additionally, we are installing docker-compose on our CentOS 8 server, so we can create and run multiple containers as a single service.

Download docker-compose package from GitHub.

[root@docker-01 ~]# curl -L https://github.com/docker/compose/releases/download/1.25.1-rc1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   617    0   617    0     0    546      0 --:--:--  0:00:01 --:--:--   546
100 16.2M  100 16.2M    0     0   184k      0  0:01:29  0:01:29 --:--:--  276k

Grant execute permissions to docker-compose command.

[root@docker-01 ~]# chmod +x /usr/local/bin/docker-compose

Check docker-compose version.

[root@docker-01 ~]# docker-compose version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-py version: 4.1.0
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

We have successfully installed Docker CE and Docker-Compose on CentOS 8. We have only explored the installation of Docker CE here

Dnsmasq Centos7

One could only guess that the rationale for lack of DNS caching in RHEL is the arguable efficiency for those systems which aren’t network connected or simply don’t need to make any DNS lookups.

There are of course such cases where you don’t need (many) DNS resolutions. I can think of:

  • a dedicated DB server
  • a private server where all hosts are listed in the hosts file

Those systems will likely issue zero to none DNS lookups while running, and DNS cache isn’t really a thing for them.

But for the most intents of running either a desktop or server RHEL machines, you will absolutely benefit from a DNS cache.

Enabling DNS cache in RHEL 7 and 8 is easy thanks to dnsmasq integration of NetworkManager.

The dnsmasq is a very lightweight caching DNS forwarder which runs great even on the tiniest hardware like your very own home router.

I won’t torture you with long instructions on how to enable the DNS cache. It’s really quick and goes down to:

yum -y install dnsmasq

cat << 'EOF' | sudo tee /etc/NetworkManager/conf.d/dns.conf 
[main]
dns=dnsmasq
EOF

systemctl reload NetworkManager

You have just made your machine already faster by running these.

For more details and fine-tuning, read on.

NetworkManager and dnsmasq

Let’s explain what happened when we ran the above commands to enable DNS caching.

In the first bit, we have installed the very essential of DNS caching – dnsmasq program.

Then we write out a file, /etc/NetworkManager/conf.d/dns.conf, with contents telling NetworkManager to enable and use its dnsmasq plugin. Then we reload NetworkManager configuration to apply our changes.

This, in turn, starts a private instance of dnsmasq program, which is bound to the loopback interface, 127.0.0.1 and listening on standard DNS port, 53.

It doesn’t end there. NetworkManager now updated /etc/resolv.conf and put nameserver 127.0.0.1 so that the whole operating system will perform DNS lookups against its dnsmasq instance.

The dnsmasq itself will use whatever nameservers you had setup in NetworkManager explicitly, or the ones provided by DHCP requests.

Very clean and beautiful integration.

Verify dnsmasq is working

Simply perform a DNS lookup using dig, against 127.0.0.1

# yum -y install bind-utils
dig +short example.com @127.0.0.1

If the output looks like a valid IP address or a list of IP addresses, then dnsmasq is working OK.

You can also check that DNS caching is working. Perform a resolution against another domain by running the following command twice:

time getent hosts foo.example.com

Observe real timing in the output reduced for the subsequent queries. E.g. first request yields:

real   0m0.048s
user   0m0.006s
sys    0m0.006s

Subsequent requests yield:

real   0m0.009s
user   0m0.006s
sys    0m0.002s

See what kind of DNS requests your system makes

To see what DNS request your system makes, you can temporarily enable logging of queries. Note that this will clear DNS cache because dnsmasq will be restarted:

echo log-queries | sudo tee -a /etc/NetworkManager/dnsmasq.d/log.conf
sudo systemctl reload NetworkManager

You can then tail or less the /var/log/messages file which will have information of requests being made. Example, on the web server that is using PaperTrail’s remote_syslog:

dnsmasq[20802]: forwarded logs6.papertrailapp.com to 2606:4700:4700::1001
dnsmasq[20802]: reply logs6.papertrailapp.com is 169.46.82.182
dnsmasq[20802]: reply logs6.papertrailapp.com is 169.46.82.183
dnsmasq[20802]: reply logs6.papertrailapp.com is 169.46.82.184
dnsmasq[20802]: reply logs6.papertrailapp.com is 169.46.82.185

This approach may be used for finding what external sites your server communicates with.

Once you’re done, don’t forget to turn off the logging:

sudo rm /etc/NetworkManager/dnsmasq.d/log.conf
sudo systemctl reload NetworkManager

How well is dnsmasq doing on your system

The dnsmasq manpage has this to say:

When it receives a SIGUSR1, dnsmasq writes statistics to the system log. It writes the cache size, the number of names which have had to removed from the cache before they expired in order to
make room for new names and the total number of names that have been inserted into the cache. The number of cache hits and misses and the number of authoritative queries answered are also given.

So we can collect DNS query stats easily:

sudo pkill --signal USR1 dnsmasq && sudo tail /var/log/messages | grep dnsmasq

The output may include, for example:

dnsmasq[31949]: cache size 400, 0/60 cache insertions re-used unexpired cache entries.
queries forwarded 30, queries answered locally 60

The 0 in 0/60 stands for “zero cache evictions”. So this number indicates that cache size is adequate. It should be as low as possible.
If that number is high, it means that cache size maybe not large enough.

We also see that 30 DNS lookups were forwarded over to upstream nameservers (misses), while 60 were satisfied directly by cache (hits).

Gathering stats like this will work well in case you only have one instance of dnsmasq. Sometimes you have more than one (e.g. libvirt may run one of its own).

It is more reliable to use the statistical information of dnsmasq that is exposed, not surprisingly, via DNS ???? The commands:

dig +short chaos txt hits.bind
dig +short chaos txt misses.bind

… give you hits and misses, respectively.

With some command line magic, you can easily calculate your DNS cache hit-ratio:

# yum -y install bc
echo "scale=2; $(dig +short chaos txt hits.bind)*100/($(dig +short chaos txt hits.bind)+$(dig +short chaos txt misses.bind))" | \
  sed 's@"@@g' | bc

The output is a percentage of DNS requests that were satisfied by DNS cache, e.g.: 80.95%.

Tuning the cache size

The default cache size of dnsmasq instance that is run by Networkmanager is 400.
This is a decent default for web servers.

For a desktop machine, you may want to increase it by large. This will assist with much less home router strain and faster network experience, especially if you’re a Chrome user. This browser does DNS caching of its own, but only as long as 1 minute – the issue that is discarded as a “feature”.

So to set DNS cache size to 20k, run:

echo cache-size=20000 | sudo tee -a /etc/NetworkManager/dnsmasq.d/cache.conf
sudo systemctl reload NetworkManager

dnsmasq and your desktop

To expand the topic of the desktop use of dnsmasq, you can also leverage it to block tracking scripts and for speeding up your browsing experience:

sudo curl https://raw.githubusercontent.com/aghorler/lightweight-dnsmasq-blocklist/master/list.txt \
  --output /etc/NetworkManager/dnsmasq.d/blocklist.conf
sudo systemctl reload NetworkManager

Finally, you may also want to improve the DNS speed by ensuring minimum TTL for DNS records that have it set too low.

echo min-cache-ttl=1800 | sudo tee -a /etc/NetworkManager/dnsmasq.d/cache.conf
sudo systemctl reload NetworkManager

This will ensure that even if a DNS record is configured with, e.g. 2 minutes TTL on remote nameserver, dnsmasq will still cache it for 30 minutes.

Note that this is acceptable for desktop machines, but not for web servers:

Phew, now I think that’s about it for dnsmasq today. Enjoy your faster DNS and be sure to subscribe for our Twitter for more fine articles ????

DOCKER INSTALL CENTOS7

CentOS 7 non-root users install source version of Docker

  1. Check if the current host has a docker group

cat /etc/group | grep docker

sudo groupadd docker

cat /etc/group | grep docker

useradd test

cat /etc/passwd | grep dev01

Add sudo permissions for new users

vi /etc/sudoers
??Add on line 92 next line
??dev01 ALL = (ALL) ALL

  1. Add the current user to the docker group (at this time the user has not joined the docker group)

gpasswd -a admin docker

https://download.docker.com/linux/static/stable/x86_64/docker-19.03.7.tgz

mkdir /docker

tar -zxvf docker-19.03.7.tgz -C /docker

cp docker/* /usr/bin/

chown root:docker /usr/bin/docker*
chown root:docker /usr/bin/containerd*
chown root:docker /usr/bin/runc
chown root:docker /usr/bin/ctr

ll /usr/bin/ | grep docker

vi /etc/systemd/system/docker.service

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target

[Service]
Type=notify
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target

chmod a+x /etc/systemd/system/docker.service
systemctl daemon-reload

vi /etc/docker/daemon.json

{
“registry-mirrors”: [“http://hub-mirror.rmohan.com”]
}

systemctl start docker

docker basic commands

docker start / stop / restart / view the status
sudo systemctl start / stop / restart / status

View docker has been mirrored
docker images

in the docker’s official website searches for the specified mirror
docker search image

Download image (without labeling the default download the latest version of the image)
docker pull Mirror Name: tag (ie tag)

Start the container (run the image-based container with the name xxx, and map the container port to the local port, and the container directory file is stored in the local directory)

docker run -d -name xxx -p Local port: Container port -v native directory: container directory image name: tag (or ID)

into the running container

docker exec -it container name (or ID) / bin / bash

container start / stop / restart / information / delete

docker start / stop / restart / inspect / rm container name (or ID)

view running containers

docker ps

view all containers (including running, stopped, not including deleted)

docker ps -a

image deletion (before deleting the image Please delete all containers related to this image)

docker rmi image name: tag (or ID)

view information about currently installed docker

docker info

yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager –add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum install docker-ce

systemctl enable docker

systemctl start docker

groupadd docker

usermod -aG docker $USER

docker volume create portainer_data

docker run -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

Docker common commands
docker ps -view all containers currently running -a display including stopped containers

docker pull -pull images

docker rmi -After deleting the image, you can match it directly based on the image name or the first letter of the tag

docker start container_id- open container (here can be container id or name)

docker stop container_id -stop container (here can be container id or name)

docker rm -delete a container (only stopped containers can be deleted)

docker build -create images using Dockerfile

docker exec -execute commands in the container, for example: docker exec -it container_id (container name or id) / bin / bash (bin / bash command or tool to execute)

docker logs –View container logs, for example: docker logs -f -t –tail 10 container_id (container name or id)

Run the container

docker run -it –rm -p 8000:80 –name aspnet_sample microsoft/dotnet__

–name container name, followed by mirror path or name

–rm delete the container after running

-p port mapping 8000 external port 80 mirroring running port mapping 8000 to 80 mirroring

-it outputs the contents of the container command line, that is, the container’s own program output is a bit similar to the foreground run in the console

-d Contrary to it Hide background run

LEMP is an acronym for Linux, Nginx (pronounced Engine X), MariaDB / MySQL, and PHP. Centos 8.1

LEMP is a software stack that includes a set of free open source tools that are used to power high traffic and dynamic websites. LEMP is an acronym for Linux, Nginx (pronounced Engine X), MariaDB / MySQL, and PHP.

Nginx is an open source, powerful and high-performance web server that can also double as a reverse proxy. MariaDB is a database system for storing user data, while PHP is a server-side scripting language for developing and supporting dynamic web pages.

Related:

In this article, you will learn how to install a LEMP server on a CentOS 8 Linux distribution.

Step 1: Update the package on CentOS 8

First, update the repository and packages on CentOS 8 Linux by running the following dnf command.

dnf update

Step 2: Install Nginx web server on CentOS 8

After the package update is complete, install Nginx with a simple command.

dnf install nginx

Install Nginx on CentOS 8

The code snippet shows that the installation of Nginx went smoothly without any problems.

After the installation is complete, configure Nginx to start automatically at system startup, and verify that Nginx is running by executing a command.

systemctl enable nginx
systemctl start nginx
systemctl status nginx

nginx -v

Step 3: Install MariaDB on CentOS 8

MariaDB is a free and open source branch of MySQL and provides the latest features that make it a better alternative to MySQL. To install MariaDB, run the command.

dnf install mariadb-server mariadb -y

To make MariaDB start automatically at system startup, run.

systemctl start mariadb
systemctl enable mariadb

The MariaDB database engine is not secure and anyone can log in without credentials. To harden MariaDB and protect it to minimize the chance of unauthorized access, run the command.

mysql_secure_installation

Step 4: Install PHP 7 on CentOS 8

Finally, we will install the last LEMP component, PHP, which is a scripted web programming language that is usually used to develop dynamic web pages.

At the time of writing this guide, the latest version is PHP 7.4. We will install it using the Remi repository. The Remi database is a free database that comes with the latest cutting-edge software version and is not available on CentOS by default.

Run the following command to install the EPEL repository.

dnf install https://dl.Fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

dnf install dnf-utils http://rpms.remirepo.net/enterprise/remi-release-8.rpm

dnf module list php

CentOS-8 – AppStream
Name Stream Profiles Summary
php 7.2 [d][e] common [d], devel, minimal PHP scripting language
php 7.3 common, devel, minimal PHP scripting language

Remi’s Modular repository for Enterprise Linux 8 – x86_64
Name Stream Profiles Summary
php remi-7.2 common [d], devel, minimal PHP scripting language
php remi-7.3 common [d], devel, minimal PHP scripting language
php remi-7.4 common [d], devel, minimal PHP scripting language

dnf module reset php

dnf module enable php:remi-7.4

dnf install php php-opcache php-gd php-curl php-mysqlnd

php -v
PHP 7.4.3 (cli) (built: Feb 18 2020 11:53:05) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Zend OPcache v7.4.3, Copyright (c), by Zend Technologies

systemctl start php-fpm
systemctl enable php-fpm

nano /etc/php-fpm.d/www.conf

user = apache
group = apache

user = nginx
group = nginx

systemctl restart nginx
systemctl restart php-fpm

cd /usr/share/nginx/html/
$echo “” > index.php

Build LAMP (Linux + Apache + MySQL + PHP) environment under CentOS 8.1

LAMP is an acronym for Linux, Apache, MySQL, and PHP, and is a popular free and open source stack used by webmasters and developers to test and host dynamic websites.
The LAMP server comes with 4 core components: Apache web server, MySQL or MariaDB database, and PHP (a popular scripting language for creating dynamic web pages).
Common LAMP architecture platform! LAMP is the most popular combination in the world, of course, there is also Nginx, which is LNMP: LAMP is more secure than NGINX,
but Nginx is more powerful than Apache in handling high concurrency. In this article, you will learn Install LAMP server on CentOS 8 Linux distribution.

Step 1: Update CentOS 8 software package
It is recommended that it is usually a good practice to update packages before starting the installation. So log in to your server and run the following command.

dnf update

dnf install httpd httpd-tools

systemctl enable httpd

systemctl start httpd

systemctl status httpd

httpd -v

rpm -qi httpd

centos8 with mariaDB

dnf install mariadb-server mariadb -y

systemctl start mariadb

systemctl enable mariadb

mysql_secure_installation

dnf install https://dl.Fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

dnf install dnf-utils http://rpms.remirepo.net/enterprise/remi-release-8.rpm

dnf module list php

dnf module reset php

dnf module enable php:remi-7.4

dnf install php php-opcache php-gd php-curl php-mysqlnd

dnf install php php-opcache php-gd php-curl php-mysqlnd

php -v

systemctl start php-fpm
systemctl enable php-fpm
systemctl status php-fpm

setsebool -P httpd_execmem 1

systemctl restart httpd

Herbal bath poweder

Nalangu Maavu is an hand made 100% natural product for herbal body wash or face wash/face mask which is popular in South India.
It is a product, made with natural ingredients and it was everyone’s choice of a good cleanser.
It is so gentle and will bring glow to your skin.

INGREDIENTS:

MINT LEAVES
Poosanthu pattai


Thirimanjana pattai
Neem powder
SANDALWOOD POWDER
AVARAM POO
NATTU KICHILI
SEEMAI KICHILI

KARBOKA ARISI

cinnamon
lavanga pathiri ilai
kalpasi
pachilai
vasambu
marikozhlunthu
poolan kizhlangu
shebagha poo
mali nanari
KORAI KILANGU
Athi mathuram
USILAM PODI
MAGILAM POO
ROSE PETALS
KASTURI MANJAL
MARU
DAVANAM
NANNARI
VETIVER
VILAMESAI ROOT
KASA KASA
RAW RICE
CHEMBARUTHI FLOWER.

Benefits:
Green Gram exfoliates dead skin cells and brightens up skin texture. Also lightens skin tone and clears tan.
Rose petals have properties and help to reduces wrinkles and prevents the skin from sun damage.As all of us Know,
Kasturi Manjal helps to treat blemishes and pimples.
Korai Kizhangu Helps to remove the unwanted hair from body and face.
Mint Leaves act as a cleaning agent and make you fresh and oil free.
Raw Rice that contains Vitamin E nourishes the skin and makes it look younger.

Preparation :

Dry all the ingredients in the sun for 2 or 3 days.
Then grind it in a flour mill nearby and store in an airtight container. You can even grind in a mixie if you have a good one.

Mix a required amount of powder with water and use it as a soap.

I am sure you will definitely love this amazing bath powder. You will be wondering to see the positive changes in your skin, once you start using this powder.

Ingredients :

Whole green gram – 1 kg

Bengal gram (Kadalai paruppu) – 200 gms

Spiked ginger – 100 gms

Vetiver – one fistful

Wild turmeric – 100 gms

Neem – one hand full of leaves (dried)

Green gram is an excellent cleanser. It is a natural beauty product. It is used in face packs, as it is a good scrubber. It brightens our skin.

Bengal gram is a tan removal agent. It is used in face packs with a few other ingredients, as it lightens our skin, clears pimples, helps to fade acne scars and to get rid of blackheads.

Spiked ginger (Poolaan kilangu or seemai kichilli kilangu in Tamil, Valiya kacholam in Malayalam) is used for enhancing skin complexion. It is the best skin conditioner. It has a nice aroma.

Vetiver, (Ramacham ver in Malayalam) has an amazing aroma. It has anti-inflammatory properties.
It is known for its soothing and cooling effects, as it is loaded with hydrating qualities. It is an anti-aging tonic. Clears acne and boosts skin health.
Wild turmeric, known as Kasthuri manjal is a great healer for skin infections and acne. It is a natural antiseptic and wonderful skin rejuvenator. This turmeric does not leave a yellow tint on the skin.

Neem, as we all know has great benefits for the skin. It is a natural antiseptic. Helps in clearing scars, removing blackheads, retains moisture, tones the skin and a great cure for any kind of skin infection.

Benefits of using Nalangu Maavu Herbal Bath Powder:
Nalangu Maavu absorbs excess oil from the skin without drying it
It helps to restore the natural pH balance of skin
Turmeric in the bath powder works wonders for inflammation
Turmeric also promotes the body’s synthesis of antioxidants and slows down visible signs of aging
Regular usage of herbal bath powder can help people with acne or pimples, by reducing oil secretion
Using Nalangu Maavu Herbal Bath Powder on a daily basis can help reduce body odor and excessive sweating
It acts like a toner, minimizing pores on the skin
Nalangu Maavu Herbal Bath Powder is antifungal, anti-bacterial and anti-microbial

How To Configure IP Address In Ubuntu 18.04 LTS

Netplan has been introduced by Ubuntu developers in Ubuntu 17.10. In this new approach, we no longer use /etc/network/interfaces file to configure IP address rather we use a YAML file. The default configuration files of Netplan are found under /etc/netplan/ directory. In this brief tutorial, we are going to learn to configure static and dynamic IP address in Ubuntu 18.04 LTS server and desktop editions.

Configure Static IP Address In Ubuntu 18.04 LTS Server

Let us find out the default network configuration file:

$ ls /etc/netplan/
50-cloud-init.yaml

As you can see, the default network configuration file is 50-cloud-init.yaml and it is obviously a YAML file.

Now, let check the contents of this file:

$ cat /etc/netplan/50-cloud-init.yaml

Add the configuration for available interfaces like eth0: and eth1:

network:
   ethernets:
     eth0:          
     addresses:
     - 192.168.1.9/24
     dhcp: false
     gateway4: 192.168.1.1
     nameservers:
        addresses:
        - 192.168.1.1
        - 8.8.8.8
        - 8.8.4.4
        search: []
     eth0:
     addresses:
     - 192.168.1.9/24
     dhcp: false
  version: 2  

How to disable Cloud-Init in a RHEL Cloud Image

So this one is pretty simple. However, I found a lot of misinformation along the way, so I figured that I would jot the proper (and most simple) process here.

Symptoms: a RHEL (or variant) VM that takes a very long time to boot. On the VM console, you can see the following output while the VM boot process is stalled and waiting for a timeout. Note that the message below has nothing to do with cloud init, but its the output that I have most often seen on the console while waiting for a VM to boot.

[106.325574} random: crng init done

Note that I have run into this issue in both OpenStack (when booting from external provider networks) and in KVM.

Upon initial boot of the VM, run the command below.

touch /etc/cloud/cloud-init.disabled

How to install Apache, PHP 7.3 and MySQL on CentOS 7.6

How to install Apache, PHP 7.3 and MySQL on CentOS 7.6

I will add the EPEL repo here to install latest phpMyAdmin as follows:

rpm –import /etc/pki/rpm-gpg/RPM-GPG-KEY*
yum -y install epel-release

Installing MySQL / MariaDB
MariaDB is a MySQL fork of the original MySQL developer Monty Widenius. MariaDB is compatible with MySQL and I’ve chosen to use MariaDB here instead of MySQL. Run this command to install MariaDB with yum:

yum -y install mariadb-server mariadb
Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the system boots) and start the MySQL server:

systemctl start mariadb.service
systemctl enable mariadb.service
Set passwords for the MySQL root account:

mysql_secure_installation

[root@server1 ~]

# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we’ll need the current
password for the root user. If you’ve just installed MariaDB, and
you haven’t set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none): <–ENTER
OK, successfully used password, moving on…

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

Set root password? [Y/n]
New password: <–yourmariadbpassword
Re-enter new password: <–yourmariadbpassword
Password updated successfully!
Reloading privilege tables..
… Success!

By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] <–ENTER
… Success!

Normally, root should only be allowed to connect from ‘localhost’. This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] <–ENTER
… Success!

By default, MariaDB comes with a database named ‘test’ that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] <–ENTER

  • Dropping test database…
    … Success!
  • Removing privileges on test database…
    … Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] <–ENTER
… Success!

Cleaning up…

All done! If you’ve completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

[root@server1 ~]

#
3 Installing Apache
CentOS 7 ships with Apache 2.4. Apache is directly available as a CentOS 7 package, therefore we can install it like this:

yum -y install httpd

Now configure your system to start Apache at boot time…

systemctl start httpd.service
systemctl enable httpd.service
To be able to access the webserver from outside, we have to open the HTTP (80) and HTTPS (443) ports in the firewall. The default firewall on CentOS is firewalld which can be configured with the firewalld-cmd command.

firewall-cmd –permanent –zone=public –add-service=http
firewall-cmd –permanent –zone=public –add-service=https
firewall-cmd –reload
Now direct your browser to the IP address of your server, in my case http://192.168.1.100, and you should see the Apache placeholder page:

Installing PHP
The PHP version that ships with CentOS as default is quite old (PHP 5.4). Therefore I will show you in this chapter some options to install newer PHP versions like PHP 7.0 to 7.3 from Remi repository.

Add the Remi CentOS repository.

rpm -Uvh http://rpms.remirepo.net/enterprise/remi-release-7.rpm
Install yum-utils as we need the yum-config-manager utility.

yum -y install yum-utils
and run yum update

yum update
Now you have to chose which PHP version you want to use on the server. If you like to use PHP 5.4, then proceed to chapter 4.1. To install PHP 7.0, follow the commands in chapter 4.2, for PHP 7.1 chapter 4.3, for PHP 7.4 use chapter 4.4 and for PHP 7.3 follow chapter 4.5 instead. Follow just one of the 4.x chapters and not all of them as you can only use one PHP version at a time with Apache mod_php.

4.1 Install PHP 5.4
To install PHP 5.4, run this command:

yum -y install php
4.2 Install PHP 7.0
We can install PHP 7.0 and the Apache PHP 7.0 module as follows:

yum-config-manager –enable remi-php70
yum -y install php php-opcache
4.3 Install PHP 7.1
If you want to use PHP 7.1 instead, use:

yum-config-manager –enable remi-php71
yum -y install php php-opcache
4.4 Install PHP 7.2
If you want to use PHP 7.2 instead, use:

yum-config-manager –enable remi-php72
yum -y install php php-opcache
4.5 Install PHP 7.3
If you want to use PHP 7.3 instead, use:

yum-config-manager –enable remi-php73
yum -y install php php-opcache
In this example and in the downloadable virtual machine, I’ll use PHP 7.3.

We must restart Apache to apply the changes:

systemctl restart httpd.service
5 Testing PHP / Getting Details About Your PHP Installation
The document root of the default website is /var/www/html. We will create a small PHP file (info.php) in that directory and call it in a browser to test the PHP installation. The file will display lots of useful details about our PHP installation, such as the installed PHP version.

nano /var/www/html/info.php
<?php
phpinfo();
Now we call that file in a browser (e.g. http://192.168.1.100/info.php)

Getting MySQL Support In PHP
To get MySQL support in PHP, we can install the php-mysqlnd package. It’s a good idea to install some other PHP modules as well as you might need them for your applications. You can search for available PHP5 modules like this:

yum search php
Pick the ones you need and install them like this:

yum -y install php-mysqlnd php-pdo
In the next step I will install some common PHP modules that are required by CMS Systems like WordPress, Joomla, and Drupal:

yum -y install php-gd php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring php-soap curl curl-devel
Now restart Apache web server:

systemctl restart httpd.service
Now reload http://192.168.1.100/info.php in your browser and scroll down to the modules section again. You should now find lots of new modules like curl etc there.:

If you don’t need the PHP info output anymore, then delete that file for security reasons.

rm /var/www/html/info.php

7 phpMyAdmin installation

phpMyAdmin is a web interface through which you can manage your MySQL databases.
phpMyAdmin can now be installed as follows:

yum -y install phpMyAdmin

Now we configure phpMyAdmin. We change the Apache configuration so that phpMyAdmin allows connections not just from localhost (by commenting out the stanza and adding the ‘Require all granted’ line):

nano /etc/httpd/conf.d/phpMyAdmin.conf
[…]
Alias /phpMyAdmin /usr/share/phpMyAdmin
Alias /phpmyadmin /usr/share/phpMyAdmin


AddDefaultCharset UTF-8


# Apache 2.4

Require ip 127.0.0.1

Require ip ::1

Require all granted

# Apache 2.2 Order Deny,Allow Deny from All Allow from 127.0.0.1 Allow from ::1


Options none
AllowOverride Limit
Require all granted

Restart Apache to apply the configuration changes:

systemctl restart httpd.service

Afterwards, you can access phpMyAdmin under http://192.168.1.100/phpmyadmin/

aws-cli

Description

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

The AWS CLI introduces a new set of simple file commands for efficient file transfers to and from Amazon S3.

Supported Services

For a list of the available services you can use with AWS Command Line Interface, see Available Services in the AWS CLI Command Reference.

AWS Command Line Interface on GitHub

You can view—and fork—the source code for the AWS CLI on GitHub in the https://github.com/aws/aws-cli project.

Versions#

VersionRelease Date
1.10.382016-06-14
1.10.352016-06-03
1.10.332016-05-25
1.10.302016-05-18

AWS CLI Cheat sheet – List of All CLI commands#

Setup#

Install AWS CLI#

AWS CLI is an common CLI tool for managing the AWS resources. With this single tool we can manage all the aws resources

sudo apt-get install -y python-dev python-pip
sudo pip install awscli
aws --version
aws configure

Bash one-liners#

cat <file> # output a file
tee # split output into a file
cut -f 2 # print the 2nd column, per line
sed -n '5{p;q}' # print the 5th line in a file
sed 1d # print all lines, except the first
tail -n +2 # print all lines, starting on the 2nd
head -n 5 # print the first 5 lines
tail -n 5 # print the last 5 lines

expand # convert tabs to 4 spaces
unexpand -a # convert 4 spaces to tabs
wc # word count
tr ' ' \\t # translate / convert characters to other characters

sort # sort data
uniq # show only unique entries
paste # combine rows of text, by line
join # combine rows of text, by initial column value

Cloudtrail – Logging and Auditing#

http://docs.aws.amazon.com/cli/latest/reference/cloudtrail/ 5 Trails total, with support for resource level permissions

# list all trails
aws cloudtrail describe-trails

# list all S3 buckets
aws s3 ls

# create a new trail
aws cloudtrail create-subscription \
    --name awslog \
    --s3-new-bucket awslog2016

# list the names of all trails
aws cloudtrail describe-trails --output text | cut -f 8

# get the status of a trail
aws cloudtrail get-trail-status \
    --name awslog

# delete a trail
aws cloudtrail delete-trail \
    --name awslog

# delete the S3 bucket of a trail
aws s3 rb s3://awslog2016 --force

# add tags to a trail, up to 10 tags
aws cloudtrail add-tags \
    --resource-id awslog \
    --tags-list "Key=log-type,Value=all"

# list the tags of a trail
aws cloudtrail list-tags \
    --resource-id-list 

# remove a tag from a trail
aws cloudtrail remove-tags \
    --resource-id awslog \
    --tags-list "Key=log-type,Value=all"

IAM#

Users#

https://blogs.aws.amazon.com/security/post/Tx15CIT22V4J8RP/How-to-rotate-access-keys-for-IAM-usershttp://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-limits.html Limits = 5000 users, 100 group, 250 roles, 2 access keys / user

http://docs.aws.amazon.com/cli/latest/reference/iam/index.html
# list all user's info
aws iam list-users

# list all user's usernames
aws iam list-users --output text | cut -f 6

# list current user's info
aws iam get-user

# list current user's access keys
aws iam list-access-keys

# crate new user
aws iam create-user \
    --user-name aws-admin2

# create multiple new users, from a file
allUsers=$(cat ./user-names.txt)
for userName in $allUsers; do
    aws iam create-user \
        --user-name $userName
done

# list all users
aws iam list-users --no-paginate

# get a specific user's info
aws iam get-user \
    --user-name aws-admin2

# delete one user
aws iam delete-user \
    --user-name aws-admin2

# delete all users
# allUsers=$(aws iam list-users --output text | cut -f 6);
allUsers=$(cat ./user-names.txt)
for userName in $allUsers; do
    aws iam delete-user \
        --user-name $userName
done

Password policy#

http://docs.aws.amazon.com/cli/latest/reference/iam/
# list policy
# http://docs.aws.amazon.com/cli/latest/reference/iam/get-account-password-policy.html
aws iam get-account-password-policy

# set policy
# http://docs.aws.amazon.com/cli/latest/reference/iam/update-account-password-policy.html
aws iam update-account-password-policy \
    --minimum-password-length 12 \
    --require-symbols \
    --require-numbers \
    --require-uppercase-characters \
    --require-lowercase-characters \
    --allow-users-to-change-password

# delete policy
# http://docs.aws.amazon.com/cli/latest/reference/iam/delete-account-password-policy.html
aws iam delete-account-password-policy

Access Keys#

http://docs.aws.amazon.com/cli/latest/reference/iam/
# list all access keys
aws iam list-access-keys

# list access keys of a specific user
aws iam list-access-keys \
    --user-name aws-admin2

# create a new access key
aws iam create-access-key \
    --user-name aws-admin2 \
    --output text | tee aws-admin2.txt

# list last access time of an access key
aws iam get-access-key-last-used \
    --access-key-id AKIAINA6AJZY4EXAMPLE

# deactivate an acccss key
aws iam update-access-key \
    --access-key-id AKIAI44QH8DHBEXAMPLE \
    --status Inactive \
    --user-name aws-admin2

# delete an access key
aws iam delete-access-key \
    --access-key-id AKIAI44QH8DHBEXAMPLE \
    --user-name aws-admin2

Groups, Policies, Managed Policies#

http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.htmlhttp://docs.aws.amazon.com/cli/latest/reference/iam/
# list all groups
aws iam list-groups

# create a group
aws iam create-group --group-name FullAdmins

# delete a group
aws iam delete-group \
    --group-name FullAdmins

# list all policies
aws iam list-policies

# get a specific policy
aws iam get-policy \
    --policy-arn <value>

# list all users, groups, and roles, for a given policy
aws iam list-entities-for-policy \
    --policy-arn <value>

# list policies, for a given group
aws iam list-attached-group-policies \
    --group-name FullAdmins

# add a policy to a group
aws iam attach-group-policy \
    --group-name FullAdmins \
    --policy-arn arn:aws:iam::aws:policy/AdministratorAccess

# add a user to a group
aws iam add-user-to-group \
    --group-name FullAdmins \
    --user-name aws-admin2

# list users, for a given group
aws iam get-group \
    --group-name FullAdmins

# list groups, for a given user
aws iam list-groups-for-user \
    --user-name aws-admin2

# remove a user from a group
aws iam remove-user-from-group \
    --group-name FullAdmins \
    --user-name aws-admin2

# remove a policy from a group
aws iam detach-group-policy \
    --group-name FullAdmins \
    --policy-arn arn:aws:iam::aws:policy/AdministratorAccess

# delete a group
aws iam delete-group \
    --group-name FullAdmins

EC2#

keypairs#

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
# list all keypairs
# http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-key-pairs.html
aws ec2 describe-key-pairs

# create a keypair
# http://docs.aws.amazon.com/cli/latest/reference/ec2/create-key-pair.html
aws ec2 create-key-pair \
    --key-name <value>

# create a new private / public keypair, using RSA 2048-bit
ssh-keygen -t rsa -b 2048

# import an existing keypair
# http://docs.aws.amazon.com/cli/latest/reference/ec2/import-key-pair.html
aws ec2 import-key-pair \
    --key-name keyname_test \
    --public-key-material file:///home/apollo/id_rsa.pub

# delete a keypair
# http://docs.aws.amazon.com/cli/latest/reference/ec2/delete-key-pair.html
aws ec2 delete-key-pair \
    --key-name <value>

Security Groups#

http://docs.aws.amazon.com/cli/latest/reference/ec2/index.html
# list all security groups
aws ec2 describe-security-groups

# create a security group
aws ec2 create-security-group \
    --vpc-id vpc-1a2b3c4d \
    --group-name web-access \
    --description "web access"

# list details about a securty group
aws ec2 describe-security-groups \
    --group-id sg-0000000

# open port 80, for everyone
aws ec2 authorize-security-group-ingress \
    --group-id sg-0000000 \
    --protocol tcp \
    --port 80 \
    --cidr 0.0.0.0/24

# get my public ip
my_ip=$(dig +short myip.opendns.com @resolver1.opendns.com);
echo $my_ip

# open port 22, just for my ip
aws ec2 authorize-security-group-ingress \
    --group-id sg-0000000 \
    --protocol tcp \
    --port 80 \
    --cidr $my_ip/24

# remove a firewall rule from a group
aws ec2 revoke-security-group-ingress \
    --group-id sg-0000000 \
    --protocol tcp \
    --port 80 \
    --cidr 0.0.0.0/24

# delete a security group
aws ec2 delete-security-group \
    --group-id sg-00000000

Instances#

http://docs.aws.amazon.com/cli/latest/reference/ec2/index.html
# list all instances (running, and not running)
# http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html
aws ec2 describe-instances

# create a new instance
# http://docs.aws.amazon.com/cli/latest/reference/ec2/run-instances.html
aws ec2 run-instances \
    --image-id ami-f0e7d19a \    
    --instance-type t2.micro \
    --security-group-ids sg-00000000 \
    --dry-run

# stop an instance
# http://docs.aws.amazon.com/cli/latest/reference/ec2/terminate-instances.html
aws ec2 terminate-instances \
    --instance-ids <instance_id>

# list status of all instances
# http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instance-status.html
aws ec2 describe-instance-status

# list status of a specific instance
aws ec2 describe-instance-status \
    --instance-ids <instance_id>

Tags#

# list the tags of an instance
# http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-tags.html
aws ec2 describe-tags

# add a tag to an instance
# http://docs.aws.amazon.com/cli/latest/reference/ec2/create-tags.html
aws ec2 create-tags \
    --resources "ami-1a2b3c4d" \
    --tags Key=name,Value=debian

# delete a tag on an instance
# http://docs.aws.amazon.com/cli/latest/reference/ec2/delete-tags.html
aws ec2 delete-tags \
    --resources "ami-1a2b3c4d" \
    --tags Key=Name,Value=

Cloudwatch#

Log Groups#

http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/WhatIsCloudWatchLogs.htmlhttp://docs.aws.amazon.com/cli/latest/reference/logs/index.html#cli-aws-logscreate a group

http://docs.aws.amazon.com/cli/latest/reference/logs/create-log-group.html
aws logs create-log-group \
    --log-group-name "DefaultGroup"

list all log groups

http://docs.aws.amazon.com/cli/latest/reference/logs/describe-log-groups.html
aws logs describe-log-groups

aws logs describe-log-groups \
    --log-group-name-prefix "Default"

delete a group

http://docs.aws.amazon.com/cli/latest/reference/logs/delete-log-group.html
aws logs delete-log-group \
    --log-group-name "DefaultGroup"

Log Streams#

# Log group names can be between 1 and 512 characters long. Allowed
# characters include a-z, A-Z, 0-9, '_' (underscore), '-' (hyphen),
# '/' (forward slash), and '.' (period).

# create a log stream
# http://docs.aws.amazon.com/cli/latest/reference/logs/create-log-stream.html
aws logs create-log-stream \
    --log-group-name "DefaultGroup" \
    --log-stream-name "syslog"

# list details on a log stream
# http://docs.aws.amazon.com/cli/latest/reference/logs/describe-log-streams.html
aws logs describe-log-streams \
    --log-group-name "syslog"

aws logs describe-log-streams \
    --log-stream-name-prefix "syslog"

# delete a log stream
# http://docs.aws.amazon.com/cli/latest/reference/logs/delete-log-stream.html
aws logs delete-log-stream \
    --log-group-name "DefaultGroup" \
    --log-stream-name "Default Stream"

AWS completer for Ubuntu with Bash#

The following utility can be used for auto-completion of commands:

$ which aws_completer
/usr/bin/aws_completer

$ complete -C '/usr/bin/aws_completer' aws

For future shell sessions, consider add this to your ~/.bashrc

$ echo "complete -C '/usr/bin/aws_completer' aws" >> ~/.bashrc

To check, type:

$ aws ec

Press the [TAB] key, it should add 2 automatically:

$ aws ec2

Creating a New Profile#

To setup a new credential profile with the name myprofile :

$ aws configure --profile myprofile
AWS Access Key ID [None]: ACCESSKEY
AWS Secret Access Key [None]: SECRETKEY
Default region name [None]: REGIONNAME
Default output format [None]: text | table | json

For the AWS access key id and secret, create an IAM user in the AWS console and generate keys for it.

Region will be the default region for commands in the format eu-west-1 or us-east-1 .

The default output format can either be text , table or json .

You can now use the profile name in other commands by using the --profile option, e.g.:

$ aws ec2 describe-instances --profile myprofile

AWS libraries for other languages (e.g. aws-sdk for Ruby or boto3 for Python) have options to use the profile you create with this method too. E.g. creating a new session in boto3 can be done like this, boto3.Session(profile_name:'myprofile') and it will use the credentials you created for the profile.

The details of your aws-cli configuration can be found in ~/.aws/config and ~/.aws/credentials (on linux and mac-os). These details can be edited manually from there.

Installation and setup#

There are a number of different ways to install the AWS CLI on your machine, depending on what operating system and environment you are using:

On Microsoft Windows – use the MSI installer. On Linux, OS X, or Unix – use pip (a package manager for Python software) or install manually with the bundled installer.

Install using pip:

You will need python to be installed (version 2, 2.6.5+,3 or 3.3+). Check with

python --version

pip --help

Given that both of these are installed, use the following command to install the aws cli.

sudo pip install awscli

Install on Windows The AWS CLI is supported on Microsoft Windows XP or later. For Windows users, the MSI installation package offers a familiar and convenient way to install the AWS CLI without installing any other prerequisites. Windows users should use the MSI installer unless they are already using pip for package management.

Run the downloaded MSI installer. Follow the instructions that appear.

To install the AWS CLI using the bundled installer

Prerequisites:

  • Linux, OS X, or Unix
  • Python 2 version 2.6.5+ or Python 3 version 3.3+
  1. Download the AWS CLI Bundled Installer using wget or curl.
  2. Unzip the package.
  3. Run the install executable.

On Linux and OS X, here are the three commands that correspond to each step:

$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
$ unzip awscli-bundle.zip
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Install using HomeBrew on OS X:

Another option for OS X

brew install awscli

Test the AWS CLI Installation

Confirm that the CLI is installed correctly by viewing the help file. Open a terminal, shell or command prompt, enter aws help and press Enter:

$ aws help

Configuring the AWS CLI

Once you have finished the installation you need to configure it. You’ll need your access key and secret key that you get when you create your account on aws. You can also specify a default region name and a default output type (text|table|json).

$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: ENTER

Updating the CLI tool

Amazon periodically releases new versions of the AWS Tool. If the tool was installed using the Python Pip tool the following command will check the remote repository for updates, and apply it to your local system.

$ pip install awscli --upgrade

List S3 buckets#

aws s3 ls

Use a named profile

aws --profile myprofile s3 ls

List all objects in a bucket, including objects in folders, with size in human-readable format and a summary of the buckets properties in the end –

aws s3 ls --recursive --summarize --human-readable s3://<bucket_name>/

Using aws cli commands#

The syntax for using the aws cli is as follows:

aws [options] <command> <subcommand> [parameters]

Some examples using the ‘ec2’ command and the ‘describe-instances’ subcommand:

aws ec2 describe-instances

aws ec2 describe-instances --instance-ids <your-id>

Example with a fake id:

aws ec2 describe-instances --instance-ids i-c71r246a