June 2020
M T W T F S S
« Mar    
1234567
891011121314
15161718192021
22232425262728
2930  

Categories

WordPress Quotes

Be Content with what you have; rejoice in the way things are. When you realize there is nothing lacking, the whole world belongs to you.
Lao Tzu
June 2020
M T W T F S S
« Mar    
1234567
891011121314
15161718192021
22232425262728
2930  

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (270)
centos8 (3)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
EXCHANGE (3)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (2)
horoscope (23)
Hyper-V (10)
IIS (15)
IPTABLES (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
SCALEIO (1)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Ubuntu (1)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)
ZIMBRA (17)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

0 visitors online now
0 guests, 0 bots, 0 members

Hit Counter provided by dental implants orange county

10 Apache Security and Hardening Tips

10 Apache Security and Hardening Tips

Tip No. 1: Disable Apache Signature and/or Apache Banner

# ServerSignature Off
# ServerTokens ProductOnl

Tip No. 2: The Trace HTTP Request

Add the following to the web-server’s configuration file. For example alter the following file in Ubuntu: /etc/apache2/apache2.conf .

* TraceEnable off

Tip 3: Remove PHP scripts that print debug info using phpinfo()

The built-in PHP function phpinfo() prints a lot of interesting internal information about the PHP environment.
It can include list of which PHP modules are enabled, and the location of various files on the web-server and other sensitive information.
Our web security scanner finds a lot of such files. It is recommended to remove these test files from a production website.

Here is a tip hpw to find such files. Look for the files with the following name: test.php, info.php, i.php and phpinfo.php in your website directory and remove them.

Tip 4: Disable directory indexing

Directory indexing is a features found in every web-server by default. When directory indexing is enabled, the web-site prints a list of files found in the website directories
when the default page does not exists (for example index.php). Directories reported can be viewed by any visitor.
It is vulnerable in the sense that these directories can contain configuration, private and backup files which can be used by the attackers
to take your server under control.

You can fix this problem by disabling the Apache autoindex module.
In some Apache installations it is called mod_autoindex.so. In Ubuntu, you just need to remove the following files:

* /etc/apache2/mods-enabled/autoindex.load
* /etc/apache2/mods-enabled/autoindex.conf

So you can do it running the following commands:

* rm -f /etc/apache2/mods-enabled/autoindex.load
* rm -f /etc/apache2/mods-enabled/autoindex.conf

Tip 5: Disable WebDAV

ake sure that WebDAV is disabled in production websites. When WebDAV is enabled, the following commands are supported by Apache: OPTIONS, PROPFIND, etc.
These commands are sensitive from computer security point of view.

* /etc/apache2/mods-enabled/dav.load
* /etc/apache2/mods-enabled/dav_fs.conf
* /etc/apache2/mods-enabled/dav_fs.load
* /etc/apache2/mods-enabled/dav_lock.load

Tip 6: Create a chroot’ed Apache environment

Tip 7: Enable PHP basedir

Tip 8: Web Stats

Tip 9: Use Google

Most of the webmasters use common web scripts and CMS or blog software. We recommend you to frequently search for security updates using Google and register for security news at your blog/CMS website.

Tip 10: Additional Steps

If your webserver runs together with MySQL server it brings additional potential security problem. MySQL can read any files located on you server including the one located in different chrooted environments. It happens because of the FILE permission. By default only MySQL root has it.
For more info about MySQL security take a look at this article ( link to GreenSQL) .

Fedora

Building High Performance webserver On Centos in Dell Servers

Performance tuning a CentOS LAMP web server for high traffic volumes

This document is prepared and Posted on August 17, 2010 by William Jamieson – Thank you very much William 🙂

Performance tune a LAMP server to handle approximately 70 full page loads per second which equated to 4,250 concurrent virtual users. We ended up doubling this expectation to 140 full page loads per second without striking issue. If this speed was maintained for 24 hours it would equate to over 12 million hits per day. This article will let you know how we achieved it.

The load tests were conducted using the HP performance center; a technology that HP obtained as part of its acquisition of Mercury for approximately USD$4.5 billion in 2006.

To find out more about the load testing software visit http://en.wikipedia.org/wiki/HP_LoadRunner

Goal:
Handle 4,250 concurrent users generating approximately 70 full page loads per second.

1 full page load consisted of:
– 1 dynamically generated PHP file using MySQL
– 4 JavaScript files
– 7 CSS files
– 8 image files

Original starting environment:
– ServerModel: Dell R300
– RAM: 2GB (2 x 1GB chips)
– Operating System: CentOS release 5.5 (Final)
– Apache: v2.2.3 (running in prefork mode)
– MySQL: v5.0.77
– PHP: v5.1.6 (as an apache module)
– eAccelerator: v0.9.5.3
– 120Mbits of bandwidth

Round 1: Initial Test
Round 1: Configuration

At the start of the process we were pretty much using the default configurations for the entire lamp stack. Linux was running iptables and ip6tables in its default configuration. eAccelerator was operating with 32MB of memory with optimization and caching enabled.

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
max_connections = 100
max_user_connections = 0
max_connect_errors = 10
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 2M
read_buffer_size = 131072
read_rnd_buffer_size = 262144
myisam_sort_buffer_size = 8M
thread_cache_size = 0
query_cache_size= 0
thread_concurrency = 10
Round 1: Results

With these settings we got up to 30 page loads per second which was 42% of our target. Interestingly, we were only operating at about 8% CPU and about 50% of our memory capacity when we hit this limit.
Round 1: Review

Looking at the apache error logs we were getting a large number of MySQL errors:
mysql_connect() [function.mysql-connect]: Too many connections in xxx.php on line 15

So the MySQL configuration seemed to be our bottleneck:

Round 2
Round 2: Configuration

We did our first major review of the Apache and MySQL performance settings and adjusted them accordingly. We doubled the Apache settings and used the ‘huge’ configuration as supplied with mysql (/usr/share/doc/mysql-server-5.0.77/my-huge.cnf).

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 16
MinSpareServers 10
MaxSpareServers 40
ServerLimit 512
MaxClients 512
MaxRequestsPerChild 8000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
# Memory usage
skip-locking
max_connections = 500
max_user_connections = 500
max_connect_errors = 999999
key_buffer = 384M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
# Try number of CPU’s*2 for thread_concurrency (eHound has 4 CPU’s)
thread_concurrency = 8

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

As an extra precaution we locked the network card in the server to use 1Gbit:
#ethtool -s eth0 speed 1000 duplex full

Edit the configuration for the network card:
#vim /etc/sysconfig/network-scripts/ifcfg-eth0

Add the following line:
ETHTOOL_OPTS=’autoneg on speed 1000 duplex full’

Restart the network:
#service network restart
Round 2: Results

With these settings we got up to 58 full page loads per second which was 59% of our target. Interestingly, we were still only operating at about 10% CPU capacity when we hit this limit but we were using approximately 70-80% of our memory.

Our MySQL errors had disappeared and there were no more errors in the Apache logs.
Round 2: Review

We were concerned that the system was starting to use swap memory which was slowing the server to a halt.

Round 3
Round 3: Configuration

We added an additional 2GB of RAM to the server so it now contained 4 x 1GB chips.
Round 3: Results

With the new RAM we still only got up to 58 full page loads per second which was 59% of our target. We were still only operating at about 10% CPU capacity but now we were only using about 40% of our memory.
Round 3: Review

Still no errors in the Apache logs and the load test farm was not receiving Apache errors. In fact it was reporting that it could not even connect to the server. This led us to believe that it was either a lack of bandwidth or a NIC/network/firewall configuration issue. After checking with our datacenter, we found that there were no inhibiting factors that would cause the problem described.

We increased the Apache & MySQL Limits and ran a different style of test.

Round 4
Round 4: Configuration

In this test we only loaded the dynamic components of the page as generated by PHP and MySQL and served by Apache. This meant that we told the load test farm not to download static content such as images, CSS or JavaScript files.

Also we increased the MySQL and Apache limits as follows:

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 280
MinSpareServers 100
MaxSpareServers 300
ServerLimit 1536
MaxClients 1536
MaxRequestsPerChild 32000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
# Memory usage
skip-locking
max_connections = 764
max_user_connections = 764
max_connect_errors = 999999
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU’s*2 for thread_concurrency (eHound has 4 CPU’s)
thread_concurrency = 8

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout
Round 4: Results

The results of this test were very interesting. We got up to 263 page loads without any issue. This consumed a lot more bandwidth than test 3 so we knew that bandwidth was not the issue. However the number of connections that both tests started to fail at were very similar.
Round 4: Review

So we knew we had a connection limit issue.

We also knew that the eAccelerator optcode cache was not dying at these high volumes, nor was MySQL, PHP or Apache.

We reviewing the kernel messages and found thousands of the following messages that were logged at the time of testing:
#cat /var/log/messages* | grep ‘Aug 15’

Aug 15 01:04:27 localhost kernel: printk: 1395 messages suppressed.
Aug 15 01:04:27 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:32 localhost kernel: printk: 1561 messages suppressed.
Aug 15 01:04:32 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:37 localhost kernel: printk: 1274 messages suppressed.
Aug 15 01:04:37 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:42 localhost kernel: printk: 1412 messages suppressed.

Further investigation revealed that the iptables/ip6tables was activated and limiting the number of connections to the box because its table was full. Ordinarily when I set up a linux server I turn iptables off because I place hardware firewalls in front of the servers. However I didn’t have the opportunity to setup this box initially, so they were still activated. I however didn’t need them, so I deactivated them.

If you still need to keep iptables running you can simply adjust the following settings:
Check the current connections limit (only works if iptables is running):
#sysctl net.ipv4.netfilter.ip_conntrack_max
65536

Change the connections limit:
#vim /etc/sysctl.conf

Add the following lines:
# conntrack limits
#inet.ipv4.netfilter.ip_conntrack_max = 65536
net.ipv4.netfilter.ip_conntrack_max = 196608

Reload the config file:
#sysctl -p

Check the new connections limit:
#sysctl net.ipv4.netfilter.ip_conntrack_max
196608

Check the current buckets limit (only works if iptables is running):
#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
8192

To change the buckets limit:
#vim /etc/modprobe.conf

Add the following lines:
options ip_conntrack hashsize=32768

Reboot the server:
#shutdown -r now

Check the new buckets limit:
#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
24576


Alternatively if you don’t need iptables like me, you can just disable them:
#service iptables stop
#service ip6tables stop
#chkconfig iptables off
#chkconfig ip6tables off

Round 5
Round 5: Configuration

This test used exactly the same configuration with iptables disabled.
Round 5: Results

Success!!! We got to 4,250 concurrent users which is about 70 pages per second (loading all additional image, CSS and JavaScript files also) with zero errors and a 0.7 second average response time. This used about 120Mbits worth of bandwidth pipe. The datacenter ended up running out of pipe before the server had any issues.

At this rate we were running at about:
– 15% CPU utilisation
– 30% Memory usage (with 4GB RAM installed)
– 400 apache threads
– 100% Bandwidth
Round 5: Review

Key findings:
– Increase your Apache and MySQL limits
– Turn off iptables
– Ensure that you have enough RAM
– Ensure that you are checking logs from MySQL, Apache, and the kernel to pick up any errors and give you clues as to how to best solve them

Round 6
Round 6: Configuration

This test used exactly the same configuration as round 5 with 250Mbit pipe instead of a 120Mbit pipe.
Round 6: Results

Success!!! We got to 140 full page loads per second (including additional images, CSS and JavaScript files also) with zero errors and still a stable 0.7 second average response time. This used the full 250Mbits worth of bandwidth pipe. The datacenter ended up running out of pipe again before the server had any issues.

At this rate we were running at about:
– 30% CPU utilisation
– 40% Memory usage (with 4GB RAM installed)
– 800 apache threads
– 100% Bandwidth
Round 6: Review

Key findings:
– Even with 250Mbits of pipe, bandwidth is still the bottleneck in this configuration.

Round 7
Round 7: Configuration

Even though our server was performing fine, we were given another server to experiment on with much higher specs.

It was a Dell R710 with 48GB of RAM and 8 2.53MHz Xeon processors running in hyper-threading mode (essentially making 16 processors).

We also had this box connected to a dedicated 4Gbit optical internet feed to give it as much bandwidth as it needed.

Everything on the box was configured the same except for Apache and MySQL (which we took the last settings and multipled them by 4) and sysctl.

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 1120
MinSpareServers 400
MaxSpareServers 1200
ServerLimit 6144
MaxClients 6144
MaxRequestsPerChild 128000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
# Memory usage
skip-locking
max_connections = 3056
max_user_connections = 3056
max_connect_errors = 999999
key_buffer = 1024M
max_allowed_packet = 4M
table_cache = 1024
sort_buffer_size = 4M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 256M
thread_cache_size = 32
query_cache_size= 64M
# Try number of CPU’s*2 for thread_concurrency (eHound has 4 CPU’s)
thread_concurrency = 32

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 64M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[myisamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[mysqlhotcopy]
interactive-timeout

We also added the following lines to sysctl:
ip_conntrack_max = 196608
net.ipv4.ip_local_port_range = 1025 65535
net.ipv4.tcp_max_tw_buckets = 1000000
net.core.somaxconn = 10000
net.ipv4.tcp_max_syn_backlog = 2000
net.ipv4.tcp_fin_timeout = 30
Round 7: Results

We got to 200 full page loads per second (including additional images, CSS and JavaScript files also) with zero errors and still a stable 0.8 second average response time. This test used 330Mbits or about 8% worth of the bandwidth available. We stopped the test simply because we didn’t need to go any higher, but potentially could have gone much higher.

At this rate we were running at about:
– 16% CPU utilisation
– 6% Memory usage (with 48GB RAM installed)
– 1227 apache threads
– 8% Bandwidth
Round 7: Review

Key findings:
– Bandwidth seem to be a much bigger bottleneck than server capability.

swap issues on Linux and clear the swap usage

swap issues on Linux

clear the swap usage

free -to (Total memory usage)

free -m (Memory usage of swap)

swapoff -a && swapon -a ( swap off and on)

free

cat /proc/swaps
sync; echo 3 > /proc/sys/vm/drop_caches

To free pagecache:
# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:
# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:
echo 3 > /proc/sys/vm/drop_caches

Note: Works well on Production Servers

PCI Compliance Disable ETags Apache

PCI Compliance Disable ETags

PCI Compliance
Disable ETags

To alleviate security risks arising from disclosure of information about files and their properties by Apache Web server, disable FileETag directive. For PCI Compliance it is required to disable ETags

Create a file at /etc/httpd/conf.d/no-etags.conf with the following:

Header unset ETag
FileETag None

Then of course restart Apache.

http://httpd.apache.org/docs/2.2/mod/core.html#FileETag

Setup Caching on Apache

Setup Caching on Apache

Setup Caching

Please note that caching will only work for non-secure data. It is not possible to cache data from a HTTPS url.
To configure caching, we’ll first have to enable it in apache:

Start yast
Go to network services
Select the “HTTP Server”
Go to “Server Modules”:
Enable these modules
cache
diskcache

Save the changes.

Note: If you forget to enable the cache module you’ll get this warning:

sjoerd@reverseproxy:/etc/apache2/vhosts.d> sudo /etc/init.d/apache2 restart
httpd2-prefork: Syntax error on line 116 of /etc/apache2/httpd.conf: Syntax error on line 26 of /etc/apache2/sysconfig.d/loadmodule.conf: Cannot load /usr/lib64/apache2-prefork/mod_disk_cache.so into server

Second disk

Add a second disk to the VM and configure it to mount on /var/cache/apache, the default location for apache cache.

Disk size: 8 GB
file system ext3, no access time
Mountpoint: /var/cache/apache

reverseproxy:~ # mount

/dev/sdb1 on /var/cache/apache type ext3 (rw,noatime,acl,user_xattr)

And set the owner:

sudo chown -R wwwrun:root /var/cache/apache

Apache Cache

Set the configuration below inside the vhost config file.

# Caching
CacheRoot /var/cache/apache
CacheEnable disk /
CacheDirLevels 1
CacheDirLength 1
CacheDefaultExpire 7200
CacheMaxExpire 86400
CacheIgnoreNoLastMod On
CacheMaxFileSize 2048000
CacheStorePrivate On

Apache Cache Resources
http://httpd.apache.org/docs/2.2/caching.html
http://httpd.apache.org/docs/2.2/mod/mod_disk_cache.html
http://www.mnot.net/cache_docs/
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
Switch From https To http

This is not really possible, if you need “ProxyPreserveHost on” in Apache. Our application needs that to work through a reverse proxy, setting it to off breaks it. We wanted to configure the reverse proxy from https on the outside, to http on the inside but that seems impossible. It is either http to http, or https to https. I tested both, and they work, but unfortunely switching from https on the outside to http on the inside does not. I experimented with rewriterules, requestheader, and a couple of settings more, no luck.
Mod Security

We want to offload the application webserver as much as possible which means we’ll also implement mod_security on the reverse proxy. This will offload and simplify the application webserver.

Mod Security 2.x has these requirements:

Apache 2.2.x (highly recommended)
Apache module mod_unique_id
libapr & libapr-util
libpcre
libxml2

All modules are already installed by default. Note that libpcre is known as ‘pcre’ on SLES.

You just have to enable the module mod_unique_id as it is not enabled by default.
Restart to make your changes effective and run httpd2 -M to see if all modules are loaded.
If everything is loaded stop apache.
Mod Security Installation

Make sure you have access to the SLES SDK Sources. Since we have an SLES Installation Update Server 11 I could download the SDK ISO DVD1 (which holds all required files) and add it to my software repository.

The ISO can be downloaded from here: SLES 11 SP1 SDK Download (A Novell account is required). The file you need to download is called: SLE-11-SP1-SDK-DVD-x86_64-GM-DVD1.iso

After installing, it’s mostly just a module but not entirely. You need to to enable two modules now, the mod_unique_id which is a normal module and can be enabled the normal way:

yast2 ? network services ? http server ? server modules
select the module and enable it

The second module to enable is mod_security. Since it is not recognized by apache as a module we have to manually add the module to the modulelist.
Find the APACHE_MODULES in the apache2 sysconfig file and add the module like below:

reverseproxy:/var/log/apache2 # vi /etc/sysconfig/apache2
APACHE_MODULES=”authz_host actions alias auth_basic authz_groupfile authn_file authz_user autoindex cgi dir include log_config mime negotiation setenvif status userdir asis cache disk_cache imagemap proxy

Restart apache and check wether the modules are running by issuing the ‘httpd2 -M’ command:

reverseproxy:/var/log/apache2 # httpd2 -M
Loaded Modules:
core_module (static)
mpm_prefork_module (static)
http_module (static)
so_module (static)
authz_host_module (shared)
actions_module (shared)
alias_module (shared)
auth_basic_module (shared)
authz_groupfile_module (shared)
authn_file_module (shared)
authz_user_module (shared)
autoindex_module (shared)
cgi_module (shared)
dir_module (shared)
include_module (shared)
log_config_module (shared)
mime_module (shared)
negotiation_module (shared)
setenvif_module (shared)
status_module (shared)
userdir_module (shared)
asis_module (shared)
cache_module (shared)
disk_cache_module (shared)
imagemap_module (shared)
proxy_module (shared)
proxy_connect_module (shared)
proxy_http_module (shared)
rewrite_module (shared)
ssl_module (shared)
unique_id_module (shared)
authz_default_module (shared)
security2_module (shared)
Syntax OK

Mod Security

Mod Security

Mod security has a default configuration file, and comes with a core rule set. The configuration works with include files which work for the modsecurity part like this:

httpd.conf
|
|– default-server.conf . . . . . . . . . set up the default server that replies to non-virtual-host requests
| `–conf.d/mod_security2.conf . . . . enable mod-security default configuration
|
`–conf.d/modsecurity/*.conf . . . . . . add the core rule set

Since this include structure is not enabled by default (because the core rule set is not enabled by default) we have to include the core rule set manually.

Create the correct directories and copy the core rule set config files to this directory:

reverseproxy:/usr/share/doc/packages/apache2-mod_security2/rules # mkdir /etc/apache2/conf.d/modsecurity
reverseproxy:/usr/share/doc/packages/apache2-mod_security2/rules # cp *.conf /etc/apache2/conf.d/modsecurity
reverseproxy:/usr/share/doc/packages/apache2-mod_security2/rules # cd /etc/apache2/conf.d/modsecurity

reverseproxy:/etc/apache2/conf.d/modsecurity # ll
-rw-r–r– 1 root root 12325 Jan 31 14:03 modsecurity_crs_10_config.conf
-rw-r–r– 1 root root 5164 Jan 31 14:03 modsecurity_crs_20_protocol_violations.conf
-rw-r–r– 1 root root 3538 Jan 31 14:03 modsecurity_crs_21_protocol_anomalies.conf
-rw-r–r– 1 root root 2496 Jan 31 14:03 modsecurity_crs_23_request_limits.conf
-rw-r–r– 1 root root 6399 Jan 31 14:03 modsecurity_crs_30_http_policy.conf
-rw-r–r– 1 root root 2720 Jan 31 14:03 modsecurity_crs_35_bad_robots.conf
-rw-r–r– 1 root root 28726 Jan 31 14:03 modsecurity_crs_40_generic_attacks.conf
-rw-r–r– 1 root root 2463 Jan 31 14:03 modsecurity_crs_45_trojans.conf
-rw-r–r– 1 root root 8268 Jan 31 14:03 modsecurity_crs_50_outbound.conf

Add the include line for the core rule set in the httpd.conf:

# Include Mod Security Core Rule Set
Include /etc/apache2/conf.d/modsecurity/*.conf

Now we will configure the config files themselves to run modsecurity first in DetectionOnly
mode to prevent the risk for false positives. We also set the logfiles correctly:

vi /etc/apache2/conf.d/mod_security2.conf:
# Basic configuration options
#SecRuleEngine On
SecRuleEngine DetectionOnly

vi /etc/apache2/conf.d/modsecurity/modsecurity_crs_10_config.conf:
SecRuleEngine DetectionOnly
SecAuditLog /var/log/apache2/modsec_audit.log
SecDebugLog /var/log/apache2/modsec_debug.log
SecDebugLogLevel 3

Now restart apache:

reverseproxy:/var/log/apache2 # /etc/init.d/apache2 start
Starting httpd2 (prefork) [Mon Jan 31 14:30:35 2011] [warn] worker http://10.10.12.20/start already used by another worker
[Mon Jan 31 14:30:35 2011] [warn] worker http://10.10.12.20/start already used by another worker

Documentation Core Rule Set

Core Rule Set Structure & Usage
====================================

To activate the rules for your web server installation:

1) You may want to edit and customize modsecurity_crs_10_config.conf.
Additionally you may want to edit modsecurity_crs_30_http_policy.conf
which enforces an application specific HTTP protocol usage.

2) Add the following line to your httpd.conf (assuming
you’ve placed the rule files into conf/modsecurity/):

Include conf/modsecurity/*.conf

3) Restart web server.

4) Make sure your web sites are still running fine.

Core Rule Set Content
=========================

In order to provide generic web applications protection, the Core Rule Set
uses the following techniques:

1. HTTP protection – detecting violations of the HTTP protocol and a locally
defined usage policy.

2. Common Web Attacks Protection – detecting common web application security
attack.

3. Automation detection – Detecting bots, crawlers, scanners and other surface
malicious activity.

4. Trojan Protection – Detecting access to Trojans horses.

5. Errors Hiding – Disguising error messages sent by the server

In addition the rule set also hints at the power of ModSecurity beyond
providing security by reporting access from the major search engines to your
site.

HTTP Protection – This first line of protection ensures that all abnormal HTTP
requests are detected. This line of defense eliminates a large number of
automated and non targeted attacks as well as protects the web server itself.
Common Web Attacks Protection Rules on the second level address the common web
application security attack methods. These are the issues that can appear in
any web application. Some of the issues addressed are:

– SQL Injection
– Cross-Site Scripting (XSS)
– OS Command execution
– Remote code inclusion
– LDAP Injection
– SSI Injection
– Information leak
– Buffer overflows
– File disclosure

Automation Detection – Automated clients are both a security risk and a
commercial risk. Automated crawlers collect information from your site, consume
bandwidth and might also search for vulnerabilities on the web site. Automation
detection is especially useful for generic detection of comments spam.

Trojan Protection – ModSecurity Core Rule Set detects access to back doors
installed on a web server. This feature is very important in a hosting
environment when some of this backdoors may be uploaded in a legitimate way and
used maliciously. In addition the Core Rule Set includes a hook for adding
an Anti-Virus program such as ClamAV for checking file uploads.

Errors Hiding – If all fails, the Core Rule Set will detect errors sent by
the web server. Detecting and blocking errors prevents attackers from
collecting reconnaissance information about the web application and also server
as a last line of defense in case an attack was not detected eariler.

Few Word of Caution
——————-

As with every new technology, using the ModSecurity Core Rule Set requires some caution:

– Every Rule Set can have false positive in new environments and any new
installation should initially use the log only Rule Set version or if no such
version is available, set ModSecurity to Detection only using the SecRuleEngine
DetectionOnly command.

After running ModSecurity in a detection only mode for a while review the evens
generated and decide if any modification to the rule set should be made before
moving to protection mode.

From the mod security manual:

SecRuleEngine

Description: Configures the rules engine.
Syntax: SecRuleEngine On|Off|DetectionOnly
Example Usage: SecRuleEngine On
Processing Phase: Any
Scope: Any
Version: 2.0.0
Dependencies/Notes: This directive can also be controlled by the ctl action (ctl:ruleEngine=off) for per rule processing.
Possible values are:
* On – process rules.
* Off – do not process rules.
* DetectionOnly – process rules but never intercept transactions, even when rules are configured to do so.

Mod Security Handling False Positives
Mod security is now configured as detection only. For now, we keep it like this, closely monitoring the mod security logfiles for false positives. When we are sure there are no more false positives (or at least nothing our customers will notice) we can set the SecRuleEngine to On.

This blog also explains how to deal with false positives: Handling False Positives
Mod Security Troubleshooting

Starting httpd2 (prefork) [Mon Jan 31 14:20:51 2011] [warn] worker http://10.10.12.20/start already used by another worker
[Mon Jan 31 14:20:51 2011] [warn] worker http://10.10.12.20/start already used by another worker
Syntax error on line 53 of /etc/apache2/conf.d/modsecurity/modsecurity_crs_10_config.conf:
Invalid command ‘SecRuleEngine’, perhaps misspelled or defined by a module not included in the server configuration

The command line was:
/usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -DSSL

‘Solution:’ The module mod_security is not enabled. Check for the module with the command ‘httpd2 -M’. If the module is really not there, add the module in /etc/sysconfig/apache2.

reverseproxy:/var/log/apache2 # /etc/init.d/apache2 restart
[Mon Jan 31 14:29:23 2011] [warn] worker http://10.10.12.20/start already used by another worker
[Mon Jan 31 14:29:23 2011] [warn] worker http://10.10.12.20/start already used by another worker
Syntax error on line 191 of /etc/apache2/conf.d/modsecurity/modsecurity_crs_10_config.conf:
ModSecurity: Failed to open the audit log file: /srv/www/logs/modsec_audit.log

‘Solution:’ The directory specified for the logs does not exist. Create the directory with this command:

reverseproxy:/var/log/apache2 # mkdir -p /srv/www/logs/

or change the location to /var/log/apache2. Of course, the same message can be displayed for /srv/www/logs/modsec_debug.log.
Testing Mod Security
You can test if mod security is running correctly by going to the index file of your website by ip-address and adding ‘?file=/etc/passwd’ to the url:

https://10.10.10.20/start/index.html?file=/etc/passwd

This will be noticed, and displayed in the log (not stopped, remember, we’re running in DetectionOnly mode):

less modsec_debug.log

[31/Jan/2011:15:46:31 +0100] [10.10.10.20/sid#7f0c98cffdc8][rid#7f0c98feb488][/start/0100_NavigationPublic.html][2] Warning. Pattern match “^[\d\.]+$” at REQUEST_HEADERS:Host. [
file “/etc/apache2/conf.d/modsecurity/modsecurity_crs_21_protocol_anomalies.conf”] [line “60”] [id “960017”] [msg “Host header is a numeric IP address”] [severity “CRITICAL”] [ta
g “PROTOCOL_VIOLATION/IP_HOST”]
[31/Jan/2011:15:46:42 +0100] [10.10.10.20/sid#7f0c98cffdc8][rid#7f0c98fe2908][/start/index.html][2] Warning. Pattern match “^[\d\.]+$” at REQUEST_HEADERS:Host. [file “/etc/apach
e2/conf.d/modsecurity/modsecurity_crs_21_protocol_anomalies.conf”] [line “60”] [id “960017”] [msg “Host header is a numeric IP address”] [severity “CRITICAL”] [tag “PROTOCOL_VIOL
ATION/IP_HOST”]
[31/Jan/2011:15:46:42 +0100] [10.10.10.20/sid#7f0c98cffdc8][rid#7f0c98fe2908][/start/index.html][2] Warning. Pattern match “(?:\b(?:\.(?:ht(?:access|passwd|group)|www_?acl)|glob
al\.asa|httpd\.conf|boot\.ini)\b|\/etc\/)” at ARGS:file. [file “/etc/apache2/conf.d/modsecurity/modsecurity_crs_40_generic_attacks.conf”] [line “114”] [id “950005”] [msg “Remote
File Access Attempt”] [data “/etc/”] [severity “CRITICAL”] [tag “WEB_ATTACK/FILE_INJECTION”]
[31/Jan/2011:15:46:42 +0100] [10.10.10.20/sid#7f0c98cffdc8][rid#7f0c98fe2908][/start/index.html][2] Warning. Pattern match “(?:\b(?:(?:n(?:et(?:\b\W+?\blocalgroup|\.exe)|(?:map|
c)\.exe)|t(?:racer(?:oute|t)|elnet\.exe|clsh8?|ftp)|(?:w(?:guest|sh)|rcmd|ftp)\.exe|echo\b\W*?\by+)\b|c(?:md(?:(?:32)?\.exe\b|\b\W*?\/c)|d(?:\b\W*?[\\/]|\W*?\.\.)|hmod.{0,40}?\+.
{0,3}x))|[\;\|\`]\W*? …” at ARGS:file. [file “/etc/apache2/conf.d/modsecurity/modsecurity_crs_40_generic_attacks.conf”] [line “133”] [id “950006”] [msg “System Command Injectio
n”] [data “/passwd”] [severity “CRITICAL”] [tag “WEB_ATTACK/COMMAND_INJECTION”]

less modsec_audit.log:

Message: Warning. Pattern match “^[\d\.]+$” at REQUEST_HEADERS:Host. [file “/etc/apache2/conf.d/modsecurity/modsecurity_crs_21_protocol_anomalies.conf”] [line “60”] [id “960017”]
[msg “Host header is a numeric IP address”] [severity “CRITICAL”] [tag “PROTOCOL_VIOLATION/IP_HOST”]
Apache-Handler: proxy-server
Stopwatch: 1296487473036980 19376 (997 2882 -)
Producer: ModSecurity for Apache/2.5.6 (http://www.modsecurity.org/); core ruleset/1.6.1.
Server: Apache/2.2.10 (Linux/SUSE)

Mod Security Resources

http://www.modsecurity.org/
http://www.modsecurity.org/documentation/faq.html
http://www.modsecurity.org/documentation/modsecurity-apache/2.5.12/html-multipage/introduction.html
Install Modsecurity
Install core rule set

 

MY SET OF RULES TO DEFEND THE WEB SERVER
SecFilterEngine On

# Make sure that URL encoding is valid
SecFilterCheckURLEncoding On

# Unicode encoding check
SecFilterCheckUnicodeEncoding On

# Only allow bytes from this range
SecFilterForceByteRange 0 255

# Only log actionable requests
SecAuditEngine RelevantOnly

# The name of the audit log file
SecAuditLog /var/log/apache2/audit_log

# Debug level set to a minimum
SecFilterDebugLog /var/log/apache2/modsec_debug_log
SecFilterDebugLevel 2

# Should mod_security inspect POST payloads
SecFilterScanPOST On

# By default log and deny suspicious requests
# with HTTP status 500
SecFilterDefaultAction “deny,log,status:500”

# Add custom secfilter rules here

Apache troubleshooting commands

Apache troubleshooting commands

Commands

ps aux | grep httpd
pstree -p | grep httpd

strace -f -o trace.txt /etc/rc.d/init.d/httpd start

Sometime Apache process, keeps on execution (Seems like Hangs), so generally trying to get the exact PHP file that is running by Apache Process, So here is my Try.

I used Strace to get the opened files by the apache process. (Get PID of
Apache process that is taking time, though you can also get it From top command)

# pstree -p -n | grep http
(This will show each files that is being processed by that Apache Proc)

# strace -p
The list of files could also be get using lsof, but that could not be of full use, as you need the files continuus

Counting Hits from Web Server Access log
# awk ‘{print $1}’ /opt/rmohan.com/access_log | grep -vE ‘^:|^common|^-‘ | sort | uniq -c | sort -nr > /var/www/reports/ips/rmohan.txt

or

# awk ‘$1>10000 {print $1}’ /opt/rmohan.com/access_log | uniq -c | sort -nr > /var/www/reports/ips/rmohan.txt

Restart Apache and check whether the modules are running by issuing the ‘httpd2 -M’ command:

reverseproxy:/var/log/apache2 # httpd2 -M
Loaded Modules:
core_module (static)
mpm_prefork_module (static)
http_module (static)
so_module (static)
authz_host_module (shared)
actions_module (shared)
alias_module (shared)
auth_basic_module (shared)
authz_groupfile_module (shared)
authn_file_module (shared)
authz_user_module (shared)
autoindex_module (shared)
cgi_module (shared)
dir_module (shared)
include_module (shared)
log_config_module (shared)
mime_module (shared)
negotiation_module (shared)
setenvif_module (shared)
status_module (shared)
userdir_module (shared)
asis_module (shared)
cache_module (shared)
disk_cache_module (shared)
imagemap_module (shared)
proxy_module (shared)
proxy_connect_module (shared)
proxy_http_module (shared)
rewrite_module (shared)
ssl_module (shared)
unique_id_module (shared)
authz_default_module (shared)
security2_module (shared)
Syntax OK

Monitor Your Website in Real-Time with Apachetop

As a webmaster, I’ve often wanted to be able to see real-time hits as they arrive. Sure, Google Analytics is a wonderful package for looking at trends over time, but there’s a delay of a few hours there, and you really can’t see data like requests per second or total bytes.

This is where the apachetop utility comes in. It’s a very simple command line utility that you can use to monitor traffic real-time. It accomplishes this by parsing the apache logfiles and displaying meaningful output to the screen.

Using Apachetop

Once you’ve installed the utility (instructions below), you can launch it by simply running apachetop from the command line. Since apachetop sometimes defaults to the wrong directory for the logfiles, you can pass in the -f parameter to specify the location of the logfile. This is also helpful when you have many virtual hosts on the same box.

apachetop -f /var/www/vhosts/howtogeek.com/statistics/logs/access_log

This is what you’ll see after a few requests have come in:

Monitoring Timeframe

The first thing to note is that the default time range for data shown is 30 seconds, so don’t expect the total counts to continue to climb forever. You can change this by passing in a few different arguments.

apachetop -H hits (Will display stats on the last x number of hits)

apachetop -T secs (Will display stats on the last x number of seconds)

I’ve been using a range of 5-10 minutes in my testing, and it really shows some useful feedback. There’s other options you can try out as well.

Filters

The next thing to note is that you can filter what gets shown in the view. To access the filters, use the f key, and you should see a small line pop up.

Hit the a key to add a filter and the line should switch. Now you can choose to filter by URL, referrer, or host.

I’m going to choose URL by hitting the u key. The filter dialog will show up near the bottom:

Since all of my articles are under the subdirectory /howto/, I’m going to enter that. Now apachetop will only show the hits relevant to hits to the articles, instead of every hit for every image.

Viewing Request Details

If you use the up/down keys, you’ll notice the cursor move up and down to allow you to select a request. (notice the * char)

If you hit the Right arrow key, you’ll be taken to the details page for that request. From here you can see the actual hosts hitting your site, as well as the referrers. I’m not going to show the hosts, since I don’t want to give out user’s IP address, but you can see the referrer here:

To go back to the list, just use the Left arrow key.

Switch Between Hosts, Referrers and URLs

If you use the d key, you can easily switch between the different views.

For instance, here I can see what traffic StumbleUpon is sending me, and then I can use the details view(right arrow) to see the exact articles that are getting hit from stumbleupon.

Help

At any point you can hit the ? or the h keys to take you to the help screen, which will give you a quick view of all the options.

I find the sort by very useful.

Installing on Ubuntu

sudo apt-get install apachetop

Installing from Source on CentOS

wget http://www.webta.org/apachetop/apachetop-0.12.6.tar.gz

yum install readline-devel

yum install ncurses-devel

tar xvzf apachetop-0.12.6.tar.gz

cd apachetop-0.12.6

./configure

make

The binary can be found in src/apachetop, and you can copy it anywhere you’d like.

Installing from Source on Ubuntu

wget http://www.webta.org/apachetop/apachetop-0.12.6.tar.gz

sudo apt-get install ncurses-dev

sudo apt-get install libreadline5-dev

tar xvzf apachetop-0.12.6.tar.gz

cd apachetop-0.12.6

./configure

make

Forward Proxy and reverse proxy

Forward Proxy and reverse proxy

Proxy server types and uses for HTTP Server (powered by Apache)

This topic provides information about proxy server types and uses.
Important: Information for this topic supports the latest PTF levels for HTTP Server for iSeries . It is recommended that you install the latest PTFs to upgrade to the latest level of the HTTP Server for iSeries. Some of the topics documented here are not available prior to this update. See http://www.ibm.com/servers/eserver/iseries/software/http/services/service.htm for more information.

Proxy servers receive requests intended for other servers and then act to fulfill, forward, redirect, or reject the requests. Exactly which service is carried out for a particular request is based on a number of factors which include: the proxy server’s capabilities, what is requested, information contained in the request, where the request came from, the intended destination, and in some cases, who sent the request.
The two most attractive reasons to use a proxy server are its ability to enhance network security and lessen network traffic. A proxy server enhances network security by providing controls for receiving and forwarding (or rejecting) requests between isolated networks, for example, forwarding requests across a firewall. A proxy server lessens network traffic by rejecting unwanted requests, forwarding requests to balance and optimize server workload, and fulfilling requests by serving data from cache rather than unnecessarily contacting the true destination server.
HTTP Server (powered by Apache) has proxy server capabilities built in. Activating these services is simply a matter of configuration. This topic explains three common proxy concepts: forward proxy, reverse proxy, and proxy chaining.
Parent topic: Concepts of functions of HTTP Server

Forward proxy

A forward proxy is the most common form of a proxy server and is generally used to pass requests from an isolated, private network to the Internet through a firewall. Using a forward proxy, requests from an isolated network, or intranet, can be rejected or allowed to pass through a firewall. Requests may also be fulfilled by serving from cache rather than passing through the Internet. This allows a level of network security and lessens network traffic.
A forward proxy server will first check to make sure a request is valid. If a request is not valid, or not allowed (blocked by the proxy), it will reject the request resulting in the client receiving an error or a redirect. If a request is valid, a forward proxy may check if the requested information is cached. If it is, the forward proxy serves the cached information. If it is not, the request is sent through a firewall to an actual content server which serves the information to the forward proxy. The proxy, in turn, relays this information to the client and may also cache it, for future requests.

Forward Proxy

The above image shows a forward proxy configuration. An intranet client initiates a request that is valid but is not cached on Server A (Proxy Server). The request is sent through the firewall to the Internet server, Server B (Content Server), which has the information the client is requesting. The information is sent back through the firewall where it is cached on Server A and served to the client. Future requests for the same information will be fulfilled by the cache, lessening network traffic (proxy caching is optional and not necessary for forward proxy to function on your HTTP Server).

For information on how to configure a forward proxy, see Set up forward proxy for HTTP Server (powered by Apache).


Reverse proxy

A reverse proxy is another common form of a proxy server and is generally used to pass requests from the Internet, through a firewall to isolated, private networks. It is used to prevent Internet clients from having direct, unmonitored access to sensitive data residing on content servers on an isolated network, or intranet. If caching is enabled, a reverse proxy can also lessen network traffic by serving cached information rather than passing all requests to actual content servers. Reverse proxy servers may also balance workload by spreading requests across a number of content servers. One advantage of using a reverse proxy is that Internet clients do not know their requests are being sent to and handled by a reverse proxy server. This allows a reverse proxy to redirect or reject requests without making Internet clients aware of the actual content server (or servers) on a protected network.

Reverse proxy

A reverse proxy server will first check to make sure a request is valid. If a request is not valid, or not allowed (blocked by the proxy), it will not continue to process the request resulting in the client receiving an error or a redirect. If a request is valid, a reverse proxy may check if the requested information is cached. If it is, the reverse proxy serves the cached information. If it is not, the reverse proxy will request the information from the content server and serve it to the requesting client. It also caches the information for future requests.

The above image shows a reverse proxy configuration. An Internet client initiates a request to Server A (Proxy Server) which, unknown to the client, is actually a reverse proxy server. The request is allowed to pass through the firewall and is valid but is not cached on Server A. The reverse proxy (Server A) requests the information from Server B (Content Server), which has the information the Internet client is requesting. The information is served to the reverse proxy, where it is cached, and relayed through the firewall to the client. Future requests for the same information will be fulfilled by the cache, lessening network traffic and load on the content server (proxy caching is optional and not necessary for proxy to function on your HTTP Server). In this example, all information originates from one content server (Server B).
For information on how to configure a reverse proxy, see Set up reverse proxy for HTTP Server (powered by Apache).

Proxy chaining

A proxy chain uses two or more proxy servers to assist in server and protocol performance and network security. Proxy chaining is not a type of proxy, but a use of reverse and forward proxy servers across multiple networks. In addition to the benefits to security and performance, proxy chaining allows requests from different protocols to be fulfilled in cases where, without chaining, such requests would not be possible or permitted. For example, a request using HTTP is sent to a server that can only handle FTP requests. In order for the request to be processed, it must pass through a server that can handle both protocols. This can be accomplished by making use of proxy chaining which allows the request to be passed from a server that is not able to fulfill such a request (perhaps due to security or networking issues, or its own limited capabilities) to a server that can fulfill such a request.
The first proxy server in a chain will check to make sure a request is valid. If a request is not valid, or not allowed (blocked by the proxy), it will reject the request resulting in the client receiving an error or a redirect. If a request is valid, the proxy may check if the requested information is cached and simply serve it from there. If the requested information is not in cache, the proxy will pass the request on to the next proxy server in the chain. This server also has the ability to fulfill, forward, redirect, or reject the request. If it acts to forward the request then it too passes the request on to yet another proxy server. This process is repeated until the request reaches the last proxy server in the chain. The last server in the chain is required to handle the request by contacting the content server, using whatever protocol is required, to obtain the information. The information is then relayed back through the chain until it reaches the requesting client.

Proxy chaining

The above image shows a proxy chaining configuration. The intranet client makes a request to Server C (Content Server FTP). Server A (Proxy Server HTTP) does not contain the requested information in cache, so the request is passed through the firewall to Server B (proxy server HTTP/FTP). Server B has both HTTP and FTP protocols and is able to change the HTTP request to an FTP request. Server C receives the FTP request and passes back the requested information to Server B. Server B, in turn, passes the fulfilled request back to the intranet client using the HTTP protocol. The request is sent through the firewall and Server A where the request is cached and given to the intranet client.

Apache as Forward Proxy:
An ordinary forward proxy is an intermediate server that sits between the client and the origin server. In order to get content from the origin server, the client sends a request to the proxy naming the origin server as the target and the proxy then requests the content from the origin server and returns it to the client. The client must be specially configured to use the forward proxy to access other sites.

A typical usage of a forward proxy is to provide Internet access to internal clients that are otherwise restricted by a firewall. The forward proxy can also use caching (mod_cache) to reduce network usage.

The forward proxy is activated using the ProxyRequests directive. Because forward proxies allow clients to access arbitrary sites through your server and to hide their true origin, it is essential that you secure your server so that only authorized clients can access the proxy before activating a forward proxy.

ProxyRequests On
ProxyVia On


Order deny,allow
Deny from all
Allow from 192.168.1

Apache as Reverse Proxy:
A reverse proxy (or gateway), by contrast, appears to the client just like an ordinary web server. No special configuration on the client is necessary. The client makes ordinary requests for content the reverse proxy then decides where to send those requests, and returns the content as if it was itself the origin.

A typical usage of a reverse proxy is to provide Internet users access to a server that is behind a firewall. Reverse proxies can also be used to balance load among several back-end servers, or to provide caching for a slower back-end server. In addition, reverse proxies can be used simply to bring several servers into the same URL space.

A reverse proxy is activated using the ProxyPass directive or the flag to the RewriteRule directive. It is not necessary to turn ProxyRequests on in order to configure a reverse proxy.

ProxyRequests Off


Order deny,allow
Allow from all

ProxyPass /foo http://foo.example.com/bar
ProxyPassReverse /foo http://foo.example.com/bar

Configuring Apache to be a forward proxy

This configuration makes Apache act as an HTTP proxy:


ProxyRequests On
ProxyVia On
#ProxyRemote * http://…:8080 Uncomment to route requests through another proxy

Order deny,allow
Deny from all
Allow from all # Not a good idea, set to allowed IP ranges

CacheRoot “/tmp”
CacheMaxExpire 24
CacheLastModifiedFactor 0.1
CacheDefaultExpire 1

ServerName my-proxy

ErrorLog “/var/log/apache2/proxy-error.log”
CustomLog “/var/log/apache2/proxy-access.log” common

Also read this.

Tips

You can use mod_rewrite to rewrite requests. To rewrite root (/) to /temporary_outage you could use the following rewrite:

RewriteCond %{HTTP_HOST} ^(www\.)?xxx\.com
RewriteRule /$ http://%{HTTP_HOST}/temporary_outage/ [P,L]

Forward Proxy works

# webproxy server1

NameVirtualHost *:80

ServerName server1
ProxyPass / http://realserver1/
ProxyHTMLURLMap http://realserver1 /

ProxyPassReverse /
ProxyHTMLInterp On
ProxyHTMLURLMap / /
RequestHeader unset Accept-Encoding

# webproxy server2
NameVirtualHost *:80

ServerName server2
ProxyPass / http://realserver2/
ProxyHTMLURLMap http://realserver2 /

ProxyPassReverse /
ProxyHTMLInterp On
ProxyHTMLURLMap / /
RequestHeader unset Accept-Encoding

# realserver2 reverse proxy
NameVirtualHost *:80

ServerName realserver2

Order deny,allow
Allow from all

ProxyPreserveHost On
ProxyPass / http://localhost:32101/
ProxyPassReverse / http://localhost:32101/

Apache Performance Tuning

Apache Performance Tuning

Forewarning:

“Premature optimization is the root of all evil.” — Donald Knuth.

Select MPM
Chose the right MPM for the right job:
prefork [default MPM for Apache 2.0 and 1.3]:
• Apache 1.3-based.
• Multiple processes, 1 thread per process, processes handle requests.
• Used for security and stability.
• Has higher memory consumption and lower performance over the newer Apache 2.0-based threaded MPMs.
worker:
• Apache 2.0-based.
• Multiple processes, many threads per process, threads handle requests.
• Used for lower memory consumption and higher performance.
• Does not provide the same level of isolation request-to-request, as a process-based MPM does.
winnt:
• The only MPM choice under Windows.
• 1 parent process, exactly 1 child process with many threads, threads handle requests.
• Best solution under Windows, as on this platform, threads are always “cheaper” to use over processes.
Configure MPM
Core Features and Multi-Processing Modules
Default Configuration

StartServers 8
MinSpareServers 5
MaxSpareServers 20
MaxClients 150
MaxRequestsPerChild 1000


StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0


ThreadsPerChild 250
MaxRequestsPerChild 0

Directives

MaxClients, for prefork MPM

MaxClients sets a limit on the number of simultaneous connections/requests that will be served.

I consider this directive to be the critical factor to a well functioning server. Set this number too low and resources will go to waste. Set this number too high and an influx of connections will bring the server to a stand still. Set this number just right and your server will fully utilize the available resources.

An approximation of this number should be derived by dividing the amount of system memory (physical RAM) available by the maximum size of an apache/httpd process; with a generous amount spared for all other processes.

MaxClients ? (RAM – size_all_other_processes)/(size_apache_process)

Use ‘ps -ylC httpd –sort:rss’ to find process size. Divide number by 1024 to get megabytes. Also try ‘top’.
Use ‘free -m’ for a general overview. The key figure to look at is the buffers/cache used value.

Use ‘vmstat 2 5’ to display the number of runnable, blocked, and waiting processes; and swap in and swap out.
Example:
• System: VPS (Virtual Private Server), CentOS 4.4, with 128MB RAM
• Apache: v2.0, mpm_prefork, mod_php, mod_rewrite, mod_ssl, and other modules
• Other Services: MySQL, Bind, SendMail
• Reported System Memory: 120MB
• Reported httpd process size: 7-13MB
• Assumed memory available to Apache: 90MB
Optimal settings:
• StartServers 5
• MinSpareServers 5
• MaxSpareServers 10
• ServerLimit 15
• MaxClients 15
• MaxRequestsPerChild 2000

With the above configuration, we start with 5-10 processes and set a top limit of 15. Anything above this number will cause serious swapping and thrashing under a load; due to the low amount of RAM available to the [virtual] Server. With a dedicated Server, the default values [ServerLimit 256] will work with 1-2GB of RAM.

When calculating MaxClients, take into consideration that the reported size of a process and the effective size are two different values. In this setup, it might be safe to use 20 or more workers… Play with different values and check your system stats.

Note that when more connections are attempted than there are workers, the connections are placed into a queue. The default queue size value is 511 and can be adjusted with the ListenBackLog directive.
ThreadsPerChild, for winnt MPM
On the Windows side, the only useful directive is ThreadsPerChild, which is usually set to a value of 250 [defaults to 64 without a value]. If you expect more, or less, concurrent connections/requests, set this directive appropriately. Check process size with Task Manager, under different values and server load.
MaxRequestsPerChild
Directive MaxRequestsPerChild is used to recycle processes. When this directive is set to 0, an unlimited amount of requests are allowed per process.
While some might argue that this increases server performance by not burdening Apache with having to destroy and create new processes, there is the other side to the argument…
Setting this value to the amount of requests that a website generates per day, divided by the number of processes, will have the benefit of keeping memory leaks and process bloat to a minimum [both of which are a common problem]. The goal here is to recycle each process once per day, as apache threads gradually increase their memory allocation as they run.
Note that under the winnt MPM model, recycling the only request serving process that Apache contains, can present a problem for some sites with constant and heavy traffic.
Requests vs. Client Connections
On any given connection, to load a page, a client may request many URLs: page, site css files, javascript files, image files, etc.
Multiple requests from one client in rapid succession can have the same effect on a Server as “concurrent” connections [threaded MPMs and directive KeepAlive taken into consideration]. If a particular website requires 10 requests per page, 10 concurrent clients will require MPM settings that are geared more towards 20-70 clients. This issue manifests itself most under a process-based MPM [prefork].

Separate Static and Dynamic Content

Use separate servers for static and dynamic content. Apache processes serving dynamic content will carry overhead and swell to the size of the content being served, never decreasing in size. Each process will incur the size of any loaded PHP or Perl libraries. A 6MB-30MB process size [or 10% of server’s memory] is not unusual, and becomes a waist of resources for serving static content.
For a more efficient use of system memory, either use mod_proxy to pass specific requests onto another Apache Server, or use a lightweight server to handle static requests:
• lighttpd [has experimental win32 builds]
• tux [patched into RedHat, runs inside the Linux kernel and is at the top of the charts in performance]
The Server handling the static content goes up front.
Note that configuration settings will be quite different between a dynamic content Server and a static content Server

mod_deflate

Reduce bandwidth by 75% and improve response time by using mod_deflate.
LoadModule deflate_module modules/mod_deflate.so

AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/x-javascript

Loaded Modules
Reduce memory footprint by loading only the required modules.
Some also advise to statically compile in the needed modules, over building DSOs (Dynamic Shared Objects). Very bad advice. You will need to manually rebuild Apache every time a new version or security advisory for a module is put out, creating more work, more build related headaches, and more downtime.
mod_expires
Include mod_expires for the ability to set expiration dates for specific content; utilizing the ‘If-Modified-Since’ header cache control sent by the user’s browser/proxy. Will save bandwidth and drastically speed up your site for [repeat] visitors.
Note that this can also be implemented with mod_headers.
KeepAlive
Enable HTTP persistent connections to improve latency times and reduce server load significantly [25% of original load is not uncommon].
prefork MPM:
KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 80
worker and winnt MPMs:
KeepAlive On
KeepAliveTimeout 15
MaxKeepAliveRequests 80

With the prefork MPM, it is recommended to set ‘KeepAlive’ to ‘Off’. Otherwise, a client will tie up an entire process for that span of time. Though in my experience, it is more useful to simply set the ‘KeepAliveTimeout’ value to something very low [2 seconds seems to be the ideal value]. This is not a problem with the worker MPM [thread-based], or under Windows [which only has the thread-based winnt MPM].
With the worker and winnt MPMs, the default 15 second timeout is setup to keep the connection open for the next page request; to better handle a client going from link to link. Check logs to see how long a client remains on each page before moving on to another link. Set value appropriately [do not set higher than 60 seconds].

SymLinks
Make sure ‘Options +FollowSymLinks -SymLinksIfOwnerMatch’ is set for all directories. Otherwise, Apache will issue an extra system call per filename component to substantiate that the filename is NOT a symlink; and more system calls to match an owner.

Options FollowSymLinks

AllowOverride
Set a default ‘AllowOverride None’ for your filesystem. Otherwise, for a given URL to path translation, Apache will attempt to detect an .htaccess file under every directory level of the given path.

AllowOverride None

ExtendedStatus
If mod_status is included, make sure that directive ‘ExtendedStatus’ is set to ‘Off’. Otherwise, Apache will issue several extra time-related system calls on every request made.
ExtendedStatus Off

ExtendedStatus
If mod_status is included, make sure that directive ‘ExtendedStatus’ is set to ‘Off’. Otherwise, Apache will issue several extra time-related system calls on every request made.
ExtendedStatus Off
Timeout
Lower the amount of time the server will wait before failing a request.
Timeout 45

Other/Specific
Cache all PHP pages, using Squid, and/or a PHP Accelerator and Encoder application, such as APC. Also take a look at mod_cache under Apache 2.2.
Convert/pre-render all PHP pages that do not change request-to-request, to static HTML pages. Use ‘wget’ or ‘HTTrack’ to crawl your site and perform this task automatically.
Pre-compress content and pre-generate headers for static pages; send-as-is using mod_asis. Can use ‘wget’ or ‘HTTrack’ for this task. Make sure to set zlib Compression Level to a high value (6-9). This will take a considerable amount of load off the server.
Use output buffering under PHP to generate output and serve requests without pauses.
Avoid content negotiation for faster response times.
Make sure log files are being rotated. Apache will not handle large (2gb+) files very well.
Gain a significant performance improvement by using SSL session cache.
Outsource your images to Amazon’s Simple Storage Service (S3).
Measuring Web Server Performance