May 2020
« Mar    


WordPress Quotes

History is a relentless master. It has no present, only the past rushing into the future. To try to hold fast is to be swept aside.
John F. Kennedy
May 2020
« Mar    

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (270)
centos8 (3)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (2)
horoscope (23)
Hyper-V (10)
IIS (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Ubuntu (1)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

0 visitors online now
0 guests, 0 bots, 0 members

Hit Counter provided by dental implants orange county

maximum-number-of-file-descriptors solaris

Before I get into details here is the bottom line. If you start MySQL on Solaris as a non-root (ie, mysql) user and for some reason you need to adjust the file descriptor resource limit for the parent shell, never use ‘ulimit -n’. This will set both the soft and hard limit and may cause MySQL to adjust the max_connections and table_open_cache configuration variables upon next startup.

Use either:

 ulimit -S -n 1024

or something like:

  prctl -n process.max-file-descriptor -t basic -v  1024 -r -i process $$

The Details

The default ‘basic’ privilege value for the resource control process.max-file-descriptor is 256. This control represents the soft ulimit for file descriptors per process. The default ‘privileged’ privilege is set to 65535, which represents the hard ulimit. A non-root user can adjust the soft limit down or up to the hard limit. Unless it has PRIV_SYS_RESOURCE a non-root user can only decrease the hard limit.

Not being aware of the default values I started a sysbench run with 256 threads. It failed with the following message:

FATAL: unable to connect to MySQL server, aborting...
FATAL: error 2001: Can't create UNIX socket (24)
FATAL: failed to connect to database server!
FATAL: thread#252: failed to connect to database server, aborting...

Sysbench fails because the action for the basic privilege process.max-file-descriptors is deny. I was running sysbench logged in as mysql. I made the mistake of increasing the ulimit to what I thought was a reasonable 1024 via ‘ulimit -n 1024’. When restarting mysqld it inherited a soft and hard limit of 1024. This had the effect of causing mysqld to recalculate the values for max_connections and table_open_cache as displayed in the error log:

090610  9:44:11 [Warning] Changed limits: max_open_files: 1024  max_connections: 886  table_cache: 64

This value is much too small for table_open_cache and is guaranteed to cause heavy contention on LOCK_open as concurrent threads increase.

Why did MySQL change table_open_cache and max_connections?

Upon startup mysqld will calculate the number of open files it wants selecting the greatest of:

  • myisam’s requirement for 2 file handles per connection
    • 10+max_connections+table_cache_size\*2
  • max_connnections\*5
  • open_files_limit

For the purpose of this discussion let’s call this max_open_files.

It then compares max_open_files with the soft resource limit for file descriptors per process. If the soft limit is less than max_open_files both the soft and hard limit will be set to max_open_files via setrlimit. If setrlimit fails the requested max_open_files is modified to the old soft limit. If the returned max_open_files is less than myisam’s requirement for 2 file handles per connection and open_files_limit has not been specified, max_connections and table_open_cache will be recalculated to fit within the new boundaries.

Now take the case where I inadvertently set both the soft and hard limit via ‘ulimit -n 1024’. I had set  open_table_cache=4096 and max_connections=2049, arbitrary large values so I could set them once and forget about them. In this case max_open_files=max_connections\*5=10245. When trying to increase the limits to 10245, the setrlimit call fails because I started mysqld as the mysql user and it does not have the privilege to increase the hard limit from 1024 to 10245. In this case max_open_files is set to 1024, which causes mysqld to recalculate max_connections and table_open_cache:

090610  9:44:11 [Warning] Changed limits: max_open_files: 1024  max_connections: 886  table_cache: 64

How to observe process.max-file-descriptor

You can use either prctl or ulimit to observe the current values of process.max-file-descriptor:

$ prctl -n process.max-file-descriptor  -i process $$
process: 20773: -ksh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
        basic             256       -   deny                             20773
        privileged      65.5K       -   deny                                 -
        system          2.15G     max   deny                                 -
$ ulimit -S -n ### soft limit
$ ulimit -H -n ### hard limit

On running processes:

# ps -ef | grep <username> | grep -v root \
| awk '{print "prctl -n process.max-file-descriptor -v <value> -r -i process " $2}' | sh -x

System wide via mdb(add to /etc/system to be presistent across reboots):

# mdb -kw
rlim_fd_cur/W <0tvalue>
rlim_fd_max/W <0tvalue>

This only works in the shell and not in .profile or .cshrc:

$ ulimit -Hn 
$ ulimit -Sn 

The same with:

$ prctl -n process.max-file-descriptor -v <value> -r -i process $$

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>




Blue Captcha Image


Protected by WP Anti Spam