May 2020
M T W T F S S
« Mar    
 123
45678910
11121314151617
18192021222324
25262728293031

Categories

WordPress Quotes

History is a relentless master. It has no present, only the past rushing into the future. To try to hold fast is to be swept aside.
John F. Kennedy
May 2020
M T W T F S S
« Mar    
 123
45678910
11121314151617
18192021222324
25262728293031

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (270)
centos8 (3)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
EXCHANGE (3)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (2)
horoscope (23)
Hyper-V (10)
IIS (15)
IPTABLES (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
SCALEIO (1)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Ubuntu (1)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)
ZIMBRA (17)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

0 visitors online now
0 guests, 0 bots, 0 members

Hit Counter provided by dental implants orange county

JAVA using More CPU

In a production environment profiling is not an option, we have seen several times that our CPU has reached almost 100%, we are running tomcat on Linux, so what is happening?

Fortunately, java comes with some great debugging tools, cooperating those tools with Linux built-in tools will let you know what is happening.

Here is what i am going to explain:

  • An introduction about java threads and its relation to what-so-called Linux LWP
  • A step-by-step
1- An introduction
as you may know, any java program starts when the JVM calls the main method, this creates a thread called the main thread and any thread you create using java code will be derived from the main thread (out-of-focus: it is always a good practice to give your threads a name, it is very useful in debugging, however, it is not required here) the same exact behavior occurs on the Linux level, the main thread for java means a process for the OS, and every thread you create using java the OS will create a Light-weight-process or LWP.
to cut it short, Java main thread = Linux process and Java thread = Linux LWP.
the idea here:
  • Ask Linux which LWP is eating the CPU.
  • Ask Java for a Thread Dump.
  • Map this LWP to a Java thread.
  • Get the part of code causing the issue.
2- A Step-By-Step:
  • Get the PID: the very first step is to know what is the Java process ID, we will use Linux tools ps and grep
    ps -A|grep java
    if you are running multiple java processes we can execute
    ps -ef|grep java
  • the next step is to get CPU usage per each LWP related to the main process, again we will use  ps  and  grep
    ps -eLo pid,lwp,nlwp,ruser,pcpu,stime,etime,args|grep {pid} > lwpthread.txt
    the file lwpthread.txt will contain some thing similar:

    PID NLWP SZ RUSER %CPU STIME ELAPSED COMMAND
    8234 8234 1110 admin 0.3 08:11 30:15 /usr/java/jdk1.6.0_24/bin/java
    8234 8245 1110 admin 99.0 08:45 10:15 /usr/java/jdk1.6.0_24/bin/java

    as you can see, we have an LWP(mapped to a java thread) eating the CPU, get the NLWP, that will be our lead in the next step.

  • the next step is to generate a java thread dump, there are two main ways to create a thread dump, the first one is to use the JDK tool jstack and pass the PID to it, the second way is to send a kill signal to the JVM, however in the first way you will have control over where you want to save the thread dump, while on the second way you will have the thread dump written on the java process standard output stream.
    as said, we are using tomcat, so the thread dump will be in catalina.out
    kill -3 {pid}
    the thread dump will be printed to the file with full stack trace. the file will contain the thread you are after, but first convert your LWP id from DEC to HEX so 8245 would be 2035, now open the thread dump with text editor and search for 2035, you will find something similar:
    "TP-Processor234786" daemon prio=10 tid=0x00002aaad8024800 nid=0x2035 runnable [0x00002aaadef29000]
    java.lang.Thread.State: RUNNABLE
    at java.util.HashMap.get(HashMap.java:303)
    at ......

and you are done!

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  

  

  

Blue Captcha Image
Refresh

*

Protected by WP Anti Spam