Running a complex web application with sessions and connections to the DB, MQ and HTTP clients brings you sooner or later into a discussion or problem with connections that have not been properly terminated by Glassfish or the client. Depending on the time-out settings in the OS and in Glassfish you might face CLOSE_WAIT connections. What is causing this is topic by itself, also helping the system to get rid of it. At the end quite likely the problem lies somewhere in the sourcecode.
In my case we found a large number of CLOSE_WAITS to Glassfish with strange IP addresses (closed enterprise app but open to public web).
Here only some helpful scripts (Linux/Ubuntu):
Find the Glassfish PID
jps | grep ASMain | awk '{ print $1 }'
#get it into variable
pid=$(jps | grep ASMain | awk '{ print $1 }')
Show all socket connections
netstat -anp | grep ESTABLISHED netstat -anp | grep CLOSE_WAIT
Show all connections to Glassfish
netstat -anp | grep CLOSE_WAIT| grep $(jps | grep ASMain | awk '{ print $1 }')
netstat -anp | grep ESTABLISHED| grep $(jps | grep ASMain | awk '{ print $1 }')
Still a bit hard to review
netstat
Now lets sort by IP and count it (ignoring the ports)
netstat -anp | grep CLOSE_WAIT| grep $(jps | grep ASMain | awk '{ print $1 }')|awk '{print $5}'|awk -F':' '{print $1}'|sort -k1|uniq -c
Creates a list like this (number of connections and the related ip address)
netstat summary
You can pack this int a cron job or even get it combined with a geolocation webservice to find the user.
