Monitoring Glassfish V3.1.2 Options

There are 2 crucial phases you want to look under the hood of your running Glassfish or inside the JVM underneath: Performance Tuning and Health Monitoring during production.

With JMX (Java Management Extensions, Wikipedia) at hand, there are a few options to choose from.

JConsonsole

The graphical monitoring tool is great for local deployment, it allows you to connect to a JVM on the same host or a remote host. It creates line graphs for your for all relevant from the moment you connect, it is perfect to observe a server while you do some testing or other actions, though it does not record any values while you are not connected. I have a hard time to get it running on a remote server and I do not favour the ‘open’ approach (see previous blog entry) which allows anyone to access the JVM with the disabled authentication settings. I also had situations where the JVM was frozen and it was no longer possible to access the JVM for monitoring, here I would rather have snapshots before the problem started together with server.log.

JConsole

Glassfish Rest Interface

Note: You need to enable the areas you want to monitor with the admin console (or the asadmin command line) because per default all are OFF.

Glassfish Monitoring Settings

Then you access the monitoring values from the browser, curl or another application (authorization required!) through the URI, like https://localhost:4848/monitoring/domain/server/jvm/memory
This also very useful to explore the available values for the asadmin approach (next option)

Restful through browser

Restful through browser

asadmin commandline

You can access all these telemetry stuff also through the asadmin command line ($GLASSFISH_HOME/bin), this comes handy for administrator who want to run cronjobs or bash scripts to take snapshots.

asadmin

./asadmin –user myadminuser –passwordfile /opt/passwordfile.txt get –monitor server.jvm.thread-system.*
Note: you need to pass admin user and password, the passwordfile contains the password in clear text:
AS_ADMIN_PASSWORD=mysupersecretpassword

glassfish@ip-10-10-10-10:/opt/app-server/bin# ./asadmin --user admin --passwordfile ~/pwd.txt get --monitor server.jvm.memory.*
server.jvm.memory.committedheapsize-count-count = 3914924032
server.jvm.memory.committedheapsize-count-description = Amount of memory in bytes that is committed for the Java virtual machine to use
server.jvm.memory.committedheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.committedheapsize-count-name = CommittedHeapSize
server.jvm.memory.committedheapsize-count-starttime = 1364197189330
server.jvm.memory.committedheapsize-count-unit = bytes
server.jvm.memory.committednonheapsize-count-count = 259194880
server.jvm.memory.committednonheapsize-count-description = Amount of memory in bytes that is committed for the Java virtual machine to use
server.jvm.memory.committednonheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.committednonheapsize-count-name = CommittedNonHeapSize
server.jvm.memory.committednonheapsize-count-starttime = 1364197189330
server.jvm.memory.committednonheapsize-count-unit = bytes
server.jvm.memory.dotted-name = server.jvm.memory
server.jvm.memory.initheapsize-count-count = 4244635648
server.jvm.memory.initheapsize-count-description = Amount of memory in bytes that the Java virtual machine initially requests from the operating system for memory management
server.jvm.memory.initheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.initheapsize-count-name = InitialHeapSize
server.jvm.memory.initheapsize-count-starttime = 1364197189330
server.jvm.memory.initheapsize-count-unit = bytes
server.jvm.memory.initnonheapsize-count-count = 24313856
server.jvm.memory.initnonheapsize-count-description = Amount of memory in bytes that the Java virtual machine initially requests from the operating system for memory management
server.jvm.memory.initnonheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.initnonheapsize-count-name = InitialNonHeapSize
server.jvm.memory.initnonheapsize-count-starttime = 1364197189330
server.jvm.memory.initnonheapsize-count-unit = bytes
server.jvm.memory.maxheapsize-count-count = 3914924032
server.jvm.memory.maxheapsize-count-description = Maximum amount of memory in bytes that can be used for memory management
server.jvm.memory.maxheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.maxheapsize-count-name = MaxHeapSize
server.jvm.memory.maxheapsize-count-starttime = 1364197189330
server.jvm.memory.maxheapsize-count-unit = bytes
server.jvm.memory.maxnonheapsize-count-count = 587202560
server.jvm.memory.maxnonheapsize-count-description = Maximum amount of memory in bytes that can be used for memory management
server.jvm.memory.maxnonheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.maxnonheapsize-count-name = MaxNonHeapSize
server.jvm.memory.maxnonheapsize-count-starttime = 1364197189330
server.jvm.memory.maxnonheapsize-count-unit = bytes
server.jvm.memory.objectpendingfinalizationcount-count-count = 0
server.jvm.memory.objectpendingfinalizationcount-count-description = Approximate number of objects for which finalization is pending
server.jvm.memory.objectpendingfinalizationcount-count-lastsampletime = 1364268183797
server.jvm.memory.objectpendingfinalizationcount-count-name = ObjectsPendingFinalization
server.jvm.memory.objectpendingfinalizationcount-count-starttime = 1364197189330
server.jvm.memory.objectpendingfinalizationcount-count-unit = count
server.jvm.memory.usedheapsize-count-count = 1841024784
server.jvm.memory.usedheapsize-count-description = Amount of used memory in bytes
server.jvm.memory.usedheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.usedheapsize-count-name = UsedHeapSize
server.jvm.memory.usedheapsize-count-starttime = 1364197189330
server.jvm.memory.usedheapsize-count-unit = bytes
server.jvm.memory.usednonheapsize-count-count = 236054792
server.jvm.memory.usednonheapsize-count-description = Amount of used memory in bytes
server.jvm.memory.usednonheapsize-count-lastsampletime = 1364268183797
server.jvm.memory.usednonheapsize-count-name = UsedNonHeapSize
server.jvm.memory.usednonheapsize-count-starttime = 1364197189330
server.jvm.memory.usednonheapsize-count-unit = bytes

Command get executed successfully.

You can use wildcards, but be aware, server.* can create easily 50.000 + lines, even server.jvm.* 5.000 + lines.
Only one parameter can be passed.
An option is to snapshot this by a cronjob into a datetime stamped file every 1 or 5 minutes.

Further Options

• Implement your own monitoring as part of your deployment, make sure your persist into a database or logfiles, so the information is accessible even the JVM is down.
  Pro: You can implement to whatever extend you want.
  Con: Learning curve for JMX.

  http://weblogs.java.net/blog/kalali/archive/2010/04/06/manage-administrate-and-monitor-glassfish-v3-java-code-using-amx-jmx

  http://docs.oracle.com/cd/E19575-01/821-0031/gcgvu/index.html

• Use an existing solution that can be deployed into the application server
  The only application I could find is LightFish by Adam Bien

  http://lightfish.adam-bien.com/