Discussion:
QSTATUS command - Interpretation
Coombs, Lawrence
2014-10-22 12:38:23 UTC
Permalink
I had an incident yesterday that is simply driving me crazy.

Application:
Two AIX servers running MQ 7.0.1.2 receives requests into a cluster queue (one on each).
Responders are four JVMs running jboss and using MDBs to receive the requests, process them and place the results on another queue. Two JVMs connect to each MQServer and read form the same cluster queue.

For a couple of hours I noticed that the QTIME (QSTATUS command) shot up to between 300000 – 700000 microseconds (which is 300 – 700 milliseconds). The QTIME is normally around 600 microseconds or less. The MSAGE would sporadically jump from 1 –to as much as 9 seconds. I could see the queue depth increasing and the MDBs were not pulling the messages even though there were ample threads running (IPPROCS > 10). Then all of sudden the messages would get picked up and processed. The UNCOMM was always NO. I had the network folks, UNIX admins investigate and they found nothing.

I recycled the JVMs and even rebooted one of the AIX servers, no change. After a couple of hours everything went back to normal.

What would cause the QTIME to get so high? These messages are non-persistent and I did not observe any memory, I/O and the CPUs were almost idle.
Also, how can the MSAGE parameter jump to 9 seconds, yet the QTIME averages stayed between the 300 - 700 millisecond range?

Any insight would be appreciated.


This message, including any attachments, is the property of Sears Holdings Corporation and/or one of its subsidiaries. It is confidential and may contain proprietary or legally privileged information. If you are not the intended recipient, please delete it without reading the contents. Thank you.


To unsubscribe, write to ***@LISTSERV.MEDUNIWIEN.AC.AT and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Luc-Michel Demey (MQ)
2014-10-22 13:06:43 UTC
Permalink
hello,
I run in a similar problem last year.
After days of testing, the answer was a fix n the network layer in AIX.
MQ was not the culprit !
I will try to find some details, but if you open a case with IBM support
you will find the APAR.

HTH, LMD.
Post by Coombs, Lawrence
I had an incident yesterday that is simply driving me crazy.
Two AIX servers running MQ 7.0.1.2 receives requests into a cluster queue (one on each).
Responders are four JVMs running jboss and using MDBs to receive the
requests, process them and place the results on another queue. Two
JVMs connect to each MQServer and read form the same cluster queue.
For a couple of hours I noticed that the QTIME (QSTATUS command) shot
up to between 300000 – 700000 microseconds (which is 300 – 700
milliseconds). The QTIME is normally around 600 microseconds or less.
The MSAGE would sporadically jump from 1 –to as much as 9 seconds. I
could see the queue depth increasing and the MDBs were not pulling the
messages even though there were ample threads running (IPPROCS > 10).
Then all of sudden the messages would get picked up and processed. The
UNCOMM was always NO. I had the network folks, UNIX admins investigate
and they found nothing.
I recycled the JVMs and even rebooted one of the AIX servers, no
change. After a couple of hours everything went back to normal.
What would cause the QTIME to get so high? These messages are
non-persistent and I did not observe any memory, I/O and the CPUs were
almost idle.
Also, how can the MSAGE parameter jump to 9 seconds, yet the QTIME
averages stayed between the 300 - 700 millisecond range?
Any insight would be appreciated.
This message, including any attachments, is the property of Sears
Holdings Corporation and/or one of its subsidiaries. It is
confidential and may contain proprietary or legally privileged
information. If you are not the intended recipient, please delete it
without reading the contents. Thank you.
------------------------------------------------------------------------
List Archive <http://listserv.meduniwien.ac.at/archives/mqser-l.html>
- Manage Your List Settings
<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
Unsubscribe
Instructions for managing your mailing list subscription are provided
in the Listserv General Users Guide available at http://www.lsoft.com
<http://www.lsoft.com/resources/manuals.asp>
--
Luc-Michel Demey
Freelance MQ Expert
Paris - France
(lmd at demey-consulting dot fr)


To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Coombs, Lawrence
2014-10-22 13:08:39 UTC
Permalink
Any help you can give would be appreciated. Why would the issue just resolve itself?

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Luc-Michel Demey (MQ)
Sent: Wednesday, October 22, 2014 8:07 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: QSTATUS command - Interpretation

hello,
I run in a similar problem last year.
After days of testing, the answer was a fix n the network layer in AIX.
MQ was not the culprit !
I will try to find some details, but if you open a case with IBM support you will find the APAR.

HTH, LMD.
Le 22/10/2014 14:38, Coombs, Lawrence a écrit :
I had an incident yesterday that is simply driving me crazy.

Application:
Two AIX servers running MQ 7.0.1.2 receives requests into a cluster queue (one on each).
Responders are four JVMs running jboss and using MDBs to receive the requests, process them and place the results on another queue. Two JVMs connect to each MQServer and read form the same cluster queue.

For a couple of hours I noticed that the QTIME (QSTATUS command) shot up to between 300000 – 700000 microseconds (which is 300 – 700 milliseconds). The QTIME is normally around 600 microseconds or less. The MSAGE would sporadically jump from 1 –to as much as 9 seconds. I could see the queue depth increasing and the MDBs were not pulling the messages even though there were ample threads running (IPPROCS > 10). Then all of sudden the messages would get picked up and processed. The UNCOMM was always NO. I had the network folks, UNIX admins investigate and they found nothing.

I recycled the JVMs and even rebooted one of the AIX servers, no change. After a couple of hours everything went back to normal.

What would cause the QTIME to get so high? These messages are non-persistent and I did not observe any memory, I/O and the CPUs were almost idle.
Also, how can the MSAGE parameter jump to 9 seconds, yet the QTIME averages stayed between the 300 - 700 millisecond range?

Any insight would be appreciated.

This message, including any attachments, is the property of Sears Holdings Corporation and/or one of its subsidiaries. It is confidential and may contain proprietary or legally privileged information. If you are not the intended recipient, please delete it without reading the contents. Thank you.

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>
--
Luc-Michel Demey

Freelance MQ Expert

Paris - France

(lmd at demey-consulting dot fr)

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

This message, including any attachments, is the property of Sears Holdings Corporation and/or one of its subsidiaries. It is confidential and may contain proprietary or legally privileged information. If you are not the intended recipient, please delete it without reading the contents. Thank you.


To unsubscribe, write to ***@LISTSERV.MEDUNIWIEN.AC.AT and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Loading...