Qmgr running out of resources, again, on AIX

[Yonny Serrano]

We have three qmgrs running on a production AIX box, with WMQ 5.3 CSD06.
Yesterday, one of them stopped answering applications' requests for
connection. We had to reboot the machine cause restarting the qmgr did
not help.
The qmgr is running fine now but we want to determine the cause of the
problem.
The error logs in /var/mqm/QMIB/errors are full of these errors:
- AMQ9508: Program cannot connect to the queue manager and
- AMQ9999: Channel program ended abnormally

The logs in /var/mqm/errors show almost only "failing process" errors.

A few months ago we had a similar problem and IBM recommended some
changes on the qmgr ini file:

CHANNELS:
MaxChannels = 4096
MaxActiveChannels = 4096
MQIBINDTYPE=FASTPATH
PPOptThreads=20
PPMaxThreads=30
PPOptProcesses=100
PPMaxProcesses=-1
TuningParameters:
AgentClassMap=(1,2,3)
AgentClassLimit1=(20,30,100)
AgentClassLimit2=(20,30,100)
AgentClassLimit3=(20,30,100)

After this change, the problem did not showed again until yesterday.

I have all the error logs, fdc and ini files, so, if you are willing to
help and need those files I can send them directly

Thanks,

Yonny R. Serrano.

pd. I have also the error log of the operating system, which is
reporting "Software defects".

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Jay H. Lang]

Send me the files...I will have a look.

jay-***@public.gmane.org

Thanks.
--
Jay H. Lang
Chief Technologist
Distributed Computing Professionals Inc.
IBM Certified Specialist - WebSphere MQ
303 277-1873 - Colorado Office
651 406-2131 - USPS Office
303 807-9700 - Cell
----- Original Message -----
From: "Yonny Serrano" <yonny.serrano-8a+***@public.gmane.org>
To: <MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org>
Sent: Thursday, August 18, 2005 10:36 AM
Subject: Qmgr running out of resources, again, on AIX

Post by Yonny Serrano
We have three qmgrs running on a production AIX box, with WMQ 5.3 CSD06.
Yesterday, one of them stopped answering applications' requests for
connection. We had to reboot the machine cause restarting the qmgr did
not help.
The qmgr is running fine now but we want to determine the cause of the
problem.
- AMQ9508: Program cannot connect to the queue manager and
- AMQ9999: Channel program ended abnormally
The logs in /var/mqm/errors show almost only "failing process" errors.
A few months ago we had a similar problem and IBM recommended some
MaxChannels = 4096
MaxActiveChannels = 4096
MQIBINDTYPE=FASTPATH
PPOptThreads=20
PPMaxThreads=30
PPOptProcesses=100
PPMaxProcesses=-1
AgentClassMap=(1,2,3)
AgentClassLimit1=(20,30,100)
AgentClassLimit2=(20,30,100)
AgentClassLimit3=(20,30,100)
After this change, the problem did not showed again until yesterday.
I have all the error logs, fdc and ini files, so, if you are willing to
help and need those files I can send them directly
Thanks,
Yonny R. Serrano.
pd. I have also the error log of the operating system, which is
reporting "Software defects".
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Jeff A Tressler]

I have not had reason to worry about this before but understand the concept
of the
switch file. In my most recent project, we plan on using JMS to access the
MQ
objects. I have not JMS knowledge.

Apparently, there is a checkbox for XA coordination in the JMS world and
when the programmer checks this, it appears to coordinate the MQ and
Oracle changes. If he does a commit, the message is gone from the queue
and the database is updated. If he does a rollback, the message remains in
the queue and the database is not changed.

Since I never compiled this switch file, nor have I changed the mq.ini file
to
configure the queue manager to use the switch file, are we doing two-phase
commits and somehow JMS is dealing with the XA issues or is JMS really just
doing single-phased commits? And how would I test for this since the code
is
doing a single commit and the updates are occurring.

Jeff Tressler

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Robert Broderick]

Lets see if I get this right.

MQSeries will act as the resource coordinator. He will, once you set up the
XA stanza, communicate to the resources (databases) that he will be the COP
for the transaction as MQSeries initializes. You will see messages in the
AMQERR01.LOG file at MQSeries start up. Good or Bad that is!!!

I believe what you will see if you can trace you JMS app is tht when you
check that box it will issue a "BEGIN" command which will tell MQ to tell
it's resources he knows about that your transaction is under his control and
it is about to start an XA LUW. After that it is exactly as you have
described. When you are done the coordinator sez commit (Phase one) and
prepare your work for backout (Phase two). Then ends the transaction. I
would suspect if there are indoubts the Phase Two allows the resources to
identify the indoubt work as a package and back it out. KEWL!!

Testing. Just break it befor the comit and see what happens. There are
examples of the XA stanza in the Admin manuals.


bobbee

Subject: XA Compliance
Date: Tue, 23 Aug 2005 15:14:10 -0400
I have not had reason to worry about this before but understand the concept
of the
switch file. In my most recent project, we plan on using JMS to access the
MQ
objects. I have not JMS knowledge.
Apparently, there is a checkbox for XA coordination in the JMS world and
when the programmer checks this, it appears to coordinate the MQ and
Oracle changes. If he does a commit, the message is gone from the queue
and the database is updated. If he does a rollback, the message remains in
the queue and the database is not changed.
Since I never compiled this switch file, nor have I changed the mq.ini file
to
configure the queue manager to use the switch file, are we doing two-phase
commits and somehow JMS is dealing with the XA issues or is JMS really just
doing single-phased commits? And how would I test for this since the code
is
doing a single commit and the updates are occurring.
Jeff Tressler
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Jeff A Tressler]

Post by Robert Broderick
MQSeries will act as the resource coordinator. He will,
once you set up the XA stanza, communicate to the
resources (databases) that he will be the COP for the
transaction as MQSeries initializes. ...
I believe what you will see if you can trace you JMS app
is tht when you check that box it will issue a "BEGIN"
command which will tell MQ to tell it's resources he
knows about that your transaction is under his control
and it is about to start an XA LUW. ...
There are examples of the XA stanza in the Admin manuals.

If I understand this correctly, if I have not set the XA
stanaz up to support XA coordination, even though the JMS
thinks it it getting Two Phase commits it is actually just
getting two SINGLE phase commits?

The JMS programmer is setting the XA checkbox but I have
not compiled the switch file or set the XA stanzas. Thus
I beleive XA coordination is not taking place. Thank you
for your help.

Jeff Tressler

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Christopher Frank]

Hello,

I think the reason here is that XA coordination with JMS requires the
involvement of an external transaction manager. I do not believe the
transaction manager that is part of the MQ queue manager can act in this
role. You would need to use the XA capabilities of the application server
in which the JMS program is running to do this coordination. If the JMS
program "thinks" it is getting two-phase commits, this is probably the
reason.

Regards,

Christopher Frank
Certified I/T Specialist - WebSphere Software
IBM Certified Solutions Expert - Websphere MQ & MQ Integrator
--------------------------------------
Phone: [phone] (t/l 653-5532) mobile: [phone]
e-mail: chrisfra-r/Jw6+rmf7HQT0dZR+***@public.gmane.org

Jeff A Tressler <jtressle-ec/***@public.gmane.org>@LISTSERV.MEDUNIWIEN.AC.AT> on 08/24/2005
11:57:45 AM

Please respond to MQSeries List <MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org>

Sent by: MQSeries List <MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org>


To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
cc:
Subject: Re: XA Compliance

Post by Robert Broderick
MQSeries will act as the resource coordinator. He will,
once you set up the XA stanza, communicate to the
resources (databases) that he will be the COP for the
transaction as MQSeries initializes. ...
I believe what you will see if you can trace you JMS app
is tht when you check that box it will issue a "BEGIN"
command which will tell MQ to tell it's resources he
knows about that your transaction is under his control
and it is about to start an XA LUW. ...
There are examples of the XA stanza in the Admin manuals.

If I understand this correctly, if I have not set the XA
stanaz up to support XA coordination, even though the JMS
thinks it it getting Two Phase commits it is actually just
getting two SINGLE phase commits?

The JMS programmer is setting the XA checkbox but I have
not compiled the switch file or set the XA stanzas. Thus
I beleive XA coordination is not taking place. Thank you
for your help.

Jeff Tressler

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Robert Broderick]

I do not know you platform BUT...there could be a good chnce the switch file
is supplied already. Otherwise the compile is pretty straight forward.

Yes, once you put the stanza in the .ini file starting the QMGR becomes
interesting!! {:-)


bobbee

Subject: Re: XA Compliance
Date: Wed, 24 Aug 2005 12:57:45 -0400

Post by Robert Broderick
MQSeries will act as the resource coordinator. He will,
once you set up the XA stanza, communicate to the
resources (databases) that he will be the COP for the
transaction as MQSeries initializes. ...
I believe what you will see if you can trace you JMS app
is tht when you check that box it will issue a "BEGIN"
command which will tell MQ to tell it's resources he
knows about that your transaction is under his control
and it is about to start an XA LUW. ...
There are examples of the XA stanza in the Admin manuals.

If I understand this correctly, if I have not set the XA
stanaz up to support XA coordination, even though the JMS
thinks it it getting Two Phase commits it is actually just
getting two SINGLE phase commits?
The JMS programmer is setting the XA checkbox but I have
not compiled the switch file or set the XA stanzas. Thus
I beleive XA coordination is not taking place. Thank you
for your help.
Jeff Tressler
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html