Discussion:
Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Potkay, Peter M (CTO Architecture + Engineering)
2013-12-11 13:48:13 UTC
Permalink
Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Tim Zielke
2013-12-11 14:04:07 UTC
Permalink
Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Ram
2013-12-11 14:23:55 UTC
Permalink
Wouldn't alias queues be helpful here?

If mqm is allowed to do this as per RFE, we would need a additional event messages to allow for audit trail in security conscious environment.

Ram Ramanathan

On Dec 11, 2013, at 9:04 AM, Tim Zielke <tim.zielke-PR+tvw7B/***@public.gmane.org> wrote:

> Hi Peter,
>
> Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.
>
> Thanks,
> Tim
>
> From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
> Sent: Wednesday, December 11, 2013 7:48 AM
> To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
> Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
>
>
> Link to review the RFE and vote for it
> http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656
>
>
> Description:
> Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue
>
> Use case:
> Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
> However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
> A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.
>
> Business justification:
> By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.
>
>
>
>
> Peter Potkay
> Application Platform Engineering @ The Hartford
> Websphere MQ, Message Broker and DataPower
> (W) 860-624-1395 / (M) 860-202-1375
>
>
>
> ************************************************************
> This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
> ************************************************************
>
>
> List Archive - Manage Your List Settings - Unsubscribe
> Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
>
>
> List Archive - Manage Your List Settings - Unsubscribe
> Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Potkay, Peter M (CTO Architecture + Engineering)
2013-12-11 14:27:28 UTC
Permalink
Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I'm thinking the change in security would not give the desired results.


Peter Potkay


From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Potkay, Peter M (CTO Architecture + Engineering)
2013-12-11 14:31:30 UTC
Permalink
“Wouldn't alias queues be helpful here?”
Indeed, but it assumes every local queue is sitting behind one or more alias queues. I know some shops do exactly this, but impossible to retrofit that solution into a big environment that doesn’t already use alias queues everywhere.

“If mqm is allowed to do this as per RFE, we would need a additional event messages to allow for audit trail in security conscious environment. ”
There isn’t an event message generated if mqm browses a get enabled queue, so I’m thinking that would be a separate RFE if an event message is desired for mqm browsing a queue.

Peter Potkay


From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Ram
Sent: Wednesday, December 11, 2013 9:24 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Wouldn't alias queues be helpful here?

If mqm is allowed to do this as per RFE, we would need a additional event messages to allow for audit trail in security conscious environment.

Ram Ramanathan

On Dec 11, 2013, at 9:04 AM, Tim Zielke <***@AON.COM<mailto:***@AON.COM>> wrote:
Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************
Tim Zielke
2013-12-11 14:46:40 UTC
Permalink
I ran a test and you are correct. :-)

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I'm thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Roger Lacroix
2013-12-11 17:25:37 UTC
Permalink
Hello Peter,

I don't mean to rain on your parade but I don't know about this.

(1) You want IBM to add an MQGET option to circumvent the inhibit
option? i.e. MQGMO_BROWSE_IF_INHIBITED or
MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM. The application would have to be
coded to know when to use it or provide the user with the ability to
select such an option.

(2) There are already several auditing/tracking solutions that solve
this very problem (yes they are commercial products) and IBM supplies
the free 'amqsaxe0.c' sample that will trace any MQ API
call. Therefore, once you have a trace of the MQ API calls, you know
exactly who did what.

Regards,
Roger Lacroix
Capitalware Inc.

At 08:48 AM 12/11/2013, you wrote:
>
>Link to review the RFE and vote for it
><http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656>http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656
>
>
>Description:
>Enhance Websphere MQ so that a suitably authorized user (member of
>the mqm group) can browse a queue even if that queue is get
>inhibited. The ability to browse would be enough for our use cases,
>but it might be beneficial to also allow the MQ Admin to
>destructively get the message(s) on the Get Inhibited queue
>
>Use case:
>Two applications in the development environment that send MQ
>messages to each other are having a bad day, each laying blame on
>the other as to what is actually being sent to the other. Classic he
>sent, she sent. Or, you're not gonna believe this, they claim that
>MQ is changing the content of the message. So we tell the consuming
>app to close the queue and then we tell the sending app to produce
>another message but only when we tell them the consuming app has
>finally managed to stop, at which point we can look at it. Sounds
>simple. Often its not.
>However, if the MQ Admin could Get Inhibit the queue yet still
>browse the queue, we could work the issue without needing the
>consuming application to stop. The MQ Admin could grab a copy of the
>message, enable the queue for gets and the consuming app would not
>have had to change anything.
>A use case for allowing the ability to destructively get a message
>might be to allow an MQ Admin to step in and save the day when an
>app is stuck in a tight loop on a poison message. Without getting
>the consuming app to stop, the MQ Admin could get inhibit the queue,
>the poison message identified and removed, and the queue get enabled.
>
>Business justification:
>By giving the MQ Administrator this ability it would allow them to
>speed up problem resolution and exonerate Websphere MQ from being
>the source of the issue.
>
>
>
>
>Peter Potkay
>Application Platform Engineering @ The Hartford
>Websphere MQ, Message Broker and DataPower
>(W) 860-624-1395 / (M) 860-202-1375
>
>
>
>
>************************************************************
>This communication, including attachments, is for the exclusive use
>of addressee and may contain proprietary, confidential and/or
>privileged information. If you are not the intended recipient, any
>use, copying, disclosure, dissemination or distribution is strictly
>prohibited. If you are not the intended recipient, please notify
>the sender immediately by return e-mail, delete this communication
>and destroy all copies.
>************************************************************
>
>
>----------
><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List Archive
>-
><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>Your List Settings -
><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>
>
>Instructions for managing your mailing list subscription are
>provided in the Listserv General Users Guide available at
><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Potkay, Peter M (CTO Architecture + Engineering)
2013-12-11 18:48:49 UTC
Permalink
1. No need for a new option. Just tweak the internal code (ha ha, easy for me to say) to not reject a browse attempt if the queue is inhibited if the user attempting the browse has mqm authority. Although....having to specify the option in addition to that might be a good way to enforce the user knows exactly what they are asking for, and so that things continue to work the way they always did if they don't specify the option.


2. If those solutions are already in place, yes. But this seems like a small trivial thing to implement and adding an exit seems or slogging through a trace is overkill.




Peter Potkay


From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
Sent: Wednesday, December 11, 2013 12:26 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hello Peter,

I don't mean to rain on your parade but I don't know about this.

(1) You want IBM to add an MQGET option to circumvent the inhibit option? i.e. MQGMO_BROWSE_IF_INHIBITED or MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM. The application would have to be coded to know when to use it or provide the user with the ability to select such an option.

(2) There are already several auditing/tracking solutions that solve this very problem (yes they are commercial products) and IBM supplies the free 'amqsaxe0.c' sample that will trace any MQ API call. Therefore, once you have a trace of the MQ API calls, you know exactly who did what.

Regards,
Roger Lacroix
Capitalware Inc.

At 08:48 AM 12/11/2013, you wrote:


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************
________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Tim Zielke
2013-12-11 19:35:52 UTC
Permalink
Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I'm thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Potkay, Peter M (CTO Architecture + Engineering)
2013-12-11 20:56:49 UTC
Permalink
That's slick, but....We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would 'hang' until the PID was continued?

Nevertheless your email is getting saved and I'm gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay


From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I'm thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Tim Zielke
2013-12-11 21:27:14 UTC
Permalink
Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That's slick, but....We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would 'hang' until the PID was continued?

Nevertheless your email is getting saved and I'm gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I'm thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org<mailto:MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Paul Clarke
2013-12-11 21:36:44 UTC
Permalink
It is perhaps worth mentioning here that if you freeze the AMQRMPPA process then ‘the other end’ will not wait indefinitely. If a response is not received after approximately ‘heartbeat seconds (plus a bit)’ then the other end will consider it a communications failure and the channel will end. So, if you are going to do this type of processing you can’t hang about

Cheers,
Paul.

Paul Clarke
www.mqgem.com

From: Tim Zielke
Sent: Wednesday, December 11, 2013 9:27 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Yes, sending the STOP signal to a process will STOP all the threads for that process.



This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:



GET disable the queue you wanted to browse

Let the message flow into the queue manager and get placed on that queue

Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager

GET enable the queue and browse the message

Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager



In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.



Thanks,

Tim



From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue



That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?



Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.







Peter Potkay



From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue



Hi Peter,



If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.



There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.



Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.



Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:



kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)

Have the sending application send its message

Browse the message from the queue

kill -s CONT 12345



Thanks,

Tim



From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue



Tim,

The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.





Peter Potkay



From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue



Hi Peter,



Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.



Thanks,

Tim



From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue





Link to review the RFE and vote for it

http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656





Description:

Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue



Use case:

Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.

However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.

A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.



Business justification:

By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.









Peter Potkay
Application Platform Engineering @ The Hartford

Websphere MQ, Message Broker and DataPower

(W) 860-624-1395 / (M) 860-202-1375







************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************




--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com




--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************




--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com




--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************




--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com



--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Tim Zielke
2013-12-11 21:58:40 UTC
Permalink
Hi Paul,

Thanks for the information. Of course, one option is to put all of those below steps into a shell script which would probably run sub-second. Or if you are feeling "MacGyverish", you can try and manually beat the heartbeat clock :-).

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Paul Clarke
Sent: Wednesday, December 11, 2013 3:37 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

It is perhaps worth mentioning here that if you freeze the AMQRMPPA process then ‘the other end’ will not wait indefinitely. If a response is not received after approximately ‘heartbeat seconds (plus a bit)’ then the other end will consider it a communications failure and the channel will end. So, if you are going to do this type of processing you can’t hang about [Smile]

Cheers,
Paul.

Paul Clarke
www.mqgem.com

From: Tim Zielke<mailto:***@AON.COM>
Sent: Wednesday, December 11, 2013 9:27 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>
Darren Douch
2013-12-11 22:20:10 UTC
Permalink
+1

I'm usually go with the flow on these 'additional function' RFEs ... but this
one's got me wondering if there's a 'vote against RFE' link. At risk of getting
called a stick in the mud, I prefer get(disabled) to mean get(disabled).

Just my two penn'orth.
Darren

On 11/12/2013 17:25, Roger Lacroix wrote:
> Hello Peter,
>
> I don't mean to rain on your parade but I don't know about this.
>
> (1) You want IBM to add an MQGET option to circumvent the inhibit option?
> i.e. MQGMO_BROWSE_IF_INHIBITED or MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM. The
> application would have to be coded to know when to use it or provide the user
> with the ability to select such an option.
>
> (2) There are already several auditing/tracking solutions that solve this very
> problem (yes they are commercial products) and IBM supplies the free
> 'amqsaxe0.c' sample that will trace any MQ API call. Therefore, once you have
> a trace of the MQ API calls, you know exactly who did what.
>
> Regards,
> Roger Lacroix
> Capitalware Inc.
>
> At 08:48 AM 12/11/2013, you wrote:
>>
>> Link to review the RFE and vote for it
>> _http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656
>> _
>>
>> *Description:
>> *Enhance Websphere MQ so that a suitably authorized user (member of the mqm
>> group) can browse a queue even if that queue is get inhibited. The ability to
>> browse would be enough for our use cases, but it might be beneficial to also
>> allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue
>>
>> *Use case:
>> *Two applications in the development environment that send MQ messages to
>> each other are having a bad day, each laying blame on the other as to what is
>> actually being sent to the other. Classic he sent, she sent. Or, you're not
>> gonna believe this, they claim that MQ is changing the content of the
>> message. So we tell the consuming app to close the queue and then we tell the
>> sending app to produce another message but only when we tell them the
>> consuming app has finally managed to stop, at which point we can look at it.
>> Sounds simple. Often its not.
>> However, if the MQ Admin could Get Inhibit the queue yet still browse the
>> queue, we could work the issue without needing the consuming application to
>> stop. The MQ Admin could grab a copy of the message, enable the queue for
>> gets and the consuming app would not have had to change anything.
>> A use case for allowing the ability to destructively get a message might be
>> to allow an MQ Admin to step in and save the day when an app is stuck in a
>> tight loop on a poison message. Without getting the consuming app to stop,
>> the MQ Admin could get inhibit the queue, the poison message identified and
>> removed, and the queue get enabled.
>>
>> *Business justification:
>> *By giving the MQ Administrator this ability it would allow them to speed up
>> problem resolution and exonerate Websphere MQ from being the source of the issue.
>>
>>
>>
>>
>> *Peter Potkay*
>> Application Platform Engineering @ The Hartford
>> Websphere MQ, Message Broker and DataPower
>> (W) 860-624-1395/ (M) 860-202-1375
>>
>>
>>
>>
>> ************************************************************
>> This communication, including attachments, is for the exclusive use of
>> addressee and may contain proprietary, confidential and/or privileged
>> information. If you are not the intended recipient, any use, copying,
>> disclosure, dissemination or distribution is strictly prohibited. If you are
>> not the intended recipient, please notify the sender immediately by return
>> e-mail, delete this communication and destroy all copies.
>> ************************************************************
>>
>> --------------------------------------------------------------------------------
>> List Archive <http://listserv.meduniwien.ac.at/archives/mqser-l.html> -
>> Manage Your List Settings
>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>> Unsubscribe
>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>
>>
>>
>> Instructions for managing your mailing list subscription are provided in the
>> Listserv General Users Guide available at http://www.lsoft.com
>> <http://www.lsoft.com/resources/manuals.asp>
>
> --------------------------------------------------------------------------------
> List Archive <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
> Your List Settings
> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe
> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>
>
>
> Instructions for managing your mailing list subscription are provided in the
> Listserv General Users Guide available at http://www.lsoft.com
> <http://www.lsoft.com/resources/manuals.asp>
>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Andrew Hickson
2013-12-12 08:37:55 UTC
Permalink
Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Potkay, Peter M (CTO Architecture + Engineering)
2013-12-12 13:11:35 UTC
Permalink
Darren,
The link I posted takes you the page for the RFE where one can vote for it. Or you can go to the comments tab of the RFE and add a note on why you would prefer an RFE is not implemented.

But here's my thinking, if the queue is Get Disabled, ain't nothing gonna stop the MQ Admin from enabling it anyway if they want to see that message. But then the MQ Admin has got to have a race with the consuming app. If the app opens the queue exclusively thinking it'll prevent anyone even the MQ Admin from opening the queue to look at the messages, ain't nothing gonna stop the MQ admin from just setting up a trace, but that takes some effort. So if the MQ Admin can get at the data anyway, why not make it easier for the MQ Admin to do their job by allowing them the ability to browse the queue even if it's get disabled? I find myself wishing for just that capability a couple times a year. It's not a big deal, but it comes up often enough where I finally decided to submit the RFE.

Here's another scenario where it would be useful, at least for me anyway. When the SNDR channel has the XMITQ disabled because the channel is stopped or retrying, sometimes I want to look at those messages on the XMITQ. One way or the other I can and will see them, but why not make my job easier and allow me as the MQ Admin to browse those messages if I have the correct option/flag specified, I am suitably authorized, and there is no risk of damage? To that last point (no risk of 'damage') I would be fine with the RFE only implementing the Browse piece and not allow destructive gets against a get disabled queue.



Peter Potkay

-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Darren Douch
Sent: Wednesday, December 11, 2013 5:20 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

+1

I'm usually go with the flow on these 'additional function' RFEs ... but this one's got me wondering if there's a 'vote against RFE' link. At risk of getting called a stick in the mud, I prefer get(disabled) to mean get(disabled).

Just my two penn'orth.
Darren

On 11/12/2013 17:25, Roger Lacroix wrote:
> Hello Peter,
>
> I don't mean to rain on your parade but I don't know about this.
>
> (1) You want IBM to add an MQGET option to circumvent the inhibit option?
> i.e. MQGMO_BROWSE_IF_INHIBITED or MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM.
> The application would have to be coded to know when to use it or
> provide the user with the ability to select such an option.
>
> (2) There are already several auditing/tracking solutions that solve
> this very problem (yes they are commercial products) and IBM supplies
> the free 'amqsaxe0.c' sample that will trace any MQ API call.
> Therefore, once you have a trace of the MQ API calls, you know exactly who did what.
>
> Regards,
> Roger Lacroix
> Capitalware Inc.
>
> At 08:48 AM 12/11/2013, you wrote:
>>
>> Link to review the RFE and vote for it
>> _http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID
>> =42656
>> _
>>
>> *Description:
>> *Enhance Websphere MQ so that a suitably authorized user (member of
>> the mqm
>> group) can browse a queue even if that queue is get inhibited. The
>> ability to browse would be enough for our use cases, but it might be
>> beneficial to also allow the MQ Admin to destructively get the
>> message(s) on the Get Inhibited queue
>>
>> *Use case:
>> *Two applications in the development environment that send MQ
>> messages to each other are having a bad day, each laying blame on the
>> other as to what is actually being sent to the other. Classic he
>> sent, she sent. Or, you're not gonna believe this, they claim that MQ
>> is changing the content of the message. So we tell the consuming app
>> to close the queue and then we tell the sending app to produce
>> another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it.
>> Sounds simple. Often its not.
>> However, if the MQ Admin could Get Inhibit the queue yet still browse
>> the queue, we could work the issue without needing the consuming
>> application to stop. The MQ Admin could grab a copy of the message,
>> enable the queue for gets and the consuming app would not have had to change anything.
>> A use case for allowing the ability to destructively get a message
>> might be to allow an MQ Admin to step in and save the day when an app
>> is stuck in a tight loop on a poison message. Without getting the
>> consuming app to stop, the MQ Admin could get inhibit the queue, the
>> poison message identified and removed, and the queue get enabled.
>>
>> *Business justification:
>> *By giving the MQ Administrator this ability it would allow them to
>> speed up problem resolution and exonerate Websphere MQ from being the source of the issue.
>>
>>
>>
>>
>> *Peter Potkay*
>> Application Platform Engineering @ The Hartford Websphere MQ, Message
>> Broker and DataPower
>> (W) 860-624-1395/ (M) 860-202-1375
>>
>>
>>
>>
>> ************************************************************
>> This communication, including attachments, is for the exclusive use
>> of addressee and may contain proprietary, confidential and/or
>> privileged information. If you are not the intended recipient, any
>> use, copying, disclosure, dissemination or distribution is strictly
>> prohibited. If you are not the intended recipient, please notify the
>> sender immediately by return e-mail, delete this communication and destroy all copies.
>> ************************************************************
>>
>> ---------------------------------------------------------------------
>> ----------- List Archive
>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
>> Your List Settings
>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>> Unsubscribe
>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=s
>> ignoff%20mqseries>
>>
>>
>> Instructions for managing your mailing list subscription are provided
>> in the Listserv General Users Guide available at http://www.lsoft.com
>> <http://www.lsoft.com/resources/manuals.asp>
>
> ----------------------------------------------------------------------
> ---------- List Archive
> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your
> List Settings
> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
> Unsubscribe
> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=si
> gnoff%20mqseries>
>
>
> Instructions for managing your mailing list subscription are provided
> in the Listserv General Users Guide available at http://www.lsoft.com
> <http://www.lsoft.com/resources/manuals.asp>
>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Tim Zielke
2013-12-12 14:11:54 UTC
Permalink
Hi Andy,

Thanks for the information. Given your concerns about locks (and that we are dealing with a Development environment, too), I would still assume the following would be acceptable, if done in a shell script.

#!/usr/bin/ksh
#
# Send a STOP signal to all channel processes. This can be done dynamically with more scripting, if needed. For a simple use case, we assume just two pids, 12345 and 12346.
kill -s STOP 12345
kill -s STOP 12346
# GET enable queue, but do it asynchronously and then sleep 1 second
runmqsc < get_enable_queue.txt &
sleep 1
# Run browse program asynchronously and then sleep 1 second
browse_prog Q1 > output.txt &
sleep 1
# Send a CONT signal to all channel processes. This can be done dynamically with more scripting, if needed. For a simple use case, we assume just two pids, 12345 and 12346.
kill -s CONT 12345
kill -s CONT 12346

The above script would only STOP the channel processes for 2 seconds. It would give the browse program a 1 second head start to slide in and browse the message before the channel processes are activated again. I would assume the queue manager could tolerate a 2 second stall if one of those inter-process locks was held by a channel process. You do have the potential where the browse program or get enable process was held up by a lock as well, and you don't actually browse the message before a channel process is woken up and gets it first, but then you would just try again.

I didn't follow how tracing can help with Peter's issue, unless there is a way to see the entire contents of the message that would be PUT on his queue in a trace. I actually tried doing that yesterday, with no success.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>
________________________________



Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Darren Douch
2013-12-12 14:14:47 UTC
Permalink
I'm not saying it wouldn't be a handy thing to be able to do Peter, it was just
the DISABLED no longer equals DISABLED bit that I didn't like.

This may sound like I'm splitting hairs, but 'simply' adding a GET(MQM_ONLY)
option to the queue would get it past my "not a fan" filter. Someone will
probably tell me now why that's a really bad idea. :)

Must have changed my password recently - did attempt to follow the RFE link
yesterday but hadn't signed on after two attempts so moved on to something else.

Darren.

On 12/12/2013 13:11, Potkay, Peter M (CTO Architecture + Engineering) wrote:
> Darren,
> The link I posted takes you the page for the RFE where one can vote for it. Or you can go to the comments tab of the RFE and add a note on why you would prefer an RFE is not implemented.
>
> But here's my thinking, if the queue is Get Disabled, ain't nothing gonna stop the MQ Admin from enabling it anyway if they want to see that message. But then the MQ Admin has got to have a race with the consuming app. If the app opens the queue exclusively thinking it'll prevent anyone even the MQ Admin from opening the queue to look at the messages, ain't nothing gonna stop the MQ admin from just setting up a trace, but that takes some effort. So if the MQ Admin can get at the data anyway, why not make it easier for the MQ Admin to do their job by allowing them the ability to browse the queue even if it's get disabled? I find myself wishing for just that capability a couple times a year. It's not a big deal, but it comes up often enough where I finally decided to submit the RFE.
>
> Here's another scenario where it would be useful, at least for me anyway. When the SNDR channel has the XMITQ disabled because the channel is stopped or retrying, sometimes I want to look at those messages on the XMITQ. One way or the other I can and will see them, but why not make my job easier and allow me as the MQ Admin to browse those messages if I have the correct option/flag specified, I am suitably authorized, and there is no risk of damage? To that last point (no risk of 'damage') I would be fine with the RFE only implementing the Browse piece and not allow destructive gets against a get disabled queue.
>
>
>
> Peter Potkay
>
> -----Original Message-----
> From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Darren Douch
> Sent: Wednesday, December 11, 2013 5:20 PM
> To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
> Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
>
> +1
>
> I'm usually go with the flow on these 'additional function' RFEs ... but this one's got me wondering if there's a 'vote against RFE' link. At risk of getting called a stick in the mud, I prefer get(disabled) to mean get(disabled).
>
> Just my two penn'orth.
> Darren
>
> On 11/12/2013 17:25, Roger Lacroix wrote:
>> Hello Peter,
>>
>> I don't mean to rain on your parade but I don't know about this.
>>
>> (1) You want IBM to add an MQGET option to circumvent the inhibit option?
>> i.e. MQGMO_BROWSE_IF_INHIBITED or MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM.
>> The application would have to be coded to know when to use it or
>> provide the user with the ability to select such an option.
>>
>> (2) There are already several auditing/tracking solutions that solve
>> this very problem (yes they are commercial products) and IBM supplies
>> the free 'amqsaxe0.c' sample that will trace any MQ API call.
>> Therefore, once you have a trace of the MQ API calls, you know exactly who did what.
>>
>> Regards,
>> Roger Lacroix
>> Capitalware Inc.
>>
>> At 08:48 AM 12/11/2013, you wrote:
>>> Link to review the RFE and vote for it
>>> _http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID
>>> =42656
>>> _
>>>
>>> *Description:
>>> *Enhance Websphere MQ so that a suitably authorized user (member of
>>> the mqm
>>> group) can browse a queue even if that queue is get inhibited. The
>>> ability to browse would be enough for our use cases, but it might be
>>> beneficial to also allow the MQ Admin to destructively get the
>>> message(s) on the Get Inhibited queue
>>>
>>> *Use case:
>>> *Two applications in the development environment that send MQ
>>> messages to each other are having a bad day, each laying blame on the
>>> other as to what is actually being sent to the other. Classic he
>>> sent, she sent. Or, you're not gonna believe this, they claim that MQ
>>> is changing the content of the message. So we tell the consuming app
>>> to close the queue and then we tell the sending app to produce
>>> another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it.
>>> Sounds simple. Often its not.
>>> However, if the MQ Admin could Get Inhibit the queue yet still browse
>>> the queue, we could work the issue without needing the consuming
>>> application to stop. The MQ Admin could grab a copy of the message,
>>> enable the queue for gets and the consuming app would not have had to change anything.
>>> A use case for allowing the ability to destructively get a message
>>> might be to allow an MQ Admin to step in and save the day when an app
>>> is stuck in a tight loop on a poison message. Without getting the
>>> consuming app to stop, the MQ Admin could get inhibit the queue, the
>>> poison message identified and removed, and the queue get enabled.
>>>
>>> *Business justification:
>>> *By giving the MQ Administrator this ability it would allow them to
>>> speed up problem resolution and exonerate Websphere MQ from being the source of the issue.
>>>
>>>
>>>
>>>
>>> *Peter Potkay*
>>> Application Platform Engineering @ The Hartford Websphere MQ, Message
>>> Broker and DataPower
>>> (W) 860-624-1395/ (M) 860-202-1375
>>>
>>>
>>>
>>>
>>> ************************************************************
>>> This communication, including attachments, is for the exclusive use
>>> of addressee and may contain proprietary, confidential and/or
>>> privileged information. If you are not the intended recipient, any
>>> use, copying, disclosure, dissemination or distribution is strictly
>>> prohibited. If you are not the intended recipient, please notify the
>>> sender immediately by return e-mail, delete this communication and destroy all copies.
>>> ************************************************************
>>>
>>> ---------------------------------------------------------------------
>>> ----------- List Archive
>>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
>>> Your List Settings
>>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>>> Unsubscribe
>>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=s
>>> ignoff%20mqseries>
>>>
>>>
>>> Instructions for managing your mailing list subscription are provided
>>> in the Listserv General Users Guide available at http://www.lsoft.com
>>> <http://www.lsoft.com/resources/manuals.asp>
>> ----------------------------------------------------------------------
>> ---------- List Archive
>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your
>> List Settings
>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>> Unsubscribe
>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=si
>> gnoff%20mqseries>
>>
>>
>> Instructions for managing your mailing list subscription are provided
>> in the Listserv General Users Guide available at http://www.lsoft.com
>> <http://www.lsoft.com/resources/manuals.asp>
>>
> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
> ************************************************************
> This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
> ************************************************************
>
> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
> in the message body (not the subject), write: SIGNOFF MQSERIES
> Instructions for managing your mailing list subscription are provided in
> the Listserv General Users Guide available at http://www.lsoft.com
> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>
>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Andrew Hickson
2013-12-12 14:30:58 UTC
Permalink
Tim,
The -d option on MQ trace controls how much user data will be displayed in
the trace.
If you set a large -d value then you should be able to trace the entire
message payload.

Regards
Andy.




From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 12/12/2013 14:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



Hi Andy,

Thanks for the information. Given your concerns about locks (and that we
are dealing with a Development environment, too), I would still assume the
following would be acceptable, if done in a shell script.

#!/usr/bin/ksh
#
# Send a STOP signal to all channel processes. This can be done
dynamically with more scripting, if needed. For a simple use case, we
assume just two pids, 12345 and 12346.
kill -s STOP 12345
kill -s STOP 12346
# GET enable queue, but do it asynchronously and then sleep 1 second
runmqsc < get_enable_queue.txt &
sleep 1
# Run browse program asynchronously and then sleep 1 second
browse_prog Q1 > output.txt &
sleep 1
# Send a CONT signal to all channel processes. This can be done
dynamically with more scripting, if needed. For a simple use case, we
assume just two pids, 12345 and 12346.
kill -s CONT 12345
kill -s CONT 12346

The above script would only STOP the channel processes for 2 seconds. It
would give the browse program a 1 second head start to slide in and browse
the message before the channel processes are activated again. I would
assume the queue manager could tolerate a 2 second stall if one of those
inter-process locks was held by a channel process. You do have the
potential where the browse program or get enable process was held up by a
lock as well, and you don't actually browse the message before a channel
process is woken up and gets it first, but then you would just try again.

I didn't follow how tracing can help with Peter's issue, unless there is a
way to see the entire contents of the message that would be PUT on his
queue in a trace. I actually tried doing that yesterday, with no
success.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>




Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Meekin, Paul
2013-12-12 14:28:04 UTC
Permalink
How about a new queue attribute - BROWSE_WHILE_INHIBITED

That way nothing gets broken in terms of exising behaviour. I agree that it is a useful feature but am also a bit uneasy about changing the expected behaviour. Of course this wouldn't help in the case where the application is only browsing too but I would think in general it would work as well.

-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Darren Douch
Sent: 12 December 2013 14:15
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I'm not saying it wouldn't be a handy thing to be able to do Peter, it was just the DISABLED no longer equals DISABLED bit that I didn't like.

This may sound like I'm splitting hairs, but 'simply' adding a GET(MQM_ONLY) option to the queue would get it past my "not a fan" filter. Someone will probably tell me now why that's a really bad idea. :)

Must have changed my password recently - did attempt to follow the RFE link yesterday but hadn't signed on after two attempts so moved on to something else.

Darren.

On 12/12/2013 13:11, Potkay, Peter M (CTO Architecture + Engineering) wrote:
> Darren,
> The link I posted takes you the page for the RFE where one can vote for it. Or you can go to the comments tab of the RFE and add a note on why you would prefer an RFE is not implemented.
>
> But here's my thinking, if the queue is Get Disabled, ain't nothing gonna stop the MQ Admin from enabling it anyway if they want to see that message. But then the MQ Admin has got to have a race with the consuming app. If the app opens the queue exclusively thinking it'll prevent anyone even the MQ Admin from opening the queue to look at the messages, ain't nothing gonna stop the MQ admin from just setting up a trace, but that takes some effort. So if the MQ Admin can get at the data anyway, why not make it easier for the MQ Admin to do their job by allowing them the ability to browse the queue even if it's get disabled? I find myself wishing for just that capability a couple times a year. It's not a big deal, but it comes up often enough where I finally decided to submit the RFE.
>
> Here's another scenario where it would be useful, at least for me anyway. When the SNDR channel has the XMITQ disabled because the channel is stopped or retrying, sometimes I want to look at those messages on the XMITQ. One way or the other I can and will see them, but why not make my job easier and allow me as the MQ Admin to browse those messages if I have the correct option/flag specified, I am suitably authorized, and there is no risk of damage? To that last point (no risk of 'damage') I would be fine with the RFE only implementing the Browse piece and not allow destructive gets against a get disabled queue.
>
>
>
> Peter Potkay
>
> -----Original Message-----
> From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On
> Behalf Of Darren Douch
> Sent: Wednesday, December 11, 2013 5:20 PM
> To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
> Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a
> get inhibited queue
>
> +1
>
> I'm usually go with the flow on these 'additional function' RFEs ... but this one's got me wondering if there's a 'vote against RFE' link. At risk of getting called a stick in the mud, I prefer get(disabled) to mean get(disabled).
>
> Just my two penn'orth.
> Darren
>
> On 11/12/2013 17:25, Roger Lacroix wrote:
>> Hello Peter,
>>
>> I don't mean to rain on your parade but I don't know about this.
>>
>> (1) You want IBM to add an MQGET option to circumvent the inhibit option?
>> i.e. MQGMO_BROWSE_IF_INHIBITED or MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM.
>> The application would have to be coded to know when to use it or
>> provide the user with the ability to select such an option.
>>
>> (2) There are already several auditing/tracking solutions that solve
>> this very problem (yes they are commercial products) and IBM supplies
>> the free 'amqsaxe0.c' sample that will trace any MQ API call.
>> Therefore, once you have a trace of the MQ API calls, you know exactly who did what.
>>
>> Regards,
>> Roger Lacroix
>> Capitalware Inc.
>>
>> At 08:48 AM 12/11/2013, you wrote:
>>> Link to review the RFE and vote for it
>>> _http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_I
>>> D
>>> =42656
>>> _
>>>
>>> *Description:
>>> *Enhance Websphere MQ so that a suitably authorized user (member of
>>> the mqm
>>> group) can browse a queue even if that queue is get inhibited. The
>>> ability to browse would be enough for our use cases, but it might be
>>> beneficial to also allow the MQ Admin to destructively get the
>>> message(s) on the Get Inhibited queue
>>>
>>> *Use case:
>>> *Two applications in the development environment that send MQ
>>> messages to each other are having a bad day, each laying blame on
>>> the other as to what is actually being sent to the other. Classic he
>>> sent, she sent. Or, you're not gonna believe this, they claim that
>>> MQ is changing the content of the message. So we tell the consuming
>>> app to close the queue and then we tell the sending app to produce
>>> another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it.
>>> Sounds simple. Often its not.
>>> However, if the MQ Admin could Get Inhibit the queue yet still
>>> browse the queue, we could work the issue without needing the
>>> consuming application to stop. The MQ Admin could grab a copy of the
>>> message, enable the queue for gets and the consuming app would not have had to change anything.
>>> A use case for allowing the ability to destructively get a message
>>> might be to allow an MQ Admin to step in and save the day when an
>>> app is stuck in a tight loop on a poison message. Without getting
>>> the consuming app to stop, the MQ Admin could get inhibit the queue,
>>> the poison message identified and removed, and the queue get enabled.
>>>
>>> *Business justification:
>>> *By giving the MQ Administrator this ability it would allow them to
>>> speed up problem resolution and exonerate Websphere MQ from being the source of the issue.
>>>
>>>
>>>
>>>
>>> *Peter Potkay*
>>> Application Platform Engineering @ The Hartford Websphere MQ,
>>> Message Broker and DataPower
>>> (W) 860-624-1395/ (M) 860-202-1375
>>>
>>>
>>>
>>>
>>> ************************************************************
>>> This communication, including attachments, is for the exclusive use
>>> of addressee and may contain proprietary, confidential and/or
>>> privileged information. If you are not the intended recipient, any
>>> use, copying, disclosure, dissemination or distribution is strictly
>>> prohibited. If you are not the intended recipient, please notify
>>> the sender immediately by return e-mail, delete this communication and destroy all copies.
>>> ************************************************************
>>>
>>> --------------------------------------------------------------------
>>> -
>>> ----------- List Archive
>>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
>>> Your List Settings
>>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>>> Unsubscribe
>>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=
>>> s
>>> ignoff%20mqseries>
>>>
>>>
>>> Instructions for managing your mailing list subscription are
>>> provided in the Listserv General Users Guide available at
>>> http://www.lsoft.com <http://www.lsoft.com/resources/manuals.asp>
>> ---------------------------------------------------------------------
>> -
>> ---------- List Archive
>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
>> Your List Settings
>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>> Unsubscribe
>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=s
>> i
>> gnoff%20mqseries>
>>
>>
>> Instructions for managing your mailing list subscription are provided
>> in the Listserv General Users Guide available at http://www.lsoft.com
>> <http://www.lsoft.com/resources/manuals.asp>
>>
> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in
> the message body (not the subject), write: SIGNOFF MQSERIES
> Instructions for managing your mailing list subscription are provided
> in the Listserv General Users Guide available at http://www.lsoft.com
> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
> ************************************************************
> This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
> ************************************************************
>
> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in
> the message body (not the subject), write: SIGNOFF MQSERIES
> Instructions for managing your mailing list subscription are provided
> in the Listserv General Users Guide available at http://www.lsoft.com
> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>
>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Doug Clark
2013-12-12 18:17:17 UTC
Permalink
What about a utility program that could browse the messages and display the MQ HEADER information. And maybe provide insight on any uncommitted messages that are just left hanging due to app failures of one sort or another.

Sent from my iPhone

> On Dec 12, 2013, at 8:28 AM, "Meekin, Paul" <paul.meekin-***@public.gmane.org> wrote:
>
> How about a new queue attribute - BROWSE_WHILE_INHIBITED
>
> That way nothing gets broken in terms of exising behaviour. I agree that it is a useful feature but am also a bit uneasy about changing the expected behaviour. Of course this wouldn't help in the case where the application is only browsing too but I would think in general it would work as well.
>
> -----Original Message-----
> From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Darren Douch
> Sent: 12 December 2013 14:15
> To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
> Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
>
> I'm not saying it wouldn't be a handy thing to be able to do Peter, it was just the DISABLED no longer equals DISABLED bit that I didn't like.
>
> This may sound like I'm splitting hairs, but 'simply' adding a GET(MQM_ONLY) option to the queue would get it past my "not a fan" filter. Someone will probably tell me now why that's a really bad idea. :)
>
> Must have changed my password recently - did attempt to follow the RFE link yesterday but hadn't signed on after two attempts so moved on to something else.
>
> Darren.
>
>> On 12/12/2013 13:11, Potkay, Peter M (CTO Architecture + Engineering) wrote:
>> Darren,
>> The link I posted takes you the page for the RFE where one can vote for it. Or you can go to the comments tab of the RFE and add a note on why you would prefer an RFE is not implemented.
>>
>> But here's my thinking, if the queue is Get Disabled, ain't nothing gonna stop the MQ Admin from enabling it anyway if they want to see that message. But then the MQ Admin has got to have a race with the consuming app. If the app opens the queue exclusively thinking it'll prevent anyone even the MQ Admin from opening the queue to look at the messages, ain't nothing gonna stop the MQ admin from just setting up a trace, but that takes some effort. So if the MQ Admin can get at the data anyway, why not make it easier for the MQ Admin to do their job by allowing them the ability to browse the queue even if it's get disabled? I find myself wishing for just that capability a couple times a year. It's not a big deal, but it comes up often enough where I finally decided to submit the RFE.
>>
>> Here's another scenario where it would be useful, at least for me anyway. When the SNDR channel has the XMITQ disabled because the channel is stopped or retrying, sometimes I want to look at those messages on the XMITQ. One way or the other I can and will see them, but why not make my job easier and allow me as the MQ Admin to browse those messages if I have the correct option/flag specified, I am suitably authorized, and there is no risk of damage? To that last point (no risk of 'damage') I would be fine with the RFE only implementing the Browse piece and not allow destructive gets against a get disabled queue.
>>
>>
>>
>> Peter Potkay
>>
>> -----Original Message-----
>> From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On
>> Behalf Of Darren Douch
>> Sent: Wednesday, December 11, 2013 5:20 PM
>> To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>> Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a
>> get inhibited queue
>>
>> +1
>>
>> I'm usually go with the flow on these 'additional function' RFEs ... but this one's got me wondering if there's a 'vote against RFE' link. At risk of getting called a stick in the mud, I prefer get(disabled) to mean get(disabled).
>>
>> Just my two penn'orth.
>> Darren
>>
>>> On 11/12/2013 17:25, Roger Lacroix wrote:
>>> Hello Peter,
>>>
>>> I don't mean to rain on your parade but I don't know about this.
>>>
>>> (1) You want IBM to add an MQGET option to circumvent the inhibit option?
>>> i.e. MQGMO_BROWSE_IF_INHIBITED or MQGMO_BROWSE_IF_INHIBITED_I_AM_MQM.
>>> The application would have to be coded to know when to use it or
>>> provide the user with the ability to select such an option.
>>>
>>> (2) There are already several auditing/tracking solutions that solve
>>> this very problem (yes they are commercial products) and IBM supplies
>>> the free 'amqsaxe0.c' sample that will trace any MQ API call.
>>> Therefore, once you have a trace of the MQ API calls, you know exactly who did what.
>>>
>>> Regards,
>>> Roger Lacroix
>>> Capitalware Inc.
>>>
>>> At 08:48 AM 12/11/2013, you wrote:
>>>> Link to review the RFE and vote for it
>>>> _http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_I
>>>> D
>>>> =42656
>>>> _
>>>>
>>>> *Description:
>>>> *Enhance Websphere MQ so that a suitably authorized user (member of
>>>> the mqm
>>>> group) can browse a queue even if that queue is get inhibited. The
>>>> ability to browse would be enough for our use cases, but it might be
>>>> beneficial to also allow the MQ Admin to destructively get the
>>>> message(s) on the Get Inhibited queue
>>>>
>>>> *Use case:
>>>> *Two applications in the development environment that send MQ
>>>> messages to each other are having a bad day, each laying blame on
>>>> the other as to what is actually being sent to the other. Classic he
>>>> sent, she sent. Or, you're not gonna believe this, they claim that
>>>> MQ is changing the content of the message. So we tell the consuming
>>>> app to close the queue and then we tell the sending app to produce
>>>> another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it.
>>>> Sounds simple. Often its not.
>>>> However, if the MQ Admin could Get Inhibit the queue yet still
>>>> browse the queue, we could work the issue without needing the
>>>> consuming application to stop. The MQ Admin could grab a copy of the
>>>> message, enable the queue for gets and the consuming app would not have had to change anything.
>>>> A use case for allowing the ability to destructively get a message
>>>> might be to allow an MQ Admin to step in and save the day when an
>>>> app is stuck in a tight loop on a poison message. Without getting
>>>> the consuming app to stop, the MQ Admin could get inhibit the queue,
>>>> the poison message identified and removed, and the queue get enabled.
>>>>
>>>> *Business justification:
>>>> *By giving the MQ Administrator this ability it would allow them to
>>>> speed up problem resolution and exonerate Websphere MQ from being the source of the issue.
>>>>
>>>>
>>>>
>>>>
>>>> *Peter Potkay*
>>>> Application Platform Engineering @ The Hartford Websphere MQ,
>>>> Message Broker and DataPower
>>>> (W) 860-624-1395/ (M) 860-202-1375
>>>>
>>>>
>>>>
>>>>
>>>> ************************************************************
>>>> This communication, including attachments, is for the exclusive use
>>>> of addressee and may contain proprietary, confidential and/or
>>>> privileged information. If you are not the intended recipient, any
>>>> use, copying, disclosure, dissemination or distribution is strictly
>>>> prohibited. If you are not the intended recipient, please notify
>>>> the sender immediately by return e-mail, delete this communication and destroy all copies.
>>>> ************************************************************
>>>>
>>>> --------------------------------------------------------------------
>>>> -
>>>> ----------- List Archive
>>>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
>>>> Your List Settings
>>>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>>>> Unsubscribe
>>>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=
>>>> s
>>>> ignoff%20mqseries>
>>>>
>>>>
>>>> Instructions for managing your mailing list subscription are
>>>> provided in the Listserv General Users Guide available at
>>>> http://www.lsoft.com <http://www.lsoft.com/resources/manuals.asp>
>>> ---------------------------------------------------------------------
>>> -
>>> ---------- List Archive
>>> <http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage
>>> Your List Settings
>>> <http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> -
>>> Unsubscribe
>>> <mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=s
>>> i
>>> gnoff%20mqseries>
>>>
>>>
>>> Instructions for managing your mailing list subscription are provided
>>> in the Listserv General Users Guide available at http://www.lsoft.com
>>> <http://www.lsoft.com/resources/manuals.asp>
>> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in
>> the message body (not the subject), write: SIGNOFF MQSERIES
>> Instructions for managing your mailing list subscription are provided
>> in the Listserv General Users Guide available at http://www.lsoft.com
>> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>> ************************************************************
>> This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
>> ************************************************************
>>
>> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in
>> the message body (not the subject), write: SIGNOFF MQSERIES
>> Instructions for managing your mailing list subscription are provided
>> in the Listserv General Users Guide available at http://www.lsoft.com
>> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>
> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>
> To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
> in the message body (not the subject), write: SIGNOFF MQSERIES
> Instructions for managing your mailing list subscription are provided in
> the Listserv General Users Guide available at http://www.lsoft.com
> Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Tim Zielke
2014-01-13 02:57:04 UTC
Permalink
Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>
________________________________



Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Andrew Hickson
2014-01-13 08:19:09 UTC
Permalink
If the JVM is simply sending STOP signals to threads without understanding
the context of those threads then it risks sending a STOP signal to an MQ
related thread while that thread holds an MQ lock, and thus freezing some
queue manager capabilities until the thread is resumed. Using ISOLATED MQ
bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense
of some performance.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous
enough"

My understanding is that when a JVM needs to run certain types of garbage
collection, the applicable JVM system thread sends a STOP signal to all
the application threads in order for the garbage collection to take place.
So it is possible if you have a long garbage collection scavenger run
that lasted 15 seconds in your JVM, all the application threads for that
JVM would receive a STOP signal and would subsequently be stopped for that
15 seconds. Does this mean that an MQ queue manager is somewhat
susceptible to what you are describing below if a server side java
application was connected to the queue manager, since my understanding is
JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>




Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Tim Zielke
2014-01-13 16:03:05 UTC
Permalink
Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>
________________________________



Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>
________________________________




Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Ian Alderson
2014-01-14 09:23:12 UTC
Permalink
Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian





Ian Alderson
MQ Technical Architect

[cid:***@ec5a282e.4c961517]

DL 0203 003 3055


________________________________
Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________

List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************
Andrew Hickson
2014-01-14 12:48:55 UTC
Permalink
There's an assertion here that Java garbage collection is blindly sending
SIGSTOP to threads in the same process regardless of their current state.
The article referenced by Tim doesn't go as far as to state this, and it's
well beyond my Java knowledge. All pre-emptive dispatching involves
potentially losing control for short periods of time while holding locks,
but the suggestion here is that threads could randomly be frozen for
prolonged periods of time, and in this environment then any hope of
meeting reasonable SLA's would seem improbable. We could really do with a
knowledgeable JVM internals expert chipping in at this time and explaining
the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter
process lock at some point, but the scope and granularity of the locks
varies massively, for example during a standard bound MQCONN a coarse lock
is obtained, which would inhibit all other standard bound MQCONN activity.
Conversely, inter process locks used in the application process during
MQPUT/MQGET are held for very few instructions and have a much smaller
scope.

Isolated bindings are independent of MQ client bindings, and do not use
amqrmppa processes. Isolated bindings use unix domain sockets and a thread
in an amqzlas0 process will exist in much the same way that a thread in an
amqzlaa0 process exists on behalf of a standard bound application. The
performance of isolated bindings is generally someplace between shared
binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



Andy,
That is a very interesting (and may I say slightly alarming) point about
JVM GC potentially freezing some part of a QMgr. There must be a lot of
JVMs out there that are connected using server bindings, and if this is a
potential issue then the window must be very very small or the effect so
slight that it is mostly not noticed. Having said that, the most common
implementation I have seen when an app server like WAS is co-located with
a qmgr is that the apps in WAS still connect using MQClient (with the
added benefit that HA failover of the QMgr is then independent of a WAS
cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ
using bindings mode, then during GC when it will send STOP signals to all
its threads, a lock on one or more IPC resources could prevent another
part of the qmgr from gaining said lock? What is the window for this
lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this
risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application
to stop using the shared IPC resources. In terms of performance, is it
fair to say that performance would be comparable to running a client
bindings on the same server (using the loopback address)? At this point I
am not clear if isolated bindings would use amqrmppa or some other process
in the absence of IPC. If my understanding is off the mark please do let
me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect



DL 0203 003 3055


Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side
java applications (like we do), below is a link to more information about
this Java garbage collection topic (in Java garbage collection terminology
it is called stop-the-world), in case you are interested. I would think
it would be helpful to at least be aware of its existence, and potential
(although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should
know before learning about GC. The term is "stop-the-world."
Stop-the-world will occur no matter which GC algorithm you choose.
Stop-the-world means that the JVM is stopping the application from running
to execute a GC. When stop-the-world occurs, every thread except for the
threads needed for the GC will stop their tasks. The interrupted tasks
will resume only after the GC task has completed. GC tuning often means
reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not
use any stop-the-world algorithms in its garbage collection, but it does
not look like this JVM is supported for WebSphere MQ according to the
manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding
the context of those threads then it risks sending a STOP signal to an MQ
related thread while that thread holds an MQ lock, and thus freezing some
queue manager capabilities until the thread is resumed. Using ISOLATED MQ
bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense
of some performance.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>




Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous
enough"

My understanding is that when a JVM needs to run certain types of garbage
collection, the applicable JVM system thread sends a STOP signal to all
the application threads in order for the garbage collection to take place.
So it is possible if you have a long garbage collection scavenger run
that lasted 15 seconds in your JVM, all the application threads for that
JVM would receive a STOP signal and would subsequently be stopped for that
15 seconds. Does this mean that an MQ queue manager is somewhat
susceptible to what you are describing below if a server side java
application was connected to the queue manager, since my understanding is
JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>





Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com






List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com






List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com



List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

**************************************************************
The information contained in this email (including any attachments
transmitted within it) is confidential and is intended solely for the use
of the named person.
The unauthorised access, copying or re-use of the information in it by any
other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by
return email to ***@ignisasset.com.

Internet communication is not guaranteed to be timely, secure, error or
virus free. We accept no liability for any harm to systems or data, nor
for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management
Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801),
Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish
Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000
and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No.
971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and
Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000.
Scottish Mutual is a registered trade mark of Scottish Mutual Assurance
Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Tim Zielke
2014-01-14 22:58:19 UTC
Permalink
I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>
________________________________



Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF1148.BCBBDE60]
DL 0203 003 3055

________________________________
Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Ian Alderson
2014-01-16 09:44:02 UTC
Permalink
Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF129A.D65CE730]
DL 0203 003 3055

________________________________
Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Tim Zielke
2014-01-16 14:26:31 UTC
Permalink
Hi Ian,

I agree with your assessment, as well. However, we have some MQ server side Java apps that are involved in performance SLAs. Switching to MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated binding performance is generally between shared and client bindings, per Andy. Assuming (and this definitely an assumption at this point) that there is the remote risk for significant queue manager slow downs due to a long running JVM stop-the-world garbage collection stopping a thread that is holding an inter-process MQ lock, it actually makes some sense for us to consider switching to MQCNO_ISOLATED_BINDING for our server side Java applications that use MQ. Anyway, I plan on following up with an IBM service request to hopefully get a more definitive answer on this topic.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF1292.BFE84830]
DL 0203 003 3055

________________________________
Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Andrew Hickson
2014-01-16 14:42:00 UTC
Permalink
The references to "safepoints" in the article below gives me hope that the
JVM doesn't blindly send SIGSTOP to other threads in the same process.
If an MQ process were to be stopped for a few seconds while holding an MQ
inter-process lock it is unlikely that anything would be recorded in the
MQ error logs (a few seconds delay could lead to too many false alarms).
If any 'exception' were reported it would be in the form of a long lock
wait FDC.

If you were advised to use ISOLATED bindings in relation to EXTSHM it's
likely to be related to the number of AIX segments potentially required to
connect 32-bit applications to MQ when EXTSHM is not enabled, and is
unlikely to be related to locking.



From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 16/01/2014 10:05
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



Tim and Andy, thanks for the extra information. I’m not overly concerned
about this potential issue but it’s certainly something to be aware of and
would be good to get a deeper understanding. From reading the responses
it would seem to me that the window of for GC to freeze a part of the qmgr
is (probably) small. But I could imagine you could increase the odds by
having an application running lots of threads and/or continually creating
lots of connections where the lock has a much larger scope (of course
creating a lot of unnecessary connections is generally regarding as bad
practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the
MQ error logs? I recall being advised by L3 support to set isolated
bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I
can’t remember the detail of the why, but I think it could have been
related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view,
especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a
J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and
although I would hope most code is now using JMS I am sure there will
still be some apps out there. The section on Thread Creation (and
cleanup) looks interesting as it might relate to how GC treats the MQ
threads.
Of course there will be lots of standalone java apps running the MQ base
classes legitimately, although I see more developers using JMS as standard
for these implementations these days, so would be interesting to know if
the potential for locking is inherently different when using either set of
classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

I did find this article that provides a little more information on the
topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the
running threads. Garbage collectors do this by signaling the threads to
stop when they come to a “safepoint”, which is a point during program
execution at which all GC roots are known and all heap object contents are
consistent. Depending on what a thread is doing it may take some time to
reach a safepoint. Safepoint checks are normally performed on method
returns and loop back edges, but can be optimized away in some places
making them more dynamically rare. For example, if a thread is copying a
large array, cloning a large object, or executing a monotonic counted loop
with a finite bound, it may be many milliseconds before a safepoint is
reached."

If there is a JVM internals expert out there (or someone knows one at IBM?
:-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending
SIGSTOP to threads in the same process regardless of their current state.
The article referenced by Tim doesn't go as far as to state this, and it's
well beyond my Java knowledge. All pre-emptive dispatching involves
potentially losing control for short periods of time while holding locks,
but the suggestion here is that threads could randomly be frozen for
prolonged periods of time, and in this environment then any hope of
meeting reasonable SLA's would seem improbable. We could really do with a
knowledgeable JVM internals expert chipping in at this time and explaining
the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter
process lock at some point, but the scope and granularity of the locks
varies massively, for example during a standard bound MQCONN a coarse lock
is obtained, which would inhibit all other standard bound MQCONN activity.
Conversely, inter process locks used in the application process during
MQPUT/MQGET are held for very few instructions and have a much smaller
scope.

Isolated bindings are independent of MQ client bindings, and do not use
amqrmppa processes. Isolated bindings use unix domain sockets and a thread
in an amqzlas0 process will exist in much the same way that a thread in an
amqzlaa0 process exists on behalf of a standard bound application. The
performance of isolated bindings is generally someplace between shared
binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>




Andy,
That is a very interesting (and may I say slightly alarming) point about
JVM GC potentially freezing some part of a QMgr. There must be a lot of
JVMs out there that are connected using server bindings, and if this is a
potential issue then the window must be very very small or the effect so
slight that it is mostly not noticed. Having said that, the most common
implementation I have seen when an app server like WAS is co-located with
a qmgr is that the apps in WAS still connect using MQClient (with the
added benefit that HA failover of the QMgr is then independent of a WAS
cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ
using bindings mode, then during GC when it will send STOP signals to all
its threads, a lock on one or more IPC resources could prevent another
part of the qmgr from gaining said lock? What is the window for this
lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this
risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application
to stop using the shared IPC resources. In terms of performance, is it
fair to say that performance would be comparable to running a client
bindings on the same server (using the loopback address)? At this point I
am not clear if isolated bindings would use amqrmppa or some other process
in the absence of IPC. If my understanding is off the mark please do let
me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect



DL 0203 003 3055


Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side
java applications (like we do), below is a link to more information about
this Java garbage collection topic (in Java garbage collection terminology
it is called stop-the-world), in case you are interested. I would think
it would be helpful to at least be aware of its existence, and potential
(although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/


Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should
know before learning about GC. The term is "stop-the-world."
Stop-the-world will occur no matter which GC algorithm you choose.
Stop-the-world means that the JVM is stopping the application from running
to execute a GC. When stop-the-world occurs, every thread except for the
threads needed for the GC will stop their tasks. The interrupted tasks
will resume only after the GC task has completed. GC tuning often means
reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not
use any stop-the-world algorithms in its garbage collection, but it does
not look like this JVM is supported for WebSphere MQ according to the
manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding
the context of those threads then it risks sending a STOP signal to an MQ
related thread while that thread holds an MQ lock, and thus freezing some
queue manager capabilities until the thread is resumed. Using ISOLATED MQ
bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense
of some performance.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>





Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous
enough"

My understanding is that when a JVM needs to run certain types of garbage
collection, the applicable JVM system thread sends a STOP signal to all
the application threads in order for the garbage collection to take place.
So it is possible if you have a long garbage collection scavenger run
that lasted 15 seconds in your JVM, all the application threads for that
JVM would receive a STOP signal and would subsequently be stopped for that
15 seconds. Does this mean that an MQ queue manager is somewhat
susceptible to what you are describing below if a server side java
application was connected to the queue manager, since my understanding is
JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>






Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com








List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com








List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments
transmitted within it) is confidential and is intended solely for the use
of the named person.
The unauthorised access, copying or re-use of the information in it by any
other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by
return email to ***@ignisasset.com.

Internet communication is not guaranteed to be timely, secure, error or
virus free. We accept no liability for any harm to systems or data, nor
for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management
Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801),
Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish
Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000
and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No.
971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and
Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000.
Scottish Mutual is a registered trade mark of Scottish Mutual Assurance
Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Ian Alderson
2014-01-16 14:53:00 UTC
Permalink
“If you were advised to use ISOLATED bindings in relation to EXTSHM it's likely to be related to the number of AIX segments potentially required to connect 32-bit applications to MQ when EXTSHM is not enabled, and is unlikely to be related to locking. “ - yes, that’s exactly what it was.

And thanks for the info on locking and logging.

Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, January 16, 2014 2:42 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

The references to "safepoints" in the article below gives me hope that the JVM doesn't blindly send SIGSTOP to other threads in the same process.
If an MQ process were to be stopped for a few seconds while holding an MQ inter-process lock it is unlikely that anything would be recorded in the MQ error logs (a few seconds delay could lead to too many false alarms). If any 'exception' were reported it would be in the form of a long lock wait FDC.

If you were advised to use ISOLATED bindings in relation to EXTSHM it's likely to be related to the number of AIX segments potentially required to connect 32-bit applications to MQ when EXTSHM is not enabled, and is unlikely to be related to locking.



From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 16/01/2014 10:05
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF12CA.A56272A0]
DL 0203 003 3055

________________________________

Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________






Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Tim Zielke
2014-01-21 15:45:42 UTC
Permalink
FYI, here was some information from my follow up service request. My take is that there is not much safe guarding from the IBM Linux JVM stand point during stop-the-world GC to prevent a Java thread that was holding an MQ inter-process lock from being "frozen" or stopped.

From IBM:
First of all, the Linux and Solaris implementations of GC are
completely different as the Linux version of Java is developed by IBM,
whereas the Solaris version comes from Oracle. Let me discuss the
Linux (IBM) implementation.

There are a number of GC policies which can be used in Java. In
Java 6 there are

optthruput - which aims to give the best throughput,
optavgpause - which sacrifices some performance to minimise GC
pause times
gencon - which aims to improve GC performance by concentrating
on young objects

The default for Java6 is optavgpause, but later versions use gencon
as it has proved to be benficial to many customers.

All the policies have some element of stop-the-world in them.
optthruput does all its work in this mode, while optavgpause does
a lot of work concurrently with Java work and then has a final
stop-the-world phase.

Thread suspension is done co-operatively. Java threads reach a safe
point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point.

The performance of GC can be monitored by adding -verbose:gc to
the Java options. The GCMV plugin to IBM Support Assistant enables
the output to be visualised in a variety of ways including highlighting
the pause times.

See http://www.ibm.com/developerworks/java/jdk/tools/gcmv/ for more
details.

The details of the Solaris GC and the available policies are different,
but I believe that fundamentals are the same.

Please see the memory Management section of the Java Diagnostics guide
at http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp for
more information about the IBM garbage collector.


My follow up question:
I did have one follow up on this comment:

"Thread suspension is done co-operatively. Java threads reach a
safe point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point."

When the java thread decides that a safe point has been reached, is it
taking into consideration in this safe point process if the thread
might be holding any locks? For example, if the java thread was
holding an inter-process lock that other processes on the server were
waiting on, this would cause a delay in the other processes until this
java thread is continued and is able to release the lock. So is the
safe point processing taking into consideration if any locks (i.e.
semaphores, mutexes, etc.) are being held by that the java thread
before it is suspended?


IBM respnonse:
The JVM only concerns itself with its own locks and not with any
non-Java locks such as semaphores or mutexes.


Thanks,
Tim

From: Tim Zielke
Sent: Thursday, January 16, 2014 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: RE: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Ian,

I agree with your assessment, as well. However, we have some MQ server side Java apps that are involved in performance SLAs. Switching to MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated binding performance is generally between shared and client bindings, per Andy. Assuming (and this definitely an assumption at this point) that there is the remote risk for significant queue manager slow downs due to a long running JVM stop-the-world garbage collection stopping a thread that is holding an inter-process MQ lock, it actually makes some sense for us to consider switching to MQCNO_ISOLATED_BINDING for our server side Java applications that use MQ. Anyway, I plan on following up with an IBM service request to hopefully get a more definitive answer on this topic.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF168C.8D734CD0]
DL 0203 003 3055

________________________________
Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________


List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Andrew Hickson
2014-01-21 16:19:37 UTC
Permalink
From an MQ internal perspective, no inter process MQ internal locks would
ever be held while executing in java, and hence no inter process MQ
internal locks could be held when a safepoint was reached, and hence no
internal MQ inter process locks can be affected by the gc.
Similarly, no internal MQ intra process native locks would ever be held
while executing in java, and the same argument also applies to these
locks.




From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 21/01/2014 15:46
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



FYI, here was some information from my follow up service request. My take
is that there is not much safe guarding from the IBM Linux JVM stand point
during stop-the-world GC to prevent a Java thread that was holding an MQ
inter-process lock from being "frozen" or stopped.

From IBM:
First of all, the Linux and Solaris implementations of GC are
completely different as the Linux version of Java is developed by IBM,
whereas the Solaris version comes from Oracle. Let me discuss the
Linux (IBM) implementation.

There are a number of GC policies which can be used in Java. In
Java 6 there are

optthruput - which aims to give the best throughput,
optavgpause - which sacrifices some performance to minimise GC
pause times
gencon - which aims to improve GC performance by concentrating
on young objects

The default for Java6 is optavgpause, but later versions use gencon
as it has proved to be benficial to many customers.

All the policies have some element of stop-the-world in them.
optthruput does all its work in this mode, while optavgpause does
a lot of work concurrently with Java work and then has a final
stop-the-world phase.

Thread suspension is done co-operatively. Java threads reach a safe
point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point.

The performance of GC can be monitored by adding -verbose:gc to
the Java options. The GCMV plugin to IBM Support Assistant enables
the output to be visualised in a variety of ways including highlighting
the pause times.

See http://www.ibm.com/developerworks/java/jdk/tools/gcmv/ for more
details.

The details of the Solaris GC and the available policies are different,
but I believe that fundamentals are the same.

Please see the memory Management section of the Java Diagnostics guide
at http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp for
more information about the IBM garbage collector.


My follow up question:
I did have one follow up on this comment:

"Thread suspension is done co-operatively. Java threads reach a
safe point and then suspend themselves. Native threads are not suspended

unless they need VM access, in which case they block at that point."

When the java thread decides that a safe point has been reached, is it
taking into consideration in this safe point process if the thread
might be holding any locks? For example, if the java thread was
holding an inter-process lock that other processes on the server were
waiting on, this would cause a delay in the other processes until this
java thread is continued and is able to release the lock. So is the
safe point processing taking into consideration if any locks (i.e.
semaphores, mutexes, etc.) are being held by that the java thread
before it is suspended?


IBM respnonse:
The JVM only concerns itself with its own locks and not with any
non-Java locks such as semaphores or mutexes.


Thanks,
Tim

From: Tim Zielke
Sent: Thursday, January 16, 2014 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: RE: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Ian,

I agree with your assessment, as well. However, we have some MQ server
side Java apps that are involved in performance SLAs. Switching to
MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated
binding performance is generally between shared and client bindings, per
Andy. Assuming (and this definitely an assumption at this point) that
there is the remote risk for significant queue manager slow downs due to a
long running JVM stop-the-world garbage collection stopping a thread that
is holding an inter-process MQ lock, it actually makes some sense for us
to consider switching to MQCNO_ISOLATED_BINDING for our server side Java
applications that use MQ. Anyway, I plan on following up with an IBM
service request to hopefully get a more definitive answer on this topic.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned
about this potential issue but it’s certainly something to be aware of and
would be good to get a deeper understanding. From reading the responses
it would seem to me that the window of for GC to freeze a part of the qmgr
is (probably) small. But I could imagine you could increase the odds by
having an application running lots of threads and/or continually creating
lots of connections where the lock has a much larger scope (of course
creating a lot of unnecessary connections is generally regarding as bad
practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the
MQ error logs? I recall being advised by L3 support to set isolated
bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I
can’t remember the detail of the why, but I think it could have been
related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view,
especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a
J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and
although I would hope most code is now using JMS I am sure there will
still be some apps out there. The section on Thread Creation (and
cleanup) looks interesting as it might relate to how GC treats the MQ
threads.
Of course there will be lots of standalone java apps running the MQ base
classes legitimately, although I see more developers using JMS as standard
for these implementations these days, so would be interesting to know if
the potential for locking is inherently different when using either set of
classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

I did find this article that provides a little more information on the
topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the
running threads. Garbage collectors do this by signaling the threads to
stop when they come to a “safepoint”, which is a point during program
execution at which all GC roots are known and all heap object contents are
consistent. Depending on what a thread is doing it may take some time to
reach a safepoint. Safepoint checks are normally performed on method
returns and loop back edges, but can be optimized away in some places
making them more dynamically rare. For example, if a thread is copying a
large array, cloning a large object, or executing a monotonic counted loop
with a finite bound, it may be many milliseconds before a safepoint is
reached."

If there is a JVM internals expert out there (or someone knows one at IBM?
:-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending
SIGSTOP to threads in the same process regardless of their current state.
The article referenced by Tim doesn't go as far as to state this, and it's
well beyond my Java knowledge. All pre-emptive dispatching involves
potentially losing control for short periods of time while holding locks,
but the suggestion here is that threads could randomly be frozen for
prolonged periods of time, and in this environment then any hope of
meeting reasonable SLA's would seem improbable. We could really do with a
knowledgeable JVM internals expert chipping in at this time and explaining
the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter
process lock at some point, but the scope and granularity of the locks
varies massively, for example during a standard bound MQCONN a coarse lock
is obtained, which would inhibit all other standard bound MQCONN activity.
Conversely, inter process locks used in the application process during
MQPUT/MQGET are held for very few instructions and have a much smaller
scope.

Isolated bindings are independent of MQ client bindings, and do not use
amqrmppa processes. Isolated bindings use unix domain sockets and a thread
in an amqzlas0 process will exist in much the same way that a thread in an
amqzlaa0 process exists on behalf of a standard bound application. The
performance of isolated bindings is generally someplace between shared
binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>




Andy,
That is a very interesting (and may I say slightly alarming) point about
JVM GC potentially freezing some part of a QMgr. There must be a lot of
JVMs out there that are connected using server bindings, and if this is a
potential issue then the window must be very very small or the effect so
slight that it is mostly not noticed. Having said that, the most common
implementation I have seen when an app server like WAS is co-located with
a qmgr is that the apps in WAS still connect using MQClient (with the
added benefit that HA failover of the QMgr is then independent of a WAS
cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ
using bindings mode, then during GC when it will send STOP signals to all
its threads, a lock on one or more IPC resources could prevent another
part of the qmgr from gaining said lock? What is the window for this
lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this
risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application
to stop using the shared IPC resources. In terms of performance, is it
fair to say that performance would be comparable to running a client
bindings on the same server (using the loopback address)? At this point I
am not clear if isolated bindings would use amqrmppa or some other process
in the absence of IPC. If my understanding is off the mark please do let
me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect



DL 0203 003 3055


Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side
java applications (like we do), below is a link to more information about
this Java garbage collection topic (in Java garbage collection terminology
it is called stop-the-world), in case you are interested. I would think
it would be helpful to at least be aware of its existence, and potential
(although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/


Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should
know before learning about GC. The term is "stop-the-world."
Stop-the-world will occur no matter which GC algorithm you choose.
Stop-the-world means that the JVM is stopping the application from running
to execute a GC. When stop-the-world occurs, every thread except for the
threads needed for the GC will stop their tasks. The interrupted tasks
will resume only after the GC task has completed. GC tuning often means
reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not
use any stop-the-world algorithms in its garbage collection, but it does
not look like this JVM is supported for WebSphere MQ according to the
manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding
the context of those threads then it risks sending a STOP signal to an MQ
related thread while that thread holds an MQ lock, and thus freezing some
queue manager capabilities until the thread is resumed. Using ISOLATED MQ
bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense
of some performance.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>





Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous
enough"

My understanding is that when a JVM needs to run certain types of garbage
collection, the applicable JVM system thread sends a STOP signal to all
the application threads in order for the garbage collection to take place.
So it is possible if you have a long garbage collection scavenger run
that lasted 15 seconds in your JVM, all the application threads for that
JVM would receive a STOP signal and would subsequently be stopped for that
15 seconds. Does this mean that an MQ queue manager is somewhat
susceptible to what you are describing below if a server side java
application was connected to the queue manager, since my understanding is
JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>






Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com








List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com








List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments
transmitted within it) is confidential and is intended solely for the use
of the named person.
The unauthorised access, copying or re-use of the information in it by any
other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by
return email to ***@ignisasset.com.

Internet communication is not guaranteed to be timely, secure, error or
virus free. We accept no liability for any harm to systems or data, nor
for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management
Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801),
Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish
Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000
and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No.
971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and
Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000.
Scottish Mutual is a registered trade mark of Scottish Mutual Assurance
Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Tim Zielke
2014-01-21 16:58:33 UTC
Permalink
Ok, that makes sense. Thanks for clarifying!

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 21, 2014 10:20 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

From an MQ internal perspective, no inter process MQ internal locks would ever be held while executing in java, and hence no inter process MQ internal locks could be held when a safepoint was reached, and hence no internal MQ inter process locks can be affected by the gc.
Similarly, no internal MQ intra process native locks would ever be held while executing in java, and the same argument also applies to these locks.




From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 21/01/2014 15:46
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>
________________________________



FYI, here was some information from my follow up service request. My take is that there is not much safe guarding from the IBM Linux JVM stand point during stop-the-world GC to prevent a Java thread that was holding an MQ inter-process lock from being "frozen" or stopped.

From IBM:
First of all, the Linux and Solaris implementations of GC are
completely different as the Linux version of Java is developed by IBM,
whereas the Solaris version comes from Oracle. Let me discuss the
Linux (IBM) implementation.

There are a number of GC policies which can be used in Java. In
Java 6 there are

optthruput - which aims to give the best throughput,
optavgpause - which sacrifices some performance to minimise GC
pause times
gencon - which aims to improve GC performance by concentrating
on young objects

The default for Java6 is optavgpause, but later versions use gencon
as it has proved to be benficial to many customers.

All the policies have some element of stop-the-world in them.
optthruput does all its work in this mode, while optavgpause does
a lot of work concurrently with Java work and then has a final
stop-the-world phase.

Thread suspension is done co-operatively. Java threads reach a safe
point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point.

The performance of GC can be monitored by adding -verbose:gc to
the Java options. The GCMV plugin to IBM Support Assistant enables
the output to be visualised in a variety of ways including highlighting
the pause times.

See http://www.ibm.com/developerworks/java/jdk/tools/gcmv/ for more
details.

The details of the Solaris GC and the available policies are different,
but I believe that fundamentals are the same.

Please see the memory Management section of the Java Diagnostics guide
at http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp for
more information about the IBM garbage collector.


My follow up question:
I did have one follow up on this comment:

"Thread suspension is done co-operatively. Java threads reach a
safe point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point."

When the java thread decides that a safe point has been reached, is it
taking into consideration in this safe point process if the thread
might be holding any locks? For example, if the java thread was
holding an inter-process lock that other processes on the server were
waiting on, this would cause a delay in the other processes until this
java thread is continued and is able to release the lock. So is the
safe point processing taking into consideration if any locks (i.e.
semaphores, mutexes, etc.) are being held by that the java thread
before it is suspended?


IBM respnonse:
The JVM only concerns itself with its own locks and not with any
non-Java locks such as semaphores or mutexes.


Thanks,
Tim

From: Tim Zielke
Sent: Thursday, January 16, 2014 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: RE: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Ian,

I agree with your assessment, as well. However, we have some MQ server side Java apps that are involved in performance SLAs. Switching to MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated binding performance is generally between shared and client bindings, per Andy. Assuming (and this definitely an assumption at this point) that there is the remote risk for significant queue manager slow downs due to a long running JVM stop-the-world garbage collection stopping a thread that is holding an inter-process MQ lock, it actually makes some sense for us to consider switching to MQCNO_ISOLATED_BINDING for our server side Java applications that use MQ. Anyway, I plan on following up with an IBM service request to hopefully get a more definitive answer on this topic.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF1697.B8EDAAD0]
DL 0203 003 3055

________________________________

Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________






Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Ian Alderson
2014-01-22 09:55:10 UTC
Permalink
Tim,
Thank you so much for posting the follow-up from the PMR.

Andy,
I take it from your response that there seems to be no risk of the GC locking any MQ IPC resources when using standard bindings mode. Which is good news!
But at the risk of flogging a dead horse, can I just clarify your statement about no MQ locks being made in java and how that applies to calls made using the the JNI native interface (i.e. done through the .so file when using bindings mode)? I only ask because the response in the PMR states
“Native threads are not suspended unless they need VM access, in which case they block at that point.”

I’m sure you’ve considered that in your answer, but as we’ve come this far into diving deep into the GC, I’d just like to remove that last tiny doubt in my mind.

Thanks,
Ian
From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 21, 2014 4:20 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

From an MQ internal perspective, no inter process MQ internal locks would ever be held while executing in java, and hence no inter process MQ internal locks could be held when a safepoint was reached, and hence no internal MQ inter process locks can be affected by the gc.
Similarly, no internal MQ intra process native locks would ever be held while executing in java, and the same argument also applies to these locks.




From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 21/01/2014 15:46
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



FYI, here was some information from my follow up service request. My take is that there is not much safe guarding from the IBM Linux JVM stand point during stop-the-world GC to prevent a Java thread that was holding an MQ inter-process lock from being "frozen" or stopped.

From IBM:
First of all, the Linux and Solaris implementations of GC are
completely different as the Linux version of Java is developed by IBM,
whereas the Solaris version comes from Oracle. Let me discuss the
Linux (IBM) implementation.

There are a number of GC policies which can be used in Java. In
Java 6 there are

optthruput - which aims to give the best throughput,
optavgpause - which sacrifices some performance to minimise GC
pause times
gencon - which aims to improve GC performance by concentrating
on young objects

The default for Java6 is optavgpause, but later versions use gencon
as it has proved to be benficial to many customers.

All the policies have some element of stop-the-world in them.
optthruput does all its work in this mode, while optavgpause does
a lot of work concurrently with Java work and then has a final
stop-the-world phase.

Thread suspension is done co-operatively. Java threads reach a safe
point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point.

The performance of GC can be monitored by adding -verbose:gc to
the Java options. The GCMV plugin to IBM Support Assistant enables
the output to be visualised in a variety of ways including highlighting
the pause times.

See http://www.ibm.com/developerworks/java/jdk/tools/gcmv/ for more
details.

The details of the Solaris GC and the available policies are different,
but I believe that fundamentals are the same.

Please see the memory Management section of the Java Diagnostics guide
at http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp for
more information about the IBM garbage collector.


My follow up question:
I did have one follow up on this comment:

"Thread suspension is done co-operatively. Java threads reach a
safe point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point."

When the java thread decides that a safe point has been reached, is it
taking into consideration in this safe point process if the thread
might be holding any locks? For example, if the java thread was
holding an inter-process lock that other processes on the server were
waiting on, this would cause a delay in the other processes until this
java thread is continued and is able to release the lock. So is the
safe point processing taking into consideration if any locks (i.e.
semaphores, mutexes, etc.) are being held by that the java thread
before it is suspended?


IBM respnonse:
The JVM only concerns itself with its own locks and not with any
non-Java locks such as semaphores or mutexes.


Thanks,
Tim

From: Tim Zielke
Sent: Thursday, January 16, 2014 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: RE: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Ian,

I agree with your assessment, as well. However, we have some MQ server side Java apps that are involved in performance SLAs. Switching to MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated binding performance is generally between shared and client bindings, per Andy. Assuming (and this definitely an assumption at this point) that there is the remote risk for significant queue manager slow downs due to a long running JVM stop-the-world garbage collection stopping a thread that is holding an inter-process MQ lock, it actually makes some sense for us to consider switching to MQCNO_ISOLATED_BINDING for our server side Java applications that use MQ. Anyway, I plan on following up with an IBM service request to hopefully get a more definitive answer on this topic.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF1751.A9BCB460]
DL 0203 003 3055

________________________________

Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________






Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________



List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Russell Finn
2014-01-22 16:47:31 UTC
Permalink
I checked with an IBM JVM developer who told me that the IBM J9 VM
definitely does not use SIGSTOP when preparing to do java garbage
collection. JVMs not implemented by IBM may behave differently.

Garbage collection is triggered when running code requests a new Java
object that does not fit in the current heap space, or calls a gc()
method. Either way, the running thread has handed over control to the JVM
at that point. The JVM does not randomly stop a running thread.


Russell Finn
russell_finn-ygUJEDcBm8rQT0dZR+***@public.gmane.org





From: Tim Zielke <tim.zielke-PR+tvw7B/***@public.gmane.org>
To: MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org,
Date: 14/01/2014 22:58
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org>



I did find this article that provides a little more information on the
topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the
running threads. Garbage collectors do this by signaling the threads to
stop when they come to a ?safepoint?, which is a point during program
execution at which all GC roots are known and all heap object contents are
consistent. Depending on what a thread is doing it may take some time to
reach a safepoint. Safepoint checks are normally performed on method
returns and loop back edges, but can be optimized away in some places
making them more dynamically rare. For example, if a thread is copying a
large array, cloning a large object, or executing a monotonic counted loop
with a finite bound, it may be many milliseconds before a safepoint is
reached."

If there is a JVM internals expert out there (or someone knows one at IBM?
:-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending
SIGSTOP to threads in the same process regardless of their current state.
The article referenced by Tim doesn't go as far as to state this, and it's
well beyond my Java knowledge. All pre-emptive dispatching involves
potentially losing control for short periods of time while holding locks,
but the suggestion here is that threads could randomly be frozen for
prolonged periods of time, and in this environment then any hope of
meeting reasonable SLA's would seem improbable. We could really do with a
knowledgeable JVM internals expert chipping in at this time and explaining
the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter
process lock at some point, but the scope and granularity of the locks
varies massively, for example during a standard bound MQCONN a coarse lock
is obtained, which would inhibit all other standard bound MQCONN activity.
Conversely, inter process locks used in the application process during
MQPUT/MQGET are held for very few instructions and have a much smaller
scope.

Isolated bindings are independent of MQ client bindings, and do not use
amqrmppa processes. Isolated bindings use unix domain sockets and a thread
in an amqzlas0 process will exist in much the same way that a thread in an
amqzlaa0 process exists on behalf of a standard bound application. The
performance of isolated bindings is generally someplace between shared
binding and client bindings.




From: Ian Alderson <Ian.Alderson-bxmlfxBpev1m9/***@public.gmane.org>
To: MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org>




Andy,
That is a very interesting (and may I say slightly alarming) point about
JVM GC potentially freezing some part of a QMgr. There must be a lot of
JVMs out there that are connected using server bindings, and if this is a
potential issue then the window must be very very small or the effect so
slight that it is mostly not noticed. Having said that, the most common
implementation I have seen when an app server like WAS is co-located with
a qmgr is that the apps in WAS still connect using MQClient (with the
added benefit that HA failover of the QMgr is then independent of a WAS
cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ
using bindings mode, then during GC when it will send STOP signals to all
its threads, a lock on one or more IPC resources could prevent another
part of the qmgr from gaining said lock? What is the window for this
lock, on a connect of pretty much any MQ operation?

Also
?Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this
risk at the expense of some performance?
I believe using ISOLATED bindings will effectively force the application
to stop using the shared IPC resources. In terms of performance, is it
fair to say that performance would be comparable to running a client
bindings on the same server (using the loopback address)? At this point I
am not clear if isolated bindings would use amqrmppa or some other process
in the absence of IPC. If my understanding is off the mark please do let
me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect



DL 0203 003 3055


Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side
java applications (like we do), below is a link to more information about
this Java garbage collection topic (in Java garbage collection terminology
it is called stop-the-world), in case you are interested. I would think
it would be helpful to at least be aware of its existence, and potential
(although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/


Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should
know before learning about GC. The term is "stop-the-world."
Stop-the-world will occur no matter which GC algorithm you choose.
Stop-the-world means that the JVM is stopping the application from running
to execute a GC. When stop-the-world occurs, every thread except for the
threads needed for the GC will stop their tasks. The interrupted tasks
will resume only after the GC task has completed. GC tuning often means
reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not
use any stop-the-world algorithms in its garbage collection, but it does
not look like this JVM is supported for WebSphere MQ according to the
manual.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding
the context of those threads then it risks sending a STOP signal to an MQ
related thread while that thread holds an MQ lock, and thus freezing some
queue manager capabilities until the thread is resumed. Using ISOLATED MQ
bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense
of some performance.



From: Tim Zielke <tim.zielke-PR+tvw7B/***@public.gmane.org>
To: MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org>





Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous
enough"

My understanding is that when a JVM needs to run certain types of garbage
collection, the applicable JVM system thread sends a STOP signal to all
the application threads in order for the garbage collection to take place.
So it is possible if you have a long garbage collection scavenger run
that lasted 15 seconds in your JVM, all the application threads for that
JVM would receive a STOP signal and would subsequently be stopped for that
15 seconds. Does this mean that an MQ queue manager is somewhat
susceptible to what you are describing below if a server side java
application was connected to the queue manager, since my understanding is
JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <tim.zielke-PR+tvw7B/***@public.gmane.org>
To: MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <MQSERIES-JX7+OpRa80QeFbOYke1v4oOpTq8/***@public.gmane.org>






Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That?s slick, but?.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ?hang? until the PID was continued?

Nevertheless your email is getting saved and I?m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I?m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com








List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com








List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments
transmitted within it) is confidential and is intended solely for the use
of the named person.
The unauthorised access, copying or re-use of the information in it by any
other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by
return email to postmaster-ddK+***@public.gmane.org

Internet communication is not guaranteed to be timely, secure, error or
virus free. We accept no liability for any harm to systems or data, nor
for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management
Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801),
Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish
Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000
and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No.
971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and
Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000.
Scottish Mutual is a registered trade mark of Scottish Mutual Assurance
Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Andrew Hickson
2014-01-23 12:03:44 UTC
Permalink
When a 'java' thread calls into the MQ JNI then any locks taken within
that JNI call will be released before the JNI call completes.
A 'native' thread using MQ code in a process with a JVM loaded will never
invoke code in the JVM while owning an inter process lock (or any
intra-process lock which could be requested by the owner of an
inter-process lock).
From what I've seen in this discussion I think we're safe at the MQ system
level. At an application level it seems you have to worry about the JVM
freezing while GC takes place (for example a message could be locked to an
application thread in a JVM), but I guess java programmers must be used to
that.



From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 22/01/2014 09:55
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>



Tim,
Thank you so much for posting the follow-up from the PMR.

Andy,
I take it from your response that there seems to be no risk of the GC
locking any MQ IPC resources when using standard bindings mode. Which is
good news!
But at the risk of flogging a dead horse, can I just clarify your
statement about no MQ locks being made in java and how that applies to
calls made using the the JNI native interface (i.e. done through the .so
file when using bindings mode)? I only ask because the response in the
PMR states
“Native threads are not suspended unless they need VM access, in which
case they block at that point.”

I’m sure you’ve considered that in your answer, but as we’ve come this far
into diving deep into the GC, I’d just like to remove that last tiny doubt
in my mind.

Thanks,
Ian
From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Tuesday, January 21, 2014 4:20 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

From an MQ internal perspective, no inter process MQ internal locks would
ever be held while executing in java, and hence no inter process MQ
internal locks could be held when a safepoint was reached, and hence no
internal MQ inter process locks can be affected by the gc.
Similarly, no internal MQ intra process native locks would ever be held
while executing in java, and the same argument also applies to these
locks.




From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 21/01/2014 15:46
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>




FYI, here was some information from my follow up service request. My take
is that there is not much safe guarding from the IBM Linux JVM stand point
during stop-the-world GC to prevent a Java thread that was holding an MQ
inter-process lock from being "frozen" or stopped.

From IBM:
First of all, the Linux and Solaris implementations of GC are
completely different as the Linux version of Java is developed by IBM,
whereas the Solaris version comes from Oracle. Let me discuss the
Linux (IBM) implementation.

There are a number of GC policies which can be used in Java. In
Java 6 there are

optthruput - which aims to give the best throughput,
optavgpause - which sacrifices some performance to minimise GC
pause times
gencon - which aims to improve GC performance by concentrating
on young objects

The default for Java6 is optavgpause, but later versions use gencon
as it has proved to be benficial to many customers.

All the policies have some element of stop-the-world in them.
optthruput does all its work in this mode, while optavgpause does
a lot of work concurrently with Java work and then has a final
stop-the-world phase.

Thread suspension is done co-operatively. Java threads reach a safe
point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point.

The performance of GC can be monitored by adding -verbose:gc to
the Java options. The GCMV plugin to IBM Support Assistant enables
the output to be visualised in a variety of ways including highlighting
the pause times.

See http://www.ibm.com/developerworks/java/jdk/tools/gcmv/ for more
details.

The details of the Solaris GC and the available policies are different,
but I believe that fundamentals are the same.

Please see the memory Management section of the Java Diagnostics guide
at http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp for
more information about the IBM garbage collector.


My follow up question:
I did have one follow up on this comment:

"Thread suspension is done co-operatively. Java threads reach a
safe point and then suspend themselves. Native threads are not suspended

unless they need VM access, in which case they block at that point."

When the java thread decides that a safe point has been reached, is it
taking into consideration in this safe point process if the thread
might be holding any locks? For example, if the java thread was
holding an inter-process lock that other processes on the server were
waiting on, this would cause a delay in the other processes until this
java thread is continued and is able to release the lock. So is the
safe point processing taking into consideration if any locks (i.e.
semaphores, mutexes, etc.) are being held by that the java thread
before it is suspended?


IBM respnonse:
The JVM only concerns itself with its own locks and not with any
non-Java locks such as semaphores or mutexes.


Thanks,
Tim

From: Tim Zielke
Sent: Thursday, January 16, 2014 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: RE: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Ian,

I agree with your assessment, as well. However, we have some MQ server
side Java apps that are involved in performance SLAs. Switching to
MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated
binding performance is generally between shared and client bindings, per
Andy. Assuming (and this definitely an assumption at this point) that
there is the remote risk for significant queue manager slow downs due to a
long running JVM stop-the-world garbage collection stopping a thread that
is holding an inter-process MQ lock, it actually makes some sense for us
to consider switching to MQCNO_ISOLATED_BINDING for our server side Java
applications that use MQ. Anyway, I plan on following up with an IBM
service request to hopefully get a more definitive answer on this topic.


Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned
about this potential issue but it’s certainly something to be aware of and
would be good to get a deeper understanding. From reading the responses
it would seem to me that the window of for GC to freeze a part of the qmgr
is (probably) small. But I could imagine you could increase the odds by
having an application running lots of threads and/or continually creating
lots of connections where the lock has a much larger scope (of course
creating a lot of unnecessary connections is generally regarding as bad
practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the
MQ error logs? I recall being advised by L3 support to set isolated
bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I
can’t remember the detail of the why, but I think it could have been
related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view,
especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a
J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and
although I would hope most code is now using JMS I am sure there will
still be some apps out there. The section on Thread Creation (and
cleanup) looks interesting as it might relate to how GC treats the MQ
threads.
Of course there will be lots of standalone java apps running the MQ base
classes legitimately, although I see more developers using JMS as standard
for these implementations these days, so would be interesting to know if
the potential for locking is inherently different when using either set of
classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

I did find this article that provides a little more information on the
topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the
running threads. Garbage collectors do this by signaling the threads to
stop when they come to a “safepoint”, which is a point during program
execution at which all GC roots are known and all heap object contents are
consistent. Depending on what a thread is doing it may take some time to
reach a safepoint. Safepoint checks are normally performed on method
returns and loop back edges, but can be optimized away in some places
making them more dynamically rare. For example, if a thread is copying a
large array, cloning a large object, or executing a monotonic counted loop
with a finite bound, it may be many milliseconds before a safepoint is
reached."

If there is a JVM internals expert out there (or someone knows one at IBM?
:-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending
SIGSTOP to threads in the same process regardless of their current state.
The article referenced by Tim doesn't go as far as to state this, and it's
well beyond my Java knowledge. All pre-emptive dispatching involves
potentially losing control for short periods of time while holding locks,
but the suggestion here is that threads could randomly be frozen for
prolonged periods of time, and in this environment then any hope of
meeting reasonable SLA's would seem improbable. We could really do with a
knowledgeable JVM internals expert chipping in at this time and explaining
the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter
process lock at some point, but the scope and granularity of the locks
varies massively, for example during a standard bound MQCONN a coarse lock
is obtained, which would inhibit all other standard bound MQCONN activity.
Conversely, inter process locks used in the application process during
MQPUT/MQGET are held for very few instructions and have a much smaller
scope.

Isolated bindings are independent of MQ client bindings, and do not use
amqrmppa processes. Isolated bindings use unix domain sockets and a thread
in an amqzlas0 process will exist in much the same way that a thread in an
amqzlaa0 process exists on behalf of a standard bound application. The
performance of isolated bindings is generally someplace between shared
binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>





Andy,
That is a very interesting (and may I say slightly alarming) point about
JVM GC potentially freezing some part of a QMgr. There must be a lot of
JVMs out there that are connected using server bindings, and if this is a
potential issue then the window must be very very small or the effect so
slight that it is mostly not noticed. Having said that, the most common
implementation I have seen when an app server like WAS is co-located with
a qmgr is that the apps in WAS still connect using MQClient (with the
added benefit that HA failover of the QMgr is then independent of a WAS
cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ
using bindings mode, then during GC when it will send STOP signals to all
its threads, a lock on one or more IPC resources could prevent another
part of the qmgr from gaining said lock? What is the window for this
lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this
risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application
to stop using the shared IPC resources. In terms of performance, is it
fair to say that performance would be comparable to running a client
bindings on the same server (using the loopback address)? At this point I
am not clear if isolated bindings would use amqrmppa or some other process
in the absence of IPC. If my understanding is off the mark please do let
me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect



DL 0203 003 3055



Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side
java applications (like we do), below is a link to more information about
this Java garbage collection topic (in Java garbage collection terminology
it is called stop-the-world), in case you are interested. I would think
it would be helpful to at least be aware of its existence, and potential
(although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/


Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should
know before learning about GC. The term is "stop-the-world."
Stop-the-world will occur no matter which GC algorithm you choose.
Stop-the-world means that the JVM is stopping the application from running
to execute a GC. When stop-the-world occurs, every thread except for the
threads needed for the GC will stop their tasks. The interrupted tasks
will resume only after the GC task has completed. GC tuning often means
reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not
use any stop-the-world algorithms in its garbage collection, but it does
not look like this JVM is supported for WebSphere MQ according to the
manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding
the context of those threads then it risks sending a STOP signal to an MQ
related thread while that thread holds an MQ lock, and thus freezing some
queue manager capabilities until the thread is resumed. Using ISOLATED MQ
bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense
of some performance.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>






Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous
enough"

My understanding is that when a JVM needs to run certain types of garbage
collection, the applicable JVM system thread sends a STOP signal to all
the application threads in order for the garbage collection to take place.
So it is possible if you have a long garbage collection scavenger run
that lasted 15 seconds in your JVM, all the application threads for that
JVM would receive a STOP signal and would subsequently be stopped for that
15 seconds. Does this mean that an MQ queue manager is somewhat
susceptible to what you are describing below if a server side java
application was connected to the queue manager, since my understanding is
JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Sending a STOP signal to an MQ server side application is dangerous
enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be
extremely dangerous. If that process happened to own an inter process lock
at the time the STOP signal arrived the entire queue manager could easily
grind to a halt. The same issue does arise for a 'simple' server side
application (for example queue manager scope locks are obtained/released
during MQCONN processing), but the scope for inadvertently stopping the
queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the
API calls for specific connections would seem like a much better way to
identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM>
To: ***@listserv.meduniwien.ac.at,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse
a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at>







Yes, sending the STOP signal to a process will STOP all the threads for
that process.

This is what I would do for your case. My understanding now is you have
an isolated queue manager on its own server with MQ Client apps coming in
from different servers to PUT and GET messages to this queue manager. If
so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for
that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for
that queue manager

In my opinion, this type of activity would only be appropriate in a
Development lifecycle, like yours. But it shouldn't negatively impact the
queue manager besides temporarily stalling the channel activity, in my
opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In
the case of MQ Client, if you freeze the PID that represents the app
consuming from the queue, I imagine you are actually freezing the PID of
amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa
would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with
that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you
might want to consider for your use case for future reference since this
is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL
signal and the STOP signal. The STOP signal will "freeze" a process from
running any longer until you send the process a subsequent CONT or
continue singal. So basically, instead of GET inhibiting the queue, you
are run inhibiting the active application processes that can GET from the
queue. Then you should be able to browse or GET your new message from the
queue, and then send the CONT signal to the consuming applications to make
them runnable again.

Here is an example. You may be already aware of how to do this, but it
might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you
want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a
signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was
in a Get with Wait loop I’m thinking the change in security would not give
the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily
change the security on the queue that the consuming message is doing a GET
on (with a subsequent REFRESH SECURITY) so that only mqm has access to the
queue (assuming the consuming message is not leveraging the mqm group for
its security)? I would think the running consuming application would pick
up the security change after the REFRESH SECURITY and then get 2035s until
the MQ Admin has browsed the message. Then you could turn the original
application security for the queue back on followed with another REFRESH
SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf
Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Request For Enhancement - Allow an MQ Admin to browse a get
inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656



Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm
group) can browse a queue even if that queue is get inhibited. The ability
to browse would be enough for our use cases, but it might be beneficial to
also allow the MQ Admin to destructively get the message(s) on the Get
Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to
each other are having a bad day, each laying blame on the other as to what
is actually being sent to the other. Classic he sent, she sent. Or, you're
not gonna believe this, they claim that MQ is changing the content of the
message. So we tell the consuming app to close the queue and then we tell
the sending app to produce another message but only when we tell them the
consuming app has finally managed to stop, at which point we can look at
it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the
queue, we could work the issue without needing the consuming application
to stop. The MQ Admin could grab a copy of the message, enable the queue
for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might
be to allow an MQ Admin to step in and save the day when an app is stuck
in a tight loop on a poison message. Without getting the consuming app to
stop, the MQ Admin could get inhibit the queue, the poison message
identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed
up problem resolution and exonerate Websphere MQ from being the source of
the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375



************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************






List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com










List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************






List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com










List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
************************************************************
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential and/or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If you
are not the intended recipient, please notify the sender immediately by
return e-mail, delete this communication and destroy all copies.
************************************************************






List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com





List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments
transmitted within it) is confidential and is intended solely for the use
of the named person.
The unauthorised access, copying or re-use of the information in it by any
other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by
return email to ***@ignisasset.com.

Internet communication is not guaranteed to be timely, secure, error or
virus free. We accept no liability for any harm to systems or data, nor
for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management
Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801),
Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish
Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000
and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No.
971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and
Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000.
Scottish Mutual is a registered trade mark of Scottish Mutual Assurance
Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Ian Alderson
2014-01-23 13:37:17 UTC
Permalink
Andy,
That’s a perfect explanation to close out any lingering concerns for me. Thanks for the clarification.

Russell, also thank you for your reply.

Cheers,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, January 23, 2014 12:04 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

When a 'java' thread calls into the MQ JNI then any locks taken within that JNI call will be released before the JNI call completes.
A 'native' thread using MQ code in a process with a JVM loaded will never invoke code in the JVM while owning an inter process lock (or any intra-process lock which could be requested by the owner of an inter-process lock).
From what I've seen in this discussion I think we're safe at the MQ system level. At an application level it seems you have to worry about the JVM freezing while GC takes place (for example a message could be locked to an application thread in a JVM), but I guess java programmers must be used to that.



From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 22/01/2014 09:55
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________



Tim,
Thank you so much for posting the follow-up from the PMR.

Andy,
I take it from your response that there seems to be no risk of the GC locking any MQ IPC resources when using standard bindings mode. Which is good news!
But at the risk of flogging a dead horse, can I just clarify your statement about no MQ locks being made in java and how that applies to calls made using the the JNI native interface (i.e. done through the .so file when using bindings mode)? I only ask because the response in the PMR states
“Native threads are not suspended unless they need VM access, in which case they block at that point.”

I’m sure you’ve considered that in your answer, but as we’ve come this far into diving deep into the GC, I’d just like to remove that last tiny doubt in my mind.

Thanks,
Ian
From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 21, 2014 4:20 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

From an MQ internal perspective, no inter process MQ internal locks would ever be held while executing in java, and hence no inter process MQ internal locks could be held when a safepoint was reached, and hence no internal MQ inter process locks can be affected by the gc.
Similarly, no internal MQ intra process native locks would ever be held while executing in java, and the same argument also applies to these locks.




From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 21/01/2014 15:46
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________




FYI, here was some information from my follow up service request. My take is that there is not much safe guarding from the IBM Linux JVM stand point during stop-the-world GC to prevent a Java thread that was holding an MQ inter-process lock from being "frozen" or stopped.

From IBM:
First of all, the Linux and Solaris implementations of GC are
completely different as the Linux version of Java is developed by IBM,
whereas the Solaris version comes from Oracle. Let me discuss the
Linux (IBM) implementation.

There are a number of GC policies which can be used in Java. In
Java 6 there are

optthruput - which aims to give the best throughput,
optavgpause - which sacrifices some performance to minimise GC
pause times
gencon - which aims to improve GC performance by concentrating
on young objects

The default for Java6 is optavgpause, but later versions use gencon
as it has proved to be benficial to many customers.

All the policies have some element of stop-the-world in them.
optthruput does all its work in this mode, while optavgpause does
a lot of work concurrently with Java work and then has a final
stop-the-world phase.

Thread suspension is done co-operatively. Java threads reach a safe
point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point.

The performance of GC can be monitored by adding -verbose:gc to
the Java options. The GCMV plugin to IBM Support Assistant enables
the output to be visualised in a variety of ways including highlighting
the pause times.

See http://www.ibm.com/developerworks/java/jdk/tools/gcmv/ for more
details.

The details of the Solaris GC and the available policies are different,
but I believe that fundamentals are the same.

Please see the memory Management section of the Java Diagnostics guide
at http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp for
more information about the IBM garbage collector.


My follow up question:
I did have one follow up on this comment:

"Thread suspension is done co-operatively. Java threads reach a
safe point and then suspend themselves. Native threads are not suspended
unless they need VM access, in which case they block at that point."

When the java thread decides that a safe point has been reached, is it
taking into consideration in this safe point process if the thread
might be holding any locks? For example, if the java thread was
holding an inter-process lock that other processes on the server were
waiting on, this would cause a delay in the other processes until this
java thread is continued and is able to release the lock. So is the
safe point processing taking into consideration if any locks (i.e.
semaphores, mutexes, etc.) are being held by that the java thread
before it is suspended?


IBM respnonse:
The JVM only concerns itself with its own locks and not with any
non-Java locks such as semaphores or mutexes.


Thanks,
Tim

From: Tim Zielke
Sent: Thursday, January 16, 2014 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: RE: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Ian,

I agree with your assessment, as well. However, we have some MQ server side Java apps that are involved in performance SLAs. Switching to MQCNO_ISOLATED_BINDING should be negligible on the SLA, since the isolated binding performance is generally between shared and client bindings, per Andy. Assuming (and this definitely an assumption at this point) that there is the remote risk for significant queue manager slow downs due to a long running JVM stop-the-world garbage collection stopping a thread that is holding an inter-process MQ lock, it actually makes some sense for us to consider switching to MQCNO_ISOLATED_BINDING for our server side Java applications that use MQ. Anyway, I plan on following up with an IBM service request to hopefully get a more definitive answer on this topic.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Ian Alderson
Sent: Thursday, January 16, 2014 3:44 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim and Andy, thanks for the extra information. I’m not overly concerned about this potential issue but it’s certainly something to be aware of and would be good to get a deeper understanding. From reading the responses it would seem to me that the window of for GC to freeze a part of the qmgr is (probably) small. But I could imagine you could increase the odds by having an application running lots of threads and/or continually creating lots of connections where the lock has a much larger scope (of course creating a lot of unnecessary connections is generally regarding as bad practice).

Andy,
If the qmgr cannot obtain a lock, would this generally be recorded in the MQ error logs? I recall being advised by L3 support to set isolated bindings once for an app running on WAS 6 and MQ 6 a few years ago, but I can’t remember the detail of the why, but I think it could have been related to AIX’s EXTSHM – however I digress


I agree it would be good to have a JVM internals expert give their view, especially if they can relate it to the MQ classes (JMS and base).
I revisited a link I had about JMS versus base java classes running in a J2EE environment.
https://www-304.ibm.com/support/docview.wss?uid=swg21266535
Using base MQ java classes in a J2EE container is not recommended, and although I would hope most code is now using JMS I am sure there will still be some apps out there. The section on Thread Creation (and cleanup) looks interesting as it might relate to how GC treats the MQ threads.
Of course there will be lots of standalone java apps running the MQ base classes legitimately, although I see more developers using JMS as standard for these implementations these days, so would be interesting to know if the potential for locking is inherently different when using either set of classes.

Thanks,
Ian

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Tuesday, January 14, 2014 10:58 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

I did find this article that provides a little more information on the topic -> http://www.infoq.com/articles/Java_Garbage_Collection_Distilled

Relevant Section:
"To bring an application to a total stop it is necessary to pause all the running threads. Garbage collectors do this by signaling the threads to stop when they come to a “safepoint”, which is a point during program execution at which all GC roots are known and all heap object contents are consistent. Depending on what a thread is doing it may take some time to reach a safepoint. Safepoint checks are normally performed on method returns and loop back edges, but can be optimized away in some places making them more dynamically rare. For example, if a thread is copying a large array, cloning a large object, or executing a monotonic counted loop with a finite bound, it may be many milliseconds before a safepoint is reached."

If there is a JVM internals expert out there (or someone knows one at IBM? :-) ), it would be great to hear what they say on the matter.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Tuesday, January 14, 2014 6:49 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: MQ locks and JVM stop-the-world

There's an assertion here that Java garbage collection is blindly sending SIGSTOP to threads in the same process regardless of their current state. The article referenced by Tim doesn't go as far as to state this, and it's well beyond my Java knowledge. All pre-emptive dispatching involves potentially losing control for short periods of time while holding locks, but the suggestion here is that threads could randomly be frozen for prolonged periods of time, and in this environment then any hope of meeting reasonable SLA's would seem improbable. We could really do with a knowledgeable JVM internals expert chipping in at this time and explaining the JVM implementation and strategy.

Pretty much all standard bound MQI calls involve taking at least one inter process lock at some point, but the scope and granularity of the locks varies massively, for example during a standard bound MQCONN a coarse lock is obtained, which would inhibit all other standard bound MQCONN activity. Conversely, inter process locks used in the application process during MQPUT/MQGET are held for very few instructions and have a much smaller scope.

Isolated bindings are independent of MQ client bindings, and do not use amqrmppa processes. Isolated bindings use unix domain sockets and a thread in an amqzlas0 process will exist in much the same way that a thread in an amqzlaa0 process exists on behalf of a standard bound application. The performance of isolated bindings is generally someplace between shared binding and client bindings.




From: Ian Alderson <***@IGNISASSET.COM<mailto:***@IGNISASSET.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 14/01/2014 09:53
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________





Andy,
That is a very interesting (and may I say slightly alarming) point about JVM GC potentially freezing some part of a QMgr. There must be a lot of JVMs out there that are connected using server bindings, and if this is a potential issue then the window must be very very small or the effect so slight that it is mostly not noticed. Having said that, the most common implementation I have seen when an app server like WAS is co-located with a qmgr is that the apps in WAS still connect using MQClient (with the added benefit that HA failover of the QMgr is then independent of a WAS cluster).

So just to make sure I understand the risk. If a JVM is connected to MQ using bindings mode, then during GC when it will send STOP signals to all its threads, a lock on one or more IPC resources could prevent another part of the qmgr from gaining said lock? What is the window for this lock, on a connect of pretty much any MQ operation?

Also
“Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance”
I believe using ISOLATED bindings will effectively force the application to stop using the shared IPC resources. In terms of performance, is it fair to say that performance would be comparable to running a client bindings on the same server (using the loopback address)? At this point I am not clear if isolated bindings would use amqrmppa or some other process in the absence of IPC. If my understanding is off the mark please do let me know.

Thanks,
Ian



Ian Alderson
MQ Technical Architect

[cid:***@01CF1840.3A6B5540]
DL 0203 003 3055

________________________________


Ignis Asset Management
Fixed Income | Equities | Real Estate | Advisors | Solutions
150 Cheapside | London | EC2V 6ET

http://www.ignisasset.com<http://www.ignisasset.com/>
http://twitter.com/IgnisAM
http://www.linkedin.com/companies/ignis-asset-management

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Monday, January 13, 2014 4:03 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Thanks, Andy!

To the other MQ administrators on this LISTSERV that support server side java applications (like we do), below is a link to more information about this Java garbage collection topic (in Java garbage collection terminology it is called stop-the-world), in case you are interested. I would think it would be helpful to at least be aware of its existence, and potential (although probably remote?) impact to MQ queue manager performance.

http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

Here is a relevant snippet from the above link:

"Returning back to Garbage Collection, there is a term that you should know before learning about GC. The term is "stop-the-world." Stop-the-world will occur no matter which GC algorithm you choose. Stop-the-world means that the JVM is stopping the application from running to execute a GC. When stop-the-world occurs, every thread except for the threads needed for the GC will stop their tasks. The interrupted tasks will resume only after the GC task has completed. GC tuning often means reducing this stop-the-world time."

I did also run across a Java technology from Azul that says it does not use any stop-the-world algorithms in its garbage collection, but it does not look like this JVM is supported for WebSphere MQ according to the manual.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Monday, January 13, 2014 2:19 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Thread on STOP signals and MQ

If the JVM is simply sending STOP signals to threads without understanding the context of those threads then it risks sending a STOP signal to an MQ related thread while that thread holds an MQ lock, and thus freezing some queue manager capabilities until the thread is resumed. Using ISOLATED MQ bindings (MQCNO_ISOLATED_BINDING) would eliminate this risk at the expense of some performance.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 13/01/2014 03:12
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________






Hi Andy,

I had a follow up on this comment:

"Sending a STOP signal to an MQ server side application is dangerous enough"

My understanding is that when a JVM needs to run certain types of garbage collection, the applicable JVM system thread sends a STOP signal to all the application threads in order for the garbage collection to take place. So it is possible if you have a long garbage collection scavenger run that lasted 15 seconds in your JVM, all the application threads for that JVM would receive a STOP signal and would subsequently be stopped for that 15 seconds. Does this mean that an MQ queue manager is somewhat susceptible to what you are describing below if a server side java application was connected to the queue manager, since my understanding is JVMs are constantly sending STOP signals to their application threads?

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Andrew Hickson
Sent: Thursday, December 12, 2013 2:38 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Sending a STOP signal to an MQ server side application is dangerous enough, but sending a STOP signal to an MQ process (e.g amqrmppa) would be extremely dangerous. If that process happened to own an inter process lock at the time the STOP signal arrived the entire queue manager could easily grind to a halt. The same issue does arise for a 'simple' server side application (for example queue manager scope locks are obtained/released during MQCONN processing), but the scope for inadvertently stopping the queue manager is much greater with an MQ internal process.
Since MQ V7 much more specific traces have been possible, and tracing the API calls for specific connections would seem like a much better way to identify the cause of issues such as this.



From: Tim Zielke <***@AON.COM<mailto:***@AON.COM>>
To: ***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>,
Date: 11/12/2013 21:27
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue
Sent by: MQSeries List <***@listserv.meduniwien.ac.at<mailto:***@listserv.meduniwien.ac.at>>
________________________________







Yes, sending the STOP signal to a process will STOP all the threads for that process.

This is what I would do for your case. My understanding now is you have an isolated queue manager on its own server with MQ Client apps coming in from different servers to PUT and GET messages to this queue manager. If so, I would do the following:

GET disable the queue you wanted to browse
Let the message flow into the queue manager and get placed on that queue
Send a STOP signal to all of the channel processes (i.e. amqrmppa) for that queue manager
GET enable the queue and browse the message
Send a CONT signal to all of the channel processes (i.e. amqrmppa) for that queue manager

In my opinion, this type of activity would only be appropriate in a Development lifecycle, like yours. But it shouldn't negatively impact the queue manager besides temporarily stalling the channel activity, in my opinion.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 2:57 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

That’s slick, but
.We are primarily MQ Clients for our applications. In the case of MQ Client, if you freeze the PID that represents the app consuming from the queue, I imagine you are actually freezing the PID of amqrmppa, and thus all of the up to 64 threads (channels) in that amqrmppa would ‘hang’ until the PID was continued?

Nevertheless your email is getting saved and I’m gonna play around with that in the Lab. Might be real handy for other use cases.



Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 2:36 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

If these applications run on Unix/Linux, here is one other trick that you might want to consider for your use case for future reference since this is a Development environment. Use the STOP signal.

There are two signals that no process can block on Unix/Linux. The KILL signal and the STOP signal. The STOP signal will "freeze" a process from running any longer until you send the process a subsequent CONT or continue singal. So basically, instead of GET inhibiting the queue, you are run inhibiting the active application processes that can GET from the queue. Then you should be able to browse or GET your new message from the queue, and then send the CONT signal to the consuming applications to make them runnable again.

Here is an example. You may be already aware of how to do this, but it might be helpful for others to document.

Let's say DIS QSTATUS shows that pid 12345 has your queue open where you want to browse a new message from it. Do the following:

kill -s STOP 12345 (this assumes you have the appropriate access to send a signal to this process. You may need a Unix/Linux admin to help, if not)
Have the sending application send its message
Browse the message from the queue
kill -s CONT 12345

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 8:27 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Tim,
The security checking is only done on the MQOPEN call, so if the app was in a Get with Wait loop I’m thinking the change in security would not give the desired results.


Peter Potkay

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Tim Zielke
Sent: Wednesday, December 11, 2013 9:04 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue

Hi Peter,

Just a thought. Would another option for your use case be to temporarily change the security on the queue that the consuming message is doing a GET on (with a subsequent REFRESH SECURITY) so that only mqm has access to the queue (assuming the consuming message is not leveraging the mqm group for its security)? I would think the running consuming application would pick up the security change after the REFRESH SECURITY and then get 2035s until the MQ Admin has browsed the message. Then you could turn the original application security for the queue back on followed with another REFRESH SECURITY.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, December 11, 2013 7:48 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Request For Enhancement - Allow an MQ Admin to browse a get inhibited queue


Link to review the RFE and vote for it
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=42656


Description:
Enhance Websphere MQ so that a suitably authorized user (member of the mqm group) can browse a queue even if that queue is get inhibited. The ability to browse would be enough for our use cases, but it might be beneficial to also allow the MQ Admin to destructively get the message(s) on the Get Inhibited queue

Use case:
Two applications in the development environment that send MQ messages to each other are having a bad day, each laying blame on the other as to what is actually being sent to the other. Classic he sent, she sent. Or, you're not gonna believe this, they claim that MQ is changing the content of the message. So we tell the consuming app to close the queue and then we tell the sending app to produce another message but only when we tell them the consuming app has finally managed to stop, at which point we can look at it. Sounds simple. Often its not.
However, if the MQ Admin could Get Inhibit the queue yet still browse the queue, we could work the issue without needing the consuming application to stop. The MQ Admin could grab a copy of the message, enable the queue for gets and the consuming app would not have had to change anything.
A use case for allowing the ability to destructively get a message might be to allow an MQ Admin to step in and save the day when an app is stuck in a tight loop on a poison message. Without getting the consuming app to stop, the MQ Admin could get inhibit the queue, the poison message identified and removed, and the queue get enabled.

Business justification:
By giving the MQ Administrator this ability it would allow them to speed up problem resolution and exonerate Websphere MQ from being the source of the issue.




Peter Potkay
Application Platform Engineering @ The Hartford
Websphere MQ, Message Broker and DataPower
(W) 860-624-1395 / (M) 860-202-1375




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________





List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>






________________________________





List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________





List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>






________________________________





List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************


________________________________





List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>



________________________________




List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


**************************************************************
The information contained in this email (including any attachments transmitted within it) is confidential and is intended solely for the use of the named person.
The unauthorised access, copying or re-use of the information in it by any other person is strictly forbidden.
If you are not the intended recipient please notify us immediately by return email to ***@ignisasset.com<mailto:***@ignisasset.com>.

Internet communication is not guaranteed to be timely, secure, error or virus free. We accept no liability for any harm to systems or data, nor for personal emails. Emails may be recalled, deleted and monitored.

Ignis Asset Management is the trading name of the Ignis Asset Management Limited group of companies which includes the following subsidiaries:
Ignis Asset Management Limited (Registered in Scotland No. SC200801), Ignis Investment Services Limited* (Registered in Scotland No. SC101825)
Ignis Fund Managers Limited* (Registered in Scotland No. SC85610) Scottish Mutual Investment Managers Limited* (Registered in Scotland No. SC88674)
Registered Office: 50 Bothwell Street, Glasgow, G2 6HR, Tel: 0141-222-8000 and Scottish Mutual PEP & ISA Managers Limited* (Registered in England No. 971504)
Registered Office: 1 Wythall Green Way, Wythall, Birmingham B47 6WG and Ignis Investment Management Limited (Registered in England No. 5809046)
Registered Office: 150 Cheapside, London, EC2V 6ET Tel: 020 3003 3000. Scottish Mutual is a registered trade mark of Scottish Mutual Assurance Limited

*Authorised and regulated by the Financial Conduct Authority.

**************************************************************

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Loading...