Discussion:
Throttle MQ Messages
AkBar Dar
2013-07-14 17:33:34 UTC
Permalink
Is there to way in MQ to Read messages in a batch? I want to throttle the pick up of the messages off the Queue so that back end Application is not overwhelmed. I know there is way in Message Broker version 10. But, I am working with MQ 6 and Broker 6.

Please advise.
Regards,
AkBar Dar

Telecommute Day: Wednesday
Disclaimer This e-mail, and any attached file(s), is intended solely for the use of the individual or entity to whom this e-mail is addressed and may contain information that is privileged, confidential or exempt from disclosure. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender or call UI at 203-499-2222 and delete this message immediately from any computer. Any other use, retention, dissemination, retransmission, printing or copying of this e-mail or its contents (including any attached files) is strictly prohibited.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
T.Rob
2013-07-14 18:15:35 UTC
Permalink
Hi AkBar,

Not sure I understand the objective. WMQ will get the messages as far down
the pipe as possible as fast as possible. Is it the delivery of messages to
the queue that you wish throttled, or the delivery of messages from the
queue to the consumer(s)?

One common problem is that a queue served by many consumers results in all
those consumers hitting the same back-end application. If the consumers can
generate more traffic than the back-end can handle, that usually means there
are too many consumers. This would be indicated if, for example, the back
end having problems was a database.

However, if the back end system is driven by a web service, the load shaping
can be done by DataPower or similar. For example, 20 consumers all try to
hit the same web service which is configured for no more than 10 requests
per second. The DataPower box simply discards 10 of the requests. The apps
driving the transactions roll back their messages on the queue and loop
around to try again. The 20 requests are spread over 2 or three seconds
instead of one.


Or did I completely misunderstand the question?

-- T.Rob
-----Original Message-----
Behalf Of AkBar Dar
Sent: Sunday, July 14, 2013 1:34 PM
Subject: Throttle MQ Messages
Is there to way in MQ to Read messages in a batch? I want to throttle the
pick up of the messages off the Queue so that back end Application is not
overwhelmed. I know there is way in Message Broker version 10. But, I am
working with MQ 6 and Broker 6.
Please advise.
Regards,
AkBar Dar
Telecommute Day: Wednesday
Disclaimer This e-mail, and any attached file(s), is intended solely for
the use
of the individual or entity to whom this e-mail is addressed and may
contain information that is privileged, confidential or exempt from
disclosure. If you are not one of the named recipient(s) or otherwise have
reason to believe that you have received this message in error, please
notify
the sender or call UI at 203-499-2222 and delete this message immediately
from any computer. Any other use, retention, dissemination,
retransmission, printing or copying of this e-mail or its contents
(including
any attached files) is strictly prohibited.
the message body (not the subject), write: SIGNOFF MQSERIES Instructions
for managing your mailing list subscription are provided in the Listserv
General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
AkBar Dar
2013-07-14 18:28:52 UTC
Permalink
Rob

You understood correctly, I am trying to research a solution that is other DataPower, if not then I got no choice to go with DataPower.


Regards,
AkBar Dar

Telecommute Day: Wednesday
________________________________________
From: MQSeries List on behalf of T.Rob
Sent: Sunday, July 14, 2013 2:15:35 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Throttle MQ Messages

Hi AkBar,

Not sure I understand the objective. WMQ will get the messages as far down
the pipe as possible as fast as possible. Is it the delivery of messages to
the queue that you wish throttled, or the delivery of messages from the
queue to the consumer(s)?

One common problem is that a queue served by many consumers results in all
those consumers hitting the same back-end application. If the consumers can
generate more traffic than the back-end can handle, that usually means there
are too many consumers. This would be indicated if, for example, the back
end having problems was a database.

However, if the back end system is driven by a web service, the load shaping
can be done by DataPower or similar. For example, 20 consumers all try to
hit the same web service which is configured for no more than 10 requests
per second. The DataPower box simply discards 10 of the requests. The apps
driving the transactions roll back their messages on the queue and loop
around to try again. The 20 requests are spread over 2 or three seconds
instead of one.


Or did I completely misunderstand the question?

-- T.Rob
-----Original Message-----
Behalf Of AkBar Dar
Sent: Sunday, July 14, 2013 1:34 PM
Subject: Throttle MQ Messages
Is there to way in MQ to Read messages in a batch? I want to throttle the
pick up of the messages off the Queue so that back end Application is not
overwhelmed. I know there is way in Message Broker version 10. But, I am
working with MQ 6 and Broker 6.
Please advise.
Regards,
AkBar Dar
Telecommute Day: Wednesday
Disclaimer This e-mail, and any attached file(s), is intended solely for
the use
of the individual or entity to whom this e-mail is addressed and may
contain information that is privileged, confidential or exempt from
disclosure. If you are not one of the named recipient(s) or otherwise have
reason to believe that you have received this message in error, please
notify
the sender or call UI at 203-499-2222 and delete this message immediately
from any computer. Any other use, retention, dissemination,
retransmission, printing or copying of this e-mail or its contents
(including
any attached files) is strictly prohibited.
the message body (not the subject), write: SIGNOFF MQSERIES Instructions
for managing your mailing list subscription are provided in the Listserv
General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

Disclaimer This e-mail, and any attached file(s), is intended solely for the use of the individual or entity to whom this e-mail is addressed and may contain information that is privileged, confidential or exempt from disclosure. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender or call UI at 203-499-2222 and delete this message immediately from any computer. Any other use, retention, dissemination, retransmission, printing or copying of this e-mail or its contents (including any attached files) is strictly prohibited.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
T.Rob
2013-07-14 18:41:12 UTC
Permalink
There's nothing pretty. The QMgr does not have capability to limit the GET
rate so either the consumers are aware of each other's state or some
outboard process intercepts the messages on a pre-queue and feeds them into
the real queue at a pre-determined rate.

One approach that I have seen is to maintain a token queue with a certain
number of messages in it. In a single transaction, the app acquires a
message from the token queue and puts it back in the same queue without
committing. The app then does it's normal processing on the main queue -
get a message, execute the back-end transaction, COMMIT. This allows the
app to have an unlimited number of consumers from many remote servers but
limits the simultaneous transactions to however many messages are in the
token queue. The content of the messages in the token queue doesn't matter,
however I've found that if the app writes some identifying information then
tracing I/O on the queue reveals valuable diagnostics as to load
distribution and app health.

It's not a perfect solution since it isn't time based, but has the advantage
of not requiring much awareness of the different app instances to one
another and therefore minimal code changes. It also allows many more
consumers than can possibly be run at once so you know the queue will always
be served, so long as one instance remains. And it's tunable. Need to run
faster? Add a message or two to the token queue. Need to throttle back?
Delete a message or two.

-- T.Rob
-----Original Message-----
Behalf Of AkBar Dar
Sent: Sunday, July 14, 2013 2:29 PM
Subject: Re: Throttle MQ Messages
Rob
You understood correctly, I am trying to research a solution that is other
DataPower, if not then I got no choice to go with DataPower.
Regards,
AkBar Dar
Telecommute Day: Wednesday
________________________________________
From: MQSeries List on behalf of T.Rob
Sent: Sunday, July 14, 2013 2:15:35 PM
Subject: Re: Throttle MQ Messages
Hi AkBar,
Not sure I understand the objective. WMQ will get the messages as far
down the pipe as possible as fast as possible. Is it the delivery of
messages
to the queue that you wish throttled, or the delivery of messages from the
queue to the consumer(s)?
One common problem is that a queue served by many consumers results in
all those consumers hitting the same back-end application. If the
consumers can generate more traffic than the back-end can handle, that
usually means there are too many consumers. This would be indicated if,
for example, the back end having problems was a database.
However, if the back end system is driven by a web service, the load
shaping can be done by DataPower or similar. For example, 20 consumers
all try to hit the same web service which is configured for no more than
10
requests per second. The DataPower box simply discards 10 of the
requests. The apps driving the transactions roll back their messages on
the
queue and loop around to try again. The 20 requests are spread over 2 or
three seconds instead of one.
Or did I completely misunderstand the question?
-- T.Rob
-----Original Message-----
On
Behalf Of AkBar Dar
Sent: Sunday, July 14, 2013 1:34 PM
Subject: Throttle MQ Messages
Is there to way in MQ to Read messages in a batch? I want to throttle
the pick up of the messages off the Queue so that back end Application
is not overwhelmed. I know there is way in Message Broker version 10.
But, I am working with MQ 6 and Broker 6.
Please advise.
Regards,
AkBar Dar
Telecommute Day: Wednesday
Disclaimer This e-mail, and any attached file(s), is intended solely
for
the use
of the individual or entity to whom this e-mail is addressed and may
contain information that is privileged, confidential or exempt from
disclosure. If you are not one of the named recipient(s) or otherwise
have reason to believe that you have received this message in error,
please
notify
the sender or call UI at 203-499-2222 and delete this message
immediately from any computer. Any other use, retention,
dissemination, retransmission, printing or copying of this e-mail or
its contents
(including
any attached files) is strictly prohibited.
the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided
in the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
the message body (not the subject), write: SIGNOFF MQSERIES Instructions
for managing your mailing list subscription are provided in the Listserv
General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
Disclaimer This e-mail, and any attached file(s), is intended solely for
the use
of the individual or entity to whom this e-mail is addressed and may
contain information that is privileged, confidential or exempt from
disclosure. If you are not one of the named recipient(s) or otherwise have
reason to believe that you have received this message in error, please
notify
the sender or call UI at 203-499-2222 and delete this message immediately
from any computer. Any other use, retention, dissemination,
retransmission, printing or copying of this e-mail or its contents
(including
any attached files) is strictly prohibited.
the message body (not the subject), write: SIGNOFF MQSERIES Instructions
for managing your mailing list subscription are provided in the Listserv
General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

Loading...