MQCB and MQCTL multi-thread issue

[Roger Lacroix]

All,

I'm stuck.  Help.

I am trying out the MQCB and MQCTL API calls(using WMQ v7.5) to have messages pushed to an async message consumer.

My code dumps out the PID (Process Id) and TID (Thread Id) of the main process and in the async message consumers.  I can see that all have the same PID, which is expected and they have different TIDs.  The issue I am having is that all message consumers are running under the same TID.  i.e. one TID for the main process and one TID for all message consumer.

I have done testing where the MQOPEN, MQCB & MQCTL are against 3 different queues and testing where they are against the same queue.  I tried using MQCTLO_THREAD_AFFINITY and without but it made no difference.  No matter what I do, I cannot get the message consumers under different TIDs (different from each other).

It is hard to say that the message consumers are running aync when they all use the same TID.  I even tried out the MQ sample code called: amqscbf0.c and got the same result.  Because all message consumers are using the same TID, the log shows that basically messages are being consumed (from the 3 queues) in a round-robin fashion.  But what I want is the 3 message consumers to run in parallel (which can only be done using different TIDs).

I have read and reread the MQ InfoCenter but I cannot figure it out.

What option/parameter do I use to get the combination MQOPEN, MQCB & MQCTL API calls to force the message consumers to be on different TIDs (from each other)?


Regards,
Roger Lacroix
Capitalware Inc.


To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Tim Zielke]

Hi Roger,

I do not know if this will meet your requirements, but have you tried doing the MQOPEN, MQCB, and MQCTL on unique connections? We have a multi-threaded application here that uses these asynch message consumer APIs, and I have seen multiple message consumers run on separate threads in a trace. I am pretty sure the message consumers were set up on different connections in the trace I reviewed. Looking at amqscbf0.c, it looks like it is using one connection.

Thanks,
Tim

From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
Sent: Sunday, November 10, 2013 10:52 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: MQCB and MQCTL multi-thread issue

All,

I'm stuck. Help.

I am trying out the MQCB and MQCTL API calls (using WMQ v7.5) to have messages pushed to an async message consumer.

My code dumps out the PID (Process Id) and TID (Thread Id) of the main process and in the async message consumers. I can see that all have the same PID, which is expected and they have different TIDs. The issue I am having is that all message consumers are running under the same TID. i.e. one TID for the main process and one TID for all message consumer.

I have done testing where the MQOPEN, MQCB & MQCTL are against 3 different queues and testing where they are against the same queue. I tried using MQCTLO_THREAD_AFFINITY and without but it made no difference. No matter what I do, I cannot get the message consumers under different TIDs (different from each other).

It is hard to say that the message consumers are running aync when they all use the same TID. I even tried out the MQ sample code called: amqscbf0.c and got the same result. Because all message consumers are using the same TID, the log shows that basically messages are being consumed (from the 3 queues) in a round-robin fashion. But what I want is the 3 message consumers to run in parallel (which can only be done using different TIDs).

I have read and reread the MQ InfoCenter but I cannot figure it out.

What option/parameter do I use to get the combination MQOPEN, MQCB & MQCTL API calls to force the message consumers to be on different TIDs (from each other)?

Regards,
Roger Lacroix
Capitalware Inc.

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Tim Zielke]

This post might be inappropriate. Click to display it.

[Roger Lacroix]

Hi Tim,

I should have included more information. I am not looking for 3
MQGETs to be performed at the same time but rather I just want the
message consumers to be on their own thread.

Each message will be less than 1KB (probably 150-200 bytes) that the
message consumer will receive. Roughly 5% (or less) of the work
performed by the thread will be related to MQ and 95% will be backend
non-MQ work.

In this case, the bottleneck is not MQ but all of the backend work
that the message consumer needs to do. Hence, if each message
consumer is started in its own thread, then the application can do
more non-MQ work (parallel processing for the backend work).

Also, for the MQCB call, I explicitly set the GMO options field to be
MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
knows that the message consumption will NOT be in a unit of work.

Right now in my testing, I have loop (counter = 3) where it does
MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
thread for the main process and 1 thread for all of the message
consumers. So, basically I have 2 threads:

- 1 thread: the main process thread can open the queue, get messages
and do backend work
- 1 thread: all message consumers that are pushed messages then it
does the backend work

2 threads are better than one but I want 4 or maybe 8 threads, so
that the application can do lots of backend work once it receive the message.

Does that make more sense?

So, does anyone know of a flag, parameter or environment variable to
tell the MQ library to start each message consumer in its own thread?

Regards,
Roger Lacroix
Capitalware Inc.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

[Tim Zielke]

Hi Roger,

That does make more sense, but my replies were more around how to make the message consumers run on their own thread. I did not test this, but based on profiling one of our multi-threaded applications that is accomplishing what you want (message consumers on their own thread), the following may work.

In your program, create 4 of your own threads that do the following (and thus create their own unique connection):
thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
thread 4 - MQCONN, MQOPEN, MQCB, MQCTL

When you are done, you will have 4 message consumers running on their own thread in your multi-threaded process.

Thanks,
Tim

-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
Sent: Monday, November 11, 2013 12:04 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Tim,

I should have included more information. I am not looking for 3
MQGETs to be performed at the same time but rather I just want the
message consumers to be on their own thread.

Each message will be less than 1KB (probably 150-200 bytes) that the
message consumer will receive. Roughly 5% (or less) of the work
performed by the thread will be related to MQ and 95% will be backend
non-MQ work.

In this case, the bottleneck is not MQ but all of the backend work
that the message consumer needs to do. Hence, if each message
consumer is started in its own thread, then the application can do
more non-MQ work (parallel processing for the backend work).

Also, for the MQCB call, I explicitly set the GMO options field to be
MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
knows that the message consumption will NOT be in a unit of work.

Right now in my testing, I have loop (counter = 3) where it does
MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
thread for the main process and 1 thread for all of the message
consumers. So, basically I have 2 threads:

- 1 thread: the main process thread can open the queue, get messages
and do backend work
- 1 thread: all message consumers that are pushed messages then it
does the backend work

2 threads are better than one but I want 4 or maybe 8 threads, so
that the application can do lots of backend work once it receive the message.

Does that make more sense?

So, does anyone know of a flag, parameter or environment variable to
tell the MQ library to start each message consumer in its own thread?

Regards,
Roger Lacroix
Capitalware Inc.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

[Paul Clarke]

Tim,

This is not unreasonable but, just to be clear to everyone listening in, it
is not necessary to create four threads to issue the MQCONNs on. You can
issue all four connects on the same thread to get the same effect.

Cheers,
Paul.

Paul Clarke
www.mqgem.com
-----Original Message-----
From: Tim Zielke
Sent: Monday, November 11, 2013 9:08 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Roger,

That does make more sense, but my replies were more around how to make the
message consumers run on their own thread. I did not test this, but based
on profiling one of our multi-threaded applications that is accomplishing
what you want (message consumers on their own thread), the following may
work.

In your program, create 4 of your own threads that do the following (and
thus create their own unique connection):
thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
thread 4 - MQCONN, MQOPEN, MQCB, MQCTL

When you are done, you will have 4 message consumers running on their own
thread in your multi-threaded process.

Thanks,
Tim

-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of
Roger Lacroix
Sent: Monday, November 11, 2013 12:04 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Tim,

I should have included more information. I am not looking for 3
MQGETs to be performed at the same time but rather I just want the
message consumers to be on their own thread.

Each message will be less than 1KB (probably 150-200 bytes) that the
message consumer will receive. Roughly 5% (or less) of the work
performed by the thread will be related to MQ and 95% will be backend
non-MQ work.

In this case, the bottleneck is not MQ but all of the backend work
that the message consumer needs to do. Hence, if each message
consumer is started in its own thread, then the application can do
more non-MQ work (parallel processing for the backend work).

Also, for the MQCB call, I explicitly set the GMO options field to be
MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
knows that the message consumption will NOT be in a unit of work.

Right now in my testing, I have loop (counter = 3) where it does
MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
thread for the main process and 1 thread for all of the message
consumers. So, basically I have 2 threads:

- 1 thread: the main process thread can open the queue, get messages
and do backend work
- 1 thread: all message consumers that are pushed messages then it
does the backend work

2 threads are better than one but I want 4 or maybe 8 threads, so
that the application can do lots of backend work once it receive the
message.

Does that make more sense?

So, does anyone know of a flag, parameter or environment variable to
tell the MQ library to start each message consumer in its own thread?

Regards,
Roger Lacroix
Capitalware Inc.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

[Roger Lacroix]

Hi Paul,

> You can issue all four connects on the same thread to get the same effect.

I actually tried this when I was fooling around with the code (even
though the standard is 1 connection per thread) but the MQCONN always
returns the same hconn. So, I thought 'nice try but it is 1
connection per thread'.

Regards,
Roger Lacroix
Capitalware Inc.

At 04:35 PM 11/11/2013, you wrote:
>Tim,
>
>This is not unreasonable but, just to be clear to everyone listening
>in, it is not necessary to create four threads to issue the MQCONNs
>on. You can issue all four connects on the same thread to get the same effect.
>
>Cheers,
>Paul.
>
>Paul Clarke
>www.mqgem.com
>-----Original Message----- From: Tim Zielke
>Sent: Monday, November 11, 2013 9:08 PM
>To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>Subject: Re: MQCB and MQCTL multi-thread issue
>
>Hi Roger,
>
>That does make more sense, but my replies were more around how to
>make the message consumers run on their own thread. I did not test
>this, but based on profiling one of our multi-threaded applications
>that is accomplishing what you want (message consumers on their own
>thread), the following may work.
>
>In your program, create 4 of your own threads that do the following
>(and thus create their own unique connection):
>thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
>thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
>thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
>thread 4 - MQCONN, MQOPEN, MQCB, MQCTL
>
>When you are done, you will have 4 message consumers running on
>their own thread in your multi-threaded process.
>
>Thanks,
>Tim
>
>-----Original Message-----
>From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On
>Behalf Of Roger Lacroix
>Sent: Monday, November 11, 2013 12:04 PM
>To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>Subject: Re: MQCB and MQCTL multi-thread issue
>
>Hi Tim,
>
>I should have included more information. I am not looking for 3
>MQGETs to be performed at the same time but rather I just want the
>message consumers to be on their own thread.
>
>Each message will be less than 1KB (probably 150-200 bytes) that the
>message consumer will receive. Roughly 5% (or less) of the work
>performed by the thread will be related to MQ and 95% will be backend
>non-MQ work.
>
>In this case, the bottleneck is not MQ but all of the backend work
>that the message consumer needs to do. Hence, if each message
>consumer is started in its own thread, then the application can do
>more non-MQ work (parallel processing for the backend work).
>
>Also, for the MQCB call, I explicitly set the GMO options field to be
>MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
>knows that the message consumption will NOT be in a unit of work.
>
>Right now in my testing, I have loop (counter = 3) where it does
>MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
>thread for the main process and 1 thread for all of the message
>consumers. So, basically I have 2 threads:
>
>- 1 thread: the main process thread can open the queue, get messages
>and do backend work
>- 1 thread: all message consumers that are pushed messages then it
>does the backend work
>
>2 threads are better than one but I want 4 or maybe 8 threads, so
>that the application can do lots of backend work once it receive the message.
>
>Does that make more sense?
>
>So, does anyone know of a flag, parameter or environment variable to
>tell the MQ library to start each message consumer in its own thread?
>
>Regards,
>Roger Lacroix
>Capitalware Inc.
>
>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>in the message body (not the subject), write: SIGNOFF MQSERIES
>Instructions for managing your mailing list subscription are provided in
>the Listserv General Users Guide available at http://www.lsoft.com
>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>
>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>in the message body (not the subject), write: SIGNOFF MQSERIES
>Instructions for managing your mailing list subscription are provided in
>the Listserv General Users Guide available at http://www.lsoft.com
>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>in the message body (not the subject), write: SIGNOFF MQSERIES
>Instructions for managing your mailing list subscription are provided in
>the Listserv General Users Guide available at http://www.lsoft.com
>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Paul Clarke]

No it isn’t Roger...you should read the MQI API Reference more carefully. By default MQHCONNs are bound to a single thread but you can connect and ask for a shared connection handle using something like MQCNO_HANDLE_SHARE_BLOCK. With this option you can create as many MQHCONNs on the same thread as you like and use those MQHCONNs on any thread you like.

Cheers,
Paul.

Paul Clarke
www.mqgem.com

From: Roger Lacroix
Sent: Monday, November 11, 2013 10:00 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Paul,

> You can issue all four connects on the same thread to get the same effect.

I actually tried this when I was fooling around with the code (even though the standard is 1 connection per thread) but the MQCONN always returns the same hconn. So, I thought 'nice try but it is 1 connection per thread'.

Regards,
Roger Lacroix
Capitalware Inc.

At 04:35 PM 11/11/2013, you wrote:

Tim,

This is not unreasonable but, just to be clear to everyone listening in, it is not necessary to create four threads to issue the MQCONNs on. You can issue all four connects on the same thread to get the same effect.

Cheers,
Paul.

Paul Clarke
www.mqgem.com
-----Original Message----- From: Tim Zielke
Sent: Monday, November 11, 2013 9:08 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Roger,

That does make more sense, but my replies were more around how to make the message consumers run on their own thread. I did not test this, but based on profiling one of our multi-threaded applications that is accomplishing what you want (message consumers on their own thread), the following may work.

In your program, create 4 of your own threads that do the following (and thus create their own unique connection):
thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
thread 4 - MQCONN, MQOPEN, MQCB, MQCTL

When you are done, you will have 4 message consumers running on their own thread in your multi-threaded process.

Thanks,
Tim

-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
Sent: Monday, November 11, 2013 12:04 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Tim,

I should have included more information. I am not looking for 3
MQGETs to be performed at the same time but rather I just want the
message consumers to be on their own thread.

Each message will be less than 1KB (probably 150-200 bytes) that the
message consumer will receive. Roughly 5% (or less) of the work
performed by the thread will be related to MQ and 95% will be backend
non-MQ work.

In this case, the bottleneck is not MQ but all of the backend work
that the message consumer needs to do. Hence, if each message
consumer is started in its own thread, then the application can do
more non-MQ work (parallel processing for the backend work).

Also, for the MQCB call, I explicitly set the GMO options field to be
MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
knows that the message consumption will NOT be in a unit of work.

Right now in my testing, I have loop (counter = 3) where it does
MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
thread for the main process and 1 thread for all of the message
consumers. So, basically I have 2 threads:

- 1 thread: the main process thread can open the queue, get messages
and do backend work
- 1 thread: all message consumers that are pushed messages then it
does the backend work

2 threads are better than one but I want 4 or maybe 8 threads, so
that the application can do lots of backend work once it receive the message.

Does that make more sense?

So, does anyone know of a flag, parameter or environment variable to
tell the MQ library to start each message consumer in its own thread?

Regards,
Roger Lacroix
Capitalware Inc.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html


--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Roger Lacroix]

Hi Paul,

> No it isn't Roger...you should read the MQI API Reference more carefully.

I may know a lot about MQ but you most certainly know a lot more!!

> By default MQHCONNs are bound to a single
thread but you can connect and ask for a shared
connection handle using something like
MQCNO_HANDLE_SHARE_BLOCK. With this option you
can create as many MQHCONNs on the same thread as
you like and use those MQHCONNs on any thread you like.

How in the world did you get that from the manual?

For MQCNO_HANDLE_SHARE_BLOCK, the manual says

Creates a shared connection. On a
MQCNO_HANDLE_SHARE_BLOCK connection, if the
connection is currently in use by an MQI call on
another thread, the MQI call waits until the current MQI call has completed.

Where does it say that option allows you to
create multiple connections per thread?

Don't get me wrong - this IS what I was looking
for. I added the option and put MQCONNX, MQOPEN,
MQCB and MQCTL in a loop and viola, I get the
number of threads for the message consumer as the
loop counter is set for. Bingo.

Should I be using MQCTLO_THREAD_AFFINITY option
with MQCTL? I tried it with and without but it
did not seem to make any difference.

Regards,
Roger Lacroix
Capitalware Inc.

At 05:09 PM 11/11/2013, you wrote:
>No it isn’t Roger...you should read the MQI
>API Reference more carefully. By default
>MQHCONNs are bound to a single thread but you
>can connect and ask for a shared connection
>handle using something like
>MQCNO_HANDLE_SHARE_BLOCK. With this option you
>can create as many MQHCONNs on the same thread
>as you like and use those MQHCONNs on any thread you like.
>
>Cheers,
>Paul.
>
>Paul Clarke
>www.mqgem.com
>
>From: <mailto:roger.lacroix-***@public.gmane.org>Roger Lacroix
>Sent: Monday, November 11, 2013 10:00 PM
>To:
><mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org>MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAT
>Subject: Re: MQCB and MQCTL multi-thread issue
>
>Hi Paul,
>
> > You can issue all four connects on the same thread to get the same effect.
>
>I actually tried this when I was fooling around
>with the code (even though the standard is 1
>connection per thread) but the MQCONN always
>returns the same hconn. So, I thought 'nice try
>but it is 1 connection per thread'.
>
>Regards,
>Roger Lacroix
>Capitalware Inc.
>
>At 04:35 PM 11/11/2013, you wrote:
>>Tim,
>>
>>This is not unreasonable but, just to be clear
>>to everyone listening in, it is not necessary
>>to create four threads to issue the MQCONNs on.
>>You can issue all four connects on the same thread to get the same effect.
>>
>>Cheers,
>>Paul.
>>
>>Paul Clarke
>>www.mqgem.com
>>-----Original Message----- From: Tim Zielke
>>Sent: Monday, November 11, 2013 9:08 PM
>>To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>>Subject: Re: MQCB and MQCTL multi-thread issue
>>
>>Hi Roger,
>>
>>That does make more sense, but my replies were
>>more around how to make the message consumers
>>run on their own thread. I did not test this,
>>but based on profiling one of our
>>multi-threaded applications that is
>>accomplishing what you want (message consumers
>>on their own thread), the following may work.
>>
>>In your program, create 4 of your own threads
>>that do the following (and thus create their own unique connection):
>>thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
>>thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
>>thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
>>thread 4 - MQCONN, MQOPEN, MQCB, MQCTL
>>
>>When you are done, you will have 4 message
>>consumers running on their own thread in your multi-threaded process.
>>
>>Thanks,
>>Tim
>>
>>-----Original Message-----
>>From: MQSeries List
>>[mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
>>Sent: Monday, November 11, 2013 12:04 PM
>>To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>>Subject: Re: MQCB and MQCTL multi-thread issue
>>
>>Hi Tim,
>>
>>I should have included more information. I am not looking for 3
>>MQGETs to be performed at the same time but rather I just want the
>>message consumers to be on their own thread.
>>
>>Each message will be less than 1KB (probably 150-200 bytes) that the
>>message consumer will receive. Roughly 5% (or less) of the work
>>performed by the thread will be related to MQ and 95% will be backend
>>non-MQ work.
>>
>>In this case, the bottleneck is not MQ but all of the backend work
>>that the message consumer needs to do. Hence, if each message
>>consumer is started in its own thread, then the application can do
>>more non-MQ work (parallel processing for the backend work).
>>
>>Also, for the MQCB call, I explicitly set the GMO options field to be
>>MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
>>knows that the message consumption will NOT be in a unit of work.
>>
>>Right now in my testing, I have loop (counter = 3) where it does
>>MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
>>thread for the main process and 1 thread for all of the message
>>consumers. So, basically I have 2 threads:
>>
>>- 1 thread: the main process thread can open the queue, get messages
>>and do backend work
>>- 1 thread: all message consumers that are pushed messages then it
>>does the backend work
>>
>>2 threads are better than one but I want 4 or maybe 8 threads, so
>>that the application can do lots of backend work once it receive the message.
>>
>>Does that make more sense?
>>
>>So, does anyone know of a flag, parameter or environment variable to
>>tell the MQ library to start each message consumer in its own thread?
>>
>>Regards,
>>Roger Lacroix
>>Capitalware Inc.
>>
>>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>>in the message body (not the subject), write: SIGNOFF MQSERIES
>>Instructions for managing your mailing list subscription are provided in
>>the Listserv General Users Guide available at http://www.lsoft.com
>>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>>
>>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>>in the message body (not the subject), write: SIGNOFF MQSERIES
>>Instructions for managing your mailing list subscription are provided in
>>the Listserv General Users Guide available at http://www.lsoft.com
>>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>>in the message body (not the subject), write: SIGNOFF MQSERIES
>>Instructions for managing your mailing list subscription are provided in
>>the Listserv General Users Guide available at http://www.lsoft.com
>>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>
>
>----------
><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List
>Archive -
><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>Your List Settings -
><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>
>
>Instructions for managing your mailing list
>subscription are provided in the Listserv
>General Users Guide available at
><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com
>
>
>----------
><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List
>Archive -
><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>Your List Settings -
><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>
>
>Instructions for managing your mailing list
>subscription are provided in the Listserv
>General Users Guide available at
><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Paul Clarke]

I agree that the manual could be a lot clearer and perhaps this is worthy of a manual update. Shared Handles have been around for many releases, nothing to do with async consume, it would be a shame if most people don't know about them.

As far as using MQCTLO_THREAD_AFFINITY is concerned I think you probably should use it. It's not really what the option is for but if I recall it will have the right affect as a side effect on a client. Be aware that the threading on the server and client is a little different and, of course, may change in the future. However, from what I remember the situation is this:

a) on the server
A thread dedicated to the connection is created as soon as MQCTL is issued and the thread will run for the life of async consume. So, MQCTLO_THREAD_AFFINITY will have no effect since it is doing it anyway.

b) on the client
By default the connection is only associated with a thread when there are messages to be delivered and more than one connection can be associated with the same thread at any one time. This reduces the thread usage. For example it is possible to have 100 connections to the queue manager all with started consumers and yet there be 0 threads in the background to deliver the messages. However, if I recall correctly then specifying MQCTLO_THREAD_AFFINITY will cause the client to dedicate a thread to the connection just like the server. So, in the case of the client I think you should indeed use MQCTLO_THREAD_AFFINITY.


Of course the bottom line is to try it and see what effect you see. If you do find it works then I think it might also be worth getting a manual update no this behaviour since if you are relying on it you don't want MQ to change the dispatching model underneath you. This isn't really what MQCTLO_THREAD_AFFINITY was about after all but it is an understandable by-product.

Cheers,
Paul.

Paul Clarke
www.mqgem.com

From: Roger Lacroix
Sent: Tuesday, November 12, 2013 12:26 AM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Paul,

> No it isn't Roger...you should read the MQI API Reference more carefully.

I may know a lot about MQ but you most certainly know a lot more!!

> By default MQHCONNs are bound to a single thread but you can connect and ask for a shared connection handle using something like MQCNO_HANDLE_SHARE_BLOCK. With this option you can create as many MQHCONNs on the same thread as you like and use those MQHCONNs on any thread you like.

How in the world did you get that from the manual?

For MQCNO_HANDLE_SHARE_BLOCK, the manual says

Creates a shared connection. On a MQCNO_HANDLE_SHARE_BLOCK connection, if the connection is currently in use by an MQI call on another thread, the MQI call waits until the current MQI call has completed.

Where does it say that option allows you to create multiple connections per thread?

Don't get me wrong - this IS what I was looking for. I added the option and put MQCONNX, MQOPEN, MQCB and MQCTL in a loop and viola, I get the number of threads for the message consumer as the loop counter is set for. Bingo.

Should I be using MQCTLO_THREAD_AFFINITY option with MQCTL? I tried it with and without but it did not seem to make any difference.

Regards,
Roger Lacroix
Capitalware Inc.

At 05:09 PM 11/11/2013, you wrote:

No it isn’t Roger...you should read the MQI API Reference more carefully. By default MQHCONNs are bound to a single thread but you can connect and ask for a shared connection handle using something like MQCNO_HANDLE_SHARE_BLOCK. With this option you can create as many MQHCONNs on the same thread as you like and use those MQHCONNs on any thread you like.

Cheers,
Paul.

Paul Clarke
www.mqgem.com

From: Roger Lacroix
Sent: Monday, November 11, 2013 10:00 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Paul,

> You can issue all four connects on the same thread to get the same effect.

I actually tried this when I was fooling around with the code (even though the standard is 1 connection per thread) but the MQCONN always returns the same hconn. So, I thought 'nice try but it is 1 connection per thread'.

Regards,
Roger Lacroix
Capitalware Inc.

At 04:35 PM 11/11/2013, you wrote:

Tim,

This is not unreasonable but, just to be clear to everyone listening in, it is not necessary to create four threads to issue the MQCONNs on. You can issue all four connects on the same thread to get the same effect.

Cheers,
Paul.

Paul Clarke
www.mqgem.com
-----Original Message----- From: Tim Zielke
Sent: Monday, November 11, 2013 9:08 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Roger,

That does make more sense, but my replies were more around how to make the message consumers run on their own thread. I did not test this, but based on profiling one of our multi-threaded applications that is accomplishing what you want (message consumers on their own thread), the following may work.

In your program, create 4 of your own threads that do the following (and thus create their own unique connection):
thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
thread 4 - MQCONN, MQOPEN, MQCB, MQCTL

When you are done, you will have 4 message consumers running on their own thread in your multi-threaded process.

Thanks,
Tim

-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
Sent: Monday, November 11, 2013 12:04 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Tim,

I should have included more information. I am not looking for 3
MQGETs to be performed at the same time but rather I just want the
message consumers to be on their own thread.

Each message will be less than 1KB (probably 150-200 bytes) that the
message consumer will receive. Roughly 5% (or less) of the work
performed by the thread will be related to MQ and 95% will be backend
non-MQ work.

In this case, the bottleneck is not MQ but all of the backend work
that the message consumer needs to do. Hence, if each message
consumer is started in its own thread, then the application can do
more non-MQ work (parallel processing for the backend work).

Also, for the MQCB call, I explicitly set the GMO options field to be
MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
knows that the message consumption will NOT be in a unit of work.

Right now in my testing, I have loop (counter = 3) where it does
MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
thread for the main process and 1 thread for all of the message
consumers. So, basically I have 2 threads:

- 1 thread: the main process thread can open the queue, get messages
and do backend work
- 1 thread: all message consumers that are pushed messages then it
does the backend work

2 threads are better than one but I want 4 or maybe 8 threads, so
that the application can do lots of backend work once it receive the message.

Does that make more sense?

So, does anyone know of a flag, parameter or environment variable to
tell the MQ library to start each message consumer in its own thread?

Regards,
Roger Lacroix
Capitalware Inc.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html


------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com



------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com


--------------------------------------------------------------------------------

List Archive - Manage Your List Settings - Unsubscribe
Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Tim Zielke]

Hi Paul,

Thanks for all the information. Before sending that last post, I had done some testing with the amqscbf0.c sample and come to the same conclusion as Roger that you can have only one connection per thread. I even played around with one of the HANDLE_SHARE options. I just didn't pick the right one! Glad to know that this is not the case, and you can have multiple connections for a given thread.

Thanks,
Tim

From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Paul Clarke
Sent: Tuesday, November 12, 2013 12:52 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: MQCB and MQCTL multi-thread issue

I agree that the manual could be a lot clearer and perhaps this is worthy of a manual update. Shared Handles have been around for many releases, nothing to do with async consume, it would be a shame if most people don't know about them.

As far as using MQCTLO_THREAD_AFFINITY is concerned I think you probably should use it. It's not really what the option is for but if I recall it will have the right affect as a side effect on a client. Be aware that the threading on the server and client is a little different and, of course, may change in the future. However, from what I remember the situation is this:

a) on the server
A thread dedicated to the connection is created as soon as MQCTL is issued and the thread will run for the life of async consume. So, MQCTLO_THREAD_AFFINITY will have no effect since it is doing it anyway.

b) on the client
By default the connection is only associated with a thread when there are messages to be delivered and more than one connection can be associated with the same thread at any one time. This reduces the thread usage. For example it is possible to have 100 connections to the queue manager all with started consumers and yet there be 0 threads in the background to deliver the messages. However, if I recall correctly then specifying MQCTLO_THREAD_AFFINITY will cause the client to dedicate a thread to the connection just like the server. So, in the case of the client I think you should indeed use MQCTLO_THREAD_AFFINITY.


Of course the bottom line is to try it and see what effect you see. If you do find it works then I think it might also be worth getting a manual update no this behaviour since if you are relying on it you don't want MQ to change the dispatching model underneath you. This isn't really what MQCTLO_THREAD_AFFINITY was about after all but it is an understandable by-product.

Cheers,
Paul.

Paul Clarke
www.mqgem.com

From: Roger Lacroix<mailto:***@ROGERS.COM>
Sent: Tuesday, November 12, 2013 12:26 AM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Paul,

> No it isn't Roger...you should read the MQI API Reference more carefully.

I may know a lot about MQ but you most certainly know a lot more!!

> By default MQHCONNs are bound to a single thread but you can connect and ask for a shared connection handle using something like MQCNO_HANDLE_SHARE_BLOCK. With this option you can create as many MQHCONNs on the same thread as you like and use those MQHCONNs on any thread you like.

How in the world did you get that from the manual?

For MQCNO_HANDLE_SHARE_BLOCK, the manual says

Creates a shared connection. On a MQCNO_HANDLE_SHARE_BLOCK connection, if the connection is currently in use by an MQI call on another thread, the MQI call waits until the current MQI call has completed.

Where does it say that option allows you to create multiple connections per thread?

Don't get me wrong - this IS what I was looking for. I added the option and put MQCONNX, MQOPEN, MQCB and MQCTL in a loop and viola, I get the number of threads for the message consumer as the loop counter is set for. Bingo.

Should I be using MQCTLO_THREAD_AFFINITY option with MQCTL? I tried it with and without but it did not seem to make any difference.

Regards,
Roger Lacroix
Capitalware Inc.

At 05:09 PM 11/11/2013, you wrote:

No it isn’t Roger...you should read the MQI API Reference more carefully. By default MQHCONNs are bound to a single thread but you can connect and ask for a shared connection handle using something like MQCNO_HANDLE_SHARE_BLOCK. With this option you can create as many MQHCONNs on the same thread as you like and use those MQHCONNs on any thread you like.

Cheers,
Paul.

Paul Clarke
www.mqgem.com<http://www.mqgem.com/>

From: Roger Lacroix<mailto:***@ROGERS.COM>
Sent: Monday, November 11, 2013 10:00 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT>
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Paul,

> You can issue all four connects on the same thread to get the same effect.

I actually tried this when I was fooling around with the code (even though the standard is 1 connection per thread) but the MQCONN always returns the same hconn. So, I thought 'nice try but it is 1 connection per thread'.

Regards,
Roger Lacroix
Capitalware Inc.

At 04:35 PM 11/11/2013, you wrote:

Tim,

This is not unreasonable but, just to be clear to everyone listening in, it is not necessary to create four threads to issue the MQCONNs on. You can issue all four connects on the same thread to get the same effect.

Cheers,
Paul.

Paul Clarke
www.mqgem.com<http://www.mqgem.com/>
-----Original Message----- From: Tim Zielke
Sent: Monday, November 11, 2013 9:08 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Roger,

That does make more sense, but my replies were more around how to make the message consumers run on their own thread. I did not test this, but based on profiling one of our multi-threaded applications that is accomplishing what you want (message consumers on their own thread), the following may work.

In your program, create 4 of your own threads that do the following (and thus create their own unique connection):
thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
thread 4 - MQCONN, MQOPEN, MQCB, MQCTL

When you are done, you will have 4 message consumers running on their own thread in your multi-threaded process.

Thanks,
Tim

-----Original Message-----
From: MQSeries List [mailto:***@LISTSERV.MEDUNIWIEN.AC.AT] On Behalf Of Roger Lacroix
Sent: Monday, November 11, 2013 12:04 PM
To: ***@LISTSERV.MEDUNIWIEN.AC.AT
Subject: Re: MQCB and MQCTL multi-thread issue

Hi Tim,

I should have included more information. I am not looking for 3
MQGETs to be performed at the same time but rather I just want the
message consumers to be on their own thread.

Each message will be less than 1KB (probably 150-200 bytes) that the
message consumer will receive. Roughly 5% (or less) of the work
performed by the thread will be related to MQ and 95% will be backend
non-MQ work.

In this case, the bottleneck is not MQ but all of the backend work
that the message consumer needs to do. Hence, if each message
consumer is started in its own thread, then the application can do
more non-MQ work (parallel processing for the backend work).

Also, for the MQCB call, I explicitly set the GMO options field to be
MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
knows that the message consumption will NOT be in a unit of work.

Right now in my testing, I have loop (counter = 3) where it does
MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
thread for the main process and 1 thread for all of the message
consumers. So, basically I have 2 threads:

- 1 thread: the main process thread can open the queue, get messages
and do backend work
- 1 thread: all message consumers that are pushed messages then it
does the backend work

2 threads are better than one but I want 4 or maybe 8 threads, so
that the application can do lots of backend work once it receive the message.

Does that make more sense?

So, does anyone know of a flag, parameter or environment variable to
tell the MQ library to start each message consumer in its own thread?

Regards,
Roger Lacroix
Capitalware Inc.

To unsubscribe, write to ***@LISTSERV.MEDUNIWIEN.AC.AT and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/>
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to ***@LISTSERV.MEDUNIWIEN.AC.AT and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/>
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
To unsubscribe, write to ***@LISTSERV.MEDUNIWIEN.AC.AT and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/>
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

________________________________
List Archive<http://listserv.meduniwien.ac.at/archives/mqser-l.html> - Manage Your List Settings<http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1> - Unsubscribe<mailto:***@LISTSERV.MEDUNIWIEN.AC.AT?subject=Unsubscribe&BODY=signoff%20mqseries>

Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com<http://www.lsoft.com/resources/manuals.asp>

[Roger Lacroix]

Hi Paul,

I did a bunch of testing of multiple connections
(2 to 10) with and without the affinity options
and I cannot see any difference. Now all of my
testing is in bindings mode because the program
will be configured as an MQ service. Hence, I
don't need to worry about client mode. So, your
comments match what I am seeing.

Regards,
Roger Lacroix
Capitalware Inc.

At 01:52 AM 11/12/2013, you wrote:
>I agree that the manual could be a lot clearer
>and perhaps this is worthy of a manual update.
>Shared Handles have been around for many
>releases, nothing to do with async consume, it
>would be a shame if most people don't know about them.
>
>As far as using MQCTLO_THREAD_AFFINITY is
>concerned I think you probably should use it.
>It's not really what the option is for but if I
>recall it will have the right affect as a side
>effect on a client. Be aware that the threading
>on the server and client is a little different
>and, of course, may change in the future.
>However, from what I remember the situation is this:
>
>a) on the server
>A thread dedicated to the connection is created
>as soon as MQCTL is issued and the thread will
>run for the life of async consume. So,
>MQCTLO_THREAD_AFFINITY will have no effect since it is doing it anyway.
>
>b) on the client
>By default the connection is only associated
>with a thread when there are messages to be
>delivered and more than one connection can be
>associated with the same thread at any one time.
>This reduces the thread usage. For example it is
>possible to have 100 connections to the queue
>manager all with started consumers and yet there
>be 0 threads in the background to deliver the
>messages. However, if I recall correctly then
>specifying MQCTLO_THREAD_AFFINITY will cause the
>client to dedicate a thread to the connection
>just like the server. So, in the case of the
>client I think you should indeed use MQCTLO_THREAD_AFFINITY.
>
>
>Of course the bottom line is to try it and see
>what effect you see. If you do find it works
>then I think it might also be worth getting a
>manual update no this behaviour since if you are
>relying on it you don't want MQ to change the
>dispatching model underneath you. This isn't
>really what MQCTLO_THREAD_AFFINITY was about
>after all but it is an understandable by-product.
>
>Cheers,
>Paul.
>
>Paul Clarke
>www.mqgem.com
>
>From: <mailto:roger.lacroix-***@public.gmane.org>Roger Lacroix
>Sent: Tuesday, November 12, 2013 12:26 AM
>To:
><mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org>MQSERIES-0lvw86wZMd9k/***@public.gmane.orgAT
>Subject: Re: MQCB and MQCTL multi-thread issue
>
>Hi Paul,
>
> > No it isn't Roger...you should read the MQI API Reference more carefully.
>
>I may know a lot about MQ but you most certainly know a lot more!!
>
> > By default MQHCONNs are bound to a single
> thread but you can connect and ask for a shared
> connection handle using something like
> MQCNO_HANDLE_SHARE_BLOCK. With this option you
> can create as many MQHCONNs on the same thread
> as you like and use those MQHCONNs on any thread you like.
>
>How in the world did you get that from the manual?
>
>For MQCNO_HANDLE_SHARE_BLOCK, the manual says
>
>Creates a shared connection. On a
>MQCNO_HANDLE_SHARE_BLOCK connection, if the
>connection is currently in use by an MQI call on
>another thread, the MQI call waits until the current MQI call has completed.
>
>Where does it say that option allows you to
>create multiple connections per thread?
>
>Don't get me wrong - this IS what I was looking
>for. I added the option and put MQCONNX,
>MQOPEN, MQCB and MQCTL in a loop and viola, I
>get the number of threads for the message
>consumer as the loop counter is set for. Bingo.
>
>Should I be using MQCTLO_THREAD_AFFINITY option
>with MQCTL? I tried it with and without but it
>did not seem to make any difference.
>
>Regards,
>Roger Lacroix
>Capitalware Inc.
>
>At 05:09 PM 11/11/2013, you wrote:
>>No it isn’t Roger...you should read the MQI
>>API Reference more carefully. By default
>>MQHCONNs are bound to a single thread but you
>>can connect and ask for a shared connection
>>handle using something like
>>MQCNO_HANDLE_SHARE_BLOCK. With this option you
>>can create as many MQHCONNs on the same thread
>>as you like and use those MQHCONNs on any thread you like.
>>
>>Cheers,
>>Paul.
>>
>>Paul Clarke
>>www.mqgem.com
>>
>>From: <mailto:roger.lacroix-***@public.gmane.org>Roger Lacroix
>>Sent: Monday, November 11, 2013 10:00 PM
>>To:
>><mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org>MQSERIES-0lvw86wZMd9k/***@public.gmane.org.AT
>>Subject: Re: MQCB and MQCTL multi-thread issue
>>
>>Hi Paul,
>>
>> > You can issue all four connects on the same thread to get the same effect.
>>
>>I actually tried this when I was fooling around
>>with the code (even though the standard is 1
>>connection per thread) but the MQCONN always
>>returns the same hconn. So, I thought 'nice
>>try but it is 1 connection per thread'.
>>
>>Regards,
>>Roger Lacroix
>>Capitalware Inc.
>>
>>At 04:35 PM 11/11/2013, you wrote:
>>>Tim,
>>>
>>>This is not unreasonable but, just to be clear
>>>to everyone listening in, it is not necessary
>>>to create four threads to issue the MQCONNs
>>>on. You can issue all four connects on the same thread to get the same effect.
>>>
>>>Cheers,
>>>Paul.
>>>
>>>Paul Clarke
>>>www.mqgem.com
>>>-----Original Message----- From: Tim Zielke
>>>Sent: Monday, November 11, 2013 9:08 PM
>>>To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>>>Subject: Re: MQCB and MQCTL multi-thread issue
>>>
>>>Hi Roger,
>>>
>>>That does make more sense, but my replies were
>>>more around how to make the message consumers
>>>run on their own thread. I did not test this,
>>>but based on profiling one of our
>>>multi-threaded applications that is
>>>accomplishing what you want (message consumers
>>>on their own thread), the following may work.
>>>
>>>In your program, create 4 of your own threads
>>>that do the following (and thus create their own unique connection):
>>>thread 1 - MQCONN, MQOPEN, MQCB, MQCTL
>>>thread 2 - MQCONN, MQOPEN, MQCB, MQCTL
>>>thread 3 - MQCONN, MQOPEN, MQCB, MQCTL
>>>thread 4 - MQCONN, MQOPEN, MQCB, MQCTL
>>>
>>>When you are done, you will have 4 message
>>>consumers running on their own thread in your multi-threaded process.
>>>
>>>Thanks,
>>>Tim
>>>
>>>-----Original Message-----
>>>From: MQSeries List
>>>[mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Roger Lacroix
>>>Sent: Monday, November 11, 2013 12:04 PM
>>>To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
>>>Subject: Re: MQCB and MQCTL multi-thread issue
>>>
>>>Hi Tim,
>>>
>>>I should have included more information. I am not looking for 3
>>>MQGETs to be performed at the same time but rather I just want the
>>>message consumers to be on their own thread.
>>>
>>>Each message will be less than 1KB (probably 150-200 bytes) that the
>>>message consumer will receive. Roughly 5% (or less) of the work
>>>performed by the thread will be related to MQ and 95% will be backend
>>>non-MQ work.
>>>
>>>In this case, the bottleneck is not MQ but all of the backend work
>>>that the message consumer needs to do. Hence, if each message
>>>consumer is started in its own thread, then the application can do
>>>more non-MQ work (parallel processing for the backend work).
>>>
>>>Also, for the MQCB call, I explicitly set the GMO options field to be
>>>MQGMO_NO_SYNCPOINT | MQOO_FAIL_IF_QUIESCING, so that the MQ library
>>>knows that the message consumption will NOT be in a unit of work.
>>>
>>>Right now in my testing, I have loop (counter = 3) where it does
>>>MQOPEN, MQCB and MQCTL calls. The result is that I end up with 1
>>>thread for the main process and 1 thread for all of the message
>>>consumers. So, basically I have 2 threads:
>>>
>>>- 1 thread: the main process thread can open the queue, get messages
>>>and do backend work
>>>- 1 thread: all message consumers that are pushed messages then it
>>>does the backend work
>>>
>>>2 threads are better than one but I want 4 or maybe 8 threads, so
>>>that the application can do lots of backend
>>>work once it receive the message.
>>>
>>>Does that make more sense?
>>>
>>>So, does anyone know of a flag, parameter or environment variable to
>>>tell the MQ library to start each message consumer in its own thread?
>>>
>>>Regards,
>>>Roger Lacroix
>>>Capitalware Inc.
>>>
>>>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>>>in the message body (not the subject), write: SIGNOFF MQSERIES
>>>Instructions for managing your mailing list subscription are provided in
>>>the Listserv General Users Guide available at http://www.lsoft.com
>>>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>>>
>>>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>>>in the message body (not the subject), write: SIGNOFF MQSERIES
>>>Instructions for managing your mailing list subscription are provided in
>>>the Listserv General Users Guide available at http://www.lsoft.com
>>>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>>>To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
>>>in the message body (not the subject), write: SIGNOFF MQSERIES
>>>Instructions for managing your mailing list subscription are provided in
>>>the Listserv General Users Guide available at http://www.lsoft.com
>>>Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
>>
>>
>>----------
>><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List
>>Archive -
>><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>>Your List Settings -
>><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>>
>>
>>Instructions for managing your mailing list
>>subscription are provided in the Listserv
>>General Users Guide available at
>><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com
>>
>>
>>----------
>><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List
>>Archive -
>><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>>Your List Settings -
>><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>>
>>
>>Instructions for managing your mailing list
>>subscription are provided in the Listserv
>>General Users Guide available at
>><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com
>
>
>----------
><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List
>Archive -
><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>Your List Settings -
><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>
>
>Instructions for managing your mailing list
>subscription are provided in the Listserv
>General Users Guide available at
><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com
>
>
>----------
><http://listserv.meduniwien.ac.at/archives/mqser-l.html>List
>Archive -
><http://listserv.meduniwien.ac.at/cgi-bin/wa?SUBED1=mqser-l&A=1>Manage
>Your List Settings -
><mailto:LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org?subject=Unsubscribe&BODY=signoff%20mqseries>Unsubscribe
>
>
>Instructions for managing your mailing list
>subscription are provided in the Listserv
>General Users Guide available at
><http://www.lsoft.com/resources/manuals.asp>http://www.lsoft.com

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html