Seperate File Systems for logs and queues - on virtual Linux servers too?

[Potkay, Peter M (CTO Architecture + Engineering)]

Historically we always broke out /var/mqm and /var/mqm/logs onto separate disks on a physical server. Then when we started heavy with physical servers in VCS clusters we made a separate cluster resource for the virtual drive for the logs and another virtual drive for the rest of the queue manager. Although with all the SAN storage in the back end a giant pool of disks fronted with cache I wondered how much benefit there really was.

We are standing up a bunch of virtual RHEL 6 servers on VMware. We are not using the Hypervisor edition. The whole virtual server is nothing but a vmdk file on SAN now. The virtual server is running in a physical ESX cluster among dozens of other virtual servers, all sharing the same physical I/O to the back end storage that hosts my virtual server and all the other virtual servers, and any additional virtual disks I may ask to have assigned to my virtual server.

So is there any benefit to making /var/mqm and /var/mqm/logs separate file systems on a virtual server? I did not find anything on IBM's site, and my PMR confirmed that unless I can guarantee separate distinct physical disks for the logs versus the queues there was no reason to make /var/mqm and /var/mqm/logs separate file systems. I can guarantee that I will NOT be able to have separate physical disks for my file systems in our SAN environment.

So I'm leaning to just asking for a single /var/mqm file system so I have a dedicated amount of disk capacity for my queue manager, and not bother with a separate file system for the logs.

Thoughts?

Oh, I always wondered, if /var/mqm/logs is separate file system, is there still a reason to break out /var/mqm/qmgrs from /var/mqm?

Peter Potkay




************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

[Andy Hickson]

From a performance perspective, anyone interested in MQ persistent message
performance is highly likely to have some form of fast write cache, and
hence the historical reasons for /var/mqm/qmgrs and /var/mqm/log being on
separate file systems no longer applies.
However from a space management and monitoring perspective there are still
strong reasons to keep /var/mqm, /var/mqm/qmgrs (or equivalent) and
/var/mqm/log (or equivalent) on separate file systems.

As a simple example, imagine that putting a large number of messages to a
queue causes space issues. If /var/mqm and /var/mqm/qmgrs are on the same
file system then the queue manager may be unable to write suitable
diagnostics detailing the resource constraint . For example, such a resource
constraint could also then cause the queue manager to not be able to create
further IPC resources which might inhibit new queue manager processes from
starting.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

[Potkay, Peter M (CTO Architecture + Engineering)]

Thank you Andy, this is helpful. And it explains why one might want to split up /var/mqm and /var/mqm/qmgrs, even if you had already split off your /var/mqm/logs.

Now, if the main benefit is related to space management and monitoring, and you are using circular logs, the amount of space being used by the circular logs is going to be static. An argument for not needing to break our /var/mqm/logs from /var/mqm? (Although you still may want to break out /var/mqm/qmgrs where the queues live for the reason mentioned below)



Another question, maybe best answered by a Linux guru, but of interest to us MQ Admins I think. In a virtual server where all your storage is mixed up with other virtual servers, is there some benefit from separate file systems that the operating system of the virtual server might be able to take advantage of for some parallel I/O processing in scope of the one virtual server, meaning maybe MQ might be able to read/write faster on the one virtual servers because you took the time to split the logs and queues onto separate file systems?



Peter Potkay



-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Andy Hickson
Sent: Wednesday, July 24, 2013 2:34 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Seperate File Systems for logs and queues - on virtual Linux servers too?

From a performance perspective, anyone interested in MQ persistent message performance is highly likely to have some form of fast write cache, and hence the historical reasons for /var/mqm/qmgrs and /var/mqm/log being on separate file systems no longer applies.

However from a space management and monitoring perspective there are still strong reasons to keep /var/mqm, /var/mqm/qmgrs (or equivalent) and /var/mqm/log (or equivalent) on separate file systems.

As a simple example, imagine that putting a large number of messages to a queue causes space issues. If /var/mqm and /var/mqm/qmgrs are on the same file system then the queue manager may be unable to write suitable diagnostics detailing the resource constraint . For example, such a resource constraint could also then cause the queue manager to not be able to create further IPC resources which might inhibit new queue manager processes from starting.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

[Andy Hickson]

Peter,
Although using circular logs puts an upper bound on the amount of log space
needed for the logs, it doesn't imply that additional logs won't be required.
If the event that a secondary log extent needs to be allocated it would be
important for queue manager availability (but not message integrity) that
space exists in which the log can be created.

From a performance tuning perspective it is also very useful to be able to
monitor the relative activity to the queue manager data and to the recovery
logs.

For these reasons I would still recommend the typical use of three file
systems for production MQ deployment.

Regards
Andy.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES

[Tim Zielke]

Hi Peter,

I am not sure if this is exactly what you were asking, but the Linux kernel does provide parallel flush threads (or pdflush depending on the 2.6 kernel version) to increase the performance of writing to the block devices. This is called congestion avoidance.

So for example, if the /var/mqm filesystem writes to block device sda4 and the /var/mqm-log filesystem writes to block device sda5, if the I/O writes get busy enough you can have flush threads writing to both block devices to improve the I/O performance.

So having more MQ filesystems that write to separate block devices does potentially increase the I/O performance. But I would think that would only be useful if your server is heavily I/O driven for the MQ filesystems, where the kernel needs to run a lot of parallel flush threads to keep up with the I/O demand. We just do the recommended /opt/mqm, /var/mqm, and /var/mqm-log for separate MQ file systems on Linux/Unix.

Thanks,
Tim


-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Potkay, Peter M (CTO Architecture + Engineering)
Sent: Wednesday, July 24, 2013 9:38 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Seperate File Systems for logs and queues - on virtual Linux servers too?

Thank you Andy, this is helpful. And it explains why one might want to split up /var/mqm and /var/mqm/qmgrs, even if you had already split off your /var/mqm/logs.

Now, if the main benefit is related to space management and monitoring, and you are using circular logs, the amount of space being used by the circular logs is going to be static. An argument for not needing to break our /var/mqm/logs from /var/mqm? (Although you still may want to break out /var/mqm/qmgrs where the queues live for the reason mentioned below)



Another question, maybe best answered by a Linux guru, but of interest to us MQ Admins I think. In a virtual server where all your storage is mixed up with other virtual servers, is there some benefit from separate file systems that the operating system of the virtual server might be able to take advantage of for some parallel I/O processing in scope of the one virtual server, meaning maybe MQ might be able to read/write faster on the one virtual servers because you took the time to split the logs and queues onto separate file systems?



Peter Potkay



-----Original Message-----
From: MQSeries List [mailto:MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org] On Behalf Of Andy Hickson
Sent: Wednesday, July 24, 2013 2:34 PM
To: MQSERIES-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org
Subject: Re: Seperate File Systems for logs and queues - on virtual Linux servers too?

From a performance perspective, anyone interested in MQ persistent message performance is highly likely to have some form of fast write cache, and hence the historical reasons for /var/mqm/qmgrs and /var/mqm/log being on separate file systems no longer applies.

However from a space management and monitoring perspective there are still strong reasons to keep /var/mqm, /var/mqm/qmgrs (or equivalent) and /var/mqm/log (or equivalent) on separate file systems.

As a simple example, imagine that putting a large number of messages to a queue causes space issues. If /var/mqm and /var/mqm/qmgrs are on the same file system then the queue manager may be unable to write suitable diagnostics detailing the resource constraint . For example, such a resource constraint could also then cause the queue manager to not be able to create further IPC resources which might inhibit new queue manager processes from starting.

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html

To unsubscribe, write to LISTSERV-0lvw86wZMd9k/bWDasg6f+***@public.gmane.org and,
in the message body (not the subject), write: SIGNOFF MQSERIES