Tag: Resource Quotas

multiple queues without host oversubscription

Posted by chrisdag – January 11, 2012

Another good tip from the mailing list …

Is there a way to assign node to multiple queues but not make the
node over subscribed? Are there any tips or reference site with the
information?



As Reuti said, one option is an RQS (resource quota set). I stole this
one from somewhere (my cluster has hosts with differing numbers of CPU
cores):


{
    name         limit_slots_to_cores_rqs
    description  Prevents core oversubscription across queues.
    enabled      TRUE
    limit        hosts {*} to slots=$num_proc
}

It’s official: 6.1 snapshot is out; major new enhancements

Posted by chrisdag – December 13, 2006

Highlights:

  • Preview release only, test carefully before even remotely considering production use
  • A tentative beta release of SGE 6.1 is scheduled for February 2007
  • No official data for full 6.1 release; official release may have additional features or components
  • A HUGE milestone with major new functionality

Read the official announcement here:

The most exciting new feature is a MAJOR step forward for the project and the product – a flexible system for implementing Resource Quotas. This feature is being developed to address at a minimum some of the biggest and most vexing configuration limitations encountered by the user community:

  • Issue #: 74: — Supporting maxujobs on a per host level
  • Issue #: 1532: — Allowing “max jobs per user” limits on a per queue basis
  • Issue #: 1644: — Allowing per-user slot limits to be set within parallel environments (PE’s).

Long time participants on the SGE mailing lists will recognize the above issues as some of the most commonly reported feature and enhancement requests rising out of the user community. The developers and project leads deserve sincere congratulations for pushing this enhancement through. The specification document looks well thought out and will likely be the foundation for future clever resource quota methods used by SGE Administrators and cluster operators.


Other additions to the 6.1 snapshot include:

  • Official support for Mac OS X on Intel and Linux on Itanium
  • ARCo joins the codebase (as reported previously)
  • The PDC patches supplied by the user community were accepted and now allow for better usage data collection on Apple Mac OS/X, IBM AIX and HP HP/UX
  • Helpful scripts and documentation for Solaris 10 users wishing to use the amazing DTrace tool for bottleneck identification and tuning