gridengine.info :

SGE 6.1u5 update is out

dag@sonsorol.org (chris) — Thu, 21 Aug 2008 17:48:10 -0400

Update release SGE 6.1u5 is out today, the announcement can be read at http://gridengine.sunsource.net/news/GE61u5-announce.html and the list of bugs fixed since the last release in the 6.1 series can be found here: http://gridengine.sunsource.net/project/gridengine/61patches.txt.

Screencast showing online upgrade to SGE 6.2

dag@sonsorol.org (chris) — Thu, 21 Aug 2008 10:48:19 -0400

Lubomir Petrik has posted a screencast recording showing the SGE 6.x to SGE 6.2 upgrade process. Thanks to Andy for finding and reporting this.

T-Shirt Contest

dag@sonsorol.org (chris) — Wed, 06 Aug 2008 13:16:07 -0400

Want a T-shirt? Be quick and email Andy. Details below.

Do you want to win a truly nice open source T-Shirt?

There are T-shirts to win in three categories:

1. Among the first 50 of you who reply *directly* to me (andy.schwierskott@sun.com) and tell us what is the single most important or interesting feature in SGE 6.2 for you, we'll draw three T-shirts.

2. Three T-Shirts goes to those persons who first report that they have upgraded their production cluster to SGE 6.2. Test-beds, eval clusters, private use doesn't count.

3. Three T-Shirts goes to those persons who will be using SGE for the first time, be it because you replace another DRM system or be it because you start using a DRM system for the first time. Requirements: it must be SGE 6.2 and it must be production use, not just a test-bed, private use or eval cluster.

We'll respect your privacy and only make your name public if you agree to it! Sun Microsystems employees may not participate.

Please feel free to populate this announcement and 'lottery' to mailing lists who take care about the SGE technology.

Regards, Andy

6.2 Officially Out

dag@sonsorol.org (chris) — Tue, 05 Aug 2008 12:13:26 -0400

Grid Engine 6.2 is officially out, follow the links in the blog post below to read DanT's excellent set of articles on "why upgrade to 6.2?".

Get it here:
http://www.sun.com/software/gridware/

This also marks the official transition to having all of the Sun SGE documentation and manuals in wiki form:
http://wikis.sun.com/display/GridEngine/Grid+Engine

SGE 6.2 Coming August 5th

dag@sonsorol.org (chris) — Sun, 03 Aug 2008 12:28:10 -0400

Unofficial word is that the official release of SGE 6.2 is coming on Tuesday, August 5th.

To read up on why this is news, check out Dan's excellent essays:

Site update

dag@sonsorol.org (chris) — Tue, 22 Jul 2008 11:21:00 -0400

If you are reading this and seeing a peach colored background then you are seeing the new default Typo 5.1.1 theme. I updated the RAILS and Typo installation on this box today and decided to at least experiment with the new theme for a bit. The article area is wider which I like (and I never got back into the Scribbish 2.0 theme to make that wider...) so I may take the lazy way out and stick with this theme.

Leave a comment if you like this theme or if you'd prefer the white-background "Scribbish 2.0" theme. Thanks!

Update: I'm in the process of editing the CSS stylesheets for the new theme that shipped with Typo 5.1.1, I like the white background, fonts and wider width of the articles. Hopefully I can get the sidebar better outlined and make a few other CSS tweaks...

Update 2: Tired of tweaking; back to the default Scribbish theme it is ...

Collection of usable ARCo queries

dag@sonsorol.org (chris) — Tue, 22 Jul 2008 09:03:13 -0400

Thanks to Jana, there is a new wiki page up at http://wiki.gridengine.info/wiki/index.php/ARCoQueries that documents some additional contributed ARCo queries for the Grid Engine Analysis & Reporting Console.

Why upgrade? DanT explains SGE from 5.x through 6.2 and beyond

dag@sonsorol.org (chris) — Fri, 18 Jul 2008 14:58:52 -0400

Dan has posted a great overview of how Grid Engine has changed since the version 5.x days, couched in the context of answering the "Why should I upgrade SGE?" questions that often come up.

I won't even excerpt it, the full article is well worth a read:
http://blogs.sun.com/templedf/entry/why_upgrade

Univa UD UniCluster Express (and Amazon EC2)

dag@sonsorol.org (chris) — Mon, 14 Jul 2008 19:55:25 -0400

Univa is a company that I became familiar with when Steve Tuecke gave a talk at the 2007 Grid Engine workshop in Regensberg, Germany. Steve's talk was called "Univa Open Source HPC Cluster and Grid Software" and you can download the PDF version here.

Steve's talk and his company were pretty interesting and he specifically hit on some areas I had mentioned in my talk on "LSF vs. SGE" where we discussed the lack of ISV and commercial entities capable of supporting the Grid Engine community with service, support, consulting, training and other "value added" activities. I came home from Germany thinking that Univa would be worth watching (also Steve had dropped some hints informally about 'some news may be happening soon...') -- and sure enough a short time later Univa became "Univa UD" as Univa and United Devices became a single company.

If you point a web browser over at http://grid.org or the corporate site at http://www.univaud.com you'll quickly see that they have a product out called "UniCluster Express".

Cluster stacks are normally not all that exciting to me but Univa is doing something different that makes it worth checking out ...

Click on through for the full article ...

The UniCluster Express stack is free, 100% open-source and contains the following core components: Grid Engine, Grid Engine Analysis & Reporting Console (ARCo), Ganglia, Globus and a custom Java based central monitoring console application.

But that is not the really interesting part. The eye catching bit is that Univa UD is offering free support for this integrated stack.

Yep.

Free support for an integrated stack including some really complex bits like SGE ARCo and the impossible-for-most-mortals Globus stack (heck, I attended a GGF meeting and actually sat through the Globus tutorial and still could barely get it functioning in my lab environment...)

Univa UD has taken already popular open-source compute farming technologies, added in a new Java based monitoring console and bundled everything up into a installable tarball. Free support comes through grid.org and I'm guessing that they plan to actually make money by offering customization, commercial support and other services to people willing to pay.

I recently took the software for a spin myself and got it integrated into the Amazon Web Services EC2 elastic compute cloud framework. My work was written up as a HOWTO whitepaper and was documented in a series of recorded screencasts.

One of the screencasts is linked to at the top of the article, all of the others are available by following this URL:
http://www.screencast.com/users/BioTeam/folders/UniCluster-in-Amazon-EC2

If you want the PDF Whitepaper you'll have to get it via a web form over at http://www.univaud.com/hpc/wp-unicluster-amazon-ec2.php.

Disclaimer/Disclosure: In my role as part-owner/employee of BioTeam, my company earned money on this project. Univa UD basically challenged us to get their stuff working in EC2 and offered to pay for a few days of consulting time. A more detailed disclaimer can be found within the whitepaper.

SGE and MPICH2 On Windows/Linux Heterogenous Systems

dag@sonsorol.org (chris) — Mon, 14 Jul 2008 19:03:29 -0400

Thanks to Jacek Strzelczyk for the new Wiki page entitled "Install and configure Grid Engine in heterogenic environment on Linux and Windows with MPICH2" that was posted earlier this week.

Feedback needed: Obsolete options and parameters considered for removal

dag@sonsorol.org (chris) — Tue, 24 Jun 2008 08:22:41 -0400

Grid Engine developers posted a list today of SGE configuration parameters and client arguments that are being considered for removal from the product because they are either obsolete or they duplicate settings found elsewhere.

The developers are seeking feedback and comments on their plans - if you have any please drop a line to the users@gridengine.sunsource.net mailing list. The current roadmap calls for these methods to be marked as 'deprecated' in the SGE 6.2 release with total removal planned for a future post-6.2 release.

The message can be found here:
http://gridengine.sunsource.net/servlets/ReadMsg?list=users&msgNo=25045

The full list of items being considered for removal can also be found after the jump ...

The parameters planned to obsolete are:

host_conf(5)
- processors
   obsolete, same as num_proc from the complex list

sched_conf(5)
- algorithm
   just default is allowed, no additional algorithms are planed
- params JC_FILTER
   huge performance impact plus may lead to wrong scheduling decisions

sge_conf(5)
- reprioritize
   redundant because hard bound to reprioritize_interval in sched_conf(5)
- shell_start_mode
   obsolete, value from queue_conf(5) is used
- set_token_cmd
   no known AFS support
- pag_cmd
   no known AFS support
- token_extend_time
   no known AFS support
- qmaster_params DISABLE_AUTO_RESCHEDULING
   equivalent to default reschedule_unknown=0:0:0
- qmaster_params merge ACCT_RESERVED_USAGE and SHARETREE_RESERVED_USAGE
   We can't imaging a use case to have these values separated
- finished_jobs
   qstat -j does not work with successful finished jobs. Code seems
   to work only with jobs going into error state.

user(5)
- delete_time
   change to internal, not changeable/visible field
   Implicit set by auto_user_delete_time

qconf(1)
- sep option
   obsolete, same as num_proc
- ks option
   obsolete, same as -kt scheduler

qmod(1)
- c option
   depreciated, use -cj or -cq
- r option
   depreciated, use -rj or -rq
- s option
   depreciated, use -sj or -sq
- us option
   depreciated, use -usj or -usq

SGE 6.2 beta 2 is out

dag@sonsorol.org (chris) — Thu, 19 Jun 2008 07:04:12 -0400

6.2b2 came out yesterday:

http://gridengine.sunsource.net/news/GE62beta2-announce.html

The list of bug fixes made since SGE 6.2 Beta 1 is online at http://gridengine.sunsource.net/project/gridengine/62patches.txt.

This is the latest beta release of SGE 6.2 and we really need more eyeballs and testers on this release to flesh out any remaining issues before 6.2 goes officially out the door.

There are some differences in 6.2 both in the install procedure as well as the daemons (sge_schedd is gone! -- It's now a thread within sge_qmaster). I posted a screencast recording of the SGE 6.2 Beta 1 installation a while back: http://gridengine.info/articles/2008/05/16/screencast-live-install-of-sge6-2-beta for those that may be interested in watching what the new install process looks like.

How to run Dytran applications under Grid Engine

dag@sonsorol.org (chris) — Wed, 28 May 2008 11:44:29 -0400

Gerhard Venter asked the users list for assistance in getting Dytran to run under Grid Engine. Once his issues were resolved, Gerhard was kind enough to write up a Wiki Entry on Dytran/SGE integration.

The wiki page is here:
http://wiki.gridengine.info/wiki/index.php/Dytran

Thanks Gerhard!

Dan reviews Grid Engine queues

dag@sonsorol.org (chris) — Wed, 28 May 2008 11:33:34 -0400

Dan is back and blogging up a storm, he recently posted a nice overview of Grid Engine queue basics. An excerpt is here:

... So, aside from governing the number of free slots on a host, what does a queue do? It controls the execution context of jobs that run in it. It determines what parallel environments are available, what file, memory, and CPU time limits should be applied, how the job should be started, stopped, suspended, and resumed, what the job's process' nice value is, etc. ...

Read the full article here:
http://blogs.sun.com/templedf/entry/intro_to_grid_engine_queues

Java DRMAA binding via JavaScript

dag@sonsorol.org (chris) — Wed, 28 May 2008 11:27:59 -0400

%  jrunscript -cp $SGE_ROOT/lib/drmaa.jar -f drmaa.js

Job 2 submitted
Job 2 has ended
Job terminated abnormally
%

Richard Hierlmeier seems to have joined the ranks of Sun Bloggers and has a facinating post up documenting how he used the javascript engine that ships with Java 6 to bind to drmaa.jar.

The post is here:

http://blogs.sun.com/rhierlmeier/entry/java_drmaa_binding_with_javascript