Category: News

Long Overdue

Posted by chrisdag – November 18, 2011

You are looking at the same old blog with a new foundation. After years of dealing with the hassle of the Typo blogging engine I’ve finally migrated the site over to the WordPress blogging platform. The blog does not look much different yet as the ‘scribish’ theme is available on both platforms.

The move to wordpress will stop the ruby and RAILS-related memory leaks that were causing this site to fall over all the time. I’m keeping a careful eye on the error logs so I can find and catch any permalink, RSS or XML related errors. Hopefully with enough work I can put in rewrite or redirect rules so that any link that worked on the “old” gridengine.info will continue to work. If you notice anything odd or want to highlight something that is broken please feel free to drop me a line at ‘dag@sonsorol.org’.

say hello to gridengine.org

Posted by chrisdag – January 28, 2011

http://gridengine.org is now live.

I chose this URL, gridengine.info as the base domain for this site because I did not want to step on any community toes and I also wanted to ensure that the site was seen as mostly a personal endeavor rather than a corporate thing I did on behalf of my employer. To be honest I was also playing around with ruby-on-rails at the time and this site gave me an excuse to deploy the Typo rails blogging engine.

When securing gridengine.info, the gridengine.org domain was also available at the time so I also picked it up. In the years since I bought that domain it has been set up to do nothing but redirect web surfers to the “official” open source Grid Engine site at http://gridengine.sunsource.net. I considered it a good use and the money spent was an inexpensive protection against domain squatters or other non-friendly interests.

With gridengine.sunsource.net dead and gone I needed to do something different with gridengine.org.

Well, it’s time to do something different. I’ve hooked up with an unofficial group of Grid Engine supporters and it looks like we are going to try to use the domain to help organize and bootstrap a new open source grid engine community in the post-Oracle era.

Not sure how much I’ll continue to post here, a lot of the helpful/howto type articles would be a natural fit to start posting over at the other site, especially if we get a team of bloggers and wiki documentation writers going.

Anyway, please spread the word about http://gridengine.org – we need help reconstituting the users- mailing list and regenerating documentation.

There are still a ton of unanswered questions about what the free and open source grid engine (or gridengine(s) !!) will look like. Lots of stuff is being hashed out and things might move a bit slowly as there are legal implications as well that need careful attention.

Welcome Back Grid Engine!

Posted by chrisdag – January 18, 2011

Grid Engine Lives!

In fantastic news, the principal engineers of the Grid Engine team have left Oracle and are going to keep on doing what they do best under the new umbrella of Univa.

Read the full press release here:

http://univa.com/about/news/press_2011/01172011.php

This is fantastic news. My main concern with the longevity of the various open source forks was how long they could continue in the absence of deep developer knowledge of the codebase. The forks can’t live long with just a parasitic community of users (like myself!) far outweighing the ratio of developers able to commit bug fixes and otherwise work on the code internals. We were optimistic already because the forks were being run by smart people but today’s announcement really and truly means that Grid Engine is going to live on in a free-to-use form.

More postings once I get my head around the news and can discern fact from PR speak!

Update:

I do “own” the gridengine.org domain name as well as this gridengine.info one.

Since there is no trademark issue with Oracle over the use of “Grid Engine” you might see gridengine.org pop up in a more useful capacity in the future.

 

 

 

Goodbye Grid Engine

Posted by chrisdag – December 24, 2010

As expected by many, Oracle has finally and clearly announced their plans to close down and stop work on an Open Source version of Grid Engine. Moving forward Grid Engine is now “Oracle Grid Engine” available as a commercially licensed software product.

Of note:

  • Oracle explicitly references open source Grid Engine fork now called Open Grid Scheduler. This is a relief to many who were hoping for some sort of signal of intent or attitude towards community efforts. A non-hostile Oracle is a good thing for people wanting to make use of the open source codebase.
  • Oracle is going further than “just words” – as the CollabNet site (gridengine.sunsource.net) site is dismantled they will be passing the components to the community. This includes the full mailing list archives, CVS repository etc.

There are two announcements you should read:

Both messages contain information and updates about plans and roadmaps that you may be interested in.

 

 

 

Grid Engine on the new Amazon Compute Cluster Instances

Posted by chrisdag – July 13, 2010

{ crossposted to blog.bioteam.net and gridengine.info }

Amazon made a very important announcement today, releasing new EC2 server types and network configurations that significantly enhance the Amazon AWS environment for people who are interested in cluster computing, compute farming and high performance computing (HPC) on the cloud.

The announcement is here for those who are interested:

http://aws.typepad.com/aws/2010/07/the-new-amazon-ec2-instance-type-the-cluster-compute-instance.html

I’m thrilled that this news is now public, the service is up for use and I can finally start testing, blogging and benchmarking in the “real” production environment.

In the next few days I’ll be blogging over on http://blog.bioteam.net, concentrating initially on seeing how storage and storage IO speeds differ on the new instance types. For life science types like myself, one of the biggest hassles in the cloud is due to the fact that we tend to be more performance bound by the speed of storage and file IO than anything else. The 10GbE networking changes and non-oversubscription of the network links along with the ability to group nodes together may mean very very interesting things are now much more feasible on the AWS platform.

Because I’m going to first concentrate on storage and IO stuff on the new offering I wanted to quickly show Grid Engine running on the new server types.

Even a single node SGE cluster can do reasonable work now as the cc1 instance type includes a pair of quad-core Nehalem CPUs along with ~23GB memory and a 10GbE ethernet backend.

We will be blogging and talking much more about how to use Chef Server to orchestrate self-assembling Grid Engine clusters and compute farms on this new service but since that may not happen until later — I just wanted to throw up a teaser post showing SGE 6.2u5 running in single-node mode on the new HPC offerings from Amazon.

qstat output showing 16 CPUs (click for full-size):

sge-cc1-1.png

 

qhost output showing system resources (click for full-size):

sge-cc1-2.png

SGE 6.2u4 update is out today

Posted by chrisdag – October 23, 2009

This is a bugfix/maintenance release, read the full announcement here.
.

As always, checking the list of fixed bugs and issues is a good way to start deciding if an upgraded is needed and how urgent it may be.

SGE workshop pictures

Posted by chrisdag – September 10, 2009

Photos from the 2009 Grid Engine Workshop in Germany. Please help me name and tag the participants!

Also, my talk on “Grid Engine & Amazon Compute Cloud” is online here:
http://blog.bioteam.net/2009/09/09/grid-engine-amazon-ec2/. The other talks and training slides are slowly making their way onto the net. We should have video from all the talks as well.

www.flickr.com

This is a Flickr badge showing items in a set called 2009 Grid Engine Workshop (Germany). Make your own badge here.

Greetings from Germany

Posted by chrisdag – September 7, 2009

sge_germany_small.jpg

The 2009 Sun HPC Workshop started today with some tutorial sessions. Dan T and I are running a Grid Engine Administration workshop right next door to the Lustre folks.

It’s alive

Posted by chrisdag – August 31, 2009

xmlqstat.png

Grid Engine 6.2 Update 3 is out

Posted by chrisdag – June 23, 2009

Important Note: Sun has changed the license terms for this release. The full release from Sun.com can only be used for 90 days for free. The courtesy binaries are still free for all to use but the distribution will not include the Amazon EC2 cloud adaptor or the excellent “sgeinspect” tool. Source code for both of these components is available under the SISSL license so theoretically community members can build versions for themselves.

The full release announcement is here:

http://gridengine.sunsource.net/news/SGE62u3-announce.html

For me, the most important new features are the SGEInspect tool (screenshots of which you can view online at http://www.flickr.com/photos/chrisdag/sets/72157617805352910/ and the exclusive host scheduling feature which now removes the need for PE-based ‘hacks’ to achieve the same goal.

The license change is interesting, I need to see how hard it is to build sgeinspect from source code, it really is a powerful new tool. It’s a shame that this won’t be part of the free distribution but then again I want Sun to make product and support revenue off of SGE so I can see the point.