This topic is one which has been under discussion for some time now. The basic idea is that using the JAXB RI from the JWSDP, we could build a set of classes which would parse qstat output, making it trivial for a developer to write an app which keeps tabs on Grid Engine. I believe the final decision was that we would not officially include such classes with Grid Engine for supportability reasons. Instead, I’m going to explain to you how to build the classes yourself. If you’re too lazy to follow these instructions, here’s a tarball of the classes I generated while writing this post, along with the source, etc.
The first thing you need to do is make sure you have the JavaTM platform, the latest JWSDP, Ant, and at least Grid Engine 6.0u7 (or an equivalent maintrunk source build) installed.
I am going to refer to several directories in this tutorial. They are:
| $JWSDP_HOME |
Root of JWSDP install |
| $JAXB_HOME |
$JWSDP_HOME/jaxb |
| $SGE_ROOT |
Root of Grid Engine install |
| $SCHEMA_HOME |
$SGE_ROOT/util/resources/schemas/qstat |
| $BIND_HOME |
Diretory where you’ll generate the classes |
Note that you don’t need to have the above set as environment variables in your shell, but if you do, you can just copy and paste commands from this tutorial.
The first step is to create an external bindings file. This file will provide the JAXB class generator with some additional information. Specifically, the external bindings file will 1) assign a package to the generated files and 2) fix a naming conflict in the qstat.xsd schema. Create a file called $SCHEMA_HOME/qstat.xjb with the following contents:
<?xml version="1.0" encoding="UTF-8"?>
<jxb:bindings version="1.0"
xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<jxb:bindings schemaLocation="qstat.xsd" node="/xs:schema">
<jxb:bindings node="//xs:complexType[@name='job_list_t']">
<jxb:bindings node="//xs:attribute[@name='state']">
<jxb:property name="stateAttribute"/>
</jxb:bindings>
</jxb:bindings>
<jxb:schemaBindings>
<jxb:package name="com.sun.grid.xml.qstat"/>
</jxb:schemaBindings>
</jxb:bindings>
</jxb:bindings>
|
This file sets the package to com.sun.grid.xml.qstat, but you can set the package to whatever suits you. Keep in mind, though, that you may want to generate a set of bindings for each of the three schemas, and you don’t want them to overlap.
Next, you’ll generate the binding classes. You do that by running:
% $JAXB_HOME/bin/xjc.sh -d $BIND_HOME -b $SCHEMA_HOME/qstat.xjb $SCHEMA_HOME/qstat.xsd
|
If all went well, you’ll see a list of the classes that are generated. Congratulations! You now have a qstat XML binding!
What do you do with it? Well, let me get you started. First, for convenience, let’s create an Ant build script to compile the binding and run a sample app. Create a file called $BIND_HOME/build.xml with the following contents:
<?xml version="1.0" standalone="yes"?>
<project basedir="." default="compile">
<path id="classpath">
<!--compile Java source files-->
<target name="compile" description="Compile all Java source files">
<echo message="Compiling the java source files..." />
<javac destdir="." debug="on">
<src path="." />
<classpath refid="classpath" />
</javac>
</target>
<target name="run" depends="compile" description="Run the sample app">
<echo message="Running the sample application..." />
<java classname="Main" fork="true">
<classpath refid="classpath" />
</java>
</target>
</project>
|
This build script is very primative. With any amount of Ant skills, you should be able to write something that better suits your needs. My goal here is only to cover the bare necessities.
Now, let’s create the sample app. Let’s write a simple app that lists the job number of name of all jobs currently in the system, called $BIND_HOME/Main.java. It might look something like this:
import java.util.*;
import com.sun.grid.*;
import javax.xml.bind.*;
public class Main {
public static void main (String[] args) throws Exception {
// Create a JAXB context
JAXBContext jc = JAXBContext.newInstance ("com.sun.grid.xml.qstat");
// Use the context to create an Unmarshaller
Unmarshaller u = jc.createUnmarshaller();
// Fork a qstat -xml
Process p = Runtime.getRuntime ().exec ("qstat -xml");
// Let the binding do it's magic
JobInfo ji = (JobInfo)u.unmarshal (p.getInputStream ());
List list = ((JobInfoT)ji.getJobInfo ().get (0)).getJobList ();
Iterator i = list.iterator ();
while (i.hasNext ()) {
JobListT jlt = (JobListT)i.next ();
System.out.println (jlt.getJBJobNumber () + ": " + jlt.getJBName ());
}
}
}
|
In the call to JAXBContext.netInstance(), we specify the package we used to generate the binding. What we get back from the call to Unmarshaller.unmarshal() is an object tree derived from the qstat output. We then walk the object tree until we get to the job list, and then we go through the jobs, printing the number and name for each one.
Clearly, the only way you will know what the object tree looks like is to 1) read the schema, 2) read the generated source files, or 3) generate JavaDocs from the generated source files. 3 is the best option, but 2 is what I actually did.
To build and run this sample app, do the following:
% ant -Djwsdp.home=$JWSDP_HOME build
% ant -Djwsdp.home=$JWSDP_HOME run
|
If you want to parse the output from qstat -j, you will need to repeat this process with the $SCHEMA_HOME/detailed_job_info.xsd schema. When processing this schema you will need to change the external bindings file a little. You’ll want a file called $SCHEMA_HOME/detailed_job_info.xjb that contains:
<?xml version="1.0" encoding="UTF-8"?>
<jxb:bindings version="1.0"
xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<jxb:bindings schemaLocation="detailed_job_info.xsd" node="/xs:schema">
<jxb:schemaBindings>
<jxb:package name="com.sun.grid.xml.job"/>
</jxb:schemaBindings>
</jxb:bindings>
</jxb:bindings>
|
Again, feel free to change the package. You’d use this binding the same way you use the other one. There is a third schema in the $SCHEMA_HOME directory, message.xsd, that you won’t likely need, but if you do, you can generate a binding for it just like you did for detailed_job_info.xsd
Clearly, what I have provided here is only a starting point. For more information about using JAXB, see the docs included with the JWSDP download. Particularly useful are the examples. An obvious next step would be to extend the build script to generate JavaDocs and to split the class files out of the source tree. Another good next step would be to customize the binding classes so that, for example, the status code gets returned as a list of strings instead of a binary or’ed int. If you have problems, let me know. This tutorial is still a work in progress, so feedback is welcome.
