Thursday, April 3, 2008

Who is that masked user?

Trying to get a better handle on usage of our cluster, I for the first time realised that Maui actually provides quite a nice way of displaying the efficiency of jobs. It doesn't sort them the way you'd like, but then that's what "sort" is for. Here's the "bottom 10" jobs on our system:

[root@gridgate gridmapdir]# showq -r|sort -n -k 4|sed -e 's/^[ \t]*//' -e '/^$/d'|head -n 10
JobName S Par Effic XFactor Q User Group MHost Procs Remaining StartTime
447 Jobs 447 of 683 Processors Active (65.45%)
550825_ R DEF 7.53 0.1 DE fus098 fusion wn019 1 7:45:00 Fri Mar 28 11:16:42
550438_ R DEF 9.31 0.1 DE fus098 fusion wn056 1 1:33:37 Fri Mar 28 05:05:04
550818_ R DEF 9.51 0.0 DE fus098 fusion wn072 1 5:19:15 Fri Mar 28 08:50:40
550439_ R DEF 9.65 0.1 DE fus098 fusion wn056 1 1:33:37 Fri Mar 28 05:05:04
550429_ R DEF 10.08 0.0 DE fus098 fusion wn062 1 00:39:47 Fri Mar 28 04:11:18
550437_ R DEF 10.19 0.1 DE fus098 fusion wn056 1 1:33:26 Fri Mar 28 05:05:04
550417 R DEF 10.28 0.1 DE fus098 fusion wn011 1 00:27:03 Fri Mar 28 03:58:28
550441_ R DEF 10.30 0.1 DE fus098 fusion wn056 1 1:33:40 Fri Mar 28 05:05:04

Looks like I need to find out who this fus098 guy is. Normally my method for doing this is to grep through /var/log/globus-gatekeeper.log but I finally got sick of this and wrote a little python script to translate the funny system used in /etc/grid-security/gridmapdir (documented here) and output the complete set of pool account mappings. I was going to implement all sorts of fancy options for outputting a particular user's mapping etc. but decided I could do what I needed with grep so I'll leave the fancification to someone else. The script is available here and here's some sample usage:

What are the mappings for users with "childs" in their DN?

[childss@gridgate childss]$ ./poolmapping |grep -i childs
dte053:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs:dteam
solovo003:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs:solovo
webcom050:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs:webcom
cosmo007:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs
cosmo004:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs:cosmo
gitest042:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs:gitest

What DN is mapped to dte053?

[childss@gridgate childss]$ ./poolmapping |grep -i dte053
dte053:/c=ie/o=grid-ireland/ou=cs.tcd.ie/l=ra-tcd/cn=stephen o. childs:dteam

5 comments:

Peter said...

How about sifting through /opt/edg/var/gatekeeper/grid-jobmap_20080403 for pool mapping, recorded per job.

Ewan said...

So - who is fus098 then?

Stephen Childs said...

Hmm I don't have /opt/edg/var/gatekeeper/grid-jobmap_20080403 on my CE. Is this something new in the gLite 3.1 CE (ours is still 3.0). In any case would this be any better than the contents of gridmapdir.

Stephen Childs said...

As for the identity of fus098 I'm going to try and get in touch with him privately to find out why his jobs are so inefficient before I publicise who he is!

Peter said...

Yes, grid-jobmap was new in glite3.1. It's a little better because it logs uid, DN, voms roles, lrmsID per job in a single row. Nice for parsing with shell utils. BTW, if you want historical efficiency plots (good for spotting bad VOs) then I recommend Paul's monami stuff. An example here: http://tinyurl.com/272h8r