************************************** 4/12/16
jobs submitted to all sites with 
memory=7.9G lifetime=10h disk=30GB
suceeded on the following sites, with very few slots, 
but also few disconnects.  3 days to run 1000 jobs.

count, return code, site:
   122 0   Caltech
   688 0   FNAL
     4 0   Nebraska
   141 0   Omaha
    21 0   UCSD
    24 0   Wisconsin


************************************** 1/6/16
current submission list:
Caltech,FERMIGRID,FNAL,MIT,Michigan,Nebraska,Omaha,SU-OG,Wisconsin,UCSD,NotreDame,MWT2,BNL
currently:
MIT not accepting opportunistic since August (technical issues)
MWT2 - no jobs since 12/20 - possibly due to 4 or 5h limit
FNAL (CMS) may be down
Cornell - not many slots, restarts, but eventually (days) succeeds
Clemson - not working well - still commissioning
Hyak_CE - 4h limit
FERMIGRID - doesn't work - OSG can't send jobs back to fermigrid

************************************** 10/12/15

  I submitted some test jobs to investigate the performance
at each OSG site.  The jobs had varying lengths from 1h to 15h.
Here is what I found so far.  Since this is a just a small
sample, it observes the performance on that day, in some respects
it might be different the next day.

These sites
Caltech Omaha SU-OG UCSD NotreDame FNAL(CMS)
all behavied very well.  Almost no restarts and all
long jobs finished.

MIT
  High restart rate, going up with length of job.
Only 1h jobs did not restart at all.
If the jobs is over 5h, it never completed.

Michigan
  Performed very well for jobs with less than 8h, after that
almost nothing completed.

Nebraska
  Many, many restarts and disconnects at all job lengths.
After 8h, nothing completed.

Wisconsin
   Many restarts and disconnects at all job legnths.  After 8h, the
restart probabiliy goes up noticeably.

Hyak_CE
   No jobs have started here since 9/23

MWT2, UChiago, BNL, Cornell, TTU can't be tested right now.