Group Todo List completed tasks

OSG Cluster

• correct SE info in BDII?
• site tests (sam, etc.)
• site-local-config.xml and storage.xml
• phedex installation and configuration
• correct SE entry in sitedb * LEPP cluster upgraded to OSG 1.0
• CRAB submission via condor_g scheduler working
• Upgrade to OSG 1.0
• Fix for authentication issues
• Fix for CE BDII advertisements
• Propagation of BDII ads to LCG BDII

CMSSW and FWLite Software Infrastructure

• fwlite::Event access to meta-data with lumi-block, run or file scope
• Finish FWLite packaging
• Implement go to run and event in fwlite::ChainEvent. This will require looping through the files, so it isn't a trivial forward to fwlite::Event
• fwlite::Event access to TriggerNames
• Update TFWLiteSelector for EDM 3XX file format changes
• Give access to current TFile* in fwlite::Event, ChainEvent
• Initial FWLite support for 3xx file format
• Two proof-of-concept FWLite Mac distributions
  • tarball built using DAR (old CMS packaging system)
  • tarball based on RPM scripts and dependencies
• Complete FWLite port to Mac Os X
• Provenance studies (cdj, dsr)
  • Characterized provenance file overhead as a function of event content and pruning strategy in the 21X EDM file format
  • Developed an EDAnalyzer to dump the module dependencies from the file provenance; analyzed the results from AOD and RECO data files
  • Recommended adding an option to keep only one layer of parentage, to be used skimming from custodial data (full provenance can be recovered using the "two file solution")
  • Implemented exact event-by-event provenance pruning in the 22X CMSSW stream
    • New unit tests
  • Developed design to reduce provenance overhead using per-process commonality to optimize what is stored
  • Combined the module dependencies with per-module timing data to estimate the maximum module-level parallelism possible in a multi-threaded CMSSW framework
• fwlite::Event, delay throwing exceptions for missing data until the handle is dereferenced to match framework functionality (cdj, dsr)
• Fixes for streamer format missing product ID high water mark (cdj, dsr)
• fwlite::Event new functions
  • return event number and timestamp
  • access BranchDescription by label
  • go to an event by event number
  • get the name of the branch matching a label
  • Access process names from processing history
• Update FWLite for EDM 21X file format

Fireworks

• New Fireworks release

DBS and WEBTOOLS

• (done) Add FileMover into WEBTOOLS
  • (done) proxy delegation for FileMover service
  • (done) new pick event interface for FIleMover
• (done) masthead, fix masthead / problem with SiteDB
• (done) Lookup and fix dbs admin service
• (done) New input page for data migration (part of admin, allow users to place request to migrate one dataset from one DBS instance to another local DBS instance, but not to global)
• (done) Add a switch between DD and DBS-API implementation
• Add lumi summary
• (done) Add data lookup based on runfino
• (done) site completion for CLI

20081216

• New pick event interface for FileMover which allow to extract desired event from provided dataset/run/lumi
• New 'base' package for WEBTOOLS, which includes definition of masthead, templates, security module
• work on grid certification for webtools
• improvements for DBS-QL (Query Language)
• unit tests for DBS installation
• search across multiple DBS instances
• added cross-section, configuration information DBS-QL

Previous tasks

• load-balancing of CMS web-services, including Data Discovery, Phedex, SiteDB, etc.
• CMS web-services put into maintenance mode and all tasks delegated to CERN operator
• maturity of DBS-QL language via exploring prototype on Data Discovery, I spent a lot of time here working with FNAL team to make final version of DBS-QL and underlying tools.
  • finalize QL syntax
  • CLI interface
  • integration with other data services
  • DD migration to DBS-QL
  • administrative interface for data operators
• DD is used widely in CMS (24/7), including monitoring at CERN and FNAL ROC centers for data productions and by all physicists to locate their data (commissioning, etc.).
• New FileMover data service, to locate/download your favorite LFN to local disk
• Put REST service into WMCore framework
• Modified WEBTOOLS framework to accept GRID certificate and pass it to web-services
• New masthead base service