Persisting Pointers

This material is obsolete. We have adopted art::Ptr as the solution.

Introduction

1. places no additional burden on the physicist programmer; that is, the pointers are "just there", just like they were before the objects were written to disk.
2. is efficient, in the sense that the work required to recover the pointers is done only if and when needed.
3. is affordable and robust: we can afford to pay the software engineers to build and maintain solution; we do not have long delays when we discover that we need new features.
4. does not depend on global state; for doing things like event mixing and overlays of signal MC with data from random triggers, the concept of "the event" might be poorly defined.

For Mu2e I propose a solution in which:

• the default behaviour gives up efficiency so that, as much as possible, the pointers are "just there".
• if the physicist-programer is willing to do some extra work, then the efficiency can be recovered.

At present, the solution is not quite complete and it does require two extra steps on the part of the physicist-programmer. One extra step occurs when first creating the object; that will remain. The other extra step occurs when reading data from an event; this extra step will go away in a later version of the framework.

The Idea Behind the Proposed Solution

Consider an object representing a digitized waveform produced in MC processing; in general this will be produced from one or more StepPointMC objects. It is useful to record which StepPointMC objects contributed to each digitized waveform object. Each of the contributing StepPointMC objects can be described uniquely by specifying the data product in which it is found and the offset, or index, within that data product. So a single precursor can be described by a class like,

struct DPIndex{
    ProductID    id;
    std::size_t index;
};

     std::vector<DPIndex> precursors;

The following shows a fragment of what a digitized waveform class might look like if we did not have to worry about persisting pointers:

struct DigiWaveform{

  StrawIndex       strawIndex;                   // Straw identifier
  unsigned short   timestamp;                    // Timestamp at the start of the first bin.
  std::vector<unsigned short> adc;               // Pulseheights in 10 ns bins.
  std::vector<StepPointMC const*> stepPointMCs;  // Pointers to precursors.

  DigiWaveform( StrawIndex s, 
                unsigned short t, 
                std::vector<unsigned short> const& a,
                std::vector<StepPointMC const*> const& p):
        strawIndex(s),
        timestamp(t),
        adc(a),
        stepPointMCs(p)
 {
 }
};

struct DigiWaveform{

  // Persistent data.
  StrawIndex       strawIndex;           // Straw identifier
  unsigned short   timestamp;            // Timestamp at the start of the first bin.
  std::vector<unsigned short> adc;       // Pulseheights in 10 ns bins.
  std::vector<DPIndex> stepPointIndices; // The StepPointMC objects that contributed to this.

  // Need some accessors for the private data.

private:

  // This data is not persisted but can be recovered as needed.
  // On readback, the bool is intialized to false and the vector to empty.
  mutable bool pointersValid;
  mutable std::vector<StepPointMC const*> stepPointMCs;

};

Adding some constructors and accessors makes the class much, much busier. But the extra code is mostly boilerplate that will not change much from one persistent class to the next. A more complete fragment is below:

struct DigiWaveform{

  // Persistent data.
  StrawIndex       strawIndex;           // Straw identifier
  unsigned short   timestamp;            // Timestamp at the start of the first bin.
  std::vector<unsigned short> adc;       // Pulseheights in 10 ns bins.
  std::vector<DPIndex> stepPointIndices; // The StepPointMC objects that contributed to this.

  // One constructor.
  DigiWaveform( StrawIndex s, 
                unsigned short t, 
                std::vector<unsigned short> const& a,
                std::vector<StepPointMC const*> const& p,
                edm::Handle<StepPointMCCollection>& c):
        strawIndex(s),
        timestamp(t),
        adc(a),
        stepPointMCs(p),
        pointersValid(true),
 {
  // construct stepPointIndices from p and c.
 }

  // A second constructor.
  DigiWaveform( StrawIndex s, 
                unsigned short t, 
                std::vector<unsigned short> const& a,
                std::vector<DPIndex> const& idx,
                edm::Event const  *event = 0 ):
        strawIndex(s),
        timestamp(t),
        adc(a),
        stepPointIndices(idx),
        pointersValid(false),
        stepPointMCs()
 {
    if ( event ){
         // construct stepPointMCs from stepPointIndices and the event.
    }
 }


  // Accessor for the pointers.
  std::vector const& getStepPointMCs( edm::Event const& event) const{

     if ( pointersValid ) return stepPointMCs;

     // Fill the pointers.
     resolveDPIndices<StepPointMCCollection>( event, stepPointIndices, stepPointMCs);

     return stepPointMCs;
  }

  // A second accessor that does not need an event but might throw.
  std::vector const& getStepPointMCs() const{
     if ( pointersValid ) return stepPointMCs;
     throw ...;
  }

  // A safety checker, should be rarely needed.
  bool pointersOK() const { return pointersValid; }

private:

  // This data is not persisted but can be recovered as needed.
  // On readback, the bool is intialized to false and the vector to empty.
  mutable bool pointersValid;
  mutable std::vector<StepPointMC const*> stepPointMCs;

};

1. The templated function resolveDPIndices knows how to fill out the pointer array given the event and the vector of DPIndices. The template argument is the data type of the data product ( not of the element within the data product).
2. In general, the accessors to the pointers will return valid data or will throw.
   1. resolveDPIndices will throw if the data type of the ProductId does not match the template argument.
   2. resolveDPIndices will throw if the data type of the pointers in stepPointMCs is not the data type of an element of template argument.
   3. resolveDPIndices will throw if it cannot find the data product.
   4. resolveDPIndices will throw if the offset is larger than the size of the data product.
3. If an above threshold ADC count is pure digitizer noise, it will not point back to any StepPointMC. So this class allows stepPointIndices to be empty and a valid stepPointMCs to be empty. In your class it might be logical to decide that empty arrays to be errors and to throw when that occurs.
4. This pattern has two inconveniences:
   1. There are extra arguments in the constructor. I don't think that this is an onerous requirement and I think that the alternatives are worse. This won't change.
   2. The accessor to the pointers requires an event as an input argument. If this is deemed too complicated, there are several solutions. These solutions reduce efficiency since they needlessly fill in pointers that are not needed. However it is probably the right short term solution because it should save person-time.

An Example

• ToyDP/inc/CrudeStrawHit.hh (click on the latest revision to see the file).
• ToyDP/src/CrudeStrawHit.cc

• ToyDP/inc/DPIndex.hh
• Mu2eUtilities/inc/resolveTransients.hh
• Mu2eUtilities/resolveDPIndices.hh

• HitMakers/src/MakeCrudeStrawHit_plugin.cc
• HitMakers/src/MCSH_Test_plugin.cc
• HitMakers/test/makehits.py
• HitMakers/test/read.py

The class CrudeStrawHits contains several public data members that describe a crude hit in a straw. Since the main purpose of this class is to serve this data, they are public, not private. A crude straw hit can come from zero or more precusors. If a hit is just salt and pepper noise, it will not have any precursors. Right now we are making hits directly from StepPointMC objects and each CrudeStrawHit can come from one or more StepPointMC objects. In the future we will make CrudeStrawHit objects from unpacked digis and each CrudeStrawHit will come from exactly one unpacked digi object.

To allow a CrudeStrawHit to describe its precursors, the class contains two data members,

    enum precursor_type { undefined, unpackedDigi, stepPointMC};
    precursor_type precursorType;
    std::vector precursorIndices;

Security, Privacy, Legal