Initial commit: dumping sharelatex contents

main.tex

+\documentclass{report}
+\usepackage[utf8]{inputenc}
+\usepackage{graphicx}
+
+\title{EIC Data Model}
+\author{David Blyth}
+\date{\today}
+
+\begin{document}
+
+\maketitle
+
+\chapter{Observations}
+A major goal of the Electron-Ion Collider (EIC) Software Consortium (ESC) is to maximize the interoperability of tracking software within the EIC community.  I.e., the goal is to enable any EIC detector effort to use any tracking software developed within the community.  In achieving this, duplicated software and tracking algorithm development effort can ultimately be minimized, and the quality of such software may be enhanced by combined effort.  Since a common software framework among efforts at different institutions is unlikely, the prevailing idea for how to achieve this goal is to develop well-defined geometry and data exchange formats at the point of track reconstruction.  This document addresses the data exchange part of this challenge.
+
+\begin{figure}
+    \begin{center}
+        \includegraphics{figures/ObservationsInChain.pdf}
+    \end{center}
+\caption[Observations in Reconstruction Chain Diagram]{\textbf{Observations in Reconstruction Chain Diagram} - Observation data model objects facilitate the exchange of reduced data going into tracking and other reconstruction steps.}
+\label{figChainDiag}
+\end{figure}
+
+In principle, the same goal of interoperability can be set for entire event reconstruction once detailed knowledge of detector hardware has been used to reduce raw data.  It is the purpose of the data model to be a recipe for the exchange and storage of these reduced data, specifically within a class of data model objects that will be referred to in this document as Observations (Figure~\ref{figChainDiag}).  The goal for the design of Observation objects is to present reduced but sufficient information required for event reconstruction when combined with relatively coarse information on the distribution of detector matter and the differentiation between sensitive and insensitive volumes that comes from the geometry exchange interface.  This document specifies a template for concrete Observation objects by describing required attributes which provide information primarily on the spacetime position of a given observation.
+
+We describe a template model for Observations in Section~\ref{secObservations}.  The primary focus of Section~\ref{secObservations} is to describe the mechanism by which Observation objects describe their noise (or uncertainty, but we will use the term noise in this document) distribution in spacetime.  This description goes into detail about the concrete data model objects ObservedPos and RandVar, which are objects in support of and required by all Observation objects.  Section~\ref{secExamples} presents high-level examples of creating ObservedPos objects from detector hits.  In Section~\ref{secCovMatrix} we describe the reduction of ObservedPos objects into a canonical covariance matrix.  Simplicity in the construction of a covariance matrix from ObservedPos objects while also providing the capability to describe complex spacetime noise distributions is a primary goal of the data model.
+
+\newpage
+\section{The Observation Template}
+\label{secObservations}
+In the absence of noise in the spacetime position of observations such as energy deposition, the description of such observations would be fairly trivial.  However, complexity is introduced not only by purely statistical noise, but also by the finite geometry of detector readout.  The typical model for the description of noise is to specify a mean point, a spatial covariance matrix, and possibly an independent temporal noise, where all noise is assumed to be normally-distributed.  However, this assumption is perhaps too reductive for some readout geometries, such as strip and fiber detectors, and detectors that constrain combinations of spatial and temporal components (TPCs).  This document proposes a description that is more informative than a simple covariance matrix, where non-normal distributions such as boxcar/uniform distributions can be specified, and complex convolutions of such distributions can be described without needlessly complicating the way the Observations are ultimately used in reconstruction.
+
+\begin{figure}
+    \begin{center}
+        \includegraphics{figures/Observation.pdf}
+    \end{center}
+\caption[Observation Template Diagram]{\textbf{Observation Template Diagram} - The grey box shows required attributes of concrete Observation objects, while the black boxes show attributes for concrete objects in support of Observations.  The ObservedPos and RandVar types are required for describing the positions of the Observation.  Each box contains a list of attributes defined by the model, where blue attribute types indicate a type dependence within the model.}
+\label{figObsDiag}
+\end{figure}
+
+Figure \ref{figObsDiag} shows a graphical representation of required attributes for Observation objects.  These attributes are
+\begin{itemize}
+    \item \textbf{Positions} - list of ObservedPos objects, each representing a possible discrete position for the given observation
+    
+    Observations from most detector types will have only one possible position.  However, some detector types can reconstruct to multiple possible discrete positions, e.g. when triangulating position from timing.  Other applications of specifying multiple positions are possible, and will be discussed later.
+    \item \textbf{Source Object IDs} - list of identifiers for simulated hit or particle objects
+    
+    These are unsigned 64-bit values that are used to hold unique identifiers for data model objects that were reduced into the form of an Observation.
+\end{itemize}
+
+\subsection{ObservedPos}
+\label{subSecVarPos}
+The ObservedPos data model object represents a multivariate position in spacetime.  The attributes of ObservedPos objects are
+\begin{itemize}
+    \item \textbf{Mean} - vector specifying the mean position in spacetime, with spatial components in mm and time in ns
+    \item \textbf{Noise} - list of random variable RandVars that contribute to overall noise
+    
+    These random noise contributions have zero mean, and are added together (implying that their distributions are convolved) to form a noise distribution that spans a 4-dimensional spacetime about the mean point.  Note that a minimum of four contributions is required, but more may be specified.
+    \item \textbf{Weight Modifier} - value to add to the probabilistic weight assigned to this discrete position
+    
+    The default weight is 1, and so for a single position or positions with equal probability, this is 0/unset.
+\end{itemize}
+
+The key concept underlying the above attributes is the specification of the distribution of a noisy position by a linear combination of one-dimensional random variables.  Each variable is described by a RandVar, which specifies the shape of the one-dimensional distribution as well as a vector sigma value that provides magnitude and direction for the random variation.  Each variable is assumed to be independent, implying a convolution of their distributions.  Mathematically, each distribution in the convolution is constrained in the other three dimensions with Dirac delta functions.  In reconstruction algorithms that assume normally-distributed variables, the convolution is of course also normally-distributed, and the problem simplifies down to addition of the sigma components in quadrature.
+
+This seemingly superfluous description of positional noise will be shown in Section~\ref{secCovMatrix} to be simple to reduce down to a canonical covariance matrix, while it maintains the information necessary to build more sophisticated reconstruction algorithms without bias that comes from an incorrect assumption of the noise distribution.
+
+\subsection{RandVar}
+The RandVar data model object represents a single variable with an arbitrary projection into experimental spacetime coordinates.  The attributes of RandVar objects are
+\begin{itemize}
+    \item \textbf{Distribution Type} - integer referencing the enumeration of types such as normal or uniform (boxcar)
+    \item \textbf{Sigma} - vector with a magnitude of the distribution's RMS and a direction along which the one-dimensional distribution is defined, with spatial components in mm and time in ns
+\end{itemize}
+
+The two distribution types which are likely most useful are normal and uniform (boxcar).  This type specification uses prior knowledge to succinctly imply the values of higher-order moments, rather than specifying those moments explicitly, which would be expensive in terms of IO.  As the EIC is a precision experiment, it is worthwhile to allow the inclusion of this information in our data model to potentially improve event reconstruction.
+
+\newpage
+\section{ObservedPos Examples}
+\label{secExamples}
+The following examples demonstrate the intended use of ObservedPos objects with Observations.  For each example, a detector readout scenario is described, followed by a list of ObservedPos positions that describe the scenario.  Attributes that are trivial (zero-valued) are left out.
+
+\subsection{Simple Strip Pad}
+Consider 0.1~mm-thick silicon with strip readouts in the x-y plane that are 0.1~mm wide and 5~cm long, and has a reconstructed time resolution of 1~ns.  For some energy deposit that induces charge collection in a single strip, we may create an Observation with the following position(s):
+
+\begin{samepage}
+\footnotesize
+\begin{itemize}
+    \item \texttt{position[0] = ObservedPos:}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($0.1/\sqrt{12}$, 0, 0, 0)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = (0, $50/\sqrt{12}$, 0, 0)}
+        \item \texttt{noise[2] = RandVar: type = UNIFORM, sigma = (0, 0, $0.1/\sqrt{12}$, 0)}
+        \item \texttt{noise[3] = RandVar: type = NORMAL, sigma = (0, 0, 0, 1)}
+    \end{itemize}
+\end{itemize}
+\end{samepage}
+
+\subsection{TPC}
+Consider a TPC hit with a resolution in the readout plane (x-y) of 0.1~mm, a drift velocity ($v_d$) of 0.05~mm/ns, a drift time ($t_d$) uncertainty of 5~ns, and a mean charge collection time ($t_0$) uncertainty of 1~ns.  Since $t_d$ and $t_0$ are our independent random variables, we must simply transform to experimental coordinates.  For experimental coordinates $z$ and $t$ we have
+\begin{equation}
+    z = v_dt_d,
+\end{equation}
+and
+\begin{equation}
+    t = t_0-t_d,
+\end{equation}
+so in $(z,t)$ the independent noise contributions are $(v_d*\sigma_{t_d},-\sigma_{t_d})=(0.05*5,-5)$ and $(0,\sigma_{t_0})=(0,1)$.  For the Observation we have the following position(s):
+
+\begin{samepage}
+\footnotesize
+\begin{itemize}
+    \item \texttt{position[0] = ObservedPos:}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = NORMAL, sigma = (0.1, 0, 0, 0)}
+        \item \texttt{noise[1] = RandVar: type = NORMAL, sigma = (0, 0.1, 0, 0)}
+        \item \texttt{noise[2] = RandVar: type = NORMAL, sigma = (0, 0, 0.25, -5)}
+        \item \texttt{noise[3] = RandVar: type = NORMAL, sigma = (0, 0, 0, 1)}
+    \end{itemize}
+\end{itemize}
+\end{samepage}
+
+\subsection{Hexagonal Pad with Alignment Uncertainty}
+\label{secHexPad}
+\begin{figure}
+    \begin{center}
+        \includegraphics{figures/HexPadWithPGrams.pdf}
+    \end{center}
+\caption[Hexagonal Pad From Parallelograms]{\textbf{Hexagonal Pad From Parallelograms} - This shape is used to demonstrate exact description of uniform distributions with complex boundaries within the data model.}
+\label{figHexPad}
+\end{figure}
+
+Consider a hexagonal readout pad in the x-y plane with 1~mm sides.  For this example, let's work in only two dimensions (x and y only, ignoring z and t coordinates), meaning that only two RandVar noise contributions are requried to span the space.  Suppose also that we want to exactly describe the hexagonal pad shape as uniform positional noise.  With a single ObservedPos, we can exactly describe a general parallelogram with uniform RandVars, but not a hexagon.  We may, however, construct the hexagonal shape from three discrete ObservedPos objects as illustrated in Figure \ref{figHexPad}.  For this scenario, we may create an Observation with the following position(s):
+
+\begin{samepage}
+\footnotesize
+\begin{itemize}
+    \item \texttt{position[0] = ObservedPos: mean = ($0.5*0.5$, $0.5*0.5*\sqrt{3}$)}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($1/\sqrt{12}$, 0)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $-0.5*\sqrt{3/12}$)}
+    \end{itemize}
+    \item \texttt{position[1] = ObservedPos: mean = ($0.5*0.5$, $-0.5*0.5*\sqrt{3}$)}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($1/\sqrt{12}$, 0)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $0.5*\sqrt{3/12}$)}
+    \end{itemize}
+    \item \texttt{position[2] = ObservedPos: mean = (-0.5, 0)}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $-0.5*\sqrt{3/12}$)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $0.5*\sqrt{3/12}$)}
+    \end{itemize}
+\end{itemize}
+\end{samepage}
+\newpage
+We may also want to include additional noise from alignment uncertainty.  Suppose that we assign normal noise contributions with RMS values of 0.1~mm separately in both the x and y directions.  We then have the following position(s):
+
+\begin{samepage}
+\footnotesize
+\begin{itemize}
+    \item \texttt{position[0] = ObservedPos: mean = ($0.5*0.5$, $0.5*0.5*\sqrt{3}$)}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($1/\sqrt{12}$, 0)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $-0.5*\sqrt{3/12})$}
+        \item \texttt{noise[2] = RandVar: type = NORMAL, sigma = (0.1, 0)}
+        \item \texttt{noise[3] = RandVar: type = NORMAL, sigma = (0, 0.1)}
+    \end{itemize}
+    \item \texttt{position[1] = ObservedPos: mean = ($0.5*0.5$, $-0.5*0.5*\sqrt{3}$)}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($1/\sqrt{12}$, 0)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $0.5*\sqrt{3/12}$)}
+        \item \texttt{noise[2] = RandVar: type = NORMAL, sigma = (0.1, 0)}
+        \item \texttt{noise[3] = RandVar: type = NORMAL, sigma = (0, 0.1)}
+    \end{itemize}
+    \item \texttt{position[2] = ObservedPos: mean = (-0.5, 0)}
+    \begin{itemize}
+        \item \texttt{noise[0] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $-0.5*\sqrt{3/12}$)}
+        \item \texttt{noise[1] = RandVar: type = UNIFORM, sigma = ($0.5/\sqrt{12}$, $0.5*\sqrt{3/12}$)}
+        \item \texttt{noise[2] = RandVar: type = NORMAL, sigma = (0.1, 0)}
+        \item \texttt{noise[3] = RandVar: type = NORMAL, sigma = (0, 0.1)}
+    \end{itemize}
+\end{itemize}
+\end{samepage}
+
+\begin{figure}
+    \begin{center}
+        \includegraphics{figures/hexPadMC.pdf}
+    \end{center}
+\caption[MC Sampling of Hexagonal Pad Described in the Model]{\textbf{MC Sampling of Hexagonal Pad Described in the Model} - Additional normally-distributed alignment uncertainty is seen as soft edges.}
+\label{figHexPadMC}
+\end{figure}
+
+To illustrate the resulting multivariate distribution, Figure \ref{figHexPadMC} shows the combined distribution sampled via Monte Carlo, where the hexagonal shape is correctly reconstructed, and the additional alignment uncertainty is clearly seen.
+
+\newpage
+\section{Covariance Matrix Construction}
+\label{secCovMatrix}
+Common general-purpose algorithms such as the Kalman filter are difficult to use without making the assumption of normally-distributed observation noise.  It will therefore be desirable to reduce the generalized noise information into a covariance matrix.  This will be done often, and must be an inexpensive calculation.
+
+The covariance matrix can be put in the form of
+\begin{equation}
+    \Sigma = \left<xx^T+\displaystyle\sum_i^n{s_is_i^T}\right>-\left<x\right>\left<x\right>^T,
+\end{equation}
+where $x$ represents the ObservedPos mean position vector, $s_i$ represents the RandVar sigma vectors associated with the ObservedPos, $n$ is the number of $s_i$ vectors, and the expectation operations are evaluated over the list of possible discrete Observation positions with appropriate weighting (i.e. $1+$ the weight modifier).
+
+For the case where there is only one ObservedPos, the expectation operators disappear and only the summation over the outer product of $s_i$ with itself remains.  In this case, there are up to $10*n$ (remember that $n>=4$) unique multiplication and addition operations to be performed in order to construct the symmetric covariance matrix, fewer for each $s_i$ component that tests equal to zero.  For a common scenario where $i$ ranges from 0 to 3, and one $s_i$ has only one component (time) while the rest contain only spatial components, the number of unique multiplications and additions is limited to 19.
+
+The pseudocode below describes the construction of a covariance matrix for an Observation, with a focus on generality and computational efficiency.  As can be seen, extra code is required to accommodate multiple ObservedPos objects.  However, this extra code does not translate to significantly more computations, and does not need to be executed when there is only one ObservedPos.
+\begin{samepage}
+\begin{verbatim}
+1:  define A to be a 4x4 upper-triangular zero matrix;
+2:  define w_tot to be a floating point zero value;
+3:  define x_tot to be a 4-dimensional zero vector;
+4:  loop over ObservedPos objects:
+5:      define B to be a 4x4 upper-triangular zero matrix;
+6:      loop over the list of RandVars for this ObservedPos:
+7:          add all unique non-zero products of this
+            RandVar's sigma vector components to the upper
+            part of matrix B;
+8:      if the number of ObservedPos objects == 1:
+9:          return B;
+10:     else:
+11:         define x to be this ObservedPos's mean value vector;
+12:         define w to be 1 + this ObservedPos's
+            weight modifier;
+13:         add all unique non-zero products of x's components
+            to the upper part of matrix B;
+14:         add B * w to A;
+15:         add x * w to x_tot;
+16:         add w to w_tot;
+17: define S to be A / w_tot;
+18: define x_avg to be x_tot / w_tot
+19: subtract all unique non-zero products of x_avg's components
+    from the upper part of matrix S;
+20: return S;
+\end{verbatim}
+\end{samepage}
+
+\end{document}