Event Store graph and injection policies

This document describes the Event Store injection policies and provides outline of existing Event Store tables. The graph snapshot of data collected by Event Store is described by a concrete example of various injection steps.

Event Store tables:

Here we represent a list of tables used by Event Store. The bold names are primary keys in those tables and italic names represents the group index (for faster lookup).

FileID

fileId

fileName

typeId

Here fileId is an unique file identifier in EventStore associated with every stored file.

KeyFile

graphId

view

run

uid

keyFileId

Here graphId is an id assigned to the graph; view represents a user skim (e.g. qcd); uid is unique id of underlying file(data), e.g. for real data it is alwasy 1, for MC it's unique id. This uid is a part of syncValue=(run#,event#,uid). keyFileId is a key file id.

RunUID

run

uid

Location

id

graphId

run

uid

locationFileId

id represents the internal unique identifier which allows store multiple entries for the same (graphId,run,uid) group index. locationFileId is a location file id.

FileType

id

type

description

id represents the internal unique identifier, type is a file type (e.g. pds, ikey, lpds, etc.)

OrphanFile

id

dateTime

user

SpecificVersion

svName

svid

svName stands for Specific Version Name (or data version name) and svid its id.

GraphPath

graphId

svid

PathDepend

parentId

childId

both parentId and childId are svid's, but show a dependencies among the graph.

Version

id

grade

timeStamp

minRunNumber

maxRunNumber

graphId

state

Available grades

Available grades in Event Store

write	read
daq-unchecked	daq
p2-unchecked	p2-postprocess
physics-unchecked	physics

Injection steps

The Greek letters will be used to describe graphId's and roman letters stand for paths within a graph. A graph represents a complete set of data visible to users from a client application. As an example we will start injection process by injecting a raw data whose parentId=0, timeStamp=0.

Injection I

The first injection attaches data to the daq graph. The injection step affects the following tables: "Version", "GraphPath", "PathDepend", "KeyFile" and "Location" and it starts by updating the "Version" table. In order to add a new graphId, the timeStamp and grade need to be unique or can be the same for a disabled state. For demonstration purposes, we add data which are represented by a path ("0" in a graph represents the parent node and "a" the child node). This path becomes the first available graph (α). At this step, new key and location files are created. As will be shown further, the keyFileId is an independent quantity. The locationFileId's would be related to the graphId.

Version 1 grade1 timeStamp1 minR maxR α active

GraphPath α a

PathDepend a NULL

KeyFile α view run uid key

Location id1 α run uid La

A concrete example for this type of graph would be: "0" represents raw data assigned to daq grade, "a" represents one pass2 dataset which was injected as "p2-unchecked" grade and "α" is the new graphId.

Injection II

Now we inject a new piece of data represented by node (b). We would like to use this data in conjunction with the original data represented by node (a). To do that we disable the original graphId=α and inject our data using the new graphId=β. After that, our data snapshot looks like

where β represents a full graph.

Version 1 grade1 timeStamp1 minR maxR α disable 2 grade1 timeStamp1 minR maxR β active

GraphPath α a β a β b

PathDepend a NULL b a

KeyFile α view run uid key β view run uid key

Location id1 α run uid La id2 β run uid La id2 β run uid Lb

Example: "0" represents data from daq grade, "a" represents pass2 data assigned to "p2-unchecked" grade, "b" represents a post-pass2 correction injected as "physics-unchecked" grade and "β" is the new graphId.

Injection III

Let's assume we made a mistake and found that data (b) need to be replaced by data (b^'). To reflect this change we choose another timeStamp.

Version 1 grade1 timeStamp1 minR maxR α disable 2 grade1 timeStamp1 minR maxR β active 3 grade1 timeStamp2 minR maxR γ active

GraphPath α a β a β b γ a γ b'

PathDepend a NULL b a b' a

KeyFile α view run uid key β view run uid key γ view run uid key

Location id1 α run uid La id2 β run uid La id2 β run uid Lb id3 γ run uid La id3 γ run uid Lb'

Example: "0" represents data from daq grade, "a" represents pass2 data assigned to "p2-unchecked" grade, "b" represents a post-pass2 correction assigned to "physics-unchecked" grade with "PP2-dataXX-version1" specificVersionName, "b^'" is post-pass2 data assigned to "physics-unchecked" grade with "PP2-dataXX-version2" specificVersionName and "γ" is the new graphId.

Injection IV

Once new data (c) are available we would like to assign them to the same timeStamp as the original processed data (a). To do that we choose another grade. The new graph (δ) represent (0,a,c) nodes.

Version 1 grade1 timeStamp1 minR maxR α disable 2 grade1 timeStamp1 minR maxR β active 3 grade1 timeStamp2 minR maxR γ disable 4 grade2 timeStamp1 minR maxR δ active

GraphPath α a β a β b γ a γ b' δ a δ c

PathDepend a NULL b a b' a c a

KeyFile α view run uid key β view run uid key γ view run uid key δ view run uid key

Location id1 α run uid La id2 β run uid La id2 β run uid Lb id3 γ run uid La id3 γ run uid Lb' id4 δ run uid La id4 δ run uid Lb

Example: "0" represents data from daq grade, "0" represents pass2 data assigned to p2-unchecked grade, "b" is post-p2 data assigned to physics-unchecked grade with "PP2-dataXX-version1" specificVersionName, "b^'" is post-p2 data assigned to physics-unchecked grade with "PP2-dataXX-version2" specificVersionName, "c" is another post-p2 data assigned to physics-unchecked grade with "PP2-dataYY-version7" specificVersionName and "δ" is some a new graphId.

Injection V

Finally we decided to combine data (0), (a), (b^') and (c) into unique graph (ε). Since data (b^') was injected using timeStamp2, we assign new graph with timeStamp2.

Version 1 grade1 timeStamp1 minR maxR α disable 2 grade1 timeStamp1 minR maxR β active 3 grade1 timeStamp2 minR maxR γ disable 4 grade2 timeStamp1 minR maxR δ active 5 grade1 timeStamp2 minR maxR ε active

GraphPath α a β a β b γ a γ b' δ a δ c ε a ε b' ε c

PathDepend a NULL b a b' a c a

KeyFile α view run uid key β view run uid key γ view run uid key δ view run uid key ε view run uid key

Location id1 α run uid La id2 β run uid La id2 β run uid Lb id3 γ run uid La id3 γ run uid Lb' id4 δ run uid La id4 δ run uid Lb id5 ε run uid La id5 ε run uid Lb' id5 ε run uid Lc

Example: "0" represents data from daq grade, "0" represents pass2 data assigned to p2-unchecked grade, "b" is post-p2 data assigned to physics-unchecked grade with "PP2-dataXX-version1" specificVersionName, "b^'" is post-p2 data assigned to physics-unchecked grade with "PP2-dataXX-version2" specificVersionName, "c" is another post-p2 data assigned to physics-unchecked grade with "PP2-dataYY-version7" specificVersionName and "ε" is a new graphId which declared to be usefull for physics analysis.

-- ValentinKuznetsov - 14 Feb 2005

• version_graph1:
  

• version_graph2:
  

• version_graph3:
  

• version_graph4:
  

• version_graph5: