DAS  3.0
Das Analysis System
MetaInfo

Description

Generic meta-information for n-tuple (including speficities to Darwin).

Wrapper for TList information typically stored in TTree::GetUserInfo(). The list includes information such as:

  • a list of generic flags, e.g. the R parameter and jet algo (flags)
  • some minimal software version information (git),
  • the list of corrections applied so far (corrections),
  • the successive commands run so far (history)
  • and anything else that is possibly relevant (the format is flexible). The information stored in the list can be converted into a config file to reproduce identical results.

To initialise the meta information in a new TTree, a config with flags is required:

MetaInfo metainfo(tree, config);

To modify the meta information of an existing TTree, only the tree is necessary, but the config may be compared to the existing meta information:

MetaInfo metainfo(tree);
metainfo.Check(config);

This is typically useful for reproducibility purposes, in the case where the config was extracted from another TTree at a more advanced stage.

A typical use case is to determine generic flags for the data processing:

bool isMC = metainfo.Get<bool>("flags", "isMC");
int year = metainfo.Get<bool>("flags", "year");

The metainfo can then be used like any other UserInfo instance. Although one is entirely free to store any sort of information, it is important to keep in mind the minimal structure of the file (see above) and respect it as much as possible. For instance, to indicate an additional correction, the preferred way is the following:

metainfo.Set<string>("corrections", "nameOfMyCorrection", "value");

#include <MetaInfo.h>

+ Inheritance diagram for MetaInfo:
+ Collaboration diagram for MetaInfo:

Classes

struct  IKnowWhatIAmDoing
 

Public Member Functions

 MetaInfo (const IKnowWhatIAmDoing &, bool git_init=true)
 
 MetaInfo (TTree *t, const boost::property_tree::ptree &config, bool git_init=true)
 
 MetaInfo (TTree *t, bool git_init=true)
 
 MetaInfo (TList *l, bool git_init=true)
 
template<typename Ptr , typename ... Args>
 MetaInfo (const Ptr &t, Args... args)
 
 ~MetaInfo ()
 
void Check (const boost::property_tree::ptree &) const
 
template<unsigned h>
unsigned Seed (std::pair< unsigned short, unsigned short > slice={1, 0}) const
 
 MetaInfo (const IKnowWhatIAmDoing &, bool git_init=true)
 
 MetaInfo (TTree *t, const boost::property_tree::ptree &config, bool git_init=true)
 
 MetaInfo (TTree *t, bool git_init=true)
 
 MetaInfo (TList *l, bool git_init=true)
 
template<typename Ptr , typename ... Args>
 MetaInfo (const Ptr &t, Args... args)
 
 ~MetaInfo ()
 
void Check (const boost::property_tree::ptree &) const
 
template<unsigned h>
unsigned Seed (std::pair< unsigned short, unsigned short > slice={1, 0}) const
 
- Public Member Functions inherited from UserInfo
template<typename... Args>
TList * List (const char *key, Args... args) const
 
TList * List () const
 
template<class T , typename... Args>
Get (const char *key, Args... args) const
 
template<typename... Args>
bool Find (const char *key, Args... args) const
 
template<class T , typename... Args>
void Set (const char *key, Args... args) const
 
 UserInfo (const char *="UserInfo")
 
 UserInfo (const boost::property_tree::ptree &)
 
 UserInfo (TList *)
 
 UserInfo (TTree *)
 
 UserInfo (TTree *, const boost::property_tree::ptree &)
 
 ~UserInfo ()
 
void Write () const
 
bool Empty () const
 
boost::property_tree::ptree Write (std::ostream &, Format=INFO) const
 
boost::property_tree::ptree Write (const std::filesystem::path &) const
 
boost::property_tree::ptree MkPtree (Format format=INFO) const
 
void ls () const
 
template<typename... Args>
TList * List (const char *key, Args... args) const
 
TList * List () const
 
template<class T , typename... Args>
Get (const char *key, Args... args) const
 
template<typename... Args>
bool Find (const char *key, Args... args) const
 
template<class T , typename... Args>
void Set (const char *key, Args... args) const
 
 UserInfo (const char *="UserInfo")
 
 UserInfo (const boost::property_tree::ptree &)
 
 UserInfo (TList *)
 
 UserInfo (TTree *)
 
 UserInfo (TTree *, const boost::property_tree::ptree &)
 
 ~UserInfo ()
 
void Write () const
 
bool Empty () const
 
boost::property_tree::ptree Write (std::ostream &, Format=INFO) const
 
boost::property_tree::ptree Write (const std::filesystem::path &) const
 
boost::property_tree::ptree MkPtree (Format format=INFO) const
 
void ls () const
 

Static Public Member Functions

static std::string libgit2_version ()
 
static std::string libgit2_version ()
 

Static Public Attributes

static std::map< std::string, std::string > versions
 

Private Member Functions

void GitInit ()
 
void PrintStatus () const
 
bool AllChangesCommitted () const
 
void SetupGit (bool)
 
void SetupVersions ()
 
void UpdateList (const bool) const
 
 MetaInfo (bool, const char *)
 
 MetaInfo (TTree *, const boost::property_tree::ptree &, bool, const char *)
 
 MetaInfo (TTree *, bool, const char *)
 
 MetaInfo (TList *, bool, const char *)
 
void GitInit ()
 
void PrintStatus () const
 
bool AllChangesCommitted () const
 
void SetupGit (bool)
 
void SetupVersions ()
 
void UpdateList (const bool) const
 
 MetaInfo (bool, const char *)
 
 MetaInfo (TTree *, const boost::property_tree::ptree &, bool, const char *)
 
 MetaInfo (TTree *, bool, const char *)
 
 MetaInfo (TList *, bool, const char *)
 

Static Private Member Functions

static int get_one_status (const char *, unsigned int, void *)
 
static int get_one_status (const char *, unsigned int, void *)
 

Private Attributes

git_repository * repo
 
int git_error
 
char sha [16]
 
const std::filesystem::path origin
 

Additional Inherited Members

- Public Types inherited from UserInfo
enum  Format {
  INFO, JSON, XML, INFO,
  JSON, XML
}
 
enum  Format {
  INFO, JSON, XML, INFO,
  JSON, XML
}
 
- Protected Member Functions inherited from UserInfo
boost::property_tree::ptree MkPtree (TList *, Format=INFO) const
 
boost::property_tree::ptree MkPtree (TList *, Format=INFO) const
 

Constructor & Destructor Documentation

◆ MetaInfo() [1/18]

MetaInfo ( bool  git_init,
const char *  origin 
)
private

Used internally to set the git repo location (last parameter).

280  : UserInfo(new TList), repo(nullptr), origin(origin)
281 {
282  int preseed = time(nullptr) % numeric_limits<int>::max();
283  Set<int>("preseed", preseed);
284 
285  SetupGit(git_init);
286  SetupVersions();
287 }

◆ MetaInfo() [2/18]

MetaInfo ( TTree *  t,
const boost::property_tree::ptree &  config,
bool  git_init,
const char *  origin 
)
private

Used internally to set the git repo location (last parameter).

293  : UserInfo(t), repo(nullptr), origin(origin)
294 {
295  if (!Empty())
296  BOOST_THROW_EXCEPTION(invalid_argument("Only a new tree's metainfo may be initialised."));
297 
298  try {
299  // flags
300  const auto& flags = config.get_child("flags");
301  Set<bool>("flags", "isMC", flags.get<bool>("isMC"));
302  Set<int >("flags", "year", flags.get<int >("year"));
303  Set<int >("flags", "R" , flags.get<int >("R" ));
304  const auto& labels = flags.get_child_optional("labels");
305  if (labels) // note: here, we use `boost::optional`,
306  // not `std::optional`, hence we have to test explicitly
307  for (const auto& label: *labels) {
308  // there are subtle differences between INFO, JSON, and XML
309  // - in INFO: the label is stored in the key and the value is empty
310  // - in JSON: the label is stored in the value and the key is empty
311  // - in XML: the label is stored in the value and the key is "item"
312  string key = label.first,
313  value = label.second.get_value<string>();
314 
315  if (key.empty() || key == "item")
316  Set<string>("flags", "labels", value);
317  else if (value.empty())
318  Set<string>("flags", "labels", key);
319  else
320  BOOST_THROW_EXCEPTION(invalid_argument("Empty labels are not allowed"));
321  }
322 
323  if (config.count("history"))
324  Set<string>("history", config.get<string>("history"));
325 
326  if (config.count("preseed"))
327  Set<int>("preseed", config.get<int>("preseed"));
328  else {
329  int preseed = time(nullptr) % numeric_limits<int>::max();
330  Set<int>("preseed", preseed);
331  }
332 
333  SetupGit(git_init);
334  // Set git information coming from the config
335  if (!git_init) {
336  Set<string>("git", "commit", config.get<string>("git.commit"));
337  Set<bool>("git", "reproducible", config.get<bool>("git.reproducible"));
338  }
339 
340  SetupVersions();
341  }
342  catch (const pt::ptree_error& e) {
343  BOOST_THROW_EXCEPTION(e);
344  }
345 }

◆ MetaInfo() [3/18]

MetaInfo ( TTree *  t,
bool  git_init,
const char *  origin 
)
private

Used internally to set the git repo location (last parameter).

349  : UserInfo(t), repo(nullptr), origin(origin)
350 {
351  if (!git_init) return;
352  GitInit();
353  UpdateList(false);
354 }

◆ MetaInfo() [4/18]

MetaInfo ( TList *  l,
bool  git_init,
const char *  origin 
)
private

Used internally to set the git repo location (last parameter).

358  : UserInfo(l), repo(nullptr), origin(origin)
359 {
360  if (!git_init) return;
361  GitInit();
362  UpdateList(true);
363 }

◆ MetaInfo() [5/18]

MetaInfo ( const IKnowWhatIAmDoing ,
bool  git_init = true 
)
inline

Constsructor for first call ever. This creates a minimal MetaInfo.

This is a low-level constructor and the produced MetaInfo lacks some information:

  • The preseed is chosen randomly.
  • The caller is responsible for populating "flags".
  • No command history is recorded.
  • git.complete is set to false. Do not use this constructor.
151  : MetaInfo(git_init, DARWIN_GIT_REPO)
152  {}

◆ MetaInfo() [6/18]

MetaInfo ( TTree *  t,
const boost::property_tree::ptree &  config,
bool  git_init = true 
)
inline

Constsructor for first call ever, where the second parameter should be extracted from the config file or from the command line.

The preseed is taken from the config file or generated on the fly in a non-reproducible way. Use getMetaInfo for see how to force the seed to a value from the config file.

Parameters
t`TTree` whose `UserInfo` will be populated
configinput config (must include flags)
165  : MetaInfo(t, config, git_init, DARWIN_GIT_REPO)
166  {}

◆ MetaInfo() [7/18]

MetaInfo ( TTree *  t,
bool  git_init = true 
)
inline

Constsructor for call of existing n-tuple. By default, the git.complete flag will be reset to false. It should be set to true after the event loop in the executable.

Parameters
t`TTree` whose `UserInfo` will be modified
git_initflag to init git repo
175  : MetaInfo(t, git_init, DARWIN_GIT_REPO)
176  {}

◆ MetaInfo() [8/18]

MetaInfo ( TList *  l,
bool  git_init = true 
)
inline

Constsructor for direct call to a list (typically from a projection). By default, the git.complete will be kept as it is.

Parameters
l`TList` that will be modified
git_initflag to init git repo
184  : MetaInfo(l, git_init, DARWIN_GIT_REPO)
185  {}

◆ MetaInfo() [9/18]

MetaInfo ( const Ptr &  t,
Args...  args 
)
inline

Generic constsructor for smart pointers.

191  : MetaInfo(&*t, args...) { }

◆ ~MetaInfo() [1/2]

~MetaInfo ( )

Destructor (releases memory & shuts down Git).

366 {
367  if (repo == nullptr) return;
368  git_repository_free(repo);
369  git_libgit2_shutdown();
370 }

◆ MetaInfo() [10/18]

MetaInfo ( bool  ,
const char *   
)
private

Used internally to set the git repo location (last parameter).

◆ MetaInfo() [11/18]

MetaInfo ( TTree *  ,
const boost::property_tree::ptree &  ,
bool  ,
const char *   
)
private

Used internally to set the git repo location (last parameter).

◆ MetaInfo() [12/18]

MetaInfo ( TTree *  ,
bool  ,
const char *   
)
private

Used internally to set the git repo location (last parameter).

◆ MetaInfo() [13/18]

MetaInfo ( TList *  ,
bool  ,
const char *   
)
private

Used internally to set the git repo location (last parameter).

◆ MetaInfo() [14/18]

MetaInfo ( const IKnowWhatIAmDoing ,
bool  git_init = true 
)
inline

Constsructor for first call ever. This creates a minimal MetaInfo.

This is a low-level constructor and the produced MetaInfo lacks some information:

  • The preseed is chosen randomly.
  • The caller is responsible for populating "flags".
  • No command history is recorded.
  • git.complete is set to false. Do not use this constructor.
151  : MetaInfo(git_init, DARWIN_GIT_REPO)
152  {}

◆ MetaInfo() [15/18]

MetaInfo ( TTree *  t,
const boost::property_tree::ptree &  config,
bool  git_init = true 
)
inline

Constsructor for first call ever, where the second parameter should be extracted from the config file or from the command line.

The preseed is taken from the config file or generated on the fly in a non-reproducible way. Use getMetaInfo for see how to force the seed to a value from the config file.

Parameters
t`TTree` whose `UserInfo` will be populated
configinput config (must include flags)
165  : MetaInfo(t, config, git_init, DARWIN_GIT_REPO)
166  {}

◆ MetaInfo() [16/18]

MetaInfo ( TTree *  t,
bool  git_init = true 
)
inline

Constsructor for call of existing n-tuple. By default, the git.complete flag will be reset to false. It should be set to true after the event loop in the executable.

Parameters
t`TTree` whose `UserInfo` will be modified
git_initflag to init git repo
175  : MetaInfo(t, git_init, DARWIN_GIT_REPO)
176  {}

◆ MetaInfo() [17/18]

MetaInfo ( TList *  l,
bool  git_init = true 
)
inline

Constsructor for direct call to a list (typically from a projection). By default, the git.complete will be kept as it is.

Parameters
l`TList` that will be modified
git_initflag to init git repo
184  : MetaInfo(l, git_init, DARWIN_GIT_REPO)
185  {}

◆ MetaInfo() [18/18]

MetaInfo ( const Ptr &  t,
Args...  args 
)
inline

Generic constsructor for smart pointers.

191  : MetaInfo(&*t, args...) { }

◆ ~MetaInfo() [2/2]

~MetaInfo ( )

Destructor (releases memory & shuts down Git).

Member Function Documentation

◆ AllChangesCommitted() [1/2]

bool AllChangesCommitted ( ) const
private

Function to check possibly uncommitted changes per file. It calls internally MetaInfo::get_one_status with a foreach loop.

Returns
true if all changes have been committed
171 {
172  // https://libgit2.org/libgit2/#HEAD/group/status/git_status_foreach
173  return git_status_foreach(repo, get_one_status, repo) == 0;
174 }

◆ AllChangesCommitted() [2/2]

bool AllChangesCommitted ( ) const
private

Function to check possibly uncommitted changes per file. It calls internally MetaInfo::get_one_status with a foreach loop.

Returns
true if all changes have been committed

◆ Check() [1/2]

void Check ( const boost::property_tree::ptree &  config) const

Compare the meta information with the config given in argument. Either warnings or errors are thrown according to the severity of the inconsistencies. It also completes the records of successive commands.

373 {
374  try {
375  const auto& flags = config.get_child_optional("flags");
376  if (flags) {
377  if (Get<bool>("flags", "isMC") != flags->get<bool>("isMC") ||
378  Get<int >("flags", "year") != flags->get<int >("year") ||
379  Get<int >("flags", "R" ) != flags->get<int >("R" ))
380  BOOST_THROW_EXCEPTION(invalid_argument("Inconsistent flags between metainfo and config"));
381 
382  const auto& labels_in_config = flags->get_child_optional("labels");
383  if (labels_in_config) // TODO?
384  for (const auto& label: *labels_in_config) {
385  string key = label.first,
386  value = label.second.get_value<string>();
387 
388  if (key.empty() || key == "item") {
389  if (Find("flags", "labels", value)) continue;
390  cerr << orange << "Warning: Adding label '" + value + "' in config, which was not found in input metainfo.\n" << def;
391  Set<string>("flags", "labels", value);
392  }
393  else if (value.empty()) {
394  if (Find("flags", "labels", key)) continue;
395  cerr << orange << "Warning: Adding label '" + key + "' in config, which was not found in input metainfo.\n" << def;
396  Set<string>("flags", "labels", key);
397  }
398  }
399  if (Find("flags", "labels")) {
400  const auto& labels_in_metainfo = MkPtree(List("flags", "labels"));
401  for (const auto& label: labels_in_metainfo) {
402  string value = label.second.get_value<string>();
403  if (labels_in_config->count(value)) continue;
404  bool found = false;
405  if (labels_in_config)
406  for (const auto& label2: *labels_in_config) {
407  string value2 = label2.second.get_value<string>();
408  found = found || (value2 == value);
409  }
410  if (!found)
411  cerr << orange << "Warning: Label '" + value + "' in metainfo was not found in config.\n" << def;
412  }
413  }
414  }
415 
416  // in case the config comes from `getMetaInfo`, this checks if anything has changed meanwhile
417  const auto& git = config.get_child_optional("git");
418  if (git) {
419  auto commit = git->get<string>("commit");
420  if (origin != Get<string>("git", "repo"))
421  cerr << orange << "Warning: inconsistent repositories between metainfo and config.\n" << def;
422  else if (origin == Get<string>("git", "repo") && sha != Get<string>("git", "commit"))
423  cerr << orange << "Warning: inconsistent commits between metainfo and config.\n" << def;
424  else if (!git->get<bool>("reproducible") || !git->get<bool>("complete"))
425  cerr << orange << "Warning: the input config was generated from an irreproducible or incomplete file.\n" << def;
426  else if (AllChangesCommitted() && Get<bool>("git", "reproducible") && Get<bool>("git", "complete"))
427  cerr << green << "Expecting to reproduce the same result as the one used to generate the input config!\n" << def;
428  }
429 
430  if (config.count("history"))
431  Set<string>("history", config.get<string>("history"));
432 
433  if (config.count("preseed")) {
434  auto c = config.get<int>("preseed"),
435  s = Get<int>("preseed");
436  if (c != s)
437  cerr << orange << "The preseed in the sample (" << s << ") differs from the preseed in the config (" << c << "). The value from the config will be " << bold << "ignored" << normal << ".\n" << def;
438  }
439  }
440  catch (const pt::ptree_error& e) {
441  BOOST_THROW_EXCEPTION(e);
442  }
443 }

◆ Check() [2/2]

void Check ( const boost::property_tree::ptree &  ) const

Compare the meta information with the config given in argument. Either warnings or errors are thrown according to the severity of the inconsistencies. It also completes the records of successive commands.

◆ get_one_status() [1/2]

int get_one_status ( const char *  p,
unsigned int  ,
void *  data 
)
staticprivate

Function to check if a file is ignored.

Returns
EXIT_SUCCESS or EXIT_FAILURE
Parameters
ppath to file
datapointer to `git_repository`
37 {
38  auto repo = static_cast<git_repository *>(data);
39  int ignore = 0;
40  // https://libgit2.org/libgit2/#HEAD/group/ignore/git_ignore_path_is_ignored
41  int git_error = git_ignore_path_is_ignored(&ignore, repo, p);
42  if (git_error != 0)
43  BOOST_THROW_EXCEPTION(fs::filesystem_error("File status could not be determined", p, error_code{git_error, git_error_category}));
44  return (ignore == 1) ? EXIT_SUCCESS : EXIT_FAILURE;
45 }

◆ get_one_status() [2/2]

static int get_one_status ( const char *  ,
unsigned int  ,
void *   
)
staticprivate

Function to check if a file is ignored.

Returns
EXIT_SUCCESS or EXIT_FAILURE

◆ GitInit() [1/2]

void GitInit ( )
private

Inits and opens git repo, sets sha to the value at running time.

48 {
49  // https://libgit2.org/libgit2/#HEAD/group/libgit2/git_libgit2_init
50  git_libgit2_init();
51 
52  if (origin.empty())
53  BOOST_THROW_EXCEPTION(fs::filesystem_error("Empty path to Git repository", origin, make_error_code(errc::invalid_argument)));
54 
55  // open repo
56  if (!(fs::exists(origin) && fs::is_directory(origin)))
57  BOOST_THROW_EXCEPTION(fs::filesystem_error("Git repository cannot be found", origin, make_error_code(errc::not_a_directory)));
58 
59  // https://libgit2.org/libgit2/#HEAD/group/repository/git_repository_open
60  git_error = git_repository_open(&repo, origin.c_str());
61  if (git_error != 0)
62  BOOST_THROW_EXCEPTION(fs::filesystem_error("Invalid git repository", origin, error_code{git_error, git_error_category}));
63 
64  // get HEAD
65  git_object * head_commit = nullptr;
66  // https://libgit2.org/libgit2/#HEAD/group/revparse/git_revparse_single
67  git_error = git_revparse_single(&head_commit, repo, "HEAD^{commit}");
68  if (git_error != 0)
69  BOOST_THROW_EXCEPTION(system_error(git_error, git_error_category, "Head commit could not be determined "
70  "(check if $DARWIN_GIT_REPO has been defined and points to the root of the git repository)"));
71  auto commit = reinterpret_cast<git_commit *>(head_commit);
72 
73  // get SHA-1
74  const git_oid * id = git_commit_id(commit);
75  // https://libgit2.org/libgit2/#HEAD/group/oid/git_oid_tostr
76  git_oid_tostr(sha, 16, id);
77 
78  // display a few properties
79  // https://libgit2.org/libgit2/#HEAD/group/commit/git_commit_author
80  const git_signature * author = git_commit_author(commit);
81  // https://libgit2.org/libgit2/#HEAD/group/commit/git_commit_message
82  cerr << underline << "latest commit" << def << ": " << git_commit_message(commit)
83  << " by " << author->name << " (" << sha << ")\n";
84 
85  // https://libgit2.org/libgit2/#HEAD/group/commit/git_commit_free
86  git_commit_free(commit);
87 }

◆ GitInit() [2/2]

void GitInit ( )
private

Inits and opens git repo, sets sha to the value at running time.

◆ libgit2_version() [1/2]

std::string libgit2_version ( )
static
Returns
string with libgit2 version.
261 {
262  int major, minor, rev;
263  // https://libgit2.org/libgit2/#HEAD/group/libgit2/git_libgit2_version
264  int git_error = git_libgit2_version(&major, &minor, &rev);
265  if (git_error != 0)
266  BOOST_THROW_EXCEPTION(runtime_error("Unable to get libgit2 version"));
267  return Form("%d.%d.%d", major, minor, rev);
268 }

◆ libgit2_version() [2/2]

static std::string libgit2_version ( )
static
Returns
string with libgit2 version.

◆ PrintStatus() [1/2]

void PrintStatus ( ) const
private

Mimicks git status.

Adapted from this example. See also the documentation.

90 {
91  git_status_list * status = nullptr;
92  // https://libgit2.org/libgit2/#HEAD/group/status/git_status_list_new
93  git_error = git_status_list_new(&status, repo, nullptr);
94  if (git_error != 0)
95  BOOST_THROW_EXCEPTION(system_error(git_error, git_error_category, "File status could not be determined"));
96  // https://libgit2.org/libgit2/#HEAD/group/status/git_status_list_entrycount
97  const size_t maxi = git_status_list_entrycount(status);
98 
99  for (size_t i = 0; i < maxi; ++i) {
100  const git_status_entry * s = git_status_byindex(status, i);
101 
102  if (s->status == GIT_STATUS_CURRENT) continue;
103  if (s->status & GIT_STATUS_IGNORED) continue;
104 
105  char istatus = ' ', // index status
106  wstatus = ' '; // worktree status
107  const char * extra = "", // for submodule info
108  * a = nullptr,
109  * b = nullptr,
110  * c = nullptr;
111 
112  if (s->status & GIT_STATUS_INDEX_NEW ) istatus = 'A';
113  else if (s->status & GIT_STATUS_INDEX_MODIFIED ) istatus = 'M';
114  else if (s->status & GIT_STATUS_INDEX_DELETED ) istatus = 'D';
115  else if (s->status & GIT_STATUS_INDEX_RENAMED ) istatus = 'R';
116  else if (s->status & GIT_STATUS_INDEX_TYPECHANGE) istatus = 'T';
117 
118  if (s->status & GIT_STATUS_WT_NEW) {
119  if (istatus == ' ') { istatus = '?'; }
120  wstatus = '?';
121  }
122  else if (s->status & GIT_STATUS_WT_MODIFIED ) wstatus = 'M';
123  else if (s->status & GIT_STATUS_WT_DELETED ) wstatus = 'D';
124  else if (s->status & GIT_STATUS_WT_RENAMED ) wstatus = 'R';
125  else if (s->status & GIT_STATUS_WT_TYPECHANGE) wstatus = 'T';
126  if (istatus == '?' && wstatus == '?') continue; // TODO?
127 
128  // A commit in a tree is how submodules are stored, so let's go take a look at its status.
129  if (s->index_to_workdir &&
130  s->index_to_workdir->new_file.mode == GIT_FILEMODE_COMMIT) {
131  unsigned int smstatus = 0;
132 
133  // https://libgit2.org/libgit2/#HEAD/group/submodule/git_submodule_status
134  if (!git_submodule_status(&smstatus, repo, s->index_to_workdir->new_file.path,
135  GIT_SUBMODULE_IGNORE_UNSPECIFIED)) {
136  if (smstatus & GIT_SUBMODULE_STATUS_WD_MODIFIED ) extra = " (new commits)";
137  else if (smstatus & GIT_SUBMODULE_STATUS_WD_INDEX_MODIFIED) extra = " (modified content)";
138  else if (smstatus & GIT_SUBMODULE_STATUS_WD_WD_MODIFIED ) extra = " (modified content)";
139  else if (smstatus & GIT_SUBMODULE_STATUS_WD_UNTRACKED ) extra = " (untracked content)";
140  }
141  }
142 
143  // Now that we have all the information, format the output.
144  if (s->head_to_index) {
145  a = s->head_to_index->old_file.path;
146  b = s->head_to_index->new_file.path;
147  }
148  if (s->index_to_workdir) {
149  if (!a) a = s->index_to_workdir->old_file.path;
150  if (!b) b = s->index_to_workdir->old_file.path;
151  c = s->index_to_workdir->new_file.path;
152  }
153 
154  cerr << istatus << wstatus << ' ' << a;
155  if (istatus == 'R') cerr << ' ' << b;
156  if (wstatus == 'R') cerr << ' ' << c;
157  cerr << extra << '\n';
158  }
159 
160  for (size_t i = 0; i < maxi; ++i) {
161  // https://libgit2.org/libgit2/#HEAD/group/status/git_status_byindex
162  const git_status_entry * s = git_status_byindex(status, i);
163  if (s->status != GIT_STATUS_WT_NEW) continue;
164  cerr << "?? " << s->index_to_workdir->old_file.path << '\n';
165  }
166  // https://libgit2.org/libgit2/#HEAD/group/status/git_status_list_free
167  git_status_list_free(status);
168 }

◆ PrintStatus() [2/2]

void PrintStatus ( ) const
private

Mimicks git status.

Adapted from this example. See also the documentation.

◆ Seed() [1/2]

unsigned Seed ( std::pair< unsigned short, unsigned short >  slice = {1,0}) const
inline

Generate the seed in a reproducible way:

\[ s = h + (k+1) \times b \]

where

  • $h$ is hard-coded in the program and should be different for every call of the functor,
  • $k$ corresponds to the (incremented) slice index,
  • and $b$ should be different for every sample (a.k.a. preseed).
Parameters
slicevariable part
220  {1,0}
221  ) const
222  {
223  unsigned b = Get<int>("preseed");
224  unsigned k = slice.second;
225  return h + (k+1) * b;
226  }

◆ Seed() [2/2]

unsigned Seed ( std::pair< unsigned short, unsigned short >  slice = {1,0}) const
inline

Generate the seed in a reproducible way:

\[ s = h + (k+1) \times b \]

where

  • $h$ is hard-coded in the program and should be different for every call of the functor,
  • $k$ corresponds to the (incremented) slice index,
  • and $b$ should be different for every sample (a.k.a. preseed).
Parameters
slicevariable part
220  {1,0}
221  ) const
222  {
223  unsigned b = Get<int>("preseed");
224  unsigned k = slice.second;
225  return h + (k+1) * b;
226  }

◆ SetupGit() [1/2]

void SetupGit ( bool  )
private

Fills in git information in the MetaInfo.

◆ SetupGit() [2/2]

void SetupGit ( bool  git_init)
private

Fills in git information in the MetaInfo.

177 {
178  bool all_changes_committed = false;
179  if (git_init) {
180  GitInit();
181  all_changes_committed = AllChangesCommitted();
182  if (!all_changes_committed) {
183  cerr << orange << "Warning: uncommited changes.\n";
184  PrintStatus();
185  cerr << def;
186  }
187  // NOTE Only set if git_init is true
188  Set<string>("git", "commit", string(sha));
189  }
190  Set<fs::path>("git", "repo" , origin );
191  Set<bool> ("git", "reproducible", all_changes_committed);
192  Set<bool> ("git", "complete" , false );
193 }

◆ SetupVersions() [1/2]

void SetupVersions ( )
private

Fills in software version information in the MetaInfo.

◆ SetupVersions() [2/2]

void SetupVersions ( )
private

Fills in software version information in the MetaInfo.

196 {
197  for (auto const& version: versions) {
198  string const& soft = version.first,
199  vers = version.second;
200  Set<string>("software", soft.c_str(), vers);
201  }
202 }

◆ UpdateList() [1/2]

void UpdateList ( const bool  ) const
private

Update list with present git information. The flags and git blocks are expected to be found, otherwise it crashes smoothly. Then it updates the Git information.

◆ UpdateList() [2/2]

void UpdateList ( const bool  assumeComplete) const
private

Update list with present git information. The flags and git blocks are expected to be found, otherwise it crashes smoothly. Then it updates the Git information.

Parameters
assumeCompleteshould assume completeness (typically `false` for event loops)
205 {
206  if (Empty())
207  BOOST_THROW_EXCEPTION(invalid_argument("No metainfo in input to update."));
208 
209  if (!Find("flags"))
210  BOOST_THROW_EXCEPTION(invalid_argument("Missing flags in input."));
211  if (!Find("git"))
212  BOOST_THROW_EXCEPTION(invalid_argument("Missing git information in input."));
213 
214  /****** git business ******/
215 
216  if (origin != Get<fs::path>("git", "repo")) {
217  cerr << orange << "Warning: inconsistent repositories between input and output.\n" << def;
218  Set<fs::path>("git", "repo", origin);
219  }
220  else if (sha != Get<string>("git", "commit")) {
221  cerr << orange << "Warning: inconsistent commits between input and output.\n" << def;
222  Set<string>("git", "commit", string(sha));
223  }
224 
225  bool all_changes_committed = AllChangesCommitted();
226  if (!all_changes_committed) {
227  cerr << orange << "Warning: uncommited changes.\n";
228  PrintStatus();
229  cerr << def;
230  }
231  auto reproducible = Get<bool>("git", "reproducible");
232  if (!reproducible)
233  cerr << orange << "Warning: input tree not reproducible.\n" << def;
234  Set<bool>("git", "reproducible", all_changes_committed && reproducible);
235 
236  auto complete = Get<bool>("git", "complete");
237  if (!complete)
238  cerr << orange << "Warning: input tree not complete.\n" << def;
239  Set<bool>("git", "complete", assumeComplete && complete);
240 
241  /****** software business ******/
242 
243  for (auto const& version: versions) {
244  string const& soft = version.first,
245  vers1 = version.second;
246  if (!Find("software", soft.c_str())) continue;
247  auto vers2 = Get<string>("software", soft.c_str());
248 
249  if (vers1 == vers2) continue;
250  cerr << orange << "Warning: inconsistent " << soft << " versions ("
251  << vers1 << " vs " << vers2 << ")\n" << def;
252  }
253 
254  /****** seeding business ******/
255 
256  if (!Find("preseed"))
257  BOOST_THROW_EXCEPTION(invalid_argument("No preseed could be found."));
258 }

Member Data Documentation

◆ git_error

int git_error
mutableprivate

C-style error code from Git operations (see source code)

◆ origin

const std::filesystem::path origin
private

path to local repo determined at compilation time

◆ repo

git_repository * repo
private

pointer to local Git repo

◆ sha

char sha
private

commit SHA at running time

◆ versions

static std::map< std::string, std::string > versions
static
Initial value:
= {
{"Darwin" , DARWIN_VERSION},
{"gpp" , Form("%d.%d.%d", __GNUC__, __GNUC_MINOR__, __GNUC_PATCHLEVEL__)},
{"Cpp" , Form("C++%ld", __cplusplus/100-2000)},
{"ROOT" , gROOT->GetVersion()},
{"Boost" , BOOST_LIB_VERSION},
{"libgit2", libgit2_version()}
}

software version


The documentation for this class was generated from the following files:
Ntupliser_cfg.cerr
cerr
Definition: Ntupliser_cfg.py:93
Darwin::Tools::UserInfo::List
TList * List() const
Access directly to main list.
Definition: UserInfo.h:201
Darwin::Tools::UserInfo::l
TList * l
main list (typically from TTree::GetUserInfo())
Definition: UserInfo.h:64
Darwin::Tools::MetaInfo::SetupVersions
void SetupVersions()
Fills in software version information in the MetaInfo.
Definition: MetaInfo.cc:195
Ntupliser_cfg.labels
dictionary labels
Definition: Ntupliser_cfg.py:31
Darwin::Tools::MetaInfo::git_error
int git_error
Definition: MetaInfo.h:68
Step::def
static const char * def
Definition: Step.h:36
Ntupliser_cfg.args
args
Definition: Ntupliser_cfg.py:11
Darwin::Exceptions::git_error_category
git_error_category
Error category for libgit2.
underline
static const char * underline
Definition: colours.h:10
Ntupliser_cfg.year
int year
Definition: Ntupliser_cfg.py:63
Ntupliser_cfg.p
p
Definition: Ntupliser_cfg.py:358
jercExample.key
string key
Definition: jercExample.py:109
Darwin::Tools::MetaInfo::GitInit
void GitInit()
Inits and opens git repo, sets sha to the value at running time.
Definition: MetaInfo.cc:47
btvExample_106X.data
data
Definition: btvExample_106X.py:10
DARWIN_GIT_REPO
#define DARWIN_GIT_REPO
Definition: main.cc:21
Darwin::Tools::MetaInfo::AllChangesCommitted
bool AllChangesCommitted() const
Definition: MetaInfo.cc:170
orange
static const char * orange
Definition: colours.h:6
Darwin::Tools::UserInfo::Find
bool Find(TList *mother, const char *key) const
Recursive finder.
Definition: UserInfo.h:107
DYToLL_M-50_13TeV_pythia8_cff_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO.version
version
Definition: DYToLL_M-50_13TeV_pythia8_cff_GEN_SIM_RECOBEFMIX_DIGI_L1_DIGI2RAW_L1Reco_RECO.py:49
normal
static const char * normal
Definition: colours.h:8
Darwin::Tools::MetaInfo::PrintStatus
void PrintStatus() const
Definition: MetaInfo.cc:89
Step::green
static const char * green
Definition: Step.h:33
DARWIN_VERSION
#define DARWIN_VERSION
Definition: version.h:1
Darwin::Tools::UserInfo::MkPtree
boost::property_tree::ptree MkPtree(TList *, Format=INFO) const
Definition: UserInfo.cc:52
Darwin::Tools::MetaInfo::MetaInfo
MetaInfo(bool, const char *)
Used internally to set the git repo location (last parameter).
Definition: MetaInfo.cc:279
Darwin::Tools::MetaInfo::repo
git_repository * repo
pointer to local Git repo
Definition: MetaInfo.h:67
Ntupliser_cfg.isMC
string isMC
Definition: Ntupliser_cfg.py:59
Darwin::Tools::MetaInfo::UpdateList
void UpdateList(const bool) const
Definition: MetaInfo.cc:204
Darwin::Tools::MetaInfo::origin
const std::filesystem::path origin
path to local repo determined at compilation time
Definition: MetaInfo.h:71
Darwin::Tools::UserInfo::Empty
bool Empty() const
Check if list is empty.
Definition: UserInfo.cc:156
Darwin::Tools::MetaInfo::get_one_status
static int get_one_status(const char *, unsigned int, void *)
Definition: MetaInfo.cc:36
Darwin::Tools::MetaInfo::sha
char sha[16]
commit SHA at running time
Definition: MetaInfo.h:70
Darwin::Tools::MetaInfo::SetupGit
void SetupGit(bool)
Fills in git information in the MetaInfo.
Definition: MetaInfo.cc:176
Darwin::Tools::MetaInfo::libgit2_version
static std::string libgit2_version()
Definition: MetaInfo.cc:260
Darwin::Tools::MetaInfo::versions
static std::map< std::string, std::string > versions
software version
Definition: MetaInfo.h:207
Darwin::Tools::UserInfo::UserInfo
UserInfo(const char *="UserInfo")
Constructor (starting from scratch)
Definition: UserInfo.cc:161
Ntupliser_cfg.flags
dictionary flags
Definition: Ntupliser_cfg.py:27
Darwin::Tools::config
@ config
activate -c option to provide config file
Definition: Options.h:25
Step::bold
static const char * bold
Definition: Step.h:35