Parameter Set

Triumvirate uses the ParameterSet class to handle parameter sets.

from triumvirate.parameters import ParameterSet

Templates

As a starting point, Triumvirate provides parameter templates which you can modify. The templates come in two formats: as a YAML file text stream or a Python dictionary.

To fetch the templates, one could use fetch_paramset_template(), with the argument 'text' for the YAML file text stream and 'dict' for the Python dictionary.

from triumvirate.parameters import fetch_paramset_template

YAML file template

Let’s first have a look at the content of the YAML file template,

parameter_template = fetch_paramset_template('text')

print(parameter_template)  # DEMO

# @file params_template.yml
# @brief Template parameter file for the Triumvirate Python package.
# @seealso https://triumvirate.readthedocs.io

---

# -- I/O -----------------------------------------------------------------

# Directories for input catalogue(s) and output measurement(s).
# The paths can be either absolute or relative to the working directory.
# If unset, the current working directory is assumed.
directories:
  catalogues:
  measurements:

# Filenames (with extension) of input catalogues.  These are relative
# to the catalogue directory.
files:
  data_catalogue:
  rand_catalogue:

# FUTURE: This parameter currently has no effect in the Python interface.
catalogue_columns: []

# FUTURE: This parameter currently has no effect in the Python interface.
catalogue_dataset:

# Tags to be appended as an input/output filename suffix.
tags:
  output:


# -- Mesh sampling -------------------------------------------------------

# Box size in each dimension (in length units).
boxsize:
  x:
  y:
  z:

# Grid cell number in each dimension.
ngrid:
  x:
  y:
  z:

# Box expansion factor: {1. (default), >1.}.
# The box is expanded by this factor in each dimension when
# box size is not set.
expand: 1.

# Nyquist cut-off in either wavenumber or separation.
# Only applicable when the grid cell number `ngrid` is unset, which is
# then determined from this and the box size.
# The measurement range is also truncated at this value if unset.
cutoff_nyq:

# Box alignment: {'centre' (default), 'pad'}.
# The catalogues are either centred or padded from the
# mininum-coordinate corner.
alignment: centre

# Padding scale: {'box' (default), 'grid'}.
# The padding scale is either the box size or the grid cell size.
# Only applicable if `alignment` is set to 'pad'.
padscale: box

# Padding factor as a multiple of the size of padding scale.
# Only applicable if `alignment` is set to 'pad'.
padfactor:

# Mesh assignment scheme: {'ngp', 'cic', 'tsc' (default), 'pcs'}.
assignment: tsc

# Interlacing switch: {true/on, false/off (default))}.
# The switch is overridden to `false` when measuring three-point statistics.
interlace: off


# -- Measurements --------------------------------------------------------

# Type of catalogue(s): {'survey', 'random', 'sim', 'none'}.
# If of type 'sim', global-plane-parallel measurements are made;
# otherwise, local-plane-parallel measurements are made.
catalogue_type:

# Type of measurement: {
#   'powspec', '2pcf', '2pcf-win',
#   'bispec', '3pcf', '3pcf-win', '3pcf-win-wa',
#   'modes', 'pairs'
# }.
statistic_type:

# Degrees of the multipoles.
degrees:
  ell1:
  ell2:
  ELL:

# Orders of wide-angle corrections.
wa_orders:
  i:
  j:

# Form of three-point statistic measurements:
# {'full', 'diag' (default), 'off-diag', 'row'}.
form: diag

# Normalisation convention: {
#   'none', 'particle' (default), 'mesh',
#   'mesh-mixed' (two-point statistics only)
# }.
norm_convention: particle

# Binning scheme: {'lin' (default), 'log', 'linpad', 'logpad', 'custom'}.
binning: lin

# Range of measurement scales.
# The binning coordinate is either wavenumbers in Fourier space,
# or separations in configuration space. [mandatory]
range: [~, ~]

# Number of measurement bins (i.e. data vector dimension).
# Must be >=2, or >=7 if padding is used in binning. [mandatory]
num_bins:

# Fixed bin index when the `form` of three-point statistics measurements
# is set to 'off-diag' or 'row'.  If 'off-diag', the bin index is the
# positive off-diagonal index; if `form` is set to 'row', the bin index
# is the row index.
idx_bin:


# -- Misc ----------------------------------------------------------------

# FFTW scheme: {'estimate', 'measure' (default), 'patient'}.
# This corresponds to the FFTW planner flags.
fftw_scheme: measure

# Use FFTW wisdom: {false (default)/off, <path-to-dir>}.
# If not `false` or non-empty, then this is the path to
# the FFTW wisdom directory; the FFTW wisdom file is either imported
# from there or exported there if the wisdom file does not yet exist;
# `fftw_scheme` must be set to 'measure' or higher (i.e. 'patient').
use_fftw_wisdom: false

# FUTURE: This parameter currently has no effect in the Python interface.
save_binned_vectors: false

# Logging verbosity level: a non-negative integer.
# Typical values are: {
#   0 (NSET, unset), 10 (DBUG, debug), 20 (STAT, status) (default),
#   30 (INFO, info), 40 (WARN, warning), 50 (ERRO, error)
# }.
verbose: 20

# Progress bar display switch: {true/on, false/off (default), <%-points>}.
# If a float is provided, the progress bar is update/displayed
# at that percentage-point interval, e.g. 10. for every 10%.
progbar: off

...

which you can save to a .yml file (named "parameter_template.yml" below),

import os.path as osp

parameter_filepath = osp.join("tmp/input", "parameter_template.yml")
with open(parameter_filepath, 'w') as parameter_file:
    parameter_file.write(parameter_template)

Python dictionary template

Alternatively, one could use the Python dictionary template:

parameter_dict = fetch_paramset_template('dict')

# DEMO: format printing of dictionaries.
from pprint import pprint  # noqa: E402
pprint(parameter_dict)

{'alignment': 'centre',
 'assignment': 'tsc',
 'binning': 'lin',
 'boxsize': {'x': None, 'y': None, 'z': None},
 'catalogue_columns': [],
 'catalogue_dataset': None,
 'catalogue_type': None,
 'cutoff_nyq': None,
 'degrees': {'ELL': None, 'ell1': None, 'ell2': None},
 'directories': {'catalogues': None, 'measurements': None},
 'expand': 1.0,
 'fftw_scheme': 'measure',
 'files': {'data_catalogue': None, 'rand_catalogue': None},
 'form': 'diag',
 'idx_bin': None,
 'interlace': False,
 'ngrid': {'x': None, 'y': None, 'z': None},
 'norm_convention': 'particle',
 'num_bins': None,
 'padfactor': None,
 'padscale': 'box',
 'progbar': False,
 'range': [None, None],
 'save_binned_vectors': False,
 'statistic_type': None,
 'tags': {'output': None},
 'use_fftw_wisdom': False,
 'verbose': 20,
 'wa_orders': {'i': None, 'j': None}}

Initialisation

From a YAML file

To initialise a parameter set from the YAML file template, let’s try:

from triumvirate.parameters import InvalidParameterError  # DEMO

try:
    paramset = ParameterSet(param_filepath=parameter_filepath)
except InvalidParameterError as exception:  # DEMO
    print(repr(exception))

InvalidParameterError('`range` parameters must be set.')

/var/folders/1_/q2lsr5v97z785g1fy4fh40sm0000gn/T/ipykernel_3394/1298854432.py:4: UserWarning: `boxsize` parameters are unset. In this case, the box size will be computed using the particle coordinate spans and the box expansion factor.
  paramset = ParameterSet(param_filepath=parameter_filepath)
/var/folders/1_/q2lsr5v97z785g1fy4fh40sm0000gn/T/ipykernel_3394/1298854432.py:4: UserWarning: `ngrid` parameters are unset. In this case, the grid cell number will be computed using the box size and the Nyquist cutoff.
  paramset = ParameterSet(param_filepath=parameter_filepath)

As one could see, ParameterSet performs validation checks on parameter values, and unset mandatory parameters result in the exception InvalidParameterError above. Some important parameters, though non-mandatory, result in a warning message being emitted if unset, and are instead inferred from the rest.

To fix the exception, (externally) edit the file "parameter_template.yml" saved above to make sure all parameters marked by [mandatory] in the file comments are set, e.g. with the following line changes:

parameter_template.yml

boxsize:
  x: 1000.
  y: 1000.
  z: 1000.
  
ngrid:
  x: 64
  y: 64
  z: 64
  
range: [0.005, 0.105]

num_bins: 10

Now let’s try to initialise the parameter set again:

parameter_set = ParameterSet(param_filepath=parameter_filepath)

From a Python dictionary

Similarly, one can initialise the parameter set from the Python dictionary template, but only after all the mandatory parameters have been set:

for ax_name in ['x', 'y', 'z']:
    parameter_dict['boxsize'][ax_name] = 1000.
    parameter_dict['ngrid'][ax_name] = 64.

parameter_dict.update({
    'range'   : [0.005, 0.105],
    'num_bins': 10,
})

parameter_set = ParameterSet(param_dict=parameter_dict)

Parameter access

One can access individual parameters from ParameterSet either as an attribute, or like a dictionary by key or with the get() method:

print("Binning scheme:", parameter_set['binning'])
print("Binning scheme:", parameter_set.get('binning'))  # equivalent
print("Binning scheme:", parameter_set.binning)         # equivalent

Binning scheme: lin
Binning scheme: lin
Binning scheme: lin

One can also extract all parameters like a dictionary using the items() method:

pprint(dict(parameter_set.items()))

{'alignment': 'centre',
 'assignment': 'tsc',
 'assignment_order': 3,
 'binning': 'lin',
 'boxsize': {'x': 1000.0, 'y': 1000.0, 'z': 1000.0},
 'catalogue_columns': [],
 'catalogue_dataset': None,
 'catalogue_type': None,
 'cutoff_nyq': None,
 'degrees': {'ELL': None, 'ell1': None, 'ell2': None},
 'directories': {'catalogues': '', 'measurements': ''},
 'expand': 1.0,
 'fftw_planner_flag': 0,
 'fftw_scheme': 'measure',
 'fftw_wisdom_file_b': '',
 'fftw_wisdom_file_f': '',
 'files': {'data_catalogue': '', 'rand_catalogue': ''},
 'form': 'diag',
 'idx_bin': None,
 'interlace': False,
 'ngrid': {'x': 64.0, 'y': 64.0, 'z': 64.0},
 'nmesh': np.float64(262144.0),
 'norm_convention': 'particle',
 'npoint': '',
 'num_bins': 10,
 'padfactor': None,
 'padscale': 'box',
 'progbar': False,
 'range': [0.005, 0.105],
 'save_binned_vectors': False,
 'space': '',
 'statistic_type': None,
 'tags': {'output': ''},
 'use_fftw_wisdom': False,
 'verbose': 20,
 'volume': np.float64(1000000000.0),
 'wa_orders': {'i': None, 'j': None}}

Passing as an argument

Now whenever a callable in Triumvirate accepts the paramset argument, you can pass parameter_set above to it. Depending on the set-up, you may need to make further modifications to your parameter_set to suit the data and/or algorithm you are using.