Quick Guide#
If you have not yet installed Triumvirate, please follow the instructions in ‘Installation’.
Using the Python package#
Preamble#
Load needed classes and functions from the relevant package modules:
from triumvirate.catalogue import ParticleCatalogue
from triumvirate.logger import setup_logger
from triumvirate.parameters import ParameterSet
from triumvirate.threept import compute_bispec_in_gpp_box
Optional logger#
A logger is useful for program tracking, and can be set up easily and passed
to callables that accept a logger argument:
# Set up the customised Triumvirate program logger.
trv_logger = setup_logger()
See also
Customised Logger for more details.
Parameter configuration#
The simplest way to initialise an instance of the parameter set class
ParameterSet is by loading an external
.yml file. A template parameter file
"params_template.yml" can be
downloaded, but the relevant parameters need to be changed accordingly
for your use.
See also
Parameter Configuration and Parameter Set for more details.
Assuming the parameter file is in the current working directory, read the parameter set from file:
parameter_set = ParameterSet("params_template.yml", logger=trv_logger)
[2023-10-03 11:58:41 (+00:00:00) INFO] Validating parameters... (entering C++)
[2023-10-03 11:58:41 (+00:00:00) INFO] ... validated parameters. (exited C++)
[2023-10-03 11:58:41 (+00:00:00) STAT C++] Parameters validated.
The parameters used here are (possibly printed in a different order):
parameter_set.print()
{'alignment': 'centre',
'assignment': 'cic',
'binning': 'lin',
'boxsize': {'x': 1000.0, 'y': 1000.0, 'z': 1000.0},
'catalogue_columns': [],
'catalogue_type': 'survey',
'degrees': {'ELL': 0, 'ell1': 0, 'ell2': 0},
'directories': {'catalogues': '', 'measurements': ''},
'files': {'data_catalogue': '', 'rand_catalogue': ''},
'form': 'diag',
'idx_bin': None,
'interlace': False,
'ngrid': {'x': 64, 'y': 64, 'z': 64},
'norm_convention': 'particle',
'num_bins': 10,
'padfactor': None,
'padscale': 'box',
'range': [0.005, 0.105],
'save_binned_vectors': False,
'statistic_type': 'bispec',
'tags': {'output': ''},
'verbose': 20,
'wa_orders': {'i': None, 'j': None},
'assignment_order': 2,
'npoint': '3pt',
'space': 'fourier'}
Catalogue data#
We will measure clustering statistics from a single simulation-like catalogue
file "mock_catalogue_sim.dat", which contains three data columns 'x',
'y' and 'z' for the Cartesian coordinates of particles.
Read the catalogue data from file:
# DEMO: warning messages are captured and reprinted here for demonstration.
import warnings # DEMO
with warnings.catch_warnings(record=True) as demo_warnings:
catalogue = ParticleCatalogue.read_from_file(
"mock_catalogue_sim.dat",
names=['x', 'y', 'z'],
logger=trv_logger
)
for demo_warning in demo_warnings:
print(demo_warning.message)
[2023-10-03 11:58:42 (+00:00:00) INFO] Original extents of particle coordinates: {'x': (-499.998, 499.998), 'y': (-500.000, 499.998), 'z': (-499.999, 500.000)} (ParticleCatalogue(source=extfile:mock_catalogue_sim.dat)).
[2023-10-03 11:58:42 (+00:00:00) INFO] Catalogue loaded: ntotal = 499214, wtotal = 499214.000, wstotal = 499214.000 (ParticleCatalogue(source=extfile:mock_catalogue_sim.dat)).
Catalogue 'nz' field is not provided and thus set to zero, which may raise errors in some computations.
Catalogue 'ws' field is not provided, so is set to unity.
Catalogue 'wc' field is not provided, so is set to unity.
Hint
As seen in the warning messages, catalogues should usually contain more data columns than just the Cartesian coordinates. For a cuboid-box catalogue of unweighted particles like the above, the Cartesian coordinates are the only necessary input data.
See also
Particle Catalogue for more details.
Clustering measurements#
As an example, we measure the diagonal global plane-parallel bispectrum
monopole \((\ell_1, \ell_2, L) = (0, 0, 0)\) (set in the parameter set above)
from the simulation-like cuboid-box catalogue loaded above, and save the
result as a .txt file to the default output file "bk000_diag.txt".
Call the function:
results = compute_bispec_in_gpp_box(
catalogue, paramset=parameter_set,
save='.txt', logger=trv_logger
)
[2023-10-03 11:58:42 (+00:00:00) INFO] Validating parameters... (entering C++)
[2023-10-03 11:58:42 (+00:00:00) INFO] ... validated parameters. (exited C++)
[2023-10-03 11:58:42 (+00:00:00) STAT C++] Parameters validated.
[2023-10-03 11:58:42 (+00:00:00) INFO] Parameter set have been initialised.
[2023-10-03 11:58:42 (+00:00:00) INFO] Binning has been initialised.
[2023-10-03 11:58:42 (+00:00:00) INFO] Offset extents of particle coordinates: {'x': (0.002, 999.998), 'y': (0.001, 999.999), 'z': (0.000, 1000.000)} (ParticleCatalogue(source=extfile:mock_catalogue_sim.dat)).
[2023-10-03 11:58:42 (+00:00:00) INFO] Catalogue box has been periodised.
[2023-10-03 11:58:42 (+00:00:00) INFO] Inserted missing 'nz' field based on particle count and box size.
[2023-10-03 11:58:42 (+00:00:00) INFO] Preparing catalogue for clustering algorithm... (entering C++)
[2023-10-03 11:58:42 (+00:00:00) INFO] ... prepared catalogue for clustering algorithm. (exited C++)
[2023-10-03 11:58:42 (+00:00:00) INFO C++] Catalogue loaded: ntotal = 499214, wtotal = 499214.000, wstotal = 499214.000 (source=extdata).
[2023-10-03 11:58:42 (+00:00:00) INFO C++] Extents of particle coordinates: {'x': (0.002, 999.998 | 999.996), 'y': (0.001, 999.999 | 999.998), 'z': (0.000, 1000.000 | 999.999)} (source=extdata).
[2023-10-03 11:58:42 (+00:00:00) INFO] Normalisation factors: 8.037847e+00 (particle; used), 2.208591e+00 (mesh), 0.000000e+00 (mesh-mixed; n/a).
[2023-10-03 11:58:42 (+00:00:00) INFO] Measuring bispectrum from a simulation-box catalogue in the global plane-parallel approximation... (entering C++)
[2023-10-03 11:58:42 (+00:00:00) INFO] ... measured bispectrum from a simulation-box catalogue in the global plane-parallel approximation. (exited C++)
[2023-10-03 11:58:42 (+00:00:00) STAT C++] Computing bispectrum from a periodic-box simulation-type catalogue in the global plane-parallel approximation...
[2023-10-03 11:58:42 (+00:00:00) STAT C++] Bispectrum term at orders (m1, m2, M) = (0, 0, 0) computed.
[2023-10-03 11:58:42 (+00:00:00) STAT C++] ... computed bispectrum from a periodic-box simulation-type catalogue in the global plane-parallel approximation.
[2023-10-03 11:58:42 (+00:00:00) INFO] Measurements saved to bk000_diag.txt.
The returned results is a dictionary containing the coordinate bin centres
and effective/average coordinate in each bin, the number of contributing
modes/pairs per bin and (the real and imaginary parts of) the raw measurements
including shot noise and the shot noise component:
# DEMO
from pprint import pprint
pprint(results)
{'bk_raw': array([-9.76634349e+08+7.92439422e-07j, 5.62308421e+09+4.87302019e-08j,
8.69371048e+08+2.05250138e-08j, 1.28499830e+09-1.49280707e-08j,
6.68698399e+08+2.50419193e-09j, 6.05966770e+08+1.17318674e-08j,
3.85704714e+08+6.54509200e-09j, 2.88772402e+08+1.79973553e-10j,
2.32705659e+08+1.49205243e-09j, 2.00352034e+08+4.20782831e-09j]),
'bk_shot': array([1.89812860e+08-1.62112435e-09j, 2.06798899e+08+6.04756658e-10j,
1.50707291e+08+3.88425279e-10j, 1.24950413e+08-9.99012797e-11j,
1.06980043e+08+3.73844445e-10j, 9.44867847e+07-2.83867665e-10j,
8.04644910e+07+3.54079996e-11j, 6.84708346e+07-1.77212147e-10j,
5.94749097e+07-1.80318917e-10j, 5.12239797e+07+1.03430924e-10j]),
'k1_bin': array([0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1 ]),
'k1_eff': array([0.01149964, 0.02047114, 0.03052422, 0.04062536, 0.05033379,
0.06040971, 0.07034279, 0.08025683, 0.09014322, 0.10008191]),
'k2_bin': array([0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1 ]),
'k2_eff': array([0.01149964, 0.02047114, 0.03052422, 0.04062536, 0.05033379,
0.06040971, 0.07034279, 0.08025683, 0.09014322, 0.10008191]),
'nmodes_1': array([ 56, 194, 488, 812, 1250, 1896, 2426, 3272, 4016, 5138]),
'nmodes_2': array([ 56, 194, 488, 812, 1250, 1896, 2426, 3272, 4016, 5138])}
The saved file contains a header with summary information about the parameters used and the input data as well as intermediary quantities:
# DEMO
with open("bk000_diag.txt", 'r') as results_file:
print(results_file.read())
# Catalogue source: extfile:mock_catalogue_sim.dat
# Catalogue size: ntotal = 499214, wtotal = 499214.000, wstotal = 499214.000
# Catalogue particle extents: ([0.002, 999.998], [0.001, 999.999], [0.000, 1000.000])
# Box size: [1000.000, 1000.000, 1000.000]
# Box alignment: centre
# Mesh number: [64, 64, 64]
# Mesh assignment and interlacing: cic, False
# Normalisation factor: 8.037846928e+00 (particle)
# Normalisation factor alternatives: 8.037846928e+00 (particle), 2.208590772e+00 (mesh), 0.000000000e+00 (mesh-mixed; n/a)
# [0] k1_cen, [1] k1_eff, [2] nmodes_1, [3] k2_cen, [4] k2_eff, [5] nmodes_2, [6] Re{bk000_raw}, [7] Im{bk000_raw}, [8] Re{bk000_shot}, [9] Im{bk000_shot}
1.000000000e-02 1.149964290e-02 56 1.000000000e-02 1.149964290e-02 56 -9.766343488e+08 7.924394224e-07 1.898128595e+08 -1.621124349e-09
2.000000000e-02 2.047114034e-02 194 2.000000000e-02 2.047114034e-02 194 5.623084213e+09 4.873020188e-08 2.067988992e+08 6.047566581e-10
3.000000000e-02 3.052421724e-02 488 3.000000000e-02 3.052421724e-02 488 8.693710478e+08 2.052501378e-08 1.507072914e+08 3.884252789e-10
4.000000000e-02 4.062536317e-02 812 4.000000000e-02 4.062536317e-02 812 1.284998304e+09 -1.492807069e-08 1.249504126e+08 -9.990127965e-11
5.000000000e-02 5.033378732e-02 1250 5.000000000e-02 5.033378732e-02 1250 6.686983986e+08 2.504191929e-09 1.069800433e+08 3.738444455e-10
6.000000000e-02 6.040971138e-02 1896 6.000000000e-02 6.040971138e-02 1896 6.059667699e+08 1.173186739e-08 9.448678469e+07 -2.838676648e-10
7.000000000e-02 7.034278912e-02 2426 7.000000000e-02 7.034278912e-02 2426 3.857047142e+08 6.545091995e-09 8.046449097e+07 3.540799964e-11
8.000000000e-02 8.025683499e-02 3272 8.000000000e-02 8.025683499e-02 3272 2.887724022e+08 1.799735532e-10 6.847083460e+07 -1.772121469e-10
9.000000000e-02 9.014322302e-02 4016 9.000000000e-02 9.014322302e-02 4016 2.327056590e+08 1.492052427e-09 5.947490971e+07 -1.803189165e-10
1.000000000e-01 1.000819060e-01 5138 1.000000000e-01 1.000819060e-01 5138 2.003520342e+08 4.207828312e-09 5.122397971e+07 1.034309241e-10
See also
Clustering Measurements for more details.
Running the C++ program#
For most users, the C++ code serves as the backend for the Python package
and the only public routine is the compiled binary executable program
triumvirate which should be run with an external parameter file.
Parameter configuration#
A template parameter file "params_template.ini"
can be downloaded, but the relevant parameters need to be changed accordingly
for your use.
See also
Parameter Configuration for more details.
For the example below, we have modified the following lines in the template parameter file:
params_template.ini#
data_catalogue_file = mock_catalogue_sim.dat
catalogue_columns = x,y,z
boxsize_x = 1000.
boxsize_y = 1000.
boxsize_z = 1000.
ngrid_x = 64
ngrid_y = 64
ngrid_z = 64
catalogue_type = sim
statistic_type = bispec
ell1 = 0
ell2 = 0
ELL = 0
bin_min = 0.005
bin_max = 0.105
num_bins = 10
Executing the program#
To run the ‘black-box’ C++ program, execute
$ <path-to-executable> <path-to-parameter-file>
in the shell.
Assuming the compiled binary executable is in the default build directory and the parameter file is in the current working directory which is the repository directory root, we execute the following to make the same measurement as before:
$ build/bin/triumvirate params_template.ini
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[2023-10-03 11:58:43 (+00:00:00) STAT] [A] Parameters and source data are being initialised.
[2023-10-03 11:58:43 (+00:00:00) STAT] [A.1] Reading parameters...
[2023-10-03 11:58:43 (+00:00:00) STAT] Parameters validated.
[2023-10-03 11:58:43 (+00:00:00) INFO] Check used-parameter file for reference: .//parameters_used.
[2023-10-03 11:58:43 (+00:00:00) STAT] [A.1] ... read parameters.
[2023-10-03 11:58:43 (+00:00:00) STAT] [A.2] Reading catalogues...
[2023-10-03 11:58:43 (+00:00:00) WARN] Catalogue 'nz' field is unavailable and will be set to the mean density in the bounding box (source=extfile:mock_catalogue_sim.dat).
[2023-10-03 11:58:44 (+00:00:01) INFO] Catalogue loaded: ntotal = 499214, wtotal = 499214.000, wstotal = 499214.000 (source=extfile:mock_catalogue_sim.dat).
[2023-10-03 11:58:44 (+00:00:01) INFO] Extents of particle coordinates: {'x': (-499.998, 499.998 | 999.996), 'y': (-500.000, 499.998 | 999.998), 'z': (-499.999, 500.000 | 999.999)} (source=extfile:mock_catalogue_sim.dat).
[2023-10-03 11:58:44 (+00:00:01) STAT] [A.2] ... read catalogues.
[2023-10-03 11:58:44 (+00:00:01) STAT] [B] Clustering statistics are being measured.
[2023-10-03 11:58:44 (+00:00:01) STAT] [B.1] Setting up binning...
[2023-10-03 11:58:44 (+00:00:01) STAT] [B.1] ... set up binning.
[2023-10-03 11:58:44 (+00:00:01) STAT] [B.2] Computing lines of sight...
[2023-10-03 11:58:44 (+00:00:01) STAT] [B.2] ... computed lines of sight.
[2023-10-03 11:58:44 (+00:00:01) STAT] [B.3] Aligning catalogues inside measurement box...
[2023-10-03 11:58:44 (+00:00:01) INFO] Extents of particle coordinates: {'x': (0.000, 1000.000 | 1000.000), 'y': (0.000, 999.996 | 999.996), 'z': (0.001, 999.997 | 999.996)} (source=extfile:mock_catalogue_sim.dat).
[2023-10-03 11:58:44 (+00:00:01) STAT] [B.3] ... aligned catalogues inside measurement box.
[2023-10-03 11:58:44 (+00:00:01) INFO] Alpha contrast: 1.000000e+00.
[2023-10-03 11:58:44 (+00:00:01) INFO] Normalisation factors: 8.037847e+00 (particle; used), 3.089711e+00 (mesh), 0.000000e+00 (mesh-mixed).
[2023-10-03 11:58:44 (+00:00:01) STAT] Computing bispectrum from a periodic-box simulation-type catalogue in the global plane-parallel approximation...
[2023-10-03 11:58:44 (+00:00:01) STAT] Bispectrum term at orders (m1, m2, M) = (0, 0, 0) computed.
[2023-10-03 11:58:44 (+00:00:01) STAT] ... computed bispectrum from a periodic-box simulation-type catalogue in the global plane-parallel approximation.
[2023-10-03 11:58:44 (+00:00:01) INFO] Measurements saved to .//bk000_diag.
[2023-10-03 11:58:44 (+00:00:01) STAT] [C] Data objects are being cleared.
[2023-10-03 11:58:44 (+00:00:01) INFO] Minimal estimate of peak memory usage: 0.1 gigabytes.
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
As before, the saved file contains a header with summary information about the parameters used and the input data as well as intermediary quantities:
# DEMO
with open("bk000_diag", 'r') as results_file:
print(results_file.read())
# Catalogue source: extfile:mock_catalogue_sim.dat
# Catalogue size: ntotal = 499214, wtotal = 499214.000, wstotal = 499214.000
# Catalogue particle extents: [(0.000, 1000.000), (0.000, 999.996), (0.001, 999.997)]
# Box size: [1000.000, 1000.000, 1000.000]
# Box alignment: centre
# Mesh number: [64, 64, 64]
# Mesh assignment and interlacing: tsc, false
# Normalisation factor: 8.037846928e+00 (particle)
# Normalisation factor alternatives: 8.037846928e+00 (particle), 3.089710869e+00 (mesh), 0.000000000e+00 (mesh-mixed)
# [0] k1_cen, [1] k1_eff, [2] nmodes_1, [3] k2_cen, [4] k2_eff, [5] nmodes_2, [6] Re{bk000_raw}, [7] Im{bk000_raw}, [8] Re{bk000_shot}, [9] Im{bk000_shot}
1.000000000e-02 1.149964290e-02 56 1.000000000e-02 1.149964290e-02 56 -9.702292027e+08 -3.420253273e-07 1.898133758e+08 1.407334952e-09
2.000000000e-02 2.047114034e-02 194 2.000000000e-02 2.047114034e-02 194 5.617152234e+09 8.492544529e-07 2.067769125e+08 -3.013608600e-10
3.000000000e-02 3.052421724e-02 488 3.000000000e-02 3.052421724e-02 488 8.716164555e+08 -6.571738503e-08 1.506636762e+08 1.326931029e-10
4.000000000e-02 4.062536317e-02 812 4.000000000e-02 4.062536317e-02 812 1.287789206e+09 -5.397597847e-09 1.249698447e+08 -1.638624364e-10
5.000000000e-02 5.033378732e-02 1250 5.000000000e-02 5.033378732e-02 1250 6.692105056e+08 -4.726288622e-10 1.070435506e+08 -1.837850334e-10
6.000000000e-02 6.040971138e-02 1896 6.000000000e-02 6.040971138e-02 1896 6.044495786e+08 1.153949022e-09 9.460508763e+07 4.027631477e-10
7.000000000e-02 7.034278912e-02 2426 7.000000000e-02 7.034278912e-02 2426 3.870081046e+08 2.337751971e-10 8.065164469e+07 -1.567788092e-10
8.000000000e-02 8.025683499e-02 3272 8.000000000e-02 8.025683499e-02 3272 2.877129959e+08 -2.396032946e-09 6.868552974e+07 -3.042787559e-11
9.000000000e-02 9.014322302e-02 4016 9.000000000e-02 9.014322302e-02 4016 2.302318466e+08 -3.731170687e-09 5.979982970e+07 -8.534409916e-11
1.000000000e-01 1.000819060e-01 5138 1.000000000e-01 1.000819060e-01 5138 1.979397038e+08 3.677058417e-09 5.157943854e+07 -2.521833190e-11
In addition, the interpreted/processed parameters used by the program are
saved to the "parameters_used*" file in the specified measurement
directory (the current working directory here), where the wildcard asterisk
is the output tag (empty here) specified in the input parameter file.
# DEMO
with open("parameters_used", 'r') as used_params_file:
print(used_params_file.read())
catalogue_dir =
measurement_dir = ./
data_catalogue_file = mock_catalogue_sim.dat
rand_catalogue_file =
catalogue_columns = x,y,z
output_tag =
boxsize_x = 1000.000
boxsize_y = 1000.000
boxsize_z = 1000.000
ngrid_x = 64
ngrid_y = 64
ngrid_z = 64
volume = 1.000000e+09
nmesh = 262144
alignment = centre
padscale = box
padfactor = 0.0000
assignment = tsc
interlace = false
assignment_order = 3
catalogue_type = sim
statistic_type = bispec
npoint = 3pt
space = fourier
ell1 = 0
ell2 = 0
ELL = 0
i_wa = 0
j_wa = 0
form = diag
norm_convention = particle
binning = lin
bin_min = 0.0050
bin_max = 0.1050
num_bins = 10
idx_bin = 0
save_binned_vectors =
verbose = 20