Cosmomc Grids

Grids of CosmoMC runs

CosmoMC has a set of scripts for generating and analysing grids of parameter runs using CosmoMC and GetDist, plus a variety of utility scripts and plotting functions. See the python readme for how to configure python on your system.

Most scripts can be run without arguments to show a list of possible input parameters. If you just want to use downloaded grid chains, you can use the plotting scripts and GetDist GUI directly without reading below; see Planck grid readme for examples.

A description for a grid of models is given in a python settings file.

If your results will be large, make a directory for the outputs on a data disk, then make a symbolic link to ./gridname in cosmomc, e.g. doing ln -s /scratch//..//mygrid mygrid, so "mygrid" is an easy shorthand for the location. This name is used as a parameter for most of the scripts.
Copy and edit python/settings_sample.py appropriately and save in e.g. python/my_settings_file.py
To generate grid directories and .ini files for CosmoMC do
python python/makeGrid.py mygrid my_settings_file
This produces sets of .ini files for each run in mygrid/iniFiles, and a directory structure for storing chain results.
Use runbatch.py to submit all the ini files produced to the queue; makeGrid will tell you what to do.

You may also need to produce a modify the supplied job submission script for your machine. See configuring job settings for your machine.

For a more complicated example of a large grid configuration see python/planck/settings_planck_2018.py. To run with settings in a module (subfolders) do e.g. "python makeGrid.py mygrid planck.settings_planck_2018". Covariance matrices (.covmat files) for Planck data with various parameter combinations are in planck_covmats folder, linked by default for grid runs.

Grid scripts take as a first argument the name of the folder containing the grid, and also many optional arguments that can be used to restrict to particular parameter or data combinations. Available scripts include:

makeGrid.py: as descried above, for setting up a grid of models. Stores the information in a "pickle" file that can then be read by the other scripts for iterating over batch items.
runbatch.py: submit runs to a job submission script
runningJobs.py: to check what is currently running and current convergence statistics.
checkConverge.py: after running runGridGetdist, this can be used to find chains with poor convergence
cleanup.py: for cleaning up files (e.g. before re-running a failed chain); can also just be used to list files for runs matching input parameters
runGridGetdist.py: runs getdist over the grid of chain results, and generating parameter constraints (.margestats files etc). This runs the fortran getdist, and produces plot_data files for plotting; run with --no_plots if you don't need these (e.g. using the GetDist GUI, or python plot scripts).
makeTables.py: script for making latex (and hence PDF) tables from .margestats and .minimum files produced by runGridGetDist.py.
To make basic tables do something like
```
 python python/makeTables.py main mytable_limit95 --limit 2
```
If you want to see how parameters shift around wrt to some baseline data combination use --changes_from_datatag datatag, which adds the fractional change in each parameter's mean in units of the baseline data combination's standard deviation; use --changes_from_paramtag paramtag to show shifts for each data combination wrt to the given baseline parameter combination. Use -h to see full list of possible options.
makePlots.py: script for making plots and comparison plots en mass.
copyGridFiles.py: to copy chains (add --chains) and/or getdist results (add --dist) to another folder or directly into a zip file, e.g.
```
 copyGridFiles.py mygrid mygrid.zip --dist --chains
```
extractDistFiles.py: for copying all files of a particular type from all the subdirectories in the grid, e.g. to extract all .covmat files, or all .pdf plots
makeCompare.py: to extract parameter constraints from .margestats files in the grid

Most scripts have standardized input parameters to restrict to subsets of grid items, e.g. filtering by importance sampling run, no importance sampling, or specific item name (pattern match).