On a multi-processor machine you can start to get good results in a few of hours. On single processors you'll need to set aside rather longer. You can also run on a cluster.
By default CosmoMC uses a simple Metropolis algorithm or an optimized fast-slow sampling method (which works for likelihood with many fast nuisance parameters like Planck). The program takes as inputs estimates of central values and posterior uncertainties of the various parameters. The proposal density can use information about parameter correlations from a supplied covariance matrix: use one if possible as it will significantly improve performance. Covariance matrices are supplied for common sets of default base parameters. If you compile and run with MPI (to run across nodes in a cluster), there is an option to dynamically learn the proposal matrix from the covariance of the post-burn-in samples so far. The MPI option also allows you to terminate the computation automatically when a particular convergence criterion is matched. MPI is strongly recommended.
There are two programs supplied cosmomc and getdist. The first is a compiled Fortran code does the actual Monte-Carlo and produces sets of .txt chain files and (optionally) .data output files (the binary .data files include the theoretical CMB power spectra etc.). The "getdist" program analyses the .txt files calculating statistics and outputs files for the requested 1D, 2D and 3D plots (and could be used independently of the main cosmomc program), and there are Fortran and pure python versions. The "cosmomc" program also does post processing on .data files, for example doing importance sampling with new data.
Please e-mail details of any bugs or enhancements to Antony Lewis. If you have any questions please ask in the CosmoCoffee computers and software forum. You can also read answers to other people's questions there.
Downloading and Compiling
To run CosmoMC you will need a Fortran 2003 (or higher) compiler - Intel Fortran 14 or higher works (earlier versions will not). Others may, but gfortran does not support Fortran 2003 well enough at the moment (see gfortran bug list; we hope to have gfortran bugs fixed soon). If you just want to analyse chains and run plots, you only need python 2.7+ (see plotting below).
Using MPI simplifies running several chains and proposal optimization. MPI can be used with OpenMP: generally you want to use OpenMP to use all the shared-memory processors on each node of a cluster, and MPI to run multiple chains on different nodes (the program can also just be run on a single CPU).
Using Visual Fortran there's no need to use the Makefile, just open the project file in the source folder, and set params.ini as the program argument. This is not set up to compile with MPI, so mainly useful for development.
See BibTex file for relevant citations.See the supplied params.ini file for a fairly self-explanatory list of input parameters and options. The file_root entry gives the root name for all files produced. Running using MPI on a cluster is recommended if possible as you can automatically handle convergence testing and stopping.
mpirun -np 4 ./cosmomc params.iniThere is also a supplied script python/runMPI.py that you may be able to adapt for submitting jobs to a job queue, e.g.
python python/runMPI.py paramsto run using the params.ini parameters file. Run runMPI.py without arguments to see a list of options (number of chains, cores, wall time, etc). You will probably need to set up a job script template file for your computer cluster (a couple of samples are supplied). See Configuring job settings for your machine.
If things go wrong check the .log and any error files in your cosmomc/scripts directory.
Input Parametersparam[paramname] = center min max start_width propose_widthThe start_width entry determines the randomly chosen dispersion of the starting position about the given centre, and should be as large or larger than the posterior (for the convergence diagnostic to be reliable, chains should start at widely separated points). Chains are restricted to stay within the bounds determined by min and max. If the parameter is not to be varied in your run, you can just specify the fixed value.
The sampler proposes changes in parameters using a proposal density function with a width determined by propose_width (multiplied by the value of the global propose_scale parameter). The propose_width should be of the order of the conditional posterior width (i.e. the expected error in that parameter when other parameters are fixed). If you specify a propose_matrix (approximate covariance matrix for the parameters), the parameter distribution widths are determined from its eigenvalues instead, and the proposal density changes the parameter eigenvectors. The covariance matrix can be computed using "getdist" once you have done one run - the file_root.covmat file. The planck_covmats/ directory is supplied with many you can use. The covariance matrix does not have to include all the parameters that are used - zero entries will be updated from the input propose widths of new parameters (the propose width should be of the size of the conditional distribution of the parameter - typically quite a bit smaller than the marginalized posterior width; generally too small is better than too large). The scale of the proposal density relative to the covariance is given by the propose_scale parameter. If your propose_matrix is significantly broader than the expected posterior, this number can be decreased.
If you don't have a propose_matrix, you can use estimate_propose_matrix = T to automatically estimate it by numerical fitting about the best fit point (with action=2 to stop, or action=0 to continue straight into an MCMC run).
Other sampling methods that were implemented in previous version have not currently been updated for the latest version.
The runMPI.py and runbatch.py submission scripts use a job template file to submit jobs to the queue on your cluster. The job_script files is supplied as sample, and you may need to make your own template specific to the configuration of your machine, depending on what queuing system you have (e.g. PBS, SLURM, etc.) Recommended steps are
export COSMOMC_job_template=job_script_mycluster export COSMOMC_coresPerNode=8 export COSMOMC_nodes=2 export COSMOMC_chainsPerNode=2 export COSMOMC_queue=default_queue export COSMOMC_walltime=24:00:00
weight like param1 param2 param3 ...
The weight gives the number of samples (or importance weight) with these parameters. like gives -log(likelihood). The getdist program and gui could be used completely independently of the cosmomc program.
Run getdist distparams.ini or python python/GetDist.py distparams.ini to process the chains specified in the parameter input file distparams.ini. This should be fairly self-explanatory, in the same format as the cosmomc parameter input file. Note that the outputs from the python and fortran versions of getdist are not identical due to slightly different algorithms, but should be consistent to better than the sampling noise. Both can produce pre-computed densities for plotting (in plot_data directory), but the python plotting scripts do not require this (they are generated on demand) - output will be faster and make far fewer files if no_plots=T.
GetDist Parameters
Output Text Files
Plotting
GetDist can be used to produce basic simple scripts for various types of plots. You can also just write plotting scripts, or use the GetDist Gui, in which case there is no need to run GetDist first.
If no_plots=F, GetDist produces scripts files to make simple 1D, 2D and 3D plots. In the fortran getdist these can be either python or matlab, set plot_ext=py or plot_ext=m for which you prefer, python is now preferred and the matlab option is not available from python getdist. The script files produced are called
Parameter labels are set in distparams.ini - if any are blank the parameter is ignored. You can also specify which parameters to plot, or if parameters are not specified for
the 2D plots or the colour of the 3D plots getdist automatically works out
the most correlated variables and uses them.
The data files used by python and Matlab are output to the plot_data directory.
Performance of the MCMC can be improved by using parameters which have a close to Gaussian posterior distribution. The default parameters (which get implicit flat priors) are
Parameters like H0 and omegal (ΩΛ) are derived from the above. Using theta rather than H0 is more efficient as it is much less correlated with other parameters. There is an implicit prior 40 < H0 < 100 (which can be changed).
The .txt chain files list derived parameters after the base parameters.
The list of parameter names and labels used in the default parameterization is listed in the supplied params_CMB.paramnames file.
Since the program uses a covariance matrix for the parameters, it knows about (or will learn about) linear combination degeneracies. In particular ln[10^10 A_s] - 2*tau is well constrained, since exp(-2tau)A_s determines the overall amplitude of the observed CMB anisotropy (thus the above parameterization explores the tau-A degeneracy efficiently). The supplied covariance matrix will do this even if you add new parameters.
Changing parameters does in principle change the results as each base parameter has a flat prior. However for well constrained parameters this effect is very small. In particular using theta rather than H_0 has a small effect on marginalized results.
The above parameterization does make use of some knowledge about the physics, in particular the (approximate) formula for the sound horizon.
To change parameterization make a new .paramnames file, then change sources/params_CMB.f90 to change the mapping of physical parameters to MCMC array indices, and also to read in your new .paramnames file.
Likelihoods that you use may also have their own nuisance parameters.
The code makes use of Fortran 2003 features and structures. There is automated code documentation, which includes modules, classes and subroutines. File and class documentation gives inheritance trees to help understand the structure, for example GeneralTypes.f90 contains the key base classes that other things are inherited from.
You are encouraged to examine what the code is doing and consider carefully
changes you may wish to make. For example, the results can depend on the
parameterization. You may also want to use different CAMB modules, e.g.
slow-roll parameters for inflation, or use a fast approximator. The main
source code files you may want to modify are
The .ini file comments should explain the other options.
Example: Since many people get this wrong, here is an illustration of what happens when generating plots from a tensor run set of chains (with prior r>0):
If you are not using parameter names you can set numbered limit parameters, e.g. limits12=0 N for parameter 12.
Incorrect result when limits[r02] is not set.
Correct result when setting limits[r02]=0 N.
For 2D plots smooth_scale_2D is the smoothing scale relative to the bin size
See the GetDist GUI readme for details of hope to use the GetDist GUI to make plots from chains. Even if you ultimately write custom scripts to optimize how things are displayed, this can be a quick and easy way to compare results and see what you want to plot.
See the CosmoMC python readme for details of how to use the plotting library.
If plot_ext=m GetDist produces Matlab '.m' files. Matlab support is not likely to be developed further, but maintained from the previous version for convenience.
To compare two different sets of chains set compare_num=1 in the .ini file, and compare1 to the root name of some chains you have previously run GetDist on.
Convergence diagnostics
The getdist program will output convergence diagnostics, both short summary information when getdist is run, and also more detailed information in the file_root.converge file. When running with MPI the first two of the parameters below can also be calculated when running the chains for comparison with a stopping criterion (see the .ini input file).
Differences between GetDist and MPI run-time statistics
GetDist will cut out ignore_rows from the beginning of each chain, then compute the R statistic using all of the remaining samples. The MPI run-time statistic uses the last half of all of the samples. In addition, GetDist will use all the parameters, including derived parameters. If a derived parameter has poor convergence this may show up when running GetDist but not when running the chain (however the eigenvalues of covariance of means is computed using only base parameters). The run-time values also use thinned samples (by default every one in ten), whereas GetDist will use all of them. GetDist will allow you to use only subsets of the chains.
Parameterizations
Using CosmoMC as a generic sampler
You can use CosmoMC to sample any custom function that you want to provide. To do this
You can use named or un-named parameters. Making your own .paramnames file to name your parameters usually makes things easier (except possibly if you have a very large number of parameters where number labels make sense).
Programming
call SetTheoryParameterNumbers(num_slow_params,num_semi_slow_params)which sets the number of hard and semi-hard parameters (the latter being things like initial power spectrum parameters). Make a new .paramnames file with the new parameter names in and change and then
call this%Initialize(Ini,Names, 'params_CMB.paramnames', Config)to set your new parameter names file.
Feel free to ask questions (and read answers to other people's) on the CosmoCoffee software forum.