CosmoBox

CosmoBox
Development environments with recent gfortran (gcc 6/7/8), MPI, OpenBLAS, python, key python modules and LaTeX
New: updated December 2019

CosmoBox is a configured virtual development environment, with everything you need to use latex and build and debug in Fortran 2008/2013, for use with CAMB, CosmoMC, and many other programs. Files that you work on are stored on your local machine, so you can use your local editor, download large data files, etc, without having to put them inside the virtual environment. There are three main flavours of CosmoBox depending on what you want

Desktop image, giving access to GUI editors inside the virtual machine for compiling and debugging, plus the ability to run python graphical interfaces like the GetDist GUI for plotting.
Docker image, for command-line compiling and running. This loads faster and is more easily customizable, but then all editing and GUI programs need to be set up and run on your local machine.
StarCluster image, for running clusters on the AWS cloud

CosmoBox Desktop

A tool called Vagrant can be used to quickly run virtual machines boxes, with your local directory visible inside ready for compilation and running. CosmoBox is supplied as a Vagrant image that can be downloaded and run using standard Vagrant methods. To get started

Download and install VirtualBox
Download and install Vagrant
Change to your source folder that you want to work with (e.g. CosmoMC main directory) and run
```
vagrant init cmbant/cosmobox
vagrant up
```
This will download and install the machine for you, and start a running instance.
The /vagrant/ directory lets you see and work on files on your host machine. If you work consistently with files under /vagrant they are stored on your local machines, so you can destroy, update or rebuild the virtual environment without losing any data.
Use "vagrant suspend" and "vagrant resume" to start and stop the virtual machine. If you want to start over, clean up, or upgrade the Vagrant box, use "vagrant destroy".

You can then use the terminal to compile and run, run the CodeBlocks IDE by clicking on the icon, or running other graphical interfaces like the GetDist GUI from your local path (e.g. assuming your ran "vagrant up" from the cosmomc directory on your local machine, run "python /vagrant/python/GetDistGUI.py").

For further help with Vagrant see Getting Started (for example you may want to change the default memory/CPU configuration by editing the produced Vagrantfile).

Alternatively, you can Download the CosmoBox virtual machine image and load it in VirtualBox directly. This is convenient if you want to make customizations or store data in the virtual machine, but you'll have to set up your own shared folders as needed to access you local machine (and access them from ubuntu under /media/).

Configuration

Current configuration is based on

Ubuntu Linux 18.04
gcc/gfortran 7.4
Python 3.7 using miniconda, including matplotlib, scipy, sympy, cython, PySide for GUI environments and jupyter for notebooks
LaTeX
OpenMPI
LAPACK with OpenBLAS
git, wget and other standard tools
CodeBlocks IDE for fortran/C editing.
docker

The password and username is ubuntu. For full details, the Packer scripts for building the virtual machine are available on GitHub.

CosmoBox Docker

Docker is a way of quickly download and running virtual environments, with a simple automated build system for new images. The CosmoBox Docker image has a recent source build of gcc 7, plus other useful dependencies and tools for running cosmology programs. On Linux, Docker does not require running a virtual machine, as it uses the same kernel. On Windows and Mac, in practice it uses VirtualBox. To install and get running:

Mac:
- Download and run Kinematic
- Use the icon at the bottom left to open a terminal
Windows:
- Follow the Instructions, or run docker from within a VirtualBox Linux virtual machine if you also need to have VirtualBox installed..
Linux: See relevant distro Instructions

At the command line, make a virtual environment and command line for building and compiling simply by running

docker run -v /local/code/source:/virtual_path -i -t cmbant/cosmobox /bin/bash

In the virtual shell you can "cd /virtual_path", and you should see your host system files under /local/code/source, and you can then run "make" etc to build and run them. On Windows you may need to use -v /c/Users:/c/Users, and then find your files in docker under /c/Users/.

It works well with OpenMP and MPI for testing (depending on how many cores you have on your machine). Additional data files that you want to install and access from within the docker container should be installed on your machine somewhere under /local/code/source.

See Docker CosmoBox on Docker Hub for the configuration of the machine. You could easily make your own images based off cmbant/cosmobox (or the small cmbant/docker-gcc-build) if you want to make a customized environment with other things pre-installed.

To run stable gcc6 version rather than latest gcc7, use cmbant/cosmobox:gcc6, or try gcc8 and devel for more recent versions. If you also want astropy, healpy, camb and jupyter installed, get cmbant/cosmologist, which adds a couple of layers on top of CosmoBox.

CosmoBox AMI and CosmoCluster

These are a bit less up to date. Amazon Web Services (EWS) provides a way to run virtual machines in the cloud; see What is EC2 for an introduction. By using a CosmoBox image you can easily compile and test code on a free micro instance, or fire up a production cluster as large as you like by using StarCluster. To get started

Set up an AWS account following the instructions.
You get free credit, so it doesn't cost anything to experiment and run small machines. You can skip the "Create IAM User" and VPC options.
To manually run a single image for compiling and testing:
- log in to the AWS account
- change the region to "Oregon" (top right)
- Select the EC2 service (which is what runs virtual machines)
- Select the AMIs section, change the search dropdown to "Public images", search for "CosmoBox", select latest and click "Launch"
- Once launched, you can can see the "Public DNS" given in the console, which you can then ssh into, e.g. something like
```
ssh ubuntu@ec2-52-24-238-121.us-west-2.compute.amazonaws.com
```
  (where the exact domain changes every time you launch an instance).
- Do what you like, and terminate the image in the AWS console when done
To run a cluster for compiling, testing and production runs:
- Follow the Instructions for setting up your config file using your AWS information
- In the StarCluster config file:
  - Set AWS_REGION_NAME = us-west-2 in the [aws config] section
  - Use NODE_IMAGE_ID = ami-3f2c130f to use the CosmoBox image
  - Choose appropriate instance types depending on what you want to do (see list)
    For example you might have something like
```
[cluster smallcluster]
NODE_IMAGE_ID = ami-3f2c130f
NODE_INSTANCE_TYPE = t2.micro

[cluster cheapcluster]
# Declares that this cluster uses smallcluster as defaults
EXTENDS=smallcluster
#8 cores
NODE_INSTANCE_TYPE = c3.2xlarge
SPOT_BID = 0.25
CLUSTER_SIZE = 2
```
- From your local machine you can now start your cluster, e.g.
```
starcluster start cheapcluster
```
  and install and configure the machine as described in the Quick Start and full manual
- For storing your code, data and products you may want to make a separate EBS volume (like a virtual hard disk), then you can use and share it between clusters, without worrying about re-installing each time or losing data when the cluster is terminated. The main image also does not have much space, so you'll run out of room if you try to install much data onto it. See the StarCluster manual page
If you want to use a different Amazon region, you'd need to move the AMI first (launch, make a snapshot, then move region). Each run that lasts 12 hours using 16 cores will typically cost about US $12 using on-demand instances, significantly less for extra nodes if the spot instance price is low (e.g. around $2).
Maintained by Antony Lewis.