You are here: CLASSE Wiki>ACC/ACL Web>OffsiteDoc (20 Sep 2023, DavidSagan)Edit Attach
Tags

Information for "Off-Site" Bmad Distribution or Conda-Forge Setup

Distribution Overview

A Distribution, (also called a Bmad Distribution), is a stand-alone subset of the ACC Code Libraries, including the Bmad toolkit used for accelerator/X-ray simulations.
  • A Distribution is meant for use on machines that are not part of the CLASSE laboratory network.
  • Distributions are freely available for download from the web. No CLASSE account is required.
  • New Distributions are generated approximately weekly.
  • Distributions contain a subset of the code in a Release. The code that is left out of Distributions is CESR specific and therefore not of interest to outside people.
  • Distributions include the Tao program (as well as many other programs).
  • Distributions can be built on Linux or Mac OS X (intel) computers. For Windows systems, WSL2 should be used.
  • All " OFF-site" work is done using Distributions.
A Distribution contains:
  1. The Bmad source code to build a collection of libraries and executables. (The term " Bmad source code" includes not only the Bmad code library as well as the subsidiary libraries, like cesr_utils, but also the code for programs like Tao.)
  2. Script files to build the libraries and executables.

Distribution names are generally of the form: bmad_dist_YYYY_MMDD-N.tgz where YYYY_MMDD is the year_month-day tag of the release and N is the version number which starts with "0" for the first version generated on a given date, etc.

Bmad has been successfully been compiled on:
  • Linux / Intel ifort compiler (v 17.0.7.259 or later).
  • Linux / gcc gfortran compiler (v 6.3 or later).
  • Mac Intel / gcc gfortran compiler (v 6.3 or later).

Note: ifort and gcc versions earlier than what is stated above are known to fail.

Note: Bmad can be built on Windows by installing a Linux distribution via WSL2.

For more information on Bmad, please see http://www.lepp.cornell.edu/~dcs/bmad/

Conda-Forge installation

Pre-compiled Bmad executables (Tao, etc) can be installed via conda-forge (a Python package manager). When installing with conda, you do not have to download and build a Bmad Distribution. Conda can be used on Windows using WSL2. Using Conda is convenient since Conda takes care of the many package dependencies that Bmad based programs have.

The following instructions for conda are for installing under MacOS or Linux. For installing on Windows, you need to install a Linux distribution under WSL2. Follow the instructions under MS Windows for setting up and running under WSL2 (read that entire section), but replace the step of building Bmad with the following steps, run under the Linux distribution that you installed under WSL2. Do not install the conda package manger for Windows, install the Linux version while running in Linux.

Important: Do not invoke the Distribution script dist_source_me if you are using conda-forge. There are conflicts such that Distributions and conda-forge cannot both be active at once.

Conda Install instructions:
  1. Install the conda package manager according to your platform: https://docs.conda.io/en/latest/miniconda.html
  2. Install Bmad via: 
    conda install -c conda-forge bmad

The executables will be added to your path. For example, type: 

tao

to run Tao.

By default this will install the latest distribution. All distributions can be listed using: 
conda search -c conda-forge bmad

A specific distribution can be installed as: 
conda install -c conda-forge bmad=20211013.0

To update the Bmad version use: 
conda update -c conda-forge bmad

To install MPI versions of programs that have MPI versions (Note: Tao does not have an MPI version) use 
conda install -c conda-forge bmad=*=mpi_openmpi*

Unlike MPI, OpenMP is always enabled with a conda installation.

Note: If you want to modify existing Bmad based program or create new ones under conda, you can link to the libraries at $CONDA_PREFIX/lib and use headers from $CONDA_PREFIX/include. The only drawback is that you will need a compatible compiler as the one used by conda-forge which can be installed with: 
conda install -c conda-forge compilers  # this will install C, CXX and FC

If it is ever desired to using a Bmad Distribution instead of using conda-forge, uninstallation can be done with the command: 

conda uninstall -c conda-forge bmad

Bmad Distribution build steps

To build a Bmad Distribution (as opposed to using Conda-Forge), do the following steps:
  1. Install the required packages for your operating system and do any required setup
  2. Download a distribution
  3. Set up the distribution environment (this step will also need to be executed any time you use Bmad)
  4. Compile the distribution

Downloading a Distribution tarball

  1. Check compiler requirements for compiling Distributions.
  2. Download a distribution tarball from the
    -------->> Distributions Directory <<--------
     The latest distribution is recommended.
  3. Unpack the tarball (substitute the actual file name for bmad_dist_YYYY_MMDD-N.tgz): 
     tar xf bmad_dist_YYYY_MMDD-N.tgz

As of 2/2020, the Distribution has the following size: 
Size of tarball:                        0.54 GB
Size with no builds:                    2.44 GB
Size with only production built:        6.42 GB
Size with debug and production builds: 13.56 GB

Note to script writers: The following will set a shell variable to the name of the latest Distribution file (without the .tgz suffix) and download the tarball 

DIST_FILE=$(curl https://www.classe.cornell.edu/~cesrulib/downloads/latest_distribution_file_prefix)
wget https://www.classe.cornell.edu/~cesrulib/downloads/tarballs/$DIST_FILE.tgz

Updating a Distribution

Sometimes it is useful to update files directly from the repository as opposed to downloading a tarball. For example, if a bug fix has been uploaded to the repository but that fix is not yet available in any tarball. However, downloading a new Distribution tarball is to be preferred since code in the repository is generally not as stable as the code in the tarballs. Having multiple Distributions is advantageous if there are problems with the latest code which, while very rare, has happened.

To update directly from the Repository, go to the distribution root directory and issue the command: 

svn update *

This does not require a CLASSE account.

Note: When doing "svn update *" for the first time, SVN might complain that you need to do an upgrade first to modify the format of the SVN information in the Distribution to match the version of the SVN client that you are using. Unfortunately, the "svn upgrade *" command is a bit brain dead and is not smart enough to skip directories that are not under SVN version control. The work around is to upgrade directories individually or in groups. For example, "svn upgrade x*" will upgrade directories whose name starts with "x", etc.

After updating a Distribution, the code needs to be recompiled.

Installing Packages for Various Operating Systems

Detailed instructions are given below for

For any other operating systems, here is some general guidance:

Besides cmake (v 2.8.5 or later) and GCC, there are several standard unix/linux tools needed for compiling the xraylib library. They are:
  • gmake
  • automake
  • autoconf
  • libtool
  • m4

The Tao program needs the following standard packages:
  • readline
  • ncurses
  • termcap
  • X11 Core and Development Libraries (libX11 and libX11-devel)

PLplot (if it is used instead of PGPLOT) requires two more packages installed on your system:
  • cairo
  • pango
  • libXt-devel (usually installs as a dependency for cairo/pango)

tip Use the appropriate package manager (yum or rpm for Red Hat based systems, etc.) to install any needed tools. Note: The name of the package can vary from platform to platform. For example, the readline package is installed with yum by installing readline-devel. A google search will generally reveal the appropriate name.

Note: The relevant compilers for your platform must appear in each user's default PATH , otherwise the scripts in the Distribution will not work!

macOS (formally Mac OS X)

Note: The SHELL for building Bmad is bash.

  • If prompted, please don't follow Apple's warning to "upgrade" to zsh.
  • If your SHELL set to zsh, please use the command below to switch back to bash: 
    chsh -s /bin/bash

Note: There are various Xcode/OS problems. See the Troubleshooting page for details.

The following tools are needed in addition to those listed in the above Libraries and Tools section:

  1. XQuartz -- provides core and development libraries for X11 X-Windows graphics support for tao and other packages that require graphical output.
  2. Xcode -- Install the latest version from the Apple developer website or get it using the Mac App Store. Once you have Xcode installed, open a terminal, run 
     sudo xcode-select --install
    Then click the Install button to install the required command line developer tools.
  3. To complete the Apple Xcode License Agreement, please type: 
    sudo xcodebuild -license

Setup with MacPorts Package Management System.

MacPorts documentation and download here.

  • The following commands will download the necessary packages. 
    sudo port install gcc12       # Any version (> 6) of gcc may be used. Except! Use gcc 12 or above with M1 Mac to avoid compiler bug.
sudo port select gcc mp-gcc12
sudo port install wget
sudo port install cmake
sudo port install gmake
sudo port install automake
sudo port install autoconf
sudo port install libtool
sudo port install m4
sudo port install pkgconfig
sudo port install pango      # Note: Needed for PLplot.
sudo port install zlib

Setup with Homebrew Package Management System.

Note: Using Homebrew and MacPorts on the same computer can lead to conflicts. It is highly recommended that only one be used.
  • With Homebrew use the following the commands: 
    brew install make
brew install gcc@9  # A different version (> 6) of gcc may be used if desired
brew install cmake
brew install automake
brew install autoconf
brew install libtool
brew install m4
brew install pkgconfig
brew install pango
brew install zlib

sudo ln -s /usr/local/bin/gcc-9 /usr/local/bin/gcc
sudo ln -s /usr/local/bin/glibtool /usr/local/bin/libtool
sudo ln -s /usr/local/bin/glibtoolize /usr/local/bin/libtoolize

export PATH=/usr/local/opt/gettext/bin:/usr/local/opt/make/libexec/gnubin:/usr/local/opt/m4/bin:/usr/local/opt/sqlite/bin:$PATH
export LDFLAGS="-L/usr/local/opt/gettext/lib -L/usr/local/opt/libffi/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/sqlite/lib"
export CPPFLAGS="-I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/sqlite/include"
export PKG_CONFIG_PATH="/usr/local/opt/libffi/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig:/usr/local/opt/sqlite/lib/pkgconfig"

  • If ACC_ENABLE_SHARED is set to "Y", you must do the following: 
    sudo mkdir /opt/local/
sudo ln -s /usr/local/lib /opt/local/lib
  • Discover the latest version of readline installed by Homebrew: 
    ls /usr/local/Cellar/readline/
  • Use the readline version name to build the symlinks in the command below. For example, if the Homebrew version of readline is 8.1.1: 
    sudo ln -s /usr/local/Cellar/readline/8.1.1/lib/lib* /usr/local/lib/
sudo ln -s /usr/local/Cellar/readline/8.1.1/lib/pkgconfig/*.pc /usr/local/lib/pkgconfig/
sudo ln -s /usr/local/Cellar/readline/8.1.1/include/readline /usr/local/include

RHEL variants (RHEL, CentOS, Scientific Linux, CERN SLC)

The instructions here assume that ACC_PLOT_PACKAGE="plplot" in dist_prefs.

For RHEL8 and its derivatives (Rocky, CentOS-8), install the required packages with 
yum install bc cmake gcc-c++ gcc-gfortran libtool make openssl-devel pango-devel readline-devel

then follow the build instructions.

For RHEL7 and its derivatives (CentOS, Scientific Linux), you will need a newer compiler than the default that is available, which will require the RHEL Developer Toolset. For RHEL7 itself, you will need to adjust your subscription accordingly. For the RHEL derivatives, you will ned need to first install the Software Collections repository as follows:

CentOS 7 
yum install centos-release-scl
Scientific Linux 7 
yum install yum-conf-softwarecollections
Next, you can install the required packages with 

yum install cmake devtoolset-10-gcc-gfortran devtoolset-10-gcc-c++ devtoolset-10-make libtool openssl-devel pango-devel readline-devel

Before starting the build process, you will need to execute 

scl enable devtoolset-10 bash

This will create a new subshell where your environment is set up to use the required compilers. If you quit and come back to the build process, you will need to re-execute this latter command to again give yourself the required build environment.

Debain Variants (Ubuntu, etc.)

Your Ubuntu installation should be at version 18.04 or higher. The required packages for a PLPlot build (the default) are installed by: 

 sudo apt-get install gfortran g++ cmake libtool-bin libreadline-dev libpango1.0-dev libssl-dev bc

When building with PGPLOT add 

sudo apt-get install libxt-dev

MS Windows

To install Bmad on Windows 10/11:
  1. Install WSL2, which will allow you to run a Linux distribution under Windows. For most users the best choice will be to install the latest Ubuntu distribution (which is what you will get by default). This may not perform well on a system with a small amount of memory (you're effectively running a second computer on your hardware). Note it is no longer necessary to install an X server, the latest WSL (Windows Store version) has one built-in.
  2. Build Bmad as described above on whatever Linux distribution you've chosen.

Distribution Environment Setup

Important! The distribution environment setup must be done before building or running programs. It is recommended that the setup commands be put in the appropriate shell startup file so that the setup is done automatically when logging in.

  1. Distributions are compiled using bash, sh, or ksh. If your default shell is not one of these, you have two options:
    • Switch the default shell to bash (recommended), or
    • Switch to the bash shell each time you want to compile or run using the command 
      /bin/bash
  2. Users should add the following text to their local account login or profile files (or put these commands in a command file and run the command file each time you create a new command window).
    • Note: Replace PATH-to-DISTRIBUTION with the full path to the directory where the Distribution archive was unpacked.
    • The following stuff should be put in your ~/.bashrc file (or ~/.bash or ~/.profile file whatever one is appropriate): 
      export DIST_BASE_DIR="PATH-to-DISTRIBUTION"
source ${DIST_BASE_DIR}/util/dist_source_me
ulimit -S -c 0
ulimit -S -s 10240 # Remove this line when using a Mac
ulimit -d unlimited
      The first ulimit prevents core dump files from being generated (takes up space and no one ever looks at them anyway). The next two ulimit lines helps prevent programs from bombing due to software limits on internal stack and heap sizes.
    • If you are using the multiple Distribution scheme as discussed above, PATH-to-DISTRIBUTION should refer the bmad_dist soft link. For example, the definition of DIST_BASE_DIR might look like: 
      export DIST_BASE_DIR="..../distribution_base_dir/bmad_dist"
    • Note: The default behavior is for the setup process to print out informational messages. To suppress this output, you may define the environment variable DIST_SETUP_QUIET to have the value "Y". This is entirely optional.
  3. If you have put the initalization commands in a shell startup file, setup the environment by simply logging out and logging back in again (and using the /bin/bash command if needed). Otherwise, if not in a shell startup file, source the command file the initialization commands are in.
  4. To see information about the Distribution that is being used by a terminal session, and to make sure the environment is properly setup, use either of the commands: 
    distinfo
accinfo
    Either command will display a summary of the active Distribution's information, build architecture, selected Fortran compiler and enviromental configuration. In particular, you should see a set of environment variables starting with the prefix ACC_. If you do not see this, the environment has not been properly setup.

Compiling a Distribution

  1. Unzip and extract the tar archive into a working directory of your choice with the command: 
    tar -zxf bmad_dist_YYYY_MMDD-N.tgz
  2. Set up the Distribution environment as explained in the Distribution Environment Setup section. Notice that the environmental variable DIST_BASE_DIR points to the root directory (with a name like to bmad_dist_YYYY_MMDD-N).
  3. Change to the root directory: 
    cd $DIST_BASE_DIR
  4. Edit the file:
    $DIST_BASE_DIR/util/dist_prefs
    This file contains the preferences for custimizing the build. Documentation for dist_prefs is contained in the file itself. In particular, one can set:
  5. After editing dist_prefs, run your initialization file by loggin out and logging back in again.
  6. The final step is to build the source. For building production libraries and executables, type: 
    util/dist_build_production
    For building debug libraries and executables, type: 
    util/dist_build_debug
  • Note: If you ever want to restart the build from scratch, before using one of the build commands above, first remove all compiled files using the command: 
    util/dist_clean

Organizing Multiple Distributions

It can be convenient to have multiple Distributions. Older Distributions can act as backups in case, for example, the results of simulation programs change. The recommended way to handle multiple distributions is to create a root directory to hold the different distributions. An example directory structure is: 

              distribution_base_dir
                       |
     +--------------------------+------------------------+---------------------+
     |                          |                        |                     |
bmad_dist_2020_0507-0     bmad_dist_2020_0821-0    bmad_dist_2121_0107-0    bmad_dist (soft link)

Here the Distribution root directory is called distribution_base_dir. This directory hold three Distributions and a directory soft link called bmad_dist. The soft link will point to the distribution currently being used. For example, if the Distribution named bmad_dist_2021_0107-0 is to be used, the soft link is created by the command (do this in the distribution_base_dir): 

ln -sfn bmad_dist_2021_0107-0 bmad_dist

The idea is that to switch from using one Distribution to another, only the bmad_dist soft link needs to be changed. The setup scripts discussed below do not have to be modified since they refer to the bmad_dist soft link.

Directories in a Distribution

Directories Containing Executables

production/bin Production executables are meant for ordinary use.
debug/bin Debug executables are meant for debugging code. Debug executables run slower but do more error checking and can be used with a debugger.

Cornell Developed Library Directories:

bmad Main simulation library.
cpp_bmad_interface C++ interface to Bmad
sim_utils Helper routines (matrix , plotting, file manipulation, etc.)

Cornell Developed Directories containing programs:

bsim Beam simulation programs
code_examples Example programs and example lattices.
regression_tests Bmad regression testing suite.
tao General simulation program.
util_programs Utility programs (EG lattice format converters, etc.)

Other Cornell Developed Directories:

bmad-doc Documentation of Bmad and associated programs.
util Utility scripts.
build_system Build system script.

3rd-Party Libraries ("Packages"):

fftw Discrete Fourier transform library
forest FPP/PTC Full Polymorphic Package/Particle Tracking Code from Etienne Forest
fgsl A Fortran interface to the GNU Scientific Library
gsl GNU Scientific Library
hdf5 library for storing and managing data.
lapack Linear Algebra PACKage
PGPLOT "the Pretty Good Plotting Package". For setup information, see the Plotting Packages wiki page.
plplot A cross-platform software package for creating scientific plots. For setup information, see the Plotting Packages wiki page.
xraylib A library for X-ray matter interaction cross sections for X-ray fluorescence applications

Building your own code using a Distribution

To build your own code, after successfully installing a Distribution, please see the Code Development wiki page.

The following is the ordered list of libraries to link to. Different programs will need different subsets of this list for linking. The libraries that the linker uses is specified by the LINK_LIBS variable in the cmake.XXX file used to compile and link your program. See the file bmad_dist/tao/cmake.tao for an example. The linker at link time will notify you if there are any missing libraries from the LINK_LIBS list. The order of the libraries in the LINK_LIBS list is important.

cpp_bmad_interface C++ interface for Bmad
bmad http://www.lns.cornell.edu/~dcs/bmad/
sim_utils Bmad utility routines
${PLOT_LINK_LIBS} pgplot or plplot; see Plot Packages wiki
${ACC_BMAD_LINK_LIBS} List of third party libraries

Licensing Issues

The FOREST FPP/PTC (Polymorphic Tracking Code) package, from Etienne Forest, has no licensing issues.

The PGPLOT (pretty good plotting) package from Tim Pearson, states on the PGPLOT web page: PGPLOT is not public-domain software. However, it is freely available for non-commercial use. The source code and documentation are copyrighted by California Institute of Technology, and may not be redistributed or placed on public Web servers without permission. The software is provided ``as is'' with no warranty.

Issues with building a Distribution: Troubleshooting Page

Please see our wiki page on Troubleshooting ACC Code Issues

Getting Help

To request help, Please see our Help and Mailing List Information wiki page for email contacts.

Old: Docker Setup

Using Docker to setup a Bmad Distribution has essentially been abandoned in favor of using Conda-Forge. The unmaintained Docker setup instructions are kept for reference.

Edit  | Attach | Print version | History: r195 < r194 < r193 < r192 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r195 - 20 Sep 2023, DavidSagan
ACC/ACL

  • ACC/ACL Web

  • CLASSE Computing Info
  • CLASSE

Create personal sidebar

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding CLASSE Wiki? Send feedback