Tags

Information for Off-Site Bmad Simulation



Distribution Overview

A Distribution, (also called a Bmad Distribution), is a stand-alone subset of the ACC Code Libraries used for accelerator/X-ray simulations.
  • A Distribution is meant for use on machines that are not part of the CLASSE laboratory network.
  • Distributions are freely available for download from the web. No CLASSE account is required.
  • New Distributions are generated approximately weekly.
  • Distributions contain a subset of the code in a Release. The code that is left out of Distributions is CESR specific and therefore not of interest to outside people.
  • Distributions can be built on Linux or Mac OS X (intel) computers. Note: The ACC Build System does not work on Windows but it is possible to build some subset of a Distribution. Please contact David Sagan if interested.
  • All " OFF-site" work is done using Distributions.
A Distribution contains:
  1. The Bmad source code to build a collection of libraries and executables. (The term " Bmad source code" includes not only the Bmad code library as well as the subsidiary libraries, like cesr_utils, but also the code for programs like Tao.)
  2. Script files to build the libraries and executables.

Distribution names are generally of the form: bmad_dist_YYYY_MMDD.tgz where YYYY_MMDD is the year_month-day tag of the release.

Bmad has been successfully been compiled on:
  • Linux / Intel ifort compiler (v 17.0.7.259 or later).
  • Linux / gcc gfortran compiler (v 5.3 or later).
  • Mac Intel / gcc gfortran compiler (v 5.3 or later).

Note: ifort and gcc versions earlier than what is stated above are known to fail.

Note: Bmad has, with restrictions, been built on Windows.

For more information on Bmad, please see http://www.lepp.cornell.edu/~dcs/bmad/



Updating a Distribution

To upgrade to more recent code, the safest way is to download (see below) the latest Distribution. Having multiple Distributions is advantageous if there are problems with the latest code which, while very rare, has happened.

The alternative to downloading a new Distribution is to use the "svn update *" command in the root directory of a Distribution. Unless you have a specific reason to do this, it is recommended that instead you preserve your old distribution and download a new distribution.

Note: When doing "svn update *" for the first time, SVN might complain that you need to do an upgrade first to modify the format of the SVN information in the Distribution to match the version of the SVN client that you are using. Unfortunately, the "svn upgrade *" command is a bit brain dead and is not smart enough to skip directories that are not under SVN version control. The work around is to upgrade directories individually or in groups. For example, "svn upgrade x*" will upgrade directories whose name starts with "x", etc.



Downloading a Distribution tarball

  1. Check compiler requirements for compiling Distributions.
  2. Download a distribution tarball from the
    Distributions DirectoryThe latest distribution is recommended.
  3. Unpack the tarball (substitute the actual file name for bmad_dist_YYYY_MMDD.tgz):
     tar xzf bmad_dist_YYYY_MMDD.tgz 

As of 2/2020, the Distribution has the following size:
Size of tarball:                        0.54 GB
Size with no builds:                    2.44 GB
Size with only production built:        6.42 GB
Size with debug and production builds: 13.56 GB
 



Updating the Distribution directly from the Repository

Sometimes it is useful to update files directly from the repository as opposed to downloading a tarball. For example, if a bug fix has been uploaded to the repository but that fix is not yet available in any tarball. However, updating via tarballs is to be preferred since code in the repository is generally not as stable as the code in the tarballs.

To update directly from the Repository, go to the distribution root directory and issue the command:

svn update *

This does not require a CLASSE account.

Libraries and Tools Needed for Compiling

Besides cmake (v 2.8.5 or later) and GCC, there are several standard unix/linux tools needed for compiling the xraylib library. They are:
  • gmake
  • automake
  • autoconf
  • libtool
  • m4

The Tao program needs the following standard packages:
  • readline
  • ncurses
  • termcap
  • X11 Core and Development Libraries (libX11 and libX11-devel)

PLplot (if it is used instead of PGPLOT) requires two more packages installed on your system:
  • cairo
  • pango
  • libXt-devel (usually installs as a dependency for cairo/pango)

tip Use the appropriate package manager (yum or rpm for Red Hat based systems, etc.) to install any needed tools. Note: The name of the package can vary from platform to platform. For example, the readline package is installed with yum using the command

 yum install readline-devel

A google search will generally reveal the appropriate name.

Note: The relevant compilers for your platform must appear in each user's default PATH , otherwise the scripts in the Distribution will not work!

macOS (formally Mac OS X)

Supported macOS versions are Catalina v10.15, Mojave v10.14 and High Sierra v10.13 - older versions may work but we can't assist you, if you have trouble.

In Catalina v10.15 the supported SHELL is still bash - don't follow Apple's warning to "upgrade" to zsh.

UPDATE 06-May-2020 - Xcode 11 BUG ALERT.

  • Due to a bug in Xcode 11, building the Bmad Distribution on macOS 10.14 Mojave will fail - Please do not upgrade to Xcode 11 with macOS 10.14 Mojave.
    • If you have upgraded to Xcode 11 with macOS 10.14 and see a build error:
      • /opt/local/lib/gcc5/gcc/x86_64-apple-darwin18/5.5.0/include-fixed/stdio.h:78:20: fatal error: _stdio.h: No such file or directory
    • please type the following command:
       cd /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs ; sudo ln -s MacOSX.sdk MacOSX10.14.sdk

  • To check which version of Xcode is installed, please type:
    system_profiler SPDeveloperToolsDataType

The following tools are needed in addition to those listed in the above Libraries and Tools section:
  1. XQuartz -- provides core and development libraries for X11 X-Windows graphics support for tao and other packages that require graphical output.
  2. Xcode -- Install the latest version from the Apple developer website or get it using the Mac App Store. Once you have Xcode installed, open a terminal, run
     sudo xcode-select --install
    Then click the Install button to install the required command line developer tools.
  3. To complete the Apple Xcode License Agreement, please type:
    sudo xcodebuild -license
  4. MacPorts -- package management system. See the installation site for downloads and instructions.

  • The following commands will download the necessary packages along with cmake and GCC using MacPorts.
    sudo port install gcc9
    sudo port select gcc mp-gcc9
    sudo port install wget
    sudo port install cmake
    sudo port install gmake
    sudo port install automake
    sudo port install autoconf
    sudo port install libtool
    sudo port install m4
    sudo port install pkgconfig
    sudo port install pango      # Only needed if PLplot is being used

Homebrew Users

  • With Homebrew use the following the commands:
    brew install make
    brew install gcc@9
    brew install cmake
    brew install automake
    brew install autoconf
    brew install libtool
    brew install m4
    brew install pkgconfig
    brew install pango
    
    sudo ln -s /usr/local/bin/gcc-9 /usr/local/bin/gcc
    sudo ln -s /usr/local/bin/glibtool /usr/local/bin/libtool
    sudo ln -s /usr/local/bin/glibtoolize /usr/local/bin/libtoolize
    
    export PATH=/usr/local/opt/gettext/bin:/usr/local/opt/make/libexec/gnubin:/usr/local/opt/m4/bin:/usr/local/opt/sqlite/bin:$PATH
    export LDFLAGS="-L/usr/local/opt/gettext/lib -L/usr/local/opt/libffi/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/sqlite/lib"
    export CPPFLAGS="-I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/sqlite/include"
    export PKG_CONFIG_PATH="/usr/local/opt/libffi/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig:/usr/local/opt/sqlite/lib/pkgconfig"

RHEL variants (RHEL, CentOS, Scientific Linux, CERN SLC)

If the version of the default GCC gfortran compiler is less than 5.3, you should install one of the RHEL Developer Toolset Software Collections, at least devtoolset-4 or greater.

To install devtoolset-6 (provides GCC gfortran version 6.2.1), see basic information below:
https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/

That page gives instructions for RHEL and CentOS. For straight Scientific Linux (not the CERN version), replace the first step of the instructions with the instructions from:
https://blog.abysm.org/2016/03/installing-developer-toolset-rhel-based-distributions/

For CERN SLC 6, follow the instructions here:
http://linux.web.cern.ch/linux/scientific6/docs/softwarecollections.shtml

Once devtoolset-6 is installed, the Bmad Distribution build system will find and initialize the devtoolset environment. If this fails, you can manually initialize the devtoolset environment (for bash shell users) by typing:

scl enable devtoolset-6 bash



Unsupported - Please be aware that we can't assist you with unsupported OSes.

Debain Variants (Ubuntu, etc.)

instructions here.

For Ubuntu, various packages are installed by:

 sudo ln -s /usr/bin/make /usr/bin/gmake
 sudo apt-get install libtool-bin
 sudo apt-get install libncurses-dev
 sudo apt-get install libreadline-dev
 sudo apt-get install libx11-dev
 sudo apt-get install libxt-dev
 sudo apt-get install gfortran



Windows Building

Installation instructions here.

Note: Running the Tao program is problematical on Windows. Also building on Windows is not supported by Cornell. It is therefore recommended that Distributions be built on Linux or Mac OSX instead.



Distribution Environment Setup

Important! The distribution environment setup must be done before building programs and must be done before running any programs. It is recommended that the setup commands be put in the appropriate shell startup file so that the setup is done automatically when logging in.
  1. Users should add the following text to their local account login or profile files.
    • Note: Replace PATH-to-DISTRIBUTION with the full path to the directory where the Distribution archive was unpacked.
    • For sh / ksh / bash users: The following stuff should be put in your ~/.bashrc file (or ~/.bash or ~/.profile file whatever one is appropriate):
      export DIST_BASE_DIR="PATH-to-DISTRIBUTION"
      source ${DIST_BASE_DIR}/util/dist_source_me
      ulimit -S -c 0
      ulimit -S -s 10240 # Remove this line when using a Mac
      ulimit -S -d 25165824
      The first ulimit prevents core dump files from being generated (takes up space and no one ever looks at them anyway). The next two ulimit lines helps prevent programs from bombing due to software limits on internal stack and heap sizes.
    • Note: The default behavior is for the setup process to print out informational messages. To suppress this output, you may define the environment variable DIST_SETUP_QUIET to have the value "Y". This is entirely optional.
  2. Run your initialization file by simply logging out and logging back in again.
  • For csh / tcsh users:
    • The Bmad Distribution environmental scripts were updated to use sh / ksh / bash, thus deprecating the attention on the support for the csh / tcsh environment. This change was to simplify maintenance and because tcsh has not been updated since 2009 (csh even longer).
    • We therefore recommend that Bmad users switch their default shell to sh / ksh / bash.
    • Else, a clunkly workaround is to simply type
      /bin/bash
      then type
      source ${DIST_BASE_DIR}/util/dist_source_me
      This works but you nolonger have any aliases that were defined in your .cshrc (or .tcshrc) file nor will csh commands (setenv, etc) work, as your shell has switched to bash.
    • Here at CLASSE, we have completely moved all users from tcsh to bash.

To see information about the Distribution that is being used by a terminal session, use the command:
distinfo

or
accinfo

Either command will display a summary of the active Distribution's information, build architecture, selected Fortran compiler and enviromental configuration.



Compiling a Distribution

  1. Unzip and extract the tar archive into a working directory of your choice with the command:
    tar -zxf bmad_dist_YYYY_MMDD.tgz
  2. Set up the Distribution envionment as explained in the Distribution Environment Setup section. Notice that the environmental variable DIST_BASE_DIR points to the root directory (with a name like to bmad_dist_YYYY_MMDD).
  3. Change to the root directory:
    cd $DIST_BASE_DIR
  4. A file named $DIST_BASE_DIR/util/dist_prefs is provided and contains the supported preferences that you may adjust to alter the behavior of the build. In particular, the compiler and plotting library are set in this file. Also whether a shared-object version of the libraries is built and whether MPI and/or OpenMP is supported is determined by the settings in this file. Documentation for dist_prefs is contained in the file itself. Edit this file as needed. For information on setting the plotting library, please click here.
  5. After editing dist_prefs, run your initialization file by loggin out and logging back in again.
  6. The final step is to build the source. For building production libraries and executables, type:
    util/dist_build_production
    For building debug libraries and executables, type:
    util/dist_build_debug
  • Note: To remove compiled files and restart from scratch run:
    util/dist_clean



Directories in a Distribution

Directories Containing Executables

production/bin Production executables are meant for ordinary use.
debug/bin Debug executables are meant for debugging code. Debug executables run slower but do more error checking and can be used with a debugger.

Cornell Developed Library Directories:

bmad Main simulation library.
cpp_bmad_interface C++ interface to Bmad
sim_utils Helper routines (matrix , plotting, file manipulation, etc.)

Cornell Developed Directories containing programs:

bsim Beam simulation programs
examples Example programs and example lattices.
regression_tests Bmad regression testing suite.
tao General simulation program.
util_programs Utility programs (EG lattice format converters, etc.)

Other Cornell Developed Directories:

lattice Lattice files for various machines.
util Utility scripts.
build_system Build system script.

3rd-Party Libraries ("Packages"):

forest FPP/PTC Full Polymorphic Package/Particle Tracking Code from Etienne Forest
fgsl A Fortran interface to the GNU Scientific Library
gsl GNU Scientific Library
hdf5 library for storing and managing data.
lapack Linear Algebra PACKage
mad_tpsa A Library for TPSA (Truncated Power Series Algebra) manipulations.
recipes_f-90_LEPP A F90 double precision version of Numerical Recipes
PGPLOT "the Pretty Good Plotting Package". For setup information, see the Plotting Packages wiki page.
plplot A cross-platform software package for creating scientific plots. For setup information, see the Plotting Packages wiki page.
xraylib A library for X-ray matter interaction cross sections for X-ray fluorescence applications


Building your own code using a Distribution

To build your own code, after successfully installing a Distribution, please see the Code Development wiki page.

The following is the ordered list of libraries to link to. Different programs will need different subsets of this list for linking. The libraries that the linker uses is specified by the LINK_LIBS variable in the cmake.XXX file used to compile and link your program. See the file bmad_dist/tao/cmake.tao for an example. The linker at link time will notify you if there are any missing libraries from the LINK_LIBS list. The order of the libraries in the LINK_LIBS list is important.
cpp_bmad_interface C++ interface for Bmad
bmad http://www.lns.cornell.edu/~dcs/bmad/
mad_tpsa Truncated Power Series Algebra package
xrlf03 Fortran wrapper for xraylib
xrl xraylib library https://github.com/tschoonj/xraylib
sim_utils Bmad utility routines
recipes_f-90_LEPP F90 Numerical Recipes converted to double precision
${PLOT_LINK_LIBS} pgplot or plplot; see Plot Packages wiki
forest Etienne Forest's FPP/PTC
fgsl Fortran wrapper for gsl
gsl Gnu Scientific library http://www.gnu.org/software/gsl/
gslcblas BLASS library used by gsl
hdf5_hl_cpp high level C++ wrapper for hdf5
hdf5_cpp C++ wrapper for hdf5
hdf5hl_fortran high level Fortran wrapper for hdf5
hdf5_fortran Fortran wrapper for hdf5
hdf5_hl high level C wrapper for hdf5
hdf5 Data model, library, and file format for storing and managing data. https://www.hdfgroup.org/HDF5/
lapack95 Fortran wrapper for lapack
lapack Linear Algebra library http://www.netlib.org/lapack/
blas Used by lapack



Licensing Issues

This distribution includes a modified version of the Numerical Recipes code. It is the responsibility of the end user to ensure that they have a valid Numerical Recipes license for any computer on which they build the Bmad distribution. Numerical Recipes licensing information (for individual computers or sites) can be obtained at www.nr.com.

The FOREST FPP/PTC (Polymorphic Tracking Code) package, from Etienne Forest, has no licensing issues.

The PGPLOT (pretty good plotting) package from Tim Pearson, states on the PGPLOT web page: PGPLOT is not public-domain software. However, it is freely available for non-commercial use. The source code and documentation are copyrighted by California Institute of Technology, and may not be redistributed or placed on public Web servers without permission. The software is provided ``as is'' with no warranty.



BMADZ Lattice Design Program

One program that is not included in the Distribution is BMADZ which is used for designing storage ring lattices and control room knobs. BMADZ was developed for designing Cornell CESR lattices but it can be applied to other storage rings as well. Note: Unless you know otherwise, it is better to use the Tao program included in the Distribution. That is, do not bother to download BMADZ unless you really know you need it.

To download BMADZ: First download a Distribution. Then, in the distribution root directory, download the cesr_utils and bmadz directories:

svn co https://accserv.lepp.cornell.edu/svn/CESR/CESR_libs/cesr_utils
svn co https://accserv.lepp.cornell.edu/svn/trunk/src/bmadz

To compile, use the following commands:

cd cesr_utils
mk
cd ../bmadz
mk



Issues with building a Distribution: Troubleshooting Page

Please see our wiki page on Troubleshooting ACC Code Issues



Getting Help

To request help, Please see our Help and Mailing List Information wiki page for email contacts.

Topic revision: r132 - 17 May 2020, DavidSagan
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding CLASSE Wiki? Send feedback