Ember now available

Posted: October 17, 2013

Access to Ember is now available. Please take note of the information listed below before you start using the cluster. We consider this first few days of access as a testing period, as it is impossible for CHPC to thoroughly test all possible running conditions. We will be monitoring the nodes and will offline any nodes if we see an issue. If you run into any problems please send a report to issues.

Below is some very important information about changes

  • The cluster is now running RHEL6 and has new versions of the batch scheduler and usage accounting software; it is now the same as what is running on Kingspeak.
  • The batch policies should be identical to what they were before ember was relocated.
  • The node numbering has changed -- very important for those who ssh to the nodes. A new listing of the node numbers is available on this CHPC wiki page
  • SSH keys have changed. You will get messages and not be able ssh to the nodes until you have either deleted your .ssh/known_hosts or edited this file to remove all entries for the ember interactive and compute nodes by searching and removing keys for ember, ember*, em*, and 172.17.4*.*
  • You need to check if there is a new RH6 build of any other package/library (e.g., OpenFoam, python,netcdf) before running – these are indicated by a _rhel6 or _rh6 after the version number. While CHPC has tested many of the builds, we may have not caught all the ones that need rebuilt for the new OS --- please let us know of any we may have missed.
  • You need to check your codes to see if they need rebuilt. We strongly recommend that you no longer use any packages/libraries in /uufs/arches/sys location; we will not be maintaining this file system moving forward. It is important that you use the RH6 version of the compilers (listed below):
    • PGI: /uufs/chpc.utah.edu/sys/pkg/pgi/std_rh6
    • GNU: /usr/bin/gcc (4.4.7) OR /uufs/chpc.utah.edu/sys/pkg/gcc/4.7.2_rh6 (also gfortran)
    • INTEL: there is no change for the path to the intel compilers
  • In the/uufs/ember.arches/sys/pkg application tree, the std, std_intel, std_pgi links for MVAPICH2 and OPENMPI point to the RHEL6 versions
  • New ember specific builds of FFTW, BOOST, AMBER12, GROMACS can be found /uufs/ember.arches/sys/pkg; the old ones have permissions set so they are not accessible/executable. New ember builds of QE and LAMMPS are being worked on.
  • For GPU node use – we have updated CUDA to 5.5 (/usr/local/cuda)