Tags

UNIX Home Disk

Please see DataStewardship for general storage recommendations at CLASSE.

Home disk quota

There is a 1GB Quota on the /home disk allocated to each CLASSE user. This is the same directory visible from Windows as \\samba.classe.cornell.edu\home\yourCLASSEuserid. The 1GB quota is implemented for two main reasons:
  1. If a rogue process runs away and fills up or overloads the disk, all users will be severely impacted. Many jobs will fail and/or general system responsiveness for all users will be severely degraded.
  2. Since the disk is backed up daily, excessive volume of files means excessive overhead. I.e. poor performance for the duration of the backup as well as increased expense in backup media.

Alternative storage locations

When choosing where to store your data, in general there are three types of disk space available.
  1. The /home disk is for essential Linux files like login scripts, browser preferences, source code, scripts, etc. It will be backed up daily and historical records will be kept.
  2. Several styles of temporary disk space are available. These are intended for files that can easily be reproduced, like libraries and executables, and for working space. They are not backed up. Available to all are spaces like /cdat/tem and /nfs/acc/temp. Other temporary spaces are primarily designed for a particular project like /nfs/ilc/sim1, 2, and 3 for CesrTA simulations, and /nfs/erl/sim for ERL users.
  3. Finally, there are several user, group, and project specific file systems. Some of these are backed up daily, and most do not implement quotas. For example, all users have a 5GB UserDisk (please see https://wiki.classe.cornell.edu/Computing/UserDisk). In addition, many projects or groups have dedicated file systems for vital, but voluminous project or group specific data, like the final results of data analyses or simulations. Because this type of usage could easily run away, filling all available space, we only allow users with specific needs to access these disks. For example, accelerator users can be given access to the -nfs/acc/user= disk. If you feel you need space on /nfs/acc/user or some other project or group-specific file system, contact your supervisor who will authorize your usage to the computer group. For the time being, no quotas are in place on most of these file systems, though we may be forced to implement generous quotas if the disk fills up too often or if backups become excessively burdensome.

Mozilla (Firefox) cache usage under Linux

In some cases, you may find that your Mozilla profiles are using a large amount of disk space. To clear some of this space and prevent them from using up more than necessary, please see FirefoxBrowser and TbirdProfiles (https://wiki.classe.cornell.edu/Computing/FirefoxBrowser and https://wiki.classe.cornell.edu/Computing/TbirdProfiles).

Evaluating usage

From Linux

Your /home disk usage can be analyzed using baobab (graphical) or du (text-based). For example:
  • baobab ~/
  • du -hcsx ~/* ~/.* --exclude='.' --exclude='..'
  • du -x ~/ | sort -nr | head

From Windows

  • Open a Windows Exporer window to \\samba.classe.cornell.edu\home
  • Right-mouse-button click on the folder named with your CLASSE userid
  • In the new popup window, select the Properties entry
  • Wait for a little while while it measures the contents of your home directory

Summary

We ask that you make use of temporary disk space and user, group, or project disk space, and keep your /home disk usage below 1 GByte. Any personal, group, or project data should be stored in user, project, or group file systems instead of the home filesystem. Please see DataStewardship (https://wiki.classe.cornell.edu/Computing/DataStewardship) for more information.

Redirecting scratch space from CLASSE home directories

In an effort to avoid filling your Linux home disk quota with scratch files, Linux login scripts move scratch space out of your home disk. In addition, this helps save CLASSE resources and avoid unnecessarily backing up scratch space.

If you've already moved these directories, they won't be moved again. However if they aren't already links, ~/.mozilla, ~/.thunderbird, ~/Downloads, and ~/.cache will be moved as follows :
  • ~/.mozilla/ and ~/.thunderbird/ to /nfs/user/$USER/
  • ~/Downloads/ and ~/.cache/ to /cdat/tem/$USER/

These changes will take affect when you login to a graphical desktop, and should be transparent to new and existing sessions.
Topic revision: r12 - 18 Oct 2018, AdminDevinBougie
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding CLASSE Wiki? Send feedback