Skip to content

Storage

There are multiple storage systems available for CEOAS researchers. Some of them are shared with and managed by other colleges and departments, while others are managed by and solely for the college to use. Historically storage on the HPC cluster was located at /, but new storage systems are intended to be added at /storage to reduce the amount of clutter in /.

Each storage is listed here with its path followed by a short description. The list is sorted in roughly the order of widely used to narrowly used. For CEOAS storage, questions and requests can be directed to CEOAS support via support@ceoas.oregonstate.edu. Other storage systems will have their contact information noted.

/ceoas - General storage

The general CEOAS storage is located at /ceoas. Every PI can request an allocation of storage here, either for themselves or their lab. While we do not enforce strict limitations under the idea that unused storage is wasted storage, we do request that PIs and lab keep their total usage at or below approximately 10TB. If we find that certain groups are using far in excess of this and the cluster is getting tight on storage, we will contact them to request they reduce how much they're storing in /ceoas.

Storage here is not backed-up, however it does have seven daily snapshots and two weekly snapshots in case of accidental modification or deletion of files. Be aware that snapshots are not a guarantee and they may be pre-emptively deleted if users are rapidly turning over data such that the snapshots are using an excessive amount of storage.

For users of non-HPC CEOAS servers, this storage can also be mounted on them by request at /home/server/hpc. Certain users find this useful for use cases such as doing model runs on the HPC and analysis on their own dedicated servers, or for making output from HPC jobs available on a web frontend.

Additionally, CEOAS users can request that a given directory be made accessible via Globus or FTP for large data transfers. Please contact CEOAS support for more information on these options.

/home - Home directories

Home directories are located at /home on all HPC servers. /home itself is a symlink to (as of this writing) /fs2/home. It's highly recommended to use /home instead of /fs2/home because the latter path can be subject to change.

Home directories have a limit of 75GB. Because quotas are not calculated in realtime, occasionally users can accidentally burst beyond that limit before the restrictions kick in. Users above the quota will be blocked from writing new files to their home directory. Oftentimes the immediate symptom is Slurm jobs immediately failing due to not being able to open files for writing log output in the user's home directory.

Because every user constantly accesses their home directory and is affected by excessive load on the home storage, users should not run anything more intensive than minimal processing out of their home directory. Users found to be performing excessive IO in their home directories will have their jobs unceremoniously terminated and contacted afterwards.

Management and provisioning of home directories is handled by the CQLS. Questions and requests regarding them can be directed to CQLS support at https://docs.hpc.oregonstate.edu/support/. While CEOAS support can generally answer questions and provide some assistance, the CQLS support staff handles all the administrative tasks.

/storage/ceoas-scratch - Fast storage

The scratch storage at /storage/ceoas-scratch is available to all CEOAS researchers for general use. It's backed by a 330TB RAID10 array of SSDs generously donated by Solidigm. As long as sufficient spare capacity remains and users are careful to clean up after themselves, we hope to avoid needing to implement any strict usage or automatic deletion policies.

/storage/ceoas-datasets - Shared datasets

The shared dataset storage at /storage/ceoas-datasets is intended to provide quick and easy access to portions of datasets commonly used across the college, such as ERA and CMIP. By having a shared repository, it reduces the amount of duplication of these datasets across other storage system, provides dedicated performance for accessing them, and allows researchers to quickly and easily access them without having to download their own copy first. Due to the size of the native datasets, the versions hosted here are only a subset of them, with shorter time frames, lower resolutions/frequencies, and/or a partial selection of variables.

For information on the datasets, please consult the README files located in their directories. Questions about datasets should be directed to their individual maintainer, as CEOAS support provides the technical infrastructure to host them but not the expertise to use them. Requests to add additional datasets can be directed to CEOAS support, but please keep in mind that our priority is storing datasets that are likely to be widely used across the college instead of only by individual researchers.

/nfs* - CQLS storage

The /nfs storage systems are the storage systems for the CQLS. Occasionally these will be used by CEOAS researchers who have either purchased their own storage from the CQLS or are collaborating with HPC users in other colleges who have done so. CEOAS researchers who have purchased storage can find their allocations at /nfs7/CEOAS. Any questions or requests regarding this storage should be directed to CQLS support.

/local - HPC software

Generally accessible software is stored in /local. Software used by the entire cluster is located at /local/cluster and managed by the CQLS. Software specifically built for CEOAS researchs is located at /local/ceoas and managed by CEOAS support. Requests to install software for use by CEOAS researchers should be sent to CEOAS support. We generally try to use Spack to provide software instead of standalone binaries and libraries; information on how to use the CEOAS-managed Spack install can be found at https://docs.hpc.oregonstate.edu/ceoas/software/.