This is a Service Level Agreement (SLA) from the Faculty of Environment for the Petabyte Environmental Tape Archive and Library (PETAL) service (services provided to the customer). A separate End User Agreement (terms agreed to by the customer) is also available.
2. DESCRIPTION OF SERVICE
The PETAL service is a Small Research Facility (SRF) managed by the Faculty of Environment (FoE) for the near-line and off-line storage and archival of research data. The service has a 10 year lifespan and is independent from the University of Leeds (UoL) repository used for storing publications for open access as required by grant funding bodies. Extension of the service beyond its lifespan will be decided dependent on its usage, running costs, available future funding and other relevant factors. The service will only be available to Faculty of Environment staff during the initial phase, and will then be rolled out to other grant holders within the UoL, dependent on available staff and capacity.
2.1. Service Classes
The service is comprised of an active archive: a combination of disk and tape that provides functionality similar to an on-line file system. Files are written to a disk cache initially before being moved to tape. The most recently accessed files have a copy retained in the cache for faster access (but the length of time that it remains in the cache is undefined).
The active archive file systems are network accessible to all University client systems, including Linux, Windows and Mac clients, in the same manner as normal file-systems, via kerberized NFSv4 and SMB protocols
2.2 Service Details
We offer two specific storage services, depending on the number of copies of the data that are required – either one or two copies of the data. Each person’s storage allocation will be associated with one of these services, dependent on the number of copies required.
Single copy on tape:
- Appropriate for non-critical data
- Data that can be recreated readily
- No Backup of this data will be provided
Two copies on tape:
- Appropriate for data that would be difficult to recreate (time consuming)
- Appropriate for data that is impossible to reproduce (unique value)
- Every tape will have a second copy synchronized twice a day
Each storage allocation will have a directory created on one of these areas where to store the data.
Please note that the single-copy on tape has no disaster recovery whatsoever: there is no possibility of retrieving the data in the event of it being affected by theft, fire, water ingress, tape failure, system failure, human error or other unforeseen circumstances.
2.3. Service Access
Access to the service is only via the SRF. There is a website available at http://petal.leeds.ac.uk that provides all required information for using the service. All requests are processed by the Petal System Administrator (SA) and forwarded to the appropriate people.
To gain access to the service, customers must complete a storage allocation form (via the website) for each project and agree to this Service Level Agreement and the End User Agreement. Once the storage allocation has been approved, an initial purchase of tapes will be required to provide the necessary storage space to deposit data. Storage space must be purchased by submitting a purchasing tapes form (via the website). The request will be authorised via the SA who will then increase the customer’s available storage capacity. The SA will interact with the Finance Office (FO) who will process and charge the transactions at the end of each month.
Customers cannot use tapes they have purchased themselves – all tapes must be purchased via the SRF.
3. END-USER EXPECTATIONS
The End User Agreement, must be read and electronically signed by the Principal Investigator (PI) before a storage allocation will be created.
Every group using the PETAL service must complete a storage allocation request form. This form names the PI who is financially and legally responsible for the data stored and any users who will be given either “read-only” or “read/write” access to the storage area.
For groups that constantly deposit data into their storage allocation, a minimum of one whole unused tape is required to be available at all times, in preparation for more data being deposited. Once there is less than one Terabyte of space left, warning emails will be sent to the PI responsible for the storage allocation, informing them that more tapes need to be purchased. For storage allocations that are a one-off deposit, the requirement of having an extra whole tape available will be waived.
If the group uses more space than has been purchased, a warning mail will be sent by the SA and the storage allocation area will be changed to “read-only” (restricting further writing) until further tapes are purchased. Groups that fail to purchase the backlog of required tapes, will have access to their storage allocation blocked until the required number of tapes are purchased.
Files should be in their final format before they are copied to the archive. The system is similar to a Write-Once-Read-Many (WORM) device – data cannot be deleted once written on tape. There is a ‘grace period’ of an hour since the last modification of the file. During this hour the system will allow the files to be deleted or modified. Once the period expires all files will be marked as read-only and deleting them will no more be possible. Only under exceptional circumstances will arrangements be made for deleting unnecessary files once written on tape. You will need to contact the SA to arrange for the removal of such files. This is a very time-consuming operation that should be avoided if possible. It is worthwhile checking that only the required files will be archived (before they are) as users must pay for all the file system space that they occupy, regardless of what is stored.
At the end of each day, any files that only exist in the disk cache will be copied onto tape. If a second copy has been requested, this copy will be also created by the end of the day. Every storage allocation will have a dedicated page on the website where users can verify that all of the files have been copied/exist on tape, before they start deleting the original copies of the data from their original location. If in doubt, please contact the SA before deleting any files.
3.2 User/Group Administration
The PI is the only person authorized to make changes to the set of users that comprise a storage allocation, to add or remove users on the access list, or to request other changes on behalf of the project. Change requests should be submitted by the PI using the send a message form (via the website). Without such approval, the SA will not follow on requests.
The PI should verify their list of users on a yearly basis. Note that while a person’s account may be terminated after a person leaves the University, his or her files will be retained within the PETAL service. This is done to ensure that shared files do not accidentally become unusable by a group when one user account becomes inactive.
The PETAL service relies on the UoL Active Directory infrastructure. All users must have a valid account and be UoL staff, students or affiliates.
The cost to the PETAL service end users is controlled through the SRF. End users pay actual costs for data storage. The cost is approximately £122 per Terabyte or approximately £306 per tape (2.5TB). There is a slight difference in price between charging for use of PETAL on grants that already exist, compared to listing the calculated indexed costs on a grant awaiting future approval. The costs associated with the SRF will be re-assessed each year by the FO and adjusted. The minimum purchase that a user can make is one tape. For further information on the charges, contact the Faculty of Environment Finance Office.
3.4 Ownership of Media
The tapes that data resides on belong to the SRF and may not be removed from the PETAL service. All research data generated is property of the UoL. Please contact the SA to copy the data onto another set of tapes for transport, or any other storage resources if necessary.
4. SERVICE EXPECTATIONS
4.1 Service Levels
The main service provided to end users is access to the disk cache via NFS and SMB exports. This is an automated system and the expectation is that it will run 24/7 with support during normal working hours (Mon-Fri 9am-5pm). There is no expectation of support outside of those hours and all requests for new storage areas or new tapes will only be dealt with during this time. Data in the system may be stored off-line and will require operator intervention before data can be accessed.
No hardware components in the system are replicated, and redundancy is only provided in specific areas (RAID arrays, power supplies).
While every reasonable and good faith effort will be made to ensure the reliability and availability of the PETAL service and of the data stored on it, access to data in the PETAL service may be affected by circumstances outside of our control.
- Response time for access to off-line data: tapes will be loaded into the system within one working day of the request being received.
- Response time for requests of new areas or tapes: will be fulfilled within one working day of the request being received.
- Response time for hardware failures: the system will be restored within two working days to allow hardware to be restored to working order by the external vendor. Hardware support from the external vendor is on a next business day basis. Resolution times may differ.
4.2 Accessing Storage
The file systems containing the archived data are potentially accessible from all UoL client machines which are capable of running kerberized NFSv4 and SMB.
For the Faculty of Environment, the disk cache can be accessed at the location below from CentOS 6 (or later) clients:
Local system administration work may be required on systems outside of the Faculty of Environment to allow NFS access – that work is outside the scope of this service.
SMB access from any University of Leeds Windows or Mac client is at:
Any data deposited into these file systems will eventually be copied onto tape, which will incur in the appropriate charge. It is the PI’s responsibility to ensure that all group users with write access understand this as the PI is financially responsible for all data deposited. Data wrongly deposited onto tape cannot be erased and the cost cannot be refunded.
4.3 Using the Service
Users should be aware that this is a near-line system utilizing tape and disk storage. We expect all normal commands to work to copy data into the cache – but performance won’t necessarily match on-line, disk-based systems. Users are expected to monitor their copy jobs to ensure they have successfully completed and take any remedial action necessary before deleting any data at source. Please contact the SA immediately if you experience any issue during the copy of the files to the archive.
4.4 Duration of Service
The hardware infrastructure supporting the PETAL service is funded through an SRF, with an initial 10 year plan. Operation of the PETAL service beyond that date may be contingent on sustainable use and further grants or University funding and cooperation.
In the event that FoE ceases to provide the PETAL service or any comparable resource, FoE will give at least 60 days advance notice. It will be the responsibility of the customer to transfer their data to other storage resources within that time window.
The PETAL website will contain information on the current usage and capacity of each project, updated weekly. It is the responsibility of the PI to monitor their usage and act accordingly. PIs will be informed when further storage is required to be purchased. PIs can also bulk load their storage allocation in preparation for a large future data deposit.
At least once a year, the SA will provide a list of users to all PIs showing the names of every user who has access to the storage allocation’s data.
Planned maintenance of the PETAL service infra-structure, including the hardware, will take place on the first Wednesday each month. The SA will broadcast the announcement via the service newsletter to which all the service users will be subscribed. The system should be considered at risk from 9am – noon on that day.
4.7 Change Management
The PETAL service will try to announce to all customers any major changes to the system a minimum of 30 days in advance. Exceptions may be critical updates and bug fixes that improve the stability of the system significantly.
As the service is shared by multiple users simultaneously, there will noticeable variations in the responsiveness of the components of the library.
The disk cache works on a FIFO (First In, First Out) principle, keeping copies of the most recently accessed files available on the disk. When files aren’t accessed, they age and become candidates for removal from the cache. If a file is accessed again, it moves back up to the top of the list and will remain in the cache until another file pushes it out of the cache.
Requests for known off-line data or large data sets should be initiated a few days before the data is required, to allow for the tapes to be retrieved, loaded and the data extracted and copied back onto disk. If the tape is on-line, it will be determined by the availability of a free tape drive and the location of the file on the tape.
4.9 User Training
Training sessions for groups can be organised with the SA. They will explain the proper use of the system and the responsibilities of the users in more detail. On-line documentation is also available describing various usage scenarios. Please use the form Send a message to arrange a meeting with the SA.