Difference between revisions of "Frontenac:MigrateOff"

From CAC Wiki
Jump to: navigation, search
(Migration Q&A)
(Migration Time Table)
Line 23: Line 23:
 
== Migration Time Table ==
 
== Migration Time Table ==
  
{| class="wikitable" | '''Difference between "old" SW (Linux) and "new" CAC (Frontenac) clusters'''
+
{| class="wikitable" | '''Migration timeline for non-paying users'''
|-
+
|'''Month (2017)'''
+
|'''Who moves ?'''
+
 
|-
 
|-
 
| December 2018 - March 2019
 
| December 2018 - March 2019
Line 39: Line 36:
 
|  
 
|  
 
* Access changes from free to charged
 
* Access changes from free to charged
* Charged accounts are active with priorities based on charge
+
* Paying users continue to access systems as before
* Compute Canada free accounts (hpcXXXX) loose access
+
* All jobs of non-paying users are terminated
* Data not covered by charged accounts are purged
+
* non-paying users from outside Ontario:
* Some exceptions apply :
+
** access stops
** grace period for residual data of RAC users, limited access
+
** data are subject to purge
** Queen's researchers & other Ontario research groups get additional time to move their data off system
+
* Ontario non-paying users (including Queen's, CAC consortium, SNO, OBI, etc):
 +
** access for data management still possible
 +
** data are migrated to tape (nearline)
 +
** no job submission
 +
** no access to compute cluster
 
|-
 
|-
 
| July 1, 2019
 
| July 1, 2019

Revision as of 13:54, 1 February 2019

Outline

Frontenac serves as our main compute cluster and is operated through the SLURM scheduler. Until March 31, allocations from the 2018 Resource Allocation Competition of Compute Canada, are running on this cluster. Furthermore, the cluster is accessed by researchers with a "contributed" priority allocation, and on an "opportunistic" scheduling basis with low priority.

Since the cluster will not be among the allocatable system for the 2019 Compute Canada allocation round ("RAC2019"), the operation of Frontenac will be on a cost-recovery basis from April 1, 2019 onward. For details about the fee structure, please see our Frontenac Fee Wiki page for details. This is an important change that affects both compute access and the usage of storage.

From April 2019 we will cannot provide compute services and/or storage capacity on Frontenac free of charge.

If you are currently using Frontenac for computations, and are looking for another system that offers cycles free of charge, please consult the available resources at Compute Canada. Allocations of larger resources require an application, but "opportunistic" use (Rapid Access Service or RAS) is available also.

Migrating off the Frontenac cluster

Note: Any data on the Frontenac file system must be moved off the system by April 2019. Retention of data beyond that time requires an arrangement involving fees. Please consult our fees guide for details. The Arrangement has to be made before the deadline of April 1, 2019 to avoid data purges.

For technical information of how to upload/download data from our system, see [[|]]

Note that the responsibility of data migration lies with the user. We do not control access to the target systems and cannot do the data transfer for you. We will assist you with technical issues but you have to arrange for the disk space on the target system.

Why ?

Unfortunately, the only way we can continue to supply the disk space is by charging a fee to cover maintenance and operating costs. We do so on a cost recovery basis, i.e. we charge as much as we must, and as little as we can.

Migration Time Table

December 2018 - March 2019
  • Monthly reminders about the need to migrate data
  • RAC access, general "opportunistic" access continues
  • Both access types are free
  • Storage is free
  • Users are asked to make arrangements for charged access
April 1, 2019
  • Access changes from free to charged
  • Paying users continue to access systems as before
  • All jobs of non-paying users are terminated
  • non-paying users from outside Ontario:
    • access stops
    • data are subject to purge
  • Ontario non-paying users (including Queen's, CAC consortium, SNO, OBI, etc):
    • access for data management still possible
    • data are migrated to tape (nearline)
    • no job submission
    • no access to compute cluster
July 1, 2019
  • All data not covered by charged accounts are purged.
  • This includes backups (tape).

We will transfer hardware from the "old" cluster (SW) to the new one (Frontenac) to accommodate the migrated users. This means that in the transition period, the old cluster will gradually become smaller while the new one grows. Dedicated hardware will be moved when its users migrate.

IMPORTANT DEADLINES

Date Migration Event
April 1, 2019
  • Access to Frontenac from free accounts ends
  • Limited access for RAC users for retrieval of new data
  • Limited access for Queen's / Ontario users for data retrieval
  • Data for closed hpcXXXX accounts are purged (non-Ontario, non-RAC)
July 1, 2017
  • All data associated with free accounts are purged
  • No access from free accounts

Migration Q&A

  • Q: Who migrates ?
A: All users who currently hold an hpcXXXX (Compute Canada) free account, and do not want a charged account.
  • Q: Where can I continue to use resources for free ?
A: Compute Canada systems offer free resources based on allocations from a competition (RAC), or to a limited extend on "opportunistic" usage (RAC).
  • Q: Can I run my programs somewhere else?
A: Most execution scripts, pipelines, etc will still work on another system, as Frontenac is versy similar to other Compute Canada systems.
  • Q: Do I copy my files over myself ?
A: Yes. We cannot do the data transfer for you. We will assist you with technical issues, but we cannot arrange for space on other systems.
  • Q: Is this optional ?
A: No. The move must be done by April 1, 2019, as we have to cover our operating and maintenance costs.
  • Q: Can I decide when to move ?
A: Yes, as long as the move is completed by April 1.
  • Q: Will this disrupt my research ?
A: We hope that you continue your research on our systems and pay the associated fee. In that case things will be continuing without any disruption. If you decide to move to another system that offers free access, you should plan the data migration as early as possible to smooth over any issues that the migration entails. We will do our best to assist you with this, but we cannot do it for you.
  • Q: I have a RAC 2018. Does this apply to me ?
A: Yes. The bulk of data has to be moved by April 2019. We may be able to help make arrangements prior to the onset of the 2019 allocation for temporary data storage. Contact us.
  • Q: I have contributed hardware to CAC. Does this apply to me ?
A: Contributed accounts were issued on the basis of a prior agreement and will not be subject to this as long as the agreement continues.

Help

If you have questions that you can't resolve by checking documentation, email to cac.help@queensu.ca.