Backup/Restore

Migration of the data between database instances can be facilitated by using a pair of commands: backup and restore.

The command backup creates a backup copy of the data in the database in the form of a large SQL script. In addition to the SQL, the archives with genomic sequences (sequence.tar), files with binary data (file_storage.tar) and user data added through the Persephone client (user_storage.tar) are created.

PS> backup
Absolute path to the storage for user data loaded via Persephone (drag&drop)? /data/WebPersephone/Users
Absolute path to the folder for backups (writable)? /data/shared/backup
Database can be backed up
The following data will be backed up:
Backup type Input path Output path Approximate size
-----------------------------------------------------------------------------------------------------------------------------
Backup folder path /data/backup/Persephone_Backup_2024-03-11
Database /data/backup/Persephone_Backup_2024-03-11/backup.sql 51.47 GB
Sequence storage /data/sequences /data/backup/Persephone_Backup_2024-03-11/sequence.tar 42.66 GB
File storage /data/FileStorage /data/backup/Persephone_Backup_2024-03-11/file_storage.tar 12.16 GB
User storage /data/WebPersephone/Users /data/backup/Persephone_Backup_2024-03-11/user_storage.tar 2.02 KB

Proceed? (Y/N)

The backup files will be stored in a sub-folder of the directory specified during the process. If needed, transfer the root backup folder (e.g., /data/shared/backup) with the sub-folders to a new location. In case of Docker, it is quite common to use a mounted folder as the destination of the backup, so that the files could be accessed from outside Docker container and copied to a new location.

The directory for the user files added though the Persephone client is not known to PersephoneShell, it is specified in the configuration of the Persephone application, so, its location should be manually entered at the prompt Absolute path to the user storage. For our standard Docker setup, the user files are stored at /data/WebPersephone/Users.

Create a new empty database or use the one that is coming with the Persephone Docker image. Start PersephoneShell on a new machine and execute the command restore. Note that the data in the docker image contains one sample map set (Arabidopsis thaliana/TAIR10). The restore command will erase this map set. When asked about the directory for storing the genomic sequences, it is OK to specify the same folder that was in use by the previous copy of the data. We are creating a new data set, so the old data, once the restore process is over, can be deleted from the database and from the file system.

PS> restore -v
Schema is not empty (1 map set(s) found). If you proceed, all the data will be lost.
Do you want to delete the data? (Y/N) Y
Absolute path to the folder which contains the backup? /shared/backup
Looking for backups in the given folder...

[##] FOLDER_NAME BACKUP_SPENT_TIME BACKUP_DATE TOTAL_FILE_SIZE_MB STATUS
----------------------------------------------------------------------------------------------------
[0] Persephone_Backup_2024-03-11 (1) 00:00:25 2024-03-11 1.17 GB OK

Select [lineNo] corresponding to the backup folder: 0
Absolute path to the user storage? /persephone/WebPersephone/Users
Restored sizes:
TYPE FILE_SIZE_MB
--------------------------------
DB 110.38 MB
Sequence storage 983.76 MB
File storage 10.00 KB
User storage 106.18 MB

Sequence folder (/persephone/sequences) is not empty. If you proceed, all the data will be lost.
Do you want to proceed? (Y/N) Y
User storage folder (/persephone/WebPersephone/Users) is not empty. If you proceed, all the data will be lost.
Do you want to proceed? (Y/N) Y
Do you want to rebuild SOLR after restoring backup? (Otherwise, run it manually) (Y/N) Y
Database can be restored
Restore process Result Approximate size
-----------------------------------------------------------------------------------------
All data will be deleted
Backup folder path /shared/backup/Persephone_Backup_2024-03-11
Database 110.38 MB
Sequence storage /persephone/sequences 983.76 MB
File storage /persephone/FileStorage 10.00 KB
User storage /persephone/WebPersephone/Users 106.18 MB

Do you want to restore this backup? (Y/N) Y

Some of the values required for the restore process, such as the location of file storage, are predefined in the PersephoneShell configuration file in the section <ConnectionSettings>: