Archiving SLIMS data

Contents

 

Wget

The wget command line tool is usually available on Linux and Mac, and can be download for Windows.  Wget is the easy option to download the all the data of a project with a single command.
In SLIMS determine the URL of your data by following the “”All Files” link. The URL in the web browser will look similar this address.
http://slimsdata.genomecenter.ucdavis.edu/Data/i5om268pkp/Unaligned/
the bold part of the URL is referred to as RANDOM_DIR below,

Via the command line please change into your destination directory on your local machine, for example:

cd /data/archive/myrun

Then modify the following command to use your data directory URL mentioned abbove. Hit enter and wait for the download to finish.

wget -r -nH -nc -R index.html* http://slimsdata.genomecenter.ucdavis.edu/Data/RANDOM_DIR/Unaligned/

Rsync

If you are using an UNIX (Linux/Apple/Solaris) system, the best way to transfer your data is via rsync.

You change to your archive directory on your local machine, for example,

cd /data/archive/myrun
rsync -avL slimsdata.genomecenter.ucdavis.edu::slims/RANDOM_DIR .

The random_string above can be found in your SLIMS (http://slims.bioinformatics.ucdavis.edu) web interface, for the run you want to download

View->View Files

at the “View Files” screen, the address bar contains the random string name similar to the following URL

http://slims.bioinformatics.ucdavis.edu/Data/79856d9a09/

where 79856d9a09 is the random string that you need to download your files.

You may repeat the above command again to make sure that you have downloaded all your files, rsync will download missing files if the previous session was interrupted, it will report 0 download file if everything on the local machine is in sync with the remote server.

 

Windows 10 and 11

Windows users can now easily install Unix terminals terminals, which provide access to wget & rsync from the command line within Windows:
The easiest option is to install the small and free MobaXTerm SSH client from mobaxterm.mobatek.net
Microsoft supports the easy and free install of an almost complete UNIX that can also be used from within Windows: docs.microsoft.com/en-us/windows/wsl/install

MobaXTerm contains a complete UNIX shell. As such there are multiple ways to download the data. We would suggest these commands for beginners wanting to download sequencing data from SLIMS:

# Create a new download directory in the windows file system (here named “seqdata”). The new directory will be located on the windows “C:” drive.

mkdir /drives/c/seqdata      

# Move your console into this new directory:

cd /drives/c/seqdata    

# Start the download within this new directory by entering the WGET command as described above on this page; for example:

wget -r -nH -nc -R index.html* http://slimsdata.genomecenter.ucdavis.edu/Data/cdn6e35XYZ/Unaligned/

# The download will start and the progress for each file will be displayed on the console. Depending on the size of the data, the download might take from 10 minutes to several hours. Do not close your PC until it has finished. The downloaded files will be located in a “Data” subdirectory, likely within further subdirectories.