Hprc banner tamu.png

Difference between revisions of "SW:rclone"

From TAMU HPRC
Jump to: navigation, search
(Dropbox =)
(Configure rclone for Google Drive)
Line 2: Line 2:
 
rclone is a tool for syncing files from HPRC clusters to a remote storage sites like Google Drive, Dropbox, Amazon's AWS and many more.  Note rclone is available on ada, terra and HPRC Lab workstations... no module required for any of them.  If you are using this from ada or terra, you probably want to use the fast transfer nodes (ftns) (i.e. ada-ftn1.tamu.edu, ada-ftn2.tamu.edu and terra-ftn.hprc.tamu.edu [note the 'hprc' in the last one]).
 
rclone is a tool for syncing files from HPRC clusters to a remote storage sites like Google Drive, Dropbox, Amazon's AWS and many more.  Note rclone is available on ada, terra and HPRC Lab workstations... no module required for any of them.  If you are using this from ada or terra, you probably want to use the fast transfer nodes (ftns) (i.e. ada-ftn1.tamu.edu, ada-ftn2.tamu.edu and terra-ftn.hprc.tamu.edu [note the 'hprc' in the last one]).
 
   
 
   
=== Configure rclone for Google Drive ===
+
=== Configure rclone for your Google Drive ===
 
* configure rclone for your destination
 
* configure rclone for your destination
 
<pre>
 
<pre>

Revision as of 10:52, 17 August 2018

rclone

rclone is a tool for syncing files from HPRC clusters to a remote storage sites like Google Drive, Dropbox, Amazon's AWS and many more. Note rclone is available on ada, terra and HPRC Lab workstations... no module required for any of them. If you are using this from ada or terra, you probably want to use the fast transfer nodes (ftns) (i.e. ada-ftn1.tamu.edu, ada-ftn2.tamu.edu and terra-ftn.hprc.tamu.edu [note the 'hprc' in the last one]).

Configure rclone for your Google Drive

  • configure rclone for your destination
rclone config
  1. choose: n) New remote
  2. pick a name for the drive. e.g. mygdrive
  3. choose the type, in this example we use 11) Google Drive
  4. select the default (blank) option for the following
  5. leave Google Application Client Id blank (default)
  6. leave Google Application Client Secret blank (default)
  7. choose: 1) Full access to all files
  8. leave root_folder_id blank (default)
  9. leave service_account_file blank (default)
  10. when it ask if you want to auto_config, answer 'n' or 'N' for no
  11. at this point, rclone will provide a URL for you to log into in order to get an access code. Open the URL in your local browser and login (in this case to Google). If successful, you will get an access code. Copy/paste the code to the rclone prompt: Enter verification code>
  12. this example is for a non-team drive so answer 'n' to the 'Configure this as a team drive?' prompt
  13. review settings and choose 'y' to save
  14. choose 'q' to exit rclone config
  • These settings will be stored in $HOME/.config/rclone/rclone.conf and can be transferred between systems as needed if you don't want to run 'rclone configure' again.

Use rclone with your Google Drive

  • see the rclone man page and --help screen for other commands
  • here's a simple backup of one's $HOME directory (from terra)
rclone copy $HOME mygdrive:home-terra
  • in your local browser watch as new files are created on your remote storage

[more to come]

Team Google Drives

Other services

Amazon AWS S3

Dropbox

more information