{"type":"doc","content":[{"type":"extension","attrs":{"layout":"default","extensionType":"com.atlassian.confluence.macro.core","extensionKey":"toc","parameters":{"macroParams":{},"macroMetadata":{"macroId":{"value":"61dfffec-332a-4896-bdcb-f615832152ce"},"schemaVersion":{"value":"1"},"title":"Table of Contents"}}}},{"type":"paragraph","content":[{"text":"Dropbox can be accessed for browsing and data transfers from Talapas login nodes using the process described below. The following steps configures an rclone remote connection to your Dropbox account.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Create UO Dropbox account","type":"text"}]},{"type":"paragraph","content":[{"text":"If you do not already have a UO affiliated Dropbox account, follow the instructions in the IS Knowledge Base article to setup your Dropbox account.","type":"text"}]},{"type":"paragraph","content":[{"text":"https://service.uoregon.edu/TDClient/2030/Portal/KB/ArticleDet?ID=102716","type":"text","marks":[{"type":"link","attrs":{"href":"https://service.uoregon.edu/TDClient/2030/Portal/KB/ArticleDet?ID=102716"}}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Setup a Dropbox app","type":"text"}]},{"type":"paragraph","content":[{"text":"Follow ","type":"text"},{"text":"only","type":"text","marks":[{"type":"em"}]},{"text":" the instructions in the section “Get your own Dropbox App ID” in the IS Knowledge Base article to create a Dropbox app.","type":"text"}]},{"type":"paragraph","content":[{"text":"https://service.uoregon.edu/TDClient/2030/Portal/KB/ArticleDet?ID=111417","type":"text","marks":[{"type":"link","attrs":{"href":"https://service.uoregon.edu/TDClient/2030/Portal/KB/ArticleDet?ID=111417"}}]}]},{"type":"paragraph","content":[{"text":"Rclone will use your Dropbox app id credentials to authenticate to your Dropbox account when configuring the remote connection. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Ensure your Dropbox app has the permissions below. This can be edited under the Permissions tab of your Dropbox App.","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"‘files.content.read’","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"‘files.content.write’","type":"text"}]}]}]},{"type":"paragraph"},{"type":"paragraph","content":[{"text":"The example below configures an rclone remote location named ","type":"text"},{"text":"dbx-remote","type":"text","marks":[{"type":"annotation","attrs":{"annotationType":"inlineComment","id":"d4b2e56f-76f9-4c89-9952-f4921024df05"}}]},{"text":". Your rclone remote name can be anything (descriptive is probably better). It will be used as a command line argument to rclone and does not need to be the same as your Dropbox app name. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Substitute and for your own Dropbox app key and secret.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Run ‘rclone config’ on a login node","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ module load rclone/1.52.2\n$ rclone config\n2020/08/05 12:01:58 NOTICE: Config file \"~/.config/rclone/rclone.conf\" not found - using defaults\nNo remotes found - make a new one\nn) New remote\ns) Set configuration password\nq) Quit config\nn/s/q> n\nname> dbx-remote\nType of storage to configure.\nEnter a string value. Press Enter for the default (\"\").\nChoose a number from below, or type in your own value\n 1 / 1Fichier\n \\ \"fichier\"\n 2 / Alias for an existing remote\n \\ \"alias\"\n 3 / Amazon Drive\n \\ \"amazon cloud drive\"\n 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)\n \\ \"s3\"\n 5 / Backblaze B2\n \\ \"b2\"\n 6 / Box\n \\ \"box\"\n 7 / Cache a remote\n \\ \"cache\"\n 8 / Citrix Sharefile\n \\ \"sharefile\"\n 9 / Dropbox\n \\ \"dropbox\"\n...\nStorage> 9\n** See help for dropbox backend at: https://rclone.org/dropbox/ **\n\nDropbox App Client Id\nLeave blank normally.\nEnter a string value. Press Enter for the default (\"\").\nclient_id> \nDropbox App Client Secret\nLeave blank normally.\nEnter a string value. Press Enter for the default (\"\").\nclient_secret> \nEdit advanced config? (y/n)\ny) Yes\nn) No (default)\ny/n> n\nRemote config\nMake sure your Redirect URL is set to \"http://localhost:53682/\" in your custom config.\nUse auto config?\n * Say Y if not sure\n * Say N if you are working on a remote or headless machine\ny) Yes (default)\nn) No\ny/n> n\nFor this to work, you will need rclone available on a machine that has\na web browser available.\n\nFor more help and alternate methods see: https://rclone.org/remote_setup/\n\nExecute the following on the machine with the web browser (same rclone\nversion recommended):\n\n\trclone authorize \"dropbox\" -- \"\" \"\"\n\nThen paste the result below:\nresult> ","type":"text"}]},{"type":"paragraph","content":[{"text":"Copy the rclone authorize line.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Run ‘rclone authorize’ on your local machine","type":"text"}]},{"type":"paragraph","content":[{"text":"Running this step on your local machine is required since Talapas login nodes do not run web browsers. ","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Install rclone 1.52.2 on your local machine if not already installed ","type":"text"}]},{"type":"paragraph","content":[{"text":"For example, to install on Linux or Mac run:","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ curl https://rclone.org/install.sh | sudo bash\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n100 4436 100 4436 0 0 5783 0 --:--:-- --:--:-- --:--:-- 5776Password:\n\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n100 15 100 15 0 0 10 0 0:00:01 0:00:01 --:--:-- 10\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n100 16.1M 100 16.1M 0 0 2267k 0 0:00:07 0:00:07 --:--:-- 3028k\nArchive: rclone-current-osx-amd64.zip\n creating: tmp_unzip_dir_for_rclone/rclone-v1.52.2-osx-amd64/\n extracting: tmp_unzip_dir_for_rclone/rclone-v1.52.2-osx-amd64/git-log.txt [empty]\n inflating: tmp_unzip_dir_for_rclone/rclone-v1.52.2-osx-amd64/rclone [binary]\n inflating: tmp_unzip_dir_for_rclone/rclone-v1.52.2-osx-amd64/rclone.1 [text]\n inflating: tmp_unzip_dir_for_rclone/rclone-v1.52.2-osx-amd64/README.html [text]\n inflating: tmp_unzip_dir_for_rclone/rclone-v1.52.2-osx-amd64/README.txt [text]\n\nrclone v1.52.2 has successfully installed.\nNow run \"rclone config\" for setup. Check https://rclone.org/docs/ for more details.\n\n$ rclone --version\nrclone v1.52.2","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Run rclone authorize on your local machine","type":"text"}]},{"type":"paragraph","content":[{"text":"On your local machine, paste the rclone authorize line you copied from the rclone config step on login node. ","type":"text"}]},{"type":"paragraph","content":[{"text":"This will trigger your browser to open a tab requesting approval for rclone to connect to Dropbox, click ‘Allow’. Copy the code provided in the output of rclone authorize and return to the Talapas login node.","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ rclone authorize \"dropbox\" -- \"\" \"\"\n2020/08/05 12:09:50 NOTICE: Config file \"~/.config/rclone/rclone.conf\" not found - using defaults\nMake sure your Redirect URL is set to \"http://localhost:53682/\" in your custom config.\nIf your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=redacted\nLog in and authorize rclone for access\nWaiting for code...\nGot code\nPaste the following into your remote machine --->\n{\"access_token\":\"redacted\",\"token_type\":\"bearer\",\"expiry\":\"2020-08-05T16:10:21.28527-07:00\"}\n<---End paste ","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Finish ‘rclone config’ on the login node","type":"text"}]},{"type":"paragraph","content":[{"text":"On the login node, paste the code in the result field.","type":"text"}]},{"type":"codeBlock","content":[{"text":"...\nThen paste the result below:\nresult> {\"access_token\":\"redacted\",\"token_type\":\"bearer\",\"expiry\":\"2020-08-05T16:10:21.28527-07:00\"}\n--------------------\n[dbx-remote]\ntype = dropbox\nclient_id = \nclient_secret = \ntoken = {\"access_token\":\"redacted\",\"token_type\":\"bearer\",\"expiry\":\"2020-08-05T16:10:21.28527-07:00\"}\n--------------------\ny) Yes this is OK (default)\ne) Edit this remote\nd) Delete this remote\ny/e/d> y\nCurrent remotes:\n\nName Type\n==== ====\ndbx-remote dropbox\n\ne) Edit existing remote\nn) New remote\nd) Delete remote\nr) Rename remote\nc) Copy remote\ns) Set configuration password\nq) Quit config\ne/n/d/r/c/s/q> q\n","type":"text"}]},{"type":"paragraph","content":[{"text":"The rclone remote configuration created is saved in ","type":"text"},{"text":"~/.config/rclone/rclone.conf","type":"text","marks":[{"type":"code"}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Run rclone","type":"text"}]},{"type":"paragraph","content":[{"text":"Documentation on rclone commands is available: ","type":"text"},{"text":"https://rclone.org/commands/","type":"text","marks":[{"type":"link","attrs":{"href":"https://rclone.org/commands/"}}]}]},{"type":"paragraph","content":[{"text":"List contents of your Dropbox","type":"text","marks":[{"type":"annotation","attrs":{"annotationType":"inlineComment","id":"647c72c5-953f-4f63-85d6-28c623f3f76b"}}]}]},{"type":"codeBlock","content":[{"text":"$ rclone lsd dbx-remote:\n -1 2020-08-06 10:13:47 -1 myFirstFolder\n$ rclone ls dbx-remote:\n 240 Get Started with Dropbox Paper.url\n 1102331 Get Started with Dropbox.pdf\n 11693 myFirstFolder/perfsonar-pschedule tests.rtf","type":"text"}]},{"type":"paragraph","content":[{"text":"Copy files","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ rclone copy --help","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"If you are familiar with ","type":"text"},{"text":"rsync","type":"text","marks":[{"type":"code"}]},{"text":", rclone always works as if you had written a trailing / - meaning \"copy the contents of this directory\". This applies to all commands and whether you are talking about the source or destination.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"Copy the contents of a folder from Dropbox to Talapas in your current working directory. In this example the folder is ‘MyFirstFolder’ and contains the file ‘perfsonar-pscheduletests.rft’","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ rclone copy dbx-remote:myFirstFolder .\n$ ls -l perfsonar-pscheduletests.rtf\n-rw-r--r-- 1 user group 11693 Jul 22 16:01 perfsonar-pscheduletests.rtf","type":"text"}]},{"type":"paragraph","content":[{"text":"Copy the contents of all Dropbox to Talapas","type":"text","marks":[{"type":"annotation","attrs":{"annotationType":"inlineComment","id":"5b929a37-c4f8-4fb4-adc4-bc2bd87fcb92"}}]}]},{"type":"codeBlock","content":[{"text":"$ rclone copy dbx-remote: /projects/pirg/shared","type":"text"}]},{"type":"paragraph","content":[{"text":"Copy a file from Talapas to Dropbox","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ rclone copy ./hello.txt dbx-remote:myFirstFolder","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"If you use the rclone sync command always run with --dry-run flag first! This command has the potential to delete data.","type":"text"}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Re-authenticate rclone","type":"text"}]},{"type":"paragraph","content":[{"text":"Rclone authentication to Dropbox expires after 4 hours and to continue you will need to reconnect. The process is similar to above as you’ll need to run rclone authorize on your local machine.","type":"text"}]},{"type":"paragraph","content":[{"text":"On the login node, note the expiration error below. Run rclone config reconnect.","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ rclone lsd dbx-remote:\n2020/08/06 10:11:36 ERROR : : error listing: Post \"https://api.dropboxapi.com/2/files/list_folder\": couldn't fetch token - maybe it has expired? - refresh with \"rclone config reconnect dbx-remote:\": oauth2: token expired and refresh token is not set\n2020/08/06 10:11:36 Failed to lsd with 2 errors: last error was: Post \"https://api.dropboxapi.com/2/files/list_folder\": couldn't fetch token - maybe it has expired? - refresh with \"rclone config reconnect dbx-remote:\": oauth2: token expired and refresh token is not set\n$ \n$ rclone config reconnect dbx-remote:\nAlready have a token - refresh?\ny) Yes (default)\nn) No\ny/n> y\nMake sure your Redirect URL is set to \"http://localhost:53682/\" in your custom config.\nUse auto config?\n * Say Y if not sure\n * Say N if you are working on a remote or headless machine\ny) Yes (default)\nn) No\ny/n> n\nFor this to work, you will need rclone available on a machine that has\na web browser available.\n\nFor more help and alternate methods see: https://rclone.org/remote_setup/\n\nExecute the following on the machine with the web browser (same rclone\nversion recommended):\n\n\trclone authorize \"dropbox\" -- \"\" \"\"\n\nThen paste the result below:\nresult>","type":"text"}]},{"type":"paragraph","content":[{"text":"On your local machine, run rclone authorize, click ‘Allow’ in your web browser to approve the connection and copy the resulting code.","type":"text"}]},{"type":"codeBlock","content":[{"text":"$ rclone authorize \"dropbox\" -- \"\" \"\"\nMake sure your Redirect URL is set to \"http://localhost:53682/\" in your custom config.\nIf your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=redacted\nLog in and authorize rclone for access\nWaiting for code...\nGot code\nPaste the following into your remote machine --->\n{\"access_token\":\"redacted\",\"token_type\":\"bearer\",\"expiry\":\"2020-08-06T14:13:44.524022-07:00\"}\n<---End paste","type":"text"}]},{"type":"paragraph","content":[{"text":"On the login node, paste the code and run rclone.","type":"text"}]},{"type":"codeBlock","content":[{"text":"...\nThen paste the result below:\nresult> {\"access_token\":\"redacted\",\"token_type\":\"bearer\",\"expiry\":\"2020-08-06T14:13:44.524022-07:00\"}\n$\n$ rclone lsd dbx-remote:\n -1 2020-08-06 10:13:47 -1 myFirstFolder","type":"text"}]}],"version":1}

Browser not supported