RetroFed Forum
Your home away from modern life.
Any Solutions for uploading files to cloud storage(e.g Google drive) and such, for people in countries with very slow upload speeds?
Post Reply
AuthorMessage
Cactus_Head programming.dev cactus_head@programming.dev
Joined: 2023-11-25
Quote

I have a collection of ebooks(8 GB atm) that i want to backup to google drive. It usually takes me 2-3 hrs to upload a 2 GB zip file(containing only the most important documents and files).

Tried to upload my ebooks, left it for 12 hrs and it still hasn't uploaded. Thought maybe its because i am using Rclone shared client ID(heavily rate limited ), so i set up my own client ID and tried again and its been 3 hrs and still hasn't upload

I am realizing now that its probably has nothing to do with rclone but rather my internet speed, specifically my upload speed(download 15-30 mps & upload 5-8 mps).

So i am wondering if anyone has solutions for people in countries with abysmal internet speeds.

22   Original
Auster thebrainbin.org Auster@thebrainbin.org
Joined: 2024-10-28
#2 · (edited) Quote Reply

  • uploading from a phone if it stays on all the time
  • uploading one at a time
  • uploading in smaller batches
  • uploading from an internet café, a library or some other place if it has internet faster than yours
  • hire a VPS, use some transfer method with a more generous timeout tolerance to transfer the files uncompressed to the VPS, and from there leave the ebooks uploading with your preferred method
  • hire a faster service if possible
9
Peffse lemmy.world Peffse@lemmy.world
Joined: 2023-09-28
#3 · Quote Reply

uploading from a phone if it stays on all the time uploading one at a time uploading in smaller batches uploading from an internet café, a library or some other place if it has internet faster than yours hire a VPS, use some transfer method with a more generous timeout tolerance to transfer the files uncompressed to the VPS, and from there leave the ebooks uploading with your preferred method h...

uploading one at a time, uploading in smaller batches

Yup, that's where I'd start. Creating a tar and splitting it into parts. Just as an example. Not sure if that guide is good or not. It was just the first thing that popped up in the search engine.

5
just_another_person lemmy.world just_another_person@lemmy.world
Joined: 2023-07-07
#4 · Quote Reply

Sounds like you need a client that does "multiparty uploads". Meaning interruptions should continue if there is a break in comms.

All the Cloud endpoints support this, you just need a better client. Do some research I guess.

3
Rivalarrival lemmy.today Rivalarrival@lemmy.today
Joined: 2023-06-11
#5 · Quote Reply

Where will you be using this upload?

If you're doing it to make it accessible on your own devices from anywhere, try Syncthing on the various devices instead. If the devices are on the same LAN from time to time, your modem won't be a bottleneck.

If you still have the sources from which you originally acquired the books, you could use a VPS to re-acquire them, and then push them to the google drive directly from the VPS. They never pass through your modem; your modem can't be the bottleneck.

3
solrize lemmy.ml solrize@lemmy.ml
Joined: 2023-06-22
#6 · Quote Reply

It's gonna take a long time, just deal with it. 12 hours is expected, right? Get some sleep and check it afterwards.

1
Cactus_Head programming.dev cactus_head@programming.dev
Joined: 2023-11-25
#7 · (edited) Quote Reply

It's gonna take a long time, just deal with it. 12 hours is expected, right? Get some sleep and check it afterwards.

It didn't even upload after 12 hours. I left it running the whole night and woke up in the morning to it still running

2
TheTechyHobbit sh.itjust.works TheTechyHobbit@sh.itjust.works
Joined: 2024-02-11
#8 · Quote Reply

Have you considered a torrent based solution?

Setup a torrenting server hosted online somewhere with fast speeds (e.g. Netherlands), sync it with gdrive, create a torrent for your books, and "download" your books to the toreenting machine.

This approach won't help with upload speed, but it should help with reliability/continuity. As long as you seed from your home computer, your books will eventually make it into your gdrive.

Although with this setup it might be easier to just torrent the ebooks to the online server from sources other than yourself.

2
fozid feddit.uk fozid@feddit.uk
Joined: 2026-01-25
#9 · Quote Reply

The only real solution to do it faster is to use faster internet or compress it further. Then alternatives are chunk it or deal with a long maybe even 24 hour upload.

1
I_am_10_squirrels beehaw.org I_am_10_squirrels@beehaw.org
Joined: 2023-06-03
#10 · Quote Reply

uploading from a phone if it stays on all the time uploading one at a time uploading in smaller batches uploading from an internet café, a library or some other place if it has internet faster than yours hire a VPS, use some transfer method with a more generous timeout tolerance to transfer the files uncompressed to the VPS, and from there leave the ebooks uploading with your preferred method h...

Mail a physical copy to someone with a faster connection and ask them to upload for you

3
IanTwenty piefed.social IanTwenty@piefed.social
Joined: 2025-10-17
#11 · Quote Reply

At 5mbps it should take about 1 hour for 2GB. It sounds like your actual speed is 2-3x lower. Can you take that up with your ISP? Are you certain your machine has the best connection within your control, i.e. directly wired into the router? Network equipment is not faulty? Have you tested with iperf within your network? Just in case there's another issue beside the slow external speed...

Another thing that springs to mind is to use a backup tool like restic that will not only compress but deduplicate your data into hundreds of small files that might make upload faster. Dedup can save significant space and you can try it out locally first. Just do restic init then restic backup PATH.

Restic can use rclone as a backend also and upload straight to google: https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#other-services

Finally there's sometimes nothing faster than physically moving data. A person jogging with a 100gb drive has great bandwidth! Is there a location with better internet within reach? A library or school perhaps?

5
Auster thebrainbin.org Auster@thebrainbin.org
Joined: 2024-10-28
#12 · (edited) Quote Reply

uploading one at a time, uploading in smaller batches Yup, that's where I'd start. Creating a tar and splitting it into parts. Just as an example. Not sure if that guide is good or not. It was just the first thing that popped up in the search engine.

Had forgotten about split archives. 7z also helps, e.g. if one wants to split into 500 MB files recursively, the person can use in Linux 7z a -r -v500m /some/path/some_name.7z /path/with/all/files/, and in Windows iirc right click in the folder > 7-Zip > Add to archive... > configure as the user needs.

2
Auster thebrainbin.org Auster@thebrainbin.org
Joined: 2024-10-28
#13 · Quote Reply

Have you considered a torrent based solution? Setup a torrenting server hosted online somewhere with fast speeds (e.g. Netherlands), sync it with gdrive, create a torrent for your books, and "download" your books to the toreenting machine. This approach won't help with upload speed, but it should help with reliability/continuity. As long as you seed from your home computer, your books will even...

If OP is going legit, torrenting is a pretty solid solition. And like the VPS and mailing suggestions, OP could torrent the files to a remote server, and from that server, upload to a 3rd party storage.

2
PetteriPano lemmy.world PetteriPano@lemmy.world
Joined: 2024-09-04
#14 · Quote Reply

I have great internets at home, but I did a lot of sailing abroad with limited roaming. I had my home server set up to download new videos from my favourite YouTube channels and put them in a shared syncthing directory.

I'd just go hit that local pub or library with free WiFi and let syncthing on my smartphone do its thing.

4
Post Reply
Back to top