Working my way to Seafile
I’ve recently started my journey into machine learning and I ran into a problem almost straight away. I have to keep jumping from different machines when travelling and this makes managing the project files difficult. Trying to keep the latest datasets and jupyter notebooks current was a pain.
My initial response was of course GitHub, with git being the perfect tool for source control, but I ran into an issue again with file size. Even the starter datasets for some of the introductory projects I was trying were in the 2-10 gigabyte range and while this is well below GitHub’s repo size limits it seemed far from sustainable to me.
We recommend repositories be kept under 1GB each. Repositories have a hard limit of 100GB. If you reach 75GB you’ll receive a warning from Git in your terminal when you push. This limit is easy to stay within if large files are kept out of the repository. If your repository exceeds 1GB, you might receive a polite email from GitHub Support requesting that you reduce the size of the repository to bring it back down.
My next thought was DropBox or Google Drive, these seemed like a good solution and in reality are, but the part of me that loves r/selfhosted really wanted me to host the solution rather than relying on a mega corporation. I’ve used OwnCloud in the past and to be honest didn’t have the best experience with it, so Nextcloud and Seafile were the next choices.
Nextcloud seemed to be much more feature rich than I need, with support for team messaging and lots of other things. It looks great and one of my mates uses it with great success. https://nextcloud.com/
Seafile took my fancy though, perhaps it’s the cool artwork or the fact it’s open source. https://www.seafile.com/en/home/
Where to host it and Setup
SoYouStart is my go to for server rental, I’ve been using them on and off for close to 10 years and the service has been great. The cheapest server that they offer at the time of writing this post has an Intel Xeon E3-1225v2 and 16GB of ram. This is a bit excessive for what I need, so a smaller box would be ideal.
Enter Kimsufi, the daughter company of SoYouStart, with servers starting at €3.99 ex. VAT. The “KS-3” fit the bill for me. A snappy little Intel Atom, 4GB of RAM and 2TB of disk space would more than suffice for the time being.
CentOS 7 was my chosen OS, used it in the past and feel fairly at home there.
Setup was a breeze, followed along with the installation script Seafile provides and 10 minutes later was up running. All that was left was a quick SSL certificate from Lets Encrypt and we are off to the races.
Now back to the task at hand, more epochs to run.