Introduction

Digital nomads are a type of people who use telecommunications technologies to earn a living and, more generally, conduct their life in a nomadic manner. Such workers often work remotely from foreign countries, coffee shops, public libraries, co-working spaces, or recreational vehicles. This is often accomplished through the use of devices that have wireless Internet capabilities such as smartphones or mobile hotspots [10].
As I am a Digital Nomad, I can personally attest to how wonderful Digital Nomadism can be. Unfortunately, the virtues of Digital Nomadism and operational objectives are not always be aligned so infrastructure and collaboration tools are an exceedingly important consideration when managing resources this way. As a manager and decision-maker for a group that utilizes Digital Nomads, I must take into consideration the processes and technologies that are necessary to drive efficiency gains while simultaneously considering their implications on the people involved in them.

Throughout the remainder of this blog I am going to focus on some of the considerations that are necessary for supporting a group of Digital Nomads that perform resource intensive computational work, such as Deep Learning. It is worth noting that many of the talking points that will follow are equally relevant to supporting a group that performs software development or data analytics with the likely exception of GPU considerations. Whether you are a decision maker for one of these groups, or a procurement professional supporting such a group, hopefully find this to be a useful guide.

A Quick Detour

Deep Learning, Machine Learning, and All The Fun Stuff

Deep learning is a subfield of machine learning. It focuses on developing and implementing algorithms that are inspired by the structure and function of the brain. According to Andrew Ng, the idea behind deep learning is that we may use brain simulations to:
  • Make learning algorithms much better and easier to use.
  • Make revolutionary advances in machine learning and AI. 
These algorithms are called artificial neural networks. Deep learning has become popular because while most learning algorithms eventually reach a plateau in their performance, deep learning is believed to be the first class of algorithms that are scalable. That is, their performance continues improving as you feed them more data (with some caveats that we won't get into here). [11]

CPU vs GPU

When most people discuss computer performance they are usually referring to CPUs. This is appropriate since for most activities, especially multi-tasking, CPUs are a highly important consideration. However, for backend matrix/vector operations such as graphics processing (i.e., playing computer games) and deep learning the GPU is king. A CPU is optimized to fetch small amounts of memory quickly while the GPU is optimized to fetch large amounts of memory not so quickly. One way to visualize the difference is using a squirt gun versus a hose in repeated water fights. The squirt gun can help you get your neighbor wet very quickly at any moment, while the hose can get them much more wet but requires much more time to setup between each fight. This is not to say that CPUs do not matter at all when performing matrix computations. Without a decent CPU one might run into some process bottlenecks.

GPU

Getting a bit more into the weeds, GPUs are computer chips that were originally developed for the purpose of rendering images which requires a heavy amount of matrix computation. As such, they have been optimized to perform this task. Coincidentally, deep learning also requires a heavy amount of matrix manipulation and so GPUs have since been re-purposed for the needs of data scientists and machine learning engineers. For a more in depth explanation see this excellent Quora post.

It is important to note that when I say GPU, as of writing this blog I really just mean NVIDIA. The reason for this is historical. NVIDIA's standard libraries made it easy for the first wave of deep learning libraries to be established in CUDA. It is for this reason, along with continued strong community support from NVIDIA, that deep learning capabilities rapidly grew. As of the publication of this blog AMD and Intel are just not truly viable options. See this post for a great analysis on the current state of GPUs.

Other Relevant Hardware

Other components that are necessary to build a deep learning rig include the following:
  • RAM: Random Access Memory. It performs short-term data storage by handling the information you are actively using so that you can access it quickly. You will likely want at least 32GB but your needs are largely defined by your use case - research & state-of-the-art capabilities requires a very different capability than Kaggle research or building a startup.
  • Hard Drive: It handles long-term data storage. You will likely want a solid-state drive (SSD) since it is much faster. If money is a concern a hybrid (SSHD) may be a viable alternative. You will likely want at least 1TB because deep learning datasets can get big.
  • Motherboard: In terms of your CPU, PCIe lanes are an important consideration. However, even more important is your ultimate use case and potential needs around using multiple GPUs. It's important to know your minimally viable considerations, and your long term objectives, since you want to ensure that the combination of your CPU and motherboard supports running the number GPUs that you are planning executing against.
  • Power Supply: You need a power supply that can produce as much power as is being consumed.
  • Case: You need something to put all of the above components in that will appropriately address protection and heat considerations.

Decisions, Decisions

Now that we have established a barebones understanding of the technologies involved in a computationally intensive rig we will simply list the options for deployment within a group that facilitate Digital Nomadism. For a more in depth analysis that provides pros and cons for each I highly recommend [1].

Tower

A full-blown piece of hardware that is used for local delivery. Not recommended.

Notebook

A high-end laptop.

Notebook + Tower

A low-to-mid tier laptop that can be used to run code remotely via the terminal (i.e., SSH-ing) on a piece of hardware.

Notebook + eGPU

A pre-built eGPU that affords plug and play capability.

Notebook + Cloud

A low-to-mid tier laptop that can be used to run code remotely via the terminal (i.e., SSH-ing) on someone else's (AWS, Google Cloud Platform, Microsoft, etc.) piece of hardware.

Costs

There are both direct and indirect costs associated with all of the options listed above. For instance, consider the opportunity costs and risks. Having a high-end laptop is great because you can run your code anywhere but it will likely come with considerations: a high upfront cost, operational risks if you are accessing data locally, upgrade and maintenance limitations, and probably a hefty weight that makes it less than desirable from a portability perspective. Alternatively, building and deploying your own rig may not be desirable - at least at the beginning - because you now have infrastructure costs and require time allocation towards management and maintenance. This leads one to believe that cloud computing might be preferable but then its highly important to consider how much data you have, costs associated with maintaining it, security risks around managing it, and execution times with your algorithms. In the long run there's a very real chance you'll spend less money not using web services after you've got yourself up and running.

Further Reading

For a better understanding of a logical enterprise approach towards strategic sourcing as it relates to this topic on an enterprise level, it may be useful to refer to some of the following links:
Alternatively, feel free to reach out to Source One's IT & Telecommunications expert David Pastore directly. Finally, for much more in depth reading on the specific topics covered above see the following links:

Works Cited

[1] Monn, Dominic. “Which hardware should I use as a Remote Machine Learning Engineer.” Towards Data Science (blog), 30 May 2018, https://towardsdatascience.com/which-hardware-should-i-use-as-a-remote-machine-learning-engineer-35af52301d3c.

[2] Fortuner, Brendan. “Building your own deep learning box.” Towards Data Science (blog), 12 Feb 2017, https://towardsdatascience.com/building-your-own-deep-learning-box-47b918aea1eb.

[3] Condo, Nick. “Build a Deep Learning Rig for $800.” Towards Data Science (blog), 22 Feb 2017, https://towardsdatascience.com/build-a-deep-learning-rig-for-800-4434e21a424f.

[4] Ragalie, Alex. “Build your own top-spec remote-access Machine Learning rig: a very detailed assembly and installation guide for a dual boot Ubuntu 16.04/Win 10 with CUDA 8 run on i7 6850K with 2x GTX 1080Ti GPUs.” Medium (blog), 2 Nov 2017, https://medium.com/@aragalie/build-your-own-top-spec-remote-access-machine-learning-rig-a-very-detailed-assembly-and-dae0f4011a8f.

[5] Biewald, Lukas. “Build a super fast deep learning machine for under $1,000.” O'Reilly (blog), 1 Feb 2017, https://www.oreilly.com/learning/build-a-super-fast-deep-learning-machine-for-under-1000.

[6] Chen, Jeff. “Why building your own Deep Learning Computer is 10x cheaper than AWS.” Medium (blog), 15 July 2019, https://medium.com/the-mission/why-building-your-own-deep-learning-computer-is-10x-cheaper-than-aws-b1c91b55ce8c.


[7] “Deep learning workstation 2018-2019 buyer's guide. BIZON G3000 deep learning devbox review, benchmark. 5X times faster vs Amazon AWS.” Bizon-Tech (blog), 10 Oct 2019, https://bizon-tech.com/blog/buyers-guide2018-bizon-g3000-gpu-deeplearning-workstation-review-benchmarks-5xtimes-faster-aws.

[8] Chen, Jeff. “How to build the perfect Deep Learning Computer and save thousands of dollars.” Medium (blog), 15 July 2019, https://medium.com/the-mission/how-to-build-the-perfect-deep-learning-computer-and-save-thousands-of-dollars-9ec3b2eb4ce2.

[9] Chen, Jeff. “Why your personal Deep Learning Computer can be faster than AWS and GCP.” Medium (blog), 15 July 2019, https://medium.com/the-mission/why-your-personal-deep-learning-computer-can-be-faster-than-aws-2f85a1739cf4.


[9] Chen, Jeff. “Why your personal Deep Learning Computer can be faster than AWS and GCP.” Medium (blog), 15 July 2019, https://medium.com/the-mission/why-your-personal-deep-learning-computer-can-be-faster-than-aws-2f85a1739cf4.


[10] Wikipedia, The Free Encyclopedia, s.v. "Digital Nomad," (accessed August 8, 2019), https://en.wikipedia.org/wiki/Digital_nomad


[11] Huang, Xin. “Andrew Ng: Deep Learning, Self-Taught Learning and Unsupervised Feature Learning.” YouTube video, 45:46. May 13, 2013. https://www.youtube.com/watch?v=n1ViNeWhC24


Share To:

James Patounas

Post A Comment:

0 comments so far,add yours