Setting up your own Docker host engine


When I first got serious about learning Docker it was for rather selfish reasons and that was to save money on my hosting by using Docker to containerize the websites I had at the time.  I’ve not finished yet but I’m getting there.  I wanted a single “rented” host running virtualization of some sort.

Digital Ocean has turned out to be a quality and economic approach to getting this done.  Digital Ocean does supply an already pre-packaged Docker application service that you can be up and running on in a very short amount of time.  I am a thick headed DIY sort of guy and decided “nope! let me just spin up a droplet and set Docker up myself.”

Following the instructions that I published in a previous article I set up the Docker engine and what follows are the instructions for connecting the docker-machine CLI tool to this new instance.  It’s not that hard to do and is fairly rewarding.

Get ‘er Done!

OS Considerations

In case you might be wondering I set my Docker hosting machine up with Ubuntu 18.04LTS.  This is a distro that I’ve learned to love given I have machines provisioned with it that except for the fact I periodically reboot all my machines about once every quarter probably could have run for years with Ubuntu.  I have one last aging Debian server that sometime this calendar year I plan to swap out with Ubuntu.

Authentication Bits

I am assuming at this point you have a machine set up with docker-ce installed and running at this point. If not go back and do that.

You want to create an account on your machine and I used the same account name as exists on my laptop.

On your hosting machine as root

The “moduser” invocation adds the user to the docker user group allowing access to the docker CLI and associated operations.

Next you’ll want to create (if you don’t already have one) a RSA ssh key

On your local host as yourself

Take the public key that is generated (~/.ssh/ and copy it to the clipboard.

On the remote host as yourself you are going to create (if it doesn’t already exist) the .ssh directory and an authorized_keys file inside the .ssh directory.

On your local host ensure you are able to ssh as yourself to the remote host without being prompted for a password.

Once that is done it is now time to initialize the docker-machine.

Setup docker-machine

What you need going into this is the following information:

  • Fully qualified domain name of the Docker host
  • IP Address of the Docker host
  • path to your ssh public key

On your local host as yourself execute the following

Where USER is the userid you are logging into the remote host with, IP is the IP Address of the remote host and FQDN is the fully qualified domain of the remote host.


You should now be able to connect to and redirect your docker CLI to the remote host.

To set up the connection do the following on your local host as yourself:

This sets environmental values in your shell that tell the docker CLI where to send the actual requests you are making to.  If you want to see what is actually set simply run

In both cases substitute FQDN with the fully qualified domain name you used earlier.


Start with something simple:


That’s it!

Getting Started with Docker and Docker Compose

Introduction and chatter.

I’ve been using Docker now as a “hobbyist” for a few years now. I use it from everything from hosting this blog to testing out new ideas I plan to use at work.  Using Docker to test new ideas is a good thing especially if the tests I am doing have a potential for destroying something.  Who cares if I screw up an ephemeral container that I can always rebuild?

Of course Docker has a serious side and while so far I’ve not needed it in a professional setting (except to test Puppet modules, but that’s another topic for another post) learning Docker can’t hurt professionally so having one’s own Docker environment is a good thing and a great way to get comfortable.

I am not going to go into great detail about setting up a hosting environment in this post.  You actually do end up setting on your desktop or laptop just following my instructions.  What I am talking about is hosting one on a remote machine in a data center. You can pretty well read between the lines to figure that out there isn’t much difference.

What you’ll Need

  • An internet connection to download packages and scripts through
  • A fairly “beefy” laptop or desktop.  In this case a machine with enough memory (8Gb+) to support running your containers.
  • docker-ce package
  • docker-compose script
  • docker-machine script


The docker-ce package is the main thing needed here. This has all the software infrastructure to create containers on your local host and the commands needed to create packages remotely.


This provides a layer on top of docker (and I’ll get into more of this in another post) that allows your to code entire infrastructures in one file. For instance for a three-tier application you could define the build of a web server, application middleware server and database server as a unit and docker-compose would handle the tasks associated with spinning them up.


This provides and interface to redirect the results of docker or docker-machine commands to a remote Docker hosting machine.


Most of if not all the instructions are geared to working on Linux and in particular Ubuntu.  My hosted Docker host machine is running Ubuntu 18.04LTS at the moment and seems to do well.  For those on the spawn of the Evil Empire there are of course differences but I don’t do windows so those instructions won’t be here.

Docker Hosting

If you want to jump right in and get a Docker hosting system there are a few ways of doing that.

Build your own

Gotta nice server of your own?  Are your renting one from any of the myriad of hosting providers that exist?  No sweat, follow the instruction for installing docker-ce and add docker-machine setup to your list of short projects and away you go.

Hosted Solutions

Both Digital Ocean and Microsoft Azure offer hosted Docker solutions.  So far of the two I am more happy with Digital Ocean’s offering and I use the heck out of it.  Again you’ll want docker-machine set up on your laptop or desktop to interact with it.  Both have different approaches to setting up the credentials you need to access your Docker machine.

Digital Ocean also has support for Kubernetes coming soon.. just saying…

Let’s Get to Work


Again I’m covering how to do this on Ubuntu and first you’ll need to point to the Docker official repositories.

As root you want to execute the following:

This will import the gpg key for the docker-ce package and install the configuration for apt to find the docker-ce repository.  Now we want to install docker-ce along with its dependencies.

NOTE: if you have a previous version of docker, docker-ce or docker-engine or your machine it is advisable that you uninstall those prior to proceeding any further.

Installing docker-ce:

Now you have docker-ce installed you might want to validate everything is working for you.


Now it is time to install docker-compose.

Become root and perform the following:

That’s it, it is that simple

If all you want is to run Docker locally you are done.   However if you are planning on using Docker against a remote host you’ll also need docker-machine. Very simply become root and run the following:

On my next article I’ll tie all this together in a neat package for you so you can start spinning up containers to your heart’s content.



Tools that I use

I am inspired to write this post due to a flood of questions in the form of “which editor is best” or “which IDE do you” on some of the forums that I either lurk or actively participate in.

These kind of questions often kick of religious wars that I refuse to participate in.  The mantra I recite all the time is “use what you are comfortable with.”  If you have no starting point now my saying what I use on a daily basis can serve as a guide.  Keep in mind that some of the tools (most of actually) have their own learning curve associated with them.

The purpose of using a tool in the first place is to make the job at hand easier or serve some sort of function that would be missing without the tool.  As means of a ridiculous example let’s take the task of driving nails into wood.  Let’s make it a hard wood like oak or walnut.  I’ve never seen anybody take a nail in their hand and press it into an oak plank to attach it to another oak plank.  Common sense would tell you not to do that.  You use a hammer.  A rock could work in place of a hammer but a hammer is designed purposefully for the job.  Rocks tend to chip when subject to the stress of striking a piece of metal repeatedly and that rock could break in your hand.  The steel in a hammer is usually tempered to a point where it can absorb the energy of striking that bit of metal repeatedly while transferring its kinetic energy to the nail to insert it into the wood.

So what do I consider to be the tools of the trade?

Operating System

I am a Linux all the way sort of person.  I cannot stand Windows or MacOS and have long standardized on Linux as my OS of choice.  I started with Linux way back when Linux was quite frankly a PITA to get installed and running correctly on any hardware much less my first Linux host as a Sparc 20 that I managed to port Linux to.  That was a project in and of itself I assure you and I don’t even know if I could duplicate that today.  If my life depended on it I suppose I’d figure it out but with all the good distributions out there why would I even need to?  Besides the best Sparc 20 is a sluggish dinosaur compared with even modern consumer grand laptops and desktops.

My distro of choice (at least this year) is Ubuntu and my rather ancient laptop (HP Pavilion dv7, 8Gb RAM) was recently upgraded from Ubuntu 14LTS to Ubuntu 18.04LTS. Still cleaning up the collateral damage but that’s another story for another time.



Basic stock in trade for anybody doing any sort of Unix/Linux related work is some form of editor. If you are editing configuration files, Docker files, Perl code or whatever at some point your going to need to use an editor.  VI? VIM? Emacs?  Guess what I use all of them.

I do the bulk of my programming for Docker and Perl in Emacs.  There… I said it.  I started using Emacs back in the late 1990 timeframe and haven’t looked back.  Its advantage is its extensibility and if you know or can learn Lisp you can do all sorts of things you couldn’t do with lighter weight editors.  That statement is  a good segue into the disadvantage of Emacs.  It is a very bloated editor since it has so many extension points and features. It is at its best when used in an environment where X11 is available such as on a laptop in the MATE desktop environment.

So what about when X11 is not available?  No sweat, that’s where VI/VIM comes in.   If all I want to do is log into a remote machine and do a quick edit on a configuration file then I’m going to use VIM instead of Emacs.  VIM comes up immediately (or near to it) after I invoke it in the command line and Emacs takes a while.  I have impatient tendencies so I really don’t want to wait forty seconds (maybe not even that long) for Emacs to come up so I can do a two second edit.


I am not horribly fond of IDE environments.  Most can do a few jobs well and not do others very well.  For the time being I am using Netbeans as my IDE of choice for any Java development I am doing. I also use Netbeans when I am writing pipeline code (shared libraries and pipelines) for Jenkins in Groovy.  I’ve also used Anjuta for C and C++ projects.

Puppet Coding

Puppet Labs has released (and continues to release new versions of) pdk or Puppet Development Kit.  It has a few warts and I’m probably going to craft a post about just using pdk.  What it helps with is laying out a Puppet module (puppet module generate does the same thing) but also creating new classes along with the skeleton of the RSPec testing code.  Very easy to use and I’ve used it since laying my hands on it long ago.

One quibble I have with PDK is its insistence during an upgrade of removing the “Emacs” entries from .gitignore.  Hate that.

Version Control

Being a bit of a Git bigot and very much a DIY kinda guy I opted to convert from Bitbucket (still love the product but…) I decided to go with GOGS as my SCM repository.  I host mine in a Docker instance. Very much reminiscent of Bitbucket in its look and feel.  My one criticism is installation is not always the smoothest operation going and not always very clear in the instructions.  Once you get it working for you though it runs very smoothly.

Favorite Scripting Language

I have been using Perl as my scripting language since 1988 or so when I downloaded from comp.sources on the USENET. Whenever I start to write a script you can bet the first line is going to be  #!/usr/bin/perl -w ninety percent of the time. Perl in my mind combines the goodness of shell programming and C.  I’ll stop short of comparing Perl to C++ in any way since the OO interface to Perl has a lot of differences and is not at all strict.

For strongly typed languages I’ll use Java, C or C++ and not in order of preference in any way. I’m firm believer in using the right tool for the job and don’t try to force fit  languages to do things they weren’t designed to do.  I also use Ruby when I’m writing extensions to Puppet (custom facts and providers) but I haven’t warmed up to it yet.

Python? Nah.  I prefer to leave snakes the hell alone. Someday I know I’ll have to use it or maintain someone else’s code in Python.  Languages are about syntax and I’ve learned enough languages that if forced to I’d pick up Python too.


It’s the details, silly Perl error

Just today I was troubleshooting what seemed to be a spurious error in the logs of a script I wrote. The script itself isn’t all that important to this discussion but what I found in troubleshooting irked me to no end and I felt like I had to write about it.

In my script I have two places where I run a “find” command via the system() call to do very specific work. In this case doing recursive permissions and file mode changes on a directory.

Take a look at the offending line?

Looks innocent doesn’t it?   What you have there is an interpolation bomb for a lack of better term for it.  The error I was seeing was “find: missing argument to `-exec'” which after a bit of thought I realized the command the shell was actually seeing was “ find mydir -type f -exec chmod 0644 {} ; ” which would be missing the backslash as find would expect to be there. Whoops.

The reason for this would be everything inside the double quotes gets interpolated and the interpolation in play here was an unnecessary escape of the semicolon.  Modifying my code with a single quote thusly:


You know… really I should be using File::Find instead… but sometimes laziness gets the better of me…


DevOps — from a practitioner’s viewpoint

So what is DevOps all about anyways?

If you ask ten people what DevOps is you will get at least ten different answers and I find that the difference in definitions has a lot to do with the person being asked perspective.  Managers see one thing, developers another and the operations team something complete different.

My both being developer and a long term Unix/Linux professional including in the realms of system administration and operations I have yet another viewpoint on this.

Citing WikiPedia DevOps combines software development (Dev) with information technology operations (Ops) with the goal in mind of shortening the systems development life cycle all at the same time delivering features, fixes and updates while staying aligned with business goals and philosophy.

To this end a lot of great toolsets are employed to make all this happen.

So what am I going to talk about in this blog?  I currently work as a consultant designing and maintaining a Puppet infrastructure for a software development team.  Along with Puppet tools I use include Jenkins, Git, Docker, Perl and others. So going just a little past the limits of DevOps I’m going to talk about my favorite scripting language Perl.

I’ll also talk about Nagios which is my tool of choice for system and network monitoring.


Copyright DevOps — In the Trenches 2019
Tech Nerd theme designed by Siteturner