Docker on Mac Problems and How to Solve Them

I have a love/hate relatiaonship with Macs. There, I admitted my dirty secret. If I were building iPhone apps, my Macbook Pro would be getting a lot more use, but I find it's not my platform of choice for game development. And I tend to prefer Linux for server development. But I really wanted a build server that could build anything, so I specifically purchased a Mac Mini two years ago to run Jenkins builds. I haven't quite gotten around to Jenkins yet, but I have attempted many other things to learn the right way to run Docker on a Mac. There are a surprising number of dead ends with Docker + Mac. Here's what I've run across, maybe it'll help someone.

I originally attempted to run an OpenVPN instance in Docker for Mac. That was impossible because of the way host networking is implemented on Mac. I spent weeks trying to create a custom bridge network, configure static routes to get the OpenVPN connected clients to reach the LAN and also pass packets to the other Docker images. Don't bother. You can't. If you want to do that, you have to run Linux in VirtualBox (which is essentially Docker Machine, the old and now no longer fashionable version of Docker for Mac). Strike one.

So, settling in to running VirtualBox, I tried to run WordPress in Docker. Given that you really want to keep the content on the local hard drive to allow for backups and not lose everything when you reboot, you have to map volumes into your Docker instances. That's expected. But because I was running in VirtualBox, I simply created a folder mapping between the host (Mac) and the VM. Don't do this. Why not?

Well, for one, it exposes the bizarre Mac case-insensitive file system to the Linux processes.
Sharing folders has significant problems resolving the file owner and permissions. chown and chmod just don't work, and a lot of Docker images will do this and fail.
You can get a little farther by forcing some Dockers to run as the VirtualBox user id with --user but some will fail because the user doesn't exist in the image, or must run as root. However, recall that the file owner user in the Mac isn't typically the same user as in VirtualBox, so this often doesn't quite work and newly created files are saved in the Mac's file system frequently are not created with permissions compatible with the user inside Docker. Figuring out which user to run as can be very frustrating. In WordPress, I managed to get it working by launching the image by running a command to adduser then run the entrypoint.sh with custom parameters to skip some checks. Digging for these workarounds are super frustrating.
Any recent MySQL database created with a shared folder will not easily be mounted from any other file system. I found this out the hard way. MySQL databases store whether the tables are saved in a case-sensitive or case-insensitive file system, and mysqld flatly refuses to start or load the database. Which means you also can't even run mysqldump to export it and reimport it. If you have a free weekend to kill, you could do this, but I wouldn't recommend it.

So, what actually is working? Finally, I stumbled on a workable combination. On the Mac, export an NFS mount. This is easy. Just edit /etc/exports and add a line like the following, obviously replacing the folder and network to match your host (Mac). Note, the -mapall=UID:GID needs to be set to the user and group of the owner of the folder you are exporting. This forces all files created, read, and written to happen as the owner of that folder, regardless of who makes those requests. This is insecure, so lock it down if you need security.

/Users/jhughes/dockerdata -mapall=501:20 -network=10.0.0.0 -mask=255.0.0.0

Check the configuration of your exports file like this:

jhughes$ sudo nfsd checkexports

To get the Mac to actually pick up this change, you have to make sure the NFS daemon is enabled and running. Although I see start/stop being used, that never exported anything until I ran nfsd without parameters. So this is the command to start it up, after you have gotten the config right:

jhughes$ sudo nfsd enable && sudo nfsd stop && sudo nfsd start && sudo nfsd

Then, if you are actually exporting your mount, the following command will tell you:

jhughes$ showmount -e
Exports list on localhost:
/Users/jhughes/dockerdata 10.0.0.0

In VirtualBox, mount that as a folder. This is super easy! Just ssh into your VirtualBox guest and edit /etc/fstab and add this line to it:

10.0.0.200:/Users/jhughes/dockerdata /media/docker-data nfs auto 0 0

Now, make sure there's a place to mount the NFS folder, and mount it there:

jhughes$ mkdir /media/docker-data
jhughes$ sudo mount /media/docker-data

That's it! Now, how about a couple of examples of running useful Docker images? I thought you'd never ask. Running a Windows share is really easy. It doesn't require special permissions, you just have to create /media/docker-data/share so it has a place to store its data.

docker run -t --restart=unless-stopped --name share -p 139:139 -p 445:445 -p 138:138 -p 137:137 -d -v /media/docker-data/share:/var/share -e "TZ=CST6CDT" -e GROUPID=0 -e USERID=0 dperson/samba -n -w "WORKGROUP" -s "share;/var/share;yes;no;yes;all;none;;Public Share"

Now you should be able to find an SMB share at the IP address of your VirtualBox. :-) A similarly useful one-liner is running a MySQL box, which is fairly picky about permissions and file ownership matching the user running the process. In this case, you can force it to run as the Mac user id, as so:

docker run -t --restart=unless-stopped -d -e MYSQL_ROOT_PASSWORD=myrootpass -v /media/docker-data/mysql/conf:/etc/mysql/conf.d -v /media/docker-data/mysql/data:/var/lib/mysql --name mysql --user 501:20 mysql

Obviously, change the root password, and set the user UID:GID to match your NFS mount. I struggled with matching up ownership and permissions so much previously. Now most things run straight out of the box, but for a few things I need to override the UID:GID. I'm stoked!

STOKED!

Thanks for reading. I'm still setting up this new system (Ghost), but I look forward to dumping more learnings onto this site in the future.