Hello everyone,
The Jedi has returned! Hahaha, I'm not dead yet but it's a while huh! Right, a lot of new stuffs for me with writing script on Ansible then Docker. So, today I will update you with "Docker - The Basic", this is a generic post, not on a specific requirement.
Alright, what is Docker, this is simple to have an overview on Docker terminology and you can find this page is a good source for nutshell explanation. Read carefully on "When To Use Docker", my understanding, this is another "image configuration management" methodology which is a lightweight and improvement from VM. So, if I have a simple mapping between apple and pear of container vs VM then you might have:
The Jedi has returned! Hahaha, I'm not dead yet but it's a while huh! Right, a lot of new stuffs for me with writing script on Ansible then Docker. So, today I will update you with "Docker - The Basic", this is a generic post, not on a specific requirement.
Alright, what is Docker, this is simple to have an overview on Docker terminology and you can find this page is a good source for nutshell explanation. Read carefully on "When To Use Docker", my understanding, this is another "image configuration management" methodology which is a lightweight and improvement from VM. So, if I have a simple mapping between apple and pear of container vs VM then you might have:
- Docker host vs ESX or KVM host
- Dockerfile vs Ansible playbook
- Docker image vs VM template
- Docker container vs VM
For sure they are not equal to be compared but the purpose for usage should be considered to be the same.
So, another item (on my case) when you consider of using docker is the system configuration vs application configuration. If you are heavily depend on system configuration such as network dns, user profile, shared file system ... get away of docker, because it shared the OS layer, which you have to create a scratch image based on a template machine with your system configuration there. This is a pain because you will have to create this again and again when you have some changes on system layer. I still can't make an image from scratch with my base machine after few days trying. Another thing is image file is a huge binary file (which docker stored it somewhere on the file system /var/lib/docker/<storage_type>/) so it's not a versionable for the configuration management. I would vote for VM on this case, you will have a based OS VM with an Ansible playbook, so now, every configuration can be changed and easily track via the playbook scripting. In the other hand, when you work on an environment purely based on the application packages configuration, this is a perfect use case for docker.
I have a tough time to verify how to use docker on my environment because I want to isolate the build environment as it might interfere other products. However, the build machine/Jenkins slave has huge dependency on the system configuration.
We have to build an application that will build some deb packages and some docker images. Then I decide to go with a docker host as a Jenkins slave then this machine will
- Create a Jenkins slave container to build the deb packages which will need some other app packages which might interfere to my current build environment on the original Jenkins slaves. I can store the Dockerfile into Git for version control
- Play as a Jenkins slave to build the docker images of the application. Why I don't build those images within the container above, I want to avoid running into Docker-in-Docker situation. Even, there should be the fix from Docker but I don't want to take risk on unknown areas.
So now, let's start!
First of all, install Docker, you can follow the guideline in docker docs. For me, I use docker ce on Ubuntu, so I select Get Docker/Docker CE/Linux/Ubuntu (on the left navigation) then I have a full guide here.
If you are not root user, then you have to run docker command with sudo because the docker daemon will create the /var/run/docker.sock for root. To get away of typing sudo on each command, you can add your user into the access control list of this sock file by using
sudo setfacl -m u:<user>:rwx /var/run/docker.sock
Or you can add your user into docker group is a permanent solution (because the docker.sock will be re-created when reload daemon). So now, you can run docker directly with your user, test your docker app version
docker --version
First of all, install Docker, you can follow the guideline in docker docs. For me, I use docker ce on Ubuntu, so I select Get Docker/Docker CE/Linux/Ubuntu (on the left navigation) then I have a full guide here.
- Update your apt repo (if necessary), install pre-requisite apps: apt-transport-https (this is required for add repo key via https), curl, ca-certificates, software-properties-common
- Add repo key from https://download.docker.com/linux/ubuntu/gpg
- Add apt repo to sources.list "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
- Update source repo then install docker-ce
If you are not root user, then you have to run docker command with sudo because the docker daemon will create the /var/run/docker.sock for root. To get away of typing sudo on each command, you can add your user into the access control list of this sock file by using
sudo setfacl -m u:<user>:rwx /var/run/docker.sock
Or you can add your user into docker group is a permanent solution (because the docker.sock will be re-created when reload daemon). So now, you can run docker directly with your user, test your docker app version
docker --version
Then now, create your workspace and place your Dockerfile into your workspace and you can start from here. You should take a look on the development best practices.
Because I want to reuse the jenkins user from domain so I create a user with the same uid from domain user in this image and add this user into sudo group in case jenkins might run any command with sudo privilege, switch to jenkins user at the end so when start the container it will run with jenkins user. Then building the docker image with
docker build -t <image_name>:<image_version> .
Remember the dot at the end is using the Dockerfile in current location
You will run the docker with the swarm CLI command
docker run -td --volume ~/.ssh:/home/jenkins/.ssh --name <container_name> <image_name>:<image_version> bash -c 'java -jar /home/jenkins/slave/swarm-client-3.9.jar -master <jenkins_url> -username <jenkins_user> -password <jenkins_pwd> -labels dockerpool -name <node_name> -description "Docker container" -executors 5 -fsroot <jenkins_home>'
I reuse some configuration setting from .ssh of the jenkins user on my docker host on this container. In detail, that I will have a config file that will use specific rsa key pair that we already created for jenkins user when connect to our Git. So I mount a volume from docker host jenkins/.ssh directory into jenkins/.ssh within the container.
You can run this container with interactive mode by using -ti, please note about using detach mode you must use together -td option instead of -d only because the bash command will exit immediately in detach mode.
To make sure you can run the whole bash command without escape on special characters on your data (such as admin password ...) you should indicate using bash command with "bash -c" then following with your command in single quote to avoid the translation.
Alright, at this stage, I have done the 1st item of the docker host machine - build a docker container as Jenkins slave for building debian packages. I can create a jenkins job that run on the slave 'dockerpool' label which is the docker container that has just been created.
Second item, I will build the docker images of my application build then push them into a docker registry. For detail of my task, we have an internal Artifactory server with the docker repository, so the target is building our docker images then push them into our internal artifactory docker repo. Then now, this docker hos machine has to connect to the artifactory docker repo as a client. I will write another post about how to use virtual, local and remote repo on artifactory, but here is for docker only. Because we haven't had a dns mapping for the docker repository setting on artifactory so I have to add this mapping manually into /etc/hosts file
<artifactory_ip> <virtual_docker_repo>.<artifactory_FQDN>
Then add this insecure registry into the daemon json configuration. Create /etc/docker/daemon.json file with content
{
"dns": ["<your_dns>", "8.8.8.8"],
"insecure-registries": ["<virtual_docker_repo>.<artifactory_FQDN>"],
"data-root": "<data_path>",
"storage-driver": "overlay2"
}
Beside "insecure-registries" entry, I have some other settings, you can find the whole list of all daemon options here.
- Select appropriate base image
- Minimize the image layers by using calling multiple commands on the same RUN, COPY, ADD with &&
- ADD vs COPY is ADD can extract file from archive.
I picked up ubuntu 16.04 official as base image then install the needed packages (I should try with other image to have a better size such as openjdk) and python modules. I need to have jenkins user in other to run this container as Jenkins slave. Here are some command that can make your container as a jenkins slave which I will use Jenkins slave swarm plugin
RUN adduser --disabled-password --gecos "" --uid <user_uid> jenkins && adduser jenkins sudo && mkdir /home/jenkins/slave && curl https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/swarm-client/3.9/swarm-client-3.9.jar -o /home/jenkins/slave/swarm-client-3.9.jar
RUN echo "jenkins ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
USER jenkins
Because I want to reuse the jenkins user from domain so I create a user with the same uid from domain user in this image and add this user into sudo group in case jenkins might run any command with sudo privilege, switch to jenkins user at the end so when start the container it will run with jenkins user. Then building the docker image with
docker build -t <image_name>:<image_version> .
Remember the dot at the end is using the Dockerfile in current location
You will run the docker with the swarm CLI command
docker run -td --volume ~/.ssh:/home/jenkins/.ssh --name <container_name> <image_name>:<image_version> bash -c 'java -jar /home/jenkins/slave/swarm-client-3.9.jar -master <jenkins_url> -username <jenkins_user> -password <jenkins_pwd> -labels dockerpool -name <node_name> -description "Docker container" -executors 5 -fsroot <jenkins_home>'
I reuse some configuration setting from .ssh of the jenkins user on my docker host on this container. In detail, that I will have a config file that will use specific rsa key pair that we already created for jenkins user when connect to our Git. So I mount a volume from docker host jenkins/.ssh directory into jenkins/.ssh within the container.
You can run this container with interactive mode by using -ti, please note about using detach mode you must use together -td option instead of -d only because the bash command will exit immediately in detach mode.
To make sure you can run the whole bash command without escape on special characters on your data (such as admin password ...) you should indicate using bash command with "bash -c" then following with your command in single quote to avoid the translation.
Alright, at this stage, I have done the 1st item of the docker host machine - build a docker container as Jenkins slave for building debian packages. I can create a jenkins job that run on the slave 'dockerpool' label which is the docker container that has just been created.
Second item, I will build the docker images of my application build then push them into a docker registry. For detail of my task, we have an internal Artifactory server with the docker repository, so the target is building our docker images then push them into our internal artifactory docker repo. Then now, this docker hos machine has to connect to the artifactory docker repo as a client. I will write another post about how to use virtual, local and remote repo on artifactory, but here is for docker only. Because we haven't had a dns mapping for the docker repository setting on artifactory so I have to add this mapping manually into /etc/hosts file
<artifactory_ip> <virtual_docker_repo>.<artifactory_FQDN>
Then add this insecure registry into the daemon json configuration. Create /etc/docker/daemon.json file with content
{
"dns": ["<your_dns>", "8.8.8.8"],
"insecure-registries": ["<virtual_docker_repo>.<artifactory_FQDN>"],
"data-root": "<data_path>",
"storage-driver": "overlay2"
}
- dns: as I will use pip install on my build script and this guide recommend the DNS misconfiguration might creates problem with pip so I put this configuration here to secure the dns
- data-root: because my docker host has limited storage and I use the shared file system on it, so the default docker root (get it from docker info, /var/lib/docker) doesn't have enough space for my docker images. I have to point the docker root to a custom location with this option.
- storage-driver: to make sure after I pointed to a custom data root, I use the default storage driver, set this value based on docker info data (Storage Driver)
After saving the daemon.json, restart docker service
sudo systemctl restart docker
When you restart docker service, a new /var/run/docker.sock has been created so you have to set acl for your user on this file again. Test your docker client to the artifactory docker registry
echo <artifactory_pwd> | docker login <virtual_docker_repo>.<artifactory_FQDN> --username <artifactory_user> --password --stdin
Login Succeeded
Then now, you can pull or push docker images from/to artifactory docker registry.
Other note when building docker swarm or list of docker images. I haven't studied on docker swarm yet, from my understanding, orchestrating the build process of list of docker images, you can use docker-compose. Get it from this guide
sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
Make sure it has execution permission
sudo chmod +x /usr/local/bin/docker-compose
You will create each Dockerfile under each image directory then create a docker compose yml file to build the images. I will not walk into detail on this at this time. Then tag the images and push them into artifactory docker registry
So the second task of my docker host is done. A side quest, I should clean up the docker images after each build to save space on my jenkins slave/docker host machine. After pushing all the built images, I will get all the images (docker image ls) then filter out the application built images then delete them (by ID, docker rmi <image_id>)
Some handy docker commands for your configuration management
docker image ls: list all docker images
docker rmi <image_id> : delete an image
docker ps -a : list all docker container (include the stopped container)
docker rm <container> : delete a container
docker start <container> : start a stopped container
docker stop <container> : stop a running container
docker build -t <image_name> . : build image from Dockerfile
docker run -ti <container_name> <image_name> : run container from image
docker exec -ti <container_id> bash: access to a running container
docker exec -ti <container_id> bash: access to a running container
You can launch the man page with docker <command> --help for more detail. My post focuses mostly on CI/CD so my knowledge will be on configuration site, not on application development, detail of Dockerfile, docker-compose ... should be in other post or when I have other job :D
Some tricks for docker run when you try to run container as a VM
docker run --privileged : allow container privilege to access to host devices ...
docker run --cidfile="docker.pid" --dns=xxx.xxx.xxx.xxx --publish-all : to access VPN resources via host VPN.
docker run -v /nfs:/nfs:shared : to mount the NFS from host to container
Some tricks for docker run when you try to run container as a VM
docker run --privileged : allow container privilege to access to host devices ...
docker run --cidfile="docker.pid" --dns=xxx.xxx.xxx.xxx --publish-all : to access VPN resources via host VPN.
docker run -v /nfs:/nfs:shared : to mount the NFS from host to container
This is for today! Have a nice weekend!
Comments
Post a Comment