Docker behind a proxy

Docker is a useful tool in my toolbox. I especially recommend the remnux contributions by Lenny Zeltser. Recently I needed a VM with an Ubuntu guest OS to run docker instances of container but the issue was that the pull had to go through a authenticated proxy.

Docker has given this scenario thought and this is what I did:

sudo mkdir /etc/systemd/system/docker.service.d
sudo touch /etc/systemd/system/docker.service.d/http-proxy.conf
sudo vi /etc/systemd/system/docker.service.d/http-proxy.conf

In this file you put:
[Service]
Environment="HTTP_PROXY=http://user:password@proxyserver:port/"

Flush the changes:
sudo systemctl daemon-reload

Test if it worked:
sudo systemctl show --property=Environment docker

This should result in the proxy you entered

Restart docker
sudo systemctl restart docker

Dockerized recon-ng

OSINT is very important in what I do and thus I've spent some time getting familiarized with recon-ng in the past. Since I like my toys as much as the next IT guy I decided to run it in a container so that I can spin it up exactly how I like it in every environment.

The dockerfile

# Based on ubuntu
FROM ubuntu:16.04
Maintainer Erik Vanderhasselt

# Set environment variables
ENV DEBIAN_FRONTEND noninteractive

# Set the mirror to the country mirror
RUN sed -i 's/archive/be.archive/g' /etc/apt/sources.list

# Upgrade Ubuntu
RUN apt-get update
RUN apt-get install -f
RUN apt-get dist-upgrade -y

# Set the timezone
# bug in ubuntu 16.04:
# https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/1554806
RUN ln -fs /usr/share/zoneinfo/Europe/Brussels /etc/localtime
RUN dpkg-reconfigure -f noninteractive tzdata

# installation of git
RUN apt-get install -y git

# Clone the recon-ng from bitbucket
RUN git clone https://bitbucket.org/LaNMaSteR53/recon-ng.git

# Remove git
RUN apt-get remove -y git

# installation of python and pip
RUN apt-get install -y python
RUN apt-get install -y python-pip

WORKDIR /recon-ng
RUN pip install -r REQUIREMENTS

# Cleaning up
RUN apt-get autoremove -y
RUN apt-get clean -y

# Execute recon-ng framework
ENTRYPOINT ["./recon-ng"]

I am spinning this container up in Belgium thus that is why the mirror is the be mirror. If you work in another country it might be a good idea to change the value. This will make the downloads a bit faster.

To make the build a bit easier I wrote a small build script
#! /bin/bash

# creation of the output directory and subdirectories
mkdir -p output

# build of the docker script
DATE=`date +%y%m%d%H%M%S`
sudo docker build -t recon-ng:$DATE -t recon-ng:latest .


And since spinning it up requires the -v to simplify the output I wrote a little script

#! /bin/bash

# create the variables
IMAGE='recon-ng'
PATH_RECON=`pwd`

# creation of the output directory
mkdir -p output

# run the build
sudo docker run --rm -it -v $PATH_RECON/output/:/root/.recon-ng/ $IMAGE

Finally while I as at it I wrote a little script to archive the result so that I am sure I don't mix databases while working on different projects in parallel.

#! /bin/bash

# archive the output directory
sudo tar cvf recon-ng.tar output/

# delete the output directory
sudo rm -Rf output/

FIR (fast incident response) in docker

FIR (Fast Incident Response) is a project by CERT Société Générale. It is a nice system to do incident tracking and I use it on a regular basis for over a year now. After a year of daily use, I gathered the users and a series of issues and wanted features where expressed.

To make things go forward in an easy way I decided it was time to dockerize the installation so the end users can give quick feedback on features under development.

Although there is a Dockerfile in the repo, I decided to make my own based on the existing one:

# Dockerfile to build FIR container
# Original Dockerfile by Kyle Maxwell https://github.com/certsocietegenerale/FIR/blob/master/docker/Dockerfile
#
# to build: docker build -t fir .
# to run: docker run -d p 8000:8000 fir
# webinterface: http://x.x.x.x.:8000
#
# default administrator: admin
# default password:  admin

# Based on ubuntu:latest
FROM ubuntu:16.04
MAINTAINER Erik Vanderhasselt

# Set environment variables
ENV DEBIAN_FRONTEND noninteractive

# Upgrade Ubuntu
RUN \
  apt-get update && \
  apt-get dist-upgrade -y && \
  apt-get autoremove -y && \
  apt-get clean

# Set the timezone
# https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/1554806
RUN ln -fs /usr/share/zoneinfo/Europe/Brussels /etc/localtime
RUN dpkg-reconfigure -f noninteractive tzdata

# Install dependencies
RUN apt-get install -y python-dev
RUN apt-get install -y python-pip
RUN apt-get install -y python-lxml
RUN apt-get install -y git
RUN apt-get install -y libxml2-dev
RUN apt-get install -y libxslt1-dev
RUN apt-get install -y libz-dev

# Install the latest version of pip
RUN pip install --upgrade pip

# create the user and group
RUN groupadd -r fir
RUN useradd -r -g fir -d /home/fir -s /usr/sbin/nologin -c "FIR user" fir

# Download FIR from Github
WORKDIR /home
RUN mkdir /home/fir
RUN cd /home/fir
RUN git clone https://github.com/certsocietegenerale/FIR.git
RUN mv FIR fir
RUN chown -R fir:fir /home/fir

# install the requirements
WORKDIR /home/fir/FIR
# remove psycopg2==2.6.2 from requirements.txt since we are not using PostgreSQL
RUN sed '/^psycopg2/d' /home/fir/FIR/requirements.txt > /home/fir/FIR/req1.txt
# run pip
RUN pip install -r /home/fir/FIR/req1.txt

# prepare to run
USER fir
ENV HOME /home/fir
ENV USER fir
WORKDIR /home/fir/FIR
RUN ./manage.py migrate
RUN ./manage.py loaddata incidents/fixtures/seed_data.json
RUN ./manage.py loaddata incidents/fixtures/dev_users.json

###############################################################################

# make it run
EXPOSE 8000
ENTRYPOINT ["/home/fir/FIR/manage.py"]
CMD ["runserver", "0.0.0.0:8000"]



To build the container you do sudo docker build -t fir .
To run the container you do sudo docker run -d p 8000:8000 fir
To access fir you point your browser to http://localhost:8000, the default login is admin and the default password is admin too.

Now you have a nice system to record your incidents which is a good start but you need incident response procedures. If you got no idea what I am talking about I recommend you read up on the documents written by ENISA, NIST, CERT.org, etc.

Dockerfiles ... creating your own docker images

In this post we are going to build our first image to go over the basics. We are going to build an OpenSSH server and expose it.
 
Creating a Dockerfile
The complete reference can be found at https://docs.docker.com/engine/reference/builder/ but I am going to use only a part of the possibilities. 

The Dockerfile is a set of instructions that allows you to build an image that can be used to a container

The first step is to create the Dockerfile
touch Dockerfile

In you favorite editor you open your Dockerfile and write
FROM ubuntu
MAINTAINER your_name _email>

We need the following line so we can spin up the container with as argurment the SSH root password.
ARG SSHD_Root_Password

Since it is best practice to keep an OS up-to-date you will need to add the following

RUN \
  apt-get update && \
  apt-get dist-upgrade -y && \
  apt-get autoremove -y && \
  apt-get clean  

The instructions above will pull down all the updates for the OS when you spin up your container.

To install the OpenSSH server we do the following

RUN \
  apt-get install -y openssh-server && \
  apt-get clean

Lets make a backup of the original config file and change the permissions so it can't be read by the whole world.

RUN cp /etc/ssh/sshd_config /etc/ssh/sshd_config.orig
RUN chmod a-w /etc/ssh/sshd_config.orig

We still need to set the root password based on the argument given to the container. To change the root password you add the following line
RUN echo "root:$SSHD_ROOT_PASSWORD" | chpasswd

It is not a security best practice to allow a root user to log in over ssh, but for simplicity sake we are going to allow it in this configuration. Never use this for production.
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config

Now we need to specify the default command that runs when the container starts, it takes a json format input and that is why the command and argument are between double quotes.
CMD ["/usr/sbin/sshd", "-D"]

Finally we need to expose the SSH port to the network
EXPOSE 22

Now that the Dockerfile is ready we need to actually build an image from it.

Building the image
Building an image is the next step that is done with docker build -t .

Notice the "." at the end, it is something you can sometimes miss.

To make it concrete the command is
docker build -t xiobe/sshd --build-arg SSHD_ROOT_PASSWORD=demo

xiobe/sshd is just the name I gave the image but it can be anything you like and as you can see we give the building argument SSHD_ROOT_PASSWORD a value of demo.

Remember that it might be handy to have the same SSH password everywhere as an admin but as an attack it is even handier since I need to obtain only one password. A better way would be to build in ssh-keys if you would like to but that is not the purpose of this demo.


Spinning the container up
The only thing that we still need to do is spin up the container. This is done with
docker run -d -P xiobe/sshd

the -d means that the docker container runs in background, when you use this option it returns the ID of the container.

the -P is to publish the exposed port to a random port this means when you spin it up docker will choose a random port on the host machine and map it to the port 22 of the container.


To see that the container is running you do
docker ps 

which also shows something like 0.0.0.0:32768->22/tcp what means the randomly selected port was in this case 32768.

When you are exposing real apps you will often do mappings with -p. In this case you would for example map port 22 on the host to the guest's 22. This would look like
docker run -d -p 22:22 xiobe/sshd



Removing an image
If you are building and it fails for some reason you will probably want to remove images after you are done. This can be done with

docker rmi

To make up your cleanup a bit faster you can clean up the untagged and unnamed images like this

docker images -q --no-trunc -f dangling=true | xargs docker rmi

Running the image after a successful build

Before you get to production grade you will most likely have to build a couple of images and run them to see if you are happy with the result.  Instead of cleaning up containers after you are done you can specify the -rm on your docker run command. This will automatically clean up after a run.

Running an applicication in docker

In the last blog post I wrote about how to run a docker container, in this blog post I explore running applications.

Running an application
Running an application inside a container is done with docker run. For example
docker run ubuntu /bin/echo "hello world"
outputs hello world

In the command above we tell docker to run the image ubuntu and to run in the container /bin/echo and pass it as parameter "hello world". When the application is executed the container is shutdown.

What is the point?
Okay, I admit hello world isn't the world best example but the point is that we ran /bin/echo in the container and passed it on the parameter "hello world". It executed and once executed it exited the container.

As we saw in the previous post docker run -i -t ubuntu /bin/bash does basically the same thing. It runs /bin/bash within the container and it is only because /bin/bash isn't finished executing that the container doesn't stop.

Cleaning up
Once your container has exited you can actually work with it again.

docker ps

Will show you all running containers and

docker ps -a

Will show you all running and exited containers.

If you have a container that isn't of any use to you any more you can just use

docker rm

To remove the container from the system.

Once you are getting the hang of it you will have a number of container with the status exited. To clean up my containers I use

docker ps -aq -f=exited | xargs docker rm

Using docker for the first time ...

In my previous post I described how to install docker, in this post we are going to download a pre-built image and run it.

To check if the docker installation is working you can run
docker info

Downloading the Ubuntu Image
We start by downloading an Ubuntu Image to play with. Ubuntu posts its image to the docker repository of images called the docker hub. To pull the image from the hub you do
docker pull ubuntu

You will see that the image is getting downloaded to your local image cache and the hashes are shown and the message "pull complete".  The hash is called the image ID and are the first 12 characters of the full name of the image. If you reissue the pull, you will get the information that the the image already exists.

The Image Cache
To get an overview of your image cache you do
docker images

This will show you the name of the images, their ID, size and when they got created. To get more information about the image you can do docker inspect followed by the IMAGE ID.

Starting the Container
To start your first container you do docker run -i -t ubuntu /bin/bash. This will open a prompt in your container. The -i flag tells docker it is an interactive container. The -t creates a pseudo-TTY and attaches it to stdin and stdout.

To exit the container you need to enter the sequence ctrl+p followed by ctrl+q. To have an overview of what is running you have docker ps -a.

To reattach the image back to the TTY you do docker attach followed by the IMAGE ID.

Quitting the Container
When you are done with the container you simply type exit and it closes your session. If you run a docker ps -a after you have exited your container you will see it described as "exited x minutes ago".

Installing Software
We will come back on how to prepare the container but I just already want to point out that if you start a container, install software and then exit the container the installed software is no longer in the container since every time you run docker run -i -t ubuntu /bin/bash a new container is started.

The same logic also applies to data, so if you want to use a container for running something and need a configuration, you got to make sure the configuration is stored outside of your container.

What Happens When You Run a Container?
When you type docker run it tells the daemon you want it to run a container. The parameters we specified are the image it needs to run and what to start if the image is running. In our case above we specified the Ubuntu image and to run /bin/bash.

1. If the image is not available on the system it will be pulled from the Docker Hub 
2. Once the image exists on the system docker will create a container.
3. The container is created with a read-write layer on top.
4. The network interface is set up to communicate with the localhost.
5. The IP address is set up.
6. The requested process is executed.

 

Installing docker.io on Mint Linux 17.2 (Rafaela)

Mint Linux is an OS that I use as desktop system and sometimes I need a server to quickly run a 'little server' to do something. One of the solutions to spin up the 'little server' is docker.

The Key Concepts
Daemon and Client
On a Linux host the docker engine runs on the host machine. You interact with the docker engine, called the docker daemon, via a docker client. It is the daemon that interacts with the containers, not the client.

If you would run on a Windows machine, you would first spin up a virtual machine that is a Linux host.

Localhost and Dockerhost
The concept localhost in the network world would mean the local computer, the term dockerhost is the machine that runs the containers.

You can use standard addressing like localhost:8000 or 0.0.0.0:8000.

Docker Infrastructure
The infrastructure consists of images, registries and containers.

Images are read-only templates. Docker images are the build component of docker. You download or build an image you use. The image can come with pre-installed software so you only need to push a configuration when you run it.

Registries are the places where you store images. You pull images from a registry and distribute images via a registry.

Containers are the things that actually run the image. A container can be started, stopped, moved and deleted.

How Do Images Work?
Docker uses images that consist of multiple layers. To do this it uses a virtual file system. Actions are done to the base image and layers are created and its relation to the other layers is documented. The result is that the distribution is rather lightweight and to update an image you only need to update the layer that is out of date.

Every image starts with a base image, in my next post it will be an Ubuntu base. By default all base images are downloaded from Docker Hub. To build the image the system uses a file with instructions called the Dockerfile. Each instruction basically creates a new layer in the image.

The images are stored locally in the docker registry. The docker hub can be considered as a public docker registry.

How Does A Container Work?
A container consists of the operating system, user-added files, and meta-data. The image tells docker what is inside the container and what processes to run the container is launched. The image is read-only but when a container is executed a read-write layer is added on top in which the application can run.

It is an important concept to grasp that when a container runs it has one process that is considered the main process and when this process is killed the container stops running.

Installation

To spin up docker on Mint I did the following:
1. Figure out the kernel version
sudo uname -r

2. Figure out the Ubuntu version
sudo uname -a
As we can see the Mint Linux is based on Ubuntu 14.04.1.

3. Update the APT repo's
sudo apt-get update

4. Install apt-transport-https and ca-certificates
sudo apt-get install apt-transport-https ca-certificates

5. Add the docker key to the system
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

6. Make the docker repo
sudo vi /etc/apt/sources.list.d/docker.list

Add the following repo (Ubuntu 14.04, is called Trusty Thar)
deb https://apt.dockerproject.org/repo ubuntu-trusty main

Close /etc/apt/sources.list.d/docker.list

7. Update the repos
sudo apt-get update

8. Install the linux-image-extra (pre-requirement for Ubuntu 14.04)
sudo apt-get install linux-image-extra-$(uname -r)

9. Install apparmor (pre-requirement for Ubuntu 14.04)
sudo apt-get install apparmor

10. Install the docker engine
sudo apt-get install docker-engine

11. Start the service
sudo service docker start

12. Test the installation with a hello-world
sudo docker run hello-world

This outputs
Hello from Docker.
This message shows that your installation appears to be working correctly.
...

To run docker without the need of sudo you can add the user who has sudo rights to the docker group.

sudo usermod -aG docker user

If you have tried to execute without the appropriate permissions you will get the following message:
Cannot connect to the Docker daemon. Is 'docker daemon' running on this host?


The service comes automatically online after a (re)boot.

Memory Tuning  
According to the docker documentation the memory overhead is around 1% of the total available memory and the performance degradation is around 10%. To enable memory and swap accounting you can edit /etc/default/grub.

sudo vi /etc/default/grub

and change add to GRUB_CMDLINE_LINUX the value cgroup_enable=memory swapaccount=1.

To take effect you need to run update-grub.

Firewall

As always it is recommended to run the local firewall (ufw). If you want to connect from the host to the container you need to do nothing, if you want to connect from another host you'll need to open port 2376/tcp or 2375/tcp. The difference between both is that 2376 is for use with TLS and 2375 isn't.

Note that this means that all communication over 2375 will thus be unencrypted.

To create the forward policy you edit /etc/default/ufw and set DEFAULT_FORWARD_POLICY="ACCEPT"

You need to reload ufw to take effect
sudo ufw reload

Finally you need to allow the incoming connections on the right ports
sudo ufw allow 2376/tcp
sudo ufw allow 2375/tcp

DNS
Ubuntu-based systems are configured in such a way that by default you can't use /etc/resolv.conf.
To configure Ubuntu 14.04 you need specify the DNS server that docker needs to use in DOCKER_OPTS in /etc/default/docker/. Set your dns server's ip address instead of the Google IP addresses.

Timestamping history

Being an incident response person means sometimes you are also asked some help for troubleshooting things. The other day I was looking at a config and wanted to know what the sysadmin did on his Linux box before calling me in.

After I helped him (it was a typo in a config file) I told him something I only learned half a year ago but I think is pretty handy. You can actually timestamp the commands in the history file.

The magical line is :
echo 'export HISTTIMEFORMAT="%d/%m/%y %T "' >> ~/.bash_profile
I also put it in my Dockerfile with the following instruction:
ENV HISTTIMEFORMAT="%d/%m/%y %T "