Docker behind a proxy

Docker is a useful tool in my toolbox. I especially recommend the remnux contributions by Lenny Zeltser. Recently I needed a VM with an Ubuntu guest OS to run docker instances of container but the issue was that the pull had to go through a authenticated proxy.

Docker has given this scenario thought and this is what I did:

sudo mkdir /etc/systemd/system/docker.service.d
sudo touch /etc/systemd/system/docker.service.d/http-proxy.conf
sudo vi /etc/systemd/system/docker.service.d/http-proxy.conf

In this file you put:
[Service]
Environment="HTTP_PROXY=http://user:password@proxyserver:port/"

Flush the changes:
sudo systemctl daemon-reload

Test if it worked:
sudo systemctl show --property=Environment docker

This should result in the proxy you entered

Restart docker
sudo systemctl restart docker

Dockerized recon-ng

OSINT is very important in what I do and thus I've spent some time getting familiarized with recon-ng in the past. Since I like my toys as much as the next IT guy I decided to run it in a container so that I can spin it up exactly how I like it in every environment.

The dockerfile

# Based on ubuntu
FROM ubuntu:16.04
Maintainer Erik Vanderhasselt

# Set environment variables
ENV DEBIAN_FRONTEND noninteractive

# Set the mirror to the country mirror
RUN sed -i 's/archive/be.archive/g' /etc/apt/sources.list

# Upgrade Ubuntu
RUN apt-get update
RUN apt-get install -f
RUN apt-get dist-upgrade -y

# Set the timezone
# bug in ubuntu 16.04:
# https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/1554806
RUN ln -fs /usr/share/zoneinfo/Europe/Brussels /etc/localtime
RUN dpkg-reconfigure -f noninteractive tzdata

# installation of git
RUN apt-get install -y git

# Clone the recon-ng from bitbucket
RUN git clone https://bitbucket.org/LaNMaSteR53/recon-ng.git

# Remove git
RUN apt-get remove -y git

# installation of python and pip
RUN apt-get install -y python
RUN apt-get install -y python-pip

WORKDIR /recon-ng
RUN pip install -r REQUIREMENTS

# Cleaning up
RUN apt-get autoremove -y
RUN apt-get clean -y

# Execute recon-ng framework
ENTRYPOINT ["./recon-ng"]

I am spinning this container up in Belgium thus that is why the mirror is the be mirror. If you work in another country it might be a good idea to change the value. This will make the downloads a bit faster.

To make the build a bit easier I wrote a small build script
#! /bin/bash

# creation of the output directory and subdirectories
mkdir -p output

# build of the docker script
DATE=`date +%y%m%d%H%M%S`
sudo docker build -t recon-ng:$DATE -t recon-ng:latest .


And since spinning it up requires the -v to simplify the output I wrote a little script

#! /bin/bash

# create the variables
IMAGE='recon-ng'
PATH_RECON=`pwd`

# creation of the output directory
mkdir -p output

# run the build
sudo docker run --rm -it -v $PATH_RECON/output/:/root/.recon-ng/ $IMAGE

Finally while I as at it I wrote a little script to archive the result so that I am sure I don't mix databases while working on different projects in parallel.

#! /bin/bash

# archive the output directory
sudo tar cvf recon-ng.tar output/

# delete the output directory
sudo rm -Rf output/

Adding disks to an LVM

One of my virtual machines ran out of disk space the other day because I wasn't sure of disk sizing when I initially started playing with it. The solution was simple the LVM had to be extended. This is how you do it:

1. sudo apt-get install system-config-lvm
2. sudo pvcreate /dev/your_disk
3. sudo vgextend VG_Name /dev/your_disk
4. sudo lvextend -1 +100%FREE LV_PATH 
5. sudo resize2fs LV_PATH
6. sudo init 6

That is it, simple right?

To determine the disk(s) you want to add you do ls /dev/sd* it will return your disks, you will probably want to add the disks with no numbers at the end.

To figure out your volume group (VG_NAME) you do sudo vgdisplay and to figure out the logical volume path (LV_PATH) you do sudo lvdisplay.

Now you screen me ... now you don't

screen is a little command that I use on a daily basis, it allows you to start a session, execute some commands, disconnect from the session while the command continues and later reconnect to it.

Starting a screen session

To start a screen session you just hit screen. Then you start whatever you need to run. By hitting ctrl+a and followed by d, you detach from your session. You see the session ID when you have exited the screen session, you will need this ID to reconnect to it.

Listing your sessions

If you have multiple sessions and want to have an overview you run screen -ls.

Reattaching to a session

When you come back and want to reattach to a session you do screen -r .

Killing a session from within a session

When you are in a session you can kill it by hitting ctrl+a and then k.

Killing a session from outside a session

When you want to kill a session from outside a screen session you do screen -X -S

Setting your editor in Ubuntu

Recently I needed to alter /etc/sudoers and this is done with visudo. The visudo default editor is Nano and I have a personal preference for vi.

To change the default editor on a system you do:

sudo update-alternatives --config editor

It will present you with a list of editors and you basically chose the number of the editor you prefer.

Setting up a DNS Server in Ubuntu

This month I have a student, Yannick Merckx, sitting next to me who is specializing in Artificial Intelligence and the goal is to leverage machine learning to detect malware using our DNS logs.

This DNS adventure gave me the idea to set up my own local DNS server up so I can block a bunch of things by making a sinkhole. The theory is simple, your local DNS server intercepts the request and does the lookup instead of the one given to you by the network/internet provider.

Installing bind9

The first step is to install a DNS server. I chose bind9 because that is one I used in the past and thus have some experience with.

sudo apt-get install bind9 bind9utils

Configuring bind9
Once the software is installed you need to configure it. The configuration lives in /etc/bind.

named.conf
named.conf is where your configuration starts. It contains a bunch of include statements.

named.conf.options
named.conf.options is where you configure the forwarders. The forwarders are the name servers your DNS server will use if it doesn't know the answer. If you want for example google's DNS servers to answer you set it like.

forwarders {
    8.8.8.8; 8.8.4.4
}

You can set multiple DNS servers, you can separate them with a semi-column (;). If you want to use other DNS servers than google you can for example use OpenDNS's servers which are 208.67.222.222 and 208.67.220.22

named.conf.local
In named.conf.local you configure what databases you want to use.

zone "xiobe.net" {
  type master;
  file "/etc/bind/db.xiobe.net";
};

zone "1.0.127.in-addr.arpa" {
  type master;
  file "/etc/bind/db.127";
};

logging {
  channel simple_log {
    file "/var/log/named/bind9.log" versions 3 size 5m;
    severity debug 10;
    print-time yes;
    print-severity yes;
    print-category yes;
  };

  category default {
    simple_log;
  };
};
 
I've set up a zone for xiobe.net, my domain, and said that the master database is located at /etc/bind/db.xiobe.net. The next zone I did exactly the same thing for the reverse lookup database.

The reason why I've set the severity to debug 10 is because this allows me to actually log the answer for the requested domain.


Finally I declared how the logging has to take place. The location of the log is specific since there is already an entry in the apparmor profile (/etc/apparmor.d/usr.sbin.named).

You have to create the directory named and the log file.
sudo mkdir /var/log/named
sudo touch /var/log/named/bind9.log
sudo chown -R bind:bind /var/log/named

The db.xiobe.net file are copies of the ones that come with /etc/bind and I just added the IP addresses for Xiobe's website so no further lookup needs to occur. In db.127 nothing changes since I want 127.0.0.1 to point to localhost.

named.conf.default-zones
In this file we don't need to change a thing. 

Testing the configuration
Testing the configuration was done by doing an nslookup.

nslookup www.linux.org

I got a reply and in the log it looked like

;; QUESTION SECTION:
;www.linux.org.            IN    A

;; ANSWER SECTION:
;www.linux.org.        4178    IN    CNAME    linux.org.
;linux.org.        2758    IN    A    104.225.135.13

In a next post I will explain how to set up the sinkhole

small update
I made a little mistake in the logging part above. I adapted the post.

Setting up an internal git server

This is a blogpost to explain how you set up an internal git server on an Ubuntu server the quick and dirty way for a small team.

Setting up the server
Setting up the users
The first thing is to set up all the users. This is done with useradd. In our example our user is named bob

sudo useradd -d /home/bob -s /bin/bash bob

In my case the users already existed. If they do not exist you still need to provide each user with a password using passwd.

sudo passwd bob

Setting up the group who can read and write to the repository server
To make management simple we are going to group all our users in a group and then manage the access on a group level.

sudo addgroup git_users

Adding the users to a group
Next we need to add the users to the group git_users.

sudo usermod -a -G git_users bob

Creating the folder structure
Next we are going to create our repositories.

cd /
sudo mkdir -p /export/git/

Creation of the actual repositories will be done later in the setup.

Setting the ownership
The ownership is still set to root who is configuring the server so we still need to set it to the git_users group so that the users have access.

sudo chgrp git_users /export/git

Set read-write permissions and a sticky bit
Our git_users group needs read and write permissions so they can get to the repositories and write to them. We will use a sticky bit to make sure all objects underneath inherit all the read-write permissions.

sudo chmod g+rws /export/git

Set up our first repository
Our first repository we set up is to manage Powershell scripts.

sudo mkdir /export/git/powershell

To check if our inheritance was correct you can do

ls -la /export/git/powershell

You should see the group have read, write and a sticky bit.

Initializing our repositories
The final step on the server is to initialize the git repository. We use for this two options. The first option is to indicate it is a new repository and is named bare. The second option we use is called shared to indicate it is shared by the whole group.

cd /export/git/powershell
sudo git init --shared --bare

Everything is ready to be used the next step is to use the repositories.

In my next blogpost I will explain how to use Git GUI on Windows system.

Setting our proxy for apt

In Debian/Ubuntu you have the command apt-get to do your updates and installation and it works all fine and well when you have a direct connection to the Internet but once you are behind a proxy you need to set a specific proxy configuration.

The reason why an export http_proxy doesn't have effect is because apt-get is executed in a sudo context and the environment variable is simply ignored. To solve this you need to do the following:

sudo touch /etc/apt/apt.conf

Edit /etc/apt/apt.conf
sudo vi /etc/apt/apt.conf

Add the following content:
Acquire::http::Proxy "http://username:password@proxy:port";

For example:
Acquire::http::Proxy "http://erik:thisIs_aLongPassword@proxy.company.com:3128";

After you have edited the file you need of course to restart the service or reboot the system to take effect. If you get the message "Extra junk at the end of file" it means something is wrong with the syntax like a missing semi-column.