Mar 07 2016

Running Strace in Docker

Category: Cloud Computing,Containers,Kerneljgoulah @ 10:40 PM

I’ve been reverse engineering a new application setup and seemed like an appropriate place to try out docker. Spinning up a lightweight and reproducible environment is the goal and containerization is a reasonably efficient way to accomplish that. As I was looking into a problem with getting some services running properly, with little debug output and sparse documentation, I reached for the old trusty strace to see what was going on. But what do you know, strace is disabled by default on Docker. Here is the error that I got:

strace: test_ptrace_setoptions_for_all: PTRACE_TRACEME doesn't work: Operation not permitted
strace: test_ptrace_setoptions_for_all: unexpected exit status 1

This is admittedly an error I hadn’t seen before, and google isn’t a ton of help on this one. As a newbie with docker, it would have been helpful to have a bit more detailed error message as to why such a common tool as strace isn’t working.

Luckily some IRC logs come to the rescue in my quest through WTFed’ness. I learned that the security around this feature has apparently evolved a bit over time, with apparmor being the older but still used security system, and secconf being only available on newer distros (and OSX running boot2docker). Confusing things further, some of the articles out there are referring to apparmor (which uses different methods for modifying its security policy).

If you are using secconf, there are a couple of things you can pass to docker run to loosen up this security policy. To allow strace specifically, you enable the system call that it relies upon to get its information (ptrace):

--cap-add SYS_PTRACE

This whole paradigm is in fact documented but none of my original searches turned up these pages. In addition to disabling ptrace, there are a slew of other system level commands that you may (or may not) need that aren’t on the docker whitelist of allowed system calls. The list of calls can be adjusted very granularly by feeding docker a json file defining your security options. Or if you are feeling up for it, you can re-enable all of them in one fell swoop with this option to docker run:

--security-opt seccomp:unconfined

Its definitely worth considering which system calls your container really needs access to though, and strace is one of those that is quite useful for debugging purposes. There will always be that balance between security and usability, and decisions to make on which direction to swing the pendulum. It’s nice to see that this kind of functionality is being supported by docker to give the container very granular access to system level calls, and it might be interesting to think about ways it could be highlighted to a surprised enduser.

Tags: , , , ,


Jan 30 2009

Compile the Linux Kernel and Create Distributable Debian Packages

Category: Deployment,Kerneljgoulah @ 6:04 PM

Introduction

Compiling a kernel is actually a fairly easy thing to do these days. I’m going to go over how to do this on a Debian box since that happens to be my distro of choice. This will work just as well on Ubuntu. You can always wait for the packaged version, but you’ll always be a little behind some of the cutting edge features. This method allows you to get the latest upgrades that are incorporated into the kernel, or even to apply cutting edge kernel patches against the kernel source.

Getting the Source

You can always find the kernel at kernel.ftp.org. Login as anonymous and with your email address as the password:

$ ftp ftp.kernel.org
Connected to pub.us.kernel.org.
220 Welcome to ftp.kernel.org.
Name (ftp.kernel.org:jgoulah): anonymous
331 Please specify the password. {email address}
Password: 

Change directories into the 2.6.x series

ftp> cd pub/linux/kernel/v2.6

We want linux-2.6.28.tar.bz2, which is the newest at the time of this article

ftp> binary
200 Switching to Binary mode.
ftp> get linux-2.6.28.tar.bz2
ftp> exit

Now you have the kernel.

You may also need these tools depending what you’ve installed so far

apt-get install kernel-package libncurses5-dev fakeroot wget bzip2 build-essential

Extract and Configure the Source

We’ll put the tarball into /usr/src

$ sudo mv linux-2.6.28.tar.bz2 /usr/src/

Extract it

$ cd /usr/src
$ sudo tar xjf linux-2.6.28.tar.bz2

Its good measure to point a symlink to your current kernel

$ sudo ln -s linux-2.6.28 linux

And change into the directory

cd /usr/src/linux

If you have any patches, now is the time to install them

bzip2 -dc /usr/src/patch.bz2 | patch -p1

Clean things up

make clean && make mrproper

Now we can finally configure the kernel. Its a really smart idea to copy your existing configuration into the current kernel as a starting point. You certainly don’t want to lose any of your current modules.

$ sudo cp /boot/config-`uname -r` .config

There is one more step to load in your old settings

$ sudo make menuconfig

Now select Load an Alternate Configuration File
Enter your config file .config when it prompts you

When you exit out make sure to save and then you can do a diff against your old config and see the new kernel options:

$ diff /boot/config-`uname -r` .config

You can go back into menuconfig to make any changes necessarily, which is typically some new module you’d like to try out. For this kernel version I’m at least enabling ext4 and minstrel.

Compiling the Kernel

$ sudo make-kpkg clean

On this command you will want to set the string that gets appended to the version in the new kernel name. I usually just do something like -custom-buildX where X is the number of times I’ve changed configurations on this kernel version and rebuilt it. You can name it whatever you like as long as it begins with a minus (-) and doesn’t contain spaces

$ sudo fakeroot make-kpkg --initrd \ 
--append-to-version=-custom-build1 kernel_image kernel_headers

Go get a sandwich or something, depending on your computer this can take a while.

Installing the Kernel

The cool part about this is we’ve just created two .deb files that can be installed on other Debian servers, no re-compilation necessary. The files will look something like this, given the above parameter to the append-to-verson option from above

linux-headers-2.6.28-custom-build1_2.6.28-custom-build1-10.00.Custom_i386.deb
linux-image-2.6.28-custom-build1_2.6.28-custom-build1-10.00.Custom_i386.deb

So install them like a regular Debian package

$ sudo dpkg -i linux-image-2.6.28-custom-build1_2.6.28-custom-build1-10.00.Custom_i386.deb
Selecting previously deselected package linux-image-2.6.28-custom-build1.
(Reading database ... 418301 files and directories currently installed.)
Unpacking linux-image-2.6.28-custom-build1 (from linux-image-2.6.28-custom-build1_2.6.28-custom-build1-10.00.Custom_i386.deb) ...
Done.
Setting up linux-image-2.6.28-custom-build1 (2.6.28-custom-build1-10.00.Custom) ...
Running depmod.
Finding valid ramdisk creators.
Using mkinitramfs-kpkg to build the ramdisk.
Other valid candidates: mkinitramfs-kpkg mkinitrd.yaird
Running postinst hook script /sbin/update-grub.
You shouldn't call /sbin/update-grub. Please call /usr/sbin/update-grub instead!

Searching for GRUB installation directory ... found: /boot/grub
Searching for default file ... found: /boot/grub/default
Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Found kernel: /boot/vmlinuz-2.6.28-custom-build1
Found kernel: /boot/vmlinuz-2.6.26-custom-2.6.26
Found kernel: /boot/vmlinuz-2.6.26-custom-build7
Found kernel: /boot/vmlinuz-2.6.26-custom-build6
Found kernel: /boot/vmlinuz-2.6.26-custom-build5
Found kernel: /boot/vmlinuz-2.6.26-custom-build4
Found kernel: /boot/vmlinuz-2.6.26-custom-build3
Found kernel: /boot/vmlinuz-2.6.26-custom-build2
Found kernel: /boot/vmlinuz-2.6.24.4-custom13
Found kernel: /boot/vmlinuz-2.6.24.4-custom12
Found kernel: /boot/vmlinuz-2.6.24.4-custom11
Found kernel: /boot/vmlinuz-2.6.24.4-custom10
Found kernel: /boot/vmlinuz-2.6.24.4-custom9
Found kernel: /boot/vmlinuz-2.6.24.4-custom8
Found kernel: /boot/vmlinuz-2.6.24.4-custom7
Found kernel: /boot/vmlinuz-2.6.24.4-custom6
Found kernel: /boot/vmlinuz-2.6.24.4-custom5
Found kernel: /boot/vmlinuz-2.6.24.4-custom4
Found kernel: /boot/vmlinuz-2.6.24.4-custom3
Found kernel: /boot/vmlinuz-2.6.24.4-custom2
Found kernel: /boot/vmlinuz-2.6.24.4-custom
Found kernel: /boot/vmlinuz-2.6.18-6-686
Updating /boot/grub/menu.lst ... done

And the kernel headers

$ sudo dpkg -i linux-headers-2.6.28-custom-build1_2.6.28-custom-build1-10.00.Custom_i386.deb
Selecting previously deselected package linux-headers-2.6.28-custom-build1.
(Reading database ... 418509 files and directories currently installed.)
Unpacking linux-headers-2.6.28-custom-build1 (from linux-headers-2.6.28-custom-build1_2.6.28-custom-build1-10.00.Custom_i386.deb) ...
Setting up linux-headers-2.6.28-custom-build1 (2.6.28-custom-build1-10.00.Custom) ...

That’s pretty much it. You can look at your grub config

$ vim /boot/grub/menu.lst

Scroll down and you’ll see an entry for your new kernel. The topmost entry is the default, but remember you can also choose a different kernel at boot

title       Debian GNU/Linux, kernel 2.6.28-custom-build1
root        (hd1,0)
kernel      /boot/vmlinuz-2.6.28-custom-build1 root=/dev/sdb1 ro
initrd      /boot/initrd.img-2.6.28-custom-build1
savedefault

You can see the correct files installed into /boot

$ ls -al /boot/*2.6.28*
-rw-r--r-- 1 root root   63928 2009-01-28 23:56 /boot/config-2.6.28-custom-build1
-rw-r--r-- 1 root root 1958212 2009-01-30 17:34 /boot/initrd.img-2.6.28-custom-build1
-rw-r--r-- 1 root root 1173217 2009-01-29 00:07 /boot/System.map-2.6.28-custom-build1
-rw-r--r-- 1 root root 2899952 2009-01-29 00:07 /boot/vmlinuz-2.6.28-custom-build1

We are done, reboot

$ sudo shutdown -r now

Conclusion

We’ve seen in how just a few commands a new kernel can be configured and installed with some additional options while keeping the current configuration. Not only that, but we’ve produced Debian package files that can be installed onto other machines. This is one easy way to upgrade your kernel across many servers without having to wait for your vendor to release it.

Tags: , , , , , , ,