The first thing I would like to mention in todays article is something that struck me as I was writing a bit of python code earlier this evening. For all of the (friendly and otherwise) jabs that the Python and Perl folks exchange, the more I work with python, I notice that there are a lot of small things that make it feel a little bit like perl to me. To some people, this might seem like an insult, but the truth is I quite like perl, and I mean the comparison as a sincere compliment. I imagine that a fair bit of this similarity is just in the fact that Python and Perl are both scripting languages, but there might be more to it. I will have to come back to this topic after a few months with Python when I’ll be better equipped to make a more informed comparison.

System Calls and Callbacks

With that segue aside, it’s time to move into the meat of todays article. I’d like to start by posting, of all things, a bit of C code that I’ve kept around to use every now and then when I need to walk through a directory structure.

void stat_file(const char* pathname)
{
    if(NULL == pathname)
        return;

    struct stat st;
    if(0 != stat(pathname, &st))
    {
        printf("error opening %s: %s\n",pathname, strerror(errno));
    }
    printf("%d %s",st.st_size,pathname);
    return;
}

void callback_traverse(const char* dirname, void (*callback)(const char*), int traversal_type, int return_type, int level=0)
{
        DIR* dir = NULL;
        struct dirent* entry;
        char* path = (char*)malloc( (255*(level+1)) * sizeof(char) );
        strcpy(path,dirname);
        strcat(path,"/");

        callback(dirname);
        if(NULL != (dir = opendir(dirname)))
        {
                while( (entry = readdir(dir)) != NULL)
                {
                        if( (entry->d_name)[0]=='.')
                                continue;
                        strcat(path,entry->d_name);
                        callback_traverse(entry->d_name,callback,traversal_type,return_type,level+1);
                        strcpy(path,dirname);
                        strcat(path,"/");
                }
        }
        closedir(dir);
        free(path);
        path=NULL;
}

and now the equivalent Python code:

def stat_file(pathname):
    r = stat(pathname)
    print r.st_size,pathname
#def

def traverse_path(pathname, fun):
    for root, dirs, files in walk(pathname):
        for file in files:
            fun(file)
        #for
    #for
#traverse_path

One thing you can see immediately is that the Python code is quite a bit shorter. The C code certainly has more error checking, a small bit of which may be useful in the python application as well, but by and large the python version is shorter. That said, I have to say that I don’t necessarily find that the Python code was easier to write; certainly part of that is just my unfamiliarity with Python, but the algorithm is basically the same, except that Python is implementing parts of it under the hood.

Easier or not though, I do have to give credit to Python since the fact is that I was able to write the application. My programming forte is mucking about at the systems level; I do certainly write applications level things from time to time, but as a systems developer, I think it’s nice that Python does at least have access to system calls. That alone is enough to move python up a notch in my internal ranking of programming languages, thanks to the ease with which I was able to write the above application, I think I can now say that I would prefer to write Python code compared to Java (and the fact is that, while I don’t write a lot of Java these days, I’ve never particularly minded it).

Looking at the above code, the important technical bit is obviously the ability to pass a function as an argument to a function. Python, with it’s at least half-hearted nod to functional programming, does at least gain one important win over C and C++ in this area; I don’t think that anyone in their right mind would argue against the fact that function pointer syntax in those languages is a bit much at times.

But, passing functions as arguments and working with systems calls are things I generally expect of any language, and it works well enough and without anything spectacular enough to merit droning on about. There was, however, one thing that came up while figuring out how to write this code that I think is deserving of a bit of a rant.

Pydoc

I like man pages. Some people use reference books or google when they are doing development (I use both on occasion) but more than anything else I like man pages for reference on all of the ins and outs of functions and structures that I’m using while doing development. This, again, probably is a symptom of my systems programming roots, but pretty much every library and system call that I use in my day-to-day development has a corresponding and well written man page describing the return values, parameters, how exactly the function works, etc. Using vim as my text editor, I have only to move the cursor over the function that I want to reference, and type ‘K’ to get the man page up on the screen for my perusal.

Python does not use man pages. There is certainly a man page for python, and plenty of available documentation, but most of the information is found through Python’s pydoc application. pydoc works pretty much like man, supporting the main features that I use with man (forward and backward scrolling, and searching). Vim is even smart enough to use Pydoc instead of man when it sees that I’m working with a python file. The problem that I’ve run into with pydoc is, to be fair, as much of a problem with vim as with pydoc itself. The problem is that, to search for a given function, you need to type the full ‘path’; so for instance to look up the ‘walk’ function, one needs to run pydoc os.path.walk. Another big problem is that pydoc os contains information about walk as well, but this information is different than the information in os.path.walk – and in both cases, simply typing ‘walk’ and then ‘K’ doesn’t open up the right pydoc page, meaning that I’m forced to manually type in !pydoc in vim rather than just using ‘K’.

I understand that this eliminates the very real possiblity of two packages having functions with the same name that need documenting, but I’d much prefer that pydoc at least try to search for what I’m looking for, and prompt me to choose one if multiple results are found.

Typing in Python

In the last entry, a reader commented that they would like me to discuss my feelings on the auto typing that Python uses, compared to typing in other languages. Since I’m still getting started on my journey to learning python, I’m not qualified to get too in depth on a discussion in this arena yet, but I thought I would at least address the issue at a high level and offer up my thoughts.

The first thing to realize is that python isn’t, strictly speaking, a dynamically typed language. Under the hood, python resembles C++ and Java more strongly than, for instance, perl. Python variables have an internal type, and in python you are prohibited from doing things like adding a number to a string, without calling some sort of conversation function at any rate. Python also allows you to explicitly cast a type, allowing you to say, for instance, foo = (int)some_function().

Fundamentally, I think that this is a good compromise, with a few potential potholes. Having an internal type does mean that you will get errors on certain types of code that perl might happily parse, leading to a more subtly broken program. Having an exception up right away avoids these problems. Unfortunately, python doesn’t go quite far enough in my opinion. While you can do something like foo = (int)3.14 to get 3; you can’t cast between integer and character types (a side effect, I believe, of both pythons string object, as well as the way python handles unicode).

The other problem with implicit rather than explicit typing is that it hides what the programmer is thinking. When you’re looking at a bit of C or C++ code, and you say “int some_value = some_function()” then the next programmer knows that (unless you are a horrible horrible programming) odds are you intend that to be a number. In python, if you type “some_value = some_function()” then you have to either explicitly comment that (and we all know how much developers love to comment code) or rely on the next programming to go along and trace through some_function to see what it returns, so that they know what type of value “some_value” should be expected to have.

That said, implicit typing isn’t all bad, and I would say that as long as you choose variable names properly (or, shudder, use something like hungarian notation) you should be good. As an interesting side node, C++0x is adding the ‘auto’ keyword for automatically determining types, meaning that this will soon be one less advantage (or disadvantage depending on your perspective and circumstances) that Python has over C++.

Closing Remarks

Another few days with Python, and I have to say that some of my initial reservations are starting to go away. There are certainly still things that I dislike about the language, but I’m starting to see that it has cases where it could indeed be useful. I don’t anticipate that it will ever become my primary language choice, but I may be getting to the point where I could see myself not minding using here or there when it would be a good fit. Over the next few days I’ll be looking at OOP in python. I’ll be updating again soon so check back regularly, and if there are any topics you’d especially like me to cover, drop me an email or leave a comment and I’ll try to address it in the next post.

Thanks for reading; codeninja out.

For someone who does not have experience with programming, getting started developing can be a daunting task. Having the advantage of being a developer, and knowing several other languages, I started out today with a general idea of what I would need to get started and write my first application. To get started I knew that I would need:

• the Python interpreter, to run the applications
• an editor, to write applications
• documentation, to know how to write the programs
• a program to write

Getting the environment up and running

Since I have been running Python applications before starting development, I knew that I already had the basic environment that I would need. For those of you following along at home, running linux you will probably find that you already have linux installed, or can easily find the packages on your installation disks or in your distribution repositories (Ubuntu and Debian users install the python2.5 and python2.5-doc packages from your repositories). OS X users should already have python installed, and Windows users can grab python from python.org

Setting up a development environment

Once again, I was able to benefit from previous experience. Since I prefer to do my C, C++ and Perl development from the command line with vim, I was ready to go with my editor of choice. Vim already had support for python syntax, and with syntax highlighting and auto-indent already enabled, I had nothing to do to get started. If you haven’t taken the time to set up a development environment already, you can grab my vimrc file, and save it to ~/.vimrc to get a head start (ubuntu users: if you haven’t done so already, you may want to install a more full featured vim (e.g. vim-gnome or vim-full) since the default vim installed on ubuntu is rather limited)

Documentation

Although man pages are a useful reference, they aren’t the best way to get started learning a new programming language. To get started, I decided to check out the Official Tutorial. The tutorial itself seems to be pretty good. I’ve made it through section 4 so far, and it seems to be a good reference. Of course, I also find having a dead-tree reference to be useful, not only for learning but for referring back to later. To supplement the official tutorial, I decided to pick up O’Reilly’s Learning Python since I’ve always been a fan of the Learning $foo series from O’Reilly. I personally prefer Learning Python to the official tutorial, since it seems to get into the details more quickly, plus it has exercises that will serve useful later when I run out of ideas for sample programs to write.

Getting Stuff Done

With an editor and interpreter ready to go, and documentation on hand, I set to work writing my first Python application. I wanted to get a basic feel for the language, and be able to start with something extremely simple. To start with, I knew that I wanted to be able to demonstrate:

• Putting text on the screen
• Using conrol structures (loops, branching)
• Create a function
• Do some math
• Use a library

I decided to start with a simple program that would calculate some interesting facts about a list of numbers, and print them to the screen. At this point I am more concerned with simply being able to use the language than being idiomatic about it. Below is my first ever Python application:

#!/usr/bin/python
import math

def fibnum(n):
    if n <= 1:
        return 1
    else:
        return fibnum(n-1) + fibnum(n-2)

def factorial(n):
    if n <= 1:
        return 1
    else:
        return n * factorial(n-1)

def isprime(n):
    if n < 2:
        return False
    for x in range(2,(int)(math.sqrt(n))+1):
        if ((n%x) == 0):
            return False
    return True

for x in range(0,10):
    print "%(x)dth fibonacci number is %(fib)d"%{"x":x,"fib":fibnum(x)}
    print '%(x)d! =%(fact)d'%{'x':x,'fact':factorial(x)}
    print x,"is",
    if not isprime(x):
        print "not",
    print "prime"

and here is the output:

0! =1
0 is not prime
1th fibonacci number is 1
1! =1
1 is not prime
2th fibonacci number is 2
2! =2
2 is prime
3th fibonacci number is 3
3! =6
3 is prime
4th fibonacci number is 5
4! =24
4 is not prime
5th fibonacci number is 8
5! =120
5 is prime
6th fibonacci number is 13
6! =720
6 is not prime
7th fibonacci number is 21
7! =5040
7 is prime
8th fibonacci number is 34
8! =40320
8 is not prime
9th fibonacci number is 55
9! =362880
9 is not prime

As you can see, the program simply iterates through the numbers 0-9 and calculates the nth fibonacci number, n!, and determines if n is prime or not, and displays this information to the screen.

With my first Python program actually written and running, it’s time to go over my thoughts on day 1. To start, I found that the syntax wasn’t that bad, but at the same time, I’m not yet convinced that it’s in any way superior to C, C++, or even perl. Significant whitespaces weren’t as big of a deal as I had expected, but I found the lack of a closing curly brace ‘}’ to end a function made it hard to visually scan and see where one block of code ends and another begins. Since we already have to use a colon ‘:’ to define the start of a block, it just seems like sheer pigheadedness to not to ahead and just end a block of code with some identifiable token. The print statement also seems to be different-for-the-sake-of-being-different. The print formatting is reminiscent of C’s printf function, but the choice of “format”%{dictionary} instead of just “format”,… seems nasty to me. I will have to see if this choice imbues any greater flexibility later that makes up for it’s quirkiness. The for in loop syntax is nice, and very similar to perl’s foreach and nearly identical to php’s for in loops, and I’m glad to see at least one language feature so far that seems reasonable and intuitive to me. Finally, to pick on last nit, making ‘True’ and ‘False’ start with capital letters is irritating; I have no idea how in the world capitalizing them is supposed to make code more intuitive or readable, so once again I’m left with the thought that for all of it’s talk about doing things better, python often does things different just for the sake of it.

Final thought for today: not as painful as it could have been, but certainly less of a joy that programming in C or Perl for me so far. There are quite a few ‘features’ that I’m on the fence about right now- I’ll have to see if they turn out to be more powerful once I’ve gotten some more Python experience under my belt. I hate having to capitalize “True” and “False”.

As a software developer, it’s important for me to stay on the leading edge of software development trends. Fail to keep up and you could just find yourself stuck maintaining legacy code in a dreary job for the rest of your life. The problem is that software development is particularly prone to Best-Thing-Ever syndrome. Programmers are often accused of having ADD, and there are few places this accusation is so aptly demonstrated than in the seemingly weekly changes in “the one true language”, “the one true paradigm”, “the one true pattern”.

I’m not talking here about enterprise programming, although enterprise development certainly has it’s share of bandwagons. What I’m really talking about here is programmer culture (or what I would have called hacker culture had the term not been bastardized). I’m not trying to say that this is a bad thing, it’s part of what makes programming culture what it is, but it’s something to be aware of.

A lot of these “greatest things ever” are good ideas, often temporarily over-hyped, but they serve a solid purpose and are useful in their context. Even the ones that aren’t useful are often at least fun to play around with. Times change to quickly, however, and nobody can be an expert in (or even familiar with) every “greatest thing ever.” It’s easy to miss things because of lack of time, and sometimes “the greatest thing ever” just doesn’t seem so great, and it gets ignored.

Python, for me, has falling into both of the latter two categories. I originally looked at Python back in 2000, because I was interested in trying to fix a bug in Redhat’s Anaconda installer. Since Python hadn’t really gained the large following that it currently has, combined with the fact that I was a fairly inexperienced developer with limited internet access at the time, I never really got very far in my ambition, and quickly moved on. Python continued to grow however.

I started hearing a lot about python in 2006 or so, and by 2007 it had really started to reach a critical mass of popularity. Hearing all of the good things about Python, and remembering back when I had tried to learn it several years ago, I decided to look back in on it early in 2007. Perhaps it was too soon since I’d had to take a COBOL class and the significant whitespace thing put me off, or maybe the stars were just aligned in a certain way, but I was not really impressed with python.

The syntax seemed ugly to me, the “there’s only one way to do it” philosophy was at odds with my personal philosophy of “give the developer as much power and flexibility as possible”, and the fact that every introduction to python I read basically told me that I was a bad developer, and my favorite languages (C++ and perl) sucked, and I should be thankful for having python to rescue me – it all added up to put me pretty dead set against python. In fact, all of those things still keep my bias against python; but in the end we can’t be a slave to our bias.

Over the last year, I’ve noticed that a lot of the applications that I’m using are written in python. Music players, Desktop widgets, version control systems, and even my favorite websites are all being re-written or replaced by versions written in python. For the most part, I’ve never had a problem with this. If a program works well, then who cares what it’s written in; and I know that a lot of people never shared my own personal bias against python. That said, the transition to more applications being written in python did have an effect; subtle at first it grew more noticeable as time went on, and eventually culminated in my decision to finally sit down and get comfortable with python.

As a Linux C and C++ developer, I always had a fairly distinct advantage over a lot of other people when it came to the Linux desktop. C and C++ were the dominant languages for a long time for even desktop applications under Linux. I had the advantage of speaking the lingua franca. Not only did free software give me the right to have and modify the code, but the fact was most code was written in my preferred languages (C, C++, perl). Using the rights afforded to my by the GPL and other free software licenses was a practical and easy thing for me to do. But every python application on my system is an application written in a foreign language; by not knowing python I was leaving myself in a position where I was unable to exercise my freedom to alter the free software on my computer.

So, all of that to say this: I’ve decided to make my peace with python. After I’ve used it for a while I may learn to like it, or I may always have a lingering hatred for the language for either technical reasons, or purely because I can’t get over my own issues, but at least, love it or hate it, I’ll be able to use it as another tool in my repertoire. And, in the spirit of keeping myself on track and honest about it, I’ve decided to document the whole process right here on tuxtips.org. I’ll be posting every couple of days with my updates on what I’ve learned, what I’m liking, and what I’m not liking. A live feed of the thoughts, trials, and tribulations of a systems programmer with a bias against python taking the dive to learn what it’s really about.

Stay tuned for part 1 tomorrow, where I’ll discuss the books and documentation I’m using to learn, my development environment, and my learning plan and goals.

Understanding the mysteries of /proc can greatly aid in administrering and programming on linux systems, because proc is itself a key into the depths of the system.  In some ways, proc can be thought of as similar to the windows registry; it contains specific files that contain information about the system, and control how the system operates.

There are two main ways that proc can be used: to monitor and keep track of processes, and to monitor and tune the kernel.

On the process side, proc has subdirectories for each currently running process.  The numeric directory entires in proc are PIDs representing the currently running processes.  Inside the directory for each PID are a number of useful files that can be used both from the command line to get information about a process, and, especially, from within the application in order to get useful information about itself.

Of particular note are:

• cmdline: this contains the command used to launch the application
• cwd: a simlink to the applications current working directory
• exe: a simlink to the application running
• status: contains useful information; especially useful for seeing memory usage

Outside of using proc to gain information regarding processes, proc also contains useful system information.  Of particular note are:

• cmdline: contains the kernel boot parameters, useful for custom init scripts
• cpuinfo: contains information about the processors or cores in the system
• loadavg: contains the system load average
• self: a simlink to the PID directory for the current process

Learning to effectively use proc can greatly ehnance the ability and ease with which you can tune your system, as well as allowing your applications to effectively manage and monitor themselves and other processes.

To start with, you will need 3 packages. I’ve provided a mirror of the packages here since I had some trouble tracking them down.

• Canna Server is a daemon that converts english characters into Kana
• Im-Ja A utility to allow you to type in romanji and get the appropriate characters
• scim A utility used by im-ja to talk to the canna server

If you don’t have it, you will also need the xmkmf utility; this was already available on slackware, and is probably available for your distribution as well.
Start by installing the canna server, scim, and im-ja in that order. Building the packages is fairly simple, just extract them, and run:

xmkmf
make Makefiles
make
make install

After the packages have been installed, you need to edit your gtk.immodules command:

sudo vim /etc/gtk-2.0/gtk.immodules

and append the following lines:

"/usr/lib/gtk-2.0/2.10.0/immodules/im-ja.so"
"im-ja" "Japanese" "gtk+" "/usr/share/locale" "ja"

Note: replace 2.10.0 with your version of Gtk+
Next, run the im-ja-conf utility, and set your shortcut keys. In my configuration, I have:

• Switch to next input mode – CTRL space
• Set direct input mode – MOD4 (windows key) n
• Toggle hiragana/direct mode – MOD4 h
• Toggle full/half-width katakana mode MOD4 k

Log out of Gnome and back in so that all of your applications will have the appropriate libraries available.
Once you are back in Gnome, if you want to enter some japanese characters, in a text box, right click and choose Input Mode -> Japanese Start typing in romanji and the characters will appear in a small floating text box. Use space to select characters or finish entering a word.
With these tools in hand, you can easily begin learning Japanese, or set up a machine that is usable for someone already familiar with the language. If you are learning Japanese, you might also find the Kanatest program useful. It will allow you to test and practice your knowledge of japanese characters!

 ~/.gnome2/nautilus-scripts 

directory.

When dealing with executable files, the paths for the selected files are passed in as command line arguments to the application. So, for C or C++ applications, you would parse the selected files by looping through the argv list. For shell scripts, Nautilus sets a number of useful environment variables that let you determine what files are selected, and what directory you are in. The environment variables set by Nautilus are:

•  NAUTILUS_SCRIPT_SELECTED_FILE_PATHS 

  · This variable lists the paths of the currently selected files, separated by newlines. This only works on local filesystems.

•  NAUTILUS_SCRIPT_SELECTED_URIS 

  · This variable contains the URIs (file://file) of the selected files, separated by newlines. This works on both local and remote filesystems.

•  NAUTILUS_SCRIPT_CURRENT_URI 

  · This contains the URI of the current directory. This works for both local and remote filesystems.

•  NAUTILUS_SCRIPT_WINDOW_GEOMETRY 

  · This provides you with the window location and geometry of the browser window.

If you choose “Open Scripts Folder” from the scripts submenu, you will also get a pop-up window giving you this information.

To give you an example of how to use Nautilus scripts, I’ve provided a few useful scripts that you can use and modify yourself to get started with Nautilus scripting.

Open Terminal Here
This script opens a gnome-terminal in the current directory, using the default profile.

#!/bin/bash
gnome-terminal --working-directory=`echo $NAUTILUS_SCRIPT_CURRENT_URI | sed -e "s/file:\/\///"`

Remove Spaces
This script replaces spaces with underscores for all files in the current directory.

#!/usr/bin/perl
foreach(split(/\n/,`ls`))
{
	($of=$_)=~s/\s/_/g;
	`mv "$_" $of`;
}

ogg2mp3
A script to convert all of the .ogg files in a directory into mp3’s

#!/bin/bash
oggdec $1.ogg
lame --preset 192 -ms -h $1.wav -o $1.mp3
rm $1.wav

Un-Subdirectory
This script moves all the files in all subdirectories into the current directory. This is useful for creating playlists and slideshows, or for re-organizing files.

#!/usr/bin/perl

chomp($top = `pwd`);

mvhere(".");

sub mvhere
{
	$dir = shift;
	chdir($dir);
	my $cwd = $top."/".$dir;
	for $f (`ls`)
	{
		chomp($f);
		if(-d $f)
		{
			mvhere($f);
		}
		elsif(-f $f)
		{
			`mv $f $top/`;
		}
	}
	chdir("..");
}

Well, there you have it. Now you can continue on customizing your own environment to automate tedious or common tasks. In the next part of this article we will discuss creating drag-and-drop scripted icons on your desktop, allowing you to script events based on dragging icons onto a special desktop scripted icon.

        //defined in dirent.h
        struct dirent* entry;

        //allocate a string with 255 characters per directory name, plus 1 character for the / between directories.
        char* path = (char*)malloc( ((255*(level+1))+level+1) * sizeof(char) );

        //this will become the parent node for subsequent calls
        //and gets placed under the parent node that was sent to us
        GtkTreeIter my_iter;
        strcpy(path,dirname);
        strcat(path,"/");

        if(NULL != (dir = opendir(dirname)))//make sure we can open the directory
        {
                //create a node on the tree for the directory
                gtk_tree_store_append(store,&my_iter,parent_iter);

                //add the current directory to the list
                gtk_tree_store_set(store,&my_iter,COL_NAME,shortname,-1);

                //start to read the subdirectories
                while( (entry = readdir(dir)) != NULL)
                {
                        //skip hidden files, . and ..
                        if( (entry->d_name)[0]==‘.’)
                        {
                                continue;
                        }

                        //create the full path to the subdirecotry
                        strcat(path,entry->d_name);

                        //recursively call fill with this subdirectory
                        fill_tree(path,entry->d_name,store,&my_iter,level+1);

                        //reset the path to the path of this directory
                        strcpy(path,dirname);
                        strcat(path,"/");
                }
        }

        //close the file handle and free memory
        closedir(dir);
        free(path);
        path=NULL;
}

fill_tree("/home/tuxtips","Home Directory",my_tree_store,NULL,0);

If you are loading a large directory structure (for example, /) you might consider spawning this off into it’s own thread, as it can take a bit of time. For something a bit more dynamic, consider removing the recursive functionality, and calling the function in response to a user clicking on an entry. Also note that, for simplicity, I’ve used 255 as the maximum directory entry size, but this is a variable set in dirent.h and varies from system to system, so be sure to look that up!

Until Next Time, Happy Hacking.

I just wanted to write up a quick post to apologize for the funky formatting of code in the last post.  I’m currently trying to figure out the best way to get code to look right inside of wordpress, without the WYSWYG editor fubaring it.  As I work out the best way to format code I’ll be going back to fix older posts, and I hope that you’ll stick with us through this refinement stage of the site.

Writing multithreaded applications is hard. Luckily it’s the same sort of hard that applies programming in general, which is to say that the bulk of the difficulty comes from learning how to think about problems correctly in order to know when and how to apply multithreading. It’s beyond the scope of this particular article to go into the design and architectural aspectects of multithreading an application, but the difficulty of mixing threading with C++ extends beyond the conceptual aspects and into the meat of the development.

If you’ve come from a background of developing threaded applications in Java, or perhaps one of the more esoteric concurrent programming languages like Erlang, or to a lesser extent, from a pure C background, you might wonder why threading in C++ is so difficult. To start with, let’s look at a bit of sample code:

#include
#include
using namespace std;

class example
{
	public:
		void say_something(char* something);
};

void example::say_something(char* something)
{
	cout<<something;
}

int main(int argc, char* argv[])
{
	example my_example;
	pthread_t my_thread;
	char message[] = {"hello world"};
	pthread_create(&my_thread,NULL,&my_example.something,s);//BROKEN!
	return 0;
}

If you were familiar with languages like Java or C, you might wonder why this wouldn’t work. To a C++ developer who has been dealing with the problem of getting C++ code to play nicely with threads, it’s an example of the difficulty of multithreading C++ applications. There are a few different problems with this code. Although some would say that it’s a syntax error, writing code like this, although not syntactically correct, is more of a symptom of misunderstanding the fundamental way that pthreads work, as well as difficulty that comes with mixing C libraries into C++ applications.

Often times examining the code more closely can help one to arrive at a solution, so let’s look at what the code is trying to do, and what might be the root cause of the failure. Fundamentally, what we have is a class, with a member function called say_something which takes a char* as an argument, and prints that char* to the screen. We want to run this function in it’s own thread, and then exit. Now that we know what we want to do, let’s look at why this particular implementation will not work. The compiler will generally give a single error and give up, but there are in fact a few different layers of errors that need to be addressed. Most of these errors come from the definition of the pthread_create function, so to start with, let’s take a look at the prototype for the function.

int pthread_create(pthread_t* thread, const pthread_attr_t* attr, void* (*start_function)(void*), void* args);

The pthread_create function takes an address of a pthread_t, attributes (which are generally NULL), a pointer to a function that returns a void pointer, and takes as it’s argument a single void pointer, and a void pointer to the argument to the function. So, of course, even were it not a class member, our say_something function, which returns void and takes a cstring would not have worked. The real problem is that pthread_create expects a pointer to a function, but we are trying to pass it a pointer to a member function.

If the problem that this poses is immediately apparent to you, feel free to skip down to the end of this paragraph and peruse the code below, but if you don’t immediately understand why this is a problem read on, it is one of the cruxes of understanding the hows and whys of multithreaded development in C++. Essentially, the issue comes down to whether the function has access to an implicit this pointer. Global functions, and static member functions are not associated with any particular object. You can, therefore, easily take the address of them, make pointers to them, and use them as callbacks. Class member functions, on the other hand, need a little bit of extra information; they need a this to associate themselves with. We can’t take the address of my_example.something because we lose the information related to the this pointer that the member function is associate with. The following code snippet may help to illustrate the point of the necessity of a this pointer.

#include
#include
using namespace std;

class example
{
	public:
	string s;
	void say_hi()
	{
		cout<<"hello "<*foo)();
	}
};

int main()
{
	example foo;
	example bar;
	foo.s = "world";
	bar.s = "there";
	foo.get_hi();
	bar.get_hi();
	void ((example::*f)()) =  bar.func();
	(foo.*f)();
	return 0;
}

Compiling and running this application gives the following illuminating output:

hello world
hello there
hello world

As you can see, although we took a function pointer from bar which had it’s string set to “there”, the output still shows foo’s string of “world”. By calling foo.f we are telling f to use foo as it’s this pointer. If it all sounds quite confusing and way too much trouble to bother with, well that’s because it usually is. You don’t often see code like the above snippet precisely because it’s difficult to read, and doesn’t buy you much in practice (There are a few cases where this is incredibly handy, and if you run into one of those situations you will now recognized it and be prepared). The reason for this was to illustrate why things we think should work when it comes to mulithreading with C++ don’t work.

So, you might think, that was a long winded explanation about why you can’t do something. The astute reader however will realize that by going into detail about why things don’t work, I have layed out the pieces to one of the most common solutions to making them work. As you may recall above, I pointed out that the limitation of requiring a valid this pointer applies only to non-static member functions. We’ve also seen that we can call a pointer-to-member-function if we explicitly provide a this pointer by using the -> or . operators. This opens up the groundwork for two solutions, one of which is easy to follow and implement, the other quite powerful.

For the first case, let us assume that we have a standard path of execution that a class will take, and that we simply want to run this path in it’s own thread. In this case we can implement a solution very similar to implementing the Runnable interface in Java. By creating a static class member function that takes a pointer to an object, we can simple use our pthread_create on this static function, and have the function call the appropriate entrance function of the data member it gets as it’s argument. Look at the code below to illustrate this point:

#include
#include
#include
using namespace std
class example
{
	public:
		string s;
		void say_hi()
		{
			cout<<"hello "get_hi();
			return NULL;
		}
};

int main()
{
	pthread_t launch_thread;
	example foo;
	foo.s = "world";
	pthread_create(&launch_thread, NULL, &example::run, static_cast(&foo));
	pthread_join(launch_thread,NULL);//added so we can see the output before we exit
	return 0;
}

In this example, we have added a static function. Since we can take the address of a static member function and treat it like a normal global function, we can easily pass this function into our pthread_create. This lets us treat each instance of the class as a single threadable unit. We can spawn off a thread to begin execution, and then let our class do it’s thing. This simple approach can solve many issues that might crop up in implementing concurrency in a C++ application. There are drawbacks however.

The previous approach limits in many ways. First of all, we are limiting ourselves to a single entry point into the class. If we want to spawn off more threads from other places, we need to create more static functions. We also need to know at compile time what functions we are going to want to call in threads. If we had a dozen possible entry points where we might want to spawn off a thread, the codebase could grow very large and very complex.

Fortunately, since we understand how to use pointer-to-member-functions, we have the benefit of another solution. Consider the following code snippet:

#include <iostream>
#include <string>
#include <pthread.h>
using namespace std;

class example
{
        public:
                string s;
                void say_hi()
                {
                        cout<<"hello "<<s<<endl;
                }

                void (example::*func())()
                {
                        return &example::say_hi;
                }

                void get_hi()
                {
                        void ((example::*foo)()) =  func();
                        (this->*foo)();
                }

                static void* thread_maker(void* dat_s);
};

typedef struct val_st
{
        example* e;
        void (example::*func)();
};

//moved this out of the class definition since we need to reference val_st structure
void* example::thread_maker(void* dat_s)
{
        struct val_st* v = static_cast<val_st*>(dat_s);
        example* e = v->e;
        void ((example::*f)()) = v->func;
        (e->*f)();
        return NULL;
}

int main()
{
        pthread_t launch_thread;
        example foo;
        example bar;
        val_st v;
        foo.s = "world";
        bar.s = "there";
        foo.get_hi();
        bar.get_hi();
        void ((example::*f)()) =  bar.func();
        (foo.*f)();
        v.e = &foo;
        v.func = foo.func();
        pthread_create(&launch_thread, NULL, &example::thread_maker, static_cast<void*>(&v));
        pthread_join(launch_thread,NULL);
        return 0;
}

In this case, we consider what we know about pointer-to-member-functions, and the this pointer, and exploit this to vastly simplify a class that might have multiple thread entry points. In order to do this, the first step is to add a structure to our application. This structure contains a pointer to a member function, and the object to become that member-function’s this pointer. This enables us to create a single static thread_maker function. We pass our structure (cast as a void*) to this static function which is spawned off in it’s own thread. We now have the ability to spawn off into a thread any arbitrary member function. We can even decide at run time what functions to spawn off. In performance-critical applications, we reduce the overhead of having layers-deep logic to bend all of our code around a single entry point, and trade off dozens or more functions with a single, slightly more difficult to read function. We now have the ability to create pthreads in C++ in a fast, concise, and elegant manner.