Add Book to My BookshelfPurchase This Book Online

Chapter 2 - Designing Threaded Programs

Pthreads Programming
Bradford Nichols, Dick Buttlar and Jacqueline Proulx Farrell
 Copyright © 1996 O'Reilly & Associates, Inc.

Example: An ATM Server
Example 2-6 is a client/server program that implements an imaginary automated teller machine (ATM) application. This server will give us an opportunity to exercise our thinking about multithreaded program design and explore more realistic—and more complicated—thread handling applications.
As shown in Figure 2-6, the example is made up of a client that provides a user interface* and a server that processes requests from the client. On disk, the server stores a database of bank accounts, each including an account ID, password, and balance.
 *The client for an ATM application should be an actual machine, but, for the purposes of this book, we'll just make it a command-line program that accepts typed-in requests. (Unfortunately, this type of client isn't realistic enough to spit out ten dollar bills.)
Figure 2-6: The ATM and bank database server example
In a typical ATM operation, a customer chooses a withdrawal from a menu presented by the client and enters the amount to be withdrawn. The client packages this information into a request that it sends to the server. The server spawns a thread that checks the user's password against the one in the database, decrements the amount of money in the user's account, and sends back an indication of whether the operation succeeded. The client and server process communicate using UNIX sockets. The client reports any information returned from the server back to the user. Multiple clients can run simultaneously.
We want the server to be capable of overlapping I/O, because the account data is stored in secondary storage and its access will require a significant amount of time. The environment is asynchronous because multiple clients may exist simultaneously, sending requests of unpredictable type, order, and frequency.
In the following sections of this chapter, we'll discuss two different implementations of this program: a serial version and a multithreaded version that uses Pthreads* The multithreaded version of the program uses the boss/worker model inside the server. The boss looks at the first field of each request, then spawns a thread or process to handle that request. When the worker completes the request, it communicates the results directly back to the client program.
 *You can obtain the complete source code for all versions of the ATM example, including that for the multiprocess version used in our performance testing in Chapter 6, from our ftp site. Throughout this chapter, we'll show only those interfaces and routines pertinent to the current discussion.
For simplicity's sake, we've partitioned the client and server into modules. The interfaces between these modules will remain unchanged throughout all versions of our example. We'll change only the dispatch and service routine module from one version to another. Table 2-1 shows the contents of the client and server modules.
Table 2-1: The ATM Example Program Modules
Module
Component
Description
Client program
User interface (main)
Prompts a customer for a request, parses the response, and makes a remote procedure call (RPC) to access the server.
RPC
Includes a procedure for each possible type of request. Each procedure copies its arguments into a buffer and passes the buffer to the communication module for transmission to the server. When a response arrives from the server, the procedure checks its return values.
Communication
Finds and passes buffers to and from the server using UNIX sockets.
Server program
Communication
Receives and transmits buffers to clients using UNIX sockets.
Dispatch (and service) routines (main)
Obtains input buffers from clients by means of the communication module, identifies the request type and copies out arguments, and calls the service routine that handles the requested operation. Together, the dispatch and service routines make up the server-side procedures of the client's RPC. When request processing is complete, the dispatch routine prepares and transmits a response buffer to the client.
Database routines
Reads from and writes to the account database file using standard file I/O.
The Serial ATM Server
If we didn't have threads, what would be the simplest implementation of the ATM server? One that comes to mind is a program that runs in a loop, processing available requests serially. If a request is available, the program processes it in a request-specific service routine and sends a response to the client. The main routine for this version of the server is shown in Example 2-6.
Example 2-6: Serial ATM Server: main Routine (atm_svr_serial.c)
   extern int
  main(argc, argv)
  int argc;
  char **argv;
  {
    char req_buf[COMM_BUF_SIZE], resp_buf[COMM_BUF_SIZE];
    int  conn;
    int  trans_id;
    int  done=0;
    atm_server_init(argc, argv);
    /* loop forever */
    for(;;) {
      server_comm_get_request(&conn, req_buf);
      sscanf(req_buf, "%d", &trans_id);
      switch(trans_id) {
        case CREATE_ACCT_TRANS:
             create_account(resp_buf);
             break;
        case DEPOSIT_TRANS:
             deposit(req_buf, resp_buf);
             break;
        case WITHDRAW_TRANS:
             withdraw(req_buf, resp_buf);
             break;
        case BALANCE_TRANS:
             balance(req_buf, resp_buf);
             break;
        case SHUTDOWN:
             if (shutdown_req(req_buf, resp_buf)) done = 1;
             break;
        default:
             handle_bad_trans_id(req_buf, resp_buf);
             break;
        }
      server_comm_send_response(conn, resp_buf);
      if(done) break;
    }
    server_comm_shutdown();
  }
return 0;
The serial version of our ATM server can process only a single request at a time, no matter how many clients are requesting service.
Handling asynchronous events: blocking with select
The server handles the asynchronous arrival of requests from clients by waiting. When the server's main routine calls server_comm_get_request, the server's communication layer uses a UNIX select call to determine which channels have data on them waiting to be read. If none do, the select call (and consequently the server_comm_get_request call) blocks until data arrives.
Handling file I/O: blocking with read/write
The server in Example 2-6 does nothing but block when performing file operations. When it issues a read or write call to the file, the server waits until the operating system completes the operation and the call returns.
The server could have used UNIX signals to access the file without blocking. If so, it would need to establish a signal handler that processes the results of its I/O requests and to register this handler with the operating system such that it takes control when the completion of an I/O request is signaled. This would allow the server to make asynchronous I/O calls that return immediately. When the request completes at a later time, the server is interrupted and put into its signal handler to process the results.
The big drawback for using asynchronous I/O in a serial server is in the complicated state management and synchronization problems that arise between the server and its signal handler. The program must keep track of the state of all in-progress requests. It must create and maintain locks for various resources (such as account records) so that they are not simultaneously accessed in program and signal contexts. Finally, the clean division of the program into modules breaks down. The communication, server, and database modules all get mixed together.
All in all, the experiment of using asynchronous I/O in a serial ATM server is a good argument for designing such a server to use threads. It's much cleaner to let a single thread wait for I/O to complete than it is to manage the complexity of signals and synchronization.
The serial version of our ATM server works—in fact, it works quite well—when the input stream of requests is light. However, the performance of the serial server degrades rapidly as more and more clients request access to its data. Clients begin to see longer and longer delays in the processing of their requests because all are blocked by server access to the database. In Chapter 6, we'll run some tests on single-threaded and multithreaded versions of the server that show the point at which it becomes inefficient to use the serial version.
What can we do to improve the performance of our server underload? It can help a lot to allow it to move on to another client request while its I/O to the database is proceeding. The next versions of our server will do just that.
The Multithreaded ATM Server
Let's add threads to our example. We'll begin by identifying those tasks we want individual threads to process. Having each request processed by a separate thread may or may not be a good starting point.
Before we pursue our design, let's step back and look again at the general criteria for selecting tasks for threads. In general we'd like to select tasks for our ATM server's threads based on whether:
 They are independent of each other.
If we assume that simultaneous accesses to the same account are rare, it makes sense to have each request processed by a separate thread. Threads will not compete for account data. No individual thread will rely on the work accomplished by another thread to complete its work.
 They can become blocked in potentially long waits.
This is true of all requests to our server, because any access to the account database could involve disk access to a file.
 They can use a lot of CPU cycles.
Our server contains no tasks that can be defined as compute intensive.
 They must respond to asynchronous events.
This is true of the manner in which the communication layer of our server accepts client requests.
 They require scheduling.
In our first pass at a multithreaded version of our ATM server we won't use scheduling. But we can imagine that certain operations could be given higher priority than others. A thread handling a shutdown request might be given priority over other requests. If we used the database to generate monthly account statements, we could give those requests lower priority than direct customer requests to bank accounts.
We must also bear in mind some key constraints our ATM server places on our program design:
 We must maintain correctness.
For the multithreaded version of our ATM server to produce correct and consistent results, we must ensure that two threads don't corrupt an account by writing information to it simultaneously. Thus, we'll use locks to protect the account data.
 We must maintain the liveliness of the threads.
We must avoid those types of programming bugs in which a worker thread obtains a lock on account data and then exits without releasing the lock. Other worker threads that subsequently attempt to obtain the lock on the same data will deadlock, waiting forever.
 We must minimize overhead.
Our threads can't spend all their time synchronizing with each other or they are not worth their overhead. As a simplification, we'll start out in our example by allocating a thread for each request. We can later enhance it by allowing threads to remain active, waiting for new requests (that is, we could use a thread pool).
Model: boss/worker model
Because it's a classic server program, we'll use the boss/worker model for our ATM server. A boss thread accepts input from remote clients through the communication module. Worker threads handle each client account request.
Figure 2-7 shows the structure of our ATM example program using the boss/worker ATM server. The boss thread is neatly encapsulated by the server's main routine. Each worker thread runs one service routine: deposit, withdraw, and so on.
Figure 2-7: The boss/worker Pthreads ATM Server
The boss thread
We'll start building our multithreaded ATM server's boss thread from our serial server's main routine. The boss thread simply manages the receipt of incoming requests using the server_comm_get_request routine. After it obtained each request from the communication module, the serial server's main routine unpacked it and called the appropriate service routine. The boss thread's main routine will create a worker thread to which it will pass the request. The worker thread begins by executing a generic request-processing routine called process_request, as shown in Example 2-7.
Example 2-7: Multithreaded ATM Server: boss Thread (atm_svr.c)
   typedef struct workorder{
          int conn;
          char req_buf[COMM_BUF_SIZE];
          } workorder_t;
  extern int
  main(argc, argv)
  int argc;
  char **argv;
  {
    workorder_t *workorderp;
    pthread_t   *worker_threadp;
    int  conn;
    int  trans_id;
    atm_server_init(argc, argv);
    for(;;) {
      /*** Wait for a request ***/
      workorderp = (workorder_t *)malloc(sizeof(workorder_t));
      server_comm_get_request(&workorderp->conn, workorderp->req_buf);
      sscanf(workorderp->req_buf, "%d", &trans_id);
      if (trans_id == SHUTDOWN) {
               .
               .
               .
               break;
               }
      /*** Spawn a thread to process this request ***/
      worker_threadp=(pthread_t *)malloc(sizeof(pthread_t));
      pthread_create(worker_threadp, NULL, process_request, (void *)workorderp);
      pthread_detach(*worker_threadp);
      free(worker_threadp);
    }
    server_comm_shutdown();
    return 0;
  }
Dynamically detaching a thread
In the code for our boss thread's main routine, we've introduced a new Pthreads call—pthread_detach. The pthread_detach function notifies the Pthreads library that we don't want to join our worker threads: that is, we will never request their exit status. If we don't explicitly tell the Pthreads library that we don't care about a thread's exit status, it'll keep the shadow of the thread alive indefinitely after the thread terminates (in the same way that UNIX keeps the status of zombie processes around). Detaching our worker threads frees the Pthreads library from storing this information, thus saving space and time. We are still responsible for freeing any space we dynamically allocated to hold the pthread_t itself.
Aside from using pthread_detach on an existing thread, you can create threads already in the detached state. We'll discuss this method in Chapter 4, Managing Pthreads.
A worker thread
In our multithreaded ATM server, each worker thread begins its life in a new request-parsing routine called process_request. This is a generic request-parsing routine that all workers use regardless of which requests they actually process. Because different service routines process different requests, the primary job of process_request is to select the proper service routine. We accomplish this by means of a simple case statement, shown in Example 2-8.
Example 2-8: Multithreaded ATM Server: Worker Thread process_request
   void process_request(workorder_t *workorderp)
  {
    char resp_buf[COMM_BUF_SIZE];
    int  trans_id;
    sscanf(workorderp->req_buf, "%d", &trans_id);
    switch(trans_id) {
        case CREATE_ACCT_TRANS:
             create_account(resp_buf);
             break;
        case DEPOSIT_TRANS:
             deposit(workorderp->req_buf, resp_buf);
             break;
        case WITHDRAW_TRANS:
             withdraw(workorderp->req_buf, resp_buf);
             break;
        case BALANCE_TRANS:
             balance(workorderp->req_buf, resp_buf);
             break;
        default:
             handle_bad_trans_id(workorderp->req_buf, resp_buf);
             break;
        }
    server_comm_send_response(workorderp->conn,
                           resp_buf);
    free(workorderp);
  }
In our ATM example, the boss thread is always active. It creates worker threads, as needed, to process requests. Each active worker could be processing a request on a different account, or each worker could be performing a separate operation on the same account. It shouldn't matter to our program. The boss thread limits the number of active worker threads in the server.
At any given time in the ATM example, a request could be in one of three places:
 Queued at the server's communication module, waiting to be picked up by the boss thread
 In the boss thread's hands, about to be passed off to a worker thread
 In the hands of a worker thread, being processed
Synchronization: what's needed
So far, we haven't shown any synchronization between the threads in our multithreaded ATM server. We'll go into the details in Chapter 3, Synchronizing Pthreads, and Chapter 4.
Right now we'll just list what synchronization we'll need:
 Accounts
Now that we have multiple workers accessing the database through the service routines (deposit, withdraw, and balance), we'll need to deal with the possibility that two routines may try to manipulate the same account balance at the same time. To prevent simultaneous access, we'll protect database accesses with a mutex variable.
 Limiting the number of workers
To keep from overloading the CPUs, the boss must limit the number of worker threads that can exist concurrently. It must maintain an ongoing count of worker threads and decrement the count as threads exit. We'll do that and add a check for exiting worker threads.
 Server shutdown
The ATM client lets privileged users shut down the server. To make our server more robust, we must ensure that the server has completed the requests that are already in progress before it stops accepting new requests and shuts itself down. We'll do this by adding code so that the boss can tell when threads are active.
Future enhancements
We'll add the synchronization we discussed to our multithreaded ATM server in Chapter 3. We'll also enhance our server throughout the remainder of this book. Among the design refinements we'll consider are:
 Thread pools
Our ATM server creates a worker thread each time it receives a request and pays the cost of thread creation each time. What if we allowed our server to reuse worker threads? When the server starts, it can create a predetermined number of workers in an idle state. Each worker thread could take requests off a queue and return to an idle state (instead of exiting) after completing each request. The reduction in overhead would pay off in performance.
 Cancellation
In a couple of situations it would be useful if the boss thread could interrupt and terminate a worker thread: to cancel an in-progress request that is no longer wanted or to support a quick shutdown.
 Scheduling
We could give some threads—possibly shutdown threads and deposit threads—priority over other threads. When a CPU becomes available, we could give these threads first crack at it.

Previous SectionNext Section
Books24x7.com, Inc © 2000 –  Feedback