Network Programming with Windows Sockets : A Thread-Safe DLL for Socket Messages

10/10/2010 3:16:36 PM

Program 1 is the DLL containing two character string (“CS” in names in this example) or socket streaming functions: SendCSMessage and ReceiveCSMessage, along with a DllMain entry point .

Program 1. SendReceiveSKST: Thread-Safe DLL
/* SendReceiveSKST.c -- Multithreaded streaming socket DLL. */
/* Messages are delimited by null characters ('\0') */
/* so the message length is not known ahead of time. Incoming */
/* data is buffered and preserved from one function call to */
/* the next. Therefore, use Thread Local Storage (TLS) */
/* so that each thread has its own private "static storage." */

#include "Everything.h"
#include "ClientServer.h"/* Defines MESSAGE records. */

typedef struct STATIC_BUF_T {
/* "staticBuf" contains "staticBufLen" characters of residual data */
/* There may or may not be end-of-string (null) characters */
char staticBuf[MAX_RQRS_LEN];
LONG32 staticBufLen;

static DWORD tlsIndex = TLS_OUT_OF_INDEXES; /* Initialize TLS index */
/* A single threaded library would use the following:
static char staticBuf[MAX_RQRS_LEN];
static LONG32 staticBufLen;

/* number of attached, detached threads and processes. */
static volatile long nPA = 0, nPD = 0, nTA = 0, nTD = 0;

/* DLL main function. */
LPVOID lpvReserved)

switch (fdwReason) {
tlsIndex = TlsAlloc();
InterlockedIncrement (&nPA);
/* There is no thread attach call for other threads created
BEFORE this DLL was loaded.
Perform thread attach operations during process attach
Load this DLL before creating threads that use DLL. */

/* Indicate that memory has not been allocated */
InterlockedIncrement (&nTA);
/* Slots are initialized to 0 */
return TRUE; /* This value is ignored */

/* Free remaining resources this DLL uses. Some thread
DLLs may not have been called. */
InterlockedIncrement (&nPD);
/* Count this as detaching the primary thread as well */
InterlockedIncrement (&nTD);
pBuf = TlsGetValue (tlsIndex);
if (pBuf != NULL) {
free (pBuf);
pBuf = NULL;
return TRUE;

/* May not be called for every thread using the DLL */
InterlockedIncrement (&nTD);
pBuf = TlsGetValue (tlsIndex);
if (pBuf != NULL) {
free (pBuf);
pBuf = NULL;
return TRUE;

default: return TRUE;

BOOL ReceiveCSMessage (MESSAGE *pMsg, SOCKET sd)
/* FALSE return indicates an error or disconnect */
BOOL disconnect = FALSE;
LONG32 nRemainRecv, nXfer, k; /* Must be signed integers */
LPBYTE pBuffer, message;

if (pMsg == NULL) return FALSE;
pBuff = (STATIC_BUF *) TlsGetValue (tlsIndex);
if (pBuff == NULL) { /* First time initialization. */
/* Only threads that need this storage will allocate it */
pBuff = malloc (sizeof (STATIC_BUF));
if (pBuff == NULL) return FALSE; /* Error */
TlsSetValue (tlsIndex, pBuff);
pBuff->staticBufLen = 0; /* Intialize state */

message = pMsg->record;
/* Read up to the null character, leaving residual data
* in the static buffer */

for (k = 0; k < pBuff->staticBufLen && pBuff->staticBuf[k] != '\0';
k++) {
message[k] = pBuff->staticBuf[k];
} /* k is the number of characters transferred */
if (k < pBuff->staticBufLen) { /* a null was found in staticBuf */
message[k] = '\0';
pBuff->staticBufLen -= (k+1); /* Adjust the buffer state */
memcpy (pBuff->staticBuf, &(pBuff->staticBuf[k+1]),
return TRUE; /* No socket input required */

/* the entire static buffer was transferred. No null found */
nRemainRecv = sizeof(tempBuff) -
sizeof(CHAR) - pBuff->staticBufLen;
pBuffer = message + pBuff->staticBufLen;
pBuff->staticBufLen = 0;

while (nRemainRecv > 0 && !disconnect) {
nXfer = recv (sd, tempBuff, nRemainRecv, 0);
if (nXfer <= 0) {
disconnect = TRUE;

/* Transfer to target message up to null, if any */
for (k = 0; k < nXfer && tempBuff[k] != '\0'; k++) {
*pBuffer = tempBuff[k];
nRemainRecv -= nXfer; pBuffer++;
if (k < nXfer) { /* null has been found */
*pBuffer = '\0';
nRemainRecv = 0;
/* Adjust the static buffer state for the next
* ReceiveCSMessage call */
memcpy (pBuff->staticBuf, &tempBuff[k+1], nXfer - k - 1);
pBuff->staticBufLen = nXfer -k - 1;
return !disconnect;

BOOL SendCSMessage (MESSAGE *pMsg, SOCKET sd)
/* Send the the request to the server on socket sd */
BOOL disconnect = FALSE;
LONG32 nRemainSend, nXfer;
LPBYTE pBuffer;

if (pMsg == NULL) return FALSE;
pBuffer = pMsg->record;
if (pBuffer == NULL) return FALSE;
nRemainSend = min(strlen (pBuffer) + 1, MAX_MESSAGE_LEN);

while (nRemainSend > 0 && !disconnect) {
/* send does not guarantee that the entire message is sent */
nXfer = send (sd, pBuffer, nRemainSend, 0);
if (nXfer <= 0) {
disconnect = TRUE;
nRemainSend -=nXfer; pBuffer += nXfer;
return !disconnect;

The DllMain function is a representative solution of a multithreaded persistent state problem, and it combines TLS and DLLs. The resource deallocation in the DLL_THREAD_DETACH case is especially important in a server environment; without it, the server would eventually exhaust resources, typically resulting in either failure or performance degradation or both. Note: This example illustrates concepts that are not directly related to sockets, but it is included here, rather than in earlier chapters, because this is a convenient place to illustrate thread-safe DLL techniques in a realistic example.

If this DLL is to be loaded dynamically, you must load it before starting any threads that use the DLL; otherwise, there will not be a DLL_THREAD_ATTACH call to DllMain.


Comments on the DLL and Thread Safety

  • DllMain, with DLL_THREAD_ATTACH, is called whenever a new thread is created, but there is not a distinct DLL_THREAD_ATTACH call for the primary thread or any other threads that exist when the DLL is loaded. The DLL’s DLL_PROCESS_ATTACH case must handle these cases.

  • In general, and even in this case (consider the accept thread), some threads may not require the allocated memory, but DllMain cannot distinguish the different thread types. Therefore, the DLL_THREAD_ATTACH case does not actually allocate any memory, and there is no need to call TlsSetValue because Windows initializes the value to NULL. The ReceiveCSMessage entry point allocates the memory the first time it is called. In this way, the thread-specific memory is allocated only by threads that require it, and different thread types can allocate exactly the resources they require.

  • While this DLL is thread-safe, a given thread can use these routines with only one socket at a time because the persistent state is associated with the thread, not the socket. The next example addresses this issue.

  • The DLL source code on the Examples file is instrumented to print the total number of DllMain calls by type.

  • There is still a resource leak risk, even with this solution. Some threads, such as the accept thread, may never terminate and therefore will never be detached from the DLL. ExitProcess will call DllMain with DLL_PROCESS_DETACH but not with DLL_THREAD_DETACH for threads that are still active. This does not cause a problem in this case because the accept thread does not allocate any resources, and even memory is freed when the process terminates. There would, however, be an issue if threads allocated resources such as temporary files; the ultimate solution would be to create a globally accessible list of resources. The DLL_PROCESS_DETACH code would then have the task of scanning the list and deallocating the resources; this is left as an exercise.

  •  Network Programming with Windows Sockets : In-Process Servers
  •  Network Programming with Windows Sockets : A Socket-Based Server with New Features
  •  Network Programming with Windows Sockets : A Socket-Based Client
  •  Network Programming with Windows Sockets : A Socket Message Receive Function
  •  Exchange Server 2010 : Operating Without Traditional Point-in-Time Backups
  •  Exchange Server 2010 : Performing Backup and Recovery for Mailbox Server Roles
  •  Exchange Server 2010 : Performing Backup and Recovery for Non-Mailbox Server Roles
  •  Exchange Server 2010 : Backup and Disaster Recovery Planning
  •  Changes to Backup and Restore in Exchange Server 2010
  •  Programming Windows Azure : Using the SDK and Development Storage
  •  Programming Windows Azure : Building a Storage Client
  •  Working with the REST API
  •  Excel Programmer : Fix Misteakes
  •  Excel Programmer : Change Recorded Code
  •  Excel Programmer : Record and Read Code
  •  Configuring Server Roles in Windows 2008 : New Roles in 2008
  •  Windows Server 2003 : Creating and Configuring Application Directory Partitions
  •  Windows Server 2003 : Configuring Forest and Domain Functional Levels
  •  Windows Server 2003 : Installing and Configuring Domain Controllers
  •  Manage Server Core
    Top 10
    Fujifilm XF1 - The Stylish Shooter
    Nikon 1 V2 - Still Fast and Handles Better
    Asustor AS-604T 4-Bay NAS Review (Part 3)
    Asustor AS-604T 4-Bay NAS Review (Part 2)
    Asustor AS-604T 4-Bay NAS Review (Part 1)
    Toshiba Satellite U925t Review (Part 3)
    Toshiba Satellite U925t Review (Part 2)
    Toshiba Satellite U925t Review (Part 1)
    iBall Andi 4.5H - Pretty In White
    The HTC Butterfly - Full HD In 5 Inches Only
    Most View
    Windows 7 : Command-Line and Automation Tools - Batch Files, Windows PowerShell
    IIS 7.0 : Managing Configuration - Backing Up Configuration, Using Configuration History & Exporting and Importing Configuration
    SharePoint 2010: Business Connectivity Services - The Secure Store Service (part 2) - Creating a Secure Store Service Application for Impersonating
    Reporting Services with SQL Azure : Starting a SQL Azure–Based Report
    Windows Server 2008 : Transport-Level Security - Active Directory Rights Management Services
    Who’s Watching You? (Part 4)
    Automating Blind SQL Injection Exploitation
    Partitioning Disks and Preparing Them for Use in Vista
    AMD Radeon HD 7850 2GB vs. Nvidia GeForce GTX 660 2GB vs. AMD Radeon HD 7870 2GB (Part 1)
    SharePoint 2010 : Workflow Modeling and Development Tools (part 1) - Microsoft Visio 2010 & SharePoint Designer 2010
    Sony VAIO Duo 11 - The Notebook-Tablet Hybrid
    Programming .NET Components : Serialization Events (part 3) - Type-Version Tolerance
    All You Need To Know About iOS 6 (Part 3)
    Windows Phone 8 Operating System Review – Part2
    Transferring Bookmarks With Internet Usernames And Passwords To A New PC
    Advanced ASP.NET : Data Caching (part 1) - Adding Items to the Cache & A Simple Cache Test
    Asus Rog Tytan CG8565 - Clash Of The Tytan
    2012 - The Year to Come (Part 2)
    Compact System Cameras : Samsung NX200, Nikon 1J1, Olympus PEN E-PM1, Panasonic GX1, Sony NEX-5N
    Nokia Lumia 620 Review - Basic Smartphone With Good Performance And Stunning Design (Part 1)