Server-side development experience summary Linux C language

After doing server-side development, you need to consider some algorithms and performance issues. After several years of development, you have some experience in this area. Now write it down and share it with you.

I mainly develop C language under Linux, so the latter implementation is based on Linux operating system
and explained in C language. Other platforms and languages ​​need to be considered similarly, but there may be some differences in implementation details. I try to reduce these differences. Note that all the content explained is based on the development of 32-bit systems!

The core of server program development is stable, and efficiency needs to be considered under the premise of stability. The main public modules are the memory pool and the thread pool. Because server programs generally run for a long time, and frequently create and release memory operations, then if you use the system's malloc and free methods, it will cause a lot of memory fragmentation in the system, which affects efficiency and stability. The main idea of ​​the memory pool is to first call the system's malloc to open up a large amount of memory, and then manage this large memory. When the memory is used in the program, the memory pool allocates memory. When the program releases the memory, it only informs the memory pool. Does not really free the memory. Thread pool, the principle is similar, when dealing with some tasks concurrently, you need to use a lot of threads, we can create a lot of threads first, and then each time there is a task to be processed, then find a free thread to let it handle the task, after the completion of the thread hangs Start. This saves the time overhead of creating a thread each time a task is created.

Preparatory Knowledge

The following is a description of some of the techniques used in the encapsulation of libraries. Because we are developing in C language, in order to balance the structure and efficiency, we need some skills to encapsulate, so that the program is modular enough to ensure the efficiency of C language.

container_of macro

First introduce this macro, this macro is what I saw in the Linux kernel, it is used to derive the entire structure through a pointer to a member of a structure Pointer, let's take an example. For example, there is one structure:
1: struct my_struct 2: { 3: int a; 4: int b; 5: char c; 6: short d; 7: };

Take a look at the following code: Br> 1: int 2: main () 3: { 4: struct my_struct mys; //create structure object 5: struct my_struct* pmys; //declare a pointer to our structure 6: char* pc = &( Mys.c); //pc points to the c member of the structure 7: int* pb = &(mys.b); //pb points to the b member of the structure 8: pmys = container_of (pc, struct my_struct, c); //pmys actually points to mys structure 9: pmys = container_of (pb, struct my_struct, b); //pmys still points to mys structure 10: //The above two lines of code are the same as pmys = &mys; 11: .. 12: }

The above code shows that a pointer to a member of a structure (such as pc or pb) can be obtained by calling container_of to get a pointer to the entire structure. This macro takes three arguments, the first argument is a pointer to a member of the structure, the second argument is the type of the construct, and the third argument is the declaration of the member pointed to by the member pointer in the first argument. The famous name in . This macro is defined as follows:
1: #define offsetof(TYPE, MEMBER)((size_t) &((TYPE *)0)->MEMBER) 2: 3: #ifdef WIN32 4: //under WIN32 ( Type safety check is not supported) 5: #define container_of(ptr, type, member) /6: ((type*)(ptr)-offsetof(type, member)) 7: 8: #else 9: //10 under linux : #define container_of(ptr, type, member)({/11: const typeof( ((type *)0)->member) *_mptr = (ptr); /12: (type *)((char*) _mptr-offsetof(type, member));}) 13: 14: #endif /* */

In the above implementation, an offsetof macro is first defined. This macro is used to calculate a member of the structure. The offset of the address relative to the starting address of this structure. Under the WIN32, you can use the system's own offset macro.

The container_of macro actually uses the address of a member of the structure minus the offset of the member relative to the start of the structure, and the address of the structure is the pointer to the structure. When implemented under linux: const typeof( ((type *)0)->member)* _mptr = (ptr); This sentence is actually doing type safety checks. The type of the pointer to the first parameter member of the macro is guaranteed to be a pointer to this member type. Typeof is a gcc extension that is used to get the type of an expression. I did not find a substitute for typeof under WIN32, so I did not do a type safety check.

With the container_of macro, what can we do? For example, if we write a linked list, the data area in the linked list is generally treated as an int in the textbook. It may actually be a complicated structure, and as a person writing a linked list, I don't know what data the user will store in the linked list. The usual practice is that the data area is a void* so that the user can use this pointer to store any object. If the user wants to store his own data, he can create an object with a custom structure and then use this pointer to point to the structure. Each element in the linked list occupies a void* in addition to the memory occupied by the data area. When developing on the server side, the amount of data is large, which wastes a lot of void*. We can solve this problem by the following methods.

Defining the structure of the elements in the linked list:
1: struct link_item 2: { 3: struct link_item* next; 4: };

The methods provided for the operation of the linked list are based on struct link_item* The method of the pointer. The user declares his own linked list element structure when it is used:
1: struct user_link_item 2: { 3: struct link_item lk_item; 4: int my_data1; 5: short my_data2; 6: //... 7: }; p> This structure has a member lk_item of type struct link_item. When we use a linked list, we always pass the address of lk_item to the method of the linked list. Obtaining the data is also a pointer to the struct link_item, but we can use the container_of macro to use the pointer to the struct link_item returned by the linked list method to derive the pointer of the struct user_link_item, so that the user can obtain the data stored in the linked list. The memory of the specific elements in such an implementation is also allocated by external users.

This method is somewhat similar to the C++ inheritance mechanism. You can consider using this method when you encounter inheritance problems.

Reference Counters

When developing server programs, we are often multi-threaded parallel processing. There is a thread-safe problem with the data to be processed. The simplest thread-safe processing is atomic operations. Each processing is done using a single instruction. The second is to use thread locks for thread synchronization. For a data stored in a data structure, it may be read in one thread, and the data is deleted in another thread. We can mimic the method used by WIN32 COM to handle this problem. Add a reference counter, increment the reference counter of the data each time you want to access the data. When the usage is complete, decrement the reference counter of the data. When the value of the reference counter is decremented to 0, the data is actually deleted. Of course, the incrementing and decrementing operations of the reference counter need to be thread-safe and can be implemented entirely using atomic operations.

All data objects that need to be thread-safe can be implemented by the inheritance of the implementation of the container_of macro mentioned above.

Structural Processing of Variable Length Fields in Communication Protocols

Server-side communication usually designs a set of protocols. In the agreement is usually the form: