Linux Kernel Networking
Programming Class
TARGETED
AT:
* C-language programmers
* Network application developers, and those porting to Linux
* Network device driver developers, and those porting to Linux
* Systems programmers who want a grasp of kernel internals
* Kernel programmers who want a good grasp of networking internals
NOT
TARGETED AT:
* Networking administrators with not much programming experience
* Programmers used to working in only Java or C++
* Anyone looking for an introduction to Linux
* Anyone looking to solve the "problem du jour" with a magic bullet
* Anyone looking for the bleeding edge
WHAT
THIS COURSE IS AND IS NOT:
Linux Kernel Network Programming class is a follow on to Linux kernel
and drivers classes.
It is designed to prepare experienced programmers to write networking
applications for Linux, and of furnishing them with a solid
understanding of the implementation of networking layer of the Linux kernel.
This class will take a layered approach to networking, working inward
from application (socket) programming methods, to the network and
transport layers, and finally through the device (datalink) layer.
While this is an ambitious class of the most advantage to those students
who already have a good knowledge of operating system architecture in
general, and Linux in specific, other students will still derive solid
benefits and will walk away with a good grasp of Linux and its
networking facilities.
This is NOT a class in network administration, performance tuning, or
kernel hacking. We will not discuss specialized hardware or cutting
edge technologies. This IS a class that will give students the
framework to pursue those areas with a good foundation.
COURSE
OBJECTIVES:
Linux Kernel Network Programming is designed to prepare experienced
programmers to write networking applications for Linux, and of
furnishing them with a solid understanding of the implementation of
networking layer of the Linux kernel.
There are three principal sub-areas which we will address:
1. The application layer: how to write socket-based applications,
dealing with addressing issues and writing both client and server
applications.
2. The network and transport layers: How the Linux kernel is structured
to deal with the various protocols and routing issues.
3. The device driver and datalink layers: How networking device drivers
are written for Linux and hardware protocols are satisfied.
In order to do this we will also have a brief introduction to Linux
kernel programming, including basic architecture, methods, and
modularization.
This class is not a general class on networking; we are going to
concentrate on Linux-specific issues.
This class is not a class on kernel hacking; we don't expect you to come
out of this fully prepared to join the upper ranks of kernel developers.
Instead, you should have a thorough grasp of the whole picture from
which you can embark down that road.
This class is not a class on performance tuning. Such a class would be
very different, and not be as code-based.
Networking is a very big subject and we can not cover all of the
details. However, upon completion of this class students should have
enough of a grasp of how Linux networking is accomplished to study
particularities and many implementation details we will not cover.
There is more printed material than we can possibly cover in class; it
is here for background reference and for flexibility in choice of topics
to suit your interests and needs.
PREREQUISITES:
Students should:
* Be proficient in the C programming language.
* Be familiar with basic Linux (Unix) utilities, such as ls, rm, grep,
tar, and have a familiarity with command shells and scripts.
* Be comfortable using any of the available text editors (i.e., emacs, vi.)
* Have seen the X-Window system before.
* Know the basics of compiling and linking programs, constructing
Makefiles etc.; i.e., be comfortable doing application developing in a
Linux or Unix environment.
* Have a good understanding of systems programming in a Unix or Linux
environment, from the stand point of writing applications.
Students who have:
1. some experience configuring and compiling kernels, and writing
kernel modules and or device drivers;
2. a good grasp of operating system fundamentals, and familiarity with
the insides of any other operating system;
3. a good knowledge of networking fundamentals...will gain much more
from this class.
4. Ambitious students could be weak on any of the above except
familiarity with the C language, and still profit from this class.
PROCEDURES:
* Each student will have a computer installed with the current Red Hat
Linux distribution.
* The emphasis will be on hands-on programming, with the time divided
roughly equally between lecture and labs.
* Where feasible labs will build upon previous lab assignments.
* Lab solutions will be made available so students can see at least one
successful implementation, and have a possible template to begin the
next lab exercise if it is a follow up.
* In addition, examples as shown during the lectures will also be made
available as part of the solutions package, in the EXAMPLES
subdirectory.
* If you have any questions or feedback on the course material after
completion of the class, you can contact your instructor, or the
courseware maintainer, Jerry Cooperstein at Coop@axian.com.
Linux Kernel Networking
Programming Class
Course Outline
Session 01: Introduction
1.1 Overview
1.2 Course Objectives
1.3 Prerequisites
1.4 Procedures
1.5 Kernel Versions and Backwards Compatibility
1.6 Platforms
1.7 Documentation
Session 02: Networking
2.1 Overview
2.2 Network Programming
2.3 Historical Considerations
2.4 Standards
2.5 Layers
2.6 Transport and Network Layers and Encapsulation
2.7 Datalink Layer and Device Drivers
2.8 Summary
2.9 Lab
Session 03: Sockets
3.1 Overview
3.2 What are Sockets?
3.3 Stream Sockets
3.4 Datagram Sockets
3.5 Raw Sockets
3.6 Byte Ordering
3.7 Summary
3.8 Lab
Session 04: Sockets - Addresses and Hosts
4.1 Overview
4.2 Socket Address Structures
4.3 Converting IP Addresses
4.4 Host Information
4.5 Summary
4.6 Lab
Session 05: Sockets - Service Port and Protocol Information
5.1 Overview
5.2 Service Port Information
5.3 Protocol Information
5.4 Summary
5.5 Lab
Session 06: Sockets - Clients
6.1 Overview
6.2 Basic Client Sequence
6.3 socket()
6.4 connect()
6.5 close() and shutdown()
6.6 Unix Client
6.7 Internet Client
6.8 Summary
6.9 Lab
Session 07: Sockets - Servers
7.1 Overview
7.2 Basic Server Sequence
7.3 bind()
7.4 listen()
7.5 accept()
7.6 Unix Server
7.7 Internet Server
7.8 Summary
7.9 Lab
Session 08: Sockets - Input/Output Operations
8.1 Overview
8.2 write(), read()
8.3 send(), recv()
8.4 sendto(), recvfrom()
8.5 sendmsg(), recvmsg()
8.6 sendfile()
8.7 socketpair()
8.8 Summary
8.9 Lab
Session 09: Sockets - Options
9.1 Overview
9.2 Getting and Setting Socket Options
9.3 fcntl()
9.4 ioctl()
9.5 getsockopt(), setsockopt()
9.6 Summary
9.7 Lab
Session 10: Sockets - select() and poll()
10.1 Overview
10.2 Multiplexed and Asynchronous Socket I/O
10.3 select()
10.4 pselect()
10.5 poll()
10.6 Signal Driven and Asynchronous I/O
10.7 Concurrent Servers
10.8 Summary
10.9 Lab
Session 11: Sockets - IPV6
11.1 Overview
11.2 What is IPV6?
11.3 Main Features
11.4 Main Functions
11.5 Writing Portable Code
11.6 Summary
11.7 Lab
Session 12: Kernel Layout and Configuration
12.1 Overview
12.2 Layout of the Kernel Source
12.3 Mixing User and Kernel Space Headers
12.4 Making a Kernel Patch
12.5 Kernel Configuration
12.6 Summary
12.7 Lab
Session 13: Modules
13.1 Overview
13.2 What are Modules?
13.3 A Simple Example
13.4 Compiling Modules
13.5 Modules vs Built-in
13.6 Loading and Unloading Modules
13.7 Module Usage Count
13.8 Module Licensing
13.9 Exporting Symbols
13.10 Summary
13.11 Lab
Session 14: Kernel Programming Preview
14.1 Overview
14.2 Process Descriptor
14.3 Memory Allocation
14.4 Transferring Data between User and Kernel Space
14.5 Jiffies
14.6 Summary
Session 15: The proc Filesystem
15.1 Overview
15.2 What is the proc Filesystem?
15.3 Creating Entries
15.4 Reading Entries
15.5 Writing Entries
15.6 Summary
15.7 Lab
Session 16: Synchronization Methods
16.1 Overview
16.2 Atomic Functions
16.3 Bit Operations
16.4 Memory Barriers
16.5 Spinlocks
16.6 Semaphores
16.7 Completion Functions
16.8 Linked Lists
16.9 Summary
16.10 Lab
Session 17: Timing and Timers
17.1 Overview
17.2 Jiffies
17.3 Precise Timing
17.4 Inserting Delays
17.5 Kernel Timers
17.6 Summary
17.7 Lab
Session 18: Sleeping, Waking, Task Queues and Tasklets
18.1 Overview
18.2 Wait Queues
18.3 Tasklets
18.4 Task Queues
18.5 Summary
18.6 Lab
Session 19: Kernel Network Code
19.1 Overview
19.2 Network Code Roadmap
19.3 Object-Oriented Methods
19.4 Main Data Structures
19.5 socket structure
19.6 proto_ops structure
19.7 sock structure
19.8 softnet_data structure
19.9 Summary
19.10 Lab
Session 20: Connections
20.1 Overview
20.2 Establishing Connections
20.3 Socket Creation
20.4 Connecting, Binding, and Listening
20.5 Accepting Connections
20.6 Socket Disconnection
20.7 Summary
20.8 Lab
Session 21: Sending and Receiving Messages
21.1 Overview
21.2 I/O Through Sockets
21.3 Sending
21.4 Receiving
21.5 sendfile()
21.6 Summary
21.7 Lab
Session 22: Kernel Network Initialization
22.1 Overview
22.2 Basic Setup
22.3 Socket Initialization
22.4 Unix Protocol Initialization
22.5 Internet Protocol Initialization
22.6 Initializing TCP
22.7 Initializing softnet
22.8 Summary
22.9 Lab
Session 23: Packet Transmission
23.1 Overview
23.2 Getting Data from the Application
23.3 Sending a TCP/UDP Packet for AF_UNIX
23.4 Sending a TCP/UDP Packet for AF_INET
23.5 Getting Data to the Device Using softirqs
23.6 Summary
23.7 Lab
Session 24: Packet Reception
24.1 Overview
24.2 Getting Data to the Application
24.3 Receiving a TCP/UDP Packet for AF_UNIX
24.4 Receiving a TCP/UDP Packet for AF_INET
24.5 Getting Data from the Device Using softirqs
24.6 Summary
24.7 Lab
Session 25: IP Forwarding
25.1 Overview
25.2 What is IP Forwarding?
25.3 Examining and Deciding
25.4 Forwarding Packets
25.5 Summary
25.6 Lab
Session 26: Kernel Network Routing
26.1 Overview
26.2 Routing in Linux
26.3 Routing Tables
26.4 Forwarding Information Base
26.5 Routing Cache
26.6 Keeping Information up to Date
26.7 Summary
26.8 Lab
Session 27: Zero-Copy Networking
27.1 Overview
27.2 What is Zero-Copy Networking
27.3 Implementation
27.4 Summary
27.5 Lab
Session 28: select() and alternatives
28.1 Overview
28.2 Problems with select() and poll()
28.3 epoll()
28.4 kevent() and kqueue()
28.5 Using Real Time Signals
28.6 Summary
28.7 Lab
Session 29: Asynchronous I/O
29.1 Overview
29.2 What is Asynchronous I/O?
29.3 The Asyncrhonous I/O API
29.4 Linux Implementation
29.5 Summary
Session 30: In Kernel Web Server Acceleration
30.1 Overview
30.2 What is in-Kernel Web Service Acceleration?
30.3 khttpd
30.4 TUX
30.5 User-space Solutions
30.6 Summary
30.7 Lab
Session 31: Network File Systems
31.1 Overview
31.2 Available Network File Systems
31.3 RPC
31.4 NFS
31.5 NFS Versions
31.6 Summary
31.7 Lab
Session 32: Introduction to Network Drivers
32.1 Overview
32.2 Network Drivers
32.3 Loading/Unloading
32.4 Initialization
32.5 Summary
32.6 Lab
Session 33: Network Driver Data Structures
33.1 Overview
33.2 netdevice Structure
33.3 sk_buff Structure
33.4 Summary
33.5 Lab
Session 34: Transmission and Reception
34.1 Overview
34.2 Packet Transmission
34.3 Packet Reception
34.4 Summary
34.5 Lab
Session 35: Network Driver Facilities
35.1 Overview
35.2 Statistics
35.3 Multicasting
35.4 Changes in Link State
35.5 ioctls
35.6 Summary
35.7 Lab