OS Lab 7 - Sockets. Datagrams
In the last lab, we looked at how the kernel decides what a task may do with a file. In this lab, processes will talk to each other through sockets. We will use local datagram sockets, which carry whole messages, each with the address of its sender. With them, a server needs a single socket and a single thread to serve many clients at once. We will also see how a program can wait for the keyboard and for a socket at the same time, still with one thread. As an application, we will write MyChat, a small chat server and client, in the style of IRC (ancestor of WhatsApp, Discord, etc.).
Learning objectives
By the end of this lab you should be able to:
- create a local socket and give it an address;
- send packets to an address with
sendto(2), and receive them, with the address of their sender, withrecvfrom(2); - say what the flag
MSG_DONTWAITchanges; - serve many clients from a single socket;
- receive packets while
readline(3)waits for the user.
Sockets
A socket is one end of a communication. The process sees it as a file descriptor. Every socket has a family, which says where the other end is, and a type, which says how the data moves. In this lab we use a single combination:
AF_UNIX: both ends are on the same machine. The address is a path, and the kernel copies the data from one process to the other, without touching the disk.SOCK_DGRAM: the data travels in datagrams, or packets, which keep their boundaries. Onesendto(2)makes one packet, and onerecvfrom(2)takes one whole packet. There is no connection: every packet is sent to an address, and the receiver learns the address of the sender, so it can reply.
Every socket which receives packets has a queue in the kernel. Packets from all senders wait there, in the order in which they arrived, until the owner of the socket takes them. With AF_UNIX, no packet is lost.
One side, the server, has an address which everybody knows. The others, the clients, send packets to that address. Both sides make the same calls:
| Server | Client |
|---|---|
socket(2): a new socket |
socket(2)
|
bind(2): give it a known address |
bind(2): give it any free address, so that the server can reply
|
recvfrom(2): a packet, and who sent it |
sendto(2): a packet, to the address of the server
|
sendto(2): the reply, to that sender |
recv(2): the reply
|
Creating a socket
Both sides start in the same way: they create a socket, and fill in an address.
socket(2)
// In: <sys/socket.h>
int socket(int domain, int type, int protocol);
| Parameter | Description |
|---|---|
domain |
The family: AF_UNIX in this lab
|
type |
SOCK_DGRAM in this lab. Others are SOCK_STREAM (a connection which carries a stream of bytes, like a pipe) and SOCK_SEQPACKET (a connection which carries packets)
|
protocol |
0: the usual protocol for the family and type |
It returns a new file descriptor, or -1 on error. The new socket has no address yet.
Addresses
An AF_UNIX address is a path, kept in a struct sockaddr_un:
// In: <sys/un.h>
struct sockaddr_un {
sa_family_t sun_family; // AF_UNIX
char sun_path[108]; // The path of the socket file
};
Every family has its own structure, but the socket functions take any of them, as a struct sockaddr * and a size. We fill in our structure, and cast its address:
struct sockaddr_un addr = { .sun_family = AF_UNIX, .sun_path = "upper.sock" };
bind(s, (struct sockaddr *)&addr, sizeof addr);
A relative path, like the one above, is relative to the current directory of the process.
bind(2)
// In: <sys/socket.h>
int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen);
bind(2) gives the socket an address, so that others can send packets to it.
On the server, the address is a path. bind(2) creates a socket file at that path, whose type ls -l shows as s. The file is only a name: no data is ever written to it. To send to the socket, a process needs write permission on this file.
bind(2) fails with EADDRINUSE if the path exists. That is why servers usually call unlink(2) on the path first.A client does not care about its own address, but it needs one: without it, the server could not reply. If addrlen covers only sun_family, the kernel picks a free name, such as @a50a0. This name is not a file, and it goes away when the socket is closed:
struct sockaddr_un me = { .sun_family = AF_UNIX };
bind(s, (struct sockaddr *)&me, sizeof(sa_family_t)); // Any free name
sendto(2) and recvfrom(2)
// In: <sys/socket.h>
ssize_t sendto(int sockfd, const void *buf, size_t len, int flags,
const struct sockaddr *dest_addr, socklen_t addrlen);
ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags,
struct sockaddr *src_addr, socklen_t *addrlen);
ssize_t recv(int sockfd, void *buf, size_t len, int flags);
sendto(2)sendslenbytes as one packet, to the socket atdest_addr.recvfrom(2)takes one packet from the queue, and returns its size. It blocks while the queue is empty. If the packet is larger thanlen, the rest of it is lost.recvfrom(2)also tells us who sent the packet. Before the call, we set*addrlento the size allocated for the structure atsrc_addr. The call writes the address of the sender into that structure, and sets*addrlento the real length of the address. To reply, we give the recieved address and length tosendto(2), asdest_addrandaddrlen.recv(2)isrecvfrom(2)without the address, for when we do not care who sent the packet.
sendto(2) fails if nobody can receive at dest_addr:
| Error | Meaning |
|---|---|
ENOENT |
There is no such socket file |
ECONNREFUSED |
The address exists, but no socket has it anymore: for example, the server has ended, and its socket file stayed |
EACCES |
We have no write permission on the socket file |
flags adds additional options to a single call. We will need one flag:
| Flag | Meaning |
|---|---|
MSG_DONTWAIT |
Do not block. If there is no packet to receive, or no room to send one, fail at once with EAGAIN
|
sendto(2) blocks until there is room again. A server must not stop because one client does not read, so it sends with MSG_DONTWAIT.The following program is a server which sends back every packet in upper case, and a client which connects to it and sends it its command line arguments:
1#include <ctype.h>
2#include <errno.h>
3#include <stdio.h>
4#include <stdlib.h>
5#include <string.h>
6#include <sys/socket.h>
7#include <sys/un.h>
8#include <unistd.h>
9
10#define PATH "upper.sock"
11
12static void print_error(const char *what)
13{
14 char msg[256];
15 snprintf(msg, sizeof msg, "%s: %s\n", what, strerror(errno));
16 write(STDERR_FILENO, msg, strlen(msg));
17}
18
19static int server(void)
20{
21 int s = socket(AF_UNIX, SOCK_DGRAM, 0);
22 if (s == -1) {
23 print_error("socket");
24 return EXIT_FAILURE;
25 }
26 struct sockaddr_un addr = { .sun_family = AF_UNIX, .sun_path = PATH };
27 unlink(PATH); // remove it in case it existed before
28 if (bind(s, (struct sockaddr *)&addr, sizeof addr) == -1) {
29 print_error("bind");
30 return EXIT_FAILURE;
31 }
32 for (;;) {
33 char buf[256];
34 struct sockaddr_un from;
35 socklen_t len = sizeof from;
36 ssize_t n = recvfrom(s, buf, sizeof buf, 0, (struct sockaddr *)&from, &len);
37 if (n == -1) {
38 print_error("recvfrom");
39 continue;
40 }
41 for (ssize_t i = 0; i < n; i++) {
42 buf[i] = toupper(buf[i]);
43 }
44 sendto(s, buf, n, 0, (struct sockaddr *)&from, len); // back to the sender
45 }
46}
47
48static int client(int count, char *words[])
49{
50 int s = socket(AF_UNIX, SOCK_DGRAM, 0);
51 if (s == -1) {
52 print_error("socket");
53 return EXIT_FAILURE;
54 }
55 struct sockaddr_un me = { .sun_family = AF_UNIX };
56 if (bind(s, (struct sockaddr *)&me, sizeof(sa_family_t)) == -1) { // any free name
57 print_error("bind");
58 return EXIT_FAILURE;
59 }
60 struct sockaddr_un server = { .sun_family = AF_UNIX, .sun_path = PATH };
61 for (int i = 0; i < count; i++) {
62 if (sendto(s, words[i], strlen(words[i]), 0, (struct sockaddr *)&server, sizeof server) == -1) {
63 print_error("sendto");
64 return EXIT_FAILURE;
65 }
66 char buf[256];
67 ssize_t n = recv(s, buf, sizeof buf, 0);
68 if (n == -1) {
69 print_error("recv");
70 return EXIT_FAILURE;
71 }
72 char line[300];
73 int len = snprintf(line, sizeof line, "%.*s\n", (int)n, buf);
74 write(STDOUT_FILENO, line, len);
75 sleep(1);
76 }
77 close(s);
78 return EXIT_SUCCESS;
79}
80
81int main(int argc, char *argv[])
82{
83 if (argc == 2 && strcmp(argv[1], "server") == 0) {
84 return server();
85 }
86 if (argc >= 2 && strcmp(argv[1], "client") == 0) {
87 return client(argc - 2, argv + 2);
88 }
89 const char *usage = "usage: app server | client <word>...\n";
90 write(STDERR_FILENO, usage, strlen(usage));
91 return EXIT_FAILURE;
92}
Start the server in the background with &, and run the client:
$ ./build/debug/app server &
$ ls -l upper.sock
srwxr-xr-x 1 john john 0 Nov 17 12:03 upper.sock
$ ./build/debug/app client hello socket world
HELLO
SOCKET
WORLD
Now start two clients at the same time:
$ ./build/debug/app client one two three & ./build/debug/app client four five six
ONE
FOUR
FIVE
TWO
THREE
SIX
Both clients are served at the same time, by a single thread. The server has one socket for all its clients, and takes the packets in the order in which they arrive, whoever sent them. The order of the lines may differ from one run to the next.
upper.sock. recvfrom(2) returns the first one, with the address of its sender, and the server replies to that address.Stop the server with kill %1, and run a client again. Its sendto(2) fails with ECONNREFUSED: the socket file is still there, but no process has a socket with that address anymore.
Waiting in readline
As part of this lab we will build a chat client.
A chat client must wait for two things at once: the user typing a line, and packets from the server. But readline(3) only returns when the user presses Enter. For this, readline can call a function of ours while it waits:
// In: <readline/readline.h>
extern int (*rl_event_hook)(void); // Called while readline waits for a key
extern int rl_done; // Set it to 1, and readline() returns
int rl_on_new_line(void); // Tells readline that the cursor is on a new, empty line
void rl_redisplay(void); // Draws the prompt and the line typed so far
While no key is pressed, readline(3) calls rl_event_hook about 10 times a second. The function must return quickly, so it must not block: it can receive with MSG_DONTWAIT, and stop when recv(2) fails with EAGAIN. Its return value is not used.
When the hook prints something, the cursor is after the prompt and the half-typed line. To print above them:
- write
"\r", which moves the cursor to the start of the line, and"\033[K", an escape sequence which asks the terminal to clear the rest of the line; - write our text, ending with
"\n"; - call
rl_on_new_line()andrl_redisplay(), so that readline draws the prompt and the typed text again, on the next line.
The following program prints the time every 3 seconds, while the user types:
1#include <stdio.h>
2#include <stdlib.h>
3#include <string.h>
4#include <time.h>
5#include <unistd.h>
6
7#include <readline/history.h>
8#include <readline/readline.h>
9
10static time_t last;
11
12static void print(const char *msg)
13{
14 write(STDOUT_FILENO, msg, strlen(msg));
15}
16
17// Prints a line above the prompt, and draws the prompt again below it.
18static void print_above(const char *msg)
19{
20 print("\r\033[K"); // Back to the start of the line, and clear it
21 print(msg);
22 rl_on_new_line();
23 rl_redisplay();
24}
25
26// readline() calls this about 10 times a second, while it waits for a key.
27static int on_idle(void)
28{
29 time_t now = time(nullptr);
30 if (now - last >= 3) {
31 last = now;
32 struct tm tm;
33 localtime_r(&now, &tm);
34 char msg[64];
35 strftime(msg, sizeof msg, "* %H:%M:%S\n", &tm);
36 print_above(msg);
37 }
38 return 0;
39}
40
41int main(void)
42{
43 last = time(nullptr);
44 rl_event_hook = on_idle;
45
46 char *line;
47 while ((line = readline("> ")) != nullptr) {
48 char msg[512];
49 snprintf(msg, sizeof msg, "you typed: %s\n", line);
50 print(msg);
51 free(line);
52 }
53 return EXIT_SUCCESS;
54}
$ ./build/debug/app
* 12:33:21
> hello wor
Start typing, and wait: the time is printed above the prompt, and what you typed is drawn again below it, so you can go on typing.
Exercises
Write the program in a new copy of the project template. As a reminder from the first lab:
mkdir -p ~/oslab && cd ~/oslab
curl -OL https://wiki.dcae.pub.ro/images/9/93/Os_lab_skeleton.zip
unzip -d lab07 Os_lab_skeleton.zip
For this lab:
- As in the last labs, do not use the standard I/O library for output: build the text with
snprintf(3)andwrite(2)it. Usereadline(3)for input. - Both the server and the client have a single thread: no
pthread_create(3), nofork(2). - The server has a single socket, and the only call in which it waits is
recvfrom(2).
MyChat
In this exercise you will write MyChat, a chat room for the users of one machine. A single program is both the server and the client, depending on its argument. To try it, start the server in one terminal, and each client in a terminal of its own, all in the folder of the project:
./build/debug/app --server
./build/debug/app --client
The client reads lines from the user. A client which has just started is not in the chat yet: the user must first pick a name with /join. The client should understand the following:
| The user types | What the client should do |
|---|---|
/join NAME |
Join the chat as NAME
|
/leave |
Leave the chat. The client keeps running, and can /join again
|
/dm NAME TEXT |
Send TEXT only to NAME
|
| Any other line | Send it to everyone in the chat |
| Ctrl+D | Leave the chat, if it joined, and end |
To understand each other, the server and the clients must agree on what a packet contains: they need a protocol. A real chat protocol, such as IRC, has to work between different programs on different machines, and takes many pages to describe.
The protocol we propose for this lab is a simple example: every packet is one struct myc_packet, sent as it is in memory. The structure is defined as follows:
// The socket file, in the current directory.
#define MYC_PATH "mychat.sock"
#define MYC_NAME_MAX 32 // Including the final '\0'
#define MYC_TEXT_MAX 256 // Including the final '\0'
enum myc_type {
// Sent by the user
MYC_U_JOIN = 1, // name: the nickname
MYC_U_LEAVE, // -
MYC_U_MSG, // text
MYC_U_DM, // name: to whom
// text
// Sent by the server
MYC_S_JOIN, // name: who joined
MYC_S_LEAVE, // name: who left
// clean: true if they sent MYC_U_LEAVE, false if they errored out
MYC_S_MSG, // name: from whom
// dm: if it was sent only to us
// text
MYC_S_ERR, // text: what went wrong
};
// Every packet has the same size, whatever its type.
struct myc_packet {
int type;
bool clean;
bool dm;
char name[MYC_NAME_MAX];
char text[MYC_TEXT_MAX];
};
In this protocol, every packet has space for all the fields, but each type uses only some of them:
| Type | Sent by | Fields | Meaning |
|---|---|---|---|
MYC_U_JOIN |
client | name |
I want to join with this name |
MYC_U_LEAVE |
client | I am leaving the chat | |
MYC_U_MSG |
client | text |
Send this to everyone |
MYC_U_DM |
client | name, text |
Send this only to name
|
MYC_S_JOIN |
server | name |
name joined. Sent to everyone in the chat, including the new user
|
MYC_S_LEAVE |
server | name, clean |
name left. clean is true if they sent MYC_U_LEAVE, and false if their client ended without it
|
MYC_S_MSG |
server | name, dm, text |
name wrote text. dm is true if it was sent only to us. A message is not sent back to its author
|
MYC_S_ERR |
server | text |
The last request failed, and why |
With this protocol, a session with a server and two clients, in three terminals, could look like this:
$ ./build/debug/app --server
serving on mychat.sock
ana joined
john joined
ana left
$ ./build/debug/app --client
> hello?
! join first: /join <name>
> /join ana
* ana joined
* john joined
> hi john
<john> hey ana
[dm] <john> a secret
> /dm nobody hi
! no such user: nobody
> /leave
* ana left
>
$ ./build/debug/app --client
> /join ana
! this name is taken
> /join john
* john joined
<ana> hi john
> hey ana
> /dm ana a secret
* ana left
> ^C
ana left with /leave and then Ctrl+D, and john with Ctrl+C. The server does not know yet that john is gone: no packet told it. The server prints whatever you find useful.
Build it in these steps:
- Talking. In
main(), pick the server or the client from the argument. The server creates the socket, removes an oldmychat.sock, binds, and receives packets withrecvfrom(2), in a loop. For each one, it prints its size. The client creates its socket, binds it to any free name, reads lines withreadline(3)until Ctrl+D, and sends each one to the server as aMYC_U_MSG. Start a server and a few clients, and type some lines. - Joining. The server keeps a table of users, for example an array of 64 elements, each with a name and an address. An empty name marks a free element. It answers
MYC_U_JOINwithMYC_S_JOINto everyone in the chat, or withMYC_S_ERRif this address has already joined, if the name is empty, contains a space, or is taken, or if the table is full. It passesMYC_U_MSGon asMYC_S_MSG. Anything else from an address which has not joined gets aMYC_S_ERR. The client sends/joinand plain lines, and setsrl_event_hookto a function which receives every waiting packet and prints it, as in the session above. - Leaving. Handle
/leave, and sendMYC_S_LEAVEwithcleanset to everyone in the chat, including the user who left. On Ctrl+D, a client which is in the chat sendsMYC_U_LEAVEbefore it ends. To know whether it is in the chat, it can remember the name of its last/join, and look forMYC_S_JOINandMYC_S_LEAVEpackets with that name. A client which ends without leaving, for example with Ctrl+C, sends nothing. The server finds out the next time it sends that user a packet:sendto(2)fails withECONNREFUSED. Then it removes the user, and sendsMYC_S_LEAVEwithcleanfalse to the others. In the same way, when the client'ssendto(2)fails withECONNREFUSEDorENOENT, the server is gone: print it, and end. - Direct messages. Handle
/dm NAME TEXT. The server sendsMYC_S_MSG, withdmset, only toNAME, orMYC_S_ERRif there is no such user.
Some rules and hints:
- The server sends with
MSG_DONTWAIT. A client which does not read must not stop the server. If a send fails withEAGAIN, that client loses the packet. - The client sends without flags, and receives, in the hook, with
MSG_DONTWAIT. - Two addresses are the same if they have the same length, and
memcmp(3)finds the same bytes in that length. Keep the length whichrecvfrom(2)gave you next to each address. - While the server sends a packet to everyone, one send can fail with
ECONNREFUSED. Removing that user sendsMYC_S_LEAVEto everyone, in the middle of the first loop. This only works if you take a user out of the table before you sendMYC_S_LEAVEabout them. struct myc_packet p = { .type = MYC_S_JOIN };sets every field which is not named to 0.snprintf(p.name, sizeof p.name, "%s", name)copies a string without overflowing. It returns the length of the whole string, so a result ofsizeof p.nameor more means that the name did not fit.- To split the line of
/dm, take the command and the name withstrtok_r(3), and then the rest of the line withstrtok_r(nullptr, "", &save). - Lines which do not come from the hook, such as an error for an unknown command, are printed with a plain
write(2): readline is not waiting at that moment.