The Open Group Base Specifications Issue 6
IEEE Std 1003.1, 2004 Edition
Copyright © 2001-2004 The IEEE and The Open Group, All Rights reserved.
A newer edition of this document exists here

NAME

strtok, strtok_r - split string into tokens

SYNOPSIS

#include <string.h>

char *strtok(char *restrict
s1, const char *restrict s2);

[TSF] [Option Start] char *strtok_r(char *restrict s, const char *restrict sep,
       char **restrict
lasts); [Option End]

DESCRIPTION

For strtok(): [CX] [Option Start] The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of IEEE Std 1003.1-2001 defers to the ISO C standard. [Option End]

A sequence of calls to strtok() breaks the string pointed to by s1 into a sequence of tokens, each of which is delimited by a byte from the string pointed to by s2. The first call in the sequence has s1 as its first argument, and is followed by calls with a null pointer as their first argument. The separator string pointed to by s2 may be different from call to call.

The first call in the sequence searches the string pointed to by s1 for the first byte that is not contained in the current separator string pointed to by s2. If no such byte is found, then there are no tokens in the string pointed to by s1 and strtok() shall return a null pointer. If such a byte is found, it is the start of the first token.

The strtok() function then searches from there for a byte that is contained in the current separator string. If no such byte is found, the current token extends to the end of the string pointed to by s1, and subsequent searches for a token shall return a null pointer. If such a byte is found, it is overwritten by a null byte, which terminates the current token. The strtok() function saves a pointer to the following byte, from which the next search for a token shall start.

Each subsequent call, with a null pointer as the value of the first argument, starts searching from the saved pointer and behaves as described above.

The implementation shall behave as if no function defined in this volume of IEEE Std 1003.1-2001 calls strtok().

[CX] [Option Start] The strtok() function need not be reentrant. A function that is not required to be reentrant is not required to be thread-safe. [Option End]

[TSF] [Option Start] The strtok_r() function considers the null-terminated string s as a sequence of zero or more text tokens separated by spans of one or more characters from the separator string sep. The argument lasts points to a user-provided pointer which points to stored information necessary for strtok_r() to continue scanning the same string.

In the first call to strtok_r(), s points to a null-terminated string, sep to a null-terminated string of separator characters, and the value pointed to by lasts is ignored. The strtok_r() function shall return a pointer to the first character of the first token, write a null character into s immediately following the returned token, and update the pointer to which lasts points.

In subsequent calls, s is a NULL pointer and lasts shall be unchanged from the previous call so that subsequent calls shall move through the string s, returning successive tokens until no tokens remain. The separator string sep may be different from call to call. When no token remains in s, a NULL pointer shall be returned. [Option End]

RETURN VALUE

Upon successful completion, strtok() shall return a pointer to the first byte of a token. Otherwise, if there is no token, strtok() shall return a null pointer.

[TSF] [Option Start] The strtok_r() function shall return a pointer to the token found, or a NULL pointer when no token is found. [Option End]

ERRORS

No errors are defined.


The following sections are informative.

EXAMPLES

Searching for Word Separators

The following example searches for tokens separated by <space>s.

#include <string.h>
...
char *token;
char *line = "LINE TO BE SEPARATED";
char *search = " ";

/* Token will point to "LINE". */ token = strtok(line, search);
/* Token will point to "TO". */ token = strtok(NULL, search);
Breaking a Line

The following example uses strtok() to break a line into two character strings separated by any combination of <space>s, <tab>s, or <newline>s.

#include <string.h>
...
struct element {
    char *key;
    char *data;
};
...
char line[LINE_MAX];
char *key, *data;
...
key = strtok(line, "    \n");
data = strtok(NULL, "   \n");
...

APPLICATION USAGE

The strtok_r() function is thread-safe and stores its state in a user-supplied buffer instead of possibly using a static data area that may be overwritten by an unrelated call from another thread.

RATIONALE

The strtok() function searches for a separator string within a larger string. It returns a pointer to the last substring between separator strings. This function uses static storage to keep track of the current string position between calls. The new function, strtok_r(), takes an additional argument, lasts, to keep track of the current position in the string.

FUTURE DIRECTIONS

None.

SEE ALSO

The Base Definitions volume of IEEE Std 1003.1-2001, <string.h>

CHANGE HISTORY

First released in Issue 1. Derived from Issue 1 of the SVID.

Issue 5

The strtok_r() function is included for alignment with the POSIX Threads Extension.

A note indicating that the strtok() function need not be reentrant is added to the DESCRIPTION.

Issue 6

Extensions beyond the ISO C standard are marked.

The strtok_r() function is marked as part of the Thread-Safe Functions option.

In the DESCRIPTION, the note about reentrancy is expanded to cover thread-safety.

The APPLICATION USAGE section is updated to include a note on the thread-safe function and its avoidance of possibly using a static data area.

The restrict keyword is added to the strtok() and strtok_r() prototypes for alignment with the ISO/IEC 9899:1999 standard.

End of informative text.

UNIX ® is a registered Trademark of The Open Group.
POSIX ® is a registered Trademark of The IEEE.
[ Main Index | XBD | XCU | XSH | XRAT ]