NAME

popen — initiate pipe streams to or from a process

SYNOPSIS

[CX] #include <stdio.h>

FILE *popen(const char *command, const char *mode);

DESCRIPTION

The popen() function shall execute the command specified by the string command. It shall create a pipe between the calling program and the executed command, and shall return a pointer to a stream that can be used to either read from or write to the pipe.

The environment of the executed command shall be as if a child process were created within the popen() call using the fork() function, and the child invoked the sh utility using the call:

execl(<shell path>, "sh", "-c", "--", command, (char *)0);

where <shell path> is an unspecified pathname for the sh utility. It is implementation-defined whether the handlers registered with pthread_atfork() are called as part of the creation of the child process.

The popen() function shall ensure that any streams from previous popen() calls that remain open in the parent process are closed in the new child process, regardless of the FD_CLOEXEC or FD_CLOFORK status of the file descriptor underlying those streams.

The mode argument to popen() is a string that specifies I/O mode:

1. If mode starts with 'r', when the child process is started, its file descriptor STDOUT_FILENO shall be the writable end of the pipe, and the file descriptor fileno(stream) in the calling process, where stream is the stream pointer returned by popen(), shall be the readable end of the pipe. The FD_CLOFORK flag shall be cleared on both the STDOUT_FILENO file descriptor passed to the child process and the file descriptor underlying the returned stream.

2. If mode starts with 'w', when the child process is started its file descriptor STDIN_FILENO shall be the readable end of the pipe, and the file descriptor fileno(stream) in the calling process, where stream is the stream pointer returned by popen(), shall be the writable end of the pipe. The FD_CLOFORK flag shall be cleared on both the STDOUT_FILENO file descriptor passed to the child process and the file descriptor underlying the returned stream.

3. If mode includes a second character of 'e', then the file descriptor underlying the stream returned to the calling process by popen() shall have the FD_CLOEXEC flag atomically set. Additionally, if the implementation creates the file descriptor for use by the child process from within the parent process, then that file descriptor shall have the FD_CLOEXEC flag atomically set within the parent process. If mode does not have a second character, the FD_CLOEXEC flag of the underlying file descriptor returned by popen() shall be clear.

4. If mode is any other value, the result is unspecified.

After popen(), both the parent and the child process shall be capable of executing independently before either terminates.

Pipe streams are byte-oriented.

RETURN VALUE

Upon successful completion, popen() shall return a pointer to an open stream that can be used to read or write to the pipe. Otherwise, it shall return a null pointer and may set errno to indicate the error.

ERRORS

The popen() function shall fail if:

[EMFILE]

{STREAM_MAX} streams are currently open in the calling process.

The popen() function may fail if:

[EMFILE]

{FOPEN_MAX} streams are currently open in the calling process.

[EINVAL]

The mode argument is invalid.

The popen() function may also set errno values as described by fork or pipe.

EXAMPLES

Using popen() to Obtain a List of Files from the ls Utility

The following example demonstrates the use of popen() and pclose() to execute the command ls* in order to obtain a list of files in the current directory:

#include <stdio.h>
...


FILE *fp;
int status;
char path[PATH_MAX];


fp = popen("ls *", "r");
if (fp == NULL)
    /* Handle error */;


while (fgets(path, PATH_MAX, fp) != NULL)
    printf("%s", path);


status = pclose(fp);
if (status == -1) {
    /* Error reported by pclose() */
    ...
} else {
    /* Use macros described under wait() to inspect `status' in order
       to determine success/failure of command executed by popen() */
    ...
}

APPLICATION USAGE

Since open files are shared, a mode 'r' command can be used as an input filter and a mode 'w' command as an output filter.

Buffered reading before opening an input filter may leave the standard input of that filter mispositioned. Similar problems with an output filter may be prevented by careful buffer flushing; for example, with fflush.

A stream opened by popen() should be closed by pclose().

The behavior of popen() is specified for values of mode of "r", "w", "re", and "we". Other modes such as "rb" and "wb" might be supported by specific implementations, but these would not be portable features. Note that historical implementations of popen() only check to see if the first character of mode is 'r'. Thus, a mode of "robert the robot" would be treated as mode "r", and a mode of "anything else" would be treated as mode "w".

If the application calls waitpid() or waitid() with a pid argument greater than 0, and it still has a stream that was called with popen() open, it must ensure that pid does not refer to the process started by popen().

To determine whether or not the environment specified in the Shell and Utilities volume of POSIX.1-2024 is present, use the function call:

sysconf(_SC_2_VERSION)

(See sysconf).

RATIONALE

The popen() function should not be used by programs that have set user (or group) ID privileges. The fork() and exec family of functions (except execlp() and execvp()), should be used instead. This prevents any unforeseen manipulation of the environment of the user that could cause execution of commands not anticipated by the calling program.

If the original and popen()ed processes both intend to read or write or read and write a common file, and either will be using FILE-type C functions (fread(), fwrite(), and so on), the rules for sharing file handles must be observed (see 2.5.1 Interaction of File Descriptors and Standard I/O Streams).

The 'e' mode modifier to popen() is necessary to avoid a data race in multi-threaded applications. Without it, the parent's file descriptor is leaked into a second child process created by one thread in the window between another thread creating the pipe via popen() then using fileno() and fcntl() on the result. Also, if the popen() implementation temporarily has the child's file descriptor open within the parent, then that file descriptor could also be leaked if it is not atomically FD_CLOEXEC for the duration in which it is open in the parent.

The standard only requires that the implementation atomically set FD_CLOEXEC on file descriptors created in the parent process when the 'e' mode modifier is in effect; implementations may also do so when the 'e' modifier is not in use, provided that the FD_CLOEXEC bit is eventually cleared before popen() completes, however, this is not required because any application worried about the potential file descriptor leak will already be using the 'e' modifier.

Implementations are encouraged to add support for a "wf" mode which creates the pipe as if by calling pipe2() with the O_CLOFORK flag and then clearing FD_CLOFORK for the read side of the pipe. This prevents the write side from leaking into child processes created by other threads, ensuring the child created by popen() will get end-of-file when the parent closes the write side (although the read side can still be leaked). Unfortunately there is no way (short of temporarily preventing other threads from creating child processes, or implementing an atomic create-pipe-and-fork system call) to implement an "rf" mode with the equivalent guarantee that the child created by popen() will be the only writer. Therefore multi-threaded applications that do not have complete control over process creation cannot rely on getting end-of-file on the stream and need to use an alternative method of indicating the end of communications.

Although the standard is clear that a conforming application should not call popen() when file descriptor 0 or 1 is closed, implementations are encouraged to handle these cases correctly.

The following two examples demonstrate possible implementations of popen() using other standard functions. These examples are designed to show FD_CLOEXEC handling rather than all aspects of thread safety, and implementations are encouraged to improve the locking mechanism around the state list to be more efficient, as well as to be more robust if file descriptor 0 or 1 is returned as either part of the pipe. Also, remember that other implementations are possible, including one that uses an implementation-specific means of creating a pipe between parent and child where the parent process never has access to the child's end of the pipe. Both of these examples make use of the following helper functions, documented but not implemented here, to do the bookkeeping necessary to properly close all file descriptors created by other popen() calls regardless of their FD_CLOEXEC or FD_CLOFORK status:

/* Obtain mutual exclusion lock, so that no concurrent popen() or
   pclose() calls are simultaneously modifying the list of tracked
   children. */
static void popen_lock(void);


/* Release mutual exclusion lock, without changing errno. */
static void popen_unlock(void);


/* Add the pid and stream pair to the list of tracked children, prior
   to any code that can clear FD_CLOEXEC on the file descriptor
   associated with stream. To be used while holding the lock. */
static void popen_add_pair(FILE *stream, pid_t pid);


/* Given a stream, return the associated pid, or -1 with errno set if
   the stream was not created by popen(). To be used while holding
   the lock. */
static pid_t popen_get_pid(FILE *stream);


/* Remove stream and its corresponding pid from the list of tracked
   children. To be used while holding the lock. */
static void popen_remove(FILE *stream);


/* If stream is NULL, return the first tracked child; otherwise,
   return the next tracked child. Return NULL if all tracked children
   have been returned. To be used while holding the lock. */
static FILE *popen_next(FILE *stream);

The first example is based on fork():

#include <stdio.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <unistd.h>
FILE *popen(const char *command, const char *mode)
{
  int fds[2];
  pid_t pid;
  FILE *stream;
  int target = mode[0] == 'w'; /* index of fds used by parent */


  /* Validate mode */
  if ((mode[0] != 'w' && mode[0] != 'r') ||
      mode[1 + (mode[1] == 'e')]) {
    errno = EINVAL;
    return NULL;
  }


  /* Create pipe and stream with FD_CLOEXEC set */
  if (pipe2(fds, O_CLOEXEC) < 0)
    return NULL;
  stream = fdopen(fds[target], mode);
  if (!stream) {
    int saved = errno;
    close(fds[0]);
    close(fds[1]);
    errno = saved;
    return NULL;
  }


  /* Create child process */
  popen_lock();
  pid = fork();
  if (pid < 0) {
    int saved = errno;
    close(fds[!target]);
    fclose(stream);
    popen_unlock();
    errno = saved;
    return NULL;
  }


  /* Child process. */
  if (!pid) {
    FILE *tracked = popen_next(NULL);
    while (tracked) {
      int fd = fileno(tracked);
      if (fd < 0 || close(fd))
        _exit(127);
      tracked = popen_next(tracked);
    }
    target = mode[0] == 'r'; /* Opposite fd in the child */
    /* Use dup2 or fcntl to clear FD_CLOEXEC on child's descriptor,
       FD_CLOEXEC will take care of the rest of fds[]. */
    if (fds[target] != target) {
      if (dup2(fds[target], target) != target)
        _exit(127);
    } else {
      int flags = fcntl(fds[target], F_GETFD);
      if (flags < 0 ||
          fcntl(fds[target], F_SETFD, flags & ~FD_CLOEXEC) < 0)
        _exit(127);
    }
    execl("/bin/sh", "sh", "-c", "--", command, NULL);
    _exit(127);
  }


  /* Parent process. From here on out, the close and fcntl system
     calls are assumed to pass, since all inputs are valid and do not
     require allocating any fds or memory. Besides, excluding
     failures due to undefined behavior (such as another thread
     closing an fd it knows nothing about), cleanup from any defined
     failures would require stopping and reaping the child process,
     which may have worse consequences. */
  close(fds[!target]);  popen_add_pair(stream, pid);
  popen_unlock();
  if (mode[1] != 'e') {
    int flags = fcntl(fds[target], F_GETFD);
    if (flags >= 0)
      fcntl(fds[target], F_SETFD, flags & ~FD_CLOEXEC);
  }
  return stream;
}

The second example is based on posix_spawn():

#include <stdio.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <unistd.h>
#include <spawn.h>
extern char **environ;
FILE *popen(const char *command, const char *mode)
{
  int fds[2];
  pid_t pid;
  FILE *stream;
  int target = mode[0] == 'w'; /* index of fds used by parent */
  const char *argv[] = { "sh", "-c", "--", command, NULL };
  posix_spawn_file_actions_t actions;
  int saved;
  FILE *tracked;


  /* Validate mode */
  if ((mode[0] != 'w' && mode[0] != 'r') ||
      mode[1 + (mode[1] == 'e')]) {
    errno = EINVAL;
    return NULL;
  }


  /* Create pipe and stream with FD_CLOEXEC set */
  if (pipe2(fds, O_CLOEXEC) < 0)
    return NULL;
  stream = fdopen(fds[target], mode);
  if (!stream) {
    saved = errno;
    close(fds[0]);
    close(fds[1]);
    errno = saved;
    return NULL;
  }


  /* Create child process */
  if (posix_spawn_file_actions_init(&actions)) {
    saved = errno;
    goto spawnerr1;
  }
  popen_lock();
  tracked = popen_next(NULL);
  while (tracked) {
    int fd = fileno(tracked);
    if (fd < 0 || posix_spawn_file_actions_addclose(&actions, fd))
      goto spawnerr2;
    tracked = popen_next(tracked);
  }
  if (posix_spawn_file_actions_adddup2(&actions, fds[!target], !target))
    goto spawnerr2;
  if (posix_spawn(&pid, "/bin/sh", &actions, NULL, (char **)argv, 
environ)) {
spawnerr2:
    saved = errno;
    posix_spawn_file_actions_destroy(&actions);
    popen_unlock();
spawnerr1:
    close(fds[!target]);
    fclose(stream);
    errno = saved;
    return NULL;
  }


  /* From here on out, system calls are assumed to pass, since all
     inputs are valid and do not require allocating any fds or memory.
     Besides, excluding failures due to undefined behavior (such as
     another thread closing an fd it knows nothing about), cleanup
     from any defined failures would require stopping and reaping the
     child process, which may have worse consequences. */
  posix_spawn_file_actions_destroy(&actions);
  close(fds[!target]);
  popen_add_pair(stream, pid);
  popen_unlock();
  if (mode[1] != 'e') {
    int flags = fcntl(fds[target], F_GETFD);
    if (flags >= 0)
      fcntl(fds[target], F_SETFD, flags & ~FD_CLOEXEC);
  }
  return stream;
}

Both examples can share a common pclose() implementation.

int pclose(FILE *stream)
{
  int status;
  popen_lock();
  pid_t pid = popen_get_pid(stream);
  if (pid < 0) {
    popen_unlock();
    return -1;
  }
  popen_remove(stream);
  popen_unlock();
  fclose(stream); /* Ignore failure */
  while (waitpid(pid, &status, 0) == -1) {
    if (errno != EINTR) {
      status = -1;
      break;
    }
  }
  return status;
}

Note that, while a particular implementation of popen() (such as the two above) can assume a particular path for the shell, such a path is not necessarily valid on another system. The above examples are not portable, and are not intended to be.

Earlier versions of this standard required the command string to be passed as the next argument after "-c" (omitting "--"). This meant that portable applications needed to take care not to pass a command string beginning with <hyphen-minus> ('-') or <plus-sign> ('+'), as it would then be interpreted as containing options. Now that implementations are required to pass the "--", applications no longer need to do this.

FUTURE DIRECTIONS

None.

SEE ALSO

2.5 Standard I/O Streams, fork, pclose, pipe, sysconf, system, wait, waitid

XBD <stdio.h>

XCU sh

CHANGE HISTORY

First released in Issue 1. Derived from Issue 1 of the SVID.

Issue 5

A statement is added to the DESCRIPTION indicating that pipe streams are byte-oriented.

Issue 6

The following new requirements on POSIX implementations derive from alignment with the Single UNIX Specification:

• The optional [EMFILE] error condition is added.

IEEE Std 1003.1-2001/Cor 2-2004, item XSH/TC2/D6/67 is applied, adding the example to the EXAMPLES section.

Issue 7

Austin Group Interpretation 1003.1-2001 #029 is applied, clarifying the values for mode in the DESCRIPTION.

SD5-XSH-ERN-149 is applied, changing the {STREAM_MAX} [EMFILE] error condition from a "may fail" to a "shall fail".

POSIX.1-2008, Technical Corrigendum 1, XSH/TC1-2008/0432 [14] is applied.

Issue 8

Austin Group Defects 411 and 1318 are applied, adding the 'e' mode modifier and FD_CLOFORK, and adding example implementations of popen() and pclose() in the RATIONALE section.

Austin Group Defect 1317 is applied, making it implementation-defined whether the handlers registered with pthread_atfork() are called.

Austin Group Defect 1440 is applied, adding a "--" argument to the specified execl() call.

Austin Group Defect 1526 is applied, making typographic changes relating to mode values for consistency with fopen().