About linux : Process-hangs-if-stderr-and-stdout-are-redirected

I’m trying to call cmake and redirect the output to a pipe.

To reproduce:

1. git clone https://github.com/avast/retdec (It seems to be every CMake-Project, gradle projects don’t work, too)
2. mkdir build&&cd build
3. Add a file test.hs:

import System.Posix.IO
import System.Process
import System.Exit
import System.IO
main :: IO ()
main = do
    (code, content) <- createProcessRedirected "cmake" ["..", "-DCMAKE_INSTALL_PREFIX=/opt/root/"]
    case code of
        ExitFailure _ -> do
            putStrLn ("OOPS" ++ content)
            return ()
        _ -> do
            putStrLn "After cmake"
            (code2, content2) <- createProcessRedirected "make" ["install", "-j8"]
            case code2 of
                ExitFailure _ -> do
                    putStrLn ("OOPS" ++ content2)
                    return ()
                _ -> do
                    putStrLn "SUCCESS"
                    return ()
-- Based on https://stackoverflow.com/a/14027387
createProcessRedirected :: String -> [String] -> IO (ExitCode, String)
createProcessRedirected command args = do
    (p_r, p_w) <- System.Posix.IO.createPipe
    h_r <- fdToHandle p_r
    h_w <- fdToHandle p_w
    (_, _, _, h_proc) <- createProcess (proc command args) {
        std_out = UseHandle h_w,
        std_err = UseHandle h_w,
        create_group = False,
        delegate_ctlc = True}
    ret_code <- waitForProcess h_proc
    content <- hGetContents h_r
    return (ret_code, content)

Just compile this file (ghc test.hs -Wall -Wextra)

If I try to execute ./test now, it will build for some time (I can see how c++/make processes are spawned), but at one point it will simply stop.
I then have this process tree:

test
  -> make install -j8
     -> /usr/bin/cmake -P cmake_install.cmake

If I attach gdb and run bt (First 6 lines):

#0  0x00007f4ac7f8c7f7 in __GI___libc_write (fd=1, buf=0x5607747dd2f0, nbytes=60) at ../sysdeps/unix/sysv/linux/write.c:26
#1  0x00007f4ac7f1068d in _IO_new_file_write (f=0x7f4ac80865a0 <_IO_2_1_stdout_>, data=0x5607747dd2f0, n=60) at fileops.c:1181
#2  0x00007f4ac7f0fa06 in new_do_write (fp=0x7f4ac80865a0 <_IO_2_1_stdout_>, data=0x5607747dd2f0 "-- Up-to-date: /opt/root/include/retdec/llvm/IR/OptBisect.h\ns.h\nexYAML.h\n.h\nnager.h\n\nayer.h\n.h\n\nt execute permission?\nif(NOT DEFINED CMAKE_INSTALL_SO_NO_EXE)\n  set(CMAKE_INSTALL_SO_NO_EXE \"0\")\nendif()"..., [email protected]=60) at /usr/src/debug/glibc-2.34-11.fc35.x86_64/libio/libioP.h:947
#3  0x00007f4ac7f11729 in _IO_new_do_write (to_do=60, data=<optimized out>, fp=<optimized out>) at fileops.c:423
#4  _IO_new_do_write (fp=<optimized out>, data=<optimized out>, to_do=60) at fileops.c:423
#5  0x00007f4ac7f0f828 in _IO_new_file_sync (fp=0x7f4ac80865a0 <_IO_2_1_stdout_>) at fileops.c:799

And then I try to continue running it and after 10 seconds I break again, I get exactly the same backtrace.

But If I comment out these both lines:

        std_out = UseHandle h_w,
        std_err = UseHandle h_w,

it works.

What am I doing wrong?

The buffer size of pipes isn’t unlimited. You’re creating a deadlock, where the child process is hanging because it’s trying to write to a buffer that’s full, and your parent process doesn’t try to read anything from the buffer until the child process has completed. To fix the problem, you need to use another thread to read from the buffer while the child process is still running. The simplest way to do this is to use readProcess or a similar function in place of createProcess. If this doesn’t give you enough flexibility to do what you want, then you’ll need to create and manage the other thread yourself, which you can see how to do by looking at how readProcess is implemented.