Merge pull request #43 from klemens-morgenstern/async_system

Async system

.travis.yml

@@ -88,8 +88,8 @@ before_install:
 
 script:
     # `--coverage` flags required to generate coverage info for Coveralls
-    - ../../../b2 testing.launcher=valgrind address-model=64 architecture=x86 toolset=$TOOLSET cxxflags="--coverage -DBOOST_TRAVISCI_BUILD -std=$CXX_STANDARD" linkflags="--coverage" -sBOOST_BUILD_PATH=.
-
+    - ../../../b2 address-model=64 architecture=x86 testing.launcher=valgrind valgrind=on toolset=$TOOLSET cxxflags="--coverage -DBOOST_TRAVISCI_BUILD -std=$CXX_STANDARD" linkflags="--coverage" -sBOOST_BUILD_PATH=. 
+    - ../../../b2 vfork address-model=64 architecture=x86 toolset=$TOOLSET cxxflags="--coverage -DBOOST_TRAVISCI_BUILD -std=$CXX_STANDARD" linkflags="--coverage" -sBOOST_BUILD_PATH=. 
 after_success:
     # Copying Coveralls data to a separate folder
     - mkdir -p $TRAVIS_BUILD_DIR/coverals

doc/Jamfile.jam

@@ -21,6 +21,7 @@ doxygen autodoc
   ../../../boost/process.hpp
   [ glob ../../../boost/process/*.hpp ]
 :
+  <doxygen:param>EXCLUDE_SYMBOLS=BOOST_ASIO_INITFN_RESULT_TYPE
   <doxygen:param>PREDEFINED=BOOST_PROCESS_DOXYGEN
   <doxygen:param>HIDE_UNDOC_CLASSES=YES
   <doxygen:param>HIDE_UNDOC_MEMBERS=YES

doc/concepts.qbk

@@ -0,0 +1,82 @@
+[section:concepts Concepts]
+In this section, some of the underlying concepts of the operating system used in this library, will be explained. 
+In the following chapters we will presume knowledge of that. Though please note, 
+that this is a short summary and not conclusive of everything that can be done.
+
+The goal of this library is to implement a portable wrapper, so that we will explain mostly what
+windows and posix have in common.   
+
+[section:pipes Pipes]
+Pipes are a facility for communication between different threads, processes and in some cases machines, the operating system provides.
+
+The typical feature of a pipe is, that it is one channel, two which two handles are given, one for reading (source), one for writing (sink).
+In that it is different than other facilities (like sockets) and provides another way to manage the connectivity: if one side of the pipe is closed 
+(i.e. the pipe is broken), the other is notified.
+
+Pipes are typically used for interprocess communication. The main reason is, that pipes can be directly assigned to the process stdio, i.e. stderr, stdin and stdout.
+Additionally, half of the pipe can be inherited to the child process and closed in the father process. This will cause the pipe to be broken when the child process exits.
+
+Though please not, that if the the same thread reads and write to a pipe, it will only talk to itself.
+
+[section:anonymous Anonymous Pipes]
+
+The usual type of pipes, are the anonymous ones. Since the have no name, 
+a handle to them can only be obtained from duplicating either handle.
+
+In this library the following functions are used for the creation of unnamed pipes:
+
+* [@http://pubs.opengroup.org/onlinepubs/7908799/xsh/pipe.html posix]
+* [@https://msdn.microsoft.com/de-de/library/windows/desktop/aa365152.aspx windows]
+
+[endsect]
+
+[section:named Named Pipes]
+
+As the name suggests, named pipes have a string identifier. This means that a 
+handle to them can be obtained with the identifier, too.
+
+The implementation on posix uses [@(http://pubs.opengroup.org/onlinepubs/009695399/functions/mkfifo.html fifos],
+which means, that the named pipe behaves like a file.
+
+Windows does provide a facility called [@https://msdn.microsoft.com/en-us/library/windows/desktop/aa365150(v=vs.85).aspx named pipes],
+which also have file-like names, but are in a different scope than the actual file system.
+
+[note The main reason named pipes are part of this library, is because they need to be internally used for asynchrounous communication on windows.]
+
+[endsect]
+
+[endsect]
+
+
+[section:process Processes]
+
+A process is an independently executable entity, which is different from a thread, in that it has it's own resources. 
+Those include memory and hardware resources.
+
+Every process is identified by a unique number[footnote it is unique as long as the process is active], called the process identification digit, `pid`. 
+
+[section:exit_code Exit code]
+A process will return an integer value indicating whether it was successful. On posix
+there are more codes associated with that, but not so on windows. Therefor there is not such encoding currently in the library.
+Howevern an exit code of zero means the process was successful, while one different than zero indicates an error. 
+[endsect]
+
+[section:termination Termination]
+Processes can also be forced to exit. There are two ways to do this, signal the process to so and wait, and just terminate the process without conditions.
+
+Usually the first approach is to signal an exit request, but windows - unlike posix - does not provide a consistent way to do this. Hence this is not part of the
+library and only the hard terminate is. 
+
+[endsect]
+
+[endsect]
+
+[section:env Environment]
+
+The environment is a map of variables local to every process. The most significant one for this library
+ is the `PATH` variable, which containes a list of paths, that ought to be searched for executables. A shell will do this automatically, 
+ while this library provides a function for that. 
+
+[endsect]
+
+[endsect]

doc/process.qbk

@@ -12,6 +12,7 @@
 ]
 
 [include introduction.qbk]
+[include concepts.qbk]
 [include tutorial.qbk]
 [include design.qbk]
 [include extend.qbk]

doc/tutorial.qbk

@@ -1,4 +1,5 @@
 [def bp::system [funcref boost::process::system bp::system]]
+[def bp::async_system [funcref boost::process::async_system bp::async_system]]
 [def bp::spawn [funcref boost::process::system bp::spawn]]
 [def bp::child [classref boost::process::child bp::child]]
 [def bp::group [classref boost::process::group bp::group]]
@@ -10,11 +11,14 @@
 [def std::system [@http://en.cppreference.com/w/cpp/utility/program/system std::system]]
 [def child_running [memberref boost::process::child::running running]]
 [def child_wait [memberref boost::process::child::wait wait]]
+[def child_wait_for [memberref boost::process::child::wait_for wait_for]]
 [def child_exit_code [memberref boost::process::child::exit_code exit_code]]
+[def group_wait_for [memberref boost::process::group::wait_for wait_for]]
 [def bp::on_exit [globalref boost::process::on_exit bp::on_exit]]
 [def bp::null [globalref boost::process::null bp::null]]
 [def child_terminate [memberref boost::process::child::terminate terminate]]
 [def group_terminate [memberref boost::process::group::terminate terminate]]
+[def group_wait [memberref boost::process::group::wait wait]]
 [def bp::std_in [globalref boost::process::std_in bp::std_in]]
 [def bp::std_out [globalref boost::process::std_out bp::std_out]]
 [def bp::std_err [globalref boost::process::std_err bp::std_err]]
@@ -24,20 +28,23 @@
 [def bp::environment  [classref boost::process::basic_environment bp::environment]]
 [def bp::native_environment  [classref boost::process::basic_native_environment bp::native_environment]]
 [def boost::this_process::environment [funcref boost::this_process::environment boost::this_process::environment]]
+[def std::chrono::seconds [@http://en.cppreference.com/w/cpp/chrono/duration std::chrono::seconds]]
 
 [def __wait_for__ [memberref boost::process::child::wait_for wait_for]]
 [def __wait_until__ [memberref boost::process::child::wait_until wait_until]]
 [def __detach__ [memberref boost::process::child::detach detach]]
 
 [def __reference__ [link process.reference reference]]
+[def __concepts__ [link boost_process.concepts concepts]]
 
 [def boost::asio::yield_context [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/yield_context.html boost::asio::yield_context]]
 [def boost::asio::coroutine [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/coroutine.html boost::asio::coroutine]]
+[def bp::env [globalref boost::process::env bp::env]]
 
 [section:tutorial Tutorial]
 
 In this section we will go step by step through the different features of
-boost.process. For a full description see the __reference__.
+boost.process. For a full description see the __reference__ and the __concepts__ sections.
 
 [section Starting a process]
 
@@ -88,7 +95,8 @@ our program will wait until the child process is completed. This is unwanted,
 especially since compiling can take a while. 
 
 In order to avoid that, boost.process provides several ways to launch a process.
-Besides the already mentioned [funcref boost::process::system system] function,
+Besides the already mentioned [funcref boost::process::system system] function and it's 
+asynchronous version [funcref boost::process::async_system async_system], 
 we can also use the [funcref boost::process::spawn spawn] function or the
 [classref boost::process::child child] class.
 
@@ -290,7 +298,7 @@ int result = c.exit_code();
 [memberref boost::process::child::wait wait] is needed]
 
 To make it even easier, you can use [@http://en.cppreference.com/w/cpp/thread/future std::future] for asynchronous operations
-(you will still need to pass a reference to a io_service) to the launching function, unless you use bp::system.
+(you will still need to pass a reference to a io_service) to the launching function, unless you use bp::system or bp::async_system.
 
 Now we will revisit our first example and read the compiler output asynchronously:
 
@@ -308,26 +316,70 @@ child c("g++", "main.cpp", //set the input
 
 ios.run(); //this will actually block until the compiler is finished
 
-auto err =  fut.get();
+auto err =  data.get();
 ```
 
 [endsect]
 [section:group Groups]
 
 When launching several processes, processes can be grouped together. 
 This will also apply for a child process, that launches other processes, 
-if they do not modifiy the group membership. E.g. if you call `make` which 
+if they do not modify the group membership. E.g. if you call `make` which 
 launches other processes and call terminate on it, 
 it will not terminate all the child processes of the child unless you use a group.
 
+The two main reasons to use groups are:
+
+# Being able two terminate child processes of the child process
+# Grouping several processes into one, just so they can be terminated at once
+
+If we have program like `make`, which does launch it's own child processes, 
+a call of child_terminate might not suffice. I.e. if we have a makefile launching `gcc` 
+and use the following code, the `gcc` process will still run afterwards:
+
+```
+bp::child c("make");
+if (!c.child_wait_for(std::chrono::seconds(10)) //give it 10 seconds
+    c.child_terminate(); //then terminate
+```
+
+So in order to also terminate `gcc` we can use a group.
+
 ```
 bp::group g;
-bp::child c1("foo", g);
-bp::child c2("bar", g);
-g.group_terminate();
+bp::child c("make", g);
+if (!g.group_wait_for(std::chrono::seconds(10))
+    g.group_terminate();
+
+c.child_wait(); //to avoid a zombie process & get the exit code
 ```
 
-Please see to the [headerref boost/process/group.hpp reference] for more information.
+Now given the example, we still call child_wait to avoid a zombie process. 
+An easier solution for that might be to use [funcref boost::process::spawn spawn].
+
+
+To put two processes into one group, the following code suffices. Spawn already 
+launches a detached process (i.e. without a child-handle), but they can be grouped, 
+to that in the case of a problem, RAII is still a given. 
+
+```
+void f()
+{
+    bp::group g;
+    bp::spawn("foo", g);
+    bp::spawn("bar", g);
+
+    do_something();
+
+    g.group_wait();
+};
+```
+
+In the example, it will wait for both processes at the end of the function unless 
+an exception occures. I.e. if an exception is thrown, the group will be terminated.
+
+
+Please see the [headerref boost/process/group.hpp reference] for more information.
 
 [endsect]
 [section:env Environment]
@@ -343,87 +395,22 @@ env["VALUE_1"] = "foo";
 
 //copy it into a environment seperate to the one of this process
 bp::environment env_ = env;  
-//add a value only to the new env
-env_["VALUE_2"] = "bar";
+//append two values to a variable in the new env
+env_["VALUE_2"] += {"bar1", "bar2"};
 
 //launch a process with `env_`
 bp::system("stuff", env_); 
 ```
 
 A more convenient way to modify the environment for the child is the 
-[globalref boost::process::env env] property.
-
-Please see to the [headerref boost/process/environment.hpp reference] for more information.
-
-
-[endsect]
-
-
-[section:coro Coroutines]
-[section:stackless Stackless Coroutines]
-
-[note This section presumes knowledge of the boost.asio 
-[@http://www.boost.org/doc/libs/release/doc/html/boost_asio/overview/core/coroutine.html stackless coroutine] feature.]
-
-Stackless coroutines can be implemented rather easily, so there is no need to
-implement extra functionality concerning boost.process. 
+[globalref boost::process::env env] property, which the example as following:
 
 ```
-struct stackless_t : boost::asio::coroutine
-{
-    bp::child c;
+bp::system("stuff", bp::env["VALUE_1"]="foo", bp::env["VALUE_2"]+={"bar1", "bar2"});
 
-    boost::asio::io_service & ios;
-
-    stackless_t(boost::asio::io_service & ios) : ios(ios) {}
-
-    void operator()(
-            boost::system::error_code ec = boost::system::error_code(),
-            std::size_t n = 0)
-    {
-        if (!ec) reenter (this)
-        {
-             c = bp::child("my_program", ios, 
-                    bp::on_exit=
-                    [this](int, const std::error_code&)
-                    {
-                        (*this)(); //this is the reentry for the coroutine
-                    });
-            yield; //yield the thing.
-        }
-    }
-};
-///post the coroutine to a io-service and run it
-int main()
-{
-    boost::asio::io_service ios;
-    ios.post(stackless_t(ios));
-    ios.run();
-    return 0;
-}
 ```
 
-[endsect]
-
-[section:stackful Stackful Coroutines]
-
-[note This section presumes knowledge of the boost.asio 
-[@http://www.boost.org/doc/libs/release/doc/html/boost_asio/overview/core/spawn.html stackful coroutine] feature.]
-
-For stackful coroutines this is not as simple, because the members of 
-`boost::asio::yield_context` are not documented. Therefore, boost.process 
-provides a simple way to use stackful coroutines, which looks as follows:
-
-```
-void cr(boost::asio::yield_context yield_)
-{
-    bp::system("my-program", yield_);
-}
-```
-
-This will automatically suspend the coroutine until the child process is finished.
+Please see to the [headerref boost/process/environment.hpp reference] for more information.
 
 [endsect]
 [endsect]
-
-[endsect]

include/boost/process.hpp

@@ -21,6 +21,7 @@
 
 #include <boost/process/args.hpp>
 #include <boost/process/async.hpp>
+#include <boost/process/async_system.hpp>
 #include <boost/process/group.hpp>
 #include <boost/process/child.hpp>
 #include <boost/process/cmd.hpp>

include/boost/process/async.hpp

@@ -101,15 +101,24 @@ on_exit=function;
 on_exit(function);
 \endcode
 
-with `function` being a callable object with the signature `(int, const std::error_code&)`.
+with `function` being a callable object with the signature `(int, const std::error_code&)` or an
+`std::future<int>`.
 
 \par Example
 
 \code{.cpp}
 io_service ios;
-spawn("ls", on_exit=[](int exit, const std::error_code& ec_in){});
+
+child c("ls", on_exit=[](int exit, const std::error_code& ec_in){});
+
+std::future<int> exit_code;
+chlid c2("ls", on_exit=exit_code);
+
 \endcode
 
+\note The handler is not invoked when the launch fails.
+\warning When used \ref ignore_error it might gte invoked on error.
+
  */
 constexpr static ::boost::process::detail::on_exit_ on_exit{};
 #endif