Summary

Template function that performs parallel iteration over a range of values.

Header

 #include "tbb/parallel_for.h"

Syntax

template<typename Index, typename Func>
Func parallel_for( Index first, Index_type last, const Func& f
                   [, partitioner[, task_group_context& group]] );

template<typename Index, typename Func>
Func parallel_for( Index first, Index_type last,
                   Index step, const Func& f
                   [, partitioner[, task_group_context& group]] );

template<typename Range, typename Body>
void parallel_for( const Range& range, const Body& body,
                   [, partitioner[, task_group_context& group]] );

Description

A parallel_for(first,last,step,f) represents parallel execution of the loop:

for( auto i=first; i<last; i+=step ) f(i);

The index type must be an integral type. The loop must not wrap around. The step value must be positive. If omitted, it is implicitly 1. There is no guarantee that the iterations run in parallel. Deadlock may occur if a lesser iteration waits for a greater iteration. The partitioning strategy is auto_partitioner when the parameter is not specified.

A parallel_for(range,body,partitioner) provides a more general form of parallel iteration. It represents parallel execution of body over each value in range. The optional partitioner specifies a partitioning strategy. Type Range must model the Range concept . The body must model the requirements in the following table.

A parallel_for recursively splits the range into subranges to the point such that is_divisible() is false for each subrange, and makes copies of the body for each of these subranges. For each such body/subrange pair, it invokes Body::operator(). The invocations are interleaved with the recursive splitting, in order to minimize space overhead and efficiently use cache.

Some of the copies of the range and body may be destroyed after parallel_for returns. This late destruction is not an issue in typical usage, but is something to be aware of when looking at execution traces or writing range or body objects with complex side effects.

When worker threads are available, parallel_for executes iterations in non-deterministic order. Do not rely upon any particular execution order for correctness. However, for efficiency, do expect parallel_for to tend towards operating on consecutive runs of values.

When no worker threads are available, parallel_for executes iterations from left to right in the following sense. Imagine drawing a binary tree that represents the recursive splitting. Each non-leaf node represents splitting a subrange r by invoking one of the splitting constructors of Range. The left child represents the updated value of r. The right child represents the newly constructed object. Each leaf in the tree represents an indivisible subrange. The method Body::operator() is invoked on each leaf subrange, from left to right.

All overloads can be passed a task_group_context object so that the algorithm’s tasks are executed in this group. By default the algorithm is executed in a bound group of its own.

Complexity

If the range and body take O(1) space, and the range splits into nearly equal pieces, then the space complexity is O(P log(N)), where N is the size of the range and P is the number of threads.

Example

This example defines a routine ParallelAverage that sets output[i] to the average of input[i-1], input[i], and input[i+1], for 1 <= i< n.

#include "tbb/parallel_for.h"
#include "tbb/blocked_range.h"

using namespace tbb;

struct Average {
    const float* input;
    float* output;
    void operator()( const blocked_range<int>& range ) const {
        for( int i=range.begin(); i!=range.end(); ++i )
            output[i] = (input[i-1]+input[i]+input[i+1])*(1/3.f);
    }
};

// Note: Reads input[0..n] and writes output[1..n-1].
void ParallelAverage( float* output, const float* input, size_t n ) {
    Average avg;
    avg.input = input;
    avg.output = output;
    parallel_for( blocked_range<int>( 1, n ), avg );
    }

Example

This example is more complex and requires familiarity with STL. It shows the power of parallel_for beyond flat iteration spaces. The code performs a parallel merge of two sorted sequences. It works for any sequence with a random-access iterator. The algorithm (Akl 1987) works recursively as follows:

1. If the sequences are too short for effective use of parallelism, do a sequential merge. Otherwise perform steps 2-6.
2. Swap the sequences if necessary, so that the first sequence [begin1,end1) is at least as long as the second sequence [begin2,end2).
3. Set m1 to the middle position in [begin1,end1). Call the item at that location key.
4. Set m2 to where key would fall in [begin2,end2).
5. Merge [begin1,m1) and [begin2,m2) to create the first part of the merged sequence.
6. Merge [m1,end1) and [m2,end2) to create the second part of the merged sequence.

The Intel® Threading Building Blocks implementation of this algorithm uses the range object to perform most of the steps. Predicate is_divisible performs the test in step 1, and step 2. The splitting constructor does steps 3-6. The body object does the sequential merges.

#include "tbb/parallel_for.h"
#include <algorithm>

using namespace tbb;

template<typename Iterator>
struct ParallelMergeRange {
    static size_t grainsize;
    Iterator begin1, end1; // [begin1,end1) is 1st sequence to be merged
    Iterator begin2, end2; // [begin2,end2) is 2nd sequence to be merged
    Iterator out;               // where to put merged sequence
    bool empty()   const {return (end1-begin1)+(end2-begin2)==0;}
    bool is_divisible() const {
        return std::min( end1-begin1, end2-begin2 ) > grainsize;
    }
    ParallelMergeRange( ParallelMergeRange& r, split ) {
        if( r.end1-r.begin1 < r.end2-r.begin2 ) {
            std::swap(r.begin1,r.begin2);
            std::swap(r.end1,r.end2);
        }
        Iterator m1 = r.begin1 + (r.end1-r.begin1)/2;
        Iterator m2 = std::lower_bound( r.begin2, r.end2, *m1 );
        begin1 = m1;
        begin2 = m2;
        end1 = r.end1;
        end2 = r.end2;
        out = r.out + (m1-r.begin1) + (m2-r.begin2);
        r.end1 = m1;
        r.end2 = m2;
    }
    ParallelMergeRange( Iterator begin1_, Iterator end1_,
                        Iterator begin2_, Iterator end2_,
                        Iterator out_ ) :
        begin1(begin1_), end1(end1_),
        begin2(begin2_), end2(end2_), out(out_)
    {}
};

template<typename Iterator>
size_t ParallelMergeRange<Iterator>::grainsize = 1000;

template<typename Iterator>
struct ParallelMergeBody {
    void operator()( ParallelMergeRange<Iterator>& r ) const {
        std::merge( r.begin1, r.end1, r.begin2, r.end2, r.out );
    }
};

template<typename Iterator>
void ParallelMerge( Iterator begin1, Iterator end1, Iterator begin2, Iterator end2, Iterator out ) {
    parallel_for(
       ParallelMergeRange<Iterator>(begin1,end1,begin2,end2,out),
       ParallelMergeBody<Iterator>(),
       simple_partitioner()
    );
}

Because the algorithm moves many locations, it tends to be bandwidth limited. Speedup varies, depending upon the system.

This section describes features of  Intel® Thread Building Blocks (Intel® TBB) that relate to general environment issues.

Problem

Choose the next work item to do, based on priorities.

Context

The scheduler in Intel® Threading Building Blocks (Intel® TBB) chooses tasks using rules based on scalability concerns. The rules are based on the order in which tasks were spawned or enqueued, and are oblivious to the contents of tasks. However, sometimes it is best to choose work based on some kind of priority relationship.

Forces

• Given multiple work items, there is a rule for which item should be done next that is not the default Intel® TBB rule.

• Preemptive priorities are not necessary. If a higher priority item appears, it is not necessary to immediately stop lower priority items in flight. If preemptive priorities are necessary, then non-preemptive tasking is inappropriate. Use threads instead.

Solution

Put the work in a shared work pile. Decouple tasks from specific work, so that task execution chooses the actual piece of work to be selected from the pile.

Example

The following example implements three priority levels. The user interface for it and top-level implementation follow:

enum Priority {
   P_High,
   P_Medium,
   P_Low
};

template<typename Func>
void EnqueueWork( Priority p, Func f ) {
   WorkItem* item = new ConcreteWorkItem<Func>( p, f );
   ReadyPile.add(item);
}

The caller provides a priority p and a functor f to routine EnqueueWork. The functor may be the result of a lambda expression. EnqueueWork packages f as a WorkItem and adds it to global object ReadyPile.

Class WorkItem provides a uniform interface for running functors of unknown type:

// Abstract base class for a prioritized piece of work.
class WorkItem {
public:
   WorkItem( Priority p ) : priority(p) {}
   // Derived class defines the actual work.
   virtual void run() = 0;
   const Priority priority;
};

template<typename Func>

class ConcreteWorkItem: public WorkItem {
   Func f;
   /*override*/ void run() {
       f();
       delete this;
   }
public:
   ConcreteWorkItem( Priority p, const Func& f_ ) :
       WorkItem(p), f(f_)
   {}
};

Class ReadyPile contains the core pattern. It maintains a collection of work and fires off tasks that choose work from the collection:

class ReadyPileType {
   // One queue for each priority level
   tbb::concurrent_queue<WorkItem*> level[P_Low+1];
public:
   void add( WorkItem* item ) {
       level[item->priority].push(item);
       tbb::task::enqueue(*new(tbb::task::allocate_root()) RunWorkItem);
   }
   void runNextWorkItem() {
       // Scan queues in priority order for an item.
       WorkItem* item=NULL;
       for( int i=P_High; i<=P_Low; ++i )
           if( level[i].try_pop(item) )
               break;
       assert(item);
       item->run();
   }
};

ReadyPileType ReadyPile;

The task enqueued by add(item) does not necessarily execute that item. The task executes runNextWorkItem(), which may find a higher priority item. There is one task for each item, but the mapping resolves when the task actually executes, not when it is created.

Here are the details of class RunWorkItem:

class RunWorkItem: public tbb::task {
   /*override*/tbb::task* execute(); // Private override of virtual method
};
...
tbb::task* RunWorkItem::execute() {
   ReadyPile.runNextWorkItem();
   return NULL;
};

RunWorkItem objects are fungible. They enable the Intel® TBB scheduler to choose when to do a work item, not which work item to do. The override of virtual method task::execute is private because all calls to it are dispatched via base class task.

Other priority schemes can be implemented by changing the internals for ReadyPileType. A priority queue could be used to implement very fine grained priorities.

The scalability of the pattern is limited by the scalability of ReadyPileType. Ideally scalable concurrent containers should be used for it.

Deadlock

Deadlock happens when threads are trying to acquire more than one lock, and each holds some of the locks the other threads need to proceed. More precisely, deadlock happens when:

• There is a cycle of threads

• Each thread holds at least one lock on a mutex, and is waiting on a mutex for which the next thread in the cycle already has a lock.

• No thread is willing to give up its lock.

Think of classic gridlock at an intersection – each car has "acquired" part of the road, but needs to "acquire" the road under another car to get through. Two common ways to avoid deadlock are:

• Avoid needing to hold two locks at the same time. Break your program into small actions in which each can be accomplished while holding a single lock.

• Always acquire locks in the same order. For example, if you have "outer container" and "inner container" mutexes, and need to acquire a lock on one of each, you could always acquire the "outer sanctum" one first. Another example is "acquire locks in alphabetical order" in a situation where the locks have names. Or if the locks are unnamed, acquire locks in order of the mutex’s numerical addresses.

• Use atomic operations instead of locks.

Convoying

Another common problem with locks is convoying. Convoying occurs when the operating system interrupts a thread that is holding a lock. All other threads must wait until the interrupted thread resumes and releases the lock. Fair mutexes can make the situation even worse, because if a waiting thread is interrupted, all the threads behind it must wait for it to resume.

To minimize convoying, try to hold the lock as briefly as possible. Precompute whatever you can before acquiring the lock.

To avoid convoying, use atomic operations instead of locks where possible.

Class members are summarized by informal class declarations that describe the class as it seems to clients, not how it is actually implemented. For example, here is an informal declaration of class Foo:

class Foo {
public:
        int x();
        int y;
        ~Foo();
};

The actual implementation might look like:

namespace internal {
        class FooBase  {
        protected:
                int x();
        };

        class Foo_v3: protected FooBase {
        private:
                int internal_stuff;
        public:
                using FooBase::x;
                int y;
        };
}

typedef internal::Foo_v3 Foo;

The example shows two cases where the actual implementation departs from the informal declaration:

• Foo is actually a typedef to Foo_v3.

• Method x() is inherited from a protected base class.

• The destructor is an implicit method generated by the compiler.

The informal declarations are intended to show you what you need to know to use the class without the distraction of irrelevant clutter particular to the implementation.

Summary

Template class for priority queue with concurrent operations.

Syntax

template<typename T, typename Alloc = cache_aligned_allocator<T> >
class concurrent_priority_queue;

Header

#include "tbb/concurrent_priority_queue.h"

Description

A concurrent_priority_queue is a container that permits multiple threads to concurrently push and pop items. Items are popped in priority order as determined by a template parameter. The capacity of the queue is unbounded, subject to memory limitations on the target machine.

The interface is similar to STL std::priority_queue except where it must differ to make concurrent modification safe.

Members

namespace tbb {
    template <typename T, typename Compare=std::less<T>,
                 typename A=cache_aligned_allocator<T> >
    class concurrent_priority_queue {
    public:
        typedef T value_type;
        typedef T& reference;
        typedef const T& const_reference;
        typedef size_t size_type;
        typedef ptrdiff_t difference_type;
        typedef A allocator_type;

        //Constructors
        concurrent_priority_queue(const allocator_type& a = allocator_type());
        concurrent_priority_queue(size_type init_capacity,
                                  const allocator_type& a = allocator_type());
        template<typename InputIterator>
        concurrent_priority_queue(InputIterator begin, InputIterator end,
                                  const allocator_type& a = allocator_type());
        concurrent_priority_queue(const concurrent_priority_queue& src,
                                  const allocator_type& a = allocator_type());
        //C++11 specific
        concurrent_priority_queue(concurrent_priority_queue&& src);
        concurrent_priority_queue(concurrent_priority_queue&& src,
                                  const allocator_type& a);
        concurrent_priority_queue(std::initializer_list<T> il,
                                  const allocator_type &a = allocator_type());

        //Assignment
        concurrent_priority_queue& operator=(const concurrent_priority_queue& src);
        template<typename InputIterator>
        void assign(InputIterator begin, InputIterator end);
        //C++11 specific
        concurrent_priority_queue& operator=(concurrent_priority_queue&& src);
        concurrent_priority_queue& operator=(std::initializer_list<T> il);
        void assign(std::initializer_list<T> il);

        void swap(concurrent_priority_queue& other);

        ~concurrent_priority_queue();

        allocator_type get_allocator() const;

        bool empty() const;
        size_type size() const;

        void push(const_reference elem);
        //C++11 specific
        void push(T&& elem);
        template<typename... Args>
        void emplace(Args&&... args);

        bool try_pop(reference elem);

        void clear();
    };
}

Summary

View set of items in a container as a recursively divisible range.

Requirements

A Container Range is a Range with the further requirements listed in below.

Model Types

Classes concurrent_hash_map and concurrent_vector both have member types range_type and const_range_type that model a Container Range.

Use the range types in conjunction with parallel_for, parallel_reduce, and parallel_scan to iterate over items in a container.

Summary

Computes reduction over a range.

Header

 #include "tbb/parallel_reduce.h"

Syntax

template<typename Range, typename Value,
         typename Func, typename Reduction>
Value parallel_reduce( const Range& range, const Value& identity,
                     const Func& func, const Reduction& reduction,
                     [, partitioner[, task_group_context& group]] );

template<typename Range, typename Body>
void parallel_reduce( const Range& range, const Body& body
                      [, partitioner[, task_group_context& group]] );

where the optional partitioner declares any of the partitioners as shown in column 1 of the Partitioners table in the Partitioners section.

Description

The parallel_reduce template has two forms. The functional form is designed to be easy to use in conjunction with lambda expressions. The imperative form is designed to minimize copying of data.

The functional form parallel_reduce(range,identity,func,reduction) performs a parallel reduction by applying func to subranges in range and reducing the results using binary operator reduction. It returns the result of the reduction. Parameter func and reduction can be lambda expressions. The table below summarizes the type requirements on the types of identity, func, and reduction.

The imperative form parallel_reduce(range,body) performs parallel reduction of body over each value in range. Type Range must model the Range concept. The body must model the requirements shown in the table below.

A parallel_reduce recursively splits the range into subranges to the point such that is_divisible() is false for each subrange. A parallel_reduce uses the splitting constructor to make one or more copies of the body for each thread. It may copy a body while the body’s operator() or method join runs concurrently. You are responsible for ensuring the safety of such concurrency. In typical usage, the safety requires no extra effort.

When worker threads are available, parallel_reduce invokes the splitting constructor for the body. For each such split of the body, it invokes method join in order to merge the results from the bodies. Define join to update this to represent the accumulated result for this and rhs. The reduction operation should be associative, but does not have to be commutative. For a noncommutative operation op, "left.join(right)" should update left to be the result of leftopright.

A body is split only if the range is split, but the converse is not necessarily so. The figure below diagrams a sample execution of parallel_reduce. The root represents the original body b0 being applied to the half-open interval [0,20). The range is recursively split at each level into two subranges. The grain size for the example is 5, which yields four leaf ranges. The slash marks (/) denote where copies (b₁ and b₂) of the body were created by the body splitting constructor. Bodies b₀ and b₁ each evaluate one leaf. Body b₂ evaluates leaf [10,15) and [15,20), in that order. On the way back up the tree, parallel_reduce invokes b₀.join(b₁) and b₀.join(b₂) to merge the results of the leaves.

Execution of parallel_reduce over blocked_range<int>(0,20,5)

The figure above shows only one possible execution. Other valid executions include splitting b ₂ into b ₂ and b ₃, or doing no splitting at all. With no splitting, b ₀ evaluates each leaf in left to right order, with no calls to join. A given body always evaluates one or more subranges in left to right order. For example, in the figure above, body b ₂ is guaranteed to evaluate [10,15) before [15,20). You may rely on the left to right property for a given instance of a body. However, you t must neither rely on a particular choice of body splitting nor on the subranges processed by a given body object being consecutive. parallel_reduce makes the choice of body splitting nondeterministically.

Example where Body b₀ processes non-consecutive subranges.

The subranges evaluated by a given body are not consecutive if there is an intervening join. The joined information represents processing of a gap between evaluated subranges. The figure above shows such an example. The body b₀ performs the following sequence of operations:

1. b₀( [0,5) )
2. b₀.join()( b₁ ) where b₁ has already processed [5,10)
3. b₀( [10,15) )
4. b₀( [15,20) )

In other words, body b₀ gathers information about all the leaf subranges in left to right order, either by directly processing each leaf, or by a join operation on a body that gathered information about one or more leaves in a similar way. When no worker threads are available, parallel_reduce executes sequentially from left to right in the same sense as for parallel_for . Sequential execution never invokes the splitting constructor or method join.

All overloads can be passed a task_group_context object so that the algorithm’s tasks are executed in this group. By default the algorithm is executed in a bound group of its own.

Complexity

If the range and body take O(1) space, and the range splits into nearly equal pieces, then the space complexity is O(P log(N)), where N is the size of the range and P is the number of threads.

Example (Imperative Form)

The following code sums the values in an array.

#include "tbb/parallel_reduce.h"
#include "tbb/blocked_range.h"

using namespace tbb;

struct Sum {
    float value;
    Sum() : value(0) {}
    Sum( Sum& s, split ) {value = 0;}
    void operator()( const blocked_range<float*>& r ) {
        float temp = value;
        for( float* a=r.begin(); a!=r.end(); ++a ) {
            temp += *a;
        }
        value = temp;
    }
    void join( Sum& rhs ) {value += rhs.value;}
};

float ParallelSum( float array[], size_t n ) {
    Sum total;
    parallel_reduce( blocked_range<float*>( array, array+n ),
                     total );
    return total.value;
}

The example generalizes to reduction for any associative operation op as follows:

• Replace occurrences of 0 with the identity element for op
• Replace occurrences of += with op= or its logical equivalent.
• Change the name Sum to something more appropriate for op.

The operation may be noncommutative. For example, op could be matrix multiplication.

Example with Lambda Expressions

The following is analogous to the previous example, but written using lambda expressions and the functional form of parallel_reduce.

#include "tbb/parallel_reduce.h"
#include "tbb/blocked_range.h"

using namespace tbb;

float ParallelSum( float array[], size_t n ) {
    return parallel_reduce(
        blocked_range<float*>( array, array+n ),
        0.f,
        [](const blocked_range<float*>& r, float init)->float {
            for( float* a=r.begin(); a!=r.end(); ++a )
                init += *a;
            return init;
        },
        []( float x, float y )->float {
            return x+y;
        }
    );
}

STL generalized numeric operations and functions objects can be used to write the example more compactly as follows:

#include <numeric>
#include <functional>
#include "tbb/parallel_reduce.h"
#include "tbb/blocked_range.h"

using namespace tbb;

float ParallelSum( float array[], size_t n ) {
    return parallel_reduce(
        blocked_range<float*>( array, array+n ),
        0.f,
        [](const blocked_range<float*>& r, float value)->float {
            return std::accumulate(r.begin(),r.end(),value);
        },
        std::plus<float>()
    );
}

Intel® Threading Building Blocks (Intel® TBB) has macros, an environment variable, and a function that reveal version and run-time information.

Version Macros

The header tbb/tbb_stddef.h defines macros related to versioning, as described below. You should not redefine these macros.

TBB_VERSION Environment Variable

Set the environment variable TBB_VERSION to 1 to cause the library to print information on stderr. Each line is of the form “TBB: tag value”, where tag and value are described below.

TBB_runtime_interface_version Function

Summary

Function that returns the interface version of the Intel® TBB library that was loaded at runtime.

Syntax

extern “C” int TBB_runtime_interface_version();

Header

#include "tbb/tbb_stddef.h"

Description

The value returned by TBB_runtime_interface_version() may differ from the value of TBB_INTERFACE_VERSION obtained at compile time. This can be used to identify whether an application was compiled against a compatible version of the Intel® TBB headers.

In general, the run-time value TBB_runtime_interface_version() must be greater than or equal to the compile-time value of TBB_INTERFACE_VERSION. Otherwise the application may fail to resolve all symbols at run time.

Context

Consider an interactive program. To maximize concurrency and responsiveness, operations requested by the user can be implemented as tasks. The order of operations can be important. For example, suppose the program presents editable text to the user. There might be operations to select text and delete selected text. Reversing the order of "select" and "delete" operations on the same buffer would be bad. However, commuting operations on different buffers might be okay. Hence the goal is to establish serial ordering of tasks associated with a given object, but not constrain ordering of tasks between different objects.

Forces

• Operations associated with a certain object must be performed in serial order.

• Serializing with a lock would be wasteful because threads would be waiting at the lock when they could be doing useful work elsewhere.

Solution

Sequence the work items using a FIFO (first-in first-out structure). Always keep an item in flight if possible. If no item is in flight when a work item appears, put the item in flight. Otherwise, push the item onto the FIFO. When the current item in flight completes, pop another item from the FIFO and put it in flight.

The logic can be implemented without mutexes, by using concurrent_queue for the FIFO and atomic<int> to count the number of items waiting and in flight. The example explains the accounting in detail.

Example

The following example builds on the Non-Preemptive Priorities example to implement local serialization in addition to priorities. It implements three priority levels and local serializers. The user interface for it follows:

enum Priority {
   P_High,
   P_Medium,
   P_Low
};

template<typename Func>
void EnqueueWork( Priority p, Func f, Serializer* s=NULL );

Template function EnqueueWork causes functor f to run when the three constraints in the following table are met.

Constraints on a given functor are resolved from top to bottom in the table. The first constraint does not exist when s is NULL. The implementation of EnqueueWork packages the functor in a SerializedWorkItem and routes it to the class that enforces the first relevant constraint between pieces of work.

template<typename Func>
void EnqueueWork( Priority p, Func f, Serializer* s=NULL ) {
   WorkItem* item = new SerializedWorkItem<Func>( p, f, s );
   if( s )
       s->add(item);
   else
       ReadyPile.add(item);
}

A SerializedWorkItem is derived from a WorkItem, which serves as a way to pass around a prioritized piece of work without knowing further details of the work.

// Abstract base class for a prioritized piece of work.
class WorkItem {
public:
   WorkItem( Priority p ) : priority(p) {}
   // Derived class defines the actual work.
   virtual void run() = 0;
   const Priority priority;
};

template<typename Func>
class SerializedWorkItem: public WorkItem {
   Serializer* serializer;
   Func f;
   /*override*/ void run() {
       f();
       Serializer* s = serializer;
       // Destroy f before running Serializer’s next functor.
       delete this;
       if( s )
           s->noteCompletion();
   }
public:
   SerializedWorkItem( Priority p, const Func& f_, Serializer* s ) :
       WorkItem(p), serializer(s), f(f_)
   {}
};

Base class WorkItem is the same as class WorkItem in the example for Non-Preemptive Priorities. The notion of serial constraints is completely hidden from the base class, thus permitting the framework to extend other kinds of constraints or lack of constraints. Class SerializedWorkItem is essentially ConcreteWorkItem from the example for Non-Preemptive Priorities, extended with a Serializer aspect.

Virtual method run() is invoked when it becomes time to run the functor. It performs three steps:

1. Run the functor

2. Destroy the functor.

3. Notify the Serializer that the functor completed, and thus unconstraining the next waiting functor.

Step 3 is the difference from the operation of ConcreteWorkItem::run. Step 2 could be done after step 3 in some contexts to increase concurrency slightly. However, the presented order is recommended because if step 2 takes non-trivial time, it likely has side effects that should complete before the next functor runs.

Class Serializer implements the core of the Local Serializer pattern:

class Serializer {
   tbb::concurrent_queue<WorkItem*> queue;
   tbb::atomic<int> count;         // Count of queued items and in-flight item
   void moveOneItemToReadyPile() { // Transfer item from queue to ReadyPile
       WorkItem* item;
       queue.try_pop(item);
       ReadyPile.add(item);
   }
public:
   void add( WorkItem* item ) {
       queue.push(item);
       if( ++count==1 )
           moveOneItemToReadyPile();
   }
   void noteCompletion() {        // Called when WorkItem completes.
       if( ‐‐count!=0 )
           moveOneItemToReadyPile();
   }
};

The class maintains two members:

• A queue of WorkItem waiting for prior work to complete.

• A count of queued or in-flight work.

Mutexes are avoided by using concurrent_queue<WorkItem*> and atomic<int> along with careful ordering of operations. The transitions of count are the key understanding how class Serializer works.

• If method add increments count from 0 to 1, this indicates that no other work is in flight and thus the work should be moved to the ReadyPile.

• If method noteCompletion decrements count and it is not from 1 to 0, then the queue is non-empty and another item in the queue should be moved to ReadyPile.

Class ReadyPile is explained in the example for Non-Preemptive Priorities.

If priorities are not necessary, there are two variations on method moveOneItemToReadyPile, with different implications.

• Method moveOneItemToReadyPile could directly invokeitem->run(). This approach has relatively low overhead and high thread locality for a given Serializer. But it is unfair. If the Serializer has a continual stream of tasks, the thread operating on it will keep servicing those tasks to the exclusion of others.

• Method moveOneItemToReadyPile could invoke task::enqueue to enqueue a task that invokes item->run(). Doing so introduces higher overhead and less locality than the first approach, but avoids starvation.

The conflict between fairness and maximum locality is fundamental. The best resolution depends upon circumstance.

The pattern generalizes to constraints on work items more general than those maintained by class Serializer. A generalized Serializer::add determines if a work item is unconstrained, and if so, runs it immediately. A generalized Serializer::noteCompletion runs all previously constrained items that have become unconstrained by the completion of the current work item. The term "run" means to run work immediately, or if there are more constraints, forwarding the work to the next constraint resolver.