Non-normal form closures, or thunks, are treated specially to ensure that exactly one copy of the thunk exists. When shipping a thunk between PEs we want to convert the closure on the source PE into a Fetch-me to the new location of the thunk on the destination PE. However, there are two problems with this:
These problems are resolved by using revertable black holes and by sending ACK messages to acknowledge every packet that contains one or more thunks. A revertable black hole acts like a normal black hole, except that it also records the original value of the closure. Should a subsequent thread enter the revertable black hole, i.e. demand the thunk that has just been shipped away, it becomes blocked. By turning a thunk into a revertable black hole, we can ensure that it can be restored in the unlikely event that the packet is rejected by the destination. One reason for rejecting the packet would be the onset of global garbage collection, which is not currently implemented. If the packet is rejected a NACK message is sent, identifying the thunks that couldn't be unpacked.
For example, if a thunk with GA A42 is packed on PE A for shipment to PE B, it is converted into a revertable black hole closure on A. When PE B unpacks the thunk, it allocates it a new GA, e.g. B59. After unpacking the entire packet B sends an ACK message to A containing a pair of GAs for every thunk shipped in the packet, e.g. (A42, B59). PE A overwrites the revertable black hole identified by A42 by a Fetch-Me to B59: the thunk's new location, and reawakens any blocked threads. Typically these threads are very short-lived: they enter the Fetch-me, send a FETCH, and block awaiting the response.
We have observed programs where a thunk is repeatedly copied from PE to PE, generating a chain of Fetch-Mes. In the worst case a FETCH message might conceivably chase a thunk endlessly round a cycle of PEs. This behaviour has not been observed.