The Coroutine Language

Logical branching: if, || &&

Coroutine if statements work similarly to base R.

Short circuiting logical operators || and && will try first their left branch, which must return a scalar logical. If not satisfied with the left branch, they will evaluate the right argument and return it directly. That is, in a coroutine yield(TRUE) && "string" may return “string”, while in base R this raises an error.

I write “may” above, because if you write || in a coroutine expression you might be using either base R || or the pausable version of ||. In general you will only be using pausable || if there is a pause somewhere on either side of it. So if you write (x == 5) || FALSE you will be using base R ||, since == does not pause. Whereas if you write (await(success) == 5) || yield("assertion") you will be using pausable ||, because there is an await on one side of it.

switch and goto

switch() statements are more strict than in base R; the input to a switch must match a given branch or it will raise an error. That is, in base R, switch(-1, "not run") or switch("bar", foo="not run") returns invisible NULL, while in a coroutine this is an error.

Additionally, switch statements support a delimited goto. Within a given switch statement, goto("other_branch") stops executing the present branch and jumps to the given sibling branch. Calling goto() without arguments re-reads the switch input again. For example, you might try to read an input file that might be in a few different formats.

file_dataset <- async({
  filename <- await( download_file(url) )
  switch(getExtension(filename),
      "txt"=readTxt(filename),
      "csv"=tryCatch(readCSV(filename), error=goto("txt")),
      "json"={
          if (!await(validateSchema(filename))) goto("txt")
          tryCatch(read_json(filename), error=goto("txt"))
      },
      "zip"= {
        unzipped <- unzip_async(filename) |> await()
        filename <- unzipped
        on.exit(unlink(unzipped))
        goto(getExtension(unzipped))
      }
  )
})

Here, if there is an error in reading a .csv or .json file we re-try ingesting it as a text file; on encountering a zip file we unzip it and try again with the new filename.

Note that if a goto appears inside of a try(..., finally={...}) call, which is itself inside a branch, the finally clause will be executed before jumping to the new branch.

Starting

• run is “eager;” executes an entire coroutine expression immediately without pausing.
• Generators are “lazy;” at the start, a gen does not compute anything when it is first constructed, it only begins executing when there is a request for data via nextOr.
• async routines are “eager;” they begin executing immediately on creation, before the constructor returns. An async runs until it gets the first await and then pauses. allowing the constructor to return. Upon receiving, the async will continue running.
• stream objects can run in either lazy or eager mode. If lazy=TRUE, they will not compute anything until something calls the channel’s nextThen method, and will pause after yield if there are no more listeners. If lazy=FALSE, a stream starts executing when constructed, will not pause after yield, and may run ahead and queue up outgoing values until it reaches an await.

Pausing

• await(p) in an async or stream, pauses execution until the given promise resolves.
• yield(val) in a generator pauses execution, returns the given value as the result of nextOr().
• yield(val) in a stream, resolves the next listener with the value given. It may or may not pause depending on whether lazy=TRUE and whether there are any more pending requests/
• yieldFrom(i) takes an iterator or a stream as argument, and yields successive values until it is exhausted. yieldFrom(iter) is basically equivalent to for (i in iter) yield(i).

Finishing

A coroutine might finish normally by reaching a call to return(...), or by simply reaching the end of its expression.

If a coroutine reaches a return call, all enclosing tryCatch(finally=) and on.exit handlers will be executed before returning the value.

• When a generator finishes normally, the present call nextOr() forces its or argument. The value returned by the generator expression is discarded.
• When an async finishes normally , its promise interface resolves with the return value.
• When a stream finishes normally, the listeners will be notified of a close after any remaining messages have been read from the channel’s outgoing queue.

Errors

A coroutine might finish abnormally if the user-level R code throws an error, or if there is an internal error in the coroutine implementation.

• When a gen or run routine finishes abnormally, all on.exit clauses are triggered, while the inciting error is allowed to propagate normally without being caught.
• If a async routine throws an error, it will be caught and the error will be forwarded to reject a promise.

Cleanup

on.exit handlers are run in a special phase after main execution finishes (either normally or abnormally). After on.exit handlers have all finished, the coroutine will continue to propagate its error or its return value.

If you were to do something as strange as return() or yield() from an on.exit handler, any pending error will be cancelled. (This is also how base R’s on.exit behaves.)

Cleanup via on.exit

A coroutine may have a list of exit handlers. The expr argument to each on.exit is pulled out at parse time to make each handler; so that their effective scope is outside of any enclosing loops or tryCatch calls.

At run time, calling on.exitregisters the given handler to be executed later.. When the coroutine finishes, either by normal return or by error, the coroutine saves the return value and starts executing each handler that was registered. After the last on.exit handler has been executed, the saved value is returned.

A typical use of on.exit might be to close a file or other resource after you are done reading; here’s a simple generator that reads a file one line at a time:

fileLines <- gen(function(...) {
  f <- file(..., open="rt")
  on.exit(close(f))
  while (length(line <- readLines(f, 1)) > 0) yield(line)
})
length(as.list(fileLines("language.Rmd")))

## [1] 306

It is possible (but rather strange) to call stop() or return() from inside an on.exit handler. In that case the new value and disposition overrides the old one.

x <- run({
    on.exit({return("this instead")})
    stop("an error")
})

## Error in eval(val, targetEnv): an error

x

## [1] "this instead"

Here he have an odd situation where an error is printed out to the console, but the value was returned from run anyway. (Base R also does this.)

Cleanup with try/finally

tryCatch(expr={...}, finally={...}) will execute expr, and after expr finishes, whether it finishes normally or not, tryCatch will then execute finally.

The finally clause will also be executed if you exit from expr without finishing it, for instance by break or next in a loop, or by goto in a switch statement.

Assuming finally finishes normally, tryCatch will either return the saved return value or throw the saved error from expr. If the finally clause throws an error then the original error will be lost.

A caveat about cleanup

Writing code in a coroutine’s on.exit clause or in a tryCatch(finally=) call does not give any guarantee that the cleanup code will ever run, because coroutines only execute “on demand.” If you make a generator designed to return ten items, then call nextOr ten times, the generator will never run its exit handler. The generator will just be paused waiting to reach end of iteration; you need to call nextOr an eleventh time to allow it to reach the end, run ts exit handlers and signal end of iteration.

finished <- FALSE
g <- gen({
  on.exit(finished <<- TRUE)
  for (i in 1:10) yield(i)
})
for (j in 1:10) nextOr(g, break)
finished

## [1] FALSE

nextOr(g, "stop")

## [1] "stop"

finished

## [1] TRUE

Similarly, an async with an on.exit or finally handler will not exevute those handlers if it is paused on an await that never resolves. With those caveats in mind, coroutines do implement both on.exit and finally constructs and they are useful for sequencing cleanup actions.

tryCatch and iteration performance

When a coroutine enters a tryCatch it has to momentarily save state so that it can wind up an R-level tryCatch on the stack and then resume. If there is a a yield or await in the tryCatch expression, the coroutine has to exit from its tryCatch and re-establish it when execution resumes.

This means that yielding from inside a tryCatch can be more costly, performance-wise, than a regular yield. This is part of what motivated the development of the [iterors package][iterors::iteror-package]; desiring a protocol for iteration that did not involve using exceptions for normal flow control. Just eliminating tryCatch caused a 2x speedup for many generators, and the iterors package contains many other optimizations besides.