How to Do tserialio

By Chris Pirazzi.

This file lives in eoe/lib/libtserialio/doc/tserialio.html.

I am not at SGI any more. By the time you read this, there may be a new tserialio expert. Check the rlog for the files below. If not, best bet for reaching me is cpirazzi@cs.princeton.edu. But please read everything here before you try and get a hold of me!

Read this document before attempting to modify tserialio. It will teach you what you need to know to avoid introducing bugs.

This document was written on 12/19/97. At this point there is a compiling tserialio driver in the kudzu tree, however it contains many serious problems and will not work in any useful sense.

Below, I will explain the proper steps to port tserialio from bonsai (6.3) to kudzu (6.5).

Where it is

On 12/18/97, tserialio consists of:

Unlike the standard STREAMS or other interface, the user does not open tserialio's device node (/dev/tty*) directly. Instead, the user links with the user-mode library libtserialio.c, and uses only its documented API entry points.

Tserialio does not have or need any flow control of any kind (not hardware/sofware, not XON/XOFF).

How to Learn tserialio

Read these things in this order:

tserialio and Timeliness: How Applications Use tserialio

Before modifying tserialio, you need to think extra carefully about timeliness.

tserialio is unlike any other driver in irix/kern. In fact, it is more like the audio and MIDI driver. typically in a driver, we do all we can to optimize bandwidth (throughput), sometimes at the cost of latency. tserialio needs exactly the opposite. tserialio is used for applications like deck control, which involve absolutely miniscule amounts of data (38400 baud, and the serial line is often not at all saturated), but involve extremely tight accuracy constraints. tserialio is 100% completely useless unless it can deliver the stated accuracy guarantees, 100% of the time. a failure means that video deck control overwrites customer data, which in the video industry means you send it back (no, it's not like Microsoft). tserialio uses a guaranteed low latency event in the kernel to run every millisecond so that it can deliver the very high accuracy timestamping and scheduling described in the tserialio man page.

If it were possible to run every millisecod, guaranteed, 100% of the time from a user-mode thread, we wouldn't need the tserialio driver, and the tserialio user-mode API would be more of a convenience than any kind of new functionality for the developer.

However, as of 12/19/97, the company has decided not to allocate the engineering resources necessary to guarantee that a user-mode thread can run every millisecond on O2 and Octane for IRIX 6.5. So O2 and Octane need the tserialio driver.

The event which the tserialio driver uses is called the millisecond profiler tick. There is a nasty, kludgy hook from irix/kern/ml called tsio_timercallback_ptr which tserialio hooks into. This tick comes at splprof, which is higher than splhi. This interrupt is not threaded, nor could it be threaded and deliver the required latencies on O2/Octane. splprof is so high that almost none of the standard kernel facilities which you're used to (semaphores, memory allocation, address space manipulation (memory mapping and unmapping), sleeping) work. About the only kernel facility which the interrupt can do is to schedule a lower-priority timeout routine (known as a timepoke), from which tserialio can use other kernel facilities such as pollwakeup().

In tserialio, as described in the comment you should have read above, the interrupt routine reads (for serial TX) or writes (for serial RX) an area of main memory containing a ringbuffer. That ringbuffer is mapped into the address space of the user process using the tserialio API. So once a TSport is set up, there is actually very little tserialio needs to do in terms of using kernel facilities.

On Onyx2/Origin, the company has promised millisecond schedulability of a user-mode process. So on those platforms, don't bother porting the tserialio driver. Instead, you should write a very simple (should be at most a few hundred lines of code) "tserialio veneer" DSO in user-mode. This veneer will have the exact same API as the driver-based tserialio.so, but it will spawn off a sproc/pthread thread which runs every millisecond (which you can do by making it pri 255 and SCHED_RR or SCHED_FIFO (see sched_setscheduler(2))) and polls the serial device using any serial interface (presumably userualio or cserialio). The veneer will use some selectable object (semaphores, pipes, ...) to implement the tserialio API's selectable file descriptor. Semaphores would be the obvious choice, but due to the bizarre semantics of us selectable semaphores, you may find that you need to find another mechanism. You may find pipes to be of use. For communication between the veneer thread and the API thread, the veneer can use the same mapped ringbuffer trick as the current API and driver do. The only difference is that the whole shebang will reside in the address space of one share group (sproc) or process (pthread).

As other platforms take on the millisecond schedulability guarantees, you can discard their tserialio driver and go with the veneer.

Update: 1998 October 27

Chris Pirazzi and I had an email exchange about latency tolerance in the tserialio driver. Below is some additional information to help clarify the scheduling needs to the tserialio driver.

The tserialio driver requires a one millisecond scheduling period with one millisecond scheduling latency. So, at the extreme, the tserialio driver can tollerate no more than 2 ms betweeen its execution. This scheduling requirement also applies to the midi and audio drivers.

Chris agreed to a more strict definition:

The tserialio timeout must be scheduled to run every 1000 us with no skew in scheduling frequency, and the maximum holdoff, i.e., the time from when a timer expires until the affected thread runs, that the tserialio timeout can tollerate is 1000 us.
Based on testing with real video decks, the above definition is overly strict, but the remainder of the statement is correct.

As of IRIX 6.5.2, the OCTANE platform has guaranteed one millisecond bounded response time, in contrast to Chris's comment on 1997 December 19. The tserialio driver runs as a periodic xthread on the IP27 (Onyx2, Origin200, Origin2000) and IP30 (OCTANE). Other platforms may be added over time by defining TSERIALIO_TIMEOUT_IS_THREADED in irix/kern/kcommondefs for each additional platform. Note: One millisecond bounded response for a platform is a prerequesite to proper execution of a threaded tserialio driver, so ensure proper bounded response before defining TSERIALIO_TIMEOUT_IS_THREADED.

--Brian Forney (bforney@sgi.com)

tserialio and Bandwidth

People use tserialio for video deck control and MIDI. Both are puny in bandwidth:
MIDI == 31250 sym/sec / (1start+8data+1stop)=10 sym/byt 
     == 3125... bytes/second

deck == 38400 sym/sec / (1start+8data+1par+1stop)=11 sym/byt
     == 3491... byt/sec
As described above, latency and accuracy are what is important to the tserialio driver.

Accuracy and Latency Requirements for Video Deck Control and Video Deck Emulation

It will help you quite a bit if you understand the kinds of applications for which tserialio is used.

tserialio is used for video deck control. Video deck control needs highly accurate scheduling and timestamping of serial bytes.

Another extremely popular application is video deck emulation. Its requirements are stricter. Video deck emulation requires low latency user-thread scheduling in addition to the high accuracy of video deck control.

Please read the following blurb about these two applications. This blurb was written with someone in mind who is stronger with OS concepts than they are with video concepts: click here for millideck.

There is some other information about video deck control, including sample code using tserialio, in the Lurker's Guide to Video at http://collabmed.engr.sgi.com/projects/lurker/.

tserialio and Mountaingate

Mountaingate is a customer with one of those "special" relationships with SGI. They want to make an O2 do video deck emulation and will supposedly sell lots of systems. Yeah we've heard it before.

Anyway, back when it seemed likely that they will sell lots of systems, we did a very carefully crafted and limited hack to the tserialio IRIX 6.3 driver which would allow them to have a small part of their driver run from tserialio's millisecond callback in the kernel. Mountaingate was extremely limited as to what they could do from this callback.

To see the contract and the precise hack, see the file mgate1.9 (or whatever the highest-numbered mgate* is right now) in eoe/lib/libtserialio/mountaingate.

As you can see, we explicitly said we were not on the hook for IRIX 6.5 support. So you can choose to omit the mountaingate hacks if you want when porting to IRIX 6.5. As you can see by diffing the "with" and "without" tserialio.c in eoe/lib/libtserialio/mountaingate/tserialio_driver_hack, they are not that complex.

There is one very lightweight support issue with Mountaingate. Because we provide them with a custom tserialio.o, we need to make sure that any patches they install do not conflict with the tserialio.o we provided. Part of our contract with Mountaingate is that we will qualify or disqualify any patches they want to install. Examples of patches which Mountaingate should not be allowed to install:

Qualifying a patch usually consists of looking at its idb file (/hosts/dist/test/patches/patchXXXX/dist/*.idb) to see what it includes, and if it includes something suspicious in the kernel, going to its source files (find /hosts/jake/proj/patches/patchXXXX -type file -print) and seeing what changed.

Mountaingate asks for a set of patches to be qualified once every few months.

Porting tserialio to IRIX 6.5

The current 12/19/97 TOT irix/kern/tserialio.c for kudzu is 1.7. This file compiles but has many severe problems. The port to kudzu was done based on a non-top-of-trunk bonsai tserialio.c, so many critical changes and fixes from bonsai did not make it into the kudzu source. In addition, several kudzu updates were done in an incorrect manner.

After surveying the changes for a week I am absolutely, 100% sure that it is worth starting over from scratch with the bonsai source. This is definitely the fastest way to get kudzu tserialio working.

Some of the items below are such that it takes me longer to explain them than it would for me to do them before I leave. But this would be a short-sighted strategy. If tserialio is to survive in the long term, or even if it is to ever get ported to NT, you need to understand the intricacies of the driver, and so I will spend my time explaining it and not putzing with it.

So here's the process you need to do unless otherwise specified, all changes are to the driver. Do these things in this order:

There are a few features you might want to add and optimizations you might want to do. Some of these optimizations you'll definitely want to do if you run on platforms with many serial ports: