Screwtape's Notepad

Real VT102 emulation with MAME

As a software developer, I spend a lot of time using the terminal, or more properly, a “terminal emulator”. Tools like iTerm2, or PuTTY, or GNOME Terminal, or the classic xterm and rxvt are all “terminal emulators”; they take raw text and formatting instructions and interpret them to display pretty text on the screen. But if these tools which are commonly called “terminals” are actually “terminal emulators”, then what’s the actual terminal they’re emulating?

What is a terminal?

These days, most terminal emulators are really emulating xterm; it’s one of the oldest and most featureful terminal emulators, so for compatibility reasons other terminal emulators tend to follow it. But xterm’s webpage says it “provides DEC VT102 and Tektronix 4014 compatible terminals”. That is, xterm is an emulator for the VT102 terminal:

DEC VT100 terminal
Jason Scott / CC BY (https://creativecommons.org/licenses/by/2.0)

(technically, that’s a VT100, but they’re quite similar)

The Digital Equipment Corporation, or DEC for short, built many different kinds of terminals, but the VT100 was the one that really took off, and all subsequent models (the VT220, VT320, VT420, VT520 and all their variants) were backwards compatible with it. Since they were one of the oldest and most featureful families of terminals, for compatibility reasons xterm decided to follow them.

Implementing a terminal emulator

xterm (and all its more modern competitors) emulate the VT102 (etc.) in basically the same way: reading through the documentation, and implementing each listed feature one-by-one. The docs say that when the terminal receives the sequence ESC [ 5 A it moves the cursor 5 lines up, so as long as our terminal does the same thing, it’s correct. However, if you’re a modern software developer, that should make you nervous: confronted with a laundry-list of features, you can expect that they almost certainly interact with each other in surprising and difficult-to-predict ways, and you’ll probably need a lot more documentation, and a detailed test suite to be confident two implementations are compatible.

Sadly, so far as I know, there is no such detailed documentation or test-suite for terminals and terminal emulators. Because they’re fundamentally about visual output, it’s quite difficult to properly test them.

Instead, they’re generally tested the old-fashioned way: running applications to see if anything looks weird. The next generation of applications are tested against the current generation of terminal emulators, the next generation of terminal emulators are tested against the current generation of applications, and so forth. This has worked out surprisingly well, but there’s a certain amount of information loss along the way. For example, the VT100 supported rendering double-width text, and xterm supports it too, but many other terminal emulators don’t, so applications don’t use it, so future terminal emulators don’t bother.

Instead of comparing each terminal emulator against previous terminal emulators, it would be neat if we could compare directly against an original VTxxx terminal. But those are extremely rare these days, and even if you want to buy one second-hand, they’re heavy and fragile.

Emulation with MAME

You might have heard of MAME in the context of video-games; after all, it was originally the Multiple Arcade Machine Emulator. However, many arcade machines were built from similar components, hooked together in similar ways, so MAME wound up being a fairly generic “computer components wired together” system, and today it emulates all kinds of computers and game consoles and computerised gadgets as well as arcade machines. In particular, and relevant to our interests, it emulates the VT102. This isn’t documentation-as-feature-list implementation, like xterm or other terminal emulators: MAME uses a copy of the original firmware, and inteprets it with an emulated CPU talking to an emulated serial port and emulated video hardware. This is about the best recreation of a VT102 you can get without taking up half your desk.

Setting up VT102 emulation with MAME

To use a MAME-emulated VT102, you’ll need a couple of things:

Launching MAME

Once you have all these things, it’s time to start up MAME. Here’s the command-line to use:

mame -biospath path/to/firmware/dir -window -nomouse -vol -15 vt102 -rs232 pty

In this command-line:

Once you launch MAME, you’ll get a warning screen about the emulation being incomplete. Hit a key to get past it. Next, there’s an information screen about the hardware being emulated. Hit a key to get past that too. Finally, the emulated VT102 should boot up. At first it will say “Wait” in the top left, then you’ll get some beeping, and you’ll see something like this:

Emulated VT102 showing a 2 in the top left

That 2 in the top left means that the hardware needs to be configured. Fair enough, it hasn’t been powered on for forty years.

Configuring the VT102

In order to configure the VT102, you’ll need to examine a diagram of the VT102 keyboard. See those labels above the numbers at the top of the keyboard? Those tell you what the number keys do in the setup screen.

You’ll also notice a big “SET UP” key at the top-left of the keyboard. Your PC keyboard doesn’t have a SET UP key, but MAME maps it to the F5 key by default. Press F5, and you should enter “SET-UP A” mode:

Emulated VT102 the first config screen

There’s many things you can configure in the SET-UP screen, including such classics as “light text on dark background or dark text on light background”, “should the cursor be a block or an underline”, and “should the keyboard click when I type a key”. However, there’s only one thing we strictly need to change to get things working.

At the bottom of the MAME window, you’ll see two labels, “ON LINE” and “LOCAL”. By default, “LOCAL” is lit, meaning that typing at the keyboard goes straight to the display, not to the remote computer. On the keyboard diagram, you’ll see the “4” key is labelled “LINE/LOCAL”. Now that we’re in SET-UP mode, press 4 to switch to “ON LINE” mode.

Once you’re done, press Shift-S to save your settings. The screen should clear, and display “Wait” in the top left for a second or two, then return you to the SET-UP A screen. Note that the VT102 is an exclusively QWERTY machine — if you’re using another keyboard layout, you’ll need to press the key that produces S on a QWERTY keyboard.

Finally, press F5 again to leave SET-UP mode.

Configuring MAME

Way back when we launched MAME, we gave it the command line option -rs232 pty which caused MAME to create a PTY device to communicate with the outside world. MAME is at one end of the PTY, we connect a shell (or any other app) to the other end, and presto! the app is running inside the emulated VT102. The question is, where is this PTY? MAME will tell us, but we have to fight it a little first.

Traditionally, to bring up MAME’s user interface, you press Tab. MAME is normally used to emulate video-games, which typically don’t have a Tab key, so this works well — but the VT102 does have a Tab key, and we’d like to be able to use it. MAME’s solution is to support multiple keyboard modes: by default MAME starts in “FULL Emulation” mode, where every key is handled by the VT102; if you press the toggle-keyboard-emulation key, it switches to “PARTIAL Emulation” mode, where most keys are handled by the VT102 but some are handled by MAME itself. Traditionally, the toggle-keyboard-emulation key is the Scroll Lock key, but on my machine it was the Insert key for whatever reason. Play around with rarely-used keys on your keyboard, and eventually you should see a message about PARTIAL Emulation:

MAME's keyboard-mode status popup

Once you’re in PARTIAL Emulation mode, you can press the same key to get back to FULL Emulation, or press the Tab key to open MAME’s emulation menu:

MAME's emulation settings menu

In this menu, the most important item is “Pseudo terminals”, about a third of the way down. You can use the cursor keys, Enter, and Escape to navigate the menu. If you highlight “Pseudo terminals” and press Enter, you’ll be taken to the “Pseudo terminals” sub-menu, which shows you the path to the terminal device connected to the VT102’s emulated serial port:

MAME's Pseudo terminals menu, stating that the RS232 port is connected
    to /dev/pts/10

Make a note of the path to the terminal device, then press Tab again to close MAME’s menu, and press the keyboard-mode key again to switch back to FULL Emulation mode.

Configuring the PTY

As I mentioned earlier, the PTY is the interface between the terminal and the application running inside it. To use a physical VT102, you connect it to a physical serial port on your computer, and (like any device) your operating system has drivers and configuration for it. When people stopped using terminals and started using terminal emulators, PTYs were invented to be a kind of fictional serial port, that supports the same configuration as a real serial port but ignores most of it.

Most, but not all. When most terminal emulators create a PTY, they give a standard, sensible configuration. MAME, for whatever reason, does not — at least, not in the version I’m using. Luckily, we can use the standard Unix serial-port-configuration tool on the PTY to match the configuration the VT102 expects. Here’s the command-line to use:

stty sane cols 80 rows 24 erase ^H ixon < /dev/pts/10

In this command-line:

Launching a shell

The terminal is waiting, the PTY is configured, it’s time to launch a shell. Here’s the command-line to use:

TERM=vt102 LC_ALL=C COLUMNS=80 LINES=24 /bin/sh </dev/pts/10 >/dev/pts/10 2>/dev/pts/10

In this command-line:

Once it’s running, you should get a shell-prompt in MAME:

A shell prompt with a visible cursor

…and now you can try out your favourite tools:

The Vim text editor running happily

Problems still to solve

The setup described above works fairly well, but unfortunately there’s still some problems I don’t know how to address.

Input buffer overflows

The VT102 talks to the outside world through a serial port that defaults to 9600 baud, which is about a kilobyte per second, or slightly under a millisecond per character. That sounds pretty slow in our modern age of gigabit networking, but for a poor little machine built in 1981 with a 6MHz CPU, it can be a bit of a strain. It’s pretty simple to take an incoming character and draw it to the screen, but when the terminal has to scroll the entire screen to make room first, that can take more than a couple of milliseconds.

To deal with this problem, the VT102 has a 128-character input buffer. Each character that arrives is put at the back of the queue, and the CPU takes characters from the front and processes them as quickly as possible. When the buffer is one quarter full, the VT102 sends a XOFF character (^S), which tells the computer at the other end to pause transmission. Hopefully, the other end will receive the XOFF and pause transmission before the buffer is filled — at 9600 baud, that’s a safety margin of about 96 milliseconds, which should be ample time.

Unfortunately, the emulated VT102 is not connected through a real serial port, it’s connected through a PTY which (at least compared to what the VT102 expects) is basically infinitely fast. As a result, when you run a command that produces a lot of output, it can be garbled and abbreviated even though XON/XOFF flow control is enabled and working — the command produces enough data to fill the buffer before the XOFF character can take effect.

The vttest program displaying an obviously-wrong test case

In this screenshot, you can see a couple of instances of the chequerboard glyph (▒) that the VT102 uses to represent “I missed some text”. There’s one near the top-left, one near the bottom-left, and one in the mid-left, a few cells left of the word “middle”.

I don’t really know what to do about this. I think ideally, MAME’s serial-port emulation should be updated to forcibly rate-limit input to match the configured baud rate, and maybe to manually implement flow control since apparently the kernel can’t do it reliably. I don’t think there’s much that can be done outside MAME.

No job-control

A modern desktop OS typically runs a special application called the “desktop shell”, which allows the user to launch other applications (or groups of applications in a “virtual desktop”), switch between them, and forcibly close them.

The Unix terminal subsystem supports similar concepts: the command-line shell is a special application which allows the user to launch other applications (or groups of applications in a “job”), switch between them (with the ^Z key and commands like fg and bg) and forcibly close them (with ^C and ^\).

The concept that allows this to work is the “controlling terminal”. Every Unix process may be controlled by some terminal; when a terminal sends the kernel a keystroke like ^C or ^Z, the kernel may interrupt or suspend some of that terminal’s processes. In order for a shell to be able to manage jobs in a particular terminal, that terminal must be the shell’s controlling terminal. However, the above instructions for launching a shell result in the shell’s controlling terminal being the terminal where it was launched, not the PTY that MAME created for us. As a result, it can’t manage multiple jobs, and certain applications (like less(1)) will work incorrectly.

Normally, it’s the terminal emulator’s responsiblity to ensure that the program running inside has the correct controlling terminal set. Unfortunately, as we’ve seen, MAME doesn’t think of itself as a terminal emulator and so it doesn’t do the normal setup. Even more unfortunately, there’s no standard tool to set the controlling terminal ourselves, in the way that stty lets us configure the terminal ourselves. We could write such a tool, of course — the most important parts would be setsid(2) to clear the current controlling terminal, and the TIOCSCTTY ioctl to set a new one, but we could do all the stty and environment-variable things too — but until the flow-control situation is fixed I don’t think it’s worth the effort.

Actually, I tell a slight lie: there is a classic tool that sets up a controlling terminal: getty. It’s normally used to manage a real hardware serial port, answering incoming dial-up modem calls and launching the login process. Once MAME creates the PTY, we can pretend it’s a real hardware serial port, run getty on it, and pretend we’re teenagers in a 1980s movie hacking into the mainframe:

A classic Unix login screen

There’s a few shortcomings with this approach, however:

If you’d like to try it regardless, here’s the command-line to use:

sudo setsid /sbin/getty pts/10 vt102

In this command-line:

Conclusion

MAME’s VT102 emulation works well enough as a novelty and for light, interactive testing, but I wouldn’t want to have to use it regularly. The situation could be made much better if MAME emulated serial port flow control, and if there were a tool to launch a program with a specific controlling terminal — or better yet, if MAME’s PTY support included launching a program inside the PTY it creates.

Some of those things probably wouldn’t even be too hard to do, but right now I’ve spent enough time playing with this stuff, and it’s time to move on.