Swift on Mac OS 9 // -dealloc
It’s April 1, and which means it’s each April Fools’ Day and the anniversary of the founding of Apple Inc. While this yr is a sober one resulting from present occasions, I believe lots of people nonetheless respect what individuals are creating and sharing to maintain spirits up, whether or not that be music or artwork or…impractical programming tasks. And whereas pranks on April Fools’ appear much less and fewer enjoyable, apparent jokes and whimsy, not at anybody’s expense, are nonetheless one thing I imagine in…and even higher if they really work.
What’s a Mac OS 9?
Twenty (!) years in the past, earlier than the macOS we all know right now, there was one other working system often called “Mac OS”. It was one of many first OSs to make use of a GUI in any respect, one thing that we just about take without any consideration nowadays. It additionally dates from the times when just one program might run at a time; due to that, even the newest model makes use of cooperative multitasking to run a number of packages—that’s, a program has to yield its time to let others run. If a program crashed or overwrote reminiscence it wasn’t purported to, there was a very good likelihood you’d need to restart the entire system.
Mac OS 9 ran on PowerPC processors, which have been additionally used within the GameCube, PS3, and Xbox 360; earlier variations of the OS had began on Motorola’s 68ok CPU sequence. Its successor Mac OS X additionally ran on PowerPC when it first launched; it wasn’t till 10.four that Apple started to change to Intel processors as a substitute, and 10.6 when PowerPC was lastly dropped.
Mac OS X was an enormous step ahead from Mac OS 9 in quite a lot of methods, together with preemptive multitasking in order that you would really run a number of issues without delay. But Apple didn’t wish to simply go away OS 9 packages behind, in order that they did two issues:
The Classic atmosphere arrange a sandbox that appeared sufficient like Mac OS 9 to run Classic Mac OS packages straight in Mac OS X. Because the Classic atmosphere was itself an app, all of the packages that ran inside it have been protected against interfering with different Mac OS X packages and vice versa. It actually was fairly efficient, and really survived longer than booting into Mac OS 9 (which by no means acquired assist for newer PowerPC processors). But its life ended with the change to Intel-powered Macs—Classic was constructed on working the directions within the authentic apps straight, solely having to offer compatibility shims for libraries. Think of it like Wine / CrossOver somewhat than VirtualBox / Parallels.
Carbon was a packaged-up model of the outdated Mac OS Toolbox APIs in order that you would write Mac OS X apps the identical manner you all the time had. You mainly simply recompiled your app and added an additional annotation saying you have been “Carbonized”. (Sound acquainted? ) Fun apart: I’ve all the time assumed because of this Core Foundation exists—to offer a typical interface between Carbon and Cocoa. I don’t even have a quotation for this, although.
Classic ended with the change to Intel processors again within the 2000s, however Carbon labored all the way in which as much as final yr, macOS Mojave. Apple by no means launched a 64-bit model of Carbon, presumably to encourage builders to maneuver to Cocoa, and with final yr’s macOS Catalina, assist for 32-bit apps was dropped
fully with only a few exceptions.
What’s the objective?
Since I discovered to program on Classic Mac OS, and years later spent a very good chunk of my profession working on Swift, I’ve had the tantalizing thought that I’d wish to write a program in Swift and run it on Mac OS 9. That is,
- I write Swift supply code that calls Carbon / Toolbox APIs.
- I compile it for PowerPC with (a model of) the Swift compiler.
- I bundle it up as needed for Mac OS 9.
Is this handy? No! Absolutely not! But neither was ROSE-Eight, and but I nonetheless discovered quite a bit doing it.
As you in all probability guessed, I managed to perform this, or I wouldn’t be scripting this weblog publish. So, with out additional ado, right here’s an image of a Swift Toolbox app working on Mac OS 9.2, on my buddy Nadine’s Power Mac G4. (Check out that blazing quick 400MHz processor!)
I assume a very good variety of individuals studying this wish to know methods to do it too!
If you wish to construct your individual PPC-capable Swift compiler, take a look at the next repositories:
git clone https://belkadan.com/source/ppc-swift-project cd ppc-swift-project git clone -b ppc-swift https://belkadan.com/source/swift git clone -b ppc-swift https://belkadan.com/source/llvm-project git clone https://github.com/apple/swift-cmark cmark make # fast begin to construct swiftc and the stripped-down stdlib
Note the listing and department names of the sub-repos, and be aware that they need to be nested contained in the ppc-swift-project repo. You may even want the
mpwemulator and a replica of the Macintosh Programmer’s Workshop instruments to construct an precise app utilizing trendy macOS.
If you need a prebuilt PPC-capable Swift toolchain, right here’s one: ppc-swift-toolchain. Note that whereas I’ve put a constructed Swift.o on this toolchain, you’ll in all probability solely have success with optimized code that doesn’t even have any remaining hyperlinks to the stdlib (i.e. all the pieces is inlined away). You may even want the
mpwemulator and a replica of the Macintosh Programmer’s Workshop instruments.
You should wish to take a look at the instance within the ppc-swift-project repo. The required flags for
Rezis usually a little finicky. (And be aware that the
carbsources are required for any Carbon app, so you may’t simply skip the Rez half should you really wish to run your app.)
If you simply wish to attempt a constructed model of BitPaint, right here’s one: BitPaint-Swift.hqx.
I’d like to listen to about something you make with these instruments! Meanwhile, should you’d like to listen to how I made this work, learn on.
The final time I used to be constructing Classic Mac OS apps, I used to be utilizing CodeWarrior. Actually, calling that “building Classic Mac OS apps” was a stretch; I used to be studying C and utilizing CodeWarrior’s terminal I/O library to get a stdin/stdout interface that Classic Mac OS didn’t have natively. (Remember, no command line!) I might attempt to get some model of CodeWarrior working once more, however that didn’t look like essentially the most handy factor. I didn’t assume I’d be capable to get the Swift compiler working on Classic, so I’d be shuttling object recordsdata forwards and backwards between OSs to get something completed.
Fortunately for me, I’m not the one one considering constructing Classic apps on trendy macOS. At some level I discovered in regards to the
mpw challenge: an emulator particularly for working Apple’s Macintosh Programmer’s Workshop instruments. And I knew it was going to work, too, as a result of Steve Troughton-Smith, (in)well-known within the Apple neighborhood for locating undocumented and prerelease options in Apple’s OSs, had written up his experiences constructing an app with
mpw that ran on System 1 all the way in which as much as trendy Mac OS X, simply by constructing with the suitable compiler and towards the suitable libraries.
If you’re considering all this, I extremely advocate trying out his weblog publish. Not solely was this the reference I used to get began, however the app you see working within the above image, BitPaint, is Troughton-Smith’s take a look at app, ported to Swift. (I did ask him forward of time if it was okay to make use of his app for a hobbyist challenge.) Longtime Mac developer Gwynne Raskind additionally gave a two-part high-level tour of the Toolbox APIs on Mike Ash’s weblog a number of years in the past (half 1 | half 2); luckily, Carbon takes care of a good quantity for us even on Mac OS 9.
So okay. What does MPW give us?
- A PowerPC compiler
- A PowerPC assembler
- A PowerPC linker
- The Classic Mac OS header recordsdata
- The Classic Mac OS library stubs, for linking towards
- A bunch of object and binary inspection instruments, which we don’t want for the completed product however which I made plenty of use of when making an attempt to debug thriller misbehavior
That’s fairly good; as Troughton-Smith’s weblog publish exhibits, it’s sufficient to construct a complete app that’ll run on Classic. My thought was to take object recordsdata produced by a contemporary compiler and feed them to the PowerPC linker, which implies I’ll moreover want:
- A modified model of the Swift compiler that helps emitting MPW-compatible object recordsdata
- Some stripped-down type of the Swift customary library and runtime (sufficient to learn in and work together with Carbon headers, a minimum of)
- An precise machine working Mac OS 9. I have one, however not the charger for it, and so I did most of my testing utilizing SheepShaver. My buddy Nadine supplied some testing on precise machines as soon as issues have been working.
And, properly, that must be it! So, off we go.
Modern compiler, traditional linker
To make issues extra manageable, I set an intermediate objective: construct an app utilizing Clang, the fashionable C compiler that ships with Xcode. Clang makes use of the identical LLVM infrastructure because the Swift compiler, so I figured I might cope with all the thing format and workflow points in Clang, after which transfer on to the Swift-specific elements.
The very first thing I did was attempt to determine what the file format was for PowerPC object recordsdata. It seems it’s a format known as XCOFF; looking for trendy documentation on this turned up an IBM reference doc. Pretty a lot nobody else makes use of this format, which was not encouraging. The first time I began wanting into this challenge, I used to be anxious I’d need to have my compiler write out meeting code after which ship that by way of the MPW PowerPC assembler…after fixing it as much as account for the variations in how LLVM and MPW print PowerPC meeting.
However, after I checked to see if LLVM supported XCOFF, I used to be in for a stroke of luck. It seems IBM has began including assist for XCOFF to LLVM simply final yr, as a part of including assist for his or her AIX OS…which runs on PowerPC. So I might ask Clang to generate XCOFF recordsdata for AIX, which implies it ought to solely be a brief step to creating it generate XCOFF recordsdata for Classic Mac OS.
At this level I remembered a little bit of trivia. Apple and IBM used to have an in depth partnership, together with Motorola. They’d even made some widespread requirements that have been used throughout platforms and CPUs, although maybe with much less affect than they’d hoped. Was it doable that AIX and Classic Mac OS used the identical calling conventions for his or her procedures, they usually might simply interoperate with none further work?
I received fortunate: the reply is (practically) sure. The AIX register conventions and stack conventions match up with those within the Mac OS Runtime Architectures information. That meant I might feed object recordsdata produced by Clang straight into MPW’s
PPCLink and get a working Classic Mac OS binary out.
I’m fairly certain my mouth fell open after I first noticed this work.
% clang -c take a look at.c -target powerpc-ibm-aix-xcoff -isystem $/Interfaces/CIncludes -integrated-as -fpascal-strings % mpw PPCLink take a look at.o $ -o Test
That ought to work with simply top-of-master-branch LLVM/Clang (and a quite simple take a look at.c). I did find yourself needing to vary LLVM in just a few methods in the long run, but it surely’s actually pretty minimal, a lot because of the IBM of us for doing the exhausting a part of the work for me!
Now do Swift
Being in a position to compile a easy take a look at program was an amazing milestone, however I needed to do a good bit of labor earlier than I might get swiftc to compile a complete BitPaint. Here are a few of the highlights:
Teach Swift in regards to the PPC/AIX goal. This principally concerned including
AIXcircumstances to change statements throughout the Swift compiler, but additionally concerned making a easy description of the Swift calling conventions for Clang (which I cribbed from the 32-bit ARM implementation) after which assuring the PPC/AIX backend that this was an okay calling conference to be utilizing. I received fortunate within the quantity of labor I needed to do right here as a result of Swift already helps 32-bit ARM, little-endian 64-bit PowerPC (when working Linux), and big-endian 64-bit s390x (one other IBM structure); all of the items have been already in place.
Add assist for Pascal strings. The Mac’s first high-level programming language was Pascal, not C! As such, the default format for strings all through the Toolbox APIs was Pascal strings (a size byte adopted by string information) somewhat than C strings (string information adopted by a null byte). With the
-fpascal-stringscommand-line flag, Clang helps static Pascal strings with the syntax
"pHello World". The
pwould get replaced by the size of the string (which have to be not more than 255 bytes) so that you simply didn’t need to depend it your self. I hacked this into Swift as properly, and whereas my implementation in all probability has issues, it was sufficient to get easy issues working.
Turn off reflection assist and practically all runtime metadata. The Swift runtime may be very highly effective, however I didn’t wish to write a lot of a runtime for this challenge, which was primarily about calling a bunch of C capabilities. Beyond that, although, the default format for Swift metadata makes heavy use of relative addressing (primarily to cut back startup time, however be taught extra right here) in addition to symbols pointing inside a world, and the LLVM XCOFF implementation doesn’t (but?) assist both. So to get to a working proof-of-concept, I aggressively commented out elements of IRGen that made use of both characteristic. I’d wish to get a few of the static metadata again in some unspecified time in the future, however reflection’s not one thing I’m ever considering. Probably.
Make a smol stdlib. The full Swift customary library has a lot of issues in it I don’t want, and a few that I wouldn’t even know how to implement. (What’s a String in a world that may’t assume Unicode?) But all of the logic to combine with C code is predicated on having some fundamental varieties in the usual library (like Int16 and UnsafeMutablePointer). What I ended up doing was taking a subset of the usual library sources, after which including further recordsdata and commenting issues out till it labored.
…haha, nope, even that wasn’t ok. My early makes an attempt at this compiled okay, however they managed to crash PPCLink after I tried to jot down a take a look at program, presumably as a result of there’s simply too many symbols in the usual library. So I minimize issues all the way down to a good smaller subset, and that (ultimately) labored. Of course, I used to be working on this similtaneously I used to be modifying the compiler to get to a working proof-of-concept, so I believe I finally went additional than I wanted to. (A bunch of symbols are solely used for runtime metadata functions.) As talked about above, non-optimized builds of non-trivial packages don’t work but, so I don’t know if I’m within the hazard zone or not, however I’d attempt to add just a few extra issues again in.
Rather than modify the precise Swift repo for this, I made a decision to maintain my stripped-down customary library separate, so yow will discover it within the ppc-swift-project repo. This may be a very good reference for somebody in search of a C-compatible, runtime-less subset of Swift, maybe for an embedded or different resource-constrained atmosphere.
Disable soar tables. LLVM optimizes change statements into soar tables when it appears prefer it’ll assist efficiency and/or code measurement, however its default implementations of soar tables additionally weren’t supported within the LLVM XCOFF implementation. I think about the AIX of us will get round to implementing this ultimately, however for now I simply disabled soar tables fully, forcing the compiler to emit switches as a sequence of
ifs as a substitute.
You can take a look at all of the adjustments within the swift and llvm-project repos, should you’re curious. Very few are applicable for upstreaming to their respective tasks, however I’ll attempt to get those that are related upstreamed in some unspecified time in the future.
Per week of mysterious failures
Having made all of the adjustments above, I had an app that labored! In Swift!
…besides, it solely labored a few of the time. I’d change one thing arbitrary and instantly occasions wouldn’t register any extra. It received so dangerous that I added a counter: after any ten occasions, exit the app. Without that, I’d get trapped, unable to even give up with out restarting the (digital) machine. Even in a seemingly working model, my buddy Nadine reported that making an attempt to make use of the Reset command induced the app to crash. What was going on?
I made a decision I needed to unravel one thing unusual I’d seen earlier: even the Clang model of this system didn’t work appropriately after I turned on optimizations. It’s doable that that was a bug in IBM’s newly-added AIX assist, or 32-bit PowerPC assist because it’s not such a typical platform, and even LLVM’s optimizations. It could possibly be that AIX and Classic Mac OS actually weren’t as comparable as I believed they have been, and so my code wasn’t agreeing with the system code on how issues have been purported to work. And it could possibly be that the optimized code was utilizing an instruction that SheepShaver didn’t assist, although that didn’t actually appear to match the signs.
And the signs have been bizarre. Some native variables have been getting corrupted, however others weren’t. So I began testing all the pieces I might consider:
- was the stack aligned correctly?
- was the stack pointer someway not getting restored correctly?
- was the glue code for cross-library calls trashing different information? (see the Mac OS Runtime Architectures information)
- was there one thing inflicting the Code Fragment Manager (dynamic linker) to place the incorrect tackle in for cross-library calls?
Without with the ability to rely on logging, I made the best textual debug output facility I might: modifying the title of a menu. I wrote C capabilities that tracked the present stack pointer to ensure it was getting restored correctly; I made good use of the
DumpPEF instruments that got here with MPW; I discovered how PEF “pidata” (“pattern-initialized data”) labored and tried to step by way of CFM relocations by hand (once more, see the Mac OS Runtime Architectures information). I even began making an attempt to decompile a few of the precise system libraries to see in the event that they have been doing something suspicious, though a bug within the precise Mac OS 9 appeared extremely unlikely. This led all the way in which to studying in regards to the “toolbox ROM”, which isn’t really ROM in any respect: it’s a boot script and a compressed set of system libraries. (It’s known as that as a result of it’s content material that used to be in ROM.) Fortunately SheepShaver already is aware of methods to load it, which meant that I might do the identical decompression after which manually cut up out the person libraries.
Yeah, I received manner off within the weeds. I discovered quite a bit, although!
Finally, I appeared on the decompiled optimized code—the C model, not the Swift model. I noticed that the variable getting corrupted was in general-purpose register 13. That’s purported to be an okay place to place information in Classic Mac OS (and in 32-bit AIX, and in 32-bit Mac OS X), however I made a decision I didn’t belief that, significantly as a result of that register had been used to trace thread-local storage in 64-bit AIX. So I marked r13 as reserved…
…and the issues went away. Optimized, non-optimized, even with
-fstack-protector-all on. And Swift.
(Debugging this took a couple of week, sadly, which led to this challenge being rather less bold than I initially wished.)
What didn’t I get to? An terrible lot, really.
- There’s no runtime in any respect, which implies no dynamic allocation (amongst different issues).
- There’s no kind metadata, which implies no generics (that aren’t optimized away).
- There’s no area metadata, which implies no key paths (that aren’t optimized away).
- There’s no Unicode assist, so no Strings. Arguably I might make a String with out Characters, or a String utilizing MacRoman because the native encoding, but it surely wouldn’t essentially look very similar to right now’s Swift.String.
- There are a bunch of different customary library issues lacking as a result of I wished to get the proof-of-concept working, but additionally as a result of
PPCLinkwas choking on giant object recordsdata. If I do get extra customary library stuff working, I’ll in all probability cut up it out of the ‘Swift’ module someway.
- I needed to mess with linkage in quite a lot of methods to make the LLVM XCOFF backend comfortable, so I’m unsure multi-file builds would work. I didn’t even take a look at it.
- I’m utilizing Carbon, which signifies that my program ought to work on older variations of Mac OS X as properly, however my buddy Nadine tried and it didn’t, and it wasn’t a precedence to determine.
- I wished to attempt making good abstractions on high of the a few of the Toolbox APIs.
- I wished to make extra sophisticated instance apps!
Maybe I’ll observe up on a few of these, however I’ve been placing plenty of effort into ensuring I might end this by April 1, so I ought to in all probability get to a few of the issues I’ve been neglecting in favor of this challenge as a substitute.
This challenge took plenty of time, though I (1) know quite a bit about compilers and (2) hacked my technique to success as a substitute of being cautious and sustaining correct software program improvement practices. But I discovered quite a bit, and I achieved a objective I’ve had behind my thoughts for a very long time.
If you made all of it the way in which to the tip of the article, right here’s a reward: BitPaint working below Classic on Mac OS X 10.2 (additionally courtesy of Nadine).
Stay secure, everybody, and assist the individuals round you when you may. And if anyone makes one thing with this challenge, I wish to hear about it!