Swift compiler driver reimplementation in Swift
Swift's compiler driver is a program that coordinates the compilation of Swift source code into various compiled results: executables, libraries, object files, Swift modules and interfaces, etc. It is the program one invokes from the command line to build Swift code (i.e.,
swiftc) and is often invoked on the developer's behalf by a build system such as the Swift Package Manager (SwiftPM) or Xcode's build system.
swift-driver project is a new implementation of the Swift compiler driver that is intended to replace the existing driver with a more extensible, maintainable, and robust code base. The specific goals of this project include:
Note: Currently, swift-driver is only compatible with trunk development snapshots from swift.org.
The preferred way to build
swift-driver is to use the Swift package manager:
$ swift build
swift-driver in place of the existing Swift driver, create a symbolic link from
ln -s /path/to/built/swift-driver $SOME_PATH/swift ln -s /path/to/built/swift-driver $SOME_PATH/swiftc
Swift packages can be built with the new Swift driver by overriding
SWIFT_EXEC to refer to the
swiftc symbolic link created above and
SWIFT_DRIVER_SWIFT_FRONTEND_EXEC to refer to the original
SWIFT_EXEC=$SOME_PATH/swiftc SWIFT_DRIVER_SWIFT_FRONTEND_EXEC=$TOOLCHAIN_PATH/bin/swift-frontend swift build
Similarly, one can use the new Swift driver within Xcode by adding a custom build setting (usually at the project level) named
SWIFT_EXEC that refers to
$SOME_PATH/swiftc and adding
-driver-use-frontend-path $TOOLCHAIN_DIR/usr/bin/swiftc to
Other Swift Flags.
swift-driver can also be built with CMake, which is suggested for
environments where the Swift Package Manager is not yet
available. Doing so requires several dependencies to be built first,
all with CMake:
-DCMAKE_OSX_ARCHITECTURES=x86_64(If building on Intel) when building
cmake -B <llbuild-build-dir> -G Ninja <llbuild-source-dir> -DLLBUILD_SUPPORT_BINDINGS="Swift" -DCMAKE_OSX_ARCHITECTURES=x86_64
Once those dependencies have built, build
cmake -B <swift-driver-build-dir> -G Ninja <swift-driver-source-dir> -DTSC_DIR=<swift-tools-support-core-build-dir>/cmake/modules -DLLBuild_DIR=<llbuild-build-dir>/cmake/modules -DYams_DIR=<yamls-build-dir>/cmake/modules -DArgumentParser_DIR=<swift-argument-parser-build-dir> cmake --build <swift-driver-build-dir>
The new Swift driver is a work in progress, and there are numerous places for anyone with an interest to contribute! This section covers testing, miscellaneous development tips and tricks, and a rough development plan showing what work still needs to be done.
For a conceptual overview of the driver, see The Swift Driver, Compilation Model, and Command-Line Experience. To learn more about the internals, see Driver Design & Internals and Parseable Driver Output.
Test using command-line SwiftPM or Xcode.
$ swift test --parallel
Integration tests are costly to run and are disabled by default. Enable them
SWIFT_DRIVER_ENABLE_INTEGRATION_TESTS environment variable. In Xcode,
you can set this variable in the scheme's test action.
$ SWIFT_DRIVER_ENABLE_INTEGRATION_TESTS=1 swift test --parallel
Some integration tests run the lit test suites in a Swift working copy.
To enable these, clone Swift and its dependencies and build them with
build-script, then set both
SWIFT_DRIVER_LIT_DIR, either in your Xcode scheme or
on the command line:
$ SWIFT_DRIVER_ENABLE_INTEGRATION_TESTS=1 \ SWIFT_DRIVER_LIT_DIR=/path/to/build/Ninja-ReleaseAssert/swift-.../test-... \ swift test -c release --parallel
swift-driver Continuous Integration runs against the most recent Trunk Development snapshot published at swift.org/download.
When developing patches that have complex interactions with the underlying
swift compiler frontend, it may be prudent to ensure that
swift-driver tests also pass against the current tip-of-trunk
swift. To do so, create an empty pull request against github.com/apple/swift and perform cross-repository testing against your
swift-driver pull request #, for example:
Using: apple/swift-driver#208 @swift-ci smoke test
@swift-ci cross-repository testing facilities are described here.
After the toolchain is installed, Xcode needs to be told to use it. This can mean two things, building the driver with the toolchain and telling the driver to use the toolchain when running.
Building with the toolchain is easy, set the toolchain in Xcode: Menu Bar > Xcode > Toolchains > select your toolchain
Running the driver requires setting the TOOLCHAINS environment variable. This tells xcrun which toolchain to use (on darwin xcrun is used to find tools). This variable is the name of the toolchain and not the path (ex:
Swift Development Snapshot). Important note: xcrun lookup is lower priority than the SWIFT_EXEC_*_EXEC family of environment variables, the tools directory, and any tools in the same directory as the driver (This includes a driver installed in a toolchain). Even though TOOLCHAINS is not highest priority it's a convenient way to run the xctest suite using a custom toolchain.
When developing on macOS without quick access to a Linux machine, using a Linux Docker is often helpful when debugging.
To get a docker up and running to the following:
docker pull swift.
$ docker run -v /path/to/swift-driver:/home/swift-driver \ --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \ --security-opt apparmor=unconfined -it swift:latest bash
$ apt-get update $ apt-get install libsqlite3-dev $ apt-get install libncurses-dev
swift test --parallelto run your tests.
Options.swift, which contains the complete set of options that can be parsed by the driver, is automatically generated from the option tables in the Swift compiler. If you need to regenerate
Options.swift, you will need to build the Swift compiler and then build
makeOptions program with a
-I that allows the generated
be found, e.g.:
$ swift build -Xcc -I/path/to/build/Ninja-ReleaseAssert/swift-.../include --product makeOptions
makeOptions and redirect the output to overwrite
$ .build/path/to/makeOptions > Sources/SwiftOptions/Options.swift
The goal of the new Swift driver is to provide a drop-in replacement for the existing driver, which means that there is a fixed initial feature set to implement before the existing Swift driver can be deprecated and removed. The development plan below covers that feature set, as well as describing a number of tasks that can improve the Swift driver---from code cleanups, to improving testing, implementing missing features, and integrating with existing systems.
TODO:: there are lots of little things to improve!
Errorthrown within the library
Options.swift. Is every option there checked somewhere in the driver?
OptionAliasso we can't make the mistake of (e.g.) asking for an alias option when we're translating options?
makeOptions.cppto translate the command-line options from Swift's repository into
DarwinToolchainto also handle iOS, tvOS, watchOS
GenericUnixToolchaintoolchain to get it working
OutputFileMapimplementation to handle all file types uniformly
swift-driver. Were the results identical? What changed?
swift-driverso it can run the Swift repository's driver test suite.