Technical Leadership

A good culture, high velocity/productivity, ownership/accountability rather than defensiveness. A good culture: helpful, useful, customer centered even if the customer is feature devs while you are platform, contagion of enthusiasm and curiosity, projection of good intentions, no egos. But there are technical bits too. With an eye to the above, this page will focus on the technical bits of helping the team thrive.


Git

Importance

Git is a coordination tool (and, I would even argue, a communication tool). It is not merely a version-control tool. Developers often do not really understand git and sometimes use it incorrectly. Like Agile, however, git is an essential part of running a development team.

Good teamwork requires solving coordination problems and communication problems and git, along with good industry conventions, helps with that.

Popularity

I personally no longer see teams that are still using subversion (last release 2020, but still fully supported) or Mercurial (e.g. Atlassian is discontuing its support with Bitbucket) or Microsoft’s Visual SourceSafe (last release 2005). According to the 2022 Stackoverflow’s Developer Survey, 1% of professional developers use Mercurial, 6% still use subversion, and 97% use git, while the 2023 survey didn’t even bother asking this question.

Some companies with giant codebases, however, roll their own and do in fact have non-git roots. Meta’s, for example, is an evolved/customized version of Mercurial (Meta’s “news for developers” blog from 2022).

Git is popular, so you need to know it. But it does require significant technical expertise and has some common gotchas that, I think, can be addressed with proper training.

training topics

  • Why is my git repo already a gig in size?

    • The problem:

      • Takes forever to clone during onboarding

      • Slows down the CI server

    • Solutions:

      • How to prevent it

      • How to treat it

      • How to deal with it

  • Rebase vs. merge vs. chery pick

  • Force-pushing, private vs. shared branches

  • Sharing git patches

  • Using git to version your releases, not just commits

  • Speeding up the finding of when a regression was introduced using the “git bisect” command. Can this be automated?

  • UI git clients vs. command line. (Command line is important to be able to automate via scripting, but I actually encourage folks to use a good UI git client because it is easier to avoid committing something by mistake you did not intend to commit; a good UI git client is really an IDE for git; operating through a keyhole can help you focus but it can also blindside you.)

  • Understand the “squash and merge” button on github. It’s a great button, but it is not equivalent to an interactive rebase to squash commits. (Instead, it appears to be behaviorally equivalent to a git diff > patch-file && git checkout main && git apply patch-file && git commit.)


Branching models

Some options, considerations, and distinctions:

  • GitHub Flow

  • Gitflow

  • A cascading model from dev to test to stage to prod (branches, not environments?)

  • Trunk-based development vs. a number of infinitely living branches

Default to GitHub Flow if you don’t need anything more complicated. It, or some version of it, tends to work well for an iOS codebase (even if it doesn’t work for the backend codebase that supports it). With iOS, you have multiple versions of the app in the wild which your backend needs to support (since they might use different versions of the backend APIs, e.g. GET /trips/v1 vs. GET/trips/v2).

You will rarely need Gitflow, but looking at it is good because (a) you will have a developer who will mention it and you won’t want to say “I don’t know what that is” (although you could) and (b) Gitflow does offer some insights.


Versioning

Some conventions you need to know:

  1. Semver

  2. Calver

  3. GET /trips vs. GET /trips/v2

  4. Although Swift Package Manager (SPM) will support both formats for semver, “v1.2.3” is oudated, whereas “1.2.3” is the newer way.

Git tag:

  • Supports semver and calver.

  • Git will not garbage-collect the commit if you tag it and delete the branch. (The release branch, for example, could be deleted; but you might want to backmerge it if it contains release-candidate fixes.)

  • Lightweight vs. annotated tags.

Dependency management:

  • Swift Package Manager (SPM), CocoaPods, and Carthage all understand semver.

  • Your dependencies (third party or not) have (or should have) versions too and, moreover, have their own dependencies with their own versions.

  • An awkward situation: Your app is using Alamofire 5 (e.g. 5.8.1) while your app’s third-party dependency (e.g. your image-fetching library) is still on Alamofire 4 or, worst still, is still using AFNetworking.

  • Lock versions or allow to update up to next minor/major?


Quality

Pre-production environments

  • your company’s servers (e.g. test, development, staging, prod)

  • Apple’s environments: production and development servers, production-signed vs. development-signed app. (In-app purchases and push notifications.)

  • your A/B testing service provider

Make it visible to the team/developers/yourself:

  • Enable the code coverage ribbon in Xcode

  • A code-coverage badge on the repo

  • Sonar Cloud

Make it stable:

  • flakey tests vs. failing tests

Coverage:

  • Quantity, but what about quality?

  • Integration tests vs. unit tests.

  • UI tests <- e.g. from TestRail

Pull requests:

  • PR reviews

  • Other PR gates


Release

Topics: empirically minded, pain points, managing risk

Creating a release candidate (RC):

  • Release cadence vs. continuous delivery

  • Are release branches necessary?

  • A/B test features, feature gate, avoid long-running branches

  • A release train in App Store Connect

  • Submission for Apple Review

Testing the RC:

  • Apple’s Sandbox: testing In-App-Purchases (IAP) and push notifications (APNs)

  • Full regression testing every time :o

  • Regressions in the RC or in the wild: (a) create a test case (e.g. in TestRail) for manual testing, (b) create an appropriate automated test.

Releasing the RC:

  • Do a phased release

  • Monitor it (e.g. for spiking crash)

  • Avoid having to release a binary if something goes wrong—think about this ahead of time; what’s your damage-control strategy?

  • If you have to release a new binary: how to hotfix vs. pausing current phased release and waiting for the next release

Releasing a feature:

  • Feature gate.

  • Don’t forget to eventually conclude your A/B test! Calender it? You defined meaningful metrics and your results are statistically significant, right?


There are three popular dependency managers for iOS development:

  • Swift Package Manager (SPM)

  • CocoaPods

  • Carthage

SPM:

  • Statically linked

  • Including binaries. (Dynamically linked must be embedded; what the “duplicate symbols” warning means.)

  • Git LFS limitations

  • Long checkout times

  • DerivedData folder

Tradeoffs and optimizations:

  • Statically linked -> faster launch times, slower build times?

  • What does the IPA actually contain? (e.g. all your dylibs too; system’s dylibs are shared among processes/apps, but yours aren’t.)

  • Understanding “embed” vs. “not embed”, “embed and sign” vs. “embed and not sign”.

  • Multiple targets sharing the same dependency

  • Dependency of dependency

  • Why use one umbrella package for all third-party dependencies?

Dependency graph

  • Horizontal vs. vertical

Dependency Management


  1. iOS specific things:

    • To speed up launch times, default to using static libs for your frameworks. That is, statically link them (which will automatically “embed” them) rather than dynamically.

    • To minimize IPA size, make sure Dead Code Stripping is enabled in Xcode. (There is also Order Files to , but either have to do it by hand or cough up the dough to subscribe to EmergeTools.)

    • Here is a great article on this from Dec 2022 by Elkins and Topolsky.

  2. Strike a balance:

    • A simple app might not need any modules. One big module might be just fine.

    • Too many modules might be over-engineering it. (But some folks do love a micro-module architecture with SPM.)

  3. Benefits of modularization:

    • Divide a problem into subproblems, so it is easier to solve.

    • Modules could reflect a meaningful division of labor. A team per module.

    • Build times.

  4. Downsides of modularization:

    • Build times: if you mess up the dependency graph, you are back to rebuilding everything whenever anything changes.

  5. What is a module anyway?

    • Library vs. framework.

    • In Apple’s ecosystem:

      A framework is a hierarchical directory that encapsulates shared resources, such as a dynamic shared library, nib files, image files, localized strings, header files, and reference documentation in a single package. … A framework is also a bundle … [However, a] framework bundle is a standard directory that the user can navigate. This makes it easier for developers to browse the framework contents and view any included documentation and header files. (Source)

    • But for Google, a framework is a shared library that is exensible and which has inverted control flow:

      For Google, two technical principles help to distinguish a framework from a library: inversion of control and extensibility. While seemingly modest, the many benefits of frameworks discussed in this article are principally derived from these principles. (Source)

Modularization


Architecture

Mobile Software Architecture

  • Mobile applications are typically backed by a cloud server, which from the point of view of a mobile dev means that the topology is “hub and spokes” (aka a “star” topology), even if there is some distributed system on the backend behind the scenes (e.g. DynamoDB is a distributed database, there are CDNs such as CloudFront, etc). Peer to peer between mobile devices is rare if not non-existent.

  • Sometimes, on mobile, by “architecture” we just mean design patterns and, indeed, the line can be blurry on iOS. Other times, it refers to modularization—how you decided to carve up the app, perhaps to reflect what team is working on what or perhaps because you were trying to improve build times (e.g. isolating pieces that do not change often and putting them at the bottom of the dependency graph).

  • Apple does not actually have architectural recommendations. Their examples are MVVM (with a view model for SwiftUI foro example) or, in the UIKit world, MVC or similar. As you scale your app, you could add a view model to MVC, along with other components (e.g. services, a persistence layer, ways to glue services and data stores together).

  • There are other options. For example: VIPER (historically) or TCA (newer). TCA is Unidirectional Flow (UDF), so essentially an implementation of the redux pattern for iOS. There are indeed reducers, for example.

  • Whatever architectural choices you make, keep in mind an offline-first architecture and related considerations. See our training page.

  • Do not forget that the user’s experience does not distinguish between backend and frontend. The app—the product—is not the IPA. It is the entire system. Historically, we have oscillated some between a thin client and a thick server and back. At one polar extreme, a dumb telnet terminal, for example, is more or less just a window into the server; where to do more work—on the server or closer to the end user—is not a business decision just as much as it is an engineering decision. The mobile client is just another module in the system. Even if it is a module made up of modules, it is still a module. Same with the server. This is so even if company growth drives us towards specialization and away from employing full-stack devs. Full-stack architecture is the holistic view.


Working with

working With Design

  • Everyone wants “pixel perfect”, but (a) screen sizes do differ and (b) SwiftUI offers fewer placement guarantees than UIKit.

  • Show Design what your implementation looks like on small screens. Take a small device and enable Display Zoom. (Not to confuse this feature with the zoom-in accessibility feature.)

  • Design should be driven by the PRD—below.

working with Product

  • Expect/ask for a Product Requirements Document (PRD) for the feature they envision

  • Do an architecture/system-design session

  • Eyeball effort (and build in a buffer)

  • Create an epic or some similar way to track the feature

  • Focus a team to deliver. Have them envision the final product. Help them minimize context-switching. Shield them from politics.

  • Iteratively, throughout, show progress to Product to make sure it’s what they want. (This is just good Agile, Scrum or Kanban; we don’t waterfall.)

working with data science

  • Ask Data Science what questions they ask about mobile (e.g. is it about end user behavior or something else?) that cannot be answered through a tool like Crashlytics.

  • Create pipelines from the app into Data Science tooling.

  • Keep in mind consistency of data, so tomorrow can be meaningfully compared with yesterday.

  • Familiarize yourself with existing data pipelines and existing tooling.

  • Some common tooling: AWS S3, Snowflake, Elastic Search, Kibana, maybe Jupyter Notebooks.


Managing down, up and across

This is not a technical topic, but important to mention. A few distinctions.

Managing across != managing down

Someone can be a beloved manager but unable to manage across—e.g. collaborate with peers to get alignment across the org—and so might be let go by upper management much to the puzzlement and dismay of developers on team the person manages.

Managing up != managing down

Similarly, someone might be great at managing up (e.g. communicate clearly upwards, set clear expectations, and deliver on those expectations on time and under budget), but be disliked by the devs on the team.

Management != leadership

In business, when we say “Leadership”, we typically mean upper management. For example, we might say “Leadership said x, y, z” just as we might say “Product gave us a PRD” or “Design gave us Figmas.” Inadvertently or quite intentionally, in industry we distinguish leadership from management, but do so in a way that does not serve us well for it merely refers to the roles people are expected to play, but not to what it is to provide good leadership or good management. In industry, typically for instance, ‘manager’ refers to someone in middle management. Note of course that a director is a manager of managers and yet no one refers to a director as a ‘manager’.

You can be a good leader but a bad manager and, I think, vice versa. If a good manager is, necessarily, a good leader, then I am open to that definition, but, if that’s how we conceptualize it, then we need to make that explicit, as we should. Conflating management with leadership, as concepts, is a mistake. It is coherent to imagine upper management, aka ‘Leadership’, to possess no leadership qualities whatsoever, just as it is coherent to imagine an engineering manager who is a great leader. Moreover, tech leads are often Individual Contributors (ICs) and so not on the management track at all.

Moreover, by not being able to work with developers directly (because it is not feasible to have a 100 direct reports for example and so the org chart must, at some point in the firm’s growth, have more than one layer), those in upper management positions suffer from an inherent informational handicap. They have to rely and trust those below them.

Some related articles on this topic of management is or is not leadership:

If you want to get philosophical here… On reflection, it does seem odd if (good) management requires (good) leadership, but leadership does not require management. After all, if leadership is truly above management, then why should it require less? But if mismanagement is a sign of poor leadership, then the distinction between management and leadership collapses. Perhaps management refers to some organizational qualities such as managing a budget (e.g. contrast with mismanagement) and more mundane everyday tasks while leadership refers to steering, inspiring, looking ahead, and leading a team (an org as a team of teams) competently in the moment (e.g. think “leading by example”).

What do we want

Ideally, we want someone who can do all three well: manage down, manage up, and manage across. At a tech firm, the sum of all three, combined with some relevant leadership qualities (which I have not defined here), makes a true leader. This

This also has the odd implication that someone who is at the top of the org chart cannot be a true leader and perhaps there either (a) end users (or other relevant customers) are the “up”—the bosses of the tech firm’s head—or (b) it is all in the person’s potential or (c) the requirement to manage up goes away when you are the top since it is not the case that you are managing up poorly (and so, vacuously true that you are managing up).

What are “relevant leadership qualities”? For a technical leader these likely include the ability to pick a direction and pick it well enough and, moreover, would in fact include special technical knowledge, assuming that knowledge is relevant to the challenges and tasks ahead. (Don’t know about “noble rhetoric”.)