University of Michigan

University of Nebraska-Lincoln

Sr. Staff Software Engineer at Twitter

As Group Tech Lead of Infrastructure Automation, I was responsible for the technical strategy to rearchitect Twitter’s foundational software layers, in service to adopting infrastructure-as-code frameworks as well as accelerating the company’s migration to Kubernetes in both its cloud and on-premise environments.  This included multiple initiatives from standardizing and automating our Kubernetes cluster lifecycle management to modernizing our bare metal data center and POP technology stacks.

Twitter

Principal Software Engineer

I joined the Periscope team following its acquisition by Twitter to help scale its AWS-based backend for global launch. As Platform Lead I was ultimately accountable for most aspects of platform design from site reliability, network engineering, security, CI/CD and overall backend developer user experience. This stack now powers many Twitter consumer-facing products in AWS including Fleets and Twitter Spaces.

Periscope’s platform is based on an infrastructure-as-code toolchain I developed to safely manage constellations of declarative infrastructure allowing a small team to deploy and operate thousands of infrastructure components spread across dozens of VPCs in more than 15 regions globally.  I partnered with engineering teams to design and implement their backends using cloud-native best practices, becoming an expert at developing against and integrating myriad AWS products and technologies like EC2, IAM, DynamoDB, Cloudformation, Kinesis, SQS, SNS, ECS, KMS, Cloudwatch, Cloudfront, Route53, and so forth, while also securely interoperating with third-party SaaS and CDN providers.

Senior Staff Software Engineer

I was a founding member of the Compute team at Twitter, where we built Mesos and Aurora to try to bring a Borg-like platform experience to Twitter engineers.  I authored and open-sourced a large swath of the Aurora ecosystem: the client, the process execution framework (Thermos), the customer configuration DSL, and many of the platform integrations required to make Twitter work as a whole, from observability to service discovery.  Later, as tech lead, I shepherded many efficiency and resiliency projects like SLA-aware scheduling and improved cluster bin-packing which resulted in meaningful impacts to Twitter’s operating statements and path to profitability.

Along the way, I developed a number of supporting technologies like the PEX packaging toolchain for Python (1.8k stars on Github), wrote most of the Python common libraries at Twitter -- now leveraged internally by thousands of tools -- and was a major contributor to the Pants build system.

As a member of the Monorail infrastructure team I helped with a number of early projects to “kill the fail whale.”

I redesigned Twitter’s feature switch system, Decider, making it more durable and auditable by storing switch updates as a changelog in git, and implemented the Zookeeper-based fan-out mechanism to distribute updates to Twitter’s server fleet.  Later, I helped the team to encapsulate major components like the User store and Tweet store so that they could be decomposed into Scala microservices.  Lastly, I helped to add multiple data center support to the monorail which included extending the feature switch framework to be DC-aware as well as adding general purpose mutation replication mechanisms for global cache coherency.

Software Engineer

As a member of the Borg core scheduler team, I worked on the roll-out of CPU oversubscription, decoupling the core scheduling algorithm from the Borgmaster, and early design discussions for the next generation scheduling system, Omega, whose design later inspired Kubernetes.

Google

Senior Software Engineer

As a member of the team that designed and built the storage infrastructure and processing pipelines for Google Book Search, I helped push the scale of what was possible in storage and network systems at the time.  I developed an LTO-based sneakernet to manage the petabytes of data transfer required to replicate book scan data between remote scanning centers and data centers, doing everything from protocol design to programming tape robots.  That work was later repurposed to inform Google’s offline storage and disaster recovery strategies.

I also developed and operated the processing frameworks that allowed various image processing and OCR teams to deploy processing algorithms at scale, not just for Google Book Search but for nascent projects like Google Street View and for oblique imagery within Google Maps

Brian Wickman