Scaling

GitHub

Scaling

GitHub

Scaling GitHub

Scaling

GitHub

Scaling GitHub

Scaling

GitHub

Scaling

GitHub

Scaling

GitHub

Scaling github

SCALING

GITHUB

scalin’

githubs

Scaling GitHub

githubs and shit

Scaling

GitHub

Scaling GitHub

Scaling

Startups

B=======D~~~~

Scaling

GitHub

“Scale”

Two problems.

Scaling is

people

technology

@holman

Organizational

jeez humans are so finicky

Happiness.

$$$

happy

productive

employees

are

happy

employees

This isn’t a “management problem”.

Everyone needs to worry about this.

Hiring an employee is the

most thing

you can do to your startup.

T O X I C

Hiring an employee is the

so how can you

TOXIC EXCITEMENT

would be a great name for a rock band

yeah, i know...

Your servers, offices, and ideas are bullshit.

Keep your employees happy.

Worry about your coworkers.

Really happy.

EMPLOYEES

NEW HIRES

Know your codebase

Know your process

Imprison your employees with happiness and

nice things and cuddly work practices.

GitHub Jail

work whenever you want

work however you want

work on what you want

health, dental, vision

get out of the way

NO MEETINGS

NO PLANNING SESSIONS

NO NEED TO BE IN THE OFFICE

chat, pull requests, email

MORE DIRECT

This is designed to retain people.

We’re at 56 employees. We haven’t lost one.

This is a huge, massive competitive advantage.

It justifies the extra expense.

Communication.

Don’t have

the server guy

who knows everything.

the billing girl

the testing dude

the customer support maven

Don’t have

the person

who knows everything.

Specialization is great,

but only having one person

is a

Reduce institutional knowledge.

wikis

Every internal GitHub talk

is automatically recorded,

uploaded, and viewable to

every future employee.

...on a Kinect-powered

Arduino-based motion-

detecting portable video

recording platform.

Your new hire is stoked to dive in,

start reading, and start contributing

...so don’t get in their way.

Hire well.

Hiring poorly is just as bad

as losing people.

Aim for really great people.

WE SELF-STARTERS

less babysitting, more code

Don’t just market your product;

Always think

about attracting

good people,

even if you’re

not hiring.

Technical

robots can be pretty finicky too

Automate.

deploys

current process overview

multi-server shell commands

new employee setup

app bootstrap

Automating now will save you way

more time down the road.

Ship.

Ship early, ship often.

5x-30x

deploys per day

master

always deployable

always green tests

always a safe rollback

Limit your deployments

staff

-only

beta users

only

@github tweets

exceptions

deploys

Graph.

everyone loves fancy graphs

quickly see trends

quickly see problems

historical data as basis for alerts

METRICS ARE GREAT

But use them wisely.

162ms

average

overall

response time

Valueless metric.

59ms

average

API

response time

with

23ms

average

raw

response time

with

2x throughput

of web

The responsiveness is a lie.

199ms

average

browser

response time

16,000

requests in the last week over

4.5s

Needed to look at the

right stuff.

throttled google

googlebot

2-3x throughput

3-4x CPU usage

had

web requests

compared to

Collect a lot of metrics,

but make sure they’re

important metrics.

GitHub scale.

Everyone has different

growth patterns.

GitHub has had three.

Launch

2008

Bare metal servers

2009

net-shard

Launch

2008

Hosted on Engine Yard

10 VMs

54GB RAM

shared GFS mount

Bare metal servers

2009

Hosted on Rackspace

16 bare metal servers

288GB of RAM

redundant disk storage

net-shard

2010

networks share a common repository

rails/rails

net-shard

For GitHub, scaling involved a lot of

predictions of future trends, then

acting appropriately.

Side Projects.

A THOUGHT EXPERIMENT:

Imagine I told you to build...

an asynchronous robot written in

coffeescript that idles in our chatroom

that can deploy the site to dozens of

servers and vps instances, record

those deploys, manage branch-level

This grew organically, over dozens of

Figure out how to let this happen. It’s hard.

Small hack days can result in

real,

imma-make-us-money

impact.

Small hack days can also keep your

developers

insanely happy

Small hack days can also lead to

learning

new techniques

Projects and Posts.

JENKINS + CAMPFIRE

github.com/github/janky

CHAT ROOM ROBOT

github.com/github/hubot

OFFICE MUSIC DJ

github.com/holman/play

BLOG: GITHUB IS MOVING TO RACKSPACE

git.io/jByrlQ

BLOG: HOW WE MADE GITHUB FAST

git.io/p5v2Ag

BLOG: UNICORN

git.io/77Onfg

Technical

Organizational

Continually refine your

process + workflow.

Worry about your

computers

, and worry

about your

humans

Thanks.

ZACH HOLMAN

zachholman.com/talks

@holman

twitter+github: