-
This weekend at Hackernews I stumbled across a great video showing how Facebook pushes new code live. It's the most advanced push deployment system I have seen. The video is excellent. If you are into software, looking for advanced build and deploy techniques, or just looking for ammunition to improve your own build and deploy process, watch this video.
Thank you Lucian very interesting, possibly one of the most professional important video I watched recently... basically it is just all the opposite to what Wordpress does! I would recommend it to all engineers and system administrators that have to de with release and risk management. I would say a quite advanced audience.
Key concepts of the video:
- Ludicrous and massive, two words that shouldn't be together.
- Lowering the risk of change through TOOLS and CULTURE
- All pushes are authorized by developers, no "thrown the code over the wall" culture
- From trunk on Sunday at 6pm they cut out the release branch which will go live on Tuesday
- Release small fix everyday but big pushes only on Tuesday and Wednesday
- The longest a change will ever way before going out is 7 days
- Facebook has 15 different frontends teams
- communication is King, if there is a big change it must be communicated to other teams releasing on time. Avoid surprises
- if you are a dev inside facebook you DO test your own changes, whether you like it or not
- they use their own product as company communication infrastructure
- communication in IRC, you can ask if your code is going live and the bot call you when it happens so you are present to support the release
- very interesting at minute 21 the show and tell for the test suite
- every dev start with 4 starts and at any bug found they go down so to keep track of the risky pushers
- tools alone won't save you. You need the right people, culture and company
LESSONS
Obviously in the world other website like Facebook there are none, in terms of engineers(500), visibility and audience. I think the lesson to take from this video is that you must create the right automated test and deploy suite that will enhance the confidence of developers to ship their changes live without fear and regret.