Over the past few years, we've been working to upgrade our data centers to run at 100 gigabits per second. To do so, we needed to deploy 100G optical connections to connect the switch fabric at higher data rates and allow for future upgradability — all while keeping power consumption low and increasing efficiency. We created a 100G single-mode optical transceiver solution, which we've shared...
View details ⇨

Designing 100G optical connections

code.facebook.com
Bryce Canyon, our next-generation high-density storage server, is designed to support more powerful processors and more memory, and improves thermal and power efficiency by taking in air underneath the chassis. Our goal was to build a platform that would not only meet our storage needs today, but also scale to accommodate new modules for future growth.

Introducing Bryce Canyon: Our next-generation storage platform

code.facebook.com
Dive deeper into Big Basin, our new GPU server that can train machine learning models that are 30 percent larger.

Introducing Big Basin: Our next-generation AI hardware

code.facebook.com
Today at the Open Compute Project Summit we announced an end-to-end refresh of our server fleet: Big Basin, our new GPU server; Bryce Canyon, our high-density storage server; and updated compute platforms in Tioga Pass and Yosemite v2. Read more about the hardware specs and performance improvements here.

The end-to-end refresh of our server hardware fleet

code.facebook.com
At today's Video @Scale event in Menlo Park, Facebook announced updates to its dynamic streaming technology to deliver high-quality 360 videos more efficiently.

End-to-end optimizations for dynamic streaming

code.facebook.com
Over the years, as our community and data needs have grown, Facebook's data infrastructure team has explored new solutions for processing data at scale. They decided to use Apache Spark for its ability to scale with a large amount of data and support for custom applications. In the most recent use case, the team leveraged the imperative side of Spark to redesign a complex pipeline for...
View details ⇨

Using Apache Spark for large-scale language model training

code.facebook.com
Registration is open for F8 2017! Join us on April 18 and 19 in San Jose, CA for our annual developer conference. Space is limited; apply today!

Register now for F8 2017!

code.facebook.com
Large-scale monitoring systems generally cannot handle large-scale analysis in real time because the query performance is too slow. To address this, we developed and recently open-sourced Beringei, a high-performance in-memory time series database. Beringei currently stores up to 10 billion unique time series and serves 18 million queries per minute, powering most of the performance and health...
View details ⇨

Beringei: A high-performance time series storage engine

code.facebook.com
Online search has traditionally been a text-driven technology, even for photos and videos. Today, we announced a search system, available in the US, that leverages image understanding to surface the most relevant photos quickly and easily. Using cutting-edge deep learning techniques to process billions of photos and understand their semantic meaning, people can find photos from their friends...
View details ⇨

Building scalable systems to understand content

code.facebook.com
Every day, more than a billion people use Facebook on mobile devices. Securing data in transit between our mobile apps and our servers helps ensure that people have a safe experience on Facebook. Over the past year, we've built and deployed Zero protocol, an experimental 0-RTT protocol over TCP based on QUIC's crypto protocol. We've seen performance improvements such as a 41 percent reduction...
View details ⇨

Building Zero protocol for fast, secure mobile connections

code.facebook.com
Facebook has been transitioning all of its data centers to IPv6-only infra, but the vast majority of people on Facebook only have access to IPv4 internet. Read more about the fix that allowed us to evolve our data center technology while still supporting our entire global community.

Legacy support on IPv6-only infra

code.facebook.com
This year's hackathons spawned everything from new hardware, to developer tools, to new ways to connect with friends and family. Check out a few of our favorites from 2016!

Favorite hacks of 2016

code.facebook.com
Journey inside our data centers to learn how Facebook built some of the most innovative and efficient infrastructure in the industry.
A huge thank you to our community of engineers and developers that continue to make Facebook's open source program a success! We look forward to building more together in 2017!

Facebook Open Source 2016 year in review

code.facebook.com
Our recently launched Recommendations feature, which turns text comments into a dynamically updating map attachment, was the result of collaboration among several teams here at Facebook. Machine learning helps recognize when a post is seeking recommendations and extracts the relevant terms, which are then fed into our local search engine to identify the places with the best match. Our Android...
View details ⇨

Made in NY: The engineering behind social recommendations

code.facebook.com
Starting today, css-layout (our open source cross-platform implementation of Flexbox) will be available as Yoga, a standalone tool that allows engineers to build layouts quickly for multiple platforms. We implemented Yoga in C for better performance, and included bindings for several languages. You can check it out on GitHub now.

Yoga: A cross-platform layout engine

code.facebook.com
When we built the Reactions feature on Facebook, we wanted it to be a high-quality, lightweight experience. This led us to develop Keyframes, a library that exports After Effects animations using the least amount of data necessary to be rendered back on the client. Today, we're open-sourcing Keyframes so others can create even more fun experiences.

Keyframes: Delivering scalable, high-quality animations to mobile clients

code.facebook.com
The Connectivity Lab team at Facebook is working on high-resolution population maps to learn how people are aggregated in various parts of the world, which will help us design the best communication technologies for each region.

Today, we started sharing the population datasets publicly because we believe this data will be useful in many other ways, including for things like humanitarian aid...
View details ⇨

Open population datasets and open challenges

code.facebook.com
With the onset of these new services like video and VR, we need to make sure our global infrastructure is designed to handle richer content at faster speeds. To meet these current requirements and plan for future bandwidth demands, we’re working toward the 100G data center.

Today we’re excited to introduce Backpack, our second-generation modular switch platform. Together with our recently...
View details ⇨

Introducing Backpack: Our second-generation modular open switch

code.facebook.com
Earlier today at Web Summit, our CTO Mike Schroepfer talked about how our rapid progress in AI is driving advances across Facebook products and research — including connectivity and VR. Research and engineering breakthroughs in each of these areas will help us make more progress toward opening the world to everyone over the next decade.

Accelerating innovation and powering new experiences with AI

code.facebook.com