There was a massive global outage of Facebook's online services: Facebook, Instagram, and Whatsapp. As bad as that is, there were reports that Facebook employees could not access the building. That made me curious.
As someone that has worked in physical security for the last 10 years the PAC system failing threw up a major red flag for me. Access control systems typically use control boards that save a local copy of their configuration so if they lose the connection to the access control software, they can still function and people can still access the building.
So what happened?
The easiest assumption is that they implemented a system that relies on a live connection to authenticate access to doors. Would a company with the resources that Facebook has really have an access control system that could fail so easy? I found that hard to believe at first, but as more reports came out it did in fact seem to be the case. From the limited information I could find on what happened it sounds like they designed an access control system that would be secure and difficult to bypass. In doing so they missed one important feature that the system would need. Access.
It was an important lesson that everyone can learn from. Sometimes you can get wrapped up in one idea and completely miss the point of what your were doing to begin with. The system was there to keep unauthorized individuals out, but also to grant access to the proper individuals. You should always have a safeguards in place to access your building and critical systems, weather that be door controllers that store an offline configuration or a physical key override.
You can always supplement your access system with other security measures such as security cameras or an alarm system that notifies police or building security if breached. This way you can implement an access control system that won't slow down incident response, but is backed by additional safeguards to ensure the building is secure.
As great as the technology can be its important to remember that it can and will fail, usually at the worst times. Always have a well established backup plan. Implement a plan to distribute keys to critical people that will be responding to incidents, or install a secure lockbox or safe with a physical combination for incident responders to access and get a key for the server rooms. Create a plan for access and ensure everyone knows that plan and how to execute it.