Twitter Passwords 'Leak' - A Speculative Explanation
Disclaimer: I have no knowledge of what actually happened at Twitter. I don't work there, neither do I have secret sources. Following post is just a for-fun exercise in guessing the most likely scenario. You can also call it "pure speculation", if you really have to.
Yesterday, most Twitter users got a scary notice that their password could have been compromised and the company was asking them to change the password as a precaution:
This sparked a wide range of emotions on social media (shocker!), significant number of users being mad at the company and some even questioning the skills of Twitter's engineering team...
Let me state some strong personal opinions:
Twitter handled disclosure and mitigation superbly. Regardless of how you feel about the likely unintentional turn of phrase from their CTO (source), the reality is that they didn't try to hide it under the rug, and that is uneqiovocally a great thing. Other companies should behave the same way.
What happened doesn't indicate that Twitter lacks engineering skills. It much more likely was just a very unfortunate sequence of events - a genuine accident.
Specifically; it's important to understand that at any but the most lousy tech companies (which Twitter is definitely not) passwords are never stored in clear-text or even encrypted. They are stored "hashed", i.e. obscured in a way that cannot be 'decrypted' even by Twitter. Which means: the only time clear-text password could be logged was before it was saved in a database. And that leads us to the most likely scenario: if somebody turned on logging of all HTTP traffic (with good intentions of monitoring health of the system), without first excluding traffic that contained passwords, then they would get the exact problem described in the disclosure: passwords would unintentionally end-up in the logs. That doesn't mean that engineers at twitter are 'stupid', because it's not an architectural/design problem per se, and neither does it mean that they do not have 'proper controls' – such logging can be turned on via simple misconfiguration and is too easy to miss in code reviews. If anything, the fact that their monitoring noticed the problem indicates that they are doing pretty well.
Accidents happen. It is unfortunate, but it is a reality. Nothing in the story indicates that Twitter did anything careless or wrong. As for us, users we need to always use two-factor authentication (which Twitter does support) and unique passwords to protect ourselves from such accidents.