Spark Summit 2014 brought the Apache Spark community together on June 30- July 2, 2014 at the The Westin St. Francis in San Francisco. It featured production users of Spark, Shark, Spark Streaming and related projects.
Telefonica is the incumbent telecommunications network operator in Spain and the fourth one in capitalization in the world. Cyber security is one of our most successful businesses worldwide. We provide monitoring and protecting clients from attacks. We analyze millions of data from multiple sources including social media, DNS records, underground internet, etc in order to generate alerts and security reports for our clients.
This use case required a Big Data component capable of processing the data and extract its information in real-time; warnings and alerts are time-sensitive in order to deal efficiently with security attacks.
Our original architecture was the typical one used for data fusion systems. It included several collectors, a processing layer based on legacy systems and a data store. The initial setup included a MongoDB database and an ad-hoc application. This solution however proved to be unfit for the specific purpose of dispatching alerts. We proposed to use Cassandra and Spark instead. This approach did manage to fulfill our original specifications as intended.
Our talk will explain the reasons why we migrated the architecture and how the adopted solution based on Spark and Cassandra solved our problem.
Telefonica and Stratio developed a Big Data architecture specific to cyber security, one of our most successful businesses worldwide.
Telefonica provides a monitoring and protection service thanks to the Stratio’s technology. It is capable of processing millions of information events from multiple sources and extract its information in real-time; warnings and alerts are time-sensitive in order to deal efficiently with security attacks.
The vision of Telefonica as the fourth telecom operation in the world and the technology of Stratio is a winning combination.
Fran J. Gomez was born in Madrid. He works as a Security Engineer and his professional career has always been associated with IT Security, even before he completed his university studies. In 2005, Fran joined Telefonica I+D Hacking Ethical Team to participate in security researches on ISP core networks technologies, which has allowed him to know deeply some of the protocols and technologies that will build the future Internet. His current research is focused on Cyberintelligence at the Security Area of Telefonica. Fran has also been participated as speaker to events such as RootedCON, RedIRIS Security Forum, CCN-CERT STIC or TEDxTelefonica.