As we prepare for ScyllaDB Summit 2019, we are producing a series of blogs highlighting this year’s featured presenters. And a reminder, if you’re not yet registered for ScyllaDB Summit, please take the time to register now!
Today we are speaking with Rahul Gaikwad and Krishna Palati of FireEye’s devops team. Their presentation at ScyllaDB Summit 2019 is entitled FireEye & ScyllaDB: Intel Threat Analysis using a graph database.
Cybersecurity is top-of-mind today, not just in the tech industry but even in the view of everyday consumers and citizens. What principles do you keep foremost in mind as the cybersecurity market evolves?
Krishna: Our goal has always been to Find Evil, Solve Crime. We help our customers identify breaches, help them understand what happened and what to do next. We know the most about cyber threats via our intel information. We use our appliances and services in conjunction with this information to make our customers most secure. We focus most on common avenues of cyberattack, for example Endpoint, Network & Email protection.
Rahul: We gather data about cyberattacks and crimes from various sources and make it meaningful and available at a centralized place to perform different kinds of analysis and reporting. Based on collected information, we identify actors, cyber criminals, attackers, hackers and their footprints, origins, patterns and relationships which helps us to respond to these cyber incidents.
To save our customers from cyberattacks, we are always up to date with technologies, trends, news, forums and data sources. We concentrate on providing security to most of digital areas like devices, email , endpoints , cloud etc.
Describe the threat intelligence system you’ve built at FireEye. How does it work?
Rahul: Our current threat intelligence system is a based on custom graph database which is built to store vast amounts of information related to the cyber threat landscape. It includes user interfaces that facilitate analyst-driven querying and additions to the graph. It allows our analysts to contribute and analyse a shared representation of the threat landscape. It centralizes, organizes, and standardizes threat intelligence data and assessments (like attribution of particular activities to a specific threat group).
This means that users/analysts can get insights of each others’ analyses, research results and use the same platform to discuss or model their findings. It also provides comprehensive definitions for FireEye’s tracked threat groups by recording all of the analytic correlations that describe what we know about each threat group and how we know it.
Krishna: We generate, collect and curate intel with our in house analysts team – it is one of the largest teams available in the private sector. Intelligence information is disseminated to our internal and external customers via subscription-based reports as well as programmatic access. To make programmatic access possible, relevant intel is converted into property graphs and ingested into our threat intelligence ecosystem. The system uses proprietary algorithms and techniques on top of this property graph database to deliver cyber intel capabilities. Our analysts use these systems to analyse, collaborate with each other and find evil effectively and quickly. In addition, FireEye systems/appliances can also access this programmatically to provide context when they hit an IOC alert.
There are a number of graph database solutions available. Of all the choices out there, what specifically prompted FireEye’s move to adopt JanusGraph and ScyllaDB?
Rahul: Before we adopt JanusGraph and ScyllaDB, we conducted internal technology evaluations and, based on our use cases and results, we realized JanusGraph with ScyllaDB DB is most suitable to fulfill our requirements.
From our tech evaluation , we found some distinguished features:
Janus Graph
- Indexing capabilities that can be controlled by the user.
- Full Text search
- Embedded as well as Server mode setup capability
- Schema Management
- Triggers
- OLAP Capabilities – Distributed Graph Processing
ScyllaDB
- Easy Cluster setup
- Self-Tuning
- Equal Load distribution
- Easy to Manage On Cloud
- Less Administration
- No GC
- Low Latencies for Read and Write
A database is only as useful to the other applications and data systems that it is connected to. What are the other key components of your overall data architecture, and how do ScyllaDB and JanusGraph integrate with them?
Krishna: Most of our business logic as explained above is in Application UI and API server layer. It’s a Java based app and uses Gremlin to run complex queries that traverse the graph database. The whole solution is deployed in the Cloud. We have load balancers, proxies, app servers, JanusGraph, ScyllaDB & ElasticSearch that typically fit into N-tier architecture.
Rahul: We have been migrating existing systems data from a relational Postgres database to the new graph store which is built up using JanusGraph. We have ScyllaDB for backend storage and AWS ElasticSearch for indexing. For one time bulk migration we are using embedded JanusGraph which directly transposes the data into backends (ScyllaDB+Elastic).
To capture delta data (live events), we have continuous replication from existing system to the new system, which leverages AWS ElastiCache as a queue. The continuous replication also uses embedded JanusGraph which pull data from ElastiCache and write to the backend systems.
Also we have developed a customised Java API layer which translates existing proprietary analyst-driven queries into Gremlin queries. Along with this API we also provide customised UI which shows graphical and tabular representations of threat data from ScyllaDB and Elastic. We are storing analyst queries and session related information in an Aurora database.
For High Availability on all of this we have multiple instances of applications, databases, and load balancers which span across multiple AWS AZ’s.
Rahul, I saw on your LinkedIn profile that you’ve been through all of the ScyllaDB University courses and displayed your achievement on your profile. What was your experience with ScyllaDB University? Would you recommend it for other Big Data practitioners?
Rahul: ScyllaDB University is a very good starting point to learn ScyllaDB. These courses are very informative and well explained. Through these courses, I gained the knowledge about ScyllaDB architecture, installation, features and advantages. The best part is that it’s free and online which covers conceptual and hands-on knowledge. I highly recommend ScyllaDB University courses to all Big Data practitioners like DBAs, System Engineers, Developers and Architects who want to understand and learn NoSQL concepts and implement through ScyllaDB.
Krishna, tell us about FireEye’s devops team and work. I read that you use Terraform and Puppet. What other go-to tools do you employ to keep your data systems running smoothly? Tell us what you turn to for your day-to-day success.
Krishna: I run the Devops for FireEye Solutions Engineering which includes Managed Defense, Intel & Incident Response business units. Our goal is to automate as much as possible via infrastructure build tools like Foreman & Terraform, configuration management tools like Puppet & Ansible, monitoring & log aggregation using Datadog. We rely heavily on Jenkins for our Continuous Integration pipelines. We do static security code analysis via Coverity. Depending on the type of app, we use Java, Django or Ruby rake to manage the data models / schema & db deployments. In addition to this, my team is also responsible for cloud operations and day to day administration of applications and database systems.
What is one thing that most people in the tech industry don’t know about yourself that you’d like to share? Tell us about a talent, a hobby, or a little-known truth.
Krishna: I like to be disconnected, be out hiking or go for a long bike ride or a run. I also love to travel, explore new cultures and cuisines every chance I get. I like to read non-fiction.
Rahul: I love to paint, travel, play cricket and read novels. And I also like to watch sci-fi and comedy movies, serials or documentaries.
Rahul and Krishna, thanks for taking the time to talk with me today. I’m sure all our JanusGraph fans will be eager to see your talk!
