Mutant Monitoring System (MMS) Day 13 – Materialized Views

This is part 13 of a series of blog posts that provides a story arc for ScyllaDB Training.

In the previous post, we went over prepared statements and how developers at Division 3 can optimize their applications more efficiently by using them. The Mutant Monitoring System had been receiving a lot of data and Division 3 wanted to find better ways to sort and store data so it can be quickly analyzed with applications. Luckily, having found an exciting feature in ScyllaDB called Materialized Views, they provided us with directives to learn how to use it to help our application developers prevent further acts of terror. In this post, we will learn what Materialized Views are and how to use it with the Mutant Monitoring System.

What are Materialized Views?

Material Views automate the tedious and inefficient work that must be done when an application maintains several tables with the same data that’s organized differently. In the past, every time the application wanted to write data, it needed to write to both tables, and reads were done directly (and efficiently) from the desired table. However, ensuring any level of consistency between the data in the two or more views required complex and slow application logic.

ScyllaDB’s Materialized Views feature moves this complexity out of the application and into the servers. This approach makes it much easier for applications to have multiple views of their data. The application just declares the additional views, new view tables are created, and on every update to the original table (known as the “base table”), the additional view tables are automatically updated as well.

Please note that Materialized Views are not production ready and are still considered experimental in ScyllaDB 2.1, 2.2 and 2.3.

To get started with Materialized Views, we will first need to bring up the ScyllaDB Cluster.

Starting the ScyllaDB Cluster

The ScyllaDB Cluster should be up and running with the data imported from the previous blog posts.

The MMS Git repository has been updated to provide the ability to automatically import the keyspaces and data. If you have the Git repository already cloned, you can simply do a “git pull” in the scylla-code-samples directory.

git clone https://github.com/scylladb/scylla-code-samples.git
cd scylla-code-samples/mms

Modify docker-compose.yml and add the following line to the environment: section of scylla-node1:

- IMPORT=IMPORT

Now we can build and run the containers:

docker-compose build
docker-compose up -d

After roughly 60 seconds, the existing MMS data will be automatically imported. When the cluster is up and running, we can learn how to create a few Materialized Views.

Starting the Load Generator

Before we create a Materialized View, we will need to use a load generator tool that will start adding tracking data for each mutant in the ScyllaDB cluster. To get started, we will first need to build the Docker container:

cd scylla-code-samples/mms/loadgen
docker build -t loadgen .

To start the load generator, run the following command:

docker run -d --net=mms_web --name loadgen loadgen

After 60 seconds, the load generator will begin populating the keyspace with data. You will be able to see when it starts by viewing the container logs:

docker logs -f loadgen

Creating the Materialized Views

Now let’s connect to a container running ScyllaDB so we can use cqlsh to interact with the cluster:

docker exec -it mms_scylla-node1_1 cqlsh

use tracking;

Now it is time to create our first Materialized View. According to our original table schema for tracking, tracking_data, our primary keys are first_name, last_name, and timestamp. When creating a Materialized View, we will need to reference all of the primary keys that were used when creating the tracking_data table:

Let’s assume that Division 3 wants to retrieve only the mutant’s timestamp at a certain location. The following Materialized View will show only the timestamp and location for each mutant without the other data columns:

Now we will be able to see the data from the Materialized View with the following command:

To narrow the results down by city, we can add a location to the select query. Let’s see who is in Cincinnati:

Conclusion

In this post, we went over Materialized Views and showed a few examples of how to create views from the Mutant Monitoring System data. With Materialized Views, developers will be able to create intensive applications that can store and sort through data faster and more efficiently with less coding because those tasks can be handled by the ScyllaDB cluster rather than in the applications themselves.

Please be safe out there and continue to monitor the mutants!

Next Steps

• Learn more about ScyllaDB from our product page.
• See what our users are saying about ScyllaDB.
• Download ScyllaDB. Check out our download page to run ScyllaDB on AWS, install it locally in a Virtual Machine, or run it in Docker.