Under the Hood: Understanding Looker's System Activity Model

Looker ships with a powerful tool called the System Activity Model to enable you to gain insights into the performance and usage of your Looker instance. In this blog post, we'll delve into the intricacies of the System Activity Model, exploring its features, benefits, and how you can leverage it to combat performance challenges and optimize your Looker experience.

What is Looker's System Activity Model?

The System Activity Model serves as a window into the inner workings of your Looker instance. It provides a comprehensive overview of various aspects, including user activity, query performance, content usage, and database performance.

It is deployed as an internal LookML project with Explores that cover all the categories that you need to have an overview of how your Looker instance is performing. It isn’t editable, and it is worth noting that it holds data from the past 90 days in most cases, so bear that in mind if you are generating reports. (You get one year of data with Elite System Activity.)

The System Activity Model enables Looker administrators to monitor, analyze, and optimize the performance of their Looker deployment. Admins have access by default, and additional users can be given access by granting them the ‘see_system_activity’ permission.

Accessing the System Activity Model

There are two ways to access the System Activity Model, via pre-built dashboards from the Looker team or by building your own queries directly from the Explores.

1. To view the pre-build dashboards, navigate to the admin section of your Looker instance. From there, you'll find a suite of pre-built dashboards that offer insights into different components of your Looker instance. These dashboards cover areas such as user activity, content usage, database performance, and instance performance.

2. To view the System Activity Explores, you can find them with the rest of your Explores in the Looker navigation bar. You’ll find Explores for query performance, PDT build times, user metrics, and much more. With these Explores, you can identify trends, track user engagement, and understand how different roles interact with your Looker deployment. This information can be invaluable for optimizing user experiences, refining role-based access controls, and improving overall platform adoption.

Skip to the Good Bit: The Most Useful Explores (In Our Opinion)

History: One of the most useful parts of the System Activity Model is the History Explore, which records every query executed within your Looker instance. This allows you to identify trends, track query performance, and diagnose any issues that may arise. For example, which queries run the slowest, the longest running Explores, and how queries are being used.

Scheduled Plan: This allows you to see an overview of when all your Looker content is scheduled, so you can plan to space out your usage to optimize performance and reduce lag at peak times of day.

Query Performance Metrics: This goes into more detail than the History Explore, by breaking down each query into its constituent parts, such as database load, compiler time, and visualization rendering time so you can understand exactly what is happening with each query. 

Content Usage: Content bloat, be gone! A lot of our customers find that they have unused or underutilized dashboards, reports, and Explores. With the Content Usage Explore, you can streamline your content library, remove redundancies, and ensure that resources are allocated efficiently. N.B.: if you want to delete Looker content the right way, check out our Content Management feature.

As always, no two Looker instances are the same, so if you have particular challenges that you are facing then it may be that some of the other Explores will be most useful to you. For example, if you are looking to reduce the number of licenses that you have, then the User Explore will be your best friend. If you aren’t sure the best diagnostics to use, feel free to reach out to us (👋 hello@spectacles.dev).

Level up with Elite System Activity

Elite System Activity is available on higher tiers of looker, and has two major benefits for larger organizations:

1. It gives you 1 year + of history, rather than the standard 90 days.

2. It exports the data to BigQuery, which reduces load on your instance and improves concurrency.

Where to Begin: An Overview of Looker’s Pre-Built Dashboards

Looker provides you with a selection of pre-built dashboards to give you a head start with navigating your diagnostic information. Here is an overview of the kind of questions that each dashboard will help you answer:

1. User Activity: How many users do you have, what roles are they, and how are they interacting with your Looker instance?

2. Content Activity: How much content do you have, what is scheduled, what is the most popular content?

3. Database Performance: Does time of day make a difference to run times? How can you increase concurrency?

4. Instance Performance: What is putting excess load on Looker performance?

5. Performance Recommendations: This dashboard will signpost you to some of the issues that you might be facing in your instance and how to solve them.

6. Errors and Broken Content: A collection of error messages that the database is flagging. IMHO this dashboard is too cluttered to be really useful, which is a big part of why Spectacles exists. Drop us a line if you are looking to do a tidy up of your troubleshooting and diagnostics. 
   

Real-world Examples

As part of our fortnightly webinar series, I hosted a session on the System Activity Explores, which you can access (for free!) here. As part of that webinar, I highlighted three examples of real world challenges that can be solved with the System Activity Model. Head to the 20 minute mark of the webinar to see a step-by-step guide on how he would answer these questions:

• Help! Looker is running really slowly - can this query run any faster?

• Ugh! We have too many schedules kicking off all at the same time. What do we do?!

• Analysis paralysis! Looker has become too hard to navigate - our users are getting lost. How do we do a spring clean?
  ‍

Keep watching to the end to see the Q&A with our Looker dev community in Slack answering questions on this topic live. If you aren’t already part of our Slack community, you can join here.

So there you have it. Looker's System Activity Model provides administrators with a powerful tool for gaining insights into their Looker instance. By exploring user activity, content usage, query performance, and database performance, you can optimize your Looker instance to drive better outcomes for your organization. As ever, if you would like any support in diagnosing issues and implementing solutions within your Looker instance, Spectacles is here to help.
‍

We always love to hear from you. Have you got any insights to share from diving into the System Activity Model? Share them with us on Slack or reach out at hello@spectacles.dev.
‍

Spectacles runs webinars every two weeks, designed for the Looker community, that enable you to uplevel your Looker instances and leverage the power of Looker across your organization. To sign up to future webinars, visit our website: https://www.spectacles.dev/webinars

‍