How to Streamline Dashboard Development in Looker

Welcome to Plant Life

Picture this: a new data analyst joins a company called Plant Life that sells houseplants 🌱 on the Internet.

Oh, you think that sounds like a small business opportunity? I’ve worked with a client in this industry and their annual revenue was an order of magnitude greater than almost every B2B SaaS tech startup I’ve consulted for; you’d be surprised at how lucrative e-commerce can be.

Anyway, given how much is at stake, it’s reasonable to expect that their internal business intelligence reporting would be consistent, robust, and well organized.

Here’s what happens. Our new data analyst opens up Looker Admin and sees the most viewed dashboard is this one:

It gets worse…

• The author of this dashboard, “Mark,” was a contractor whose term was cut unexpectedly short.
• When Mark was building the dashboard, people started sharing it. It’s not done, but now it’s the most popular dashboard at the company.
• New data requests are added as new tiles to the already 100+ tile dashboard; old or broken tiles are never cleaned up and some tiles show the same data.
• The top complaints from stakeholders are that the dashboard takes too long to load, some tiles directly conflict with each other, and the data team takes too long to fix broken tiles.

This happened because there was no way to distinguish between dashboards still being worked on and ones that were ready for everyone to use.

As a result, a half-baked solution became a critical piece of reporting for the company, leaving the data team in the lurch and stakeholders unsatisfied at their level of service.

A lifecycle for dashboards

Listen, there’s a better way to manage the development, deployment, and deprecation of dashboards.

We can adopt the concept of “lifecycle management” to assign stages of life to our fledgling dashboard as it grows up, goes to college, has a career, and eventually moves to Florida to play golf all day. Each stage of life comes with appropriate expectations for quality, availability, and maintenance.

Here’s a proposed Looker content lifecycle that clearly communicates the stage of a dashboard. With this, data teams can better manage expectations for support requests (e.g. service-level objectives, or SLOs).

Development

• Where? Personal Folder
• Who can view? Self
• SLO? None

Staging

• Where? A department folder (e.g. Finance) under Shared Folders
• Who can view? Anyone
• SLO? None (use at your own risk)

Production

• Where? A department Board (optionally a LookML dashboard when there are higher stability requirements)
• Who can view? Anyone
• SLO? 99% uptime, <1 hour response time, clear escalation process, fully managed by data team

The exact SLO definition will vary per organization, but it’s important that you set concrete expectations for reliability that include uptime, response time, and resolution time. If a dashboard has no SLO, stakeholders should not expect any timely level of service and should use at their own risk.

Luckily for Plant Life, we’ve been contracted to re-architect their Looker setup at the behest of their new data analyst. They’ve asked us, unsurprisingly, to rebuild Mark’s Business Metrics Dashboard.

Let’s see how we can implement our content lifecycle plan and make content at Plant Life more reliable.

Use your Personal Folder for development

We need a judgment-free zone where anyone can work on a dashboard, so we’ll teach all dashboard creators to save works in progress to their Personal Folders.

There’s a problem though. In Looker, by default, any user can view any other user’s Personal Folder. As admins, we want to turn this off so that business users don’t mistakenly start using dashboards that aren’t yet ready for primetime.

We’ll remove the All Users group from the Users top-level folder (screenshot below taken from Looker’s docs on access levels).

Individual users can also remove the All Users group from their Personal Folder themselves using Manage Access.

Note that if you’ve had Looker support enable the Closed system option for your Looker instance, this is already taken care of.

With this setup in place, users can safely develop in-progress dashboards without worrying that other folks may become dependent on them.

Use Shared Folders for staging and review

Dashboards that are ready for review by others should be moved from the Personal Folder where it was developed to Shared Folders. From there, stakeholders can begin stress testing the data and giving feedback.

Within Shared Folders, each team should have its own subfolder (e.g. Product, Marketing, Finance).

It’s critical at this stage to log all review and discussion in a single place, similar to GitHub’s pull request feature. Maintaining this log is essential, as it streamlines future conversations by clarifying the scope and reasons for past decisions, reducing the need for time-consuming discussions. Here are some suggestions on spaces to manage that discussion:

• Avoid Slack, especially DMs: Public Slack channels are much better than DMs because the conversation is transparent and searchable to the whole organization.
• In-person or virtual meeting reviews need to be documented: If doing an in-person or virtual meeting to review, make sure to take notes of the action items and, most importantly, share those action items in writing.
• Use Jira (or comparable ticketing system): Even though ticketing systems tend to be unpopular among business users, I think they are still the easiest place to consolidate discussion around discrete pieces of work. If business users are particularly averse to using Jira, a nice compromise I’ve found is starting a thread in a public Slack channel and pasting a link to that thread in the ticket.

Here’s the new workflow the Plant Life analyst would go through to get their dashboard reviewed:

• They move the dashboard to the Executive folder within Shared Folders.
• They share the link in the corresponding Jira ticket and ping reviewers (typically the end business user).
• If there’s no response in Jira within 24 hours, the analyst reaches out in a public Slack channel to their end business user and asks them to review.
• If the end user provides feedback directly in Slack, the analyst copies the link to the thread and pastes it in the Jira ticket.

Once the design is stakeholder-approved (i.e. some message in writing from them along the lines of “looks good”), we’ll need to go through a final step to communicate that this dashboard is ready for broad usage.

Use Boards (and LookML dashboards) for production

Dashboards that have been approved for broad usage should be pinned to a team’s corresponding Board.

Boards are more effective than Folders for organizing:

• You can add sections and context to groups of dashboards and sort them based on importance.
• You can convert huge dashboards with many tiles (the main culprit for slow-loading woes) into smaller ones, grouping them together in a Board section.
• You can re-use dashboards across boards.
• You can pin arbitrary URLs to boards (i.e. non-Looker dashboards), which is especially useful if your company uses other BI tools.

Imagine how much better an experience it is for business users to view content on a Board…

…than in a Shared Folder.

Using Boards establishes a clear separation between Production and Staging; if a business user accesses a dashboard from a Board, they can be confident that it is reliable.

To incentivize usage of Boards over Folders, the data team needs to commit to maintaining a strong SLO for Board-pinned dashboards, for instance:

• 99% uptime. No broken tiles and a relatively fast load time (e.g. <10s).
• <1 hour response time during business hours. In cases where there are data discrepancies or errors, someone from the data team should be able to triage the issue in less than an hour from when the issue was raised.
• Clear escalation process. After an error is triaged, the stakeholder should not have to ask who is responsible for fixing, the turnaround time, or the reason for the issue. After the fix, the fixer needs to file an incident report so that the data team can plan to make sure the issue doesn’t happen again.

Optional: Convert to LookML dashboards

For even greater dashboard stability, we recommend converting production dashboards to LookML dashboards.

Here are some instances where you could benefit from a LookML dashboard.

• Serving data to external users, i.e. clients or customers who live outside your company.
• Very popular dashboards that internal users rely on daily.
• Executive dashboards that support high-stakes decisions.

Be warned—converting a dashboard to a LookML dashboard introduces some overhead, especially when it comes to making future changes. For this reason, we don’t recommend converting all dashboards into LookML, especially for narrow or short-lived use cases (e.g. an A/B test).

It’s easier to manage a higher SLO for a LookML dashboards than a user-created dashboard.

• Version controlled. Easy to rollback to previous versions if something breaks.
• Errors caught at LookML validation time. Renamed references get alerted before you can even commit them. The Spectacles content validator or Looker content validator will test this, but it’s better to alert the developer earlier in the process.
• Well-suited for multi-instance deployments. There’s no additional scripting required to deploy LookML dashboards to a separate production instance since it flows through version control.

Let’s see how this process plays out in our Plant Life example for our analyst:

1. First, they copy the LookML code from the Staging dashboard.

2. They create a new dashboards folder under logical in the LookML project.
3. They paste the LookML dashboard code as a new file e.g. business_metrics.dashboard.lkml.

4. They merge changes just like normal LookML code.
5. Finally, they pin the LookML dashboard (located in the Shared LookML Dashboards folder) to the Executive Board.

This example highlights another key benefit of LookML dashboards: we can rename Explores or columns and update the dashboard references in a single merge. If someone merges LookML that renames an Explore, all downstream user-defined dashboards will break and we’d have to go to the Content Validator to update the references. This will always result in some downtime. With LookML dashboards there is zero downtime since we can update the dashboard LookML code along with the explore rename code in a single merge.

To edit a LookML dashboard, the analyst could edit the YAML directly, or follow these steps.

1. Copy the LookML dashboard to their Personal Folder; this automatically converts it back to a user-defined dashboard that is editable via drag and drop.
2. Edit the copy.
3. Get the LookML of the copy and paste it in the LookML dashboard file, making sure that the dashboard value is unchanged so the dashboard maintains the same link.
4. Delete the Personal Folder copy.

Deprecating old dashboards

Dashboards are forever, right? …or was it diamonds? In my experience, dashboards are forever, though they probably shouldn’t be. To put a cherry on top of our content lifecycle, we need a robust approach to dashboard deprecation.

Periodically, you should go through the content in your Looker instance and delete content that isn’t being used.

In our content lifecycle, this would be content…

• In Personal Folders that was created long ago and never made it out of development
• In Shared folders that has since been upgraded to production as a LookML dashboard
• On Boards or in Shared folders, but the business has decided to move away from

I’d recommend hosting a quarterly “spring cleaning” meeting with all of your team members. The purpose of the meeting would be to go through every dashboard in your team’s Shared Folder and make a decision to clean up old dashboards:

• Do nothing (i.e. “this dashboard is still in review”)
• Pin to Board (i.e. “this dashboard is ready to move to Production”)
• Convert to LookML (i.e. “this dashboard is critical and needs higher reliability”)
• Schedule for deletion in 7d (i.e. “we can get this data from somewhere else, but need some time to migrate”)
• Delete now (i.e. “this dashboard is irrelevant and can be deleted now”)
• Transfer ownership (i.e. “Teddy has left the company, let’s transfer to Jenny”)

I used to run this meeting when I was a data engineer at Better and found it to be one of the most highly attended, engaging, and productive meetings I’ve organized. Give it a try!

Once you’ve identified the content that needs to be deleted, you can delete it manually or script it. It’s best practice to give content owners some advance notice and the reason why you’re deleting their dashboards.

You can use Spectacles' Content Management tool to do all of this in one go. Spectacles displays all content in your instance, who’s using it, how much it’s being used, and allows you to schedule content for deletion. Once scheduled, Spectacles notifies the content owners and gives them a chance to opt out.

Here’s how this would work in our Plant Life example.

1. We’d decide to deprecate the old version of the Business Metrics Dashboard.
2. In Spectacles, we’d schedule the deletion of the dashboard for 14 days from now.
3. Spectacles notifies the owner of the dashboard that the dashboard will be deleted, giving them a chance to opt out or contact the responsible Looker admin.

It’s the circle of life

It's clear that the content lifecycle plan outlined isn't just a recommendation; it's a blueprint for Looker administrators to nurture their dashboards effectively. This approach ensures that from the initial development in Personal Folders to the final display on production Boards, each dashboard matures through stages with clear expectations and service level objectives.

In essence, adopting this lifecycle plan is not just about managing dashboards; it's about cultivating a data-driven culture that values precision, clarity, and continuous improvement. For Plant Life, this means a Looker setup that's as robust and dependable as the business it supports, poised to grow alongside the company.

Compare and contrast how easy our new Plant Life Executive Board will be to maintain…

...compared to the old one.

Data analysts will benefit the most from this new process:

• No more getting stiffed maintaining an unmaintainable Hack Week dashboard that was built two years ago.
• A smaller, clearer set of “production-grade” dashboards to support versus the status quo of hundreds of dashboards with unclear expectations.
• Isolated development environments that encourage creativity and experimentation.

Business users benefit as well. This setup will drive greater clarity, focus, and trust in the data your team produces.

Paired with Spectacles content management tool, this content lifecycle policy enables data teams to evolve from “dashboard creators” to data product owners, driving greater efficiency and influence across their organization.

‍

If you're a current Spectacles customer, reach out to support to enable the new Content Management tool shown here. If you're not a customer, book a demo today.

‍