Improving
Redpanda's
onboarding
Goals
Redpanda Cloud adoption was a high-touch manual process driven by Sales and Customer Success. Despite this the median time for users to onboard was ~60 days. I owned the company-wide KPI to lower time to onboard, which I reported on a weekly basis to leadership.
Roles
Improving onboarding was an ongoing process where I owned the overall goals, projects and outcomes. This required collaborating with multiple teams to set the vision, prioritize items during planning and often doing the tactical design work.
Outcomes
96% proof-of-concept success rate
Median time-to-onboard decreased from ~60 days to as low as ~2 days
Project details
01
Defining a success metric
Collaborating with Product Management leaders I defined the onboarding success metric as 20 MBs total data received for a new organization.
This metric was chosen as a reasonable threshold of a substantive workload (more than exploring). New organization was chosen as there may be multiple users and resources created and we wanted to measure the overall success, not of individual users or resources.
20 MBs
Total data received for a new organization
02
Measuring the onboarding user journey
Working with our data analytics team I defined the onboarding user journey steps to track and measure:
- Sign up
- Organization created
- Namespace created
- Cluster created
- Users created
- ACLs created
- Topics created
- 20 mbs total data received
Unfortunately some of these data points could not be included in the funnel due to technical constraints in collecting certain metrics. With the data points we had we were able to create dashboards that gave an overview of the onboarding funnel, success rates and time to success. It also allowed for analyzing data for by organization.
03
Research
Research was an ongoing process with various types of quantitative and qualitative research was done as the product evolved. This included
- Weekly analysis of the onboarding analytics was done to identify where and why delays or friction points were occurring. In addition to the analytics data I analyzed Salesforce data and reached out to Customer Success. I reported the metrics and insights to leadership on a weekly basis.
- User interviews were done intermittently with the goal of identifying friction and drop off points and the amount of effort needed to complete tasks
- Heuristic evaluations were done early in the product's evolution. This was done as part of the onboarding process for one of my reports.
04
Insights
INSIGHT 01
Assumption that the user is an expert
Redpanda’s marketing, documentation and products are appropriate for expert users familiar with data streaming, however, for non-experts it was difficult to understand what Redpanda does and how to use it, making onboarding difficult.
- Little explanation of what Redpanda does
In marketing there are mainly comparisons to Kafka (the foundational technology). - Few use cases or industries listed - Marketing
Customer case studies are available in marketing but they are not categorized by industry or use case. - Documentation is not sufficient for a non-expert user. There is no Cloud quick start and documentation mixes Cloud and self-managed content.
INSIGHT 02
Little guidance provided in-product
Once a user has access to the product their is little explanation or guidance to help the user.
- After sign-up the user lands in the zero state for Namespaces, with no explanation or guidance.
- Few explanations or help links
- No recommended next steps after cluster creation
INSIGHT 03
Private networking is complex
With Redpanda's Bring-Your-Own-Cloud (BYOC) architecture Redpanda clusters can be deployed into a user's Cloud infrastructure, while being fully-managed from Redpanda's controlplane. BYOC increases security and keeps data within a customers infrastructure, however, it greatly increases the complexity of networking between Redpanda's and the customer's infrastructure. This complexity is exacerbated by Network admins teams usually being separate from the team working with Redpanda.
INSIGHT 04
Using the CLI was cumbersome
Authentication with Redpanda's Command Line Interface (CLI) was difficult as users had no guidance to create a user, and permissions and then use that information as environment variables in the user's local development system.
export REDPANDA_BROKERS="<bootstrap-server-address>"
export REDPANDA_SASL_MECHANISM="<SCRAM-SHA-256 or SCRAM-SHA-512>”
export REDPANDA_SASL_USERNAME="<username>"
export REDPANDA_SASL_PASSWORD=”<password>"
CLI (rpk) documentation was very technical and did not give examples of how to do common tasks.
05
Solutions
Solution 01
Create default resources with hello world data on sign-up
When a new user sign's up a default serverless cluster is created, pre-populated with data to get started with. A welcome modal is shown that gives recommended next steps to view the data.
Solution 02
Simplifying CLI login and cluster selection
Get started with rpk (Redpanda's command-line-interface) was complex for users as they had to create a user and assign permissions in the GUI and then set several environment variables in the CLI. We simplified this flow by integrating Single-Sign-On so a user can authenticate using their user and password, Google or Github login. Cluster selection in the CLI was also simplified; if there was only one cluster it was selected by default, if there were multiple users would be prompted to select one from a list.
Solution 03
Code samples in GUI and CLI
To simplify developing applications, code samples were made accessible in both the GUI and CLI. I led this effort and created the designs and also developed the code samples based on existing tutorials.
In the GUI a "How to connect" card was added to the cluster overview with an API section that allowed users to select a programming language and view a relevant code sample that was customized to that cluster's configuration.
In the CLI a new command was created that allowed users to choose a language and then download a sample application to their local environment.
Solution 04
Networking guidance
Private network configuration was a common user friction point. This was exacerbated in Enterprises where network admins were often on a different team from the one trying Redpanda. To help reduce the friction, in-product guidance was added to help users calculate IP ranges (CIDR blocks) and set up VPC peering.
Solution 05
Quickstart in documentation
One basic item that was missing was a Cloud quickstart in documentation. Collaborating with the documentation team I advocated for and got prioritized the creation of the Cloud quick start.
06
Outcomes
The following outcomes were derived from an analysis I did of ~1 year of data from the analytics data warehouse and Salesforce. One point to note: Customer Success was involved in helping users in some of the POCs, the amount of help varied by customer.
OUTCOME 01
96% proof-of-concept success rate
96% companies were successful in their POC, as measured by reaching 20 mbs received or a successful sales outcome. There were more POCs successful than 20 mbs received. This suggests that we should revisit the threshold for mbs received.
OUTCOME 02
Median time-to-onboard decreased from ~60 days to as low as ~2 days
There was a large drop in time-to-onboard in June 2023 that remained consistent to October 2023. Post October the median has trended higher but the P10 and P25 remained stable. Many improvements to the onboarding flow were made that most-likely contributed to the drop in time-to-onboard. The largest contributing factor to delay in onboarding was due to customer (see outcome 04), for example, holidays, project pauses, etc.
OUTCOME 03
53% of customers took less than 22 days to onboard, with 27% taking less than 7 days
This was a large improvement from the highest median of~60 days.
OUTCOME 04
Internal customer delays was the largest factor for delayed time-to-onboard
I gathered this information by manually parsing Salesforce records for each customer.
