Growing Data Scientists Into Manager Roles

In this post, Ricky Chachra, Research Science Manager at Lyft, provides insight for companies looking to home-grow their promising individual contributors (ICs) into effective managers. He reflects on his journey at Lyft, where he started as a data/research scientist and transitioned into a science management role. In his new-found capacity, Ricky is helping codify the responsibilities of Science Managers at Lyft. This post is related to a forthcoming companion post by Joshua Nguyen as they compare frameworks born out of their individual experience.

On Being the Eighth Data Scientist at Lyft

I joined Lyft in June 2015, as their eighth Data Scientist. Back in those days, Lyft was a company of about 350, housed in a small building in San Francisco’s hip Mission District. It is now well more than 3,500, with staff presence in several major American cities. Seeing both myself and the company work through various stages of our ongoing maturation, I had a distinct vantage point from where I have drawn the following reflections about growing ICs into managers.

What first drove me to Lyft is also what keeps me there today, more than three years later. We share many of the same character traits that are best summarized as being friendly, fast-growing, socially and ethically conscious, farseeing, and driven to excellence. I’m proud to confess that after interviewing at Lyft in March 2015, I felt such an attraction that I immediately dropped other companies out of consideration.

Where I was in my personal life in mid-2015 could also be said of how Lyft manifested then: we were both relatively junior in the social strata, not decisively sure about our raisons d'être, less certain about our near-term prospects, and still learning to stand in this chaotic world. Now 2018 is almost over, and Lyft and I still mirror each other: we’re more confident, no longer “junior”, our networks have expanded, we have a say in important matters, a clarity in our purpose, and the roadmaps to getting where we’re going.

In retrospect, it’s quite incredible that right at the start of my role as an individual contributor, I was placed in the highly visible Marketplace team and entrusted to single-handedly drive the development of our dynamic pricing algorithms. These algorithms respond to and regulate the crucial demand-supply balance on our platform and many company executives and general managers pay careful attention to them. Empowered by supportive colleagues, and brimming with ideas and determined to make an impact, I had my Lyft-off into the honeymoon period with all engines on full power. Unbeknownst to me, turbulence was brewing behind the scenes in senior and cross-functional manager circles. Within a few months, I came to be in the middle of challenging debates about metrics, project priorities, and serious staffing issues on engineering and product. Several events unfolded that taught me a lot about myself and others, informed my career decisions, and rapidly evolved my views on the practice of management.

To keep-up with the bursting headcount growth, some of the most brilliant ICs in data science and engineering transitioned into management. It was the most rational thing to do in the given context, something any fast-growing startup would do. In retrospect, though, we failed to appreciate that excellence in technical abilities doesn’t necessarily translate into excellence in management. Today, no one at Lyft would deny that management is a distinct field that is more akin to the arts. Understanding management comes from having experience, formal education, mentorship, a keen observation of role models in action, and self-study.

Seeing how manager actions were not always positively affecting my team’s performance, our well-being, and career progression, I promised myself that I would chart my own path to find what it takes to excel in management. Management was not completely foreign to me. Prior to joining Lyft, I was an in-house management consultant within the Chief Analytics Office at IBM. Growing further in this direction seemed like a natural progression. Yet, when I focused on management within the broader field of data science, a bigger can of worms opened.

The Rising Need for Data Science Managers

If you look at the rate at which the data scientist (DS) population is continuing to climb, you can’t help but realize that this explosive growth must have sounded alarm bells even among seasoned executives. After all, is there another new profession in recent history that has scaled as quickly. The advances in data science have brought us a panoply of choices on multiple fronts. From the ever-increasing means of data capture, storage, and modeling to algorithms and complex ethical matters, executives have many serious choices to make. However, two factors, the relative infancy of this profession and the uniqueness of every company’s situation preclude outright resolution of problems in this space by the so-called method of “following best practices” ─ we can’t just imitate what everyone else does. Several iteration rounds may pass before norms start to take hold, if they ever do, causing companies to find solutions to their problems de novo and in situ. That’s all good, but who will lead the search for solutions as the scramble to outcompete continues?

Companies that are looking for visionary DS managers are already turning over the first of many stones on the problem trail. There is an industry-wide shortage of DS managers. We felt this shortage acutely at Lyft where, due to rapid growth, the DS IC headcount more than quintupled between mid-2015 and late-2017 while the manager count remained stationary, resulting in a staggering average IC:manager ratio of 15:1! In practical terms, managers, not due to any fault of their own, were spread too thin. Weekly one-on-one meetings felt like extended standups; there were hardly any career discussions or time for two-way feedback. If you asked a manager (or even an IC), what they thought was the purpose of manager:IC relationships, I’m not sure you would be satisfied with the spectrum of answers. If you think about it, could a relationship ever be wholesome until all parties have reached a higher-level understanding of its purpose? Lyft has come a long way in how we go about management now. With more eyes than ever on the details, several committed people are driving rapid improvements in our processes. To say the least, we are increasingly more bottoms-up and geared to accelerate IC-growth.

As the leaders at Lyft brought an increased focus on filling DS manager positions, my resolve to transition to the management side was crystallizing. I didn’t have a great measure of confidence, so I started by taking little “risks” like speaking-up more frequently when I saw an issue and offering to help my manager whenever I noted he was getting overwhelmed. With the passage of time, I became increasingly self-trusting, thanks to the positive reinforcement I got from my peers. They listened to my views and they started encouraging me to consider the path to management! My interest in management had previously taken me to IBM, then remained dormant, so it was rejuvenating when it came right back. In an exceedingly supportive environment, I eventually transitioned to the management pathway at Lyft in July 2018. Here I share some insights from my transition journey with an intent to help companies looking to propel their business savvy ICs into management roles. It is also my hope that ICs interested in management will read and reflect on the pieces they need to put together as they chart their own path.

Growing from a Data Scientist IC into a Data Science Manager

By late 2017, when I was sure that I wanted to transition to management, I made my intentions known to my manager. He was supportive and identified two important steps prescribed in the transition protocol at Lyft.

First, get a promotion as an IC so I could come to a level where I would be eligible to transition to manager. Getting promoted entailed demonstrating my abilities at the next level and summarizing my impact in a compelling promotion application. Having already been unsuccessful at getting promoted in early-2017, I was not taking anything for granted ─ Lyft has strict standards. I curtailed most of my social activities and worked harder with ongoing projects to cross the finish line ahead of the early-2018 review cycle. Since promotion decisions are made behind closed doors, I remained anxious until the news became official.

Second, go through a management fitness assessment by entering a 10 week trial period as the acting manager of two direct reports. The company finalized its decision to move me into management based on the “strongly recommended” rating my evaluators gave me at the end. I spent a considerable amount of time working out how I would perform during the assessment period. My process is summarized below.

Preparing for the assessment period in the data science manager career path

I was fortunate to have a thoughtful, disciplined, and goal-oriented external mentor whom I met through Everwise, a subscription mentoring platform. I leaned on him to gain an understanding of what steps I should take to improve my chances of finding success in my transition to management. Following his recommendations, I read blogs he pointed me to, I wrote down my personal strengths and development areas, and reflected a great deal on my past experiences of being managed. I also interviewed two science managers at Lyft about their perspectives on what they were looking for in other managers. Before entering the assessment period, I distilled my understanding by identifying four areas that manager candidates should demonstrate strengths in: mentorship, interpersonal relationships, communication skills, and business skills. Although I am not aware of formalized fitness assessment metrics used at Lyft, it seems reasonable that evaluators would be looking at these four areas.

Mentoring skills

I owe a lot in my life to the great mentors I’ve had. At the same time, I’ve also experienced being on the receiving side of poor mentorship --- both, in academia and in industry. My standard of personal excellence vehemently rejects any compromise on this critically important matter. Early managers will likely have the responsibility of junior ICs as direct reports. For effective mentoring of junior ICs, DS managers must not only possess impactful IC skills at a level well above foundational, but also the ability to coach and groom raw technical talent. I was fortunate that in the three years of working as an IC at Lyft, I mentored two Ph.D. students as interns, both of whom had a strong impact on their projects. My interns gave great feedback about me to my manager which infused him with confidence to encourage me to pursue the managerial path.

Excellence in managing interpersonal relationships

Numerous opportunities allowed me a close study of how my own morale and that of my teammates were affected due to actions resulting from the prevailing perspectives on manager-IC relationships. Through readings and reflections in my personal time, I came to distill the following understanding which I elaborated in a document that I shared within Lyft: To excel in management, individuals need to continuously develop the emotional intelligence that comes with an increasing level of self-awareness. At least four kinds of relationships in the workplace come into the spotlight for managers: An empowerment-centered relationship with direct reports, a dependability-garnering relationship with leadership and other DS managers, a collaboration-fostering relationship with cross-functional managers, and most importantly, a strong self-relationship where any residual authoritarian tendencies are constantly negated. Purposeful working relationships can only be nurtured with clarified intentions.

Communicating effectively with the technical and non-technical audience

Data science literature is rife with subtle terminology encompassing nuances in sundry experimentation and modeling methods, algorithm design, and metrics and their implications. Oftentimes data science managers have to discuss their choices and work with non-scientists (product and engineering) so that meaningful tasks, realistic expectations, and roadmaps and sprint plans can be generated. In addition to communication skills, the ability to inspire a genuine interest in understanding different viewpoints is critical in this function. Once again, I was fortunate that as often the only DS IC on a project team, I had regular and productive interactions with some of the most patient and outstanding product and engineering managers in the industry. It is my understanding that these colleagues also supported my candidacy for management.

Seeing the big picture, as it relates to business

Beyond performance evaluations, promotion decisions, and compensation calibrations, data science managers attend various planning meetings wherein business strategies or roadmaps aligned to them are generated. Naturally, business-savvy DS ICs will be more engaged in such meetings and can have a stronger impact on a roadmap that they will help craft. Thanks to my long-standing interest in business, I had taken more than a handful of courses at Cornell’s business school along the way to earning my Ph.D. in engineering. These courses helped me gain familiarity in common business frameworks and executive communication styles. My astute manager might have taken notice because a few times he had me attend important planning meetings on his behalf.

Grooming Interested ICs for Management

Either managers will identify the ICs they see with management potential or interested ICs will step forward on their own once the company defines the skill requirement and the transition criteria. Grooming can be a long process, but it must reach a place of confidence for a high chance of success in an assessment period. As each individual starts at a different place in their maturation, managers, mentors, and ICs may want to co-create a personalized plan for skill development. For this to be possible in a systematic way, managers may have to purposefully create the environment or perturb existing processes to allow for grooming. My manager started asking me to own low-risk projects that provided me with some of the experience I now need as a manager. For example, I led the recruiting and staffing of our interns two years in a row and I was the ramp-up partner to some new hires.

Lyft has significantly diversified since I started. Newer possibilities to engage budding managers are being formalized. In particular, two of the more recent initiatives we’re investing in, learning and development, and diversity and inclusion programs, offer opportunities for ICs to become well-rounded in company matters, whether or not they intend to pursue the management path.

Once an individual’s candidacy is sufficiently established and the mutual interest in pursuing the transition to management is crystallizing, managers may consider pairing these candidates with current managers for mentorship. My external mentor was a critical resource in helping me focus on the right things at the appropriate time. Additionally, my manager connected me with a kind software engineering director who generously gave me valuable guidance and shared candid insights on management processes within Lyft. One can imagine how internal mentors can not only guide the mentees but also provide a pulse-check to the company’s leadership about how far along their mentees are in their personal development plan.

Concluding Remarks

As every company’s and every IC’s stories and journeys are unique, I surely do not expect the above reflections to apply out of the box. Nonetheless, they should stimulate conversation and movement to help create a healthy data science management environment. Data science managers who were once ICs at the same company can undeniably provide the impetus to implement improvements in areas that they had felt lacking. They could also prove to be vital collaborators of managers hired from other companies who bring fresh perspectives and new ideas to the table.