Working with remote teams – a long-term perspective

I started the Qubole office in Bangalore in Jan 2012, roughly at the same time the company itself started, and ran it until Oct 2020. The India team eventually grew to ~200 out of a worldwide team of ~320. The majority of R&D, half of Technical Support, and later parts of Marketing / Customer Success were based out of India. HQ was based out of the Bay Area throughout this time period (and housed most of S&M, G&A, and parts of the above teams).

• 

• 

While offshored development centers, particularly in India and East Europe, abound – a few things stand out about our/my experience:

• the remote office was established at the inception
• the work being delegated was core product development in a complex technology area
• the longevity of my personal experience in this role

The topic of how best to manage such a distributed setup, what we learned, and best practices are a common one that comes up for discussions with others exploring a remote office. I am putting my observations down in this post for future reference. (Twitter thread at https://bit.ly/2PXs62v, LinkedIn at https://bit.ly/3tpnn8o for discussion)

Trust

The #1 thing that comes to mind when thinking of a distributed setup is the level of trust across the teams. In our case, we started on a great footing – with the two founders (with a long working relationship) spread across the two offices. Others may not be as fortunate – and there’s nothing I feel that’s more important than the choice of a remote site-head. At the end of the day – all conflicts and issues can be resolved if there is trust (and empathy – a slightly different topic).

On this matter, we were fortunate enough to get some early advice to over-invest in employee travel across the two offices to get physical face-time. This was invaluable. The impact of this investment during the early days of our journey was particularly high – since early employees tend to take up important roles in the company as it grows. If the leaders are well connected and have a high trust relationship – the teams under them tend to as well. While we could not keep up the same level of travel intensity (per employee) as we scaled, the rapport across the core team established during the early days was a massive help.

A generalization of the advice I had received is to (at least) over-invest in travel for those in leadership and coordination roles. IMO – spreading such travel out so there’s a constant exchange of people should be a higher priority than expensive global gatherings. While the latter has its own use – the former is just better use of $$ when budgets are limited. Finally, transplanting people was a super-effective way of creating more trust. We were able to relocate some engineers from India to the US (some even in leadership roles) and new offices in Singapore/UK were started by veteran Sales folks from our India/US offices. These created a tightly knit global organization.

I will end this section with my favorite example of the unexpected rapport that sometimes results in the team from international travel – in this case, our then VP of Sales (Marcy) and senior QA engineer (Monika) hanging out in Palo Alto and Bangalore:

• 

• 

Empathy

Empathy is different from trust and it took me a while to wake up to this one. I still find it easiest to illustrate through examples:

• There’s a production issue in the US timezone and the field teams there are having a terrible day taking fire from customers. The India engineering team goes through the motions of on-call and responding to the issue – but does not quite get the impact of what happened. Soon enough direct messages start coming in – “the engineers don’t seem to care”.
• There’s a hot feature ask. It seems reasonable, but un-exceptional. The India engineering team can’t quite figure out why people are so worked up about it (even after Dollar numbers are quoted).
• The India engineering team is totally overloaded with tasks and operational stuff – many are awake well into the night on a regular basis. Targeted features inevitably slip. The US teams are distraught when this happens (“Why didn’t we see this coming? What will we say to customers? ..”) and develop a negative impression of the manager/team (“cannot deliver”).

My overall take here was that there is little substitute to direct visibility into people’s lives. Watching someone deal with an irate customer, seeing an insipid customer wake up suddenly when a new feature discussion starts, realizing people are responding regularly to escalations at 2AM – these observations have an effect on the receiver. When individuals differ in their observations, they differ in their reactions and behaviors. An engineer facing an irate customer would figure out the quickest patch that can go to production safely. An engineer getting a Jira ticket assigned, even the highest priority one, may approach things differently.

We never found a silver bullet solution to this issue. Having people cross-play roles (Engineers in Support stint for example), exposing ICs to stakeholders from other teams, lots of communication from the leadership team, cultural values espousing customer and team empathy, and of course travel – all these helped make the situation manageable (but not perfect).

Anecdotes

• Scheduling Cross-Geo Meetings: An interesting aside on the topic of empathy is the seemingly trivial matter of scheduling cross-continent team meetings. With the majority of the engineering team in India – the default schedules for such meetings often became lop-sided in being more friendly to IST hours. This was perceived as unfair (feedback we received early in our lifecycle) – and periodic meetings were re-scheduled to alternate (in some ratio) between IST and PST daytime hours.
• Promotions: Another place where differences in perspectives manifested themselves was during performance reviews and promotion discussions. It was not rare to find entirely divergent views of the same person on the two continents. A person maybe effective in their jobs – but weak in their communication and coordination with remote stakeholders (and sometimes the reverse as well!). The latter has a disproportionately large impact in a distributed setup – something that can catch people by surprise.
• Code-Reviews: The quality of life for developers is often affected a lot by the speed at which they can get feedback on and closeout pull-requests. In a distributed setup – unblocking remote team members waiting on such requests at a higher priority becomes an important practice.

On-Site Management and HR coverage

One thing we settled on fairly early was to try to have an on-site manager for every employee. In cases where the reporting was necessarily remote – someone on-site would play a dotted line manager. The on-site manager would cover the people management role in particular. There were multiple reasons for this:

• Employees would often find it easier to raise sensitive issues to a (on-site) manager who they interacted with in person
• The on-site manager would likely have a lesser cultural gap
• On compensation issues, the on-site manager would have a better grasp of the local job market and be able to make better judgment calls
• The on-site manager would also better plugged into the grapevine and spot people issues, proactively, earlier. Sudden changes in office attendance timings, manner of speech, and so on may be undetectable remotely – but easy to sight in person.
• Finally – the on-site manager would often be in a good place to help in hiring and planning any team expansion.

If the on-site manager was a dotted line, the solid-line (remote) manager would work with them from a people management perspective – particularly during performance reviews. In much the same way, having designated HR personnel for each employee from an engagement perspective – ideally located in the same geo if possible – helps a lot in snuffing out issues early (and v.v.).

Tango Structure

A common reporting structure we arrived at over time, that worked really well for us, is illustrated below. The Manager on one continent would be managing a global (product) team – with remote team members having a dotted line to the Director (who is located on the other continent). I am christening it as the Tango structure for reference purposes:

Getting it Wrong

It was also my observation that teams that did not implement an on-site management structure often performed worse:

• Their team members were sometimes directionless. Some remote managers weren’t able to provide sufficient direction – and worse – didn’t realize that their reports lacked direction. Team members felt inhibited in giving direct feedback to their managers (with whom they may even not get sufficient face-time)
• In almost all cases, the lack of an on-site manager inhibited additional hiring – even when the same would have benefited the business. This is natural. It is difficult for a manager to hire remotely without a trusted partner.
• Remote teams without on-site managers would often get compensation wrong – and then be fire-fighting when there was the inevitable blowback.
• Similarly, on-site managers may be able to suggest strategies for implementing business objectives that may not be obvious at HQ.

Case for a Country Manager

For the above reasons, I also came to a conclusion that for larger remote teams (say 200+) – a person with an explicit responsibility to lead the remote office in totality is important. Aside from a strong focus and authority on people-related issues highlighted above, the country manager can serve as a trusted sounding board for the executive team and participating in planning organizational capacity at a global level, and a peer for remote managers in helping them grow and manage their teams in a remote office.

Apples are not Oranges

One of my most frustrating experiences (even as a Founder) was to fight back against rules that did not apply well across geographies. The biggest one of these was managing budgets to a Headcount, instead of to a $$ Budget. It seems that while financial plans are all about $$ – financial models will often target headcounts as a proxy. But trying to apply this to both the US and India uniformly results in a total breakdown:

• The min/max cash compensation (not including stock comp. and bonuses) in a Bay Area startup may at worst be 1:4
• The min/max cash compensation in India can easily be 1:100. In particular – compensations start off low for college graduates and increase rapidly with experience.

Managers in India can therefore choose to substantially increase low cost staff while only moderately decreasing higher cost ones within the same $$ envelope (and v.v). Some of these adjustments can be made dynamically depending on what talent is available and the work queue. Managing to a fixed headcount plan becomes destructive in these situations.

Even when it comes to managing global departments like Engineering or Marketing – the availability of low-cost labor leads to a different set of organizational design points than in the US. In Engineering teams – the amount of manual testing and whether Operations should be done by specialist staff (as opposed to engineers) – are questions whose answers are intertwined with underlying labor costs.

Part of the case for a Country Manager is to bridge this divide. But even at much smaller sizes – organizations should empower managers in remote locations to execute their plans and goals with full freedom – with the only constraints being core principles and values and $$ budget numbers.

Distributing Work and Career Paths

Amongst the most common tradeoffs teams face is how to split work across remote offices. The standard wisdom is to assign projects so they are localized to one of the offices. A related problem, but one that is not obvious at the beginning, is thinking about career growth for each individual (which inevitably means being able to offer leadership opportunities at some point).

We often found the project-based assignment wisdom insufficient. It does not sufficiently account for the kind of work people did (often ad-hoc) and some projects (particularly the high profile ones) need larger teams. Such projects also benefit from participation across offices – spreading knowledge of the project across the teams and providing localized reference points for stakeholder teams. Instead, we found a more base set of axioms useful over time:

• Ensure each team has a minimum-sized sub-team in each Geo (or time-zone) they are staffed out of. The right number for this is probably (at least) 3.
• That the minimum-sized sub-team has at least one senior member.
• That the people in each sub-team leverage each other for collaborative work like code & design reviews, pair-coding and testing, operational coverage, and so on – to the extent possible.
• That career growth ladders offering team leadership opportunities be thought ahead using the Tango structure as a potential template.

Project/Team level synchronization necessarily requires periodic global discussions – and that is manageable. But it becomes unworkable to have people wait for 12 hours for feedback on the smallest of issues or not having a clear career growth plan.

Impact on Product teams

In a smaller organization – Inbound and Outbound product roles are often combined (and together make for a more interesting role). One of the challenges with remote engineering teams though is that they can be remote from the bulk of customers as well. Both close interaction with customers (and non-customers!) and real-time interactions with engineers as they iterate on the product are invaluable and indispensable. Short of insane working hours – the only sensible way to achieve these goals is to have a specialization of inbound/outbound roles in a setup like Qubole’s. This does not have to be a hard split – but there has to be an understanding and arrangement that PMs close to engineers spend more time building and iterating on the product and those close to customers spend more time on user/market research (and areas like product analytics can be shared or owned on either end).

Common Knowledge

Towards the end of my stint – I increasingly became obsessed with the notion of Common Knowledge (of the Blye-Eyed fame) and saw it as fundamental to running a well harmonized distributed setup. A simplistic way of describing the problem is that

Employees in remote offices do not run into each other in the coffee room.

As a result – many things that are common knowledge in a single office (spreading via the grapevine, including across departments) – are not common knowledge in a distributed setup. The problem of Empathy described earlier in this post is just a specialized case of a more general problem. Lack of common knowledge is the root cause of much friction, missed expectations, lack of trust, and so on. (These problems are, of course, not limited to distributed teams. They just run into it faster).

In the absence of informal channels of information dissemination – organizations must design formal ones. This can be counter-intuitive when companies are at a small size (and any formal process is seen as a sign of early bureaucratic onset) – but it is vital in distributed setups. While it is not possible to capture the entire gamut of strategies to tackle this in a single post, but some of the top ones that pop out:

• The Communication Trident: I ended up asking for important communication to be sent via all the 3 mechanisms below. Missing out on any of these inevitably leads to missed communication and expectations.
  
  • Real-time notifications (Email/Slack)
  • Long-term reference (wiki or shared doc discoverable via a portal/search)
  • In-person meetings with those that must be informed (repeated to accommodate different time zones if required)
    
• Public Rewards & Recognition to reinforce core values: This is something we mostly got right from an early stage – with quarterly and spot awards that cited core values associated with the recognition. (And those awards and their citations were widely communicated using the Trident strategy above).
  
• Process and Transparency in important decisions of global interest. In my domain – the most important of these was the product and engineering roadmap – its content, dates, as well as how it was arrived at. By 2019 – we had a periodic calendar of activities around each release cycle – starting with company strategic priorities, release planning, execution, and post-delivery retrospectives with appropriate stakeholders pulled in at each stage (and all deliberations and results captured and broadcast as per the Trident above). I wish we had the wisdom to have implemented it much sooner. A fuller description of this process needs a separate post.
  
  These considerations also apply at individual team levels. It was not uncommon to see design presentations in a team meeting being presented as a “done deal”. Remote team members are taken by surprise – when did the design go out for review? They were never part of the loop. Meanwhile, the other half of the team thinks everyone already had an opportunity to chime in (having often done some in-person brainstorming). Having a clear process for such group activities and over-communicating are important to avoid snafus like this.
  
• Some distributed teams reported significantly better alignment after implementing daily or high-frequency standup meetings (at a mutually convenient time).

Some other strategies were captured in the section on Empathy. We also, like most firms, had quarterly all-hands meetings, periodic senior management updates around key OKRs, etc. In spite of all these investments – achieving higher Common Knowledge was a work in progress. In one of my last discussions on this topic, one of our engineers who moved from India to the US recently, told me the following (paraphrased):

Perspective changes being located in the US (vs. India). All hands were not sufficient. Direct interaction with sales team members helps. In the US – we can just go for coffee with Sales Engineers. We can have more white-space discussion in such settings rather than just around Jira tickets only.

Karthik Borkar

Stress

Coordinating across a distributed team adds to meetings and working time in late and wee hours. While this is not problematic over a short time period, it does add up over longer ones – particularly for senior management and those in coordination roles. It is better to accept this reality than to believe that there is a free lunch here. More subtly, the impact may not show up in actual stress (say, frazzled people) – but opportunity cost. People regularly on early morning and late-night calls with a full day job in between may simply have less mental space left to experiment, explore and innovate. (concurrently reading Charles Duhigg’s Power of Habit – the concept of willpower as a finite resource seems to have a strong overlap with this issue).

Coordination overhead is not unique to a distributed setup – but meetings within the same timezone create dramatically lower stress than across timezones. There is also a bit of a zero-sum game here – while we want to have more face-to-face time across continents to achieve more trust, empathy, and information flow, but that time has an opportunity cost and adds to stress.

I will leave this section with some key takeaways:

1. Distributed setups need to pay attention to cross-continent meeting overload earlier in the company lifecycle (it can quietly sneak up) and design processes to avoid it.
2. It is particularly important to spare those whose job roles or talents require deep-work from coordination/communication overload.
3. Running efficient meetings is always important – but is particularly important for remote meetings being scheduled in early/late hours whose availability is at a premium. Lots of teams get this wrong. (Running efficient meetings is well-covered territory on the internet and best to skip details of here)

Miscellaneous

Its time to wrap up this already oversized post, but it would be remiss to not mention some other issues:

• Diversity: There is a definite impact of an offshore development center on hiring diversity in the US. Many people self-select themselves out of such a setup. The distribution of such folks across different population segments is not uniform.
  
• Local Reference Points: In spite of many attempts to formalize processes and over-communicate – stakeholders just loved having a local person they could go talk to for instant updates and information. We saw this particularly in the case of release/product manager roles – but also for key engineering and operations staff.
  
• Geographical bias in Customer Feedback: Nature of customer use cases, their expectations in terms of support and pricing, competitive landscape – all these tend to vary a lot by geography. In our case – customers in Bay Area, the larger North American market, Europe, and India/APAC – were all somewhat different. With the bulk of the engineering team based in India though (and often interacting with APAC customers directly) – I felt we were at risk of giving higher weightage to the requirements of customers in this region (whereas N.A was clearly our biggest market).

In Conclusion

Like most other things in life and technology – remote teams are relatively simple to start with – but complexity arrives over time with scaling and longevity. We are headed, like it or not, towards a world even more distributed than the one analyzed in this post. Not 2 or 3 offices – but developers scattered over all over the world. They will have issues similar to the ones described here (and sometimes much worse – try scheduling meetings with participants all the way from Pacific to Singapore time). We will inevitably also adapt to this brave new world. I hope the insights shared here prevent some common stumbles and help remote teams be better prepared for the journey ahead.

Post-Script

I find it easier to remember things that went wrong than what went well. Second – I don’t know what I don’t know. A few more years would have taught a few more lessons for sure. Finally, every company is different – in the level of culture gap and the experience level of the team for example. We built an apolitical and transparent culture at Qubole (that made many things easier) – but were significantly understaffed and underskilled in coordination roles (that made many things worse). YMMV.