Transition from product to infra: first month at ShapeBlue

, , ,

Anurag Awasthi shares his experiences after one month in the ShapeBlue Engineering team

Let me start by briefly introducing myself. I hold a bachelors and master’s in computer science and engineering from the Indian Institute of Technology, Kanpur. I have been working in technology for more than 5 years now and have had the privilege of working with some world class scientists and engineers at various organizations. Some of my past employers are Microsoft Research Redmond, Pocketgems Inc, and Twitter Inc. During this time, I have strived to be a generalist and explore full stack development in a consumer-focused product development. In my free time I enjoy going on hikes, camping, painting and I experiment cooking different cuisines.

In late 2017, after spending 5 years in a heavy consumer focused product development, I wanted to expand my horizons, so I worked on my own startup to explore the world of cloud computing. It worked with moderate success for some time before I ran into an old friend of mine who had now become a PMC member in Apache CloudStack. That’s when I first heard of ShapeBlue – a company which was working very actively and contributing directly to the open source community of Apache CloudStack. The company works in distributed teams and is working with some big names in the industry as its customers. It sounded a little too good to be true. I had an interview with the company, and it became obvious that the subtitle of the company “the Cloud Specialists” wasn’t just a sales pitch but accurately reflected the true horsepower of expertise that the company carried. I saw incredible opportunities to grow technically and peers to learn from, and I joined within 4 days of the interview!

Having spent a little more than a month at the company I think I have reached a place where I can provide a comparison between a company focusing on opensource community development vs. a big name private company focusing on proprietary software. I will also try to highlight some of the pros and cons of working full time in a distributed company such as ShapeBlue. Two primary things that any engineering focused person would be concerned about are the development process (challenges, rewards, etc) and team environment (interactions with peers, support and expectations, etc).

Development Process

Overall, there are many similarities between the development process in an open source project and a traditional organization. A feature begins with a feature request that needs to be presented in a specific format, followed by discussions on proposed solutions, implementation, code reviewer approval and merge. The difference lies in the challenges that arise due to challenges with communication between collaborating parties, and time consumed because of a globally distributed team.

Technologies involved in an open source project are truly diverse, and multiple choices are available at each stage of a feature’s development. Naturally, each choice comes with its own challenges as one is responsible for the due-diligence work. ShapeBlue has its own CI test environment and test engineers, but the community as a whole does not have a fixed approach to testing. As one is directly contributing to the community project, the necessity of one’s own due diligence and initiative cannot be emphasized enough. Even for a simple task such as machine setup, one needs to explore options and wisely set up a preferred system for development. This is unlike a larger organization, which mostly would have a pre-tested and often pre-configured system ready for development, and managed by a separate IT department. This becomes a fun experience on its own when one jumps into development of an open source project that touches a multitude of system components like CloudStack. I started on a MacBook, but very soon switched to ThinkPad / Ubuntu because of the better community support available on the latter platform and the limit of RAM on Macbooks.

The Apache Way

Each Apache project community has its own set of principles and are determined by the Apache way! Since it’s not just managers controlling the flow of a project, feature development is usually slower than private development too. Each discussion requires genuine intentions and effort to be heard, and the necessary feedback from the community. This can be frustrating for the impatient ones, but following the discussions in turn brings in new learnings. It also means that any feature gets reviewed by a broad set of people. So, it has its advantages.

From Consumer to Infra

The true challenge lies when one transitions from a consumer product to an infra product, such as CloudStack. As one can imagine, orchestrating a truly scalable cloud deployment is a massive task that CloudStack does. Clearly, it is expected to be more challenging to be a true cloud specialist as compared to being a mobile engineering specialist, or a web specialist, and so on. In just my first month I have had to revisit most of my university learning and learn many new network concepts, linux administration and refer to some new design patterns in code architecture. Such large exposure is rare to most developers in larger, private organizations because they are abstracted in modern, product focused development. Now whether this is good or bad is a difficult call to make as it narrows down to one’s preferences. Personally, it’s been a rewarding experience, but I’ve had mentorship support from within ShapeBlue. It would definitely be a slightly harder and much slower transition otherwise, and would require patience.

Team Environment

An open source project is determined by its contributors and does not have a specific team that regularly goes on outings or hangs out at office parties. Most contributors don’t share an office location and often work in different time zones. Given most of us actively participate socially as well as professionally with our team members in larger organizations, I suppose that’s a disadvantage as it means social life with colleagues after work is negligible. ShapeBlue similar in structure and we have team members distributed across many time zones. The communication happens primarily over Slack, but is not a hurdle as much as in wider community as team members are one ping away. So that solves most of the problems of working in a distributed team. There is also a unique culture in the company that leaves the responsibility for being productive on the employee and provides good support to do so. If someone has worked in a controlled environment with a vertical hierarchy, it’s a little difficult to get accustomed to a flat structure in the beginning. But new perspectives open up in this environment which include thinking more creatively, independently, and having a better work life balance.

Hopefully this would help anyone thinking of transitioning from a monotonous iOS / Android / Web / Backend development in a larger company to a more technically enriching project such as CloudStack. On personal level, the move has been full of several new delightful experiences with some new challenges that don’t really pay back directly in skills in the short run but pile up benefits in the long run.