The University of São Paulo (USP) is Latin America’s largest university and Brazil’s most prestigious academic institution. It produces more doctorate degrees annually than any other university in the world and ranks fifth in the number of scientific articles published. It is one of the world’s leading research institutions, with 100,000 students, 6,000 professors and 17,000 employees spread over 11 campuses.
USP successfully built a private cloud environment (Cloud USP) that would help them overcome the challenges of having 150 disparate IT environments and the ever-increasing demands for compute and storage in their dynamic research environment.
USP saw the opportunity to create a federated “cloud of clouds” across all of Brazil’s leading academic institutions. The ability of these institutions to share computing resources would drive collaboration and shared research across those organisations.
Needed a platform that could rapidly scale
The first iteration of Cloud USP consolidated the University’s 150 data centres down to 6 and brought all of its corporate, educational and research environments together into a single private cloud environment with user self-service and pooling of resources. This initial project was hugely successful and allowed the university to cut physical storage footprint by 90% (despite data growth in excess of 300%) and greatly increase their IT operational efficiency.
Cloud USP has been so successful that it has resulted in a need for the platform to rapidly scale over the next few years as Cyrano Rizzo, Office of IT at USP explained. “Cloud USP is already a massive environment, but demand from our departments’ mean that we needed to scale the compute infrastructure by 300% over the coming year. We, therefore, had to plan the future technology for the platform carefully and make sure that we chose something that was both proven, scalable and gave us the agility we needed to cope with future requirements.”
Moving to an open-source platform
USP had used a vendor distribution of Apache CloudStack, but that presented challenges both in terms of the required features and cost-at-scale for the upcoming project. Whereas, moving to an open-source platform would enable USP to quickly develop any new functionality they require. It would also provide the added advantage of allowing the University to be directly involved with the core development and maintenance of the technology. This would ultimately free them from commercially driven roadmaps and allow them to focus on what they really need for their rapidly evolving environment.
Apache CloudStack was the natural choice to meet these needs, as it was the basis for their existing vendor technology. This would mean that USP would not have undergone a steep learning curve in order to migrate as they were already actively participating in the Apache CloudStack community.
Cyrano explained, “Moving to open source is not primarily about the cost for us. It is about our ability to directly contribute the features we need to the platform. CloudStack, as an Apache project, is a very mature and well-governed open source community. We like very much that it is a project driven by its users and not by software vendors. A vendor-driven development approach simply does not give us the agility we need.”
With the new open-source infrastructure now in place, USP has been able to identify the benefits of the change, as Cyrano explained. “We are now manufacturer-independent which is a major plus as we have the autonomy we want to be able to freely customise and improve the software.”
New features to allow cloud federation
The most important new features developed by ShapeBlue enhanced the security model of CloudStack to allow USP to provide a seamless user experience for users across different institutions. This means that users can provision infrastructure resources in the cloud environments of any participating organisation. As these features have been contributed into the open source Apache CloudStack project, it also allows new institutions to easily join this federation.
“I believe that the federation will be a great thing,” said Cyrano. “Now it’s possible to interact between autonomous private cloud computing environments without the need to authenticate again. It also enables collaborative research between different academic institutes while maintaining the researcher identity in their own university. This means the researcher needs just one identity to authenticate in any cloud environment of the universities that participate in the RNP CAFe. In the future, we will be able to transfer virtual resources such as instances, templates, disks etc from one cloud to another and integrate networks.”
Other supporting features were developed such as integrating a PaaS offering (based on open source Tsuru), the networking functionality of CloudStack was also extended for the University and a mechanism to track the financial usage of their cloud infrastructure developed.
Working with ShapeBlue
In order to realise their vision, USP decided that they needed a partner who could provide professional expertise and experience. After a public tender process, ShapeBlue were chosen.
Cyrano said. “We selected ShapeBlue to work with us on this project because we were extremely impressed with their knowledge, professionalism and their widespread experience of implementing environments like ours. There is no other company who have.”
ShapeBlue consultants were able to carry out the migration to Apache CloudStack in 3 months but were also heavily involved in a number of other important tasks as Marco Sinhoreli, Managing Consultant of ShapeBlue Brazil, explained. “We helped develop an additional feature set for the USP cloud environment. As well as greatly increasing the scale of the environment.
USP and ShapeBlue have been working on many customisations and improvements. For example, routing VPC to enable the capability of routing the tiers end-to-end as well as quota services that permits limit tenants by currency, all of them open-source and authorised by Apache CloudStack.
Commenting on working with ShapeBlue, Cyrano concluded. “They are a great partner and, like me, are passionate about Apache CloudStack. I cannot stress enough the importance of the open-source community and the key role that ShapeBlue play in that. They have such in-depth knowledge of the product coupled with the experience of working with it in the real world. Without that we would not have been able to upgrade, what is one the largest private cloud environments in the world, so quickly.
“I would highly recommend ShapeBlue to anyone working on Apache CloudStack. Their reputation within the community is second to none and they are doing some amazing things in the cloud computing sector.”
Giles is CEO and founder of ShapeBlue and is responsible for overall company strategy, strategic relationships, finance and sales.
He is also a committer and PMC member of the Apache CloudStack project, and Chairman of the European Cloudstack User Group, actively helping promote brand awareness of the technology.
