Open-Source at Spot – 10 Months Later, How Have We Done?
Very few trading firms, which I know of, give back to the open-source community. Until March 2014, Spot was no different from most of the trading firms out there; we took, but we did not give back. Conversely, we realized that our platform would not have been where it was without open-source. As an engineering team, we are “standing on the shoulders of giants” – not giving back felt wrong.
“If I have seen further it is only by standing on the shoulders of giants.”
So we decided to embrace the open-source community by contributing back. We shared some of our software, launched a blog, and talked openly about our approaches to a variety of difficult problems that we have faced.
Our objective was to give back, stimulate thinking and promote collaboration. Getting this effort off the ground was not easy. It required a lot of support from our leadership team, and was harder than I had first expected.
The first challenge, was what to open-source? Like most companies, we have code that we believe gives us a competitive advantage in our industry. So, how do we separate intellectual property, from the commoditized software that everyone else in our industry is busy producing?
We did something relatively unusual in the finance industry; we trusted the judgment of our engineering team. We focused on software that does not give us an edge, but is vital to our operation. Over the long haul, it is our collaborative culture, rather than a particular toolset, that provides our competitive advantage. Service orchestration and dependency management is a common problem shared amongst many firms. Subsequently, we decided to open-source Zoom.
The biggest hurdle we faced was balancing commercial demands with our open-source objective. This is where we have continued to struggle the most. Building the generic solution was really hard, and we ended up satisfying our pressing business need over a generic solution that could be easily used elsewhere.
Although as it stands, Zoom isn’t easily transferrable, we see the following as areas that would make Zoom more generic:
- Use dependency injection to abstract 3rd-party (and internal) classes with respect to event notification, data storage, and logging
- Fully automate the build process using NPM, Bower, and PIP package managers, and a task runner such as Grunt
- Create a test instance via a script (or better yet, Grunt task)
- Provide better documentation using Markdown and APIDOC
So how have we done?
On the blog front, we have had a consistent stream of posts going out, with key engineers at Spot sharing their experiences in this and other external forums. With over 19,000 unique page views on our engineering blog, it’s gratifying to know that we have had a lot of people interested in our perspective.
Overall, we have made solid progress with our open-source efforts over the last 10 months, and I am proud of what we have achieved. We have an active healthy code base out there, and a solution that we love. It has provided a tremendous amount of benefit to our firm, and we are passionate about its evolution.
Conversely, we have struggled to balance the insanely competitive demands of our industry with our original open-source objective. Zoom is not generic enough, and cannot be easily leveraged by others. From that perspective, our goal was noble – but we have faltered in our execution.
We continue to wrestle with this dynamic, as we are resource constrained, but our objective is to continue to refactor Zoom to be more “open.” However, if either our industry or the open-source community championed Zoom, we could work through these challenges together.
Many people have worked really hard at Spot to enable us to contribute back and collaborate more. Our HR and Compliance teams have helped us overcome a number of hurdles. Albert Yu has been our chief proofreader and poster on the blog.
Nick Moskwinski, Justin Lee, Jack Miner, Jose Jurado, Mike Sanzo, Paul Dapolito, and David Shrader have been the primary creative minds behind Zoom. Andrew Girin and Rezi Andoni continue to act as the Product Owners for the solution.
David Shrader, with his pioneering engineer’s perspective, has been a key driver behind our overall open-source momentum.
We are committed to continuing down this “open” road, and I am really excited to see where we are in 10 months’ time.
Questions or comments? Email us at firstname.lastname@example.org