Introduction
bet365’s Bet Builder is a feature that allows Customers to construct their own personalised pre-match or In-Play Soccer bet. For example:
- Luis Suarez to score the first goal
- Uruguay to win 2 – 0
- Over 8 corners in the 2nd half
Previously this bet would need to be individually priced. Typically the Customer requests a tailored bet via a social network. The bet is then priced and the Customer sent a notification that their bet is available and / or a link to allow them to place their bet.
What made this different from the majority of our competitors is that it works in real-time. A Customer can construct a tailored bet on site with an instant price and place that bet there and then.
This article discusses some of the technical decisions that allowed us to deliver this functionality.
From concept to system
Bet Builder was conceived by the Sports Analysis team. This team is responsible for the design and implementation of the sportsbook trading models, and typically do so using the .NET stack. Bet Builder was no exception to this and a proof-of-concept was gradually refined until the required functional success criteria were met. Now proven, it became time to understand what would be involved in turning this PoC into a production-ready system. After spending the required time to "spin-up" on the concept, we then had to make some important decisions in order to choose the best way forward. Like most things in the I.T. field, "best" is a trade-off between multiple and, often, opposing forces. We had to balance:
- Integration with our trading and betting platforms
- Deployment infrastructure
- Time to Market
- Required performance characteristics
- Support and maintenance
bet365 has long leveraged Linux and it is it our preferred OS for highly available, highly concurrent systems. This gave us an interesting challenge: how to take a .NET software artefact and run it on Linux.
A potential avenue was .NET Core but this relatively nascent approach was rejected.
We have had considerable success with non-.NET technologies on Linux – our use of Erlang has previously been discussed and because of this we decided to implement a limited port of the proof-of-concept code into 3 different languages: Erlang, Elixir, and Go. All three of these are designed with concurrency in mind.
Given the sheer volume of transactions we deal with at bet365 it’s not too much of a leap to see why concurrency that is lightweight, simple to reason about, and safe(r) to implement is so appealing. As each port was completed it was measured to understand its performance characteristics. All three fared well but Go was the winner in terms of processing speed.
Next we considered how the product would be maintained in the future, and by whom. The logical choice was to have its development continued by the Sports Analysis team. As such, we needed to be comfortable that the language was similar to those currently used in the team. Combining its syntax with the compile-time safety afforded by it being statically typed, we felt that Go would lend itself more readily to this aim. A trivial syntax comparison example:
Visual Basic .NET
Dim all As New List(Of Integer)({1, 2, 3, 4, 5})
Dim total As Integer
For Each value As Integer In all
total += value
NextC#
List<int> all = new List<int>() {1, 2, 3, 4, 5};
int total = 0;
foreach (int value in all)
{
total += value;
}Go
all := []int32{1, 2, 3, 4, 5}
var total int32
for _, value := range all {
total += value
}Erlang
All = [1, 2, 3, 4, 5],
Total = lists:foldl(fun(N, Acc) -> Acc + N end, 0, All).Elixir
all = [1, 2, 3, 4, 5]
total = List.foldl(all, 0, fn(n, acc) -> acc + n end)Even using this contrived example you can see the similarity between the .NET and Go imperative style and syntax. Go’s support for method receivers would allow us to maintain a similar object structure to the PoC code.
Porting approach and mechanism
Having selected Go as the weapon-of-choice we then focussed on turning the partial port into a fully-fledged system. Time to Market was aggressively driving our development schedule and we were still actively developing the original proof-of-concept code. We consciously decided to adopt more of a functional programming approach to the concurrency-focussed areas of the system by reducing mutable state, and deliberately chose to stay as faithful as possible to the original proof-of-concept for the more volatile, business logic-rich areas.
These latter areas of code were more object-oriented in nature. Keeping both structural and syntactical parity would allow us to port the underlying .NET code changes with lower risk as, in almost all instances; we could compare the two implementations line-by-line for the on-going port effort.
We accepted the fact that the result would eschew Go’s idioms as we felt it was more important for us to deliver something quickly and accurately. Speaking of rapid delivery, we soon realised more benefits provided by the Go ecosystem. Fast compilation times coupled with the simple deployment of single binaries reduced our development-to-deployment times, enabling us to port code in smaller increments efficiently.
We were able to keep pace with the underlying .NET development due to the similarities in syntax; choosing one of the alternative languages would have presented us with a much larger undertaking. Judicious use of mutable state and pointers allowed us to get performance wins and also accurately reflect .NET’s argument passing mechanism.
What’s next?
Bet Builder has proven to be very successful in terms of Customer take-up, system performance, and system stability. It experiences thousands of requests per minute and typically responds in a few hundred microseconds with almost no downtime. As we extend the Bet Builder system further we plan to adopt more of Go’s idioms to benefit from decreased code verbosity and greater opportunities for concurrency.
We will also leverage our original decision to keep much of the code similar to the original proof-of-concept by more easily on boarding our current .NET developers into the Go codebase.