Arbitrage Betting Scrapers on Puppeteer
Arbitrage Betting Scrapers on Puppeteer
2019-02-12 11:23:00
Arbitrage Betting Scrapers on Puppeteer

We had been long wondering about the sustainability of arbitrage betting. If this was a thing and how this could be automated and scaled. Thats why we gave it a go! For those who don’t yet know what arbitrage is, it is the way to place bets and win no matter the outcome. That is why it is also called sure betting.

NOTE! Many betting websites allow arbitrage betting others don’t. The same applies for web scrapping. Please honor each website’s terms of use. This experiment was performed for the sole purpose of technical expertise on the specific tools and coding.

We built an automated scraping system that crawls multiple online betting sites. It then gathers the data to calculate possible arbitrage. The end goal was to asynchronously notify the user to take action.

Single website scraper

First we built a web scraper with Puppeteer. With a few lines of code you can have it visiting a page and interacting with it in all ways that a real user could. The majority of the work required is coming up with the right CSS selectors for the elements you ll need to read or interact with.

Automating for multiple websites

After getting this working for one site, we made it working for multiple. We located the code referring to this site’s specifics (e.g. CSS selectors, urls, clicking on “accept cookies” etc) and we refactored it to be dynamically created from a config file. Now all we had to do to get betting data from other websites was to create just the configs for them! Creating the configs was a matter of visiting the page and finding out the selectors to use.

Arbitrage calculation

Up to now we have a puppeteer application that spawns multiple browsers to fetch betting data from multiple websites. The next step is to calculate the arbitrages, if any. We persist the data in a database and created a script to process those. There’s a lot trickyness to this but in the end we made it working and found our first arbitrage!

Possible expansions

  • Automate account management to all betting sites
  • Automate bet placements coordinated from the arbitrage algorithm
  • UI to enter the maximum and minimum limits for betting as well as betting frequency
  • UI to add/remove websites without coding
  • UI to add/remove more sports per website

Other posts

WordPress CI / CD in Kubernetes using Flux

WordPress CI / CD in Kubernetes using Flux

Integrating WordPress in a Kubernetes Flux CI/CD pipeline.
Kubernetes Cluster & CI/CD made easy

Kubernetes Cluster & CI/CD made easy

We created ESM, a project that creates production ready Kubernetes clusters with built-in CI/CD, TLS issuance, monitoring and more in…
Our go-to Frontend stack

Our go-to Frontend stack

The most efficient and modern frontend technologies to work with.