Skip to content
  • About
  • Home

Performance Testing

Exploring Nonfunctional Testing

  • About
  • Home
  • Toggle search form

Industry Benchmarks for Performance Testing – Lessons from High Frequency Trading

Posted on 09/05/202310/06/2023 By lefty No Comments on Industry Benchmarks for Performance Testing – Lessons from High Frequency Trading

I work in energy trading. Most European power is traded through some of our software at one point. I work on automated trading so this is where my current nonfunctional interests lie. First, one of my biggest problems is that we do not really have good nonfunctional benchmarks. Or rather we do not have any at all. And then the other thing: legislation does not allow high frequency energy trading, but for us to have a competitive product, we still need to be fast, albeit not frequent.

So I am looking for people who will violently disagree with the post below, will point out all my failing, including those not in the text and who will also, kindly but angrily, add several excellent sources for nonfunctional testing metrics for energy traders in here.

Rapid Response Trading

First things first, as I just invented it, what is rapid response trading (RRT). As opposed to high frequency trading (HFT) where automated systems may execute thousands of transactions every second on FX or equity markets, energy markets are legally bound not to allow more than a couple of transactions a second. So the race is not going to be in the fast and furious trading in commodities, but the name of the game will be rapid response trading. This means that you want to respond to favourable market triggers faster than any of your competitors. While you may need to wait another 500 milliseconds for the next chance to act, you want that action to be within milli, micro or nanoseconds once the conditions are right.

To be able to figure out whether we are in the right ballpark, we need to be able to compare results, ours to those of everybody else. For energy trading, there are no public benchmarks as to how fast we ought to be. We have widely accepted numbers and metrics as to what a decent energy trading platform is supposed to perform like. This has a couple of ramifications. First, we need to establish our own. Then we ought to go out and ask our customers what they think ought to be the good numbers. Then we actually should get a largish dataset from all the major exchanges, EPEX, Nord Pool and ICE and analyse those to see how fast is fast nowadays in our neck of the woods.

So for want of better sources, I decided to look at adjacent industries. I am currently ploughing through the following documents, which provide general guidelines and governance ideas for various kinds of exchange-based trading and financial institutions from a general nonfunctional perspective.

Any others possible sources for benchmarking that I should be reading?

  • Competition among high-frequency traders, and market quality https://www.ecb.europa.eu/pub/pdf/scpwps/ecb.wp2290~b5fec3a181.en.pdf
  • EEX Market Data https://www.eex.com/en/market-data/
  • Nord Pool Data and Reports https://www.nordpoolgroup.com/en/download-center/
  • Risk-preparedness in the electricity sector https://eur-lex.europa.eu/legal-content/EN/TXT/?toc=OJ%3AL%3A2019%3A158%3ATOC&uri=uriserv%3AOJ.L_.2019.158.01.0001.01.ENG
  • Action and measures on energy prices https://energy.ec.europa.eu/topics/markets-and-consumers/action-and-measures-energy-prices_en
  • Electricity market design https://energy.ec.europa.eu/topics/markets-and-consumers/market-legislation/electricity-market-design_en
  • Paper on Cybersecurity in the Clean Energy for All Europeans Package https://www.ceer.eu/documents/104400/-/-/d70764d8-9cab-9f4a-848b-6c3a4e1bd6b0
  • Cybersecurity Policies https://digital-strategy.ec.europa.eu/en/policies/cybersecurity-policies
  • Directive on measures for a high common level of cybersecurity across the Union (NIS2 Directive) https://digital-strategy.ec.europa.eu/en/policies/nis2-directive
  • Critical infrastructure and cybersecurity https://energy.ec.europa.eu/topics/energy-security/critical-infrastructure-and-cybersecurity_en
  • EU Emissions Trading System (ETS) data viewer https://www.eea.europa.eu/data-and-maps/dashboards/emissions-trading-viewer-1
  • Gas and electricity market reports https://energy.ec.europa.eu/data-and-analysis/market-analysis_en
  • Quantifying the high-frequency trading “arms race” https://www.bis.org/publ/work955.htm
  • Guidance on cyber resilience for financial market infrastructures https://www.bis.org/cpmi/publ/d146.htm
  • Cyber-resilience: Range of practices https://www.bis.org/bcbs/publ/d454.pdf
  • and a couple more from bis
Benchmarking, Load Testing, Rapid Response Testing Tags:Benchmarking and Metrics in Performance Testing Series, Latency

Post navigation

Previous Post: Five habits of good engineers
Next Post: Ensuring Software Testers’ Impact

Related Posts

Benchmarking and Metrics in Performance Testing Series Availability Testing
Benchmarking and Metrics in Performance Testing Series Part 2 Benchmarking
Benchmarking und Metriken Benchmarking

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • Benchmarking and Metrics in Performance Testing Series Part 2
  • The Importance of Stakeholder Management in Pre-Launch Performance Testing
  • Ensuring Software Testers’ Impact
  • Industry Benchmarks for Performance Testing – Lessons from High Frequency Trading
  • Five habits of good engineers

Archives

  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023

Categories

  • Availability Testing
  • Benchmarking
  • Business Performance Testing
  • Load Testing
  • Performance Testing
  • Performance Testing Methodology
  • Rapid Response Testing
  • Robustness Testing
  • Stakeholder Management
  • Technical Performance Testing
  • Volume Testing

Copyright © 2026 Performance Testing.

Powered by PressBook Masonry Dark