Evolutionary search for robust trading strategies

chevron-icon
Back
project-presentation-img
Robert Haas
Project Owner

Evolutionary search for robust trading strategies

Funding Awarded

$40,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0 (0)

Status

  • Overall Status

    🛠️ In Progress

  • Funding Transfered

    $5,000 USD

  • Max Funding Amount

    $40,000 USD

Funding Schedule

View Milestones
Milestone Release 1
$5,000 USD Transfer Complete TBD
Milestone Release 2
$3,000 USD Pending TBD
Milestone Release 3
$12,000 USD Pending TBD
Milestone Release 4
$12,000 USD Pending TBD
Milestone Release 5
$8,000 USD Pending TBD

Status Reports

Mar. 23, 2023

Status
🧐 Fair, but could have been better
Summary

Focusing entirely on milestone 3 of the related G3P project.

Full Report

Video Updates

Robert Haas – Kickoff presentation Evolutionary Search

15 March 2023

Project AI Services

No Service Available

Overview

This project uses grammar-guided genetic programming to automatically search for robust and creative trading strategies on user-defined markets, indicators, metrics and external timeseries data from predictive models, fundamental analysis or elsewhere. The main challenge in the design of trading strategies is not the automated search process, although an advanced one will be used here, but rather the design and implemention of a proper backtesting approach to evaluate candidate strategies that avoids common pitfalls such as overfitting to the provided data or optimizing the wrong goal. This challenge will be tackled by in-depth literature research about state-of-the-art statistical evaluation techniques and review of existing backtesting libraries. Consequently, the search itself will be tuned by initially developing simple and then increasingly advanced grammars that define different search spaces of candidate trading strategies.

Both 1) the evaluation method for candidate strategies and 2) the grammars for defining suitable spaces of candidate strategies will be integrated into an open-source software package and distributed via GitHub and PyPI. Based on this package, a service will be developed for SingularityNET's AI platform, where users can provide data plus various preferences to the algorithm and it will return evolved trading strategies that perform well on historical data and have a reasonably high chance to actually perform similarly well on live markets. These strategies can then be used either for manual trading or as part of an automated trading system, which involves further aspects such as portfolio management and position sizing for multiple strategies. When the markets undergo a regime change after some time, the strategies can easily be adapted by making use of the service again.

Proposal Description

Compnay Name

Antecedens e.U.

Problem Description

There are two central problems that make it difficult to design good trading strategies:

  1. The space of possible trading strategies is infinite and most of them do not perform well. This means a hard optimization problem has to be solved: finding good elements in a vast set of alternatives, most of which do not have sufficient quality to count as acceptable solution.
  2. Proper evaluation of trading strategies is difficult. While backtesting appears like a straightforward way to test and evaluate candidate strategies on historical market data, there are many pitfalls that lead to over-estimating a strategy's actual performance on live markets. For example, there is the problem of overfitting to given data, which means that the identified patterns are too fine-grained and specific, so they do not generalize well to future dynamics. Beyond that, there is also the problem of maximizing misleading performance measures that do not reflect the actually desired goal, such as being consistently profitable in live markets without suffering from major downturns. This means the optimization problem is not only hard because of the vastness of the search space, but also because of the difficulty of defining a reliable quality measure for candidate solutions. Metaphorically speaking, not only do we search for a needle in a haystack, we also have difficulties with correctly recognizing a needle when we hold one in our hands.

Solution Description

This project addresses both problems:

  1. Grammar-guided genetic programming is a set of advanced general-purpose optimization methods that can evolve programs and expressions in any context-free language. These methods perform well on many combinatorial search spaces. Their main advantage over other methods is that the search space can be defined and changed very easily, e.g. to restrict a large space to a small region of interest, which is essentially a sub-language within a more comprehensive one. In the context of trading, this means that the vast space of possible trading strategies can be flexibly restricted to finite subsets that are of interest to a user. For beginners it could be a space of simple strategies that rely only on one or two indicators with a predefined range of parameter values, such as window sizes for moving averages. For advanced traders it could be a space of complex strategies that rely on multiple timeseries, a range of indicators, external data from prediction models and eventually a non-trivial integration of all these sources into bid, sell and hold actions.
  2. Statistical evaluation methods for timeseries data and risk-adjusted performance measures for trading strategies allow to identify robust patterns, which not only give hypothetical returns in backtesting simulations on historical data, but also allow to achieve good performance in trading on live markets, at least until a regime change happens and a redesign becomes necessary.

Project Benefits for SNET AI platform

I) Platform benefits

This project benefits SingularityNET in three ways:

  1. It adds a service to the SingularityNET decentralized AI platform.
  2. It calls a service on the SingularityNET platform. This proposal is a special use case that builds on grammar-guided genetic programming. Please note that a preliminary prototype is already available, so even if the other proposal should be rejected, this proposal could utilize the working prototype to evolve robust trading strategies, yet without a service call and less efficiently than in the case of both proposals being accepted.
  3. It creates an open-source package for evolving robust trading strategies, which in part or in total could find usage in trading tools developed by SingularityNET's spin-off SingularityDAO and SingularityNET's partner project Autonio.

II) Ethical value

This project aligns with SingularityNET's vision of a benevolent, democratic and inclusive AGI in two ways:

  1. The created software is considered to be ethically neutral, because identifying robust patterns in timeseries data is in itself neither benevolent nor malevolent.
  2. The created software aims to democratize the use of advanced evolutionary learning methods for automatically constructing robust trading strategies. These capabilities are currently mainly available to ML/AI specialists and institutions, but less so to individual retail traders. This is partly due to the technical complexity of developing or reusing these methods, and partly due to the conceptual complexity of suitably applying them without stumbling into various pitfalls.

Competitive Landscape

Unfortunately there is almost no incentive to make well-performing systems for finding trading strategies publically available. Therefore most demonstrations that look good on paper tend to fail when applied to live markets. Also, if evolutionary algorithms are applied to optimize trading strategies, usually they are used in a quite restricted fashion, e.g. just evolving numerical values of a strategy that is fixed in its form. In contrast, grammar-guided genetic programming methods allow to easily define a variety of search spaces and to search for optimal form and parameters of trading strategies at the same time without any manual adjustments

Marketing & Competition

Visibility for the created package and service will be generated at locations where developers, researchers and retail traders usually seek for implementations of backtesting and other trading-related software, such as

  • Q&A sites: StackExchange, Reddit, Quora
  • Hosting sites: GitHub, PyPI
  • Software listings: GitHub collections, reference sections of related Wikipedia entries

Additional exposure by SingularityNET's channels is also highly appreciated.

Long Description

I) Methodology and its implications

The two central points that this project has to solve are explained in the problem and solution description. In essence, the challenges are to 1) define proper search spaces of candidate trading strategies with context-free grammars and 2) establish a robust backtesting approach based on modern statistical evaluation techniques to evaluate the quality of candidate strategies. Once these two ingredients (grammar, objective function) are developed in sufficiently high quality, the methods from grammar-guided genetic programming (G3P) can be applied in order to evolve near-optimal solutions in vast search spaces of various forms. What these G3P methods are and why they can be flexibly applied to the domain of trading strategies is explained in the description of the other proposal with the title “Grammar-guided genetic programming”. Their main advantage in this context is that they can not only evolve parameters, but also the form of trading strategies at the same time. For this reason, the discovered solutions can be much more creative and surprising than with other automated search procedures. This also means that there might be less competition for such strategies on the markets, because more conventional strategies are usually rendered inefficient over time when many traders start using them.

 

II) Prototype example that highlights potential and pitfalls

An initial example how the application of these methods to the search for robust trading strategies can look like in practice is contained in the website mentioned at the end of this proposal. Preliminary prototype code is used to successfully evolve a simple trading strategy based on four tunable moving averages and a bit of flexibility in the strategy's form. The found solution significantly outperforms a simple buy-and-hold strategy on historical stock price data of Alphabet Inc. (GOOG). Please note, however, that while this strategy looks very promising on the training data, it is very unlikely that it would perform well on live markets, because it is highly overfitted to the historical data and won't generalize well to future market dynamics. To overcome this deceptive problem, a much more sophisticated backtesting strategy needs to be implemented, so that overfitting is prevented efficiently with modern statistical evaluation techniques (e.g. k-fold cross-validation, bootstrapping, Bayesian techniques, …) and major drawdowns when operating on a live market are prevented by using performance metrics that involve proper risk assessment and mitigation. In the concrete example the Sortino ratio was maximized, which already is a risk-adjusted measure of returns rather than looking at pure profits, yet there are many other options out there to set realistic optimization goals, such as Sharpe ratio, Calmar ratio, other advanced metrics and combinations thereof. Other factors to consider are the complexity of a strategy, the number of trades it performs and associated fees, as well as dangers of accidentally using data in a look-ahead fashion in backtesting, which means that the strategy could not be traded in a live setting.

 Images showing a prototype result:

 
 

 

In the example shown in this image, a strategy was evolved that in backtesting performs 193 trades over a period of roughly 8 years. It generates a hypothetical return of 3509% (or 35x) that significantly outperforms the buy-and-hold return of 703% (or 7x), which comes from simply holding the GOOG stock over 8 years, by almost a factor 5. This is highly unrealistic and a hint of severe overfitting. Another hint, shown only on the website, is that two out of the four freely selected moving averages are very similar and short-term, which indicates that the found patterns are overly specific to the historical data and won't generalize well into the future. This problem could easily be prevented by using a slightly different grammar, i.e. redefining the search space so that these low values for window size are not included. A preference such as this one, together with a choice of market and training period, could directly come from a user that invokes the service.

III) Interface design

The exact design of the the package and service interfaces (=which inputs, which outputs) is part of the proposed work (literature research, design) and partly depends on the final choice of backtesting framework and experimental results with different metrics. The goal is to give a lot of freedom to advanced users, but also guidance to beginning traders in form of suggested parameter combinations that lead to the search for simple but potentially very robust trading strategies. Such strategies also have the benefit that they are less prone to overfitting and can be easily interpreted by a human trader. Therefore they can be used by hand rather than being executed by an automated trading system that closely watches live markets all the time and quickly reacts to any signals the strategy generates.

AI Services

Proposal Video

Placeholder for Spotlight Day Pitch-presentations. Video's will be added by the DF team when available.

  • Total Milestones

    5

  • Total Budget

    $40,000 USD

  • Last Updated

    11 Mar 2024

Milestone 1 - Project start

Status
😀 Completed
Description

Contract signed

Deliverables

Budget

$5,000 USD

Link URL

Milestone 2 - Design phase

Status
🧐 In Progress
Description

Review of literature and backtesting libraries, design of search spaces and objective functions summarized in a written document

Deliverables

Budget

$3,000 USD

Link URL

Milestone 3 - Development phase 1

Status
😐 Not Started
Description

Release of v0.1.0 on GitHub with objective function and preliminary grammars

Deliverables

Budget

$12,000 USD

Link URL

Milestone 4 - Development phase 2

Status
😐 Not Started
Description

Release of v0.2.0 on GitHub and PyPI with refined grammars, consistent parameterization and testing suite

Deliverables

Budget

$12,000 USD

Link URL

Milestone 5 - Documentation and integration phase

Status
😐 Not Started
Description

Release of v1.0.0 on GitHub and PyPI with frozen API. Documentation website. AI Service and its donation to SNET.

Deliverables

Budget

$8,000 USD

Link URL

Join the Discussion (0)

Reviews & Rating

New reviews and ratings are disabled for Awarded Projects

Sort by

0 ratings

Summary

Overall Community

0

from 0 reviews
  • 5
    0
  • 4
    0
  • 3
    0
  • 2
    0
  • 1
    0

Feasibility

0

from 0 reviews

Viability

0

from 0 reviews

Desirabilty

0

from 0 reviews

Usefulness

0

from 0 reviews