Integrate SYBIL as an Ocean Predictoor

chevron-icon
Back
project-presentation-img
Kevin R. C.
Project Owner

Integrate SYBIL as an Ocean Predictoor

Funding Requested

$64,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 4.5 (4)

Overview

This is a notable application proposal for the existing service SYBIL: The General-Purpose Forecaster. It will integrate SYBIL (plus Deep SYBIL if funded) into Ocean Protocol's platform as a Predictoor Bot. SYBIL's Predictoor Bot creates prediction feeds, or streams of forecasts, of the short-term crypto price movements (up or down). For example, on the top 10 cryptos by market cap like BTC, ETH, or ADA. These feeds can be a forward-looking indicator to determine investors' general outlook on the crypto landscape. Because Predictoor Bots can earn or lose money depending on their accuracy, SYBIL's Predictoor will be deployed on Ocean's testnet as primarily a research and informational tool.

Proposal Description

Company Name (if applicable)

Temporai

How our project will contribute to the growth of the decentralized AI platform

This project bridges Ocean Protocol and SingularityNET's platforms on an application level. It can directly engage Ocean Predictoor users with SYBIL, an SNET service, once integrated as a Predictoor Bot. This project is well-aligned with the Artificial Superintelligence Alliance (ASI) goals of unifying the decentralized AI stack, as outlined in its Vision Paper. It showcases this merger's groundbreaking potential by providing an immediate, low-hanging use case of crypto price predictions.

The core problem we are aiming to solve

The core 'problem' we aim to solve is how to produce consistently high-quality prediction feeds on the Ocean Predictoor platform. Starting with crypto price prediction feeds. Ocean's stated goal is to have 10,000+ prediction feeds on every subject matter, ranging from weather/climate to transportation/logistics to crypto/finance. These prediction feeds provide great value as they inform us of probable future events, allowing us to plan better and take decisive actions.

Ocean's Predictoor website currently provides crypto price prediction feeds on the top 10 cryptocurrencies based on market cap (i.e., BTC, ETH, ADA). Their accuracy ranges from 50-55% for an average span of 4 weeks on an hourly interval.

By both improving the accuracy of the prediction feeds and expanding their coverage (e.g., for tx fees or hash rates in addition to price), these feeds can become signals (or leading indicators) that enable crypto investors/traders to anticipate market conditions (i.e., boom/bust cycles) and adjust their crypto positions accordingly.

Our specific solution to this problem

Using the Predictoor Bot template found in Ocean Protocol's pdr_backend GitHub repo, we will customize the Bot to call the SYBIL General-Purpose Forecaster service via the SNET CLI. You can check the SYBIL service status and metadata with the following respective commands:

snet service print-service-status temporai sybil
snet service print-metadata temporai sybil

At every 5min or 1hr prediction feed interval, our customized SYBIL Predictor Bot runs through the following steps:

  1. source training data set from historical prediction feeds (i.e., autoregression) and other supplementary Ocean data sources (i.e., exogenous variables)
  2. call SYBIL's Train function and pass in this training set
  3. call SYBIL's Forecasting function for the next n timesteps, the prediction feed length
  4. convert SYBIL's forecast outputs into Ocean's two-sided up/down Bernoulli predictions
  5. package these predictions into a prediction feed object
  6. publish this prediction feed onto the Predictoor platform

We will iteratively backtest, evaluate SYBIL's accuracy, and tweak the SYBIL Predictoor bot and the SYBIL service itself to improve the feed's performance. We scale by either spinning up multiple Predictoor bots for each prediction feed or having one bot produce multiple prediction feeds. We start with T10 crypto price feeds and expand from there.

We will only publish feeds on Ocean's Testnet since we can earn/lose money on its Mainnet. Putting on Mainnet and sharing the P/L with SNET community can be a future work.

Project details

Ocean's Predictoor framework has two ways of calling the AI forecasting model: through simulation or running the actual Predictoor Bot, either on testnet or mainet. Both ways require building the AI model using Ocean Predictoor's internal AimodelFactory class. Within that class, Ocean currently has several scikit-learn classification models for the Predictoor bots to use. They include the following:

  • DummyClassifier: scikit-learn's DummyClassifier with strategy="most-frequent", where the model predicts the most frequent class label during training
  • LinearLogistic: scikit-learn's LogisticRegression with max_iter=1000, which is the most amount of iterations for the LR model's optimizer to converge
  • LinearLogistic_Balanced: scikit-learn's LogisticRegression with max_iter=1000 and class_weight="balanced", meaning its class weights to be inversely proportional to the class frequencies
  • LinearSVC: scikit-learn's SVC, Support Vector Classification.

This project aims to fork the pdr_backend repo and modify it so that the SYBIL service is among the AimodelFactory options, alongside LogisticRegression and SVC. This forked repo can then be used to spin up the SYBIL Predictoor Bot. SYBIL will be the most advanced model option as it allows the bot to ensemble to various forecasting models that SYBIL provides, from statistical time-series models (i.e., ARIMA, ETS, TBATS) and to machine learning (i.e., LightGBM) and even more advanced models like NeuralProphet. If Deep SYBIL (D-SYBIL) is also funded this round, then D-SYBIL's deep learning models can be utilized by SYBIL, and therefore be accessible by the Predictoor bot downstream too!

Ultimately, to call SYBIL, I can simply change the approach to "SYBIL" in the ppss.yaml, which is the data/model config file with parameters to execute to run either simulation or bot. You can find it in the aimodel_ss section of the file:

________________________________________________________________________________________________________

aimodel_ss:
    max_n_train: 5000 # no. epochs to train model on
    autoregressive_n: 1 # no. epochs that model looks back, to predict next
    approach: SYBIL # SYBIL | LinearLogistic | LinearLogistic_Balanced | LinearSVC | Constant
    weight_recent: 10x_5x # 10x_5x | None
    balance_classes: None # SMOTE | RandomOverSampler | None
    calibrate_probs: CalibratedClassifierCV_Sigmoid # CalibratedClassifierCV_Sigmoid | CalibratedClassifierCV_Isotonic | None

________________________________________________________________________________________________________

We will develop the SYBIL module (as specified in Milestone 6) that calls the SYBIL SNET service under the hood and include it in AimodelFactory in the forked repo.

The competition and our USPs

Yes

Describe how your solution distinguishes itself from other solutions (if exist) and how it will succeed in the market.

Our solution distinguishes itself from other solutions in the following ways:

  • Our proven track record: We have successfully deployed SYBIL onto the SNET marketplace and are continuously releasing new versions of the service. Additionally, we are testing and refining SYBIL with a variety of time-series datasets. We will apply our rigorous methodology when evaluating SYBIL Predictoor Bot's crypto prediction feeds.
  • Our deep team: We have six team members, all with relevant experience in AI development. Our team is composed of data scientists, research scientists, and SWEs.
  • Our focus on interoperability: Our AI services (in this case SYBIL) are designed to be interoperable with both end applications (Predictoor) and other services (MAUQ and D-SYBIL) alike. As our services grow and augment each other's capabilities, they will only serve as a force multiplier that greatly benefits their end applications, like Ocean's Predictoor platform.

Our team

Our SYBIL Predictoor integration team consists of the following:

  • Kevin R.C.: project owner, data scientist, ML research engineer, prior crypto/blockchain experience (bio here)
  • Daniel S: data scientist, ML research scientist
  • Joseph K.: SWE, AWS cloud infra, SNET deployment
  • Parv B.: data science grad intern - Predictoor bot integration
  • Sairam V.: data science grad intern - Predictoor bot integration
  • Chinmay D.: data science grad intern - Predictoor bot integration
View Team

What we still need besides budget?

Yes

Describe the resources you still need

Temporai is seeking a part-time growth engineer with a multifaceted skillset in engineering, product management, and business development. Roles include:

  • Providing support for different Temporai projects like Predictoor integration
  • Liaise between external stakeholders and internal technical/modeling teams
  • Idealate, test, and evaluate various SYBIL Predictoor prediction feeds
  • Creating, implementing, and evaluating ideas or strategies for growth
  • Represent Temporai in external meetings with SNET, Ocean, and ecosystem

Brownie points out that you are highly passionate about the intersection of AI + crypto/Web3 and time-series forecasting. If you think this is you, please reach out to kevin@tempor.ai.

Existing resources we will leverage for this project

Yes

Description of existing resources

Ocean Predictoor's public GitHub code repo to create and run Predictoor Bots: pdr-backend 

Open Source Licensing

gnu

Describe license details and, if applicable, list any components that are not subject to this license.

We will eventually open-source the integration source code under the GNU GPL - GNU General Public License, just like for SYBIL and MAUQ. This source code is the forked Ocean Protocol's pdr_backend GitHub repo that can call SYBIL service as one of its models, as specified in the Long Description of your project.

Links and references

Here are the resource links to our prior awarded services:

SYBIL Service (DF-R2)

  • Proposal: link
  • YouTube Overview: link
  • YouTube Demo: link
  • Architectural Design Report Doc: link
  • GitHub Public Code Repo: link
  • SNET Marketplace: link

Onboard NeuralProphet Service (DF-R2)

  • Proposal: link
  • GitHub Public Code Repo: link
  • SNET Marketplace: link

MAUQ Service (DF-R3)

  • Proposal: link
  • YouTube Overview: link
  • GitHub Public Code Repo: link

Additional videos

See the YouTube links in the Links and references section above.

AI services (New or Existing)

SYBIL General-Purpose Forecaster

How it will be used

As explained in the Project Details section we will create a Predictoor Bot that will call the SYBIL service under the hood.

Proposal Video

Placeholder for Spotlight Day Pitch-presentations. Video's will be added by the DF team when available.

  • Total Milestones

    8

  • Total Budget

    $64,000 USD

  • Last Updated

    20 May 2024

Milestone 1 - API Calls & Hostings

Description

This milestone represents the required reservation of 25% of your total requested budget for API calls or hosting costs. Because it is required we have prefilled it for you and it cannot be removed or adapted.

Deliverables

You can use this amount for payment of API calls on our platform. Use it to call other services or use it as a marketing instrument to have other parties try out your service. Alternatively you can use it to pay for hosting and computing costs.

Budget

$16,000 USD

Milestone 2 - Contract Sign

Description

Finalize milestone details and sign contract with SNET

Deliverables

- Finalize milestone details (including allocation of API Calls budget across development milestones) - Sign contract with SNET - Assemble team and set up tooling for SYBIL Integration to Ocean Predictoor (including JIRA GitHub Databricks AWS Cloud etc.)

Budget

$2,000 USD

Milestone 3 - Design Report

Description

Finalize SYBIL Predictoor integration design report

Deliverables

- Liaise with the Ocean Protocol team to get Predictoor framework specifications including the current state of Predictoor Bots running on Testnet Mainnet whether they plan to expand their prediction feeds to other cryptos (i.e. AI Cryptos) beyond T10 market cap and even other domains (i.e. weather/climate) and whether we can leverage other Ocean data sources to feed into the Predictoors - Write-up architectural design report detailing these Predictoor specifications SYBIL Predictoor Bot architecture how it calls SYBIL service back-end payment scheme (i.e. AGIX or ASI) predictoor feeds output executing stakes with Ocean Sapphire EVM smart contract and backtesting/evaluation of these feeds in the Ocean Testnet

Budget

$8,000 USD

Milestone 4 - Enhance SYBIL Code Base

Description

Enhance and refactor SYBIL's code base to make it more robust and integrative with the Ocean Predictoor framework

Deliverables

- Streamline and simplify internal SYBIL architecture modules including its model factory and wrapper classes - Add more in-code comments and documentation - Develop a formula or algorithm to transform SYBIL's regression forecast into binary probabilistic classification (i.e. 50/50 60/40 80/20 etc.) for Ocean Predictoor's two-sided marketplace scheme

Budget

$8,000 USD

Milestone 5 - Optimize SYBIL Run Time

Description

Optimize SYBIL so it so its Train and Forecast function can run within the 5-minute or hourly frequency set by Ocean Predictoor platform

Deliverables

- Further optimize SYBIL infrastructure including parallel processing so we can that ability to run SYBIL base models in parallel (i.e. Kubernetes Microservices PySpark) - Select the base models and specify the hyperparameters for SYBIL to run on the 5min and 1hr intervals respectively (e.g. don't want to include TBATs or deep RNN base models for the 5min interval due to the time it takes to train these models)

Budget

$4,000 USD

Milestone 6 - SYBIL Predictoor

Description

Implement Predictoor Bot that calls SYBIL service back-end

Deliverables

- Develop module to automatically trigger SYBIL service using the SNET-CLI and return its JSON output - Integrate module into the Predictoor Bot to form SYBIL Predictoor modify and enhance the Predictoor code base from Ocean to suit the SYBIL service call - Utilize historical prediction feeds data set as well as other Ocean data sources (if possible) as input data to train SYBIL at every interval

Budget

$10,000 USD

Milestone 7 - Prediction Feeds

Description

Create and evaluate several crypto price prediction feeds (as well as other crypto indicators if possible) on Ocean Predictoor's Testnet

Deliverables

- Create prediction feeds using the SYBIL Predictoor Bot on the Testnet based on the Predictoor specifications as outlined in the design report - Develop backtesting and evaluation processes to gauge the accuracy of the prediction feeds - Iterate and tweak and SYBIL Predictoor Bot the module that calls the SYBIL service and/or the SYBIL service to improve the prediction feed accuracy above > 50% if not mor

Budget

$10,000 USD

Milestone 8 - Final Report and Promotion

Description

Finalize SYBIL Predictoor integration write up a final report and promote/demo this project

Deliverables

- Write up a final report for SYBIL Predictoor detailing its implementation and prediction feed performance - Work on demoing outreach and promotion of SYBIL Predictoor (including demoing its prediction feeds) with the assistance of SNET Ocean and the wider ASI ecosystem to gain visibility of this cross-platform use case

Budget

$6,000 USD

Join the Discussion (0)

Reviews & Rating

Sort by

4 ratings
  • 0
    user-icon
    TrucTrixie
    May 12, 2024 | 3:56 PM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 5
    Analyze disadvantages currently applied technology

    Try to evaluate for yourself which of the 8 milestones you outlined plays the most important role. And I'd like you to go into more detail on that milestone. That further shows the professionalism and dedication of you and your colleagues. I understand the team's professionalism when clearly defining the problem that needs to be solved - then coming up with a solution that integrates SYBIL as a forecasting tool. These activities all revolve around SYBIL, taking SYBIL as the center to operate and perfect it. This is good because it brings long-term benefits, but in return, we should also analyze the downsides and limitations of SYBIL and the applied technology so that the community knows the problem in an objective way. Thank you.

  • 0
    user-icon
    BlackCoffee
    May 12, 2024 | 11:59 AM

    Overall

    4

    • Feasibility 5
    • Viability 4
    • Desirabilty 4
    • Usefulness 4
    Should a reserve budget be established?

    The budget explanation seems reasonable, which is understandable because it comes from a professional. As for the reserve budget, has the team thought about this? I think there should be a reserve budget because it is a reserve fund for unexpected things that happen to the team during project implementation - when there is a reserve budget, the implementation will not be interrupted because money factor.
    The advantage of this proposal is that the total amount requested represents the correct value for the allocated capital. That means waste or extravagance is almost non-existent. I like this.

  • 0
    user-icon
    Max1524
    May 12, 2024 | 3:22 AM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 4
    • Usefulness 4
    More detailed technical analysis

    This proposal applies today's most advanced technology to the field of bridging between the two platforms Ocean Protocol and SingularityNET. As we always know, building bridges between platforms and blockchains is always a hot topic and the goal of many developers. Kevin R.C has many proposals to take advantage of technology and bring high-tech products to life. I cannot deny the feasibility and benefits that the proposal brings. I highly encourage such suggestions to continue, even continuing through many Rounds to come. If possible, Kevin can explain more clearly the technical aspects of technology so that the community can understand. A good way to clarify feasibility.

  • 0
    user-icon
    Joseph Gastoni
    May 10, 2024 | 6:06 AM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 4
    • Usefulness 5
    It has potential but requires careful execution

    Integrating SYBIL with Ocean Protocol's Predictoor framework has potential but requires careful execution to ensure the accuracy and value of the generated predictions. Building trust and transparency with users regarding the limitations of AI predictions in financial markets is essential.

    Feasibility:

    • High: The project seems technically feasible as it leverages existing components (SYBIL, Ocean Predictoor framework).
    • Integration and customization of the Predictoor Bot require some development effort.

    Viability:

    • Moderate: The success depends on the accuracy and market value of SYBIL's predictions.
    • Competition exists in crypto prediction services, so a strong value proposition is crucial.

    Desirability:

    • Moderate: Improved crypto price prediction is desirable for investors, but the accuracy needs to be proven.
    • The integration with Ocean Protocol can be attractive for users of both platforms.

    Usefulness:

    • Moderate: The project can be a valuable tool for generating crypto price prediction feeds on Ocean's platform.
    • The actual usefulness of the predictions for investors depends on their accuracy and potential for alpha generation (outperforming the market).

    Besides, the project should consider:

    • Transparency about the limitations and risks associated with using AI-based predictions for financial decision-making is crucial.
    • A clear strategy for evaluating and improving the accuracy of SYBIL's predictions is important.

    Here are some strengths of this project:

    • Leverages existing technology (SYBIL, Ocean Predictoor) for a new application.
    • Contributes to the growth of both SingularityNET and Ocean Protocol by creating a bridge between their platforms.
    • Provides a low-hanging fruit use case to showcase the potential of AI integration in decentralized finance.

    Here are some challenges to address:

    • Demonstrating the accuracy and market value of SYBIL's predictions compared to existing services.
    • Earning trust from investors who may be skeptical about AI-based financial predictions.
    • Ensuring transparency and responsible use of the predictions, considering the potential risks involved.

    By focusing on improving the accuracy of SYBIL's predictions and clearly communicating their limitations, this project can increase its value for crypto investors and the overall Decentralized AI ecosystem.

Summary

Overall Community

4.5

from 4 reviews
  • 5
    2
  • 4
    2
  • 3
    0
  • 2
    0
  • 1
    0

Feasibility

4.8

from 4 reviews

Viability

4.5

from 4 reviews

Desirabilty

4

from 4 reviews

Usefulness

4.5

from 4 reviews