Upgrade of the Stable Diffusion service

Compnay Name

Service Details

This proposal was created following open questions made on another proposal which was delivered in Pool A by end December 2022 (Integration of Maistream open-source AIs - https://proposals.deepfunding.ai/collaborate/9ad5eb8a-5b19-41fb-a5d9-52b6d18e0844).

The limitations and proposal for improvement, as described in this proposal, were shared with the SNet team during January 2023. This highlighted the opportunity to improve the existing service and also put it on par with the levels of expectations and quality of design, access, test, and documentation already pitched in the Pool A proposal.

The two proposals (the one in Pool A and this one from Pool B) are totally independent and do not address the use of the same open source AI (Stable Diffusion and mini-DallE are solely for this Pool B proposal), although the same team would work on both projects if both were selected, and there could be some benefits in using the same API models, tools, and infrastructure on both projects. Since both are fully open-source, they could also act as good templates for the integration of future open-source AIs into SNet.

The risks on this project are very low, and very much mitigated by:

• Payment on delivery (therefore most of the risk is shifted onto the development/delivery team)
• An agreement in principle for cooperation between the original developers of the service (from the SNet team) and us.
• The experience from the writer of this proposal who has delivered similar virtualized open-source AIs for other funded projects (Cardano fund 7, project TaChiKu).

Problem Description

The intention behind the delivery of the original service was good, and it currently acts as a good Proof of Concept (PoC) for integrating open-source AI into the SNet platform. However, this service currently stops short at the PoC stage, and is not much useable for code integration (via API) and/or production stage.

Here below is a screenshot of the current service UI:

Current UI for the service

At first sight, the limitations that we can attribute to this service are:

• The UI is very basic
• There is only one single parameter (input text)
• There is no choice of model (let alone custom trained model)
• There is no API
• It requires payment in AGIX after only 15 tries
• It can be slow to output results

Those limitations do not invalidate the PoC status and the effort spent so far to get there. But they do act as a severe constraint on the use of the service, both from an end-user and a developer point of view:

1. End-user: the simplistic UI does not make it much useable.
2. Developer: the lack of parametrization, customization, and access via API makes it very un-useable.

=> As a consequence, this service is not used, and the SNet platform does not benefit much from its existence.

Focussing on the Stable Diffusion AI as an example, there are many parameters that a developer would want to access on such a service. See below an example of parameters I would expect to access as a developer, but only the first one is available in the current service.

List of parameters a dev would expect to access

In addition, as a developer, I would want the capability to upload my own trained model and make use of the service capacity (GPU power, simple but secure access via API). The service does not offer this. In the best of worlds, I would also want to train my models with the service...

Overall, here below is a list of limitations that I think prevent the use of this service by a dev team:

• No access to all AI parameters
• No ability to upload a custom model
• No ability to train a model
• No ability to call APIs from code
• No ability to integrate securely
• Already cost money during test phase (after 15 tries)
• Is slow to deliver output
• Does not indicate any ability to scale (when in production)
• Is using open-source AI but is not made open-source (what do you hide?)

If those limitations are not addressed, it is my humble view that the service will not be used, therefore does not create much value and gives a bad impression that the SNet platform is populated by PoC services, not production-ready ones.

Solution Description

The improvement of this service would be carried out to mitigate the limitations listed above. Here below is the list of high level tasks which would be implemented:

• Create a GitHub open-source repo to host all the service code
• Include the entire Docker config in the open-source repo
• To improve the speed, run the Python AI program inside a web server (avoids warm-up time each time the service is called when it runs the AI as a program)
• Re-host the AI service on cost effective GPU servers
• Build documented web server end-points, accepting all possible parameters
• Build an open-source API which can be called and integrated from code
• Manage a proper authentication into the service, with possible Authorization levels for test/prod
• Allow authenticated (paying) users to upload and make use of custom models
• Make the environment free to use for unauthenticated users (test mode with limitations)
• Build test scripts for availability and robustness
• Create a small sample app that integrates with the service (to act as example)
• Create a video to showcase the use of Stable Diffusion via SNet (for marketing use)
• Document and package the final solution on GitHub

Note that this solution does not fully address the question of marketing outside of the SNet community, which remains an important point, but it creates a starting point where the service becomes production-ready, ad is fully useable, integrable, and scalable.

In short, this proposal would transform a very much unused Proof of Concept into a great utility service that did not exist before, and which can benefit the whole SNet ecosystem, since Stable Diffusion and mini-DallE are the two most in-demand image generation AIs.

Milestone & Budget

Total cost = $13,500

Delivery schedule: test-ready within 4-5 weeks ; full production-ready within 8-10 weeks.

Deliverables

Service License Info

Revenue Sharing

If the services cross the threshold of $1,000 revenue per month, 10% of the additional revenue will be fed back into the SNET/DeepFunding wallets. This condition will remain valid for 10 years after first onboarding the service and will be applicable to this service or any subsequent iteration of this service on the platform.

Related Services

The existing service can be found here:

• Creator: SNet team (Sergey Shalyapin)
• Location:
  https://beta.singularitynet.io/servicedetails/org/snet/service/image-generation

Marketing & Competition

To help boost the profile of SingularityNet, we will also deliver at least one high quality marketing video showcasing the integration of Stable Diffusion into SingularityNet.

The 25% budget allocation for hosting will also be used to make the access to the service free of charge when in test mode, therefore removing a barrier to entry for integration.

Related Links

The Pool A proposal for delivering open-sourced AIs is here:

Further links below are provided as example of what can be expected from us in terms of delivery and quality:

API standard and documentation:

Sample REST API doc produced for a prior project funded by Cardano in the category supported by SingularityNet: https://tachiku.com/doc/

Open source repo and code quality:

Location of our current public GitHub for running Stable Diffusion via a web server: https://github.com/incubiq/stable_diffusion

Sample open source repo for a Cardano fund 8 project: https://github.com/incubiq/sign_in_with_wallet

Monthly video report on project progression (here was the final video report):

Sample video reports of what was delivered for Cardano (TaChiKu, funded in the SingularityNet category): https://youtu.be/OPSa7qocXQc